Commit Graph

559 Commits

Author SHA1 Message Date
Jason Wen
08e85808c5 Merge branch 'upstream/openpilot/master' into sync-20251114
# Conflicts:
#	.github/workflows/ci_weekly_run.yaml
#	.github/workflows/raylib_ui_preview.yaml
#	.github/workflows/tests.yaml
#	.gitmodules
#	README.md
#	SConstruct
#	common/api.py
#	common/params_keys.h
#	docs/CARS.md
#	msgq_repo
#	opendbc_repo
#	panda
#	selfdrive/car/tests/test_car_interfaces.py
#	selfdrive/controls/controlsd.py
#	selfdrive/controls/lib/latcontrol.py
#	selfdrive/controls/lib/latcontrol_angle.py
#	selfdrive/controls/lib/latcontrol_pid.py
#	selfdrive/controls/lib/latcontrol_torque.py
#	selfdrive/controls/tests/test_latcontrol.py
#	selfdrive/monitoring/helpers.py
#	selfdrive/ui/SConscript
#	selfdrive/ui/main.cc
#	selfdrive/ui/qt/body.h
#	selfdrive/ui/qt/home.cc
#	selfdrive/ui/qt/home.h
#	selfdrive/ui/qt/network/networking.cc
#	selfdrive/ui/qt/network/networking.h
#	selfdrive/ui/qt/network/wifi_manager.cc
#	selfdrive/ui/qt/offroad/developer_panel.cc
#	selfdrive/ui/qt/offroad/developer_panel.h
#	selfdrive/ui/qt/offroad/experimental_mode.cc
#	selfdrive/ui/qt/offroad/firehose.cc
#	selfdrive/ui/qt/offroad/firehose.h
#	selfdrive/ui/qt/offroad/onboarding.cc
#	selfdrive/ui/qt/offroad/onboarding.h
#	selfdrive/ui/qt/offroad/settings.cc
#	selfdrive/ui/qt/offroad/settings.h
#	selfdrive/ui/qt/offroad/software_settings.cc
#	selfdrive/ui/qt/onroad/alerts.cc
#	selfdrive/ui/qt/onroad/annotated_camera.h
#	selfdrive/ui/qt/onroad/buttons.cc
#	selfdrive/ui/qt/onroad/buttons.h
#	selfdrive/ui/qt/onroad/driver_monitoring.cc
#	selfdrive/ui/qt/onroad/hud.cc
#	selfdrive/ui/qt/onroad/hud.h
#	selfdrive/ui/qt/onroad/model.cc
#	selfdrive/ui/qt/onroad/model.h
#	selfdrive/ui/qt/onroad/onroad_home.cc
#	selfdrive/ui/qt/onroad/onroad_home.h
#	selfdrive/ui/qt/request_repeater.h
#	selfdrive/ui/qt/sidebar.cc
#	selfdrive/ui/qt/sidebar.h
#	selfdrive/ui/qt/util.cc
#	selfdrive/ui/qt/widgets/cameraview.h
#	selfdrive/ui/qt/widgets/controls.cc
#	selfdrive/ui/qt/widgets/controls.h
#	selfdrive/ui/qt/widgets/input.cc
#	selfdrive/ui/qt/widgets/input.h
#	selfdrive/ui/qt/widgets/prime.cc
#	selfdrive/ui/qt/widgets/prime.h
#	selfdrive/ui/qt/widgets/ssh_keys.h
#	selfdrive/ui/qt/widgets/toggle.h
#	selfdrive/ui/qt/widgets/wifi.cc
#	selfdrive/ui/qt/widgets/wifi.h
#	selfdrive/ui/qt/window.cc
#	selfdrive/ui/qt/window.h
#	selfdrive/ui/tests/cycle_offroad_alerts.py
#	selfdrive/ui/tests/test_ui/run.py
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_fr.ts
#	selfdrive/ui/translations/main_ja.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_nl.ts
#	selfdrive/ui/translations/main_pl.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
#	selfdrive/ui/ui.cc
#	selfdrive/ui/ui.h
#	system/manager/build.py
#	system/version.py
2025-11-16 02:50:28 -05:00
Maxime Desroches
8a77534d02 fix zipapp with multilang (#36511)
* fix

* fix

* fix

* more
2025-10-27 22:47:41 -07:00
Jason Wen
f57de1c5b2 version: a new beginning (#1411)
* version: a new beginning

* changelog

* singular

* show ours

* actual

* readjust

* updated

* more

* official spelling

* more

* sync

* fix

* send it

* push

* we never had this lol

* syncs
2025-10-21 17:12:57 -04:00
Jason Wen
698e0ca00f migration: new branch names (#1225)
* migration: new branch names

* more migration

* update channel type

* no more var

* update

* more

* more
2025-09-07 23:23:03 -04:00
DevTekVE
5d110bcee5 ci: prebuilt process improvement & tag of staging prebuilt source (#1190)
* ci: add validate-test-on-staging-c3 branch to deployment triggers and enhance stable branch handling

* fix long overdue mistake lol

* ci: add condition to wait for start on push events in build workflow

* Fix extra version identifier

* no need for this, i validated what I needed

* only care for release tags, not any

* fix: update versioning logic to use build date and run number for tagging

* fix: update tagging logic and enhance commit message format in build scripts

* fix: refine tagging condition to exclude tag pushes for stable branches

* fix: add extra version identifier to output for better version tracking

* trying to keep things clean and simple

* bugfix
2025-08-30 14:35:05 +02:00
Jason Wen
798e9071d8 Merge branch 'upstream/openpilot/master' into sync-20250823
# Conflicts:
#	.github/workflows/release.yaml
#	README.md
#	RELEASES.md
#	common/params_keys.h
#	docs/CARS.md
#	opendbc_repo
#	panda
#	release/build_stripped.sh
#	selfdrive/controls/lib/longitudinal_planner.py
#	selfdrive/modeld/modeld.py
#	selfdrive/ui/feedback/feedbackd.py
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_fr.ts
#	selfdrive/ui/translations/main_ja.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
#	system/version.py
#	uv.lock
2025-08-24 14:52:21 -04:00
Adeeb Shihadeh
ef2bb7f2fc release: build orphaned branch (#36047) 2025-08-22 20:06:12 -07:00
Adeeb Shihadeh
b4cc4ea8e2 Update README.md 2025-08-20 15:44:23 -07:00
Adeeb Shihadeh
154f655335 update release checklist 2025-08-20 15:44:13 -07:00
Adeeb Shihadeh
560c503871 new release flow (#36021)
* new release flow

* Update README.md
2025-08-19 11:19:58 -07:00
Adeeb Shihadeh
2148e2dff2 build_devel: clean submodules 2025-08-18 19:48:08 -07:00
Jason Wen
97c2d7e655 Merge branch 'upstream/openpilot/master' into sync-20250809
# Conflicts:
#	common/params_keys.h
#	opendbc_repo
#	panda
#	selfdrive/car/cruise.py
#	selfdrive/controls/controlsd.py
#	selfdrive/selfdrived/selfdrived.py
#	selfdrive/test/process_replay/process_replay.py
#	selfdrive/ui/qt/setup/setup.cc
#	tinygrad_repo
2025-08-10 02:05:05 -04:00
Maxime Desroches
fbbb2ef5d0 update release checklist 2025-08-05 16:57:39 -07:00
Adeeb Shihadeh
c321fa72e2 one more 2025-08-05 16:08:14 -07:00
Adeeb Shihadeh
2c654500d2 update release README 2025-08-05 16:07:29 -07:00
Jason Wen
a93f1caf1f ci: dynamic submodule check for build_release (#1114)
* ci: dynamic submodule check for build_release

* test opendbc diff

* somem fix

* this way

* use path

* use master branch instead

* less verbose

* test bump

* test 1 more sub change

* unbump

* only echo if there's a diff
2025-08-02 23:11:59 -04:00
Jason Wen
f3c4501db1 docker: sunnypilot images (#1104)
* docker: sunnypilot images

* update ref for now

* update

* actually nightly

* rename

* enable now

* revert
2025-08-02 01:15:07 -04:00
Jason Wen
6e76d0ca86 ci: more ref change to master 2025-07-31 14:10:45 -04:00
James Vecellio-Grant
6d41ce2032 ci: Refactor model building workflows (#1096)
* Tinygrad bump from sync-20250627

* bump tinygrad_repo

* Reformat metadata generator to match driving_models.json

* bump tinygrad

* Revert "bump tinygrad"

This reverts commit f479dfd502.

* revert me after SP model compiled

* Model recompiled successfully, initiate "revert me after SP model compiled"

This reverts commit 95706eb688.

* The "FillMe" placeholder caused an extra 10 seconds of work

* bump to 22Jul2025

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* Allow more dynamic short names

This should hopefully be future-proof for now.. It's robust enough to return the correct word-digit format (see example on how it generates from given display name below):

'Last Horizon V2 (November 22, 2024)' -> LHV2
'Alabama (November 25, 2024)' -> ALABAMA
'PlayStation (December 03, 2024)' -> PLAYSTAT
'Postal Service (December 09, 2024)' -> PS
'Null Pointer (December 13, 2024)' -> NP
'North America (December 16, 2024)' -> NA
'National Public Radio (December 18, 2024)' -> NPR
'Filet o Fish (March 7, 2025)' -> FOF
'Tomb Raider 2 (April 18, 2025)' -> TR2
'Tomb Raider 3 (April 22, 2025)' -> TR3
'Tomb Raider 4 (April 25, 2025)' -> TR4
'Tomb Raider 5 (April 25, 2025)' -> TR5
'Tomb Raider 6 (April 30, 2025)' -> TR6
'Tomb Raider 7 (May 07, 2025)' -> TR7
'Down to Ride (Revision: May 10, 2025)' -> DTR
'SP Vikander Model (May 16, 2025)' -> SPVM
'VFF Driving (May 15, 2025)' -> VFFD
'Secret Good Openpilot (May 16, 2025)' -> SGO
'Vegetarian Filet o Fish (May 29, 2025)' -> VFOF
'Down To Ride (Revision: May 30, 2025)' -> DTR
'Vegetarian Filet o Fish v2 (June 05, 2025)' -> VFOFV2
'Kerrygold Driving (June 08, 2025)' -> KD
'Tomb Raider 10 (June 16, 2025)' -> TR10
'Organic Kerrygold (June 17, 2025)' -> OK
'Liquid Crystal Driving (June 21, 2025)' -> LCD
'Vegetarian Filet o Fish v3 (June 21, 2025)' -> VFOFV3
'Vibe Model [Custom Model]' -> VMCM
'Tomb Raider 13 (June 27, 2025)' -> TR13
'Aggressive TR (June 28, 2025)' -> ATR
'Tomb Raider 14 (June 30, 2025)' -> TR14
'Cookiemonster Tomb Raider (July 02, 2025)' -> CTR
'Down to Ride (Revision: July 07, 2025)' -> DTR
'Simple Plan Driving (July 07, 2025)' -> SPD
'Down to Ride (Revision: July 08, 2025)' -> DTR
'Tomb Raider 15 (July 09, 2025)' -> TR15
'Tomb Raider 15 rev-2 (July 11, 2025)' -> TR15R2
'Le Tomb Raider 14 (July 14, 2025)' -> LTR14
'Le Tomb Raider 14h (July 17, 2025)' -> LTR14H
'Tomb Raider 16 (July 18, 2025)' -> TR16
'Tomb Raider 16v2 (July 21, 2025)' -> TR16V2

* Update build-all-tinygrad-models.yaml

* Update build-all-tinygrad-models.yaml

* No need to sleep 3 seconds, just send it

* try dynamic

* cleanup

* Update build-single-tinygrad-model.yaml

* bc devtekve said. also, this is repetitive af

* Revert "bc devtekve said. also, this is repetitive af"

This reverts commit 3a0c1562de.

* maybe we could use a script instead that both build all

That both build all and sunnypilot-build-model reference

* refactor: consolidate model building steps into a single workflow

* tweak

* tweakx2

* tweakx3

* tweakx4

* dunno dunno...

* output dir

* lots of changes

* Revert "lots of changes"

This reverts commit 4aadb0ee29.

* fail if all fail

* no inputs needed

* make it easier for us

* note failure and exit 0

* Update build-all-tinygrad-models.yaml

* not needed unless we really want it

* Update build-single-tinygrad-model.yaml

* Merge branch 'sync-20250627-tinygrad' of github.com:sunnypilot/sunnypilot into sync-20250627-tinygrad

* retry for failed ?

* always run this step because sometimes one build fails

which causes the matrix to fail, but most builds still have uploaded artifacts.

* strip

* no escape

* Update build-all-tinygrad-models.yaml

* Test case from terminal run

(openpilot) james@Mac sunnypilot % jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \\([^)]*\\)"; "")), is_20hz}]' \
  /Users/james/Documents/GitHub/sunnypilot-docs/docs/driving_models_v6.json > matrix.json

mkdir -p output
touch "output/model-Tomb Raider 16v2 (July 21, 2025)-544"
touch "output/model-Space Lab Model (July 24, 2025)-547"
touch "output/model-Space Lab Model v1 (July 24, 2025)-548"

built=(); while IFS= read -r line; do built+=("$line"); done < <(
  ls output | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ \([^)]*\)//' | awk '{gsub(/^ +| +$/, ""); print}'
)

jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \
  'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' \
  matrix.json > retry_matrix.json

cat retry_matrix.json

[]
(openpilot) james@Mac sunnypilot %

* always

* great success

* add suffix to retry artifact so it doesn't conflict

* retry to get_model too

* and there haha

* unnecessary hyphen

* compare built to missing. include retries

* adjust copy of artifacts.

* Update build-all-tinygrad-models.yaml

* Update model selector versioning and add documentation

* Update retry condition for failed models in build-all-tinygrad-models.yaml

* Update retry condition for failed models in build-all-tinygrad-models.yaml

* Update build-single-tinygrad-model.yaml

* false

* default none because why not

* red diff? i think?

* meh ... not needed i guess

* error error error

* Nayan is watching... always watching mike wazowski

* string all the way

* lots of retries just in case because im scared

* more robust

* ONLY ONE!!!!!!

* delete.... a lot

* fix artifacts

* fix artifacts

* make sure each is unique :)

* skip files like artifact duhhhh

* artifact name dir

* concurrency

* copy here

* Update build-single-tinygrad-model.yaml

* Update build-single-tinygrad-model.yaml

* bump

* bump tinygrad

* max parallel? if not, i have the other remedy ready in build-all

* revert me!

* I resynced tinygrad woo hoo

* setup shouldnt fail

* pull

* big ole diff

* condition

* Update build-all-tinygrad-models.yaml

* not always() never always() never!!!

* not failure instead of great success

* Update build-all-tinygrad-models.yaml

* yay that worked. lets invoke build-single one last time

* these arent used and are just taking up 250MB space

* really frog?

* bump back to 3

* self-hosted, tici

* rename to trigger tests

* 2 and done

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-07-31 09:15:06 -07:00
DevTekVE
9bdfd46b8f ci: updating gh runner and adjust restore process (#1022)
* chore(ci): parameterize and update GitHub runner version to 2.325.0

* ci: refactor GitHub runner service installation logic

Improved service installation workflow by introducing service template creation and cleaning up redundant checks. Enhanced restoration logic with clearer system state assessment and simplified flow for better maintainability.
2025-07-05 20:55:27 +02:00
DevTekVE
ea8eaed1aa Merge remote-tracking branch 'comma/master' into sync-20250627
# Conflicts:
#	README.md
#	opendbc_repo
#	panda
#	selfdrive/ui/qt/offroad/settings.cc
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_fr.ts
#	selfdrive/ui/translations/main_ja.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
#	system/manager/build.py
#	system/ui/spinner.py
#	tinygrad_repo
#	tools/lib/framereader.py
Merge branch 'comma-202506127-bedcb896644528aed6af448e63eeadb3dd8b2c77' into sync-20250627

# Conflicts:
#	opendbc/safety/tests/libsafety/SConscript
Merge branch 'comma-202506127-bedcb896644528aed6af448e63eeadb3dd8b2c77' into sync-20250627

# Conflicts:
#	opendbc/safety/tests/libsafety/SConscript
Merge remote-tracking branch 'comma/master' into sync-20250627

# Conflicts:
#	opendbc/safety/tests/libsafety/SConscript
Merge branch 'comma-20250627-1020d355584265391eb3acb556e4353b581fa9c0' into sync-20250627
Merge branch 'comma-20250627-1020d355584265391eb3acb556e4353b581fa9c0' into sync-20250627
Sync: `commaai/opendbc:master` into `sunnypilot/opendbc:master-new`

Sync: `commaai/opendbc:master` into `sunnypilot/opendbc:master-new`
Sync: `commaai/panda:master` into `sunnypilot/panda:master-new`

Sync: `commaai/panda:master` into `sunnypilot/panda:master-new`
2025-06-28 21:43:38 +02:00
Adeeb Shihadeh
5f559cfcc7 make it easy to copy/paste 2025-06-18 15:50:31 -07:00
Adeeb Shihadeh
42fc89a0e5 update release checklist 2025-06-18 15:48:10 -07:00
discountchubbs
cf94bb130b Merge branch commaai/openpilot:master into branch sunnypilot/sunnypilot:master-new 2025-05-27 16:28:45 -04:00
DevTekVE
3c36374bed ci: build upstream (tinnygrad) models from our pipeline (#498)
* Add job to fetch and upload model artifacts

Introduced a new `get_model` job to retrieve and upload model artifacts as part of the workflow. Updated the `build_model` job to download and clean up model files before building, ensuring an isolated and streamlined process. These changes improve modularity and artifact handling in the CI pipeline.

* Refine model file handling in build workflow

Restrict deletion and download commands to ".onnx" files to prevent unintended removal or overwriting of non-ONNX files. This ensures a more targeted and safer workflow for handling model artifacts.

* Update model artifact handling in build workflow

Changed artifact upload path to include only .onnx files and commented out the deletion of existing models. Also added a dependency for the build stage to ensure proper execution sequence.

* Update build workflow to refine model artifact handling

Removed unused code for deleting models and fixed the artifact path. Added a reference file logging custom name and upstream branch for better traceability. These changes streamline the workflow and improve clarity.

* Set dynamic run-name for sunnypilot build workflow

Added a customizable `run-name` field in the sunnypilot build model workflow. This allows more informative and dynamic naming based on input parameters such as `custom_name` or `upstream_branch`. Improves clarity and traceability in workflow runs.

* Add customization options and metadata generation to build

Introduce new inputs for file name prefix and 20Hz model specification in the workflow. Enable renaming of model files, generation of file hashes, and creation of a `metadata.json` file containing build details. This improves flexibility and enhances artifact information management.

* remove EOF

* No need for this hack anymore

* Add support for compiling ONNX models to Tinygrad format

* Add model directory environment variable and metadata generation script

* Fix quoting in model generator script command

* Refactor conditional flag for 20Hz input in build model script

* meh

* Add powersave management to build model workflow

* Add environment variable setup to build model workflow

* make it path call it a day

* kill me

* kill me x2

* mmm

* mmm

* Generate metadata.pkl

* Generate metadata.pkl

* ignore failure on getting model metadata

* Cleanup the existing onnxs

* Fix rm command syntax in build model script

* Cleaning a little

* Add optional file name parameter to generate_metadata function

* update

* stg

* linter

* more formatting

* Update sunnypilot model build workflow: improve custom naming, handle 20Hz default, and add commit date as output.

* Update commit date format in workflow script

Changed the commit date format in sunnypilot-build-model.yaml to a more human-readable format ('Month DD, YYYY'). This improves readability and consistency in the workflow outputs.

* Fix output naming in sunnypilot-build-model workflow

Corrected the output name from 'model_date' to 'commit_date' to match its usage. This ensures consistency and prevents potential errors in the workflow.

Add debug echo for model date in GitHub workflow

This change adds a debug echo statement to display the model date in the sunnypilot-build-model workflow. It helps verify the correct commit date is being set during the workflow process.

* Update BUILD_DIR path in sunnypilot workflow

Replaced hardcoded BUILD_DIR with a dynamic path using `github.workspace`. This enhances flexibility and ensures compatibility with different environments or runners.

* Revert "Update BUILD_DIR path in sunnypilot workflow"

This reverts commit 376971b616.

* Refactor model metadata generation to use short names

Replaced `file_name` with generated `short_name` for metadata and file naming. Introduced `create_short_name` function to simplify short name creation from custom names. Updated scripts and workflow to reflect the new parameter and improve metadata handling.

* `Use timezone-aware UTC timestamp for build_time`

Updated the code to use `datetime.now(timezone.utc)` instead of `datetime.utcnow()`. This ensures that the generated timestamps are timezone-aware, improving consistency and correctness in the output.

* Refactor short name generation logic in model metadata

Enhanced the `create_short_name` function to better handle name conversion by incorporating new rules for handling parentheses, alphanumeric words, and version pairs. Updated related variable names in metadata functions for improved clarity and consistency.

* Fix syntax error in assigning 'models' key in metadata

Corrected a typo where 'models' was missing quotes, causing it to be interpreted as a variable instead of a string key. This ensures proper metadata dictionary structure.

Update model metadata structure and JSON generation

Revised metadata to include detailed download URIs and a type field for artifacts. Enhanced the JSON generator with new fields like environment, runner, and overrides for better configurability. These changes aim to improve metadata clarity and compatibility with downstream processes.

* Refactor variable name from "escaped_name" to "short_name"

Renamed "escaped_name" to "short_name" across function parameters and internal logic for clarity and consistency. This improves readability and better aligns with its use case in the metadata generation process. No functional changes were introduced.

* Refactor variable names and improve type annotations.

Updated parameter names to align with Python naming conventions, enhancing readability and consistency. Added type annotations for function parameters to improve code clarity and maintainability.

* Refactor type annotations and clarify comments.

Updated type annotations to use "Str" for consistency with custom typing conventions. Improved clarity of a comment describing the normal case logic in the shortening function.

* Fix dict formatting in model_generator.py

Moved the "overrides" key for better alignment and consistency in the dictionary format. This improves readability without changing functionality.

Refactor model short name generation logic

Simplify and clarify the `create_short_name` function by improving readability and condensing comments. Adjust casing and truncation logic for single-word cases and handle versioned names more intuitively.

* Ensure custom file names are converted to lowercase.

When renaming files using a custom short name, enforce lowercase conversion to maintain consistency and prevent potential issues with case-sensitive file systems. This change applies to both the tinygrad and metadata file names.

* Use UTC alias for timezone in datetime imports

Replaced `timezone.utc` with `UTC` from the `datetime` module for clarity and consistency. This simplifies the representation of time zones and aligns with Python's updated best practices.

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-05-25 10:50:09 +02:00
Cameron Clough
8097a92515 zipapp pack (#35253)
Used to ship python UI in agnos without an openpilot clone

* add a main method to target

* pack script

* validate inputs

* refactors

* copy into temp, dont keep this

* cleanup

* help messages

* rename to pack.py

* pack.py

* updates for device

* moar

* don't use cereal

* just log normally

* use importlib.resources

* revert

* Revert "don't use cereal"

This reverts commit 7208524d422d88a1b07e209359aeb25e8b3bf4e7.

* fix cereal?

* cleanup

* Revert "cleanup"

This reverts commit 921edfe5020f244dbdf4f26767af7c98ca837d1c.

* cython hotfix

* Reapply "cleanup"

This reverts commit 9b54552f784dea1b1eb4ffc03937571e4fc851ba.

* more cleanup

* any script?

* slightly clearer

* rm print

* nothing python should use SVGs

---------

Co-authored-by: Trey Moen <trey@moen.ai>
2025-05-16 22:24:03 +01:00
DevTekVE
898f30744f ci: avoid deadlock on dev-c3 branch build when PR labeled dev-c3 (#839)
* ci: enhance PR checks to validate individual check runs before merging and ignore reset-and-squash as candidate for fail

* reset and squash script shouldn't will be cancelled only if a push to master has been made which always should take priority. The rest will be put on hold to avoid parallel squash scripts running

* Set concurrencty at workflow level?

* Set concurrencty at workflow level?

* change trigger to pull_request to validate

* Reducing the types of "pull request" events

* Keep it as target

* Playing eith run name a little

* Change to pull request not target

* simplifying

* set to just pull request without target

* maintain as pull request target
2025-04-20 13:04:35 +02:00
DevTekVE
224f43b6fa CI: Improve dev-c3-new auto build (#825)
* force commit to force PR

* ci: add job to manage PR labels and remove trust label on new commits

* Add pull request trigger

* empty commit to test

* ci: update pull request label management logic and remove FORCE_RUN flag

* ci: update trust label management to use TRUST_FORK_PR_LABEL

* ci: update workflow to manage PR labels and adjust trigger conditions

* ci: rename jobs in workflow for consistency and clarity

* ci: add unleash-nightly-squash branch to workflow

* ci: add checkout step with GITHUB_TOKEN to workflow

* force wait

* what happens if negated the wait

* ci: add 'Wait for Tests' action to monitor workflow execution

* ci: update wait-for-tests action conditions and cleanup

* ci: refine conditions for managing PR labels and waiting for tests

* ci: enhance PR label conditions for workflow execution

* Cant use env on if

* maybe

* ci: update pull request workflow conditions and cleanup

* Missing end of line

* ci: rename workflow and update job name for clarity

* Cleaning

* ci: refine conditions for pull request handling in workflow

* ci: update handling of TRUST_FORK_LABEL in PR processing

* ci: remove fork trust warning from PR processing

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-04-15 15:53:29 -04:00
DevTekVE
b9d584245f ci: update squash script for external contributors (#811)
* Get head ref info

* Simpify adding the comment with the newest flagg by gh cli

* Dynamic origin

* Enhance PR processing: add label handling for forked PRs and improve variable naming

* Refactor PR label handling: use constant for trust-fork label and improve comments

* Fix remote addition in PR processing: change subprocess call to not check for errors

* Refactor PR comment handling: support multiple comments and improve clarity

* Refactor PR processing: streamline comment handling and improve error management

* Refactor add_pr_comments function: specify comments type as list of strings

* Maybe we don't prevent "behind" branches to be merged as lon as the PR itself is mergeable

* lint, leave me alone

---------

Co-authored-by: Discountchubbs <159560811+Discountchubbs@users.noreply.github.com>
2025-04-14 19:29:33 +02:00
DevTekVE
6b3f75bbf0 CI: refactor Squash and Merge with simplified branch merging (#726)
* Refactor squash and merge script for improved simplicity

Simplified the squash_and_merge.py script by replacing redundant utility functions and consolidating logic. Enhanced usability by aligning command-line arguments and leveraging streamlined git operations to improve maintainability and reliability.

* Fix argument names in squash PR script

Renamed CLI arguments from '--base' and '--source' to '--target' and '--base' to align with expected input format. This ensures the script runs correctly with proper argument mapping.

* Fix incorrect base branch argument in squash script

    Updated the `--base` argument to use `source_branch` instead of `branch` to ensure the squash script processes the correct base branch. Also adjusted the command to include `branch` as a separate argument for clarity and correctness.

* Reset to a clean state after squash error.

Add a `git reset --hard` command to ensure the repository returns to a clean state after encountering errors during the squash and merge process. This prevents lingering changes from affecting subsequent operations.

* Improve error handling in squash_and_merge_prs.py

Capture and display both stdout and stderr in error cases to provide more informative feedback. Adjust the PR comment to include available output for better debugging.

* Refactor PR squash process to enhance error handling.

Modify subprocess handling to use `result.returncode` for error checks instead of relying on exceptions. Consolidate error output retrieval and logging for better clarity, while maintaining the workflow for resetting changes on failure.

* Fix incorrect return in PR processing loop

Replaced `return` with `continue` to ensure all PRs in the loop are processed before exiting. This prevents premature termination of the function and ensures accurate success count reporting.

* Simplify subprocess output handling in squash_and_merge.py

Replaced labeled print statements with direct output of stdout and stderr. This change ensures cleaner logs and remains consistent with the function's purpose of output handling during subprocess execution.

* Update subprocess.run calls to use capture_output parameter

Replaced `stdout` and `stderr` with the `capture_output` parameter for cleaner and more concise subprocess handling. Also removed extraneous whitespace for improved code readability.

* testing moving the squash script given that it's called iteratively and switching branch might miss it

* format

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-29 21:54:49 +00:00
Jason Wen
45c8f5c9d5 ci: Update PR title formatting for Squash and Merge (#711)
Changed the PR reference format from (`#123`) to (PR-123) in squash commit messages
to prevent GitHub from automatically adding reference comments to PRs when
`master-dev-c3-new` is force pushed.
2025-03-24 22:56:19 -04:00
DevTekVE
f0abacbc59 Merge branch 'refs/heads/master-at-17032025' into sync-17032025
# Conflicts:
#	opendbc_repo
#	selfdrive/car/card.py
#	selfdrive/car/tests/test_car_interfaces.py
#	selfdrive/car/tests/test_models.py
#	selfdrive/controls/controlsd.py
#	selfdrive/controls/lib/tests/test_latcontrol.py
#	selfdrive/pandad/panda.h
#	system/manager/process_config.py
Sync: `commaai/opendbc:master` into `sunnypilot/opendbc:master-new`
2025-03-18 01:05:34 -04:00
Jason Wen
3177142bfc ci: update existing comment during Squash and Merge (#674)
* format

* ci: update existing comment during Squash and Merge

* dynamic target branch

* test label

* try again

* no f

* test

* should be edit-last

* quote

* fix arg

* final

* bold it

* revert
2025-03-17 03:49:27 -04:00
Adeeb Shihadeh
561d33d81e remove '-release' suffix 2025-03-15 20:55:03 -07:00
Jason Wen
70f8225efc Merge branch 'upstream/openpilot/master' into sync-20250312
# Conflicts:
#    opendbc_repo
#    selfdrive/car/tests/test_models.py
2025-03-13 02:41:16 -04:00
Adeeb Shihadeh
96641ac12c no gh actions in release 2025-03-12 14:56:40 -07:00
DevTekVE
a4ef5ad982 Merge remote-tracking branch 'comma/master' into sync-20250309
# Conflicts:
#	.github/workflows/ui_preview.yaml
#	common/params_keys.h
#	msgq_repo
#	opendbc_repo
#	panda
#	selfdrive/ui/qt/offroad/firehose.h
#	selfdrive/ui/tests/test_ui/run.py
#	system/manager/process_config.py
2025-03-09 14:53:03 +01:00
Jason Wen
290383128e Sync: Fix conflicts 2025-03-09 00:53:06 -05:00
Adeeb Shihadeh
c3a83ccc63 simplify release build 2025-03-06 16:00:48 -08:00
Adeeb Shihadeh
54f1b00447 Log git source commit (#34749)
* Log git source commit

* strip

* fix trim

* no trim
2025-03-02 13:11:38 -08:00
DevTekVE
1a8dd310ae Model: split modeld into it's own contained modeld implementation (#642)
* Add support for TinyGrad model runner processing

Introduced a new function `is_tinygrad_model` to detect TinyGrad as an active model runner. Updated the `is_stock_model` logic to account for TinyGrad models and added a new process entry for TinyGrad in the model manager. This enables handling TinyGrad models alongside existing configurations.

adding modeld back

Add support for `modeld_v2` and update paths for consistency

Updated `SConscript` files to integrate `modeld_v2` alongside `modeld` and adjusted script paths for correct metadata handling. Adjusted various configurations and scripts, such as `labeler.yaml` and `build_release.sh`, to include `modeld_v2` and ensure cohesive project structure.

Refactor imports to use updated `modeld_v2` paths.

Replaced outdated `modeld` references with their `modeld_v2` counterparts for consistency and clarity across the codebase. Also updated `.gitignore` to accommodate new directory structure. This change ensures better maintainability and alignment with the new directory schema.

Refactor and reorganize modeld to sunnypilot/modeld_v2 structure.

Moved and renamed `modeld` components to the new `sunnypilot/modeld_v2` directory for better organization and modularity. Updated imports and file references to align with the new structure, ensuring compatibility and functionality. Streamlined project structure to improve maintainability and future development.

* typo

* Use `stock` model runner and refactor model checks.

Replaces outdated model detection logic with unified `stock` runner integration, simplifying the decision flow for model selection. Includes `stock` as a new enum in the `Runner` type and updates affected references accordingly.

* Handle missing 'sim_pose' in model outputs gracefully.

Added conditional checks to ensure the code handles cases where 'sim_pose' is absent in the model outputs. Fallback behaviors use 'plan' data when 'sim_pose' is unavailable, preventing potential errors and enhancing robustness.
2025-03-02 20:49:30 +01:00
Adeeb Shihadeh
095088ea91 Deprecate master-ci (#34744) 2025-03-01 13:16:44 -08:00
DevTekVE
cce0f14976 CI: Fix runner install script (#639)
Not sure why it stopped working before but this should work now
2025-03-01 15:29:31 +01:00
ZwX1616
29ca4aa7de modeld refactor: supercombo -> vision+policy (#34718)
* should be ok

* update tg

* type

* wrong model

* is no more

* update readme

* add back

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-02-27 17:00:56 -08:00
DevTekVE
403d77ff3f ci: squash and merge script update to allow merging less restrictively (#610)
Fix branch status check for merge readiness

Previously, any non-"CLEAN" status prevented merging. The check now specifically fails only if the branch is "BEHIND", allowing other statuses to proceed if valid. This ensures more accurate merge validations.
2025-01-22 13:41:56 +01:00
DevTekVE
3e4be4a393 ci: Add branch reset workflow and improve squash script (#579)
* Add nightly branch reset workflow and improve squash script

Introduced a GitHub Actions workflow to reset and squash PRs nightly for the `master-dev-c3-new-test` branch. Enhanced `squash_and_merge.py` to handle more specific exit codes and `squash_and_merge_prs.py` to streamline PR processing. Updated argument handling in scripts and added validation for squash script execution.

* Forcing to show up

* UnForcing to show up

* Refactor branch handling to use inputs/environments directly

Removed intermediate step for setting branch variables and updated logic to use `inputs` or environment variables directly. Simplifies the workflow and improves maintainability by reducing redundancy and reliance on unnecessary outputs.

* Fix Python script invocation in CI workflow.

Replaced implicit script execution with an explicit `python3` command to ensure compatibility and consistency in the workflow. This change resolves potential issues with shell defaults or system configurations.

* Update branch reset logic in workflow script

Replaces checkout-based branch reset with deletion and recreation to ensure the target branch correctly points to the source branch. This change handles cases where the target branch may already exist.

* Refactor PR data handling to parse JSON input.

Updated the script to parse and handle PR data as JSON, ensuring proper data structure during processing. Adjusted functions to operate on parsed JSON instead of raw arguments for improved clarity and error handling.

* Refactor PR processing to streamline branch handling

Removed redundant `fetch_pr_branches` function and integrated branch fetching directly into `process_pr`. Simplified subprocess calls for clarity and added branch cleanup to prevent conflicts. This improves code maintainability and execution efficiency.

* Add PR number to squash and merge commit titles

This change appends the PR number to the title used in squash and merge commits, improving traceability and clarity in the commit history. The modification ensures easier identification of commits linked to specific pull requests.

* Enhance PR processing with sorting and additional checks

Implemented sorting of PRs by creation date and added checks for merge conflicts, commit data availability, and status check completion. Updated nightly squash script to provide detailed feedback via PR comments for skipped PRs. These changes improve the reliability and traceability of the merge process.

* Add traceback logging to error handling in squash_and_merge_prs

Enhanced error reporting by including full traceback details when an exception occurs in the `process_pr` function. This aids in debugging by providing more context on failures.

* Refactor and add debug output to PR merge check logic

Simplified multi-line statements into single lines for clarity and added debug `print` statements to log `merge_status` and its output. These changes enhance readability and facilitate debugging of the merge conflict check process.

* Add GITHUB_TOKEN to environment for CI workflow

Ensure the workflow has access to the GITHUB_TOKEN for authentication. This is necessary for interacting securely with the GitHub API during the CI process. Without this, some steps may fail due to lack of authorization.

* Enable scheduled workflow execution at midnight UTC

Reactivates the cron schedule for the workflow to run daily at midnight UTC. This ensures the workflow executes automatically without manual triggers, maintaining regular updates or checks.

* test

* Update workflow to trigger and monitor selfdrive tests

Replaced the direct triggering of the prebuilt workflow with a step to trigger and wait for the completion of selfdrive tests. Ensures prebuilt workflow runs only if selfdrive tests succeed, improving reliability of the CI process.

* Refine workflow trigger to fetch run URL and ID

Updated the workflow script to extract the run URL and derive the workflow ID from it. This ensures more accurate handling and tracking of GitHub Actions runs.

* Simplify selfdrive test triggering in workflow

Replaces custom script with a reusable GitHub Action to trigger and wait for selfdrive test completion. This improves maintainability and reduces complexity in the workflow file. Adjusts subsequent prebuilt workflow trigger to ensure compatibility.

* Remove duplicate prebuilt workflow trigger step

The redundant step for triggering the sunnypilot prebuilt workflow has been removed. This cleanup avoids unnecessary duplication and ensures a more streamlined workflow definition.

* Update selfdrive test trigger to use GitHub CLI commands

Replaced the third-party action with GitHub CLI for triggering and monitoring selfdrive tests. This change improves maintainability and reduces reliance on external dependencies. Updated related steps to ensure compatibility with the new approach.

* Add delay to ensure selfdrive tests workflow starts

Introduce a 120-second sleep before fetching the latest run ID to allow sufficient time for the selfdrive tests action to initialize. This prevents potential issues caused by attempting to retrieve the run ID too early.

* Improve push step to check for diffs before execution

Added logic to verify if there are differences between local and remote branches before attempting a push. This prevents unnecessary pushes and skips subsequent workflows when no changes are detected. Updated dependent steps to conditionally run based on the presence of changes.

* Update target branch and improve squash comment clarity

Renamed the default target branch from `master-dev-c3-new-test` to `nightly` in the workflow configuration. Enhanced squash process comments to dynamically reflect the `target_branch` value for better clarity and consistency.

* Add missing newline at end of file

Ensures the file complies with POSIX standards by including a newline at the end. This improves consistency and prevents potential issues with some tools or systems.

* Update default target branch and disable nightly schedule

Changed the default target branch to 'master-dev-c3-new' for workflow consistency. Commented out the nightly schedule to temporarily disable automated runs. No functional changes were made to other parts of the workflow.

* Refactored squash and merge scripts for improved PR handling

In this commit, significant updates have been made to the 'squash_and_merge.py' and 'squash_and_merge_prs.py' scripts related to how pull requests (PRs) are processed.

A 'source-branch' argument has been added to the argument parser. The merging command has been changed from 'merge' to 'rebase'.

The PR processing function has been refined. Specifically, PR validation is now a separate function confirming the conditions 'branch name', 'commit data', 'check pass status', and 'mergeability'. Now, any failures under these conditions result in skipping the PR with an appropriate warning.

The target branch is deleted if it exists, before a new one is created from the source branch. The squash script now runs with more structured arguments.

These changes generally improve PR handling in CI/CD pipelines, making them more efficient and error-resistant.

* Add 'PullRequest' to .codespellignore

This update includes 'PullRequest' in the .codespellignore file to prevent it from being flagged as a spelling error. It helps streamline code reviews and reduces false positives during spell checks.
2025-01-22 09:30:53 +01:00
Jason Wen
afde2771f8 Merge branch 'upstream/openpilot/master' into sync-20250116
# Conflicts:
#    .github/workflows/ui_preview.yaml
#    opendbc_repo
#    panda
#    release/release_files.py
#    selfdrive/test/process_replay/ref_commit
#    selfdrive/ui/tests/test_ui/run.py
2025-01-16 17:12:53 -05:00
Adeeb Shihadeh
57abd8b0c2 Remove release files (conceptually) (#34379)
* Remove release files (conceptually)

* fix
2025-01-13 19:31:21 -08:00
DevTekVE
2f2b13382b CI: update install script to allow restore of github runner after agnos update (#562)
* Refactor GitHub runner installation script

Enhance the script with modular functions for better readability and maintenance. Add support for restoration mode, enabling the recovery of existing runner configurations. Improve error handling, argument parsing, and system configuration setup.

* Ensure runner directory existence before restoration

Add a prerequisite check to verify the runner directory exists, preventing restoration failures due to a missing directory. Adjust exit code logic for proper handling when no restoration is required.

* rw not needed until a bit later

* Refactor system configuration handling and remount logic.

Reorganized system setup functions and centralized remount logic for consistency and clarity. Improved script structure by separating utility and setup functions, ensuring proper remounting behavior with a robust exit trap. Reduced redundancy and enhanced maintainability.

* silly change
2025-01-12 11:37:02 +01:00
DevTekVE
63f0237e7a prebuilt: Squash and merge script (#497)
* Add script for squash and merge workflow automation

Introduced `squash_and_merge.py` to automate the process of squashing a source branch and merging it into a target branch. The script handles backups, commit message creation, stashing, cleanup, and includes optional push functionality. Provides user prompts and error handling to simplify complex merge workflows.

* temp1

* Switch to PEP 604 style for optional and generic type hints

Replaces `Optional` and `List` with the simpler `X | None` and `list[X]` syntax introduced in Python 3.10. Updated all function signatures accordingly for consistency and modernity. Also made minor string formatting consistency adjustments.

* Fix type annotation for temp_branch variable

Updated the temp_branch variable to include a proper type annotation (`str | None`). This change improves code clarity and aligns with Python typing standards.

* more hints
2025-01-04 09:43:03 +01:00