* Add tinygrad ref testing
* BaseDir and give a fake ref id to test
* one more before the real thing
* Add the correct ref
* test
* Update test_tinygrad_ref.py
* Update tinygrad_ref
* Update test_tinygrad_ref.py
* Update tinygrad_ref
* This SHOULD FAIL
* Revert "This SHOULD FAIL"
This reverts commit 58862f8a73.
* bit of red
* Move ref to models so compiled branches can read it
* step one
* step 2
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update tinygrad_ref.py
* Update build-all-tinygrad-models.yaml
* bump to fail test
* Revert "bump to fail test"
This reverts commit 4f58991f32.
* pytest should take care of it
* lint
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* add raylib ui
* test
* this is better for now
* rm
rm
* finalize it
* need this?
* ?
* shite
shite
* try
* ?
* huh
* simp
* ?
* wtf is going on
* ???????????????
* lock
* stash
* no 2 packages
* Revert "stash"
This reverts commit 9efb0d9bda6a6309e7a567634d1921bf1cd0fb59.
* debug
* noo
* debug
* ?
* and
* yeah yeah
* init one
* 2
* i wonder
* oooh
* make sure
* fix dat
* try this
* see if wifiman
* forgot
* ?
* ???
* fuck this we can rewrite it later
* ci: dynamic submodule check for build_release
* test opendbc diff
* somem fix
* this way
* use path
* use master branch instead
* less verbose
* test bump
* test 1 more sub change
* unbump
* only echo if there's a diff
* Tinygrad bump from sync-20250627
* bump tinygrad_repo
* Reformat metadata generator to match driving_models.json
* bump tinygrad
* Revert "bump tinygrad"
This reverts commit f479dfd502.
* revert me after SP model compiled
* Model recompiled successfully, initiate "revert me after SP model compiled"
This reverts commit 95706eb688.
* The "FillMe" placeholder caused an extra 10 seconds of work
* bump to 22Jul2025
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Allow more dynamic short names
This should hopefully be future-proof for now.. It's robust enough to return the correct word-digit format (see example on how it generates from given display name below):
'Last Horizon V2 (November 22, 2024)' -> LHV2
'Alabama (November 25, 2024)' -> ALABAMA
'PlayStation (December 03, 2024)' -> PLAYSTAT
'Postal Service (December 09, 2024)' -> PS
'Null Pointer (December 13, 2024)' -> NP
'North America (December 16, 2024)' -> NA
'National Public Radio (December 18, 2024)' -> NPR
'Filet o Fish (March 7, 2025)' -> FOF
'Tomb Raider 2 (April 18, 2025)' -> TR2
'Tomb Raider 3 (April 22, 2025)' -> TR3
'Tomb Raider 4 (April 25, 2025)' -> TR4
'Tomb Raider 5 (April 25, 2025)' -> TR5
'Tomb Raider 6 (April 30, 2025)' -> TR6
'Tomb Raider 7 (May 07, 2025)' -> TR7
'Down to Ride (Revision: May 10, 2025)' -> DTR
'SP Vikander Model (May 16, 2025)' -> SPVM
'VFF Driving (May 15, 2025)' -> VFFD
'Secret Good Openpilot (May 16, 2025)' -> SGO
'Vegetarian Filet o Fish (May 29, 2025)' -> VFOF
'Down To Ride (Revision: May 30, 2025)' -> DTR
'Vegetarian Filet o Fish v2 (June 05, 2025)' -> VFOFV2
'Kerrygold Driving (June 08, 2025)' -> KD
'Tomb Raider 10 (June 16, 2025)' -> TR10
'Organic Kerrygold (June 17, 2025)' -> OK
'Liquid Crystal Driving (June 21, 2025)' -> LCD
'Vegetarian Filet o Fish v3 (June 21, 2025)' -> VFOFV3
'Vibe Model [Custom Model]' -> VMCM
'Tomb Raider 13 (June 27, 2025)' -> TR13
'Aggressive TR (June 28, 2025)' -> ATR
'Tomb Raider 14 (June 30, 2025)' -> TR14
'Cookiemonster Tomb Raider (July 02, 2025)' -> CTR
'Down to Ride (Revision: July 07, 2025)' -> DTR
'Simple Plan Driving (July 07, 2025)' -> SPD
'Down to Ride (Revision: July 08, 2025)' -> DTR
'Tomb Raider 15 (July 09, 2025)' -> TR15
'Tomb Raider 15 rev-2 (July 11, 2025)' -> TR15R2
'Le Tomb Raider 14 (July 14, 2025)' -> LTR14
'Le Tomb Raider 14h (July 17, 2025)' -> LTR14H
'Tomb Raider 16 (July 18, 2025)' -> TR16
'Tomb Raider 16v2 (July 21, 2025)' -> TR16V2
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* No need to sleep 3 seconds, just send it
* try dynamic
* cleanup
* Update build-single-tinygrad-model.yaml
* bc devtekve said. also, this is repetitive af
* Revert "bc devtekve said. also, this is repetitive af"
This reverts commit 3a0c1562de.
* maybe we could use a script instead that both build all
That both build all and sunnypilot-build-model reference
* refactor: consolidate model building steps into a single workflow
* tweak
* tweakx2
* tweakx3
* tweakx4
* dunno dunno...
* output dir
* lots of changes
* Revert "lots of changes"
This reverts commit 4aadb0ee29.
* fail if all fail
* no inputs needed
* make it easier for us
* note failure and exit 0
* Update build-all-tinygrad-models.yaml
* not needed unless we really want it
* Update build-single-tinygrad-model.yaml
* Merge branch 'sync-20250627-tinygrad' of github.com:sunnypilot/sunnypilot into sync-20250627-tinygrad
* retry for failed ?
* always run this step because sometimes one build fails
which causes the matrix to fail, but most builds still have uploaded artifacts.
* strip
* no escape
* Update build-all-tinygrad-models.yaml
* Test case from terminal run
(openpilot) james@Mac sunnypilot % jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \\([^)]*\\)"; "")), is_20hz}]' \
/Users/james/Documents/GitHub/sunnypilot-docs/docs/driving_models_v6.json > matrix.json
mkdir -p output
touch "output/model-Tomb Raider 16v2 (July 21, 2025)-544"
touch "output/model-Space Lab Model (July 24, 2025)-547"
touch "output/model-Space Lab Model v1 (July 24, 2025)-548"
built=(); while IFS= read -r line; do built+=("$line"); done < <(
ls output | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ \([^)]*\)//' | awk '{gsub(/^ +| +$/, ""); print}'
)
jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \
'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' \
matrix.json > retry_matrix.json
cat retry_matrix.json
[]
(openpilot) james@Mac sunnypilot %
* always
* great success
* add suffix to retry artifact so it doesn't conflict
* retry to get_model too
* and there haha
* unnecessary hyphen
* compare built to missing. include retries
* adjust copy of artifacts.
* Update build-all-tinygrad-models.yaml
* Update model selector versioning and add documentation
* Update retry condition for failed models in build-all-tinygrad-models.yaml
* Update retry condition for failed models in build-all-tinygrad-models.yaml
* Update build-single-tinygrad-model.yaml
* false
* default none because why not
* red diff? i think?
* meh ... not needed i guess
* error error error
* Nayan is watching... always watching mike wazowski
* string all the way
* lots of retries just in case because im scared
* more robust
* ONLY ONE!!!!!!
* delete.... a lot
* fix artifacts
* fix artifacts
* make sure each is unique :)
* skip files like artifact duhhhh
* artifact name dir
* concurrency
* copy here
* Update build-single-tinygrad-model.yaml
* Update build-single-tinygrad-model.yaml
* bump
* bump tinygrad
* max parallel? if not, i have the other remedy ready in build-all
* revert me!
* I resynced tinygrad woo hoo
* setup shouldnt fail
* pull
* big ole diff
* condition
* Update build-all-tinygrad-models.yaml
* not always() never always() never!!!
* not failure instead of great success
* Update build-all-tinygrad-models.yaml
* yay that worked. lets invoke build-single one last time
* these arent used and are just taking up 250MB space
* really frog?
* bump back to 3
* self-hosted, tici
* rename to trigger tests
* 2 and done
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* Tinygrad bump from sync-20250627
* bump tinygrad_repo
* Reformat metadata generator to match driving_models.json
* bump tinygrad
* Revert "bump tinygrad"
This reverts commit f479dfd502.
* revert me after SP model compiled
* Model recompiled successfully, initiate "revert me after SP model compiled"
This reverts commit 95706eb688.
* The "FillMe" placeholder caused an extra 10 seconds of work
* bump to 22Jul2025
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Allow more dynamic short names
This should hopefully be future-proof for now.. It's robust enough to return the correct word-digit format (see example on how it generates from given display name below):
'Last Horizon V2 (November 22, 2024)' -> LHV2
'Alabama (November 25, 2024)' -> ALABAMA
'PlayStation (December 03, 2024)' -> PLAYSTAT
'Postal Service (December 09, 2024)' -> PS
'Null Pointer (December 13, 2024)' -> NP
'North America (December 16, 2024)' -> NA
'National Public Radio (December 18, 2024)' -> NPR
'Filet o Fish (March 7, 2025)' -> FOF
'Tomb Raider 2 (April 18, 2025)' -> TR2
'Tomb Raider 3 (April 22, 2025)' -> TR3
'Tomb Raider 4 (April 25, 2025)' -> TR4
'Tomb Raider 5 (April 25, 2025)' -> TR5
'Tomb Raider 6 (April 30, 2025)' -> TR6
'Tomb Raider 7 (May 07, 2025)' -> TR7
'Down to Ride (Revision: May 10, 2025)' -> DTR
'SP Vikander Model (May 16, 2025)' -> SPVM
'VFF Driving (May 15, 2025)' -> VFFD
'Secret Good Openpilot (May 16, 2025)' -> SGO
'Vegetarian Filet o Fish (May 29, 2025)' -> VFOF
'Down To Ride (Revision: May 30, 2025)' -> DTR
'Vegetarian Filet o Fish v2 (June 05, 2025)' -> VFOFV2
'Kerrygold Driving (June 08, 2025)' -> KD
'Tomb Raider 10 (June 16, 2025)' -> TR10
'Organic Kerrygold (June 17, 2025)' -> OK
'Liquid Crystal Driving (June 21, 2025)' -> LCD
'Vegetarian Filet o Fish v3 (June 21, 2025)' -> VFOFV3
'Vibe Model [Custom Model]' -> VMCM
'Tomb Raider 13 (June 27, 2025)' -> TR13
'Aggressive TR (June 28, 2025)' -> ATR
'Tomb Raider 14 (June 30, 2025)' -> TR14
'Cookiemonster Tomb Raider (July 02, 2025)' -> CTR
'Down to Ride (Revision: July 07, 2025)' -> DTR
'Simple Plan Driving (July 07, 2025)' -> SPD
'Down to Ride (Revision: July 08, 2025)' -> DTR
'Tomb Raider 15 (July 09, 2025)' -> TR15
'Tomb Raider 15 rev-2 (July 11, 2025)' -> TR15R2
'Le Tomb Raider 14 (July 14, 2025)' -> LTR14
'Le Tomb Raider 14h (July 17, 2025)' -> LTR14H
'Tomb Raider 16 (July 18, 2025)' -> TR16
'Tomb Raider 16v2 (July 21, 2025)' -> TR16V2
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* No need to sleep 3 seconds, just send it
* try dynamic
* cleanup
* Update build-single-tinygrad-model.yaml
* bc devtekve said. also, this is repetitive af
* Revert "bc devtekve said. also, this is repetitive af"
This reverts commit 3a0c1562de.
* maybe we could use a script instead that both build all
That both build all and sunnypilot-build-model reference
* refactor: consolidate model building steps into a single workflow
* tweak
* tweakx2
* tweakx3
* tweakx4
* dunno dunno...
* output dir
* lots of changes
* Revert "lots of changes"
This reverts commit 4aadb0ee29.
* fail if all fail
* no inputs needed
* make it easier for us
* note failure and exit 0
* Update build-all-tinygrad-models.yaml
* not needed unless we really want it
* Update build-single-tinygrad-model.yaml
* Merge branch 'sync-20250627-tinygrad' of github.com:sunnypilot/sunnypilot into sync-20250627-tinygrad
* retry for failed ?
* always run this step because sometimes one build fails
which causes the matrix to fail, but most builds still have uploaded artifacts.
* strip
* no escape
* Update build-all-tinygrad-models.yaml
* Test case from terminal run
(openpilot) james@Mac sunnypilot % jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \\([^)]*\\)"; "")), is_20hz}]' \
/Users/james/Documents/GitHub/sunnypilot-docs/docs/driving_models_v6.json > matrix.json
mkdir -p output
touch "output/model-Tomb Raider 16v2 (July 21, 2025)-544"
touch "output/model-Space Lab Model (July 24, 2025)-547"
touch "output/model-Space Lab Model v1 (July 24, 2025)-548"
built=(); while IFS= read -r line; do built+=("$line"); done < <(
ls output | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ \([^)]*\)//' | awk '{gsub(/^ +| +$/, ""); print}'
)
jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \
'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' \
matrix.json > retry_matrix.json
cat retry_matrix.json
[]
(openpilot) james@Mac sunnypilot %
* always
* great success
* add suffix to retry artifact so it doesn't conflict
* retry to get_model too
* and there haha
* unnecessary hyphen
* compare built to missing. include retries
* adjust copy of artifacts.
* Update build-all-tinygrad-models.yaml
* Update model selector versioning and add documentation
* Update retry condition for failed models in build-all-tinygrad-models.yaml
* Update retry condition for failed models in build-all-tinygrad-models.yaml
* Update build-single-tinygrad-model.yaml
* false
* default none because why not
* red diff? i think?
* meh ... not needed i guess
* error error error
* Nayan is watching... always watching mike wazowski
* string all the way
* lots of retries just in case because im scared
* more robust
* ONLY ONE!!!!!!
* delete.... a lot
* fix artifacts
* fix artifacts
* make sure each is unique :)
* skip files like artifact duhhhh
* artifact name dir
* concurrency
* copy here
* Update build-single-tinygrad-model.yaml
* Update build-single-tinygrad-model.yaml
* bump
* bump tinygrad
* max parallel? if not, i have the other remedy ready in build-all
* revert me!
* I resynced tinygrad woo hoo
* setup shouldnt fail
* pull
* big ole diff
* condition
* Update build-all-tinygrad-models.yaml
* not always() never always() never!!!
* not failure instead of great success
* Update build-all-tinygrad-models.yaml
* yay that worked. lets invoke build-single one last time
* these arent used and are just taking up 250MB space
* really frog?
* bump back to 3
* self-hosted, tici
* rename to trigger tests
* split sync
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* ci: refactor PR handling with output file storage and streamlined input parsing
* try runner.temp
* base64 it
* simple simple
* double it and give it to the next person
* One more test
* Apply suggestions from code review
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* gh workflows: Create build single and build all models
Recompile action, and build single model action...
I'm tired, and this could definitely most probably be wrong.
on push : Revert me!
Revert "on push : Revert me!"
This reverts commit c6ab17adad981585f93c285e9d8a418e4c975879.
checkout gitlab
Update build-single-tinygrad-model.yaml
Whoops. ubuntu-latest not self-hosted
self-hosted is comma device
gh api because artifacts
Dynamic long
Update build-single-tinygrad-model.yaml
* FoF to FOF, because spellchecker is no fun
* ignore
* Create new dir and json by default since its full recompile
red diff, who doesn't like it
* Minimal patch to ssh agent from yesterday
* Update .codespellignore
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* gate
* more
* more
* check
* more
* fix
* more
* check
* Revert "check"
This reverts commit c30941bae0cbdb8f908883658bf1b0a4839c2be1.
* back
* more
* like before
* again
* more
* Add job to fetch and upload model artifacts
Introduced a new `get_model` job to retrieve and upload model artifacts as part of the workflow. Updated the `build_model` job to download and clean up model files before building, ensuring an isolated and streamlined process. These changes improve modularity and artifact handling in the CI pipeline.
* Refine model file handling in build workflow
Restrict deletion and download commands to ".onnx" files to prevent unintended removal or overwriting of non-ONNX files. This ensures a more targeted and safer workflow for handling model artifacts.
* Update model artifact handling in build workflow
Changed artifact upload path to include only .onnx files and commented out the deletion of existing models. Also added a dependency for the build stage to ensure proper execution sequence.
* Update build workflow to refine model artifact handling
Removed unused code for deleting models and fixed the artifact path. Added a reference file logging custom name and upstream branch for better traceability. These changes streamline the workflow and improve clarity.
* Set dynamic run-name for sunnypilot build workflow
Added a customizable `run-name` field in the sunnypilot build model workflow. This allows more informative and dynamic naming based on input parameters such as `custom_name` or `upstream_branch`. Improves clarity and traceability in workflow runs.
* Add customization options and metadata generation to build
Introduce new inputs for file name prefix and 20Hz model specification in the workflow. Enable renaming of model files, generation of file hashes, and creation of a `metadata.json` file containing build details. This improves flexibility and enhances artifact information management.
* remove EOF
* No need for this hack anymore
* Add support for compiling ONNX models to Tinygrad format
* Add model directory environment variable and metadata generation script
* Fix quoting in model generator script command
* Refactor conditional flag for 20Hz input in build model script
* meh
* Add powersave management to build model workflow
* Add environment variable setup to build model workflow
* make it path call it a day
* kill me
* kill me x2
* mmm
* mmm
* Generate metadata.pkl
* Generate metadata.pkl
* ignore failure on getting model metadata
* Cleanup the existing onnxs
* Fix rm command syntax in build model script
* Cleaning a little
* Add optional file name parameter to generate_metadata function
* update
* stg
* linter
* more formatting
* Update sunnypilot model build workflow: improve custom naming, handle 20Hz default, and add commit date as output.
* Update commit date format in workflow script
Changed the commit date format in sunnypilot-build-model.yaml to a more human-readable format ('Month DD, YYYY'). This improves readability and consistency in the workflow outputs.
* Fix output naming in sunnypilot-build-model workflow
Corrected the output name from 'model_date' to 'commit_date' to match its usage. This ensures consistency and prevents potential errors in the workflow.
Add debug echo for model date in GitHub workflow
This change adds a debug echo statement to display the model date in the sunnypilot-build-model workflow. It helps verify the correct commit date is being set during the workflow process.
* Update BUILD_DIR path in sunnypilot workflow
Replaced hardcoded BUILD_DIR with a dynamic path using `github.workspace`. This enhances flexibility and ensures compatibility with different environments or runners.
* Revert "Update BUILD_DIR path in sunnypilot workflow"
This reverts commit 376971b616.
* Refactor model metadata generation to use short names
Replaced `file_name` with generated `short_name` for metadata and file naming. Introduced `create_short_name` function to simplify short name creation from custom names. Updated scripts and workflow to reflect the new parameter and improve metadata handling.
* `Use timezone-aware UTC timestamp for build_time`
Updated the code to use `datetime.now(timezone.utc)` instead of `datetime.utcnow()`. This ensures that the generated timestamps are timezone-aware, improving consistency and correctness in the output.
* Refactor short name generation logic in model metadata
Enhanced the `create_short_name` function to better handle name conversion by incorporating new rules for handling parentheses, alphanumeric words, and version pairs. Updated related variable names in metadata functions for improved clarity and consistency.
* Fix syntax error in assigning 'models' key in metadata
Corrected a typo where 'models' was missing quotes, causing it to be interpreted as a variable instead of a string key. This ensures proper metadata dictionary structure.
Update model metadata structure and JSON generation
Revised metadata to include detailed download URIs and a type field for artifacts. Enhanced the JSON generator with new fields like environment, runner, and overrides for better configurability. These changes aim to improve metadata clarity and compatibility with downstream processes.
* Refactor variable name from "escaped_name" to "short_name"
Renamed "escaped_name" to "short_name" across function parameters and internal logic for clarity and consistency. This improves readability and better aligns with its use case in the metadata generation process. No functional changes were introduced.
* Refactor variable names and improve type annotations.
Updated parameter names to align with Python naming conventions, enhancing readability and consistency. Added type annotations for function parameters to improve code clarity and maintainability.
* Refactor type annotations and clarify comments.
Updated type annotations to use "Str" for consistency with custom typing conventions. Improved clarity of a comment describing the normal case logic in the shortening function.
* Fix dict formatting in model_generator.py
Moved the "overrides" key for better alignment and consistency in the dictionary format. This improves readability without changing functionality.
Refactor model short name generation logic
Simplify and clarify the `create_short_name` function by improving readability and condensing comments. Adjust casing and truncation logic for single-word cases and handle versioned names more intuitively.
* Ensure custom file names are converted to lowercase.
When renaming files using a custom short name, enforce lowercase conversion to maintain consistency and prevent potential issues with case-sensitive file systems. This change applies to both the tinygrad and metadata file names.
* Use UTC alias for timezone in datetime imports
Replaced `timezone.utc` with `UTC` from the `datetime` module for clarity and consistency. This simplifies the representation of time zones and aligns with Python's updated best practices.
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* enable cache for Mac brew and scons
* bump
* save cache by default, explicitly opt-out
* Delete bump-ci
---------
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
Enhance action to fail on non-successful workflow runs
Add logic to check the conclusion of the watched workflow run. If the run ends with a non-successful status, the action now exits with an error to improve error handling and ensure reliability.
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* Refactor workflow to centralize branch source handling
Moved branch resolution logic to a single `SOURCE_BRANCH` variable for consistency and readability. Updated related conditionals and parameters to use this new variable, reducing redundancy and improving maintainability of branch-specific configurations.
* Refactor workflow to use repository-defined variables.
Replaced hardcoded branch names with repository-defined variables for improved configurability and maintainability. This ensures flexibility by allowing changes via repository settings without modifying the workflow file.
