Commit Graph

14452 Commits

Author SHA1 Message Date
DevTekVE
bd49be185f ci: enable macos runners again (#495)
no need to block macos anymore. Turns out runners are free on github public repos
2024-12-23 01:03:21 +01:00
DevTekVE
37e493ce0d CI: Add GitHub Action to build models from upstream (#491)
* Add support for custom model naming in workflow

This update introduces a new optional input `custom_name` to the `sunnypilot-build-model` GitHub workflow. If provided, it customizes the artifact name; otherwise, the default naming scheme is used. This enhances flexibility for build artifact identification.

* fix name

* Update cache key logic in build workflow

Replaced `github.ref_name` with a fallback to `github.head_ref || github.ref_name` in cache keys to handle branches and pull requests more reliably. This ensures consistent cache retrieval and improves workflow efficiency.

* Simplify artifact upload logic in CI workflow

Consolidated redundant artifact upload steps into a single action, leveraging a fallback mechanism for naming. This streamlines the workflow, reducing duplication and improving maintainability.

* Make publish step resilient and add conditional notify

Added `continue-on-error` to the publish step to allow workflow progression even if it fails. Introduced a condition to notify step that depends on publish outcome being successful. These changes improve workflow reliability and error handling.

* Fix conditional syntax in GitHub Actions workflow

Updated the conditional statement in the notify job to use the correct GitHub Actions expression syntax. This ensures proper evaluation of the publish step's success outcome.

* Update condition for notify step in GitHub Actions workflow

Replaced the deprecated syntax `${{ steps.publish.outcome == 'success' }}` with the recommended `success()` function. This ensures compatibility with newer GitHub Actions features and improves maintainability.

* Update build workflow to handle PR drafts and improve concurrency

Added support for pull request draft events to trigger the workflow, ensuring drafts are not missed. Removed `continue-on-error` in the publish step to improve reliability and prevent silent failures. Simplifies and enhances workflow robustness.

* Update pull request workflows and add draft check logic

Removed unnecessary pull request event types and drafts settings, consolidating logic. Added a condition to skip publishing for non-draft pull requests, ensuring better control over the deployment process.
2024-12-23 00:37:02 +01:00
DevTekVE
e114dc5a6f CI: Use environments for manual approvals on GH Actions for feature branches (#493)
* We no longer need to dispatch manually a workflow, it will wait for the publish to be approved for feature branches.
* The notification template has been extracted to a variable for the repo, meaning no need to do commit push to update it! (https://github.com/sunnypilot/sunnypilot/settings/variables/actions/DISCORD_GENERAL_UPDATE_NOTICE)
* The configuration for what can be auto deployed is now also a variable, no need to commit a change to update that (https://github.com/sunnypilot/sunnypilot/settings/variables/actions/AUTO_DEPLOY_PREBUILT_BRANCHES)
* The configuration for what sends a message on `#dev-feedback` channel on discord is also configurable (https://github.com/sunnypilot/sunnypilot/settings/variables/actions/DEV_FEEDBACK_NOTIFICATION_BRANCHES)
2024-12-22 18:18:21 +01:00
DevTekVE
ad537fcb89 CI: Build prebuilts and models from github and self hosted runners (#476)
* Create a GitHub Actions workflow for synchronizing the repository to GitLab and enhance GitLab CI settings

A new GitHub Actions workflow for mirroring the current repository to GitLab is added. This workflow is triggered by both push and delete events in addition to manual triggering. The role of GitHub actions runner is defined through a series of steps. A new '.gitlab-ci.yml' build configuration file is also introduced, with a more comprehensive definition of variables, jobs, and pipeline rules for its utilization in the GitLab CI/CD pipeline.

Further, other changes include the addition of scripts for installing and uninstalling GitLab CI runner, along with modifying the SCons build system configuration file to include custom cache directory. Moreover, 'release_files.py' has been revised to include additional blacklisted and whitelisted files specific to Sunnypilot, ensuring suitable settings for the CI flow.

The improvements facilitate a smoother integration between GitHub and GitLab, powerfully harnessing the capabilities of both platforms for more efficient and effective CI/CD pipelines and version control management.

* not needed for this

* Update workflow to build model from upstream repository

Revised the CI workflow to build directly from the upstream `commaai/openpilot` repository. Simplified configuration, removed unused steps, and added support for specifying the upstream branch dynamically via inputs.

* Update SConstruct to allow passing arbitrary cache_dir

Modified the SConstruct file to enable setting a custom cache directory via arguments. This enhances flexibility in configuring cache paths during the build process.

* Refactor build workflow to improve branch handling logic.

Reorganized conditions for setting environment variables, replacing repository_dispatch with workflow_dispatch for prebuilt builds. Added a fallback error message for unsupported triggers to improve robustness. This enhances clarity and ensures compatibility with the intended workflow triggers.

* test test test

* test test test

* test test test

* Enable publication flag during build configurations

Added `SHOULD_PUBLISH=true` to all relevant build configuration steps to ensure proper handling of publishing logic. Updated environment variables to include this flag for downstream usage. Removed the error-check step for unsupported configurations.

* Simplify publish condition in workflow logic

Replaced the previous condition for publishing with a single output variable, `should_publish`, to streamline logic and improve maintainability. This change reduces redundancy and makes the workflow more adaptable to future updates.

* Simplify restore key patterns in build workflow.

Removed unnecessary trailing dashes in SCons cache restore keys to streamline and slightly improve key matching logic. This ensures better consistency with the current workflow setup.

* Update cache key usage in build workflow

Replaced `github.ref_name` with `github.head_ref` for cache keys to ensure accurate branch-specific caching. Added fallback restore-keys for master branches to improve cache efficiency and reduce redundant builds.

* Improved debug logging in GitHub actions workflow

This commit refines the debug logging in our GitHub Actions workflow for the Sunnypilot build. This change provides more granularity, enabling logging only during debug mode, which helps to keep the runtime logs less cluttered during normal operations. This includes the conditionally displaying of environmental variables, GitHub output contents, and directory listings.

Additionally, debug mode verbosity was added to rsync commands to aid troubleshooting file transfers during the build process.

Blank lines were also reduced for better readability and cleaner code presentation.

* test diff path

* Refactor SCons cache handling in build workflow

Replaced hardcoded cache directory paths with an environment variable (`SCONS_CACHE_DIR`) for better maintainability and flexibility. Updated related workflow steps to utilize the new variable and adjusted cache key usage. Removed unused `BASE_BUILD_NUMBER` variable to clean up the configuration.

* Update cache key to include commit SHA in workflow

This change adds the commit SHA to the cache key in the GitHub Actions workflow. It ensures more precise caching by differentiating builds based on the specific commit, reducing potential conflicts.

* clean

Update GitHub runner service to set environment variables

Updated the ExecStart command to explicitly set HOME, USER, LOGNAME, and MAIL environment variables. This ensures the runner operates with the correct environment configuration, improving reliability and compatibility.

Refactor GitHub runner service ExecStart command.

Replaced direct command execution with running the service as a specific user using `su`. This improves compatibility and aligns with best practices for user-based execution. No functional changes are expected.

Update GitHub runner service to set environment variables

The ExecStart command now sets HOME, USER, LOGNAME, and MAIL environment variables for the runner process. This ensures proper environment initialization for the designated user, improving compatibility and reliability during execution.

Refactor GitHub runner service template handling.

Revised the `modify_service_template` function to create a properly structured service template for the GitHub runner. Updated service permissions, execution parameters, and enabled the function call to ensure usage during runner setup.

* Refactor GitHub Runner installer to improve argument parsing.

Reworked the script to implement flexible and explicit command-line argument parsing using flags like `--token`, `--repo`, and `--start-at-boot`. Added support for setting default values and enabling/disabling auto-start based on the `--start-at-boot` flag. Improved error handling and usage messaging for better user experience.

* Update cache keys in sunnypilot build workflow

Modified the cache keys to include `github.ref_name` for more precise caching and restore behavior. This improves build consistency by better differentiating between branches and refs. No changes to the overall workflow logic.

* Remove 'tinygrad/*' from release file exclusions

This change modifies the release file exclusions by removing 'tinygrad/*'. The adjustment ensures that files in the 'tinygrad' directory are now included in the release process, aligning with updated packaging requirements.

* Simplify release file exclusions list.

Removed redundant and unnecessary file patterns from the exclusions list in `release_files.py`. This streamlines the file handling process and reduces maintenance overhead.

* Refactor SCons cache path and key structure.

Updated the SCons cache directory path to `SCONS_CACHE_DIR` for clarity and consistency. Improved the caching key structure to include `github.head_ref` for better cache differentiation and restore hierarchy. Adjusted related build instructions to reflect the new environment variable.

* Refactor branch configuration in CI workflow

Standardize branch name handling by replacing hardcoded values with environment variables. This improves maintainability and simplifies updates to branch names across the workflow. Updated references to use the new dynamic environment variable approach.
2024-12-21 20:19:48 +01:00
Jason Wen
cf74e6416c Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#488) 2024-12-19 10:57:30 -05:00
devtekve
7592669d74 test_processes: update ref logs to a20d7a4 2024-12-19 15:46:00 +00:00
DevTekVE
a20d7a4279 Bump panda for sync 2024-12-19 16:30:02 +01:00
DevTekVE
e4fc6ffe7a Merge remote-tracking branch 'comma/master' into sync-20241219-3
# Conflicts:
#	panda
#	selfdrive/test/process_replay/ref_commit
2024-12-19 16:29:02 +01:00
DevTekVE
cf30110d65 CI: cherry pick maxime's fix for pipelines docker (#487)
set python upper bound to 3.13 (#34286)

* try

* test

* ...

* wow

* y

* docker

* <

(cherry picked from commit 9c9b273a3e)

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2024-12-19 15:50:20 +01:00
DevTekVE
a48d43ec2b CI: adding recurring sync to lfs (#489)
Allows us to automatically keep our LFS in sync with comma's and also to manually perform a sync if we need to. Even able to sync the LFS from a given commit hash or a given branch. Useful for model stuff.
2024-12-19 15:32:40 +01:00
Maxime Desroches
9c9b273a3e set python upper bound to 3.13 (#34286)
* try

* test

* ...

* wow

* y

* docker

* <
2024-12-18 22:17:28 -08:00
Harald Schäfer
383893d39e Long planner get accel: new function args (#34288)
* Change function args

* typo

* typo

* ref commit
2024-12-18 16:41:57 -08:00
YassineYousfi
1a7c284445 National Public Radio Model 📻 (#34259)
* f3a009b7-dcb9-41f3-8917-6fcb3cec37bf/400

* 65f26b40-56c9-4c6c-a3ac-e1788bd52567/400
2024-12-18 16:13:47 -08:00
Lukas
af5082089e cabana: issue filtering by addresses in FindSignal tool (#34283)
fixed filtering by addresses issue in cabana
2024-12-18 14:11:05 -08:00
Harald Schäfer
17ca6389e1 Tinygrad runner (#34261)
* squash

* dmonitoringmodeld: use cl transform (#34235)

* needs cleanup

* only if tici

* bump tinygrad

* check width

* base modelframe

* .

* need to be args

* more cleanup

* no _frame in base

* tici only

* its DrivingModelFrame

* .6 is fair

---------

Co-authored-by: Comma Device <device@comma.ai>

* Update tinygrad

* tg upstream

* bump tg

* bump tg

* debug

* attr

* misc cleanup

* whitespace

* remove

* Add TODOs to make python proc for modelrunners

* whitespace

---------

Co-authored-by: ZwX1616 <zwx1616@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2024-12-18 11:58:59 -08:00
Adeeb Shihadeh
ff97a43c50 fix typo 2024-12-17 20:42:44 -08:00
ZwX1616
9c3aa2e2dc camerad: add os04+4.6mm lsc profile (#34280)
* draft

* ifdef in cl

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-12-17 20:32:08 -08:00
Shane Smiskol
7ffad1935d bump msgq (#34278)
bump
2024-12-17 15:43:51 -08:00
Maxime Desroches
155d842a3b set plot limits for touch events viz (#34277)
lim
2024-12-17 15:10:13 -08:00
Dean Lee
d40fd1956d wifimanager: use asyncCall to avoid timeout when adding connections (#34273)
* use asyncCall to avoid timeout when adding Wi-Fi connections

* use async call for addTetheringConnection

* only this change

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2024-12-17 14:48:08 -08:00
Shane Smiskol
857133635c ui: wait for lte connection to update before reactivating (#34275)
async
2024-12-17 14:35:43 -08:00
Shane Smiskol
f149083e4a ui: initialize tethering connection on startup (#34274)
* initialize hotspot connection on init

* better place

* fix
2024-12-17 14:25:31 -08:00
Adeeb Shihadeh
247ee2bda8 bump panda for new USB VID 2024-12-17 11:18:41 -08:00
Jason Wen
c8fe86c552 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#484) 2024-12-17 07:04:03 -05:00
Jason Wen
952354c847 test_processes: update ref logs to fffa98e 2024-12-17 06:49:31 -05:00
Jason Wen
fffa98ee85 Merge branch 'upstream/master' into sync-20241712
# Conflicts:
#	.github/workflows/ci_weekly_run.yaml
#	opendbc_repo
#	panda
#	selfdrive/test/process_replay/ref_commit
2024-12-17 06:27:31 -05:00
Adeeb Shihadeh
e317485200 tici: fix device types (#34269) 2024-12-16 19:44:43 -08:00
Shane Smiskol
3da346e2e4 AGNOS: fix update loop (#34268)
fix update loop
2024-12-16 18:30:08 -08:00
Adeeb Shihadeh
6c1314baf9 camerad: only build debayer kernel when necessary (#34267) 2024-12-16 17:22:59 -08:00
Maxime Desroches
71b02f8001 hardwared: non blocking read for touch events (#34263)
* slow

* slow

* non blocking

* 10

* try

* simple

* int

* test

* get

* try

* clean

* read all

* nested

* simpler

* indent

* cleanup
2024-12-16 17:10:47 -08:00
Jason Wen
a984903298 Hyundai: Allow controls with Sport and Manumatic Gears (#34113)
* Hyundai CAN: Explicitly parse gear shifter values for `EV, HEV, PHEV`

* for this pr

* more segments

* found 4.0!

* only print when spornt=4.0 is found

* new outputs

* bump opendbc

* Update selfdrive/car/car_specific.py

* delete notebook

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2024-12-16 16:51:04 -08:00
Jason Wen
c69cd934af Setup sunnypilot package (#483)
Setup sunnypilot as a package
2024-12-16 19:47:29 -05:00
Adeeb Shihadeh
bedbe6fd94 agnos 11.4 (#34250) 2024-12-16 16:46:03 -08:00
ZwX1616
7352e612a2 dmonitoringmodeld: use cl transform without tinygrad (#34266)
* merge

* why

* self.buf_size

* 0.05 more than with tg due to copy

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-12-16 16:29:06 -08:00
Jason Wen
35278ba63b ui: Allow Qt spinner/text/setup/reset/updater to build on macOS (#34265)
ui: Allow spinner/text/setup/reset/updater to build on macOS
2024-12-16 16:25:46 -08:00
Adeeb Shihadeh
a82116ac46 camerad: fix VisionBuf freeing (#34264)
Co-authored-by: Comma Device <device@comma.ai>
2024-12-16 15:12:17 -08:00
Adeeb Shihadeh
b2930682ff tici: only write eSIM connection once 2024-12-16 14:41:36 -08:00
Harald Schäfer
5018cf75ff North America Model (#34260)
* 0e1c9c12-0472-4a0c-8963-611ad240ec62/400

* rm outputs
2024-12-16 14:17:31 -08:00
Harald Schäfer
a98210aeec modeld: ort helpers (#34258)
* ort helpers

* import from ort helpers

* import that too

* linter

* linter

* linter
2024-12-16 13:10:00 -08:00
commaci-public
11fb0b95d2 [bot] Update Python packages (#34256)
Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>
2024-12-16 10:28:17 -08:00
Edward Wang
ea444ec340 Standardize ENV=* formatting in Dockerfiles (#34253)
fix ENV formatting
2024-12-16 09:47:32 -08:00
Mike Busuttil
cf4fae5464 CTF.md typo (#34257)
typo
2024-12-16 09:37:29 -08:00
commaci-public
833a67b019 [bot] Update Python packages (#34251)
Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>
2024-12-15 21:05:29 -08:00
Adeeb Shihadeh
8558928864 add branch guide to the readme 2024-12-15 13:36:51 -08:00
Adeeb Shihadeh
df2bf83846 op/switch: more robust switching 2024-12-15 09:39:23 -08:00
Adeeb Shihadeh
d735db6113 rm cppcheck (#34248) 2024-12-14 23:17:45 -08:00
Adeeb Shihadeh
b6233838eb macOS: disable brew auto update (#34247) 2024-12-14 23:10:04 -08:00
Adeeb Shihadeh
ba0e7c4719 hardware: add helper for setting IR power (#34245)
* hardware: add helper for setting IR power

* fix
2024-12-14 13:50:25 -08:00
Maxime Desroches
70fa0ab4c1 debug: touch events plot (#34242)
* replay

* remove
2024-12-14 13:09:59 -08:00
Harald Schäfer
f6885dcbec Revert Tinygrad (#34243)
* Revert "dmonitoringmodeld: use cl transform (#34235)"

This reverts commit 684b0b9d4d.

* Revert "load model before calling convert_fp16_to_fp32"

This reverts commit 31606a7d15.

* Revert "bump tinygrad"

This reverts commit 44f58ff758.

* Revert "Tinygrad runner (#34171)"

This reverts commit 7b5a4fbb03.

* Allow init buffer

* typo
2024-12-14 12:15:36 -08:00