Commit Graph

274 Commits

Author SHA1 Message Date
Justin Newberry
6f0affbcd2 CI: disable buildjet on PR's from forks (#29979)
* fix fork CI

* linter

* needed for docker build too

* slightly more clear

* you should have gone for the head
2023-09-20 13:15:23 -07:00
Adeeb Shihadeh
4b7b234b9e CI: login to docker hub on BuildJet (#29961)
* CI: login to docker hub on BuildJet

* shell

* try quotes

* forward

* try this

* can't believe this is broken

* try

* fix process replay
2023-09-18 15:53:16 -07:00
Adeeb Shihadeh
079a51cfdf Revert "CI: login to dockerhub on aarch64 runners (#29915)"
This reverts commit c9ec7bc2a3.
2023-09-17 15:16:07 -07:00
Kacper Rączy
f0b8ecd14f macOS: add prompt about support in setup script (#29939)
* Add prompt about the state of macos support in mac_setup

* Change words

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2023-09-15 19:05:32 -07:00
Justin Newberry
c6f1f36789 CI: reduce unittest timeout (#29937)
reduce timeout
2023-09-15 14:38:51 -07:00
Kacper Rączy
c9ec7bc2a3 CI: login to dockerhub on aarch64 runners (#29915)
* DOCKER_HUB_LOGIN command

* Add docker creds

* checkout runner name

* test again

* Use runner.name to distinguish between buildjet and GH runners
2023-09-14 17:08:49 -07:00
Adeeb Shihadeh
7ef8898624 don't run test twice 2023-09-14 14:08:24 -07:00
Vivek Aithal
e9edf1300a locationd: Unit tests for different scenarios (#29908)
* add different scenarios for locationd

* add locationd scenario tests to unit tests
2023-09-14 13:59:21 -07:00
Vivek Aithal
01011c6c1d liblocationd: No longer used (#29909)
remove liblocationd
2023-09-13 18:59:32 -07:00
Justin Newberry
a87d62ffa6 CI: fix setup retry sleep time (#29877) 2023-09-11 23:08:04 -07:00
Justin Newberry
f9a60072f7 CI: never remove existing labels (#29885) 2023-09-11 23:04:53 -07:00
Adeeb Shihadeh
2c1dfc2f34 use pytest for cpp tests (#29833)
* add pytest-cpp package (#29828)

* add pytest-cpp package

* disable for now

* use in ci

* use pytest cpp

* use my branch

* add missing tests

* that doesn't make sense

* it was merged upstream

* replace TEST_RLOG_URL with a publicly accessible url

* PR Cleanup

* assert instead

* update poetry lock

* add comment about  pytest-cpp

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
Co-authored-by: deanlee <deanlee3@gmail.com>
2023-09-11 14:01:50 -07:00
Justin Newberry
50b954d595 CI: fix pr labeler permissions (#29862)
* fix pr labeler

* fix that
2023-09-09 01:02:49 -07:00
Justin Newberry
aa52cf3e81 CI: auto label PR's (#29842)
* auto label

* on pull request

* checkout

* yaml not yml

* recursive here too

* start with just car stuff and CI

* require all

* no fps

* add translations and fix CI
2023-09-08 17:24:19 -07:00
Kacper Rączy
c417669cbd docker: fix cl tag (#29858)
unset TARGET_ARCHITECTURE before running build
2023-09-08 17:20:03 -07:00
Justin Newberry
d3e5f66fc1 CI: fix aarch64 cache (#29857)
was not passed in
2023-09-08 17:13:18 -07:00
Kacper Rączy
b09ec6bb5b docker: fix multiarch tag push (#29850)
Fix multiarch merge script
2023-09-08 15:03:40 -07:00
Kacper Rączy
1344a93a35 docker: multiarch images (amd64/aarch64) (#29691)
* arch specific tags

* merge multiarch script

* arm64 build and merge jobs

* update tag script

* casual ci test

* change image to 2204

* docker -v test

* arch specific naming

* Remove test_buildjet

* build -arch images only when CURRENT_ARCH_BUILD is set

* support new tags in tag_multiarch

* Toggle CURRENT_ARCH_BUILD on in ci

* Docker common

* Fix condition

* Fix path to docker_common

* Add more stuff to common

* Add build_arm job

* composite job for building

* Run checkout before build composite

* add shell arg

* move timeout to selfdrive_tests

* TARGET ARCHITECTURE var

* Support TARGET_ARCHITECTURE in workflow

* Rewrite to Single build job with matrix

* Remove shebang from docker_common

* Attempt for fix build matrix

* Remove setup arch

* build matrix for docker push

* Use 2vcpu for arm build

* 2vcpu for docker_push too

* temporarly unlock docker_push

* Remove requirement for target arch when pushing

* Unset target architecture in docker_push cl

* fix sha tags

* Rename action to compile-openpilot

* move push_image line to tag_multiarch step

* arch suffix for scons cache

* cache_key_prefix for setup-with-retry

* Re-disable docker_push on non-master

* Add newlines
2023-09-08 14:46:39 -07:00
Adeeb Shihadeh
c96fe132e8 CI: increase model replay timeout 2023-09-08 10:41:54 -07:00
Justin Newberry
b0a71d4553 Pytest: enforce a default timeout (#29793)
* pytest: enforce default timeout

* disable that for now

* 30 second timeout on tests
2023-09-07 17:53:51 -07:00
Justin Newberry
dde225221e CI: run unittests in parallel with xdist (#29756)
* parallel tests

* review suggesions

* add to pyproject

* add a bit more buffer on that

* fix rare athena issue

* remove from pypoetry
2023-09-07 16:29:36 -07:00
Justin Newberry
917f71d446 CI: retry setup on failure (#29785)
* try a setup action

* should be uses

* fix that formatting

* try conclusion

* continue on error

* try without hyphens

* only when failure

* make it optional

* continue on error

* those don't fail anymore

* what about 3 failures

* remove stuff for debugging

* cleanup

* review suggestions

* change that too

* fix pj
2023-09-06 11:11:15 -07:00
Adeeb Shihadeh
7b6afbc162 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
2023-09-05 18:52:40 -07:00
Justin Newberry
ca435d3447 CI: use pytest for unittesting (#29709)
* switch to pytest

* static analysis

* fix the helpers

* static analysis

* remove that

* more parallel test

* cleanup

* static analysis

* durations min and tolerance

* reduce those iterations

* sleep time could be decreased too

* in case it's negative

* try using caching

* remove parallelization for first PR

* PR Cleanup

* reduce diff

* fix that

* bump panda
2023-09-04 11:54:35 -07:00
Adeeb Shihadeh
75081b2e54 don't rerun those here 2023-08-31 18:34:20 -07:00
Adeeb Shihadeh
afd35bf7ec CI: weekly poetry lock (#29738)
* CI: weekly poetry lock

* add that back

* title

* cleanup
2023-08-31 14:27:59 -07:00
Justin Newberry
ab05bff07d CI: fix proc replay download cache permissions (#29636)
chmod that
2023-08-27 09:45:16 -07:00
Adeeb Shihadeh
d35beffbce CI: use buildjet for process replay (#29277)
* squash for pr

* backoff to 4vcpu

* review suggestions

* simplify

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
2023-08-25 18:50:21 -07:00
Justin Newberry
c46965fc27 CI: use local scons cache instead of building a separate image (#29634)
* use local

* added ci_sim

* revert scons

* dont need these anymore

* same for sim

* fix dockerfiles
2023-08-25 16:06:17 -07:00
Dean Lee
2709cd6538 common/tests: combine multiple test binaries into one (#29623) 2023-08-25 10:35:39 -07:00
Justin Newberry
c0b9458a45 CI: move cache dir to workspace directory (#29622)
* comma dir

* mkdir

* fix issues

* test

* fix that

* fix

* do the other ones too

* missed that

* fix

* fix pr comments by moving into two different directories

* fix location

* fix that

* only change current

* fix
2023-08-25 01:41:04 -07:00
Adeeb Shihadeh
36cc41f619 CI: remove redundant apt installs (#29617)
* CI: remove redundant apt installs

* split build and test

* faster build
2023-08-24 17:32:15 -07:00
Adeeb Shihadeh
e863828051 CI: workaround syntax error in azure lib (#29618) 2023-08-24 17:18:11 -07:00
Justin Newberry
90cb417caa CI: use GITHUB_ENV for env variables (#29616)
export doesn't work across steps
2023-08-24 17:03:25 -07:00
Justin Newberry
ede9f2cda5 CI: fix docker push (#29614)
* fix those

* fix docker push too

* dont need that twice
2023-08-24 16:30:33 -07:00
Justin Newberry
3d9f238bf9 CI: use scripts for docker build (#29599)
* use scripts

* remove common

* do login in the workflow

* separate step

* cleanup

* one script is cleaner

* keep it consistent

* review suggestions

* move in ci too

* PR cleanup
2023-08-24 16:09:08 -07:00
Adeeb Shihadeh
4917e22615 Minimal build (#29600)
* add build flag for minimum build

* add minimal build

* update

* fix pj

* bump opendbc

* update subaru
2023-08-24 14:58:30 -07:00
Justin Newberry
da7504e1e7 CI: Normalize file permissions (#29584)
* normalize file permissions

* comment

* add comment

* remove that

* shell

* should be on the top
2023-08-23 23:50:51 -07:00
Justin Newberry
2fdde9b77d CI: switch to --cache-to type=inline for buildkit cache (#29575)
this is cleaner
2023-08-23 17:41:33 -07:00
Justin Newberry
2e4f8e6608 CI: Docs rebuild base image (#29566)
* docs rebuild base image

* fix order
2023-08-23 15:33:23 -07:00
Kacper Rączy
659795b1db devcontainer: handle hosts without X11 (#29537)
* Setup xauthority link on host in initializeCommand

* Create a dummy xauthority if it does no exist on host machine

* Remove xauthority line from ci test
2023-08-23 10:26:38 -07:00
Kacper Rączy
df9564e172 devcontainer: CI fix (#29534)
Export xauthority before running devcontainer
2023-08-22 14:14:14 -07:00
Robbe Derks
d71a7190f1 Devcontainer (#29259)
* mvp devcontainer. builds, runs and UI works

* fix indentation

* enable color and fix openpilot_env path

* separate dockerfile for devcontainer

* use prebuilt image

* Fix openpilot_env setup

* Add dev container section in readme

* Fix typo

* Fix typo again

* Move dev dockerfile to .devcontainer dir

* Update README

* Add note about x11 forwarding on macos

* Update dev container json

* Fix typo

* Override DISPLAY on macOS

* Mention XAUTHORITY for macOS setup in README

* fix typo

* Add devcontainer CI test

* Rename to devcontainer

* Fix git issues

* ripgrep

* Link instead of vscode manual

* Replace raw path with containerWorkspaceFolder

* Fix typo

* Remove dev tools workflow

* Link away X11 forwarding setup isntructions for mac

* Remove fork consrtaint, add test run of scons

* Add submodules as safe.directories in postStartCommand

* Remove openpilot_env references

* Add scons cache volume

---------

Co-authored-by: Robbe Derks <robbe@localhost.localdomain>
Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
2023-08-22 11:47:09 -07:00
Adeeb Shihadeh
91dc8457ae Revert Qt MOC hacks (#29528)
* Revert "CI: prevent moc from running in cache population step (#29401)"

This reverts commit d1aedccfbe.

* Revert "SCons: respect cache read only (#29437)"

This reverts commit 59abb469ea.

* don't change moc path
2023-08-22 11:16:58 -07:00
Kacper Rączy
d1aedccfbe CI: prevent moc from running in cache population step (#29401)
* Do not remove moc_files from cache

* Add cache cleanup script

* Use cache cleanup script

* Dry-run in cache population command

* Add newline

* Temporarly disable master only constraint for cache cleanup step

* Add test lines

* one more

* Fix shebang

* Remove debug lines

* Bring back master branch constraints
2023-08-22 10:47:56 -07:00
Justin Newberry
3492e80d19 CI: use xdist for car model tests (#29514)
* bump panda

* try multithreaded for car models

* need pytest-cov

* upgraded to poetry 1.5.1 and regenerated
2023-08-21 20:51:27 -07:00
Justin Newberry
ec5e3bef95 CI: simulator disable pull to save local space (#29517)
we don't need to pull since the image is different most of the time
2023-08-21 20:01:52 -07:00
Justin Newberry
9188c56069 CI: remove old docs base image push (#29512)
remove that push
2023-08-21 15:32:41 -07:00
Justin Newberry
197460e061 CI: use scons cache mount for building simulator and docs (#29436)
* build docker image with scons cache

* use setup

* scons cache

* added the dockerfile

* add the shell

* docs use cache

* dont pull lfs for docs

* simulator too

* dont pull the cache image

* fix inputs

* inputs

* booleans aren't supported

* bool not supported

* bool isnt supported

* fix

* readonly cache

* fix sim cache too

* respect readonly

* gitignore and move to correct folder

* also copy old

* move to tools/ci

* move to selfdrive/test

* just make the cache writable, it's discarded anyway

* minimize diff in this pr
2023-08-21 15:11:31 -07:00
Kacper Rączy
0f7be5b1e6 Setup cleanup vol. 2 (#29456)
* code from setup cleanup #29419

update ci

update ci

export variables for mac os

check env

what is in the profile script?

install pyenv the same everywhere

temporarily disable brew cache

temporarily disable brew cache

try fixing pyenv virtualenv-init

try this

inject shims path earlier

try alternate install method

switch back to brew install

try         eval "$(pyenv init --path)"

add eval "\$(pyenv init --path)"

* Add local .env file to the cache list

* Change .env paths

* debug line

* order?

* remove .env

* Add .venv to caches

---------

Co-authored-by: Greg Hogan <gregjhogan@gmail.com>
2023-08-21 13:30:48 -07:00