Commit Graph

254 Commits

Author SHA1 Message Date
Justin Newberry
dde225221e CI: run unittests in parallel with xdist (#29756)
* parallel tests

* review suggesions

* add to pyproject

* add a bit more buffer on that

* fix rare athena issue

* remove from pypoetry
2023-09-07 16:29:36 -07:00
Justin Newberry
917f71d446 CI: retry setup on failure (#29785)
* try a setup action

* should be uses

* fix that formatting

* try conclusion

* continue on error

* try without hyphens

* only when failure

* make it optional

* continue on error

* those don't fail anymore

* what about 3 failures

* remove stuff for debugging

* cleanup

* review suggestions

* change that too

* fix pj
2023-09-06 11:11:15 -07:00
Adeeb Shihadeh
7b6afbc162 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
2023-09-05 18:52:40 -07:00
Justin Newberry
ca435d3447 CI: use pytest for unittesting (#29709)
* switch to pytest

* static analysis

* fix the helpers

* static analysis

* remove that

* more parallel test

* cleanup

* static analysis

* durations min and tolerance

* reduce those iterations

* sleep time could be decreased too

* in case it's negative

* try using caching

* remove parallelization for first PR

* PR Cleanup

* reduce diff

* fix that

* bump panda
2023-09-04 11:54:35 -07:00
Adeeb Shihadeh
75081b2e54 don't rerun those here 2023-08-31 18:34:20 -07:00
Adeeb Shihadeh
afd35bf7ec CI: weekly poetry lock (#29738)
* CI: weekly poetry lock

* add that back

* title

* cleanup
2023-08-31 14:27:59 -07:00
Justin Newberry
ab05bff07d CI: fix proc replay download cache permissions (#29636)
chmod that
2023-08-27 09:45:16 -07:00
Adeeb Shihadeh
d35beffbce CI: use buildjet for process replay (#29277)
* squash for pr

* backoff to 4vcpu

* review suggestions

* simplify

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
2023-08-25 18:50:21 -07:00
Justin Newberry
c46965fc27 CI: use local scons cache instead of building a separate image (#29634)
* use local

* added ci_sim

* revert scons

* dont need these anymore

* same for sim

* fix dockerfiles
2023-08-25 16:06:17 -07:00
Dean Lee
2709cd6538 common/tests: combine multiple test binaries into one (#29623) 2023-08-25 10:35:39 -07:00
Justin Newberry
c0b9458a45 CI: move cache dir to workspace directory (#29622)
* comma dir

* mkdir

* fix issues

* test

* fix that

* fix

* do the other ones too

* missed that

* fix

* fix pr comments by moving into two different directories

* fix location

* fix that

* only change current

* fix
2023-08-25 01:41:04 -07:00
Adeeb Shihadeh
36cc41f619 CI: remove redundant apt installs (#29617)
* CI: remove redundant apt installs

* split build and test

* faster build
2023-08-24 17:32:15 -07:00
Adeeb Shihadeh
e863828051 CI: workaround syntax error in azure lib (#29618) 2023-08-24 17:18:11 -07:00
Justin Newberry
90cb417caa CI: use GITHUB_ENV for env variables (#29616)
export doesn't work across steps
2023-08-24 17:03:25 -07:00
Justin Newberry
ede9f2cda5 CI: fix docker push (#29614)
* fix those

* fix docker push too

* dont need that twice
2023-08-24 16:30:33 -07:00
Justin Newberry
3d9f238bf9 CI: use scripts for docker build (#29599)
* use scripts

* remove common

* do login in the workflow

* separate step

* cleanup

* one script is cleaner

* keep it consistent

* review suggestions

* move in ci too

* PR cleanup
2023-08-24 16:09:08 -07:00
Adeeb Shihadeh
4917e22615 Minimal build (#29600)
* add build flag for minimum build

* add minimal build

* update

* fix pj

* bump opendbc

* update subaru
2023-08-24 14:58:30 -07:00
Justin Newberry
da7504e1e7 CI: Normalize file permissions (#29584)
* normalize file permissions

* comment

* add comment

* remove that

* shell

* should be on the top
2023-08-23 23:50:51 -07:00
Justin Newberry
2fdde9b77d CI: switch to --cache-to type=inline for buildkit cache (#29575)
this is cleaner
2023-08-23 17:41:33 -07:00
Justin Newberry
2e4f8e6608 CI: Docs rebuild base image (#29566)
* docs rebuild base image

* fix order
2023-08-23 15:33:23 -07:00
Kacper Rączy
659795b1db devcontainer: handle hosts without X11 (#29537)
* Setup xauthority link on host in initializeCommand

* Create a dummy xauthority if it does no exist on host machine

* Remove xauthority line from ci test
2023-08-23 10:26:38 -07:00
Kacper Rączy
df9564e172 devcontainer: CI fix (#29534)
Export xauthority before running devcontainer
2023-08-22 14:14:14 -07:00
Robbe Derks
d71a7190f1 Devcontainer (#29259)
* mvp devcontainer. builds, runs and UI works

* fix indentation

* enable color and fix openpilot_env path

* separate dockerfile for devcontainer

* use prebuilt image

* Fix openpilot_env setup

* Add dev container section in readme

* Fix typo

* Fix typo again

* Move dev dockerfile to .devcontainer dir

* Update README

* Add note about x11 forwarding on macos

* Update dev container json

* Fix typo

* Override DISPLAY on macOS

* Mention XAUTHORITY for macOS setup in README

* fix typo

* Add devcontainer CI test

* Rename to devcontainer

* Fix git issues

* ripgrep

* Link instead of vscode manual

* Replace raw path with containerWorkspaceFolder

* Fix typo

* Remove dev tools workflow

* Link away X11 forwarding setup isntructions for mac

* Remove fork consrtaint, add test run of scons

* Add submodules as safe.directories in postStartCommand

* Remove openpilot_env references

* Add scons cache volume

---------

Co-authored-by: Robbe Derks <robbe@localhost.localdomain>
Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
2023-08-22 11:47:09 -07:00
Adeeb Shihadeh
91dc8457ae Revert Qt MOC hacks (#29528)
* Revert "CI: prevent moc from running in cache population step (#29401)"

This reverts commit d1aedccfbe.

* Revert "SCons: respect cache read only (#29437)"

This reverts commit 59abb469ea.

* don't change moc path
2023-08-22 11:16:58 -07:00
Kacper Rączy
d1aedccfbe CI: prevent moc from running in cache population step (#29401)
* Do not remove moc_files from cache

* Add cache cleanup script

* Use cache cleanup script

* Dry-run in cache population command

* Add newline

* Temporarly disable master only constraint for cache cleanup step

* Add test lines

* one more

* Fix shebang

* Remove debug lines

* Bring back master branch constraints
2023-08-22 10:47:56 -07:00
Justin Newberry
3492e80d19 CI: use xdist for car model tests (#29514)
* bump panda

* try multithreaded for car models

* need pytest-cov

* upgraded to poetry 1.5.1 and regenerated
2023-08-21 20:51:27 -07:00
Justin Newberry
ec5e3bef95 CI: simulator disable pull to save local space (#29517)
we don't need to pull since the image is different most of the time
2023-08-21 20:01:52 -07:00
Justin Newberry
9188c56069 CI: remove old docs base image push (#29512)
remove that push
2023-08-21 15:32:41 -07:00
Justin Newberry
197460e061 CI: use scons cache mount for building simulator and docs (#29436)
* build docker image with scons cache

* use setup

* scons cache

* added the dockerfile

* add the shell

* docs use cache

* dont pull lfs for docs

* simulator too

* dont pull the cache image

* fix inputs

* inputs

* booleans aren't supported

* bool not supported

* bool isnt supported

* fix

* readonly cache

* fix sim cache too

* respect readonly

* gitignore and move to correct folder

* also copy old

* move to tools/ci

* move to selfdrive/test

* just make the cache writable, it's discarded anyway

* minimize diff in this pr
2023-08-21 15:11:31 -07:00
Kacper Rączy
0f7be5b1e6 Setup cleanup vol. 2 (#29456)
* code from setup cleanup #29419

update ci

update ci

export variables for mac os

check env

what is in the profile script?

install pyenv the same everywhere

temporarily disable brew cache

temporarily disable brew cache

try fixing pyenv virtualenv-init

try this

inject shims path earlier

try alternate install method

switch back to brew install

try         eval "$(pyenv init --path)"

add eval "\$(pyenv init --path)"

* Add local .env file to the cache list

* Change .env paths

* debug line

* order?

* remove .env

* Add .venv to caches

---------

Co-authored-by: Greg Hogan <gregjhogan@gmail.com>
2023-08-21 13:30:48 -07:00
Dean Lee
3eef63af9b common: C++ RateKeeper (#29374)
* c++ RateKeeper

* add to files_common

* use util::random_int

* improve monotor_time

* remove ~ratekeeper
2023-08-20 21:27:02 -07:00
Adeeb Shihadeh
ff068dd722 move mypy config into pyproject.toml (#29501)
* move mypy config into pyproject.toml

* rm reference
2023-08-20 21:08:04 -07:00
Harald Schäfer
bc4b75822a update process replay timeout (#29478)
* Always dict differ

* Update selfdrive_tests.yaml

* bump up more

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2023-08-20 17:24:33 -07:00
Justin Newberry
1944945d9d Revert "setup cleanup (#29419)" (#29451)
This reverts commit ad32816fd6.
2023-08-17 13:23:06 -07:00
Greg Hogan
ad32816fd6 setup cleanup (#29419)
* setup cleanup

* update ci

* update ci

* export variables for mac os

* check env

* what is in the profile script?

* install pyenv the same everywhere

* temporarily disable brew cache

* temporarily disable brew cache

* try fixing pyenv virtualenv-init

* try this

* inject shims path earlier

* try alternate install method

* switch back to brew install

* try         eval "$(pyenv init --path)"

* add eval "\$(pyenv init --path)"
2023-08-17 08:51:16 -07:00
Justin Newberry
ad00595ba5 CI: use setup script for docs and simulator (#29434)
* use setup

* dont pull lfs for docs

* fix inputs

* bool not supported

* bool isnt supported
2023-08-16 17:59:42 -07:00
Kacper Rączy
ae649ec68d macOS: fix GH actions cache (#29399)
Add lz4 and xz to dep cache exceptions
2023-08-14 16:13:35 -07:00
Justin Newberry
f8cbdcfa40 CI: docs speedup by caching first stage (#29397)
* docs speedup

* fix tags on docs build

* build docs without build-push-action
2023-08-14 14:36:17 -07:00
Adeeb Shihadeh
b11b14aff8 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
2023-08-12 17:15:16 -07:00
Adeeb Shihadeh
0ced56b2ea remove xx dependencies (#29263)
* rm xx deps

* mv that

* fix

* fix a path

* relock

* relock

* update

* update sentry

* update

* revert sentry

* Update lock

* Dummy pip package

* Not needed with poetry

* dont need dummy

---------

Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
2023-08-11 20:05:44 -07:00
Justin Newberry
2482b58968 CI: move other caches out of /tmp (#29348)
* also move other caches

* macos can stay
2023-08-11 17:41:23 -07:00
Justin Newberry
ace4d78e31 CI: move cache to home directory (#29347)
move cache to home dir
2023-08-11 15:45:38 -07:00
Justin Newberry
a0f399677e CI: use default caching action (#29339) 2023-08-11 14:03:10 -07:00
Adeeb Shihadeh
cd4bb412f5 CI: bump up model test timeout 2023-08-10 17:29:56 -07:00
Adeeb Shihadeh
e53f9597f3 remove rest of pylint (#29302)
* remove rest of pylint

* fix release build
2023-08-09 19:03:36 -07:00
Adeeb Shihadeh
ae8252c14a CI: move longitudinal tests into unit tests job (#29245)
* move long into unit test job

* try without these

* add that back
2023-08-07 06:48:06 -07:00
Adeeb Shihadeh
75ab960557 CI: add timeouts to selfdrive tests (#29239)
* CI: add timeouts to selfdrive tests

* ugh, so slow

* will have to come back to this one
2023-08-04 18:14:14 -07:00
Adeeb Shihadeh
f714e7cb24 CI: update cache action (#29241)
* update that

* update one more
2023-08-04 17:59:55 -07:00
Adeeb Shihadeh
0df2cd4fa8 CI: only run on master + PRs (#29242) 2023-08-04 17:56:30 -07:00
Adeeb Shihadeh
8633e5f8ab CI: add timeouts to tools jobs (#29238) 2023-08-04 15:35:21 -07:00