Commit Graph

265 Commits

Author SHA1 Message Date
Justin Newberry
490613e5e4 CI: fix setup retry sleep time (#29877)
old-commit-hash: a87d62ffa6
2023-09-11 23:08:04 -07:00
Justin Newberry
58ea2bac72 CI: never remove existing labels (#29885)
old-commit-hash: f9a60072f7
2023-09-11 23:04:53 -07:00
Adeeb Shihadeh
1f8459d285 use pytest for cpp tests (#29833)
* add pytest-cpp package (#29828)

* add pytest-cpp package

* disable for now

* use in ci

* use pytest cpp

* use my branch

* add missing tests

* that doesn't make sense

* it was merged upstream

* replace TEST_RLOG_URL with a publicly accessible url

* PR Cleanup

* assert instead

* update poetry lock

* add comment about  pytest-cpp

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
Co-authored-by: deanlee <deanlee3@gmail.com>
old-commit-hash: 2c1dfc2f34
2023-09-11 14:01:50 -07:00
Justin Newberry
d1fc58bf81 CI: fix pr labeler permissions (#29862)
* fix pr labeler

* fix that
old-commit-hash: 50b954d595
2023-09-09 01:02:49 -07:00
Justin Newberry
693183a946 CI: auto label PR's (#29842)
* auto label

* on pull request

* checkout

* yaml not yml

* recursive here too

* start with just car stuff and CI

* require all

* no fps

* add translations and fix CI
old-commit-hash: aa52cf3e81
2023-09-08 17:24:19 -07:00
Kacper Rączy
fc4972539f docker: fix cl tag (#29858)
unset TARGET_ARCHITECTURE before running build
old-commit-hash: c417669cbd
2023-09-08 17:20:03 -07:00
Justin Newberry
5ac2ef0e3f CI: fix aarch64 cache (#29857)
was not passed in
old-commit-hash: d3e5f66fc1
2023-09-08 17:13:18 -07:00
Kacper Rączy
73c25cc743 docker: fix multiarch tag push (#29850)
Fix multiarch merge script
old-commit-hash: b09ec6bb5b
2023-09-08 15:03:40 -07:00
Kacper Rączy
f8c6ebd3ad docker: multiarch images (amd64/aarch64) (#29691)
* arch specific tags

* merge multiarch script

* arm64 build and merge jobs

* update tag script

* casual ci test

* change image to 2204

* docker -v test

* arch specific naming

* Remove test_buildjet

* build -arch images only when CURRENT_ARCH_BUILD is set

* support new tags in tag_multiarch

* Toggle CURRENT_ARCH_BUILD on in ci

* Docker common

* Fix condition

* Fix path to docker_common

* Add more stuff to common

* Add build_arm job

* composite job for building

* Run checkout before build composite

* add shell arg

* move timeout to selfdrive_tests

* TARGET ARCHITECTURE var

* Support TARGET_ARCHITECTURE in workflow

* Rewrite to Single build job with matrix

* Remove shebang from docker_common

* Attempt for fix build matrix

* Remove setup arch

* build matrix for docker push

* Use 2vcpu for arm build

* 2vcpu for docker_push too

* temporarly unlock docker_push

* Remove requirement for target arch when pushing

* Unset target architecture in docker_push cl

* fix sha tags

* Rename action to compile-openpilot

* move push_image line to tag_multiarch step

* arch suffix for scons cache

* cache_key_prefix for setup-with-retry

* Re-disable docker_push on non-master

* Add newlines
old-commit-hash: 1344a93a35
2023-09-08 14:46:39 -07:00
Adeeb Shihadeh
13c3bf2cbd CI: increase model replay timeout
old-commit-hash: c96fe132e8
2023-09-08 10:41:54 -07:00
Justin Newberry
5b1e60f5da Pytest: enforce a default timeout (#29793)
* pytest: enforce default timeout

* disable that for now

* 30 second timeout on tests
old-commit-hash: b0a71d4553
2023-09-07 17:53:51 -07:00
Justin Newberry
37e4b2cba9 CI: run unittests in parallel with xdist (#29756)
* parallel tests

* review suggesions

* add to pyproject

* add a bit more buffer on that

* fix rare athena issue

* remove from pypoetry
old-commit-hash: dde225221e
2023-09-07 16:29:36 -07:00
Justin Newberry
e468072f1f CI: retry setup on failure (#29785)
* try a setup action

* should be uses

* fix that formatting

* try conclusion

* continue on error

* try without hyphens

* only when failure

* make it optional

* continue on error

* those don't fail anymore

* what about 3 failures

* remove stuff for debugging

* cleanup

* review suggestions

* change that too

* fix pj
old-commit-hash: 917f71d446
2023-09-06 11:11:15 -07:00
Adeeb Shihadeh
53e0f5eb49 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: 7b6afbc162
2023-09-05 18:52:40 -07:00
Justin Newberry
839e72de70 CI: use pytest for unittesting (#29709)
* switch to pytest

* static analysis

* fix the helpers

* static analysis

* remove that

* more parallel test

* cleanup

* static analysis

* durations min and tolerance

* reduce those iterations

* sleep time could be decreased too

* in case it's negative

* try using caching

* remove parallelization for first PR

* PR Cleanup

* reduce diff

* fix that

* bump panda
old-commit-hash: ca435d3447
2023-09-04 11:54:35 -07:00
Adeeb Shihadeh
0107dfb391 don't rerun those here
old-commit-hash: 75081b2e54
2023-08-31 18:34:20 -07:00
Adeeb Shihadeh
fbfc0e460d CI: weekly poetry lock (#29738)
* CI: weekly poetry lock

* add that back

* title

* cleanup
old-commit-hash: afd35bf7ec
2023-08-31 14:27:59 -07:00
Justin Newberry
47eb055283 CI: fix proc replay download cache permissions (#29636)
chmod that
old-commit-hash: ab05bff07d
2023-08-27 09:45:16 -07:00
Adeeb Shihadeh
68dd600872 CI: use buildjet for process replay (#29277)
* squash for pr

* backoff to 4vcpu

* review suggestions

* simplify

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: d35beffbce
2023-08-25 18:50:21 -07:00
Justin Newberry
643a13f27e CI: use local scons cache instead of building a separate image (#29634)
* use local

* added ci_sim

* revert scons

* dont need these anymore

* same for sim

* fix dockerfiles
old-commit-hash: c46965fc27
2023-08-25 16:06:17 -07:00
Dean Lee
00f900e8c4 common/tests: combine multiple test binaries into one (#29623)
old-commit-hash: 2709cd6538
2023-08-25 10:35:39 -07:00
Justin Newberry
e128aedc63 CI: move cache dir to workspace directory (#29622)
* comma dir

* mkdir

* fix issues

* test

* fix that

* fix

* do the other ones too

* missed that

* fix

* fix pr comments by moving into two different directories

* fix location

* fix that

* only change current

* fix
old-commit-hash: c0b9458a45
2023-08-25 01:41:04 -07:00
Adeeb Shihadeh
06aea7d935 CI: remove redundant apt installs (#29617)
* CI: remove redundant apt installs

* split build and test

* faster build
old-commit-hash: 36cc41f619
2023-08-24 17:32:15 -07:00
Adeeb Shihadeh
8f9dcca26a CI: workaround syntax error in azure lib (#29618)
old-commit-hash: e863828051
2023-08-24 17:18:11 -07:00
Justin Newberry
4063a92683 CI: use GITHUB_ENV for env variables (#29616)
export doesn't work across steps
old-commit-hash: 90cb417caa
2023-08-24 17:03:25 -07:00
Justin Newberry
630b440536 CI: fix docker push (#29614)
* fix those

* fix docker push too

* dont need that twice
old-commit-hash: ede9f2cda5
2023-08-24 16:30:33 -07:00
Justin Newberry
6847f819d1 CI: use scripts for docker build (#29599)
* use scripts

* remove common

* do login in the workflow

* separate step

* cleanup

* one script is cleaner

* keep it consistent

* review suggestions

* move in ci too

* PR cleanup
old-commit-hash: 3d9f238bf9
2023-08-24 16:09:08 -07:00
Adeeb Shihadeh
61cf541322 Minimal build (#29600)
* add build flag for minimum build

* add minimal build

* update

* fix pj

* bump opendbc

* update subaru
old-commit-hash: 4917e22615
2023-08-24 14:58:30 -07:00
Justin Newberry
de7257c22c CI: Normalize file permissions (#29584)
* normalize file permissions

* comment

* add comment

* remove that

* shell

* should be on the top
old-commit-hash: da7504e1e7
2023-08-23 23:50:51 -07:00
Justin Newberry
7a532e02f3 CI: switch to --cache-to type=inline for buildkit cache (#29575)
this is cleaner
old-commit-hash: 2fdde9b77d
2023-08-23 17:41:33 -07:00
Justin Newberry
e54655626d CI: Docs rebuild base image (#29566)
* docs rebuild base image

* fix order
old-commit-hash: 2e4f8e6608
2023-08-23 15:33:23 -07:00
Kacper Rączy
3681c1d1b2 devcontainer: handle hosts without X11 (#29537)
* Setup xauthority link on host in initializeCommand

* Create a dummy xauthority if it does no exist on host machine

* Remove xauthority line from ci test
old-commit-hash: 659795b1db
2023-08-23 10:26:38 -07:00
Kacper Rączy
e250480db6 devcontainer: CI fix (#29534)
Export xauthority before running devcontainer
old-commit-hash: df9564e172
2023-08-22 14:14:14 -07:00
Robbe Derks
a04cbced36 Devcontainer (#29259)
* mvp devcontainer. builds, runs and UI works

* fix indentation

* enable color and fix openpilot_env path

* separate dockerfile for devcontainer

* use prebuilt image

* Fix openpilot_env setup

* Add dev container section in readme

* Fix typo

* Fix typo again

* Move dev dockerfile to .devcontainer dir

* Update README

* Add note about x11 forwarding on macos

* Update dev container json

* Fix typo

* Override DISPLAY on macOS

* Mention XAUTHORITY for macOS setup in README

* fix typo

* Add devcontainer CI test

* Rename to devcontainer

* Fix git issues

* ripgrep

* Link instead of vscode manual

* Replace raw path with containerWorkspaceFolder

* Fix typo

* Remove dev tools workflow

* Link away X11 forwarding setup isntructions for mac

* Remove fork consrtaint, add test run of scons

* Add submodules as safe.directories in postStartCommand

* Remove openpilot_env references

* Add scons cache volume

---------

Co-authored-by: Robbe Derks <robbe@localhost.localdomain>
Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
old-commit-hash: d71a7190f1
2023-08-22 11:47:09 -07:00
Adeeb Shihadeh
b38403f2ca Revert Qt MOC hacks (#29528)
* Revert "CI: prevent moc from running in cache population step (#29401)"

This reverts commit 34634b95aa15dc608315016c40857f3e16ca4ff9.

* Revert "SCons: respect cache read only (#29437)"

This reverts commit 77cb4da6381fdf94ea98c55d6fcc30db4d24dcf6.

* don't change moc path
old-commit-hash: 91dc8457ae
2023-08-22 11:16:58 -07:00
Kacper Rączy
0c17a4f4d0 CI: prevent moc from running in cache population step (#29401)
* Do not remove moc_files from cache

* Add cache cleanup script

* Use cache cleanup script

* Dry-run in cache population command

* Add newline

* Temporarly disable master only constraint for cache cleanup step

* Add test lines

* one more

* Fix shebang

* Remove debug lines

* Bring back master branch constraints
old-commit-hash: d1aedccfbe
2023-08-22 10:47:56 -07:00
Justin Newberry
3e6ce7d6cd CI: use xdist for car model tests (#29514)
* bump panda

* try multithreaded for car models

* need pytest-cov

* upgraded to poetry 1.5.1 and regenerated
old-commit-hash: 3492e80d19
2023-08-21 20:51:27 -07:00
Justin Newberry
e022ffe528 CI: simulator disable pull to save local space (#29517)
we don't need to pull since the image is different most of the time
old-commit-hash: ec5e3bef95
2023-08-21 20:01:52 -07:00
Justin Newberry
fe844afac7 CI: remove old docs base image push (#29512)
remove that push
old-commit-hash: 9188c56069
2023-08-21 15:32:41 -07:00
Justin Newberry
7205b92bb6 CI: use scons cache mount for building simulator and docs (#29436)
* build docker image with scons cache

* use setup

* scons cache

* added the dockerfile

* add the shell

* docs use cache

* dont pull lfs for docs

* simulator too

* dont pull the cache image

* fix inputs

* inputs

* booleans aren't supported

* bool not supported

* bool isnt supported

* fix

* readonly cache

* fix sim cache too

* respect readonly

* gitignore and move to correct folder

* also copy old

* move to tools/ci

* move to selfdrive/test

* just make the cache writable, it's discarded anyway

* minimize diff in this pr
old-commit-hash: 197460e061
2023-08-21 15:11:31 -07:00
Kacper Rączy
1d47429f8e Setup cleanup vol. 2 (#29456)
* code from setup cleanup #29419

update ci

update ci

export variables for mac os

check env

what is in the profile script?

install pyenv the same everywhere

temporarily disable brew cache

temporarily disable brew cache

try fixing pyenv virtualenv-init

try this

inject shims path earlier

try alternate install method

switch back to brew install

try         eval "$(pyenv init --path)"

add eval "\$(pyenv init --path)"

* Add local .env file to the cache list

* Change .env paths

* debug line

* order?

* remove .env

* Add .venv to caches

---------

Co-authored-by: Greg Hogan <gregjhogan@gmail.com>
old-commit-hash: 0f7be5b1e6
2023-08-21 13:30:48 -07:00
Dean Lee
97631ec362 common: C++ RateKeeper (#29374)
* c++ RateKeeper

* add to files_common

* use util::random_int

* improve monotor_time

* remove ~ratekeeper
old-commit-hash: 3eef63af9b
2023-08-20 21:27:02 -07:00
Adeeb Shihadeh
bd27be2b78 move mypy config into pyproject.toml (#29501)
* move mypy config into pyproject.toml

* rm reference
old-commit-hash: ff068dd722
2023-08-20 21:08:04 -07:00
Harald Schäfer
3022f06e1e update process replay timeout (#29478)
* Always dict differ

* Update selfdrive_tests.yaml

* bump up more

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: bc4b75822a
2023-08-20 17:24:33 -07:00
Justin Newberry
1988b4805e Revert "setup cleanup (#29419)" (#29451)
This reverts commit bc9c60a3f955bb5b96dc41e6072cab47191f8896.
old-commit-hash: 1944945d9d
2023-08-17 13:23:06 -07:00
Greg Hogan
cce2c732c6 setup cleanup (#29419)
* setup cleanup

* update ci

* update ci

* export variables for mac os

* check env

* what is in the profile script?

* install pyenv the same everywhere

* temporarily disable brew cache

* temporarily disable brew cache

* try fixing pyenv virtualenv-init

* try this

* inject shims path earlier

* try alternate install method

* switch back to brew install

* try         eval "$(pyenv init --path)"

* add eval "\$(pyenv init --path)"
old-commit-hash: ad32816fd6
2023-08-17 08:51:16 -07:00
Justin Newberry
edbd5d330e CI: use setup script for docs and simulator (#29434)
* use setup

* dont pull lfs for docs

* fix inputs

* bool not supported

* bool isnt supported
old-commit-hash: ad00595ba5
2023-08-16 17:59:42 -07:00
Kacper Rączy
4096be66e4 macOS: fix GH actions cache (#29399)
Add lz4 and xz to dep cache exceptions
old-commit-hash: ae649ec68d
2023-08-14 16:13:35 -07:00
Justin Newberry
1424636288 CI: docs speedup by caching first stage (#29397)
* docs speedup

* fix tags on docs build

* build docs without build-push-action
old-commit-hash: f8cbdcfa40
2023-08-14 14:36:17 -07:00
Adeeb Shihadeh
e76af64065 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
old-commit-hash: b11b14aff8
2023-08-12 17:15:16 -07:00