Commit Graph

1344 Commits

Author SHA1 Message Date
Justin Newberry
405d09714c Tests: expand temporary directory helpers for more directories (#29735)
* create helpers for testing

* document it

* fix other tests
2023-08-31 14:00:31 -07:00
Mitchell Goff
1eb60516cb Move update_calibration to modeld.py (#29688)
* Move update_calibration to modeld.py

* Updated ref commit
2023-08-30 14:12:52 -07:00
Justin Newberry
f31b4ad25e Tests: remove sleep and prints from maneuver tests (#29711)
remove sleep and prints
2023-08-30 00:02:55 -07:00
Justin Newberry
5dc7028f91 Tests: logmessaged reduce global state (#29680)
* reduce global state

* use a decorator here too

* use that one

* use base imports

* ipchandler in setup

* move to common dir

* move to helpers

* wip

* fix the decorator
2023-08-29 13:36:26 -07:00
Kacper Rączy
bb14ba4e5e process_replay: clean env vars after run (#29694)
Clean environ vars at the end of replay
2023-08-28 19:44:40 -07:00
Harald Schäfer
55b391509d Revert: Locationd: enable laikad (#29687)
* Revert: Locationd: enable laikad

* update ref
2023-08-28 16:27:11 -07:00
Adeeb Shihadeh
d35beffbce CI: use buildjet for process replay (#29277)
* squash for pr

* backoff to 4vcpu

* review suggestions

* simplify

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
2023-08-25 18:50:21 -07:00
Justin Newberry
c46965fc27 CI: use local scons cache instead of building a separate image (#29634)
* use local

* added ci_sim

* revert scons

* dont need these anymore

* same for sim

* fix dockerfiles
2023-08-25 16:06:17 -07:00
Justin Newberry
584239ceb5 CI: add comment about building locally (#29635)
add comment about build
2023-08-25 15:49:07 -07:00
Adeeb Shihadeh
4d6bb32df2 process replay: parallel by default (#29633)
* process replay: parallel by default

* more is helpful
2023-08-25 14:41:43 -07:00
Mitchell Goff
72a3c987c0 Rewrite modeld in python (#29230)
* Added modeld.py (WIP)

* No more VisionIpcBufExtra

* Started work on cython bindings for runmodel

* Got ONNXModel cython bindings mostly working, added ModelFrame bindings

* Got modeld main loop running without model eval

* Move everything into ModelState

* Doesn't crash!

* Moved ModelState into modeld.py

* Added driving_pyx

* Added cython bindings for message generation

* Moved CLContext definition to visionipc.pxd

* *facepalm*

* Move cl_pyx into commonmodel_pyx

* Split out ONNXModel into a subclass of RunModel

* Added snpemodel/thneedmodel bindings

* Removed modeld.cc

* Fixed scons for macOS

* Fixed sconscript

* Added flag for thneedmodel

* paths are now relative to openpilot root dir

* Set cl kernel paths in SConscript

* Set LD_PRELOAD=libthneed.so to fix ioctl interception

* Run from root dir

* A few more fixes

* A few more minor fixes

* Use C update_calibration for now to exactly match refs

* Add nav_instructions input

* Link driving_pyx.pyx with transformations

* Checked python FirstOrderFilter against C++ FirstOrderFilter

* Set process name to fix test_onroad

* Revert changes to onnxmodel.cc

* Fixed bad onnx_runner.py path in onnxmodel.cc

* Import all constants from driving.h

* logging -> cloudlog

* pylint import-error suppressions no longer needed?

* Loop in SConscript

* Added parens

* Bump modeld cpu usage in test_onroad

* Get rid of use_nav

* use config_realtime_process

* error message from ioctl sniffer was messing up pyenv

* cast distance_idx to int

* Removed cloudlog.infos in model.run

* Fixed rebase conflicts

* Clean up driving.pxd/pyx

* Fixed linter error
2023-08-25 14:36:26 -07:00
Adeeb Shihadeh
c9e227a9c0 update azure-storage-blob (#29411)
Co-authored-by: Cameron Clough <cameronjclough@gmail.com>
2023-08-25 22:01:10 +01:00
Harald Schäfer
f1b8a86464 K1FD: generate kalman gain at init (#29610)
* K1FD: generate kalman gain at init

* Update interfaces.py

* Update interfaces.py

* Update interfaces.py

* Update ref_commit
2023-08-25 13:44:03 -07:00
Justin Newberry
90cb417caa CI: use GITHUB_ENV for env variables (#29616)
export doesn't work across steps
2023-08-24 17:03:25 -07:00
Harald Schäfer
8205590624 Locationd: enable laikad (#29570)
* experiment

* llk back

* update refs

* update ref

* no internet

* Update ref
2023-08-24 16:51:45 -07:00
Justin Newberry
1ee6ed4b16 Linter: remove pylint (#29611)
remove pylint
2023-08-24 16:30:54 -07:00
Justin Newberry
3d9f238bf9 CI: use scripts for docker build (#29599)
* use scripts

* remove common

* do login in the workflow

* separate step

* cleanup

* one script is cleaner

* keep it consistent

* review suggestions

* move in ci too

* PR cleanup
2023-08-24 16:09:08 -07:00
Harald Schäfer
8a7281d285 Laikad upgrades: prep for laikad lld (#29545)
* Laikad upgrades: prep for laikad lld

* Update ref commit
2023-08-23 00:13:10 -07:00
Adeeb Shihadeh
e0e139a233 soundd: update cpu usage 2023-08-22 15:16:38 -07:00
Harald Schäfer
fda3106c28 Revert laikad llk (#29535)
* Revert "bump laika (#29522)"

This reverts commit 3523bef732.

* Revert "Locationd: switch to laikad (#29278)"

This reverts commit fd3b8a3d75.
2023-08-22 14:12:51 -07:00
Adeeb Shihadeh
6a5725bb38 sensord: update cpu usage after removing light sensor 2023-08-22 14:03:15 -07:00
Harald Schäfer
fd3b8a3d75 Locationd: switch to laikad (#29278)
* use_laika

* Update test

* fix spacing

* fix proc replay

* small fix

* Try correct std

* cast correctly

* experimental

* simpler

* update reset thresholds

* improve

* Ready for live running

* outlier reject

* update laika

* add fine speeds

* more experiments

* fixes

* all in laika

* more small tweaks

* outlier rejection not needed

* fix conflict

* no more gpslocation

* update test

* refactor to laika

* runs again

* Update ref
2023-08-21 22:51:02 -07:00
Justin Newberry
197460e061 CI: use scons cache mount for building simulator and docs (#29436)
* build docker image with scons cache

* use setup

* scons cache

* added the dockerfile

* add the shell

* docs use cache

* dont pull lfs for docs

* simulator too

* dont pull the cache image

* fix inputs

* inputs

* booleans aren't supported

* bool not supported

* bool isnt supported

* fix

* readonly cache

* fix sim cache too

* respect readonly

* gitignore and move to correct folder

* also copy old

* move to tools/ci

* move to selfdrive/test

* just make the cache writable, it's discarded anyway

* minimize diff in this pr
2023-08-21 15:11:31 -07:00
Adeeb Shihadeh
a9626f95b6 add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
2023-08-20 20:49:55 -07:00
Harald Schäfer
1dea20e574 Proc replay: field tolerance unused (#29479)
field tolerance unused
2023-08-20 13:05:18 -07:00
Shane Smiskol
1a08c2986e Revert "radard: tie radard frequency to modelV2 vol. 2 (#29240)" (#29474)
* Revert "static analysis fixes"

This reverts commit a1e85f3661.

* Revert "radard: tie radard frequency to modelV2 vol. 2 (#29240)"

This reverts commit 460f5c8e0b.

* update refs
2023-08-18 22:56:43 -07:00
Shane Smiskol
e4ecb511c9 upadte process replay refs 2023-08-17 00:45:44 -07:00
Kacper Rączy
f07e4fb49a process_replay: prevent creating orphaned processes if replay fails during setup (#29426)
Fix the ability to cleanup, if process container fails during start
2023-08-16 10:37:19 -07:00
Justin Newberry
7ec52da33a Subaru: calculate steering angle rate (#29409)
* calculate rate

* added adeeb's comment

* move to selfdrive/car

* that makes way more sense

* much simpler

* fix rate

* preglobal doesn't have a counter

* update ref

* merge

* update ref after merging
2023-08-15 15:28:46 -07:00
Jason Young
90c4bbe8e4 car interfaces: cleanup tire stiffness calculation (#28263)
* cleanup tire stiffness calculation

* bump cereal

* think these todos still apply

* gm: same default

* revert comments

* bump

* update refs

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2023-08-15 14:17:08 -07:00
Adeeb Shihadeh
09ffce8073 add small ui for jenkins runners (#29382) 2023-08-13 18:01:51 -07:00
Adeeb Shihadeh
d6857996d3 cleanup unused python packages (#29353)
* rm unused python packages

* lock
2023-08-11 21:34:47 -07:00
Kacper Rączy
460f5c8e0b radard: tie radard frequency to modelV2 vol. 2 (#29240)
* radard: tie radard frequency to modelV2

Accumulate parsed messages until state is updated (toyota)

Same for honda

Rename rr to something more descriptive

* Change _update method name, since signature has changed

* Update ref commit

* Some renames

* Check for number of cans
2023-08-11 18:07:06 -07:00
Justin Newberry
e4ead4f183 Ruff: b905 (strict zip) (#29336)
* added mutable default args

* most of the Bs

* add comment about lrucache

* b905
2023-08-11 16:13:51 -07:00
Justin Newberry
62c1e65924 Ruff: enable most of bugbear (#29320)
* added mutable default args

* most of the Bs

* add comment about lrucache
2023-08-11 15:33:49 -07:00
Justin Newberry
8793cbff40 Ruff: enable flake8-builtins (#29315)
* enable flake8-builtins

* replace any with contains

* fix typo in pack

* fix type

* format is from the parent module, has to be enabled

* item_id

* fix item_id

* disable for id since that's what the remote server returns
2023-08-10 18:58:22 -07:00
Justin Newberry
4a9afd7554 Ruff: comprehensions conventions (#29317)
ignore c408
2023-08-10 16:40:30 -07:00
Adeeb Shihadeh
670fa9af5f pytest unit test running (#29261)
* setup config

* collect passes

* pytest system/ works
2023-08-10 08:23:19 -07:00
Mitchell Goff
c04e5d12fc Non-Inflatable Model 🎈 (#29003)
* Added all maneuvers to navInstruction message

* Added nav instruction logic to modeld

* New model: fcee01c1-96bb-414f-b00d-e4a994a00922/700

* Fixed bug in navd

* Added sharp/slight modifiers

* Updated refs
2023-08-09 19:56:45 -07:00
Adeeb Shihadeh
c53545621e jenkins: cleanup all orphaned git locks (#29296) 2023-08-09 15:14:50 -07:00
Justin Newberry
7fdd9fc37c CI: Pylint to ruff (#29294)
* pylint to riff

* pylint to riff

* pylint to riff

* revert more

* undo exclude removal

* exclude tinygrad

* set line length

* pylint exclusions to ruff

* same excludes as old linter

* fix tools QA

* remove unrequired check

* revert linting third_party

* ignore e402
2023-08-09 13:26:54 -07:00
Justin Newberry
cf64b27168 Linter: Fix >160 line length test (#29287)
* test

* fix line length

* fix line length

* fix file size

* revert poetry

* cleanup for PR
2023-08-08 17:13:35 -07:00
Justin Newberry
a91764954b Revert "CI: pylint to ruff (#29276)" (#29284)
This reverts commit cb31a53d8a.
2023-08-08 14:24:04 -07:00
Justin Newberry
cb31a53d8a CI: pylint to ruff (#29276)
* pylint to riff

* pylint to riff

* pylint to riff

* revert more

* undo exclude removal

* exclude tinygrad
2023-08-08 13:41:30 -07:00
Kacper Rączy
1774b4fc61 Revert "radard: tie radard frequency to modelV2 (#29063)" (#29243)
This reverts commit 255f5c13f2.
2023-08-04 20:25:40 -07:00
Kacper Rączy
255f5c13f2 radard: tie radard frequency to modelV2 (#29063) 2023-08-03 22:10:21 -07:00
Kacper Rączy
547a033a3c process_replay: capture process output (#29027)
* Add ProcessOutputProxy

* Move launcher to its own field

* Move ProcessOutputCapture to its own file

* Return itself from __enter__ of OpenpilotPrefix

* Integrate ProcessOutputCapture into process_replay

* Add note about capture_output_store to README

* ipykernel import is optional

* Disable type checking for link_with_current_proc

* Remove assertion

* Decode outputs to utf-8

* read(self): return empty buf if its none

* Fix type annotations

* Replace fifo with regular file, to avoid hitting fifo size limit
2023-07-31 16:30:58 -07:00
Adeeb Shihadeh
3476205569 increase min duration of steer saturated alert (#29191)
* increase min duration of steer saturated alert

* update refs
2023-07-31 12:48:11 -07:00
Shane Smiskol
12ed894362 dmonitoringmodeld: initialize model first (#29080)
* hacky fix

* even better

* ordering

* clean up

* fix model replay

* add comment

* catch sigint
2023-07-23 09:28:22 -07:00
Kacper Rączy
4793772c47 test_fuzzy: fix keyword argument (#29083)
Fix fuzzy test
2023-07-21 14:58:14 -07:00