Commit Graph

35 Commits

Author SHA1 Message Date
Shane Smiskol
12ed894362 dmonitoringmodeld: initialize model first (#29080)
* hacky fix

* even better

* ordering

* clean up

* fix model replay

* add comment

* catch sigint
2023-07-23 09:28:22 -07:00
Kacper Rączy
2c617e0c7a process_replay: modeld and dmonitoringmodeld support (#28463)
* vipc support. modeld and dmonitoringmodeld configs

* Update cereal

* Add vision meta

* Use vision meta in process replay

* Exclude modeld and dmonitoringmodeld from test_processes

* Refactor other pieces of code to use vision meta

* remove unnecessary imports

* Add MAX_FRAMES_PER_SEGMENT

* cameraState migration

* Move camera state migration to migration module

* model_replay refactor

* Fix lint issues

* Reduce number of vipc buffers to 2

* Update spinner message

* Update model replay ref

* Support the new log order

* Replace CameraFrameSizes with dict. Add encode_index field

* Migrate camera states based on encode indices

* Refactor process replay

* Refactor model replay

* Remove unnecessary print

* Update model replay ref

* Make camera state migration optional

* Remove unused import

* Add ignore_alive_pubs field

* Add assertion for carParams inside fingerprinting callback

* Remove WideCameraOnly param setup

* Refactor ModeldCameraSyncRcvCallback

* Update model replay ref commit

* Fix camera state migration

* Only migrate camera states when process has vision pubs

* Refactor model replay again

* Fix linter issues

* One more linter fix

* Update model replay ref commit
2023-06-22 17:36:25 -07:00
Kacper Rączy
64377a88f9 process_replay: helpers (#28367)
* process_replay helper

* Refactor

* refactor check_enabled uses

* fix __init__

* noqa in __init__ imports

* Move compare_logs outside of process_replay

* Move save_logs to tools.lib.helpers

* Remove save_log from compare_logs

* fix lint issues
2023-06-06 10:31:13 -07:00
Mitchell Goff
2d34083cd8 Test navmodel (#28272)
* Test navmodel

* removed comment

* Updated model_replay ref

* read tiles from azure bucket

* updated ref commit
2023-05-30 20:33:01 -07:00
Adeeb Shihadeh
5f66a9a621 jenkins fixups 2023-01-10 16:20:27 -08:00
Mitchell Goff
5ab2fc16ab model_replay: use last N frames of segment so navRoute is valid (#26790)
* model_replay: use last N frames of segment so navRoute is valid

* Use llk logMonoTime for MapRenderState.locationMonoTime

* Record mapRenderState

* update refs

Co-authored-by: Comma Device <device@comma.ai>
2022-12-20 14:19:21 -08:00
Adeeb Shihadeh
f3a711953a navmodel: check images in replay test (#26747)
* move position

* log images

* enable test

* update refs

* ignore

Co-authored-by: Comma Device <device@comma.ai>
2022-12-09 22:56:32 -08:00
Adeeb Shihadeh
1da6f6a6e2 nav model replay (#26697)
* nav model replay

* set token

* fix gh actions

* update refs

* fix pc
2022-12-07 19:36:30 -08:00
Adeeb Shihadeh
188c2c1c5e model replay speedup (#26181)
* model replay speedup

* less frames too

Co-authored-by: Comma Device <device@comma.ai>
2022-10-20 20:58:58 -07:00
HaraldSchafer
cb0b7375b7 Rocket Launcher Model (#25963)
* 1456d261-d232-4654-8885-4d9fde883894/440 6b7d7cec-ead8-40f3-86cc-86d52c9b03fe/300

* compute only 9 tokens: 1456d261-d232-4654-8885-4d9fde883894/440 6b7d7cec-ead8-40f3-86cc-86d52c9b03fe/300

* tinygrad: cleanup gather

* 1456d261-d232-4654-8885-4d9fde883894/440 6b7d7cec-ead8-40f3-86cc-86d52c9b03fe/700

* empty commit for tests

* bump tinygrad

* dont use tinygrad matmul for now

* bump tinygrad

* 1456d261-d232-4654-8885-4d9fde883894/440 e63ab895-2222-4abd-a9a5-af86bb70e260/700

* float16 1456d261-d232-4654-8885-4d9fde883894/440 e63ab895-2222-4abd-a9a5-af86bb70e260/700

* increase steer rate cost

* Revert "increase steer rate cost"

This reverts commit 74ce9ab9be7ef17ecfec931f96851b12f37f2336.

* fork tinygrad

* empty commit for tests

* basics

* Kinda works

* new lat

* new tuning

* Move LATMPCN so scons compiles

* Update long weights

* Add tinygrad optim

* Update model ref

* update weights

* Update ref

* Try

* Error message for field ignore

* update model regf

* ref commit

* Fix onnx test

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
2022-10-05 21:43:38 -07:00
Adeeb Shihadeh
6590fb2b93 pre-commit: add codespell (#25571) 2022-08-30 11:20:55 -07:00
Shane Smiskol
4602e5a8f9 process replay: print correct ref path (#25448)
* fix log paths ref printing in process replay

* rm that

* this is easier to read

* fix model_replay
2022-08-16 00:32:42 -07:00
HaraldSchafer
041458f632 Falcon Punch Model: turn cutting improvements (#25413)
* simplified change to mpc dynamics

* add jerk pts

* increase jerk cost

* increase jerk pts multipler to master value

* Add final commit

* 1456d261-d232-4654-8885-4d9fde883894/440 ac1a6744-85b0-4ec6-8ba7-608d0717b8f1/750

* some copies are useful

* update model replay ref

* less frames in model replay onnx cpu

* 1456d261-d232-4654-8885-4d9fde883894/440 264b67f5-3f52-4b58-b11f-58dd8aaf08bf/950

* 1456d261-d232-4654-8885-4d9fde883894/440 236fc556-fba3-4255-8ccf-684b22637160/950

* c9d10c64-bea4-41ec-8ca3-d8c886fda172/440 26d73dd2-862a-44ae-bbdd-32cc4f397ad7/900

* Fix couple tests

* Update ref

* Unused for now

* Add lateral factor comment

* Unused variable

Co-authored-by: nuwandavek <vivekaithal44@gmail.com>
Co-authored-by: Bruce Wayne <yassine@comma.ai>
Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
Co-authored-by: Bruce Wayne <batman@gpu06.internal>
2022-08-12 00:47:59 -07:00
YassineYousfi
3afe7464df Model file too big: Revert "New model: minor fixes" (#25390)
Revert "New model: minor fixes (#25368)"

This reverts commit 55bf938504.
2022-08-08 12:42:46 -07:00
HaraldSchafer
55bf938504 New model: minor fixes (#25368)
* 1456d261-d232-4654-8885-4d9fde883894/440 ac1a6744-85b0-4ec6-8ba7-608d0717b8f1/750

* some copies are useful

* update model replay ref

* less frames in model replay onnx cpu

Co-authored-by: Bruce Wayne <yassine@comma.ai>
Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
2022-08-05 17:28:15 -07:00
HaraldSchafer
772b748689 Cleanup calibration code (#25119)
* First attempt

* worksish

* tests pass

* cleanup

* get rid of garbahe

* fix name

* Still used in xx

* add debug functions

* used

* Revert "used"

This reverts commit 276e8ebab06d2d4f0e9927ba32b7d8aca2bf88c3.

* Update ref
2022-08-01 20:35:14 -07:00
Adeeb Shihadeh
ef00bf76c1 model_replay: update format_diff call 2022-07-25 00:08:30 -07:00
ZwX1616
1f2f9ea9c9 fullframe DM model (#24860)
* Revert "put cereal on master"

This reverts commit a8ccd8f838.

* Revert "Revert fullframe DM model (#24812)"

This reverts commit c646eeee0a.

* revert revert cereal

* clip6

* 0.8 is fair

* Fiction compensation should be based on error

* Update refs

* Add deadzone

* not that

* good mg

* ref

* ref

* ee8f

* minor tweak

* ref

* recompile

* ref

* cereal

* match driverstatus

* new ref

* new ref

* pass token through jenkins credentials

* quote

* fix snpe dead weights

* final ref

Co-authored-by: Harald Schafer <harald.the.engineer@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-06-20 16:24:51 -07:00
Adeeb Shihadeh
0fce5d9045 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
2022-06-11 23:19:27 -07:00
Adeeb Shihadeh
1139fe507b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
2022-06-11 16:38:24 -07:00
ZwX1616
c646eeee0a Revert fullframe DM model (#24812)
* Revert "fullframe DM: flip RHD yaw to use matching thresholds"

This reverts commit 2ac6931003.

* Revert "fullframe DM model (#24762)"

This reverts commit d6c07a6b15.

* revert cereal
2022-06-10 16:16:46 -07:00
ZwX1616
d6c07a6b15 fullframe DM model (#24762)
* get log

* simplify two nonsense

* not needed

* libyuv is a joke

* clean up

* try small

* fast but not bad

* working

* clean up driverview

* simplified

* thats mirrored

* smol

* tweak

* ref is screen

* w/ ee

* update camera model

* no if TICI

* start

* update pose thresh

* less cpu more dsp

* new libyuv

* new snpe

* add files

* test

* should be fast

* update out len

* trigger test

* use master snpe

* add cereal

* update cereal

* refactor parsing

* missing ;

* get

* wrong type

* test model

* use driver data

* 10829278-72fe-4283-a118-2cef959ce174/1550

* no pf

* adapt driverview

* ;

* rhd learner

* update libyuv buildi x64

* ad4337ea

* remove blink slack

* test

* no

* use toggle

* b16

* fix for nv12

* 5b02cff5 both

* update test

* update cereal

* update cereal

* update cereal

* v2 packets

* revert libyuv

* no /

* update snpemodel

* ;

* memcpy

* fix test

* use toggle in driverview

* update power

* update replay

* Revert "update replay"

This reverts commit 1d0979ca59dbc89bc5890656e9501e83f0556d50.

* update model ref

* halve cpu

* fake 8bit onnx runner

* same thresh as report

* cereal master

Co-authored-by: Comma Device <device@comma.ai>
2022-06-08 20:13:46 -07:00
HaraldSchafer
a2c6b30c98 Fix onnx model replay (#24702)
* No more C2

* Add test

* Various fixups

* Use ONNXCPU

* rm prints

* LFS pull

* Use sim docker

* Try sim docker again

* Print counter

* Make sure it can fail

* Fails correctly

* comments

* Add more install to opencl base docker

* vim not needed in base
2022-06-03 10:43:09 -07:00
George Hotz
ea5b8cdfb1 nv12: encoderd avoids a full frame copy (#24519)
* rgb to nv12

* nv12 works (w memcpy)

* correct now

* no copy

* fix nv12 with fast debayer

* reverts of unused stuff

* ui use nv12

* comment out thumbnails for now

* rebase fix

* dm read nv12

* model read nv12

* fix ffmpeg encoder

* thumbnails from nv12

* replay to nv12

* python framereader support nv12

* remove hardcoded frame/buffer sizes

* fix build

* ffmpeg encoder fix buffers

* small cleanup

* reduce power usage test

* fix cpu usage test

* fix snapshot

* fix loggerd test

* bump cereal

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
2022-06-01 17:18:28 +02:00
Adeeb Shihadeh
32a9ccfc37 visionipc python shim (#24660) 2022-05-25 20:49:56 -07:00
grekiki
17e33978cd Mypy fixes for --check-untyped-defs (#24372)
more type fixes
2022-05-12 14:08:11 +02:00
Adeeb Shihadeh
5c48e7bc86 remove comma two support (#24248)
* remove comma two support

* cleanup release files

* little more

* more libs

* no more gralloc

* add snpe back
2022-04-18 17:55:23 -07:00
HaraldSchafer
48154703a7 Fix tici bigmodel (#23817)
* Fix bigmodel on tici: cam intrinsics arent static

* Give TICI route own name for import

* Make function useable in offline test

* typo

* change tici ecam focal to trained focal

* No shadow variable

* Fixed ref

* Force update this time

* Update model replay ref commit
2022-02-21 19:57:04 -08:00
Willem Melching
85efde269d bigmodel (#23684)
* Added wide cam vipc client and bigmodel transform logic

* Added wide_frame to ModelState, should still work normally

* Refactored image input into addImage method, should still work normally

* Updated thneed/compile.cc

* Bigmodel, untested: 44f83118-b375-4d4c-ae12-2017124f0cf4/200

* Have to initialize extra buffer in SNPEModel

* Default paramater value in the wrong place I think

* Move USE_EXTRA to SConscript

* New model: 6c34d59a-acc3-4877-84bd-904c10745ba6/250

* move use extra check to runtime, not on C2

* this is always true

* more C2 checks

* log if frames are out of sync

* more logging on no frame

* store in pointer

* print sof

* add sync logic

* log based on sof difference as well

* keep both models

* less assumptions

* define above thneed

* typo

* simplify

* no need for second client is main is already wide

* more comments update

* no optional reference

* more logging to debug lags

* add to release files

* both defines

* New model: 6831a77f-2574-4bfb-8077-79b0972a2771/950

* Path offset no longer relevant

* Remove duplicate execute

* Moved bigmodel back to big_supercombo.dlc

* add wide vipc stream

* Tici must be tici

* Needs state too

* add wide cam support to model replay

* handle syncing better

* ugh, c2

* print that

* handle ecam lag

* skip first one

* so close

* update refs

Co-authored-by: mitchellgoffpc <mitchellgoffpc@gmail.com>
Co-authored-by: Harald Schafer <harald.the.engineer@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
2022-02-19 16:06:31 -08:00
Ewout ter Hoeven
55390d273f Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh
66ee3e9ef9 model replay: more consistent replay (#23237)
* first cleanup

* don't send calib and desire

* should always be valid

* little more
2021-12-15 16:12:24 -08:00
Dean Lee
348d2d2b0d rename yuv streams (#23071)
* rename yuv streams

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-11-29 18:14:59 -08:00
Willem Melching
f7c46c6949 Do not run code on version.py import (#23063)
* Do not run code on version.py import

* fix athena
2021-11-29 19:38:55 +01:00
ZwX1616
1ec8f03ec6 test: Replay both models on C2/C3 (#23019)
* add dcam stream

* delete model_test

* both in jenkins

* update spinner

* get filename fn

* they are not random names

* update route

* new ref commit

* temporarily resize tmp

* clean up unlog send

* need calib 0

* revert device

* cleanup

* arg

Co-authored-by: Comma Device <device@comma.ai>
2021-11-24 15:59:19 -08:00
Adeeb Shihadeh
cf6d133638 fix up model tests + tools (#21071)
* unlogger: send yuv stream

* fix up model test tools

* fix unlogger

* rename model replay

* bump cereal

* test in actions

* no ci for now
2021-05-28 23:08:08 -07:00