Commit Graph

883 Commits

Author SHA1 Message Date
Lukas Petersson
444d265821 process replay: support running in parallel (#24534)
* prefix params

* set env

* prefix in manager

* filesystem except

* dont delete manager folder

* Update selfdrive/common/params.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug same path

* remove cleanup + same default

* dont use filesystem lib

* param symlink path

* prefix helpers

* path

* dont delete d

* parallel loop

* refactor loop

* msgq

* clean msgs

* spelling

* nestable pool

* spelling

* logreaders in parallel

* bugfix

* assert msgq

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* assert in cereal

* folder exists

* create dirs in test

* bump cereal

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* PR feedback

* params path in basedir

* ref commit

* param path

* bugfix

* upload_only

* param path

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* msgq path name

* python concurrency.features

* progress bar

* remove progress bar from compare logs

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* defaultdict

* context manager

* update refs

* dont get logs if upload only

* upload refs in parallel

* cleanup

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* cleanup

* text

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
2022-05-18 19:59:30 -07:00
HaraldSchafer
e2e5378dd3 Revert "Joint model (#24521)" (#24581)
This reverts commit 7996fd37ef.
2022-05-18 10:56:45 -07:00
Adeeb Shihadeh
3771ccc655 controlsd cpu has crept up 2022-05-17 19:42:58 -07:00
HaraldSchafer
9f8b03753d Revert torque control (#24565)
* torque reversal start

* Fix carmodel tests

* Update ref

* update ref

* Elif is better than if
2022-05-17 19:00:57 -07:00
George Hotz
72b52f8b7d make debayer faster (#24557)
* remove local caching

* remove local caching

* reduce camerad power

* break stupid imx390 black level support

* handle edges

* now 13ms, vignetting is 'slightly' less correct

* halfs->floats and inlines, down to 12.9ms

* oops, fix float

* val from 12 is ushort

* don't decide vignetting in the debayer kernel

* 7.77 ms

* adding back black level support was free

* Revert "adding back black level support was free"

This reverts commit a841d17727886807a040dcf856ab22480fa93eff.

* minor

* rip out unused gain and black level, remove print

* save 150mW

* fix replay test

* fix top/bottom rows

* lame left right edge fix

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
2022-05-17 10:18:36 -07:00
HaraldSchafer
7996fd37ef Joint model (#24521)
* update models

* wip lanelines mhp parsing

* typos

* newer models

* 95995a49-db0c-4261-8776-b90780dc2a8c/600

* Use laneful policy to prevent planner aggression

* c8c3ab69-bd2c-4d1c-a7f8-9e8457e4827b/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* 7205c5af-7532-4215-ad9b-678789054db7/995

* bfb34fa3-1d72-4987-854f-43d42ee01015/950

* change cost

* 8ae2477a-b452-4601-b04c-d3af66ea5e98/950

* Update heading cost

* 9b991314-94f5-4f0b-872e-58d95266e4d0/333

* 9b991314-94f5-4f0b-872e-58d95266e4d0/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* undo lateral planner changes

* rm whitespace

* update ref

* c0acc19d-f2f1-47ae-8838-5382c6c6c7f5/950

* c0acc19d-f2f1-47ae-8838-5382c6c6c7f5/950

* update model refg

* Update model ref

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
2022-05-16 17:26:23 -07:00
Willem Melching
31af13101b Add unittest for alert callbacks (#24551)
add alert callback test
2022-05-16 16:25:52 +02:00
Joost Wooning
2294fcdbe7 debayering: update test ref commit 2022-05-16 13:37:16 +02:00
Adeeb Shihadeh
ab75c53792 replay: cleanup unused RGB buffers (#24537)
* replay: cleanup unused RGB buffers

* update replay cam

* little more
2022-05-14 19:55:12 -07:00
Shane Smiskol
d6c6275163 Fix process replay regen (#24517)
* fix regen issues

* compress like uploader
2022-05-14 19:53:08 -07:00
George Hotz
94b9972eb7 encoderd: enable (#24492)
* enable encoderd

* correct enable line

* fix loggerd tests

* fix power draw and cpu tests

* correct cpu for encoderd

* fix a bug, video_writer is shared

* fix issue with not recording dcam

* add recording state

* wooo tests pass. encode id keeps counting

* core 3

* loggerd then encoderd

* stop loggerd first

* core 3 always online

* see the camera when we see encoder packet

* encoderd on small core uses 37%

* remove encoder logic from loggerd

* delete unit test that doesn't really make sense anymore

Co-authored-by: Comma Device <device@comma.ai>
2022-05-13 17:56:02 -07:00
Joost Wooning
489fbb74b0 camerad: combine debayering and rgb_to_yuv opencl kernels (#24452)
* camerad: combine debayering and rgb_to_yuv opencl kernels

* fix border

* fix snapshot

* rename function

* update camerad cpu usage

* update camerad power draw

* vignetting equal to previously

* test other local worksize

* use less floats

* reduce amount of code

* move barrier back

* make faster

* fix corners

* cleanup

* cleanup

* allow more jitter on driverState timing

Co-authored-by: Comma Device <device@comma.ai>
2022-05-13 16:49:31 +02:00
Adeeb Shihadeh
ef81afc354 Revert "Joint Model (#24213)"
This reverts commit eff08cb17f.
2022-05-12 22:52:45 -07:00
Adeeb Shihadeh
c643d74609 jenkins: workaround lfs bug 2022-05-12 22:08:25 -07:00
YassineYousfi
eff08cb17f Joint Model (#24213)
* update models

* wip lanelines mhp parsing

* typos

* newer models

* 95995a49-db0c-4261-8776-b90780dc2a8c/600

* Use laneful policy to prevent planner aggression

* c8c3ab69-bd2c-4d1c-a7f8-9e8457e4827b/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* 7205c5af-7532-4215-ad9b-678789054db7/995

* bfb34fa3-1d72-4987-854f-43d42ee01015/950

* change cost

* 8ae2477a-b452-4601-b04c-d3af66ea5e98/950

* Update heading cost

* 9b991314-94f5-4f0b-872e-58d95266e4d0/333

* 9b991314-94f5-4f0b-872e-58d95266e4d0/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* undo lateral planner changes

* rm whitespace

* update ref

Co-authored-by: Harald Schafer <harald.the.engineer@gmail.com>
2022-05-12 19:47:33 -07:00
Adeeb Shihadeh
affeb2ef73 jenkins: prune lfs objects 2022-05-12 19:38:58 -07:00
Shane Smiskol
d686f9e527 process replay: save logs to folder (#24509)
* save logs to folder for proc replay

* use fakedata dir

* fix

* only once and where needed
2022-05-12 19:19:47 -07:00
Shane Smiskol
934bc1e6fb process replay: don't print diff when only uploading (#24514)
when only uploading no need to print success status or update diff

don't print succeed/fail when only uploading
2022-05-12 17:05:22 -07:00
Shane Smiskol
6518ecca8f Add desired curvature to controlsState (#24510)
* Add desired_curvature to controlsState

* clean up

* bump

* bump

* update refs from GitHub!

Co-authored-by: Jason Shuler <jshuler@gmail.com>
2022-05-12 16:59:52 -07:00
Joost Wooning
7ff3e32c2b debayering: update ref commit for debayer test 2022-05-12 14:56:32 +02:00
grekiki
17e33978cd Mypy fixes for --check-untyped-defs (#24372)
more type fixes
2022-05-12 14:08:11 +02:00
Shane Smiskol
d2d3b7b823 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-05-11 15:03:05 -07:00
HaraldSchafer
29afd53d88 Latcontrol torque: fix integrator induced ping pong (#24458)
* Latcontrol torque: fix integrator induced ping pong

* Reset on disengage since unwind resets anywayh

* Might be overkill

* rm whitespace

* update ref
2022-05-06 22:17:21 -07:00
Joost Wooning
49e10dc773 debayering: fill outer image borders (#24441)
* debayering: fill outer image borders

* correct conditions

* try something else

* remove global check

* update ref

Co-authored-by: Comma Device <device@comma.ai>
2022-05-06 18:27:41 +02:00
Willem Melching
9ef17a877f sanity check on debayer time in CI (#24453)
* check debayer time

* set actual threshold

* also print mean and max
2022-05-06 17:27:54 +02:00
Willem Melching
85975e3dd9 test_debayer.py some PC fixes (#24449)
* test_debayer.py some PC fixes

* fix rgb shape
2022-05-06 16:32:44 +02:00
Joost Wooning
9da9b773eb test for comparing debayer results to ref commit (#24420)
* test for comparing debayering results to ref commit

* fix run on device

* print more useful diff

* limit to diff output

* remove env var

* ref

* more useful diff

* use real local work size

* remove pyopencl install

* update ref

* include rgb_to_yuv

* fix diff

* get kernel build out of loop

Co-authored-by: Comma Device <device@comma.ai>
2022-05-05 16:05:37 +02:00
ntegan1
9a44d4c4e0 LatControlTorque: log desired/actual lateral accel (#24406)
* log torque lateral accel

* bump cereal

* update refs

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2022-05-03 14:18:46 -07:00
Adeeb Shihadeh
986a991a09 body: log charging status (#24409)
* body: log charging status

* update refs
2022-05-03 13:27:19 -07:00
Willem Melching
77a6f3d034 delay bz2 compression from logging to uploading (#24392)
* remove log_name

* log without compression

* fix tests

* remove extension for bootlog

* another test fix

* uploader compresses

* also compress in athena

* only compress qlog

* more generic check in do_upload

* fix bootlog compression

* lower loggerd cpu usage

* dont link against bz2

* set core affinity to little cluster

* handle old files
2022-05-03 14:09:17 +02:00
HaraldSchafer
8cdeaf9aee Latcontrol torque: update tuning (#24357)
* Little more chill

* Update ref

* Update refs
2022-05-02 14:42:18 -07:00
Shane Smiskol
4190d2608a Change safetyParam to uint16_t (#24376)
* bump panda & cereal

* bump panda

* rest of references

* regen TOYOTA with safety param 578, TOYOTA2 with 329

* regen rest of routes

* update ref

* bump cereal
2022-05-02 13:34:31 -07:00
Keyvan Fatehi
0bbfc63c9c body: add fuelGauge to body carState (#24386) 2022-05-01 18:41:50 -07:00
Adeeb Shihadeh
be748c0d6a controlsd: better alert precedence for system malfunctions (#24366)
* controlsd: better alert precedence for system malfunctions

* down to 20

* move that

* check can flags

* update refs

* update refs
2022-04-29 13:36:25 -07:00
Willem Melching
a3cfa444d3 ui: combine OnroadHud into NvgWindow (#24369)
* ui: combine OnroadHUD and NVGWindow

* draw hud first

* cleanup

* removed commented line

* fix text rendering

* increase cpu usage
2022-04-29 10:55:44 -07:00
George Hotz
b7ddce8bac camerad: less ioctls try 2 (#24371)
* premap the buffers

* memory manager

* free buffers properly, alignment seems okay

* update camerad CPU usage

* cam_sync_fd

* useless line, and use the define

* cheap prereqs for multistream

Co-authored-by: Comma Device <device@comma.ai>
2022-04-29 18:09:12 +02:00
George Hotz
5ddd2000b2 Revert "camerad: don't remap everything every time (#24334)" (#24370)
This reverts commit fb7d84875b.
2022-04-29 07:59:03 -07:00
Shane Smiskol
c1caca104f Support RAV4 Hybrid 2022 with stock longitudinal (#23969)
* add panda flag for toyota stock long with camera

* clean up

* Add 2022 RAV4 Hybrid from Philly

* fix wrong fw in interface, did this ever work?

* Must be a hybrid

* no radar parsing

* fix can error

* move to own platform

* generate docs

* fix

* Add 2022 Rav4 XSE Australia fingerprint parameters (#24303)

* Update values.py

Add 2022 Rav4 XSE Australia

* add commas

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* bump panda

* wait, the camera doesn't even send 0x343, right?

* use a set instead, more obvious

* don't test without a parser

* bump panda

* flip panda flag

* bump panda

* add commas

* regen and update refs

* set to none by default

* revert parenthesis

* update comment

* bump panda

* regen and update refs

* add test models and update readme

* bump to master

Co-authored-by: BrettLynch123 <34538435+BrettLynch123@users.noreply.github.com>
2022-04-28 22:37:46 -07:00
Shane Smiskol
66dd8f934c safety param: make uint16_t (#24362)
* only use half of the available bytes

* move to set_safety_model

* regen and update refs

* last two

* bump to master
2022-04-28 17:56:34 -07:00
George Hotz
fb7d84875b camerad: don't remap everything every time (#24334)
* premap the buffers

* memory manager

* free buffers properly, alignment seems okay

* update camerad CPU usage

* cam_sync_fd

* useless line, and use the define

* cheap prereqs for multistream

Co-authored-by: Comma Device <device@comma.ai>
2022-04-28 11:32:32 -07:00
Adeeb Shihadeh
3e098c9ba5 Toyota: add safety param flag for alt brake message (#24341)
* Toyota: add safety param flag for alt brake message

* update refs

* update refs

* bump panda
2022-04-27 12:08:11 -07:00
Shane Smiskol
e834b4d9be LatControlTorque: fix deadzone and missing steer saturated warning (#24294)
* fix steer saturated alert and deadzone

* fix and formatting

* update refs

* update refs

* not needed

* remove
2022-04-27 01:18:05 -07:00
Shane Smiskol
efa98d2a06 HKG: only allow engagement on user button press (#24140)
* use deque for prev_cruise_buttons

* stash

* disallow engagement if resume or decel isn't pressed

* fix buttons

* reduce chances of causing 30-frame fault

Fix clu11

drive down cancel times

* Revert "reduce chances of causing 30-frame fault"

This reverts commit 0f54c051bf5a22b1f584718ec46e78bc3a82a0dd.

* consider pause/resume button

* stash

* Revert "stash"

This reverts commit 551ca7be6c45729f0747abc81cd81894cd621c32.

* sadly some cars need 8 op frames (identical to 4 updates from CLU11)

* add main button

* 6 should be safe

* use max

* clean up

* change to 4 samples and process all messages received, like panda

* bump panda

* test: replay segments with known re-engagements on-device

* Revert "test: replay segments with known re-engagements on-device"

This reverts commit 9730c3c14f942b82b6ed5ef2e81b8ae0126f3006.

* need prev_cruise_buttons as we don't run every CAN message through carstate

* more generic

* extend

* Update selfdrive/car/interfaces.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* 3% faster at 1000000 loops

* update refs

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-04-26 13:27:28 -07:00
Shane Smiskol
60b7114024 onnxruntime: initialize CUDA provider before predictions (#24330)
* initialize CUDA runtime before predictions

* dmond pauses modeld execution...

* done in other PR
2022-04-26 11:49:32 -07:00
Shane Smiskol
a8e632bb98 regen: send vipc_client_extra frames (#24331)
* send extra frames to fix regen

* for sanity

* not used

* minimal changes
2022-04-26 00:15:09 -07:00
Joost Wooning
f84fdc86bb UI: remove memcpy on frame receive (#24318)
* UI: remove frame memcpy

* fix freezing

* fix onroad test
2022-04-25 13:45:55 -07:00
Adeeb Shihadeh
13e1c0b339 Revert "UI: remove RGB buffer usage (#24298)"
This reverts commit b9907e86b2.
2022-04-24 11:56:30 -07:00
Shane Smiskol
f4d1703b42 longActive: consider longitudinal control (#24295)
* consider longitudinal control

* update refs
2022-04-22 20:32:36 -07:00
Adeeb Shihadeh
b88fb037ab tici downclock (#24088)
* tici downclock

* fix that

* gp ref

* lower that a bit

* that was a scam

* perf

* update power

* increase that

Co-authored-by: Comma Device <device@comma.ai>
2022-04-22 14:14:59 -07:00
Joost Wooning
b9907e86b2 UI: remove RGB buffer usage (#24298)
* ui: use yuv buffer

* cleanup

* fix watch3

* replay always send yuv

* fix drawing of path

* fix onroad test

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-04-22 12:55:37 -07:00