Commit Graph

68 Commits

Author SHA1 Message Date
Adeeb Shihadeh
3c48bc47b7 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
old-commit-hash: 0fce5d9045
2022-06-11 23:19:27 -07:00
Lukas Petersson
35f277516e process replay: upload-only bug (#24722)
dont look for lr with upload only
old-commit-hash: 50a8b30580
2022-06-02 17:25:44 -07:00
Shane Smiskol
6e6f8422fc process replay: fix tested_cars
old-commit-hash: 6a21cc83ff
2022-06-02 14:20:19 -07:00
Shane Smiskol
490a836dd6 process replay: fix updating refs (#24716)
fix updating refs
old-commit-hash: 90af436a12
2022-06-02 13:49:21 -07:00
Lukas Petersson
a16a20005b process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 86f73a507e
2022-06-01 22:02:42 -07:00
ClockeNessMnstr
eeccd04c11 process replay: only download used logs (#24661)
* only download wanted logs

* cleanup

* cache as an option

* cleanup

* Readme

* Revert "cache as an option"

This reverts commit 060ed4ade548696505e3185fc7d20f1897072ded.

* cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 3fb22c0289
2022-06-01 21:46:45 -07:00
Shane Smiskol
222d8818d7 process replay: fix updating refs after regen (#24656)
* fix updating refs after regen

* match spacing

* same as before

* fix

* minimize diff

* update

* not used
old-commit-hash: dff70876c3
2022-05-26 00:39:33 -07:00
Lukas Petersson
81af5dacdf process replay: update regen body route (#24648)
update to a preserved route
old-commit-hash: 3e975eb3aa
2022-05-24 17:07:10 -07:00
Lukas Petersson
d1e21cef79 process replay: support running in parallel (#24534)
* prefix params

* set env

* prefix in manager

* filesystem except

* dont delete manager folder

* Update selfdrive/common/params.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug same path

* remove cleanup + same default

* dont use filesystem lib

* param symlink path

* prefix helpers

* path

* dont delete d

* parallel loop

* refactor loop

* msgq

* clean msgs

* spelling

* nestable pool

* spelling

* logreaders in parallel

* bugfix

* assert msgq

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* assert in cereal

* folder exists

* create dirs in test

* bump cereal

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* PR feedback

* params path in basedir

* ref commit

* param path

* bugfix

* upload_only

* param path

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* msgq path name

* python concurrency.features

* progress bar

* remove progress bar from compare logs

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* defaultdict

* context manager

* update refs

* dont get logs if upload only

* upload refs in parallel

* cleanup

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* cleanup

* text

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 444d265821
2022-05-18 19:59:30 -07:00
Shane Smiskol
28dfaa0abe process replay: save logs to folder (#24509)
* save logs to folder for proc replay

* use fakedata dir

* fix

* only once and where needed
old-commit-hash: d686f9e527
2022-05-12 19:19:47 -07:00
Shane Smiskol
5a0778518d process replay: don't print diff when only uploading (#24514)
when only uploading no need to print success status or update diff

don't print succeed/fail when only uploading
old-commit-hash: 934bc1e6fb
2022-05-12 17:05:22 -07:00
grekiki
7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cd
2022-05-12 14:08:11 +02:00
Shane Smiskol
024f42bae5 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d2d3b7b823
2022-05-11 15:03:05 -07:00
Shane Smiskol
121b708ba1 Change safetyParam to uint16_t (#24376)
* bump panda & cereal

* bump panda

* rest of references

* regen TOYOTA with safety param 578, TOYOTA2 with 329

* regen rest of routes

* update ref

* bump cereal
old-commit-hash: 4190d2608a
2022-05-02 13:34:31 -07:00
Shane Smiskol
754b93997a Support RAV4 Hybrid 2022 with stock longitudinal (#23969)
* add panda flag for toyota stock long with camera

* clean up

* Add 2022 RAV4 Hybrid from Philly

* fix wrong fw in interface, did this ever work?

* Must be a hybrid

* no radar parsing

* fix can error

* move to own platform

* generate docs

* fix

* Add 2022 Rav4 XSE Australia fingerprint parameters (#24303)

* Update values.py

Add 2022 Rav4 XSE Australia

* add commas

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* bump panda

* wait, the camera doesn't even send 0x343, right?

* use a set instead, more obvious

* don't test without a parser

* bump panda

* flip panda flag

* bump panda

* add commas

* regen and update refs

* set to none by default

* revert parenthesis

* update comment

* bump panda

* regen and update refs

* add test models and update readme

* bump to master

Co-authored-by: BrettLynch123 <34538435+BrettLynch123@users.noreply.github.com>
old-commit-hash: c1caca104f
2022-04-28 22:37:46 -07:00
Shane Smiskol
32075d4681 safety param: make uint16_t (#24362)
* only use half of the available bytes

* move to set_safety_model

* regen and update refs

* last two

* bump to master
old-commit-hash: 66dd8f934c
2022-04-28 17:56:34 -07:00
Adeeb Shihadeh
211fc8fc68 Toyota: add safety param flag for alt brake message (#24341)
* Toyota: add safety param flag for alt brake message

* update refs

* update refs

* bump panda
old-commit-hash: 3e098c9ba5
2022-04-27 12:08:11 -07:00
Shane Smiskol
ebec632ea2 Make safetyParam unsigned (#24160)
* bump panda & cereal

* make unsigned in boardd

* update health packet version

bump panda

* fix definition

* bump

* regen TOYOTA2 route

* upload route

* oh, ndog param wasn't being set

* regenerate routes with non-zero safety params

* update refs

* regen routes and update refs

* bump panda

* update refs

* update refs

* bump submodules to master
old-commit-hash: 6e62df65d7
2022-04-21 00:01:05 -07:00
Shane Smiskol
500034211d regen: check if regenerated route is enabled (#24207)
* check if regenerated route is enabled

* elif

* initialize CUDA runner before all other processes

* Revert "initialize CUDA runner before all other processes"

This reverts commit 124e3dd4d3408b6649f40b3e6e7f217ca347f59f.

* add back fixes

* fix typo
old-commit-hash: 1a5ddfaedd
2022-04-13 18:01:16 -07:00
Adeeb Shihadeh
a8546e8bf9 body: set fault state (#24125)
* body: set fault state

* fix that

* update refs
old-commit-hash: 6c5b0685da
2022-04-04 23:05:40 -07:00
Adeeb Shihadeh
9e0a09868a body: only control while engaged (#24114)
* body: only control while engaged

* bump panda

* remove that

* bump panda

* update refs

* update refs
old-commit-hash: 00ebd10b87
2022-04-04 19:37:21 -07:00
Adeeb Shihadeh
78bb03345e better support non-car robots (#24106)
* better support non-car robots

* bump cereal

* update refs
old-commit-hash: e40c36f22b
2022-04-01 20:03:20 -07:00
Igor Biletskyy
df806cdbeb car port: comma body (#24019)
* body FPv2

* ..

* ..

* ..

* Temp, REVERT!

* more cleanup

* typo

* ..

* del eyes

* should work?

* fix

* new dbc

* ..

* fixes

* static analysis

* cln balancing code

* no test route

* excluded_interfaces

* THE DOCS!

* comments on steer/speed mixin

* switch to bus 0

* less UDS

* FAKE bus 0 vin and fingerprint

* FAKE locationd

* Keep steady and remove handcoded offset

* Improve startup sequence, get closer to stock openpilot

* Forgot to define angle

* lowercase

* revert that

* little cleanup

* change safety model

* update refs

* body is gold

* handle no llk

* oops

* litte more

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 031f79ba88
2022-03-31 17:16:15 -07:00
Willem Melching
962201fff1 Ensure controlsd can engage again in process replay (#23556)
* hacks to make process replay engage again

* dont change that

* enable engagement check again

* chmod +x

* first working regen

* proper logMonoTime

* fix video framerate consistency

* mpc is valid again

* proper alive checks

* revert loggerd change

* ensure engaged in regen, fix managerState, peripheralState

* ubloxRaw is unused

* add submaster config for gm

* regen all services we can run without HW

* fix loggerd

* loggerd: matroska without the extension

* update ref
old-commit-hash: d3d10e014a
2022-01-21 13:13:34 +01:00
Ewout ter Hoeven
f206ebd054 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
old-commit-hash: 332f568a82
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven
a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh
e0456c4989 Mazda: move CX-9 out of dashcam mode (#22717)
* move cx9 out of dashcam

* no alc below 28

* process replay route
old-commit-hash: 2252c6bba7
2021-10-31 22:02:36 -07:00
Robbe Derks
58c99158a1 Make pandaState and safetyMode a list (#22454)
* wip: move to pandaStates

* bump cereal

* wip: SafetyMode struct

* move to safetyMode

* fix typo

* this can be None

* fix potential empty pandaStates list

* fix thermald

* fix controlsd

* rename safetyModes to safetyConfigs

* update process_replay

* fix test_models

* bump cereal
old-commit-hash: 91987f38d4
2021-10-08 17:54:34 +02:00
Adeeb Shihadeh
1244074094 add TSS2 replay route (#21953)
* add TSS2 replay route

* skip on sim

* do not set on import time

* update ref

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 86f68ff22e
2021-08-31 10:48:21 -07:00
Adeeb Shihadeh
bc0803868f update prius test route (#21791)
old-commit-hash: f7dc898853
2021-07-29 20:04:07 -07:00
Harald Schafer
6aca5f9f38 update
old-commit-hash: 1979005e82
2021-07-09 16:31:34 -07:00
HaraldSchafer
7e87930a20 Refactor lateral lag compensation (#21334)
* add T_IDXS

* refactor

* fix test

* unused

* typo

* needs casting

* Update selfdrive/controls/lib/drive_helpers.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* deprecate field

* regen all

* new segs

* add todo

* split back

* clean

* bad names

* do in controls

* add arg

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 6838e1c82c
2021-06-30 14:19:39 -07:00
Adeeb Shihadeh
d31397b84b segment regen (#21148)
* start seg regen

* send vipc

* fix up some paths

* set fp

* fix no entries

* engages

* seperate camera procs

* send real frames

* regen test routes

* nice argparser

* fix valgrind test

* move that

* fix that
old-commit-hash: 4a1aec27ef
2021-06-28 21:44:40 -07:00
robbederks
a1157cc762 Tesla AP2 port (#20074)
* squashed the PR

* remove AP passthrough

* disengage on steer override

* bump panda

* add test routes

* bump panda

* tesla is still dashcam only

* dashcam only

* rerun CI

* add to release files
old-commit-hash: cdfba105ed
2021-06-23 14:56:59 +02:00
Adeeb Shihadeh
b700c1bda3 Update VW process replay route (#21142)
old-commit-hash: 7d8480cffe
2021-06-04 15:29:13 -07:00
Adeeb Shihadeh
9531bf9d88 fix process replay on local files (#20758)
old-commit-hash: c45dbbc737
2021-04-26 20:50:47 -07:00
Jason Young
8df4311821 VW MQB: Update to Audi A3 Mk3 (#20489)
* Add A3 and A3 e-tron FPv2

* Update README

* Catch up with model renaming

* Trim duplicate FW values

* Add 2014 A3 (bankster)

* Update model-year range

* Revert more model name changes

* Revert more model name changes

* More FW for 2017 A3 e-tron (fastzeek)

* Update A3 models and packages in README

* Update A3 packages again

* AUDI_A3 -> AUDI_A3_MK3

* Update test route reference

* CAN valid should pass for VW now

* Update process_replay route

* Expand tested A3 range (we have known 2014-15,17)

* Revert "CAN valid should pass for VW now"

This reverts commit 73a4b4886f572045ada05cd13e60790ca3e17a94.

* revert route changes and update refs

* update refs again

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 48fa8ffb33
2021-03-30 14:49:46 -07:00
Adeeb Shihadeh
aa194a3cbb VW: update process replay route (#20239)
* VW: update process replay route

* update refs
old-commit-hash: f3f44e43e6
2021-03-04 14:58:28 -08:00
HaraldSchafer
5092214913 Different lag comp (#20024)
* more sensible

* different names for different segs

* new ref

* this is checked somewhere
old-commit-hash: c3707328f5
2021-02-05 14:01:22 -08:00
HaraldSchafer
5c2c6691b2 remove old model packet (#19769)
* remove publisher

* move to V2

* radard new model

* fix plant

* change packety

* need hack here too

* change to new

* this has been wrong all along

* no more model msg

* subscribe to new model

* not needed anymore

* make work

* need to ignore that too

* should pass tests, needs car test

* fix process replay

* no more poly

* update refs

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 124100d0fa
2021-01-19 17:15:16 -08:00
Adeeb Shihadeh
77817dbd50 update VW and nissan test routes
old-commit-hash: 13ea3ccf6d
2021-01-19 14:22:50 -08:00
HaraldSchafer
61c5e9e690 Update Subaru and GM test routes (#19828)
* 2 more

* update refs

* note?

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: a4098d7bb0
2021-01-18 15:54:02 -08:00
HaraldSchafer
c6d9b9565a Mpc rework2 (#19660)
* start again

* need that too

* this actually works

* not needed

* do properly

* still works

* still works

* still good

* all G without ll

* still works

* all still good

* cleanup building

* cleanup sconscript

* new lane planner

* how on earth is this silent too....

* update

* add rotation radius

* update

* pathplanner first pass

* misc fixes

* fix

* need deep_interp

* local again

* fix

* fix test

* very old

* new replay

* interp properly

* correct length

* another horrible silent bug

* like master

* fix that

* do doubles

* different delay compensation

* make robust to empty msg

* make pass with hack for now

* add some extra

* update ref for increased leg

* test cpu usage on this pr

* tiny bit faster

* purge numpy

* update ref

* not needed

* ready for merge

* try again after recompile

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 158210cde8
2021-01-14 18:43:50 -08:00
HaraldSchafer
3b911d61bc Extra test segs (#19749)
* new routes

* update refs

* new ref
old-commit-hash: 95eb5b6fcb
2021-01-13 21:56:29 -08:00
HaraldSchafer
9c8660d0e2 Test segs 08 (#19701)
* start updating routes

* update that too

* new ref commit

* try again
old-commit-hash: 307ba858db
2021-01-08 16:39:47 -08:00
Adeeb Shihadeh
c64a8356fe move to github container registry + CI speedup (#19621)
old-commit-hash: 59d7c5e64a
2020-12-31 14:57:01 -08:00
Adeeb Shihadeh
6e078827b3 check for safety mode mismatch (#2443)
old-commit-hash: 89d02dcf97
2020-11-04 11:12:15 -08:00
Adeeb Shihadeh
68b4380545 update process replay test routes (#2482)
* update bosch route

* update rfs

* bump refs
old-commit-hash: 3795920750
2020-11-03 18:44:35 -08:00
Adeeb Shihadeh
e94809f5bd add cars with LQR and INDI lat control to process replay (#2363)
* add cars that use LQR and INDI lat control to process replay

* bump ref commit
old-commit-hash: 516da927f9
2020-10-19 21:18:50 -07:00
Adeeb Shihadeh
a58dc92f20 paramsd process replay test (#2118)
* paramsd process replay test

* cleanup
old-commit-hash: 1e5a935ff0
2020-09-01 18:31:26 -07:00