Commit Graph

71 Commits

Author SHA1 Message Date
Adeeb Shihadeh
8d98d8c657 process replay: add Ram route (#25063) 2022-07-07 10:28:55 -07:00
Gijs Koning
edf170103e Process replay: Fix subtest diff (#25054)
Fix subtest diff
2022-07-07 02:14:31 -07:00
Shane Smiskol
ea241bf3dc FW fingerprinting: log all FW versions (#25042)
* get_fw_versions returns all fw versions with request's brand

* keep track of everything received

* debug

* need to regen or write a hack in build_fw_dict

* to be safe, still replace old responses within same brands (hyundai responds to two queries, can fix later)

to be safe, still replace old responses within same brands (hyundai responds to two queries, can fix later)

* update test_fw_query_on_routes

* clean up

* better name

* slightly cleaner

* fix test_startup unit test

del

* fix imports

* fix test_fw_fingerprint

fix test_fw_fingerprint

fix

* fingerprint on all FW_VERSIONS, not just brands with requests

* support old routes in test_fw_query_on_routes

* regen and update refs

* similar function style to before

* better comment

* space

switch name

* try to exact match first

* useless else

* fix debug script

* simpler dictionary

* bump cereal to master
2022-07-06 19:42:58 -07:00
Gijs Koning
c49f997be5 Add laikadOffline subtest to process replay. (#24995)
* Add subtests to process replay.
Adds laikadOffline subtest

* Update cpp.

* Update ref

* Update ref again

* Update ref again

* update ref

* Fix disabling fetching orbits

* Add proc name to event exception

* update ref

* Update setup_env

* Fix offline test and update refs
2022-06-30 08:39:12 -07:00
Shane Smiskol
170ed3d761 process replay: clean up common code (#24855)
* regen and process replay clean up

* test_fuzzy actually uses fingerprint hardcoding

fix

* revert

* revert

* this can be a url or path so just print full variable
2022-06-14 12:03:30 -07:00
Adeeb Shihadeh
0fce5d9045 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
2022-06-11 23:19:27 -07:00
Lukas Petersson
50a8b30580 process replay: upload-only bug (#24722)
dont look for lr with upload only
2022-06-02 17:25:44 -07:00
Shane Smiskol
6a21cc83ff process replay: fix tested_cars 2022-06-02 14:20:19 -07:00
Shane Smiskol
90af436a12 process replay: fix updating refs (#24716)
fix updating refs
2022-06-02 13:49:21 -07:00
Lukas Petersson
86f73a507e process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-06-01 22:02:42 -07:00
ClockeNessMnstr
3fb22c0289 process replay: only download used logs (#24661)
* only download wanted logs

* cleanup

* cache as an option

* cleanup

* Readme

* Revert "cache as an option"

This reverts commit 060ed4ade548696505e3185fc7d20f1897072ded.

* cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-06-01 21:46:45 -07:00
Shane Smiskol
dff70876c3 process replay: fix updating refs after regen (#24656)
* fix updating refs after regen

* match spacing

* same as before

* fix

* minimize diff

* update

* not used
2022-05-26 00:39:33 -07:00
Lukas Petersson
3e975eb3aa process replay: update regen body route (#24648)
update to a preserved route
2022-05-24 17:07:10 -07:00
Lukas Petersson
444d265821 process replay: support running in parallel (#24534)
* prefix params

* set env

* prefix in manager

* filesystem except

* dont delete manager folder

* Update selfdrive/common/params.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug same path

* remove cleanup + same default

* dont use filesystem lib

* param symlink path

* prefix helpers

* path

* dont delete d

* parallel loop

* refactor loop

* msgq

* clean msgs

* spelling

* nestable pool

* spelling

* logreaders in parallel

* bugfix

* assert msgq

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* assert in cereal

* folder exists

* create dirs in test

* bump cereal

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* PR feedback

* params path in basedir

* ref commit

* param path

* bugfix

* upload_only

* param path

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* msgq path name

* python concurrency.features

* progress bar

* remove progress bar from compare logs

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* defaultdict

* context manager

* update refs

* dont get logs if upload only

* upload refs in parallel

* cleanup

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* cleanup

* text

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
2022-05-18 19:59:30 -07:00
Shane Smiskol
d686f9e527 process replay: save logs to folder (#24509)
* save logs to folder for proc replay

* use fakedata dir

* fix

* only once and where needed
2022-05-12 19:19:47 -07:00
Shane Smiskol
934bc1e6fb process replay: don't print diff when only uploading (#24514)
when only uploading no need to print success status or update diff

don't print succeed/fail when only uploading
2022-05-12 17:05:22 -07:00
grekiki
17e33978cd Mypy fixes for --check-untyped-defs (#24372)
more type fixes
2022-05-12 14:08:11 +02:00
Shane Smiskol
d2d3b7b823 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-05-11 15:03:05 -07:00
Shane Smiskol
4190d2608a Change safetyParam to uint16_t (#24376)
* bump panda & cereal

* bump panda

* rest of references

* regen TOYOTA with safety param 578, TOYOTA2 with 329

* regen rest of routes

* update ref

* bump cereal
2022-05-02 13:34:31 -07:00
Shane Smiskol
c1caca104f Support RAV4 Hybrid 2022 with stock longitudinal (#23969)
* add panda flag for toyota stock long with camera

* clean up

* Add 2022 RAV4 Hybrid from Philly

* fix wrong fw in interface, did this ever work?

* Must be a hybrid

* no radar parsing

* fix can error

* move to own platform

* generate docs

* fix

* Add 2022 Rav4 XSE Australia fingerprint parameters (#24303)

* Update values.py

Add 2022 Rav4 XSE Australia

* add commas

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* bump panda

* wait, the camera doesn't even send 0x343, right?

* use a set instead, more obvious

* don't test without a parser

* bump panda

* flip panda flag

* bump panda

* add commas

* regen and update refs

* set to none by default

* revert parenthesis

* update comment

* bump panda

* regen and update refs

* add test models and update readme

* bump to master

Co-authored-by: BrettLynch123 <34538435+BrettLynch123@users.noreply.github.com>
2022-04-28 22:37:46 -07:00
Shane Smiskol
66dd8f934c safety param: make uint16_t (#24362)
* only use half of the available bytes

* move to set_safety_model

* regen and update refs

* last two

* bump to master
2022-04-28 17:56:34 -07:00
Adeeb Shihadeh
3e098c9ba5 Toyota: add safety param flag for alt brake message (#24341)
* Toyota: add safety param flag for alt brake message

* update refs

* update refs

* bump panda
2022-04-27 12:08:11 -07:00
Shane Smiskol
6e62df65d7 Make safetyParam unsigned (#24160)
* bump panda & cereal

* make unsigned in boardd

* update health packet version

bump panda

* fix definition

* bump

* regen TOYOTA2 route

* upload route

* oh, ndog param wasn't being set

* regenerate routes with non-zero safety params

* update refs

* regen routes and update refs

* bump panda

* update refs

* update refs

* bump submodules to master
2022-04-21 00:01:05 -07:00
Shane Smiskol
1a5ddfaedd regen: check if regenerated route is enabled (#24207)
* check if regenerated route is enabled

* elif

* initialize CUDA runner before all other processes

* Revert "initialize CUDA runner before all other processes"

This reverts commit 124e3dd4d3408b6649f40b3e6e7f217ca347f59f.

* add back fixes

* fix typo
2022-04-13 18:01:16 -07:00
Adeeb Shihadeh
6c5b0685da body: set fault state (#24125)
* body: set fault state

* fix that

* update refs
2022-04-04 23:05:40 -07:00
Adeeb Shihadeh
00ebd10b87 body: only control while engaged (#24114)
* body: only control while engaged

* bump panda

* remove that

* bump panda

* update refs

* update refs
2022-04-04 19:37:21 -07:00
Adeeb Shihadeh
e40c36f22b better support non-car robots (#24106)
* better support non-car robots

* bump cereal

* update refs
2022-04-01 20:03:20 -07:00
Igor Biletskyy
031f79ba88 car port: comma body (#24019)
* body FPv2

* ..

* ..

* ..

* Temp, REVERT!

* more cleanup

* typo

* ..

* del eyes

* should work?

* fix

* new dbc

* ..

* fixes

* static analysis

* cln balancing code

* no test route

* excluded_interfaces

* THE DOCS!

* comments on steer/speed mixin

* switch to bus 0

* less UDS

* FAKE bus 0 vin and fingerprint

* FAKE locationd

* Keep steady and remove handcoded offset

* Improve startup sequence, get closer to stock openpilot

* Forgot to define angle

* lowercase

* revert that

* little cleanup

* change safety model

* update refs

* body is gold

* handle no llk

* oops

* litte more

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-03-31 17:16:15 -07:00
Willem Melching
d3d10e014a Ensure controlsd can engage again in process replay (#23556)
* hacks to make process replay engage again

* dont change that

* enable engagement check again

* chmod +x

* first working regen

* proper logMonoTime

* fix video framerate consistency

* mpc is valid again

* proper alive checks

* revert loggerd change

* ensure engaged in regen, fix managerState, peripheralState

* ubloxRaw is unused

* add submaster config for gm

* regen all services we can run without HW

* fix loggerd

* loggerd: matroska without the extension

* update ref
2022-01-21 13:13:34 +01:00
Ewout ter Hoeven
332f568a82 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven
55390d273f Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh
2252c6bba7 Mazda: move CX-9 out of dashcam mode (#22717)
* move cx9 out of dashcam

* no alc below 28

* process replay route
2021-10-31 22:02:36 -07:00
Robbe Derks
91987f38d4 Make pandaState and safetyMode a list (#22454)
* wip: move to pandaStates

* bump cereal

* wip: SafetyMode struct

* move to safetyMode

* fix typo

* this can be None

* fix potential empty pandaStates list

* fix thermald

* fix controlsd

* rename safetyModes to safetyConfigs

* update process_replay

* fix test_models

* bump cereal
2021-10-08 17:54:34 +02:00
Adeeb Shihadeh
86f68ff22e add TSS2 replay route (#21953)
* add TSS2 replay route

* skip on sim

* do not set on import time

* update ref

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-08-31 10:48:21 -07:00
Adeeb Shihadeh
f7dc898853 update prius test route (#21791) 2021-07-29 20:04:07 -07:00
Harald Schafer
1979005e82 update 2021-07-09 16:31:34 -07:00
HaraldSchafer
6838e1c82c Refactor lateral lag compensation (#21334)
* add T_IDXS

* refactor

* fix test

* unused

* typo

* needs casting

* Update selfdrive/controls/lib/drive_helpers.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* deprecate field

* regen all

* new segs

* add todo

* split back

* clean

* bad names

* do in controls

* add arg

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-06-30 14:19:39 -07:00
Adeeb Shihadeh
4a1aec27ef segment regen (#21148)
* start seg regen

* send vipc

* fix up some paths

* set fp

* fix no entries

* engages

* seperate camera procs

* send real frames

* regen test routes

* nice argparser

* fix valgrind test

* move that

* fix that
2021-06-28 21:44:40 -07:00
robbederks
cdfba105ed Tesla AP2 port (#20074)
* squashed the PR

* remove AP passthrough

* disengage on steer override

* bump panda

* add test routes

* bump panda

* tesla is still dashcam only

* dashcam only

* rerun CI

* add to release files
2021-06-23 14:56:59 +02:00
Adeeb Shihadeh
7d8480cffe Update VW process replay route (#21142) 2021-06-04 15:29:13 -07:00
Adeeb Shihadeh
c45dbbc737 fix process replay on local files (#20758) 2021-04-26 20:50:47 -07:00
Jason Young
48fa8ffb33 VW MQB: Update to Audi A3 Mk3 (#20489)
* Add A3 and A3 e-tron FPv2

* Update README

* Catch up with model renaming

* Trim duplicate FW values

* Add 2014 A3 (bankster)

* Update model-year range

* Revert more model name changes

* Revert more model name changes

* More FW for 2017 A3 e-tron (fastzeek)

* Update A3 models and packages in README

* Update A3 packages again

* AUDI_A3 -> AUDI_A3_MK3

* Update test route reference

* CAN valid should pass for VW now

* Update process_replay route

* Expand tested A3 range (we have known 2014-15,17)

* Revert "CAN valid should pass for VW now"

This reverts commit 73a4b4886f572045ada05cd13e60790ca3e17a94.

* revert route changes and update refs

* update refs again

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-03-30 14:49:46 -07:00
Adeeb Shihadeh
f3f44e43e6 VW: update process replay route (#20239)
* VW: update process replay route

* update refs
2021-03-04 14:58:28 -08:00
HaraldSchafer
c3707328f5 Different lag comp (#20024)
* more sensible

* different names for different segs

* new ref

* this is checked somewhere
2021-02-05 14:01:22 -08:00
HaraldSchafer
124100d0fa remove old model packet (#19769)
* remove publisher

* move to V2

* radard new model

* fix plant

* change packety

* need hack here too

* change to new

* this has been wrong all along

* no more model msg

* subscribe to new model

* not needed anymore

* make work

* need to ignore that too

* should pass tests, needs car test

* fix process replay

* no more poly

* update refs

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-01-19 17:15:16 -08:00
Adeeb Shihadeh
13ea3ccf6d update VW and nissan test routes 2021-01-19 14:22:50 -08:00
HaraldSchafer
a4098d7bb0 Update Subaru and GM test routes (#19828)
* 2 more

* update refs

* note?

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-01-18 15:54:02 -08:00
HaraldSchafer
158210cde8 Mpc rework2 (#19660)
* start again

* need that too

* this actually works

* not needed

* do properly

* still works

* still works

* still good

* all G without ll

* still works

* all still good

* cleanup building

* cleanup sconscript

* new lane planner

* how on earth is this silent too....

* update

* add rotation radius

* update

* pathplanner first pass

* misc fixes

* fix

* need deep_interp

* local again

* fix

* fix test

* very old

* new replay

* interp properly

* correct length

* another horrible silent bug

* like master

* fix that

* do doubles

* different delay compensation

* make robust to empty msg

* make pass with hack for now

* add some extra

* update ref for increased leg

* test cpu usage on this pr

* tiny bit faster

* purge numpy

* update ref

* not needed

* ready for merge

* try again after recompile

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-01-14 18:43:50 -08:00
HaraldSchafer
95eb5b6fcb Extra test segs (#19749)
* new routes

* update refs

* new ref
2021-01-13 21:56:29 -08:00
HaraldSchafer
307ba858db Test segs 08 (#19701)
* start updating routes

* update that too

* new ref commit

* try again
2021-01-08 16:39:47 -08:00