Commit Graph

114 Commits

Author SHA1 Message Date
Shane Smiskol
6acf763db4 Volkswagen: fingerprint on VIN chassis code (#32148)
* add function signature and behavior comment

* add test

* move chassis codes to platform config!

* add a shared chassis code test

* function

* test matching

* this commit isn't complete yet

* Revert "this commit isn't complete yet"

This reverts commit ae77d5cd54e1f43d390fb70c4da38a95ac34f8da.

* need to check WMI

* TODO: test WMI

* test wmi

* radar FW sanity check

* fix test

* fixes from merge

fixes from merge

* whoops

* fix static analysis!

* do match_fw_to_car

match_fw_to_car takes vin

* makes sense to keep it one function, and we can return exact or fuzzy!

* clean up

* kinda pointless

* fix more tests

* back to function being only fuzzy

* revert test_fw_fingerprint

* revert test_fw_fingerprint

* simplify

* clean up/fixes

* rename test

* less duplicatey WMI descriptions

* fix

* convert to enum

* I am confident about these WMIs

* these are also good

* we support 5N AUS/NZ and NAR (North American) AX Tiguans

fixes

* Tiguan also Mexico

* only one user for caddy

* got from the test route

* check that the gateway type matches the platform (each platform has 1 or 2 types)

* ~gateway~ -> exact FW match

* remove re

* ensure WMIs are set

* actually no reason to delete

* move comment up to the platform config

* proper wmis typing

* spacing

* flip
2024-04-12 22:00:03 -07:00
Shane Smiskol
16a473b3aa debug: set OBD multiplexing mode for VIN (#32130) 2024-04-08 15:42:31 -07:00
Shane Smiskol
72eb17012d fix static analysis (#32082)
fix ltl
2024-04-01 20:08:22 -07:00
Shane Smiskol
56e716bf86 fw_versions: add OBD multiplexing to debugging view 2024-04-01 19:08:27 -07:00
Cameron Clough
fb81cfe3c4 fw_versions: add more type hints (#31577)
* fw_versions: add more type hints

* cleanup

* fixes

* more implicit optional

* Update selfdrive/car/fw_versions.py

* backslash is unnecessary inside parenthesis

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2024-03-05 15:33:39 -08:00
Cameron Clough
80da3aee14 mypy: use implicit-optional (#31590)
* mypy: set implicit-optional = true

* find and replace '| None = None' -> '= None' in function args
2024-02-25 13:29:18 -08:00
Cameron Clough
995250ae49 use pyupgrade to update to new typing syntax (#31580)
* add pyupgrade hook

* run pyupgrade (pre-commit run -a)

* ruff --fix

* Revert "add pyupgrade hook"

This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.

* revert changes to third_party/

* manual type fixes

* explicit Optional wrapping capnp objects
2024-02-24 16:41:23 -08:00
Shane Smiskol
74ec33f7c7 IsoTpParallelQuery: type hinting (#31370)
* start typing

* fixes

* we only plan on using this for lists

* sort

* this is the correct way to type it

* it knows this!

* it's getting smarter
2024-02-07 22:53:55 -08:00
Shane Smiskol
c9bd4e4c0d FW_QUERY_CONFIGS: type annotate (#31265)
* annotate

* fix

* clean up

* test

* clean up

* space

* fix
2024-02-01 21:01:31 -08:00
Shane Smiskol
341f8420db fw_versions: type annotate test_brand_ecu_matches (#31272)
* test this

* we
2024-02-01 15:11:10 -08:00
Shane Smiskol
2c0f7b8727 FPv2: support FwQueryConfig with no FW versions (#31227)
* bump

* from https://github.com/commaai/openpilot/pull/27929

* get VIN on bolt!

* might as well try on other gms

* remove vin

* ugh gm is going to be slow

* fix

* should really fix this

* revert

* happy?1

* fix unit test

* bump

* functional addressing must be an OBD gateway feature, this does nothing

* fix vin response

* fix addr!

* finally fix fw_versions bugs since boardd IsOnroad refactor

* for

* only bus 0

* clean up

* Update selfdrive/car/gm/values.py

* ChatGPT re-write

* filter out did

* todo

* oof

* preview: what multiple DIDs per ECU would look like in the future

* Revert "preview: what multiple DIDs per ECU would look like in the future"

This reverts commit 88f0d8611e638de644adc5feabade848c03d59e4.

* function to get all ecus

* we can remove this!

* can also do this!

* and this one too :o

* consistency

* yay

* add tests

* revert GM stuff

* another PR

* reads better

* revert rest of gm

* use that

* interesting

* these are exactly the same (with ordering differences)

* Revert "these are exactly the same (with ordering differences)"

This reverts commit a9e918dc351d24f1b7c8a05c7ec402e057a0e09a.

* flip
2024-01-30 02:08:08 -08:00
Shane Smiskol
c3fcf75737 GM camera ACC: get VIN from camera (#31224)
* bump

* gm vin

* that's not right

* only check relevant buses

* instead try queries for each bus first

* clean up

* use default retry in function

* all
2024-01-29 23:24:21 -08:00
Shane Smiskol
5a8686b439 fw_versions: fix params.remove 2024-01-29 22:36:29 -08:00
Shane Smiskol
a4cf7745e8 fix manually running fw_versions.py (#31226) 2024-01-29 21:44:00 -08:00
Shane Smiskol
289868297e VIN: query on bus 0 to log data (#31165)
* query on bus 0

* fix

* log vin_rx_bus

* fix!
2024-01-25 18:01:49 -08:00
Shane Smiskol
780f1222ad fw_versions: add extra_fw_versions param (#31132)
* add param

* fix

* fix
2024-01-23 17:21:48 -08:00
Shane Smiskol
29536324c1 Toyota: combine more hybrid platforms (#31057)
* Combine

* migrate test models segs

* fix highlander engine

* no change
2024-01-18 18:51:24 -08:00
Justin Newberry
28eed156e0 Move all fingerprints to fingerprints.py (#30790)
* move fingerpints

* fix circular imports

* fix circular imports

* fix auto fingerprint

* fix those tests

* and controlsd

* add labeler rule

* no defaultdict
2023-12-18 14:04:27 -08:00
Adeeb Shihadeh
35f819c823 swaglog.py goes in common/ (#30631)
* swaglog.py goes in common/

* all of these go in release

* we'll even include the pyx
2023-12-06 17:27:51 -08:00
Shane Smiskol
365bdd3422 Revert "FW query: query one sub-address in parallel" (#29550)
Revert "FW query: query one sub-address in parallel (#28675)"

This reverts commit 861ceb2fee.
2023-08-23 01:41:28 -07:00
Shane Smiskol
861ceb2fee FW query: query one sub-address in parallel (#28675)
* put one subaddr in parallel_addrs

* p

* remove print

* update test refs

* test

* revert test changes

* do same in present ECU query, lower threshold to catch (~0.82 to ~0.72 right within bounds)

* type

* clean up

* update test ref
2023-08-23 01:31:04 -07:00
Adeeb Shihadeh
a9626f95b6 add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
2023-08-20 20:49:55 -07:00
Shane Smiskol
f5a361a111 FPv2: fingerprint on current brand's candidates (#28435)
* pass in brand

* needs to be flipped for exact matching since it's subtractive

* new helper

* makes more sense flipped

* clean up
2023-08-11 18:47:12 -07:00
Shane Smiskol
09ce869ee7 fw_versions: clean up some typing (#29350)
* another common type

* here too

* unused
2023-08-11 18:37:17 -07:00
Justin Newberry
4a9afd7554 Ruff: comprehensions conventions (#29317)
ignore c408
2023-08-10 16:40:30 -07:00
Shane Smiskol
082fe6868d FPv2: fast fuzzy fingerprinting (#29233)
* fast fuzzy fingerprinting

* use match_fw_to_car helper

* fix and comment

* alone

* Update selfdrive/car/fw_versions.py

* Update selfdrive/car/fw_versions.py
2023-08-04 16:10:49 -07:00
Shane Smiskol
56614124f0 FPv2: don't add debug candidates to versions globally (#28562)
* debugging

* this test works, but find a better place

* make test more generic to start testing more than timings

no need to run again in another test function. eventually we
can simulate ECUs and have IsoTpParallelQuery work properly

* Revert "make test more generic to start testing more than timings"

This reverts commit 58e8536b1d8b138902d373e1e1194e70c9eb4b24.

* 2nd try

* clean up

* not even MappingProxyType works

* works

* test that too

* clean up

* refactor to avoid modification online

* revert tests

* more revert

* no longer needed

* clean up
2023-06-15 16:32:32 -07:00
Shane Smiskol
f788edb6a5 HKG: use platform codes to fuzzy fingerprint (#28531)
* get gas/ev/hev from FW (not all correct, poc)

* add test for essential ecus for fuzzy fingerprinting

* kinda works

* stash

* clean up

* add code

* simpler

* use the function

* test it with our cars

* no re

no re

no re

* debugging

* handle empty dict

* simpl

* this is promising

start on making existing fingerprinting functions use the config, instead of entirely replacing them

* needs to allow 1 match

* lay out how this should look

* changes

* executable

* some work

* use config

* fuzzy ecus

* config test

* comment and some clean up

* test platform codes

* use regex, simpler and fixes bug

* in func

* rm bad func

* typing for new func and remove old from dc

* todo done

* tested!

* remove fake platform codes

* thought we needed this, but actually...

* not needed

* not applicable any more

* use config for essential ecus

* first draft of test to make adding/removing fuzzy FP platform intentional

* compile

* clean up test

* even cleaner

* fix default ecus type

* temp fix

* this is mostly in tests now

* test every fuzzy ecu fw returns one platform code

* experiment with dates

* Revert "experiment with dates"

This reverts commit 3251b9cc5c3ca41ca92c8b75ad9b2234b720aa0b.

* clean that up

* comment

* test

* work on all cars

* fix fuzz_fw_fingerprint

* comment

* get first by search

* bit more clean up

* and more

* use compiled pattern for nicer syntax

* default

* flip dat around, much cleaner

* clean up hyundai test a bit

* flip order

same here

* rename test and flip subTest

* fix pylint

* revert fw changes

revert fw changes

* line

* add original functions to test

* needs to be a list

* cmt

* draft (need to count one ecu as a match)

* tiny clean up

* todo: date range

* only in notebook

* remove comment (still can be either list or set)

* same, only notebook

* more consistent signature

* copilot inspired

* copilot no good

* test for date parsing

* better name

* good, now we don't have to worry about the dates mismatching in another test/logic

* comment up+

* some stuff

* clean up

fix test

fix test

* test

* comment

* use utils

* clean up (utils are cleaner and less buggy)

* clean up (utils are cleaner and less buggy)

* fixup test

* use a dash (prettier) and remove some platforms that can fingerprint now!

* compile global pattern

* same as what we do in values

* remove comments

* fuzzy_get_platform_codes is one or none here

* more clean up

* sort imports

* woah woah woah

* add comment

* fix reassigning different types

* add types

* adapt fuzzy test recently added (nice it caught this!)

* update lock

* options

comments

* stash

* comments and fixes

* better comments

* better

* test: run on exact fuzzy matching logic, same results!

* use match_fw_to_car

* test all fw

* ex

* unused random

* this is a possibility

* this is more clear

* fix

* revert

* revert to needing both ECUs to match to reduce false positives, test

* fix excluded platform test :( but it's ok

* add comment

* we actually want to only test fuzzy ecus to make it explicit

* fix mypy

* comment for tomorrow

* just add matches with fuzzy FP

* add comment

* this was the cleanest I could think of, but still working on it. not very easy to understand

* think this is better, but also worse...

* comment: reframing how this works

* revert back to what we did before

* was swapped

* else set

* remove old comment

* fixes from merge

* remove fuzzy_min_match_count from this pr

* fix static analysis

* also unused

* different method first draft

* copy existing fuzzy func

* check all possible ecus exist, only platform codes, slightly refactor main loop

* fix

* Revert recent

Revert "fix"

This reverts commit 5cdb7bda835f1e48e750ab4195e891afe08e11ea.

Revert "check all possible ecus exist, only platform codes, slightly refactor main loop"

This reverts commit d3e918fa20fa4ce881445850f5f7428a3c11adf8.

Revert "copy existing fuzzy func"

This reverts commit 34c8c0545097c84f55f4b4f61907606c93760ddd.

Revert "different method first draft"

This reverts commit b91139055d7d1802c1eb726504798c156a183c9c.

* new func

* fixup test

* remove changes from v1 from fw_versions.py

* clean up a bit

* return part as part of code

* fix test

* add original fuzzy function

* add an ecu match if the date is within range (or date doesn't exist)

* add format for what we're going to do

* not working stash

* the exact matching function does more of what we want with less code and less custom logic

* we don't care about found versions, only codes and dates

* actually we do have an exception

* this works pretty nicely now

* up here

* this is better

* some minor clean up

* old function=now junk

* fix platform code test

* remove old platform code function

* now rename _new to

* use FW_QUERY_CONFIG

* clean up imports

* rename that too

* one line

* correct typing

correct typing

* draft tests

* so that works

* fixup excluded platform test now too

* this is tested by excluded platform test

* test parts and dates

* remove old comment

* old import

* take platform code stuff out of FwQueryConfig

* fix test

* revert debug script

* flip order

* make this a set by default

* revert this part

* correct typing

* clean up comments

* clean that test up too/pylint

* combine these three tests ina clean way

* not right

* more general

* be consistent with quotes

* comment

* comment

* comment in fw_versions

* flip order

* this is more readable

* could test all this, but it's tested in test_hyundai and doesn't do a lot here

* only assert brands which use this

* invalidate all CAN FD ICE and hybrid

* tuple

* can get away without filtering

* add comment reasons

* fix

* some review suggestions

* this works (first draft)

* this is better

* script to print platform codes and dates

* sanity check for dates are in correct ecus and platforms

* mypy

* better variable name and comment

* rename

* same

* slightly better name

* subset

* exclude platforms and live car without dates

* consistent

* self explan

* better name

* test to make sure the functions agree

* clean that up

* comment

* we get other responses from queries not in DB, only check any

* not used or typed
2023-06-15 02:50:45 -07:00
Shane Smiskol
e771660fa3 FPv2: rename live FW versions dict (#28550)
rename
2023-06-15 01:29:55 -07:00
Shane Smiskol
fcc268b8be exact matching: use set for invalid (#28537)
use set
2023-06-13 20:00:54 -07:00
Shane Smiskol
3234243c97 FPv2: add comment about nuance of fuzzy fingerprinting (#28465)
add comment describing this nuance of fuzzy fingerprint function
2023-06-08 20:47:55 -07:00
Shane Smiskol
9e8936be98 fuzzy fingerprint: track number of matching ECUs, not FW versions (#28423)
* first attempt trying to do it implicitly bad

* back to lists and explicitly do this

* continue checking if candidate switches on these, that's fine to check

* debugging

* clean up

* more clean up

* spot free

* only need to create/update on change

* fix test

* draft

* Revert "draft"

This reverts commit dbe32f58b172eddc8f9699044c6a5956e753305d.

* Revert "fix test"

This reverts commit 1d34269fe2de35d314aa0658fa47a78a08d93e08.

* fix test for real

* first draft of test

* this should be good

* can replace existing fuzzy test

* rm

* matching

* test non-empty fws

* TEST DOWN HERE

* no test

* unique + fix skip message + comment + clean up

* less confusing

* ing

* better
2023-06-07 00:03:01 -07:00
Shane Smiskol
0acc30d264 FPv2: test minimum fuzzy fingerprint ECUs (#28433)
* basic messy test

* draft

* clean up

* clean up

* clean up

* clean up

* never was fuzzy, but be explicit

* switch it up
2023-06-06 21:38:10 -07:00
Shane Smiskol
2b07ec4bde FPv2: brand filter helper (#28439)
* pass in brand

* needs to be flipped for exact matching since it's subtractive

* new helper

* makes more sense flipped

* clean up

* not that stuff here

* not here
2023-06-06 21:14:03 -07:00
Shane Smiskol
2dda9a0532 match_fw_to_car: pass down log (#28434)
pass in log from match_fw_to_car
2023-06-06 20:28:56 -07:00
Shane Smiskol
7c26fc716a match_fw_to_car_exact: get config once per candidate (#28384)
lesss
2023-06-04 18:57:11 -07:00
Shane Smiskol
23430dd35f fw_versions: add some typing (#28369)
* typing

* sort

* brand addrs

* don't type match_fw_to_car_fuzzy

* or this unfortunately

* no space

* add back

* and this
2023-06-02 13:02:04 -07:00
Shane Smiskol
9b1095a27e FPv2: only query brands with matching present ECUs (#27697)
* speed up fingerprinting for CAN fp cars

* add comment to describe this behavior

* more explicit, no defaultdict

* do this later

* fix

* a little more clear I think
2023-04-18 21:34:49 -07:00
Shane Smiskol
6f35d23c06 FPv2: match brand against extra ecus (#27688)
* query extra ecus (still need to match)

* match against extra ecus

* clean p

* clean p

* switch that around

* can clean that up

* values
2023-03-25 01:07:58 -07:00
Shane Smiskol
084e83cda1 FPv2: replace temporary variable with config (#27692)
* query extra ecus (still need to match)

* match against extra ecus

* clean p

* clean p

* switch that around

* can clean that up

* values

* rm

* not used
2023-03-25 01:02:11 -07:00
Shane Smiskol
9e023b3ead FPv2: only set multiplexing when needed (#27669)
* only set multiplexing if bus matters

* these don't matter anymore
2023-03-24 00:29:48 -07:00
Shane Smiskol
2ad9eec882 FPv2: log OBD multiplexed requests (#27667)
* log multiplexed requests

* bump cereal

* bump cereal to master
2023-03-24 00:25:37 -07:00
Shane Smiskol
42449b482d boardd: ability to switch between ELM safety params (#27656)
* indecisive

* rename to generic FW query

* remove code and update comment

* we need this to start off, unless we set multiplexing immediately

* draft

* draft 2

* try that

* can't do this either, boardd might read Enabled after removing, but before setting new Request param

* this should work

* use one less param

* fix params

* match behavior (set all pandas to safety param of 1, disabling multiplexing for fingerprinting

* clean up (some tests may temp break)

* fix param name and sort

* time it

* yes it does matter

* add to hyundai's bus 5 query

* remove hyundai for now

* this should work

* clean up

* clean up

* flip argument around, clean up

* fix test_startup

* some clean up

* rm line

* comment makes more sense

* required typing

* clean up common type

* comments

* Update selfdrive/car/car_helpers.py

* line

* whoops, need to set before vin!

* fix debug

* annoying

* more debugging

* bug fix (needs both keys always)

* debuGG

debuGG

* Revert "debuGG"

This reverts commit 55b2f429324c0b92d5cfb2cabf8b20db1e166248.

* Revert "more debugging"

This reverts commit 02934c3403ad5270f03093508b704c151d1ccb2a.

* Revert "annoying"

This reverts commit 8b4e5e09989f9a0217e3ec1c0ba68735929b7366.

* clean that up

* bumpback

* bumpback

* every second write param

* flip

* stuff

* move up?

* fix timing out in CI

* rm
2023-03-23 00:14:31 -07:00
Shane Smiskol
727a94cc78 fw_versions: cleanup common types (#27658)
* use common type

* use it in fw_versions

* add typing to queries
2023-03-22 21:58:04 -07:00
Shane Smiskol
5dde8f2c84 Write boardd param to continue startup (#27354)
disable at the bottom
2023-02-15 15:52:10 -08:00
Shane Smiskol
c4b84783a0 Honda Bosch: gather available ECUs from camera (#27180)
* add flag

* actually use in fw_versions.py

* simpler

* not here

* Comment

* better name

* add param and block

* keys

keys

* block

* with a value

with a value

* add query for bosch PT bus

* different name

* fix

* .

* fix test

fix test

* add cloulogs

* mark as logging
2023-02-14 17:55:41 -08:00
Shane Smiskol
f29e19cf42 FPv2: log responses from data collection queries (#27345)
* log responses from logging/debugging queries

* don't add logging FW to FW dict for fingerprinting

* flip?

* fine before

* log if extra in debug ecus

* clean up

* test extra ecus too

* Revert "test extra ecus too"

This reverts commit 8f7867844db95f48631348551551148bf504e37b.

* bump to master
2023-02-14 16:59:20 -08:00
Shane Smiskol
2a6de71cc0 get_present_ecus: check available pandas (#27181)
* add num_pandas to get_present_ecus

* always bugged me

* move this to function signature
2023-02-01 18:38:00 -08:00
Shane Smiskol
7bf70bf7d8 CAN-FD HKG: query FW versions from camera (#26063)
* add adas essential ecus

* add adas ecu and query

* add queries

* add name

* after

* presence of adas ecu

* Revert "presence of adas ecu" (POC)

This reverts commit ab88a7e7df32e1c02a175b81848bd4112b2e5c69.

* no whitelist for debugging

* Apply suggestions from code review

* add adas response

* remove adas version

* temp

* read pandaStates

* works in debug script

* only fwdCamera on tucson

* fix pandaStates reading

* fix test_startup

* fix

* simpler

* use existing socket

* pass in number of pandas

* need to create sm using outcome of fingerprinting, which uses sm

fix

* move default argument

* use sock

* always ignore

always ignore

* add canfd fingerprint test

* Update selfdrive/car/hyundai/tests/test_hyundai.py

* set
2022-10-17 23:04:06 -07:00
Shane Smiskol
7e9961b9ac FPv2: support collecting versions for specific ecus (#25699)
* Add VMCU address for EV6

* Rename vmcu

* add to tests

add to tests

* rename to more generic name

* more explicit

* remove print

* Like this much better, removes subtle fingerprinting problems

* clean up

* add test and clean up

* remove hyundai stuffs

* global

* Fpv2Config class

* fix missing fw versions from import order

* unused

* revert for now

* test for fpv2 configs with subtests

* subtests don't work that way

* remove this

* .

* intersection

* print ecus

* shorter

* fix typing

* use config
2022-10-13 17:40:07 -07:00