Commit Graph

148 Commits

Author SHA1 Message Date
Justin Newberry
c03fe6bb13 add example of using the comma car segments database (#31117)
* add vin fingerprinting example

* perhaps!

* remove segment

* easier
old-commit-hash: 389b8ca30d
2024-01-22 18:23:07 -08:00
Justin Newberry
f447062b66 CommaCarSegments: public database of segments for each platform (#31114)
* comma car segments

* comma car segments for test_models

* oneliner

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* better name

* not used here

* sort

* remove print

* better comment

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: caa9153974
2024-01-22 17:33:03 -08:00
Justin Newberry
b29eea47ce sanitizer: sanitize vin number (#31090)
old-commit-hash: 2c86c023fe
2024-01-21 12:22:32 -05:00
Justin Newberry
3d0ae1aaaa prevent typing issue in azure_container (#31044)
prvent that issue
old-commit-hash: 840db1f1b7
2024-01-17 15:34:24 -08:00
Justin Newberry
09b95b715c cleanup azure handling (#31034)
* wip cleanup

* Wip

* fixes

* fix

* cleanup

* keep this for now

* dest
old-commit-hash: 3f0b56b364
2024-01-17 14:21:47 -08:00
Justin Newberry
a9676e7086 auto_source: optimize api calls and use head to determine if file exists (#31025)
* fast

* catch all

* source

* fix file_exists

* remove duplicate reset

* test multiple loops

* iterations

* cleanup imports
old-commit-hash: 2967cada71
2024-01-16 17:24:37 -08:00
Justin Newberry
5df01b3d22 LogReader: fix plotjuggler not reading properly (#31023)
* fix plotjuggler

* simpler

* fix juggler

* fix that
old-commit-hash: 61fe705160
2024-01-16 12:40:49 -08:00
Justin Newberry
4b7edec2b0 LogReader: move opci to tools/lib (#31021)
move opci
old-commit-hash: 7d400112df
2024-01-16 10:58:51 -08:00
Justin Newberry
35dcd7ccd8 LogReader: support lists of identifiers (#31020)
* support lists of logs

* mark as slow instead
old-commit-hash: 0739d5d7b2
2024-01-16 10:40:00 -08:00
Justin Newberry
13e7366b6d LogReader: support only_union_types (#31019)
support only-union-types
old-commit-hash: 4da8d16514
2024-01-16 10:03:55 -08:00
Adeeb Shihadeh
96e9a7d3ce CI: speedup unit test job (#30995)
* start

* mv to buildjet

* fix
old-commit-hash: e17e66180c
2024-01-13 21:16:35 -08:00
Justin Newberry
c8cc30112d rename segmentrangereader to logreader (#30981)
* Replace multilogiterator

* replace logreader

* update readme

* fix from_Bytes

* new section

* reset before iter

* selector

* fix internal

* whitespace
old-commit-hash: 3d73512c07
2024-01-12 12:53:50 -08:00
Justin Newberry
a729e102c7 Replace multilogiterator (#30980)
old-commit-hash: 9f1b72ac79
2024-01-12 11:43:32 -08:00
Justin Newberry
1b65d5cd85 segmentrangereader: support direct parsing (#30973)
* use correct source

* revert

* cleanup imports

* clean

* direct parsing

* rename

* move up

* fixes

* fix that

* better error message
old-commit-hash: eb09294fc2
2024-01-11 16:56:48 -08:00
Justin Newberry
11449845e7 segmentrangereader: support more sources (#30970)
* support more sources

* remove this
old-commit-hash: fb994ae782
2024-01-11 12:55:19 -08:00
Justin Newberry
3b86243f1f SegmentRangeReader: log type selector (#30960)
* log type selector

* test this too

* and this

* invalid selector

* use strenum

* don't hardcode size

* oneline
old-commit-hash: be814ed447
2024-01-11 10:25:24 -08:00
Justin Newberry
9e6c6d92b0 segmentrangereader: support sort_by_time (#30954)
support sort-by-time
old-commit-hash: cfb23eb2d1
2024-01-10 13:36:55 -08:00
Justin Newberry
1f434b2714 SegmentRangeReader: new format for reading multiple segments (#30940)
* segment range reader

* rename that

* revert that

* cleanup

* revert this for now

* revert this for now

* Fix + test

* rm that

* rm that

* use for auto_fingerprint

* simpler

* for notebook too

* match numpy indexing

* just use numpy directly

* remove that

* spacing

* spacing

* use qlog for auto fingerprint

* add 'read mode'

* pass in read mode

* add test for modes

* numpy indexing

* fix that case

* more examples

* fix the notebook

* cleanup the notebook

* cleaner

* fix those
old-commit-hash: 0d126e1e9e
2024-01-10 11:55:19 -08:00
Justin Newberry
7aecd2f91d url_file: fix non-200 files being cached (#30886)
* fix + test

* fix unclosed

* easier to read

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* fix that

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: fba521ecc6
2024-01-02 12:42:08 -08:00
Adeeb Shihadeh
207073dc4c fix unused
old-commit-hash: 51a93cf9aa
2023-12-17 16:32:56 -08:00
royjr
d41f215df8 dependency: remove pycurl package (#30771)
* Update qcomgpsd.py

* Update url_file.py

* remove pycurl

* requests -> urllib3

* unused

* redundant

* fix import

* Revert "requests -> urllib3"

This reverts commit 7ca39e618917986f50356519d2795a432e3a5280.

* headless

* fix trail

* use requests.exceptions.RequestException

* use fp.tell

* fix indents

* reorder imports

* change timeout

* fix debug timing

* remove exception

* add timeout

* missing headers

* move to constructor

* move import

* unused import

* fix debug

* try

* no retries
old-commit-hash: 70624ffc81
2023-12-17 16:25:25 -08:00
Greg Hogan
e91032efb3 replace common.file_helpers.mkdirs_exists_ok with python os.makedirs funtion (#30618)
replace common.file_helpers.mkdirs_exists_ok with python os.makedirs function
old-commit-hash: db35dcd0b5
2023-12-06 09:55:29 -08:00
Justin Newberry
661132be5a Pytest: more post-test path cleanup (#30556)
* test cleanup

* missed this one

* remove that

* static class
old-commit-hash: 4086795c29
2023-11-30 09:55:48 -08:00
Kacper Rączy
6f51ee8267 URLFile: exception type for failed requests (#30330)
URLFileException for URLFile request errors
old-commit-hash: c27e977475
2023-10-25 16:20:12 -07:00
Greg Hogan
a544533476 framereader: eliminate tempfile usage (#30289)
old-commit-hash: 68acb26aa2
2023-10-18 19:48:26 -07:00
Greg Hogan
99d094a006 fix ffprobe on device (#30286)
* fix ffprobe on device

* use FileReader

* no more requests
old-commit-hash: 59e947dc6c
2023-10-18 18:31:14 -07:00
Justin Newberry
ef1acd4e6c Proc Replay: cleanup iterable log typing (#30261)
* cleanup log typing

* use iterable

* fix those

* Cleanup

* cleanup

* missed that one
old-commit-hash: 1820b5f259
2023-10-17 12:58:28 -07:00
Greg Hogan
a62749f527 vidindex improvements (#30196)
* vidindex improvements

* fix spelling
old-commit-hash: 6f98a987af
2023-10-05 22:48:50 -07:00
Greg Hogan
e0469bdde6 filereader cleanup (#30191)
* filereader cleanup

* make name return url

* remove unused imports
old-commit-hash: 86b90a8ba2
2023-10-05 19:45:46 -07:00
Greg Hogan
29bb2cf867 convert vidindex to python (#30176)
* convert vidindex to python

* fix whitespace

* corrupt file option

* fix up typings

* fix return type

* update framereader

* change length delimiter to uint32 value

* change length to uint32 value

* move url_file changes to separate PR

* cleanup caching

* revert whitespace change

* fix frame type param type
old-commit-hash: f8e488f881
2023-10-05 18:40:24 -07:00
Justin Newberry
a9153302a3 URLFile: don't cache non-existent file's lengths (#30071)
* urlfile empty

* simplify with mock

* better test name

* PR cleanup

* cleanup the length file
old-commit-hash: fa51bbc236
2023-10-04 12:57:56 -07:00
YassineYousfi
eb7cad40dd framereader: add cache_dir argument (#29904)
* framereader: add cache_dir argument

* make it an env var
old-commit-hash: 880157f5a6
2023-09-14 14:36:32 -07:00
Greg Hogan
b5375cc82d importing auth_config.py should not have side effects (#29893)
import should not create directory
old-commit-hash: 140a4ba558
2023-09-12 18:00:18 +00:00
Justin Newberry
dc6078a6c3 System: use paths for download folders too (#29818)
* use paths for download folders too

* trailing slash

* reset that
old-commit-hash: 73eda51a11
2023-09-07 12:49:03 -07:00
Justin Newberry
3b91610c0b Precommit: ensure executable bit is set (#29784)
* precommit ensure executable

* exclude tinygrad

* bump submodules

* exclude tinygrad globally
old-commit-hash: 4e69937d0d
2023-09-05 16:33:26 -07:00
Justin Newberry
1cc555c37b Tests: expand temporary directory helpers for more directories (#29735)
* create helpers for testing

* document it

* fix other tests
old-commit-hash: 405d09714c
2023-08-31 14:00:31 -07:00
Justin Newberry
279bd020dc Tests: logmessaged reduce global state (#29680)
* reduce global state

* use a decorator here too

* use that one

* use base imports

* ipchandler in setup

* move to common dir

* move to helpers

* wip

* fix the decorator
old-commit-hash: 5dc7028f91
2023-08-29 13:36:26 -07:00
Justin Newberry
b08c5d7517 Tests: test_caching reduce global state (#29681)
old-commit-hash: dd9b0231c6
2023-08-29 10:06:32 -07:00
Justin Newberry
c4f322ce2d Linter: remove pylint (#29611)
remove pylint
old-commit-hash: 1ee6ed4b16
2023-08-24 16:30:54 -07:00
Dean Lee
0984736972 cpplint: add filter build/include_subdir (#29585)
old-commit-hash: 21da5a216f
2023-08-24 15:42:17 +01:00
Adeeb Shihadeh
16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b6
2023-08-20 20:49:55 -07:00
Adeeb Shihadeh
e76af64065 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
old-commit-hash: b11b14aff8
2023-08-12 17:15:16 -07:00
Justin Newberry
d487c0501f Ruff: enable most of bugbear (#29320)
* added mutable default args

* most of the Bs

* add comment about lrucache
old-commit-hash: 62c1e65924
2023-08-11 15:33:49 -07:00
Justin Newberry
715f028a0e Ruff: enable flake8-builtins (#29315)
* enable flake8-builtins

* replace any with contains

* fix typo in pack

* fix type

* format is from the parent module, has to be enabled

* item_id

* fix item_id

* disable for id since that's what the remote server returns
old-commit-hash: 8793cbff40
2023-08-10 18:58:22 -07:00
Justin Newberry
7d6d3dc9ea Ruff: flake8 pie (misc lints) (#29318)
flake8 pie
old-commit-hash: af3ed37c5c
2023-08-10 15:28:51 -07:00
Shane Smiskol
d78ebfdc45 replay: new demo route (#29215)
* new demo route

* replace other references (except gps test)

* Apply suggestions from code review

* it's 13 segments
old-commit-hash: acc732efbb
2023-08-03 13:18:44 -07:00
Kacper Rączy
2edd65ad4e process_replay: helpers (#28367)
* process_replay helper

* Refactor

* refactor check_enabled uses

* fix __init__

* noqa in __init__ imports

* Move compare_logs outside of process_replay

* Move save_logs to tools.lib.helpers

* Remove save_log from compare_logs

* fix lint issues
old-commit-hash: 64377a88f9
2023-06-06 10:31:13 -07:00
Adeeb Shihadeh
3a968c67e1 loggerd: move to system/ (#27534)
old-commit-hash: 94eb215980
2023-03-08 11:20:49 -08:00
Adeeb Shihadeh
20de7c4b55 bootlog: add helper to get a bootlog by segment id
old-commit-hash: 285fd56a1d
2022-10-04 17:47:37 -07:00
Adeeb Shihadeh
0325a33752 add pyside2 package (#25602)
old-commit-hash: c6b749fb96
2022-08-30 15:10:52 -07:00