commaci-public
1e92af9b3c
[bot] Update Python packages ( #33436 )
...
* Update Python packages
* bump
* specify any list of sources
ooh this is pretty nice
* update
* y
* fix
* fix that
* bump
---------
Co-authored-by: Vehicle Researcher <user@comma.ai >
Co-authored-by: Shane Smiskol <shane@smiskol.com >
old-commit-hash: 239cc69a20b7d3c2f285bd1907cecad9c9604d50
2024-09-03 18:57:28 -07:00
Shane Smiskol
9da055853f
LogReader clean ups ( #33446 )
...
* buggin me
* no caps
* remove at least one level of indirection
* no recursion, remove a bunch of junk
* default is now allfault ??
* back
* fix tests
old-commit-hash: 5796bf12b6dac4f3fc8e55ecd9785c73044d8a08
2024-09-03 18:34:18 -07:00
Shane Smiskol
730924e7f2
LogReader: formatting and remove unnecessary type alias
...
old-commit-hash: 34305bec091c1088249fe6fe8df8de64a7ce1f86
2024-09-03 17:58:50 -07:00
Kacper Rączy
786aff27da
azure_container: pass the filename explicitely when using get_url ( #33407 )
...
* Pass filename explicitely when using azure container
* Do not update segments in test_processes
* openpilotci_source_zst
* Fix static analysisc
old-commit-hash: b5958ebb600f503f585a1080236a0f26b4f5c6a0
2024-08-29 21:51:55 -07:00
Jimmy
937ed5f700
Add testing closet to LogReader source ( #33242 )
...
* add testing closet to logreader source
* check source availability
* use internal_source_available
old-commit-hash: 118932acd3ba74cabcfeebd7993f94e5dcbbcda5
2024-08-11 15:10:14 -07:00
Hoang Bui
6a2b4dedfc
tools/logreader: add desc arg to run_across_segments() ( #33226 )
...
old-commit-hash: 9e648e428da38eeb4b096e291129841bdb7d40e8
2024-08-08 07:51:14 -07:00
Shane Smiskol
7ce33fbe2b
process replay: use zst ( #33138 )
...
* test
* update refs to zst
* update
old-commit-hash: dfd387520e88fa68f0485b6ce8dcf5befa82bafa
2024-07-30 01:46:03 -07:00
Shane Smiskol
7b67994a05
use zstandard library ( #33110 )
...
* use zstandard
* run 10
* bye bye
* this was a little overzealous
* clean up
old-commit-hash: 1b3b5ab75857465f58ec8a687db04c8ee0470e05
2024-07-28 02:03:47 -07:00
Shane Smiskol
539c828861
tools: keep re pattern file small ( #33084 )
...
remove bz2 from re file
old-commit-hash: e68bb26e14d4e85f61ab3f58ac9667ca9d6ea565
2024-07-25 22:05:10 -07:00
Shane Smiskol
309ccf6482
LogReader: support multiple log extensions in route ( #33066 )
...
* support rlog.zst
* TODO
old-commit-hash: e38a1428aa20aad134b4ad9fd1abab33a1de70b9
2024-07-24 14:47:33 -07:00
Shane Smiskol
5333e8ba20
LogReader: show total segment count in missing rlog prompt
...
old-commit-hash: 3e41b1b287c956ed17ab065cf4391aae05333e52
2024-07-01 22:43:48 -07:00
StefanLekanic-SYRMIA
5887093eed
LogReader: print number of missing rlogs ( #32830 )
...
* count rlogs
* minimization changes
* cache to avoid duplicate checks
* no real point in checking qlogs to just show the warning
---------
Co-authored-by: Shane Smiskol <shane@smiskol.com >
old-commit-hash: 504dd4543dd7fd6c466817faa82cda42b6b55542
2024-06-26 22:15:13 -07:00
Shane Smiskol
9f50db15c0
LogReader: try zst on internal source ( #32751 )
...
* internal source list files like azure api
* messy but works
* no limit
* simpler
* clean up
* clean up
* clean up
* that's obvious
* better
* we need to unfortunately return a url, so best to take a naive approach for now
* todo
* fix
* clean up
old-commit-hash: b45caf403366ed4161516e49b9e433805d9de6c4
2024-06-14 01:12:19 -07:00
Shane Smiskol
644156420b
LogReader: revert broken check
...
it returns files OR None
old-commit-hash: 68e22faa296d335f2d497c52339f536f761602c3
2024-06-14 00:30:14 -07:00
Shane Smiskol
8f402f29a3
SegmentRange: remove timestamp ( #32750 )
...
* add deprecation warning
* remove timestamp
* Update logreader.py
old-commit-hash: d5ce4b308588f2bd4eee300110ddcd66e787a110
2024-06-13 23:19:32 -07:00
Shane Smiskol
ed80e9357a
LogReader: improve error messages ( #32747 )
...
* better error messages
* clean up
old-commit-hash: 3ede1e2a7c850622beef2549315f320aa018105a
2024-06-13 22:37:00 -07:00
Shane Smiskol
e88cbb0162
add zstd Python library ( #32731 )
...
* add zstd
* fix
* uploader
* logreader: zst support
* caps
* only logreader
old-commit-hash: 04a93dd849461df7a3496706c6172d95e83149c9
2024-06-12 21:14:23 -07:00
Dean Lee
450d012166
logreader.py: optimize event loading and sorting in _LogFileReader ( #32729 )
...
* improve logfilereader
* less changes
old-commit-hash: c64bca97bac68462a63b47d33aedf869d21ac97e
2024-06-12 14:20:06 -07:00
Justin Newberry
529699f3fe
logreader: fix auto source + interactive modes ( #31847 )
...
* interactive modes
* these exceptions don't matter
old-commit-hash: a8192920737216fe07bd2f273efdee67669b8483
2024-03-12 20:04:17 -07:00
Shane Smiskol
bc7664aa2b
LogReader: fix sort by time and union types ( #31565 )
...
* fix :(
* test_sort_by_time
* this isn't required
* not slow, and just compare sorted
* messy
* works
* clean up
* clean up
* not here
* clean up
* clean up
* clean up
* makes network call
---------
Co-authored-by: Justin Newberry <justin@comma.ai >
old-commit-hash: e964c5944d4342b6e08ca872eacf9d524a49b479
2024-03-08 02:49:24 -08:00
Cameron Clough
9bd90112d0
use pyupgrade to update to new typing syntax ( #31580 )
...
* add pyupgrade hook
* run pyupgrade (pre-commit run -a)
* ruff --fix
* Revert "add pyupgrade hook"
This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.
* revert changes to third_party/
* manual type fixes
* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae4967943ee33a3699a1d89a7a770d95e9
2024-02-24 16:41:23 -08:00
Justin Newberry
8dbdd4736c
test_logreader: test internal scenarios ( #31484 )
...
* test source scenario
* test source scenario
* fix
old-commit-hash: 16d13395536e84ddf492b2352d827d33befd601e
2024-02-15 12:10:56 -08:00
Justin Newberry
42f292b3ce
test_logreader: test interactive mode + fix typo in AUTO_INTERACTIVE ( #31481 )
...
* fix spelling
* test interactive
* remove that
* test taht
* move that
old-commit-hash: 8aee0d2af2c9e9c1755afa2b46f68b199958ccdb
2024-02-15 11:24:27 -08:00
Justin Newberry
edd4649d24
LogReader: add typing hints ( #31464 )
...
logreader typing
old-commit-hash: 33cf6bda9ef6b1ed19f3a0fed4a5914a414ae653
2024-02-14 10:34:17 -08:00
Justin Newberry
38fc6f9dd6
LogReader: retain old behavior for direct parsing of files ( #31419 )
...
* maintain exception
* test that head is not called
* annoying mock
* test with cache
old-commit-hash: 1436f576df5f43f78b9253b5b2072fbf1414d9be
2024-02-14 10:05:25 -08:00
Shane Smiskol
5c16ae62d8
SegmentRange: simplify slice ( #31455 )
...
* simplify slicing
* rm
old-commit-hash: 8fe9bc7a6977f9f901b1ddf8eb0916ac80224644
2024-02-14 04:17:03 -08:00
Shane Smiskol
4c7ad7f3b7
tools/lib: format code ( #31454 )
...
easier to read. pylint used to catch all this stuff, but it's mostly missing in ruff :'(
old-commit-hash: 0846175f44ac9364527087db4ec54767e6884fca
2024-02-14 01:56:17 -08:00
Justin Newberry
8b5f1235f4
live_logreader to its own file ( #31436 )
...
own file
old-commit-hash: d6762c3035fac31cc9582c685c07441d61da0f83
2024-02-13 13:29:52 -08:00
Justin Newberry
fbf5ada69f
LogReader: live_logreader helpers ( #31416 )
...
live helper
old-commit-hash: 0a92c5bf966530335e804a8f240b10ae982bfcb2
2024-02-12 12:58:25 -08:00
Greg Hogan
1384c10fb0
logreader: skip internal source if not available ( #31400 )
...
* logreader: skip internal source if not available
* raise exception
* but only when appropriate
old-commit-hash: 667693b8c2ca7ef0b0457823cd76b93b22764038
2024-02-10 00:20:43 -08:00
Justin Newberry
9b4b5d3cf5
LogReader: skip file_exists check for comma_api ( #31251 )
...
* valid file
* missed this one
old-commit-hash: 6010a39bf8cb030b5b10e1d0657f9291467b6872
2024-01-31 16:43:58 -08:00
Justin Newberry
16e0907129
LogReader: add ``.filter`` ( #31222 )
...
add filter
old-commit-hash: c782fbc34e05922765959ee3c3b6bba38c0150fb
2024-01-29 17:33:47 -08:00
Justin Newberry
223b1ed065
LogReader: add tqdm back to run_across_segments ( #31216 )
...
tqdm
old-commit-hash: 7835f9cce05b2f8743bbd632dd6a3093741e6bd2
2024-01-29 13:18:49 -08:00
Justin Newberry
abfa8fcda9
CommaCarSegments: BIG Database! ( #31169 )
...
* sanetized selector
* god damn
* add to re
* slog branch
* slog
* source
* big
* not slog
* cleanup
* no space
* works!
* trigger
* main!
old-commit-hash: 824ddffec8c67758f3ea080c75e805d4b27f38ef
2024-01-27 01:07:29 -08:00
Justin Newberry
9edc902106
LogReader: fail-fast on invalid segments ( #31152 )
...
* don't check all
* is
old-commit-hash: edd2428b6067381ffaa25420d6fd9f0f142038a8
2024-01-25 11:09:09 -08:00
Justin Newberry
7d76ac8eba
LogReader: replace get_first_message with LogReader.first ( #31146 )
...
* first
* str
old-commit-hash: dd0c474e6cc09d17cd07b51098d826dc40bc5403
2024-01-24 10:21:57 -08:00
Justin Newberry
67bb21219c
LogReader: no redownloading on multiple iterations ( #31141 )
...
* no redownload
* sort
old-commit-hash: 88dcaa51c4d1fcc338d44f55134593760334ae23
2024-01-23 23:54:11 -08:00
Justin Newberry
0d558d02ee
auto_source: add commaCarSegments source ( #31137 )
...
add source
old-commit-hash: ac71c30ad5d8a5f303ac7a08cc73b8a4b0174498
2024-01-23 21:40:04 -08:00
Justin Newberry
6a1d7fd732
comma_api_source: fix qlogs ( #31136 )
...
fix qlogs
old-commit-hash: 3d9b483e107e119fd5322e00494a769ef0e081bc
2024-01-23 20:51:15 -08:00
Justin Newberry
8768be968b
LogReader: don't use route for max_segment ( #31129 )
...
* dont use route
* fix and cleanup
old-commit-hash: d574013d7bda94a43b29095887e176b9723d13cb
2024-01-23 14:51:45 -08:00
Justin Newberry
c5c522c984
plotjuggler: auto fallback to qlogs with prompt ( #31110 )
...
* qlog fallback
* show a warning for that
* qlog fallback
* fix
* wording
old-commit-hash: 65490bb3c55fa81d5144eba2c09f054b73179cc4
2024-01-23 12:53:19 -08:00
Justin Newberry
c03fe6bb13
add example of using the comma car segments database ( #31117 )
...
* add vin fingerprinting example
* perhaps!
* remove segment
* easier
old-commit-hash: 389b8ca30d62f81ae40a7d026a299ce3930183da
2024-01-22 18:23:07 -08:00
Justin Newberry
f447062b66
CommaCarSegments: public database of segments for each platform ( #31114 )
...
* comma car segments
* comma car segments for test_models
* oneliner
Co-authored-by: Shane Smiskol <shane@smiskol.com >
* better name
* not used here
* sort
* remove print
* better comment
---------
Co-authored-by: Shane Smiskol <shane@smiskol.com >
old-commit-hash: caa9153974c21fd827d0e991f1789cf3e81b5c3b
2024-01-22 17:33:03 -08:00
Justin Newberry
b29eea47ce
sanitizer: sanitize vin number ( #31090 )
...
old-commit-hash: 2c86c023fe2d3072910ed33ff97cd6d3ddf2915d
2024-01-21 12:22:32 -05:00
Justin Newberry
a9676e7086
auto_source: optimize api calls and use head to determine if file exists ( #31025 )
...
* fast
* catch all
* source
* fix file_exists
* remove duplicate reset
* test multiple loops
* iterations
* cleanup imports
old-commit-hash: 2967cada71fd1b40e86356585fa5d68301d9f1c8
2024-01-16 17:24:37 -08:00
Justin Newberry
5df01b3d22
LogReader: fix plotjuggler not reading properly ( #31023 )
...
* fix plotjuggler
* simpler
* fix juggler
* fix that
old-commit-hash: 61fe705160ddb9e5c55292c7f702c5502e2c0d2e
2024-01-16 12:40:49 -08:00
Justin Newberry
4b7edec2b0
LogReader: move opci to tools/lib ( #31021 )
...
move opci
old-commit-hash: 7d400112df6692ef026245b03951345cc36dd83e
2024-01-16 10:58:51 -08:00
Justin Newberry
35dcd7ccd8
LogReader: support lists of identifiers ( #31020 )
...
* support lists of logs
* mark as slow instead
old-commit-hash: 0739d5d7b25f1fbcb70a4dd2ef9b5071afb2c526
2024-01-16 10:40:00 -08:00
Justin Newberry
13e7366b6d
LogReader: support only_union_types ( #31019 )
...
support only-union-types
old-commit-hash: 4da8d165144404e7356e611e68b036ecb33126ef
2024-01-16 10:03:55 -08:00
Justin Newberry
c8cc30112d
rename segmentrangereader to logreader ( #30981 )
...
* Replace multilogiterator
* replace logreader
* update readme
* fix from_Bytes
* new section
* reset before iter
* selector
* fix internal
* whitespace
old-commit-hash: 3d73512c0702598d27e04039976fe0a91d5952f0
2024-01-12 12:53:50 -08:00
