Commit Graph

305 Commits

Author SHA1 Message Date
Harald Schäfer
34fed9f908 URLFILE: Need to catch max retry (#36815)
Need to catch max retry
2025-12-09 10:19:10 -08:00
Bruce Wayne
5393308d03 Logreader: print errors 2025-12-02 15:54:21 -08:00
Harald Schäfer
cfb0a1c18c URLFile multirange (#36740)
* url file multirange

* cleanup urlfile

* time

* fixup

* raise

* Diskfile
2025-12-01 23:11:03 -08:00
Adeeb Shihadeh
7521fd11e2 common: rename atomic_write_in_dir -> atomic_write (#36733)
rename
2025-11-30 15:08:32 -08:00
Harald Schäfer
d3532d7d6f URLFile: catch more (#36712)
* catch

* linter has a point
2025-11-28 17:25:54 -08:00
Adeeb Shihadeh
ee8970dc42 ui: add route-based profiler (#36576)
* ui: add route-based profiler

* cleanup

* this is stupid
2025-11-05 16:23:33 -08:00
Adeeb Shihadeh
954b567b9b merge a bunch of misc stuff into common.utils (#36463)
just utils
2025-10-24 20:45:56 -07:00
Shane Smiskol
870d19f33d Reapply "File sourcing: Not all files are logs (#36025)"
This reverts commit 3570022b9a.

Fix test
2025-08-19 19:59:50 -07:00
Shane Smiskol
60c34a0837 LogReader: run source test (#36031)
run "slow" test
2025-08-19 19:58:47 -07:00
Maxime Desroches
3570022b9a Revert "File sourcing: Not all files are logs (#36025)"
This reverts commit 18b7ddef8f.
2025-08-19 17:11:53 -07:00
Harald Schäfer
18b7ddef8f File sourcing: Not all files are logs (#36025)
* Not all files are logs

* more refactor

* linting ok

* fix tests

* import exception

* whoops forgot to git add

* fix

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2025-08-19 16:25:13 -07:00
Shane Smiskol
5ec9aee216 File sourcing: simplify return type (#36028)
* rm str | none pattern

* clean up

* more clean up

* stash

* Revert "stash"

This reverts commit 3e2472160cc97e9d11922137757d9ef942a0312d.

* fix da prints

* fix cmt
2025-08-19 15:39:44 -07:00
Shane Smiskol
6005b12f94 format logreader 2025-08-19 15:04:17 -07:00
Shane Smiskol
8ec61991ee LogReader sourcing: remove redundant file existence checks (#35991)
* speed up sourcing but avoiding checking for existence of collected files already from previous sources

* clean up

* been meaning to make them return dicts

* no longer true

* no longer true

* clean up

* more

* more

* revert
2025-08-14 19:28:37 -07:00
Shane Smiskol
1eef956cad LogReader sourcing: return dict (#35994)
* new return type

* fix test

* why not
2025-08-14 19:19:37 -07:00
Shane Smiskol
aa91a02db8 LogReader sourcing: check comma API source before CI source (#35992)
sort
2025-08-14 18:26:19 -07:00
Shane Smiskol
e0f51bdbb6 Reapply "LogReader: wrap events to cache which() (#35882)" (#35909)
* Reapply "LogReader: wrap events to cache which() (#35882)"

This reverts commit ba2dced54c.

* fix lr

* speed up

* clean up

* more

* should be fast

* clean up

* only supports Event

* rmrmr

* bye

* simple

* gix
2025-08-08 23:42:54 -07:00
Shane Smiskol
52a4b52628 FileName clean up (#35938)
two spaces!
2025-08-06 14:07:02 -07:00
Harald Schäfer
d1e0a60408 Filename refactor: no enum (#35930)
* conflict

* typing

* typing

* no value

* fix typing

* whitespace

* whitespace

* unused

* Reapply "Filename: minor refactor (#35927)"

This reverts commit 8c7d53004f.

* unused import

* done
2025-08-05 20:37:09 -07:00
Adeeb Shihadeh
8c7d53004f Revert "Filename: minor refactor (#35927)"
This reverts commit 96313fa4c0.
2025-08-05 19:07:17 -07:00
Harald Schäfer
96313fa4c0 Filename: minor refactor (#35927)
* Filename

* rest of refactor
2025-08-05 16:29:25 -07:00
Adeeb Shihadeh
ba2dced54c Revert "LogReader: wrap events to cache which() (#35882)"
This reverts commit 0ebee55050.
2025-08-02 15:53:20 -07:00
Shane Smiskol
0ebee55050 LogReader: wrap events to cache which() (#35882)
* speed up lr

* lazy caching

* clean up

* it fast

* stash

* stash

* chatgpt code is bad as usual

* clean up

* clean up

* clean up

* clean up

* clean up

* clean up

* match behavior

* cmt
2025-08-01 19:07:16 -07:00
Adeeb Shihadeh
a53746f8df ugh api is duplicated 2025-07-29 15:00:11 -07:00
Harald Schäfer
4d39d50858 Route: add more helpers for now (#35783)
* add helpers

* another name

* whitespace

* f string

* add log id

* add logid

* another dumb function

* add more crap

* fix naming

* whitespace

* fstring
2025-07-22 18:50:56 -07:00
Harald Schäfer
103b4df18c Log filenames: make tuples (#35771)
* make tuples

* spelling: this linter is hardcore
2025-07-19 19:19:13 -07:00
Shane Smiskol
ffc8ea5a86 LogReader: use file name constants (#35765)
use common
2025-07-18 20:29:22 -07:00
Shane Smiskol
e1fa04b678 Remove qcamera.ts.gz filename (#35766)
rm
2025-07-18 20:20:34 -07:00
Shane Smiskol
48892e339d LogReader: remove testing closet source (#35763)
rm
2025-07-18 19:21:04 -07:00
Shane Smiskol
fbbad834f3 LogReader: cache internal source available (#35761)
* cache

* fix

* match behavioir
2025-07-18 18:18:02 -07:00
Shane Smiskol
278781e5af LogReader: extend FileName (#35760)
extend filename
2025-07-18 17:00:39 -07:00
Shane Smiskol
43ad1754da LogReader: fix broken test (#35758)
fix lr test
2025-07-18 16:17:01 -07:00
Shane Smiskol
4a5385a5b0 LogReader: support connect URLs (#35757)
* support connect

* clean up
2025-07-18 16:03:30 -07:00
Bruce Wayne
dbe172f7fd Fragile xx code needs this 2025-07-18 15:39:00 -07:00
Shane Smiskol
618a25a612 LogReader: simplify sourcing logic and improve AUTO performance (#35753)
* sourcing supports any file type!

* stuff

* no camera for now

* i guess these are fine without

* rename

* get rid of these too!

* fix

* fix

* this is better

* start to clean up!

* better

* holy

holy

* clean up sources

* more robust

* working but needs some clean up

* clean up

* remove some trash

* nl

* auto_source can only return if it finds acceptable logs from sources

* double negative is confusing

* default

* list isn't hashable

* fix typing

* clean up

* speed up -- test_models got zst before bz2 in openpilotci, so do that (some segments have both bz2 and zst!)

* don't be a hero

* same behavior for now
2025-07-18 05:15:55 -07:00
Shane Smiskol
b50b351b15 LogReader: remove commaCarSegments selector (#35750)
* a source should not be a readmode... and readmode should be logtype

* fix that

* fixup
2025-07-18 02:17:11 -07:00
Shane Smiskol
862a816215 LogReader: add data endpoint option (#35751)
* add data endpoint option

* clean up
2025-07-17 23:12:08 -07:00
Shane Smiskol
239646b20b Revert "clean up"
This reverts commit 5339a13c34.

don't push to master, kids
2025-07-17 21:43:21 -07:00
Shane Smiskol
5339a13c34 clean up 2025-07-17 21:42:47 -07:00
Shane Smiskol
c553c1f872 LogReader: support passing list of sources (#35749)
* far too long

* this is a better experience

* no rename for now
2025-07-17 19:54:38 -07:00
Harald Schäfer
54da96dbdf Route: add name format (#35743)
* needed for now

* TODO

* Add TODO
2025-07-17 18:22:36 -07:00
Shane Smiskol
632f7594a4 Ban time v2 (#35744)
* ban it!

* some legitimate blunders

* even more!

* this is from commaai/one

* here too

* this too

* ignore rest

* one liner

* Revert "one liner"

This reverts commit d96a1624032aaad829a005b523e579d953cfe690.
2025-07-17 16:50:52 -07:00
commaci-public
3a2c4a855a [bot] Update Python packages (#35726)
* Update Python packages

* dep

* ref

* hint

---------

Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-07-14 13:48:55 -07:00
Jason Young
b1972af6db tools: fix log suffix for commaCarSegments v2 (#35725) 2025-07-14 05:37:33 -04:00
Adeeb Shihadeh
2b56a6c37e Revert "tools: update to commaCarSegments v2 (#35660)"
This reverts commit 12a5d8c2db.
2025-07-09 10:58:47 -07:00
Jason Young
12a5d8c2db tools: update to commaCarSegments v2 (#35660) 2025-07-08 18:31:44 -04:00
Maxime Desroches
6a37d8a89e fix framereader indent 2025-06-19 10:38:26 -07:00
Harald Schäfer
87a6e369aa Framereader: minor cleanup (#35577)
* No wrapping

* unused test

* another list

* mypy

* cleaner

* Revert "cleaner"

This reverts commit ccc1446b9d649d64b20175e22a66e135c44b21e5.

* mypy
2025-06-19 09:49:51 -07:00
Harald Schäfer
5f3d876aaa model replay: framereader cache (#35576)
* Simpler cache version

* cachetools

* different LRU

* lint

* smaller

* just write LRU

* mypy

* same length
2025-06-18 16:29:22 -07:00
Harald Schäfer
d9b6c16037 Cleanup framereader (#35513)
* squash

* misc cleanup

* no LLM garbage

* misc fixes

* typo

* fix CI

* fix hints

* LLM soo wordy

* improve
2025-06-09 22:39:35 -07:00