mawei/sunnypilot
1
0
Fork 0
mirror of https://github.com/sunnypilot/sunnypilot.git synced 2026-02-19 04:13:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
15,573 Commits 185 Branches 163 Tags
a53746f8dfbce15e20cf2ef2c36e28ebb86780e9
Commit Graph

95 Commits

Author SHA1 Message Date
Harald Schäfer
103b4df18c Log filenames: make tuples (#35771) 
* make tuples

* spelling: this linter is hardcore
 2025-07-19 19:19:13 -07:00
Shane Smiskol
ffc8ea5a86 LogReader: use file name constants (#35765) 
use common
 2025-07-18 20:29:22 -07:00
Shane Smiskol
48892e339d LogReader: remove testing closet source (#35763) 
rm
 2025-07-18 19:21:04 -07:00
Shane Smiskol
278781e5af LogReader: extend FileName (#35760) 
extend filename
 2025-07-18 17:00:39 -07:00
Shane Smiskol
4a5385a5b0 LogReader: support connect URLs (#35757) 
* support connect

* clean up
 2025-07-18 16:03:30 -07:00
Shane Smiskol
618a25a612 LogReader: simplify sourcing logic and improve AUTO performance (#35753) 
* sourcing supports any file type!

* stuff

* no camera for now

* i guess these are fine without

* rename

* get rid of these too!

* fix

* fix

* this is better

* start to clean up!

* better

* holy

holy

* clean up sources

* more robust

* working but needs some clean up

* clean up

* remove some trash

* nl

* auto_source can only return if it finds acceptable logs from sources

* double negative is confusing

* default

* list isn't hashable

* fix typing

* clean up

* speed up -- test_models got zst before bz2 in openpilotci, so do that (some segments have both bz2 and zst!)

* don't be a hero

* same behavior for now
 2025-07-18 05:15:55 -07:00
Shane Smiskol
b50b351b15 LogReader: remove commaCarSegments selector (#35750) 
* a source should not be a readmode... and readmode should be logtype

* fix that

* fixup
 2025-07-18 02:17:11 -07:00
Shane Smiskol
862a816215 LogReader: add data endpoint option (#35751) 
* add data endpoint option

* clean up
 2025-07-17 23:12:08 -07:00
Shane Smiskol
c553c1f872 LogReader: support passing list of sources (#35749) 
* far too long

* this is a better experience

* no rename for now
 2025-07-17 19:54:38 -07:00
Shane Smiskol
27d882cc0a Revert "logreader.py: concurrent file existence checks" (#34942) 
Revert "logreader.py: concurrent file existence checks (#34875)"

This reverts commit e11b239b2e.
 2025-03-26 14:45:35 -07:00
Shane Smiskol
e652f435f9 max_lat_accel.py: use parallel processing (#34936) 
* use run_across_segments

* disable tqdm
 2025-03-25 13:26:16 -07:00
Dean Lee
e11b239b2e logreader.py: concurrent file existence checks (#34875) 
* concurrent file existence checks

* upper bound on parallel queries

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
 2025-03-24 15:46:42 -07:00
Shane Smiskol
65d667e60e Revert "logreader: avoid repeated prompts in auto_strategy when rlogs are missing" (#34919) 
Revert "logreader: avoid repeated prompts in auto_strategy when rlogs are mis…"

This reverts commit b8229ebca6.
 2025-03-21 17:28:19 -07:00
Dean Lee
b8229ebca6 logreader: avoid repeated prompts in auto_strategy when rlogs are missing (#34890) 
* avoid repeated prompts in auto_strategy when rlogs are missing

* shorter

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
 2025-03-21 14:43:48 -07:00
Adeeb Shihadeh
071f8ba658 LogReader: add time_series helper 2025-02-19 16:43:54 -08:00
Dean Lee
9ec54f59c6 loggerd: switch logging from raw file format to zstd compressed files (#34549) 
* switch logging from raw file format to zstd compressed files

* more zst suffix

* compress bootlog

* remove class RawFile

* Optimize ZstdFileWriter by adding input caching

* use ZSTD_compressStream2

* cleanup

* LOG_COMPRESSION_LEVEL=10

* space

* add zst suffix to LOGS_SIZE_RATE
 2025-02-08 12:55:31 -08:00
Shane Smiskol
2af9f68147 LogReader: more specific exceptions (#33914) 
* more specific logreader exceptions

* huh

* fix
 2024-11-01 15:59:46 -07:00
commaci-public
1e92af9b3c [bot] Update Python packages (#33436) 
* Update Python packages

* bump

* specify any list of sources

ooh this is pretty nice

* update

* y

* fix

* fix that

* bump

---------

Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 239cc69a20b7d3c2f285bd1907cecad9c9604d50
 2024-09-03 18:57:28 -07:00
Shane Smiskol
9da055853f LogReader clean ups (#33446) 
* buggin me

* no caps

* remove at least one level of indirection

* no recursion, remove a bunch of junk

* default is now allfault ??

* back

* fix tests
old-commit-hash: 5796bf12b6dac4f3fc8e55ecd9785c73044d8a08
 2024-09-03 18:34:18 -07:00
Shane Smiskol
730924e7f2 LogReader: formatting and remove unnecessary type alias 
old-commit-hash: 34305bec091c1088249fe6fe8df8de64a7ce1f86
 2024-09-03 17:58:50 -07:00
Kacper Rączy
786aff27da azure_container: pass the filename explicitely when using get_url (#33407) 
* Pass filename explicitely when using azure container

* Do not update segments in test_processes

* openpilotci_source_zst

* Fix static analysisc
old-commit-hash: b5958ebb60
 2024-08-29 21:51:55 -07:00
Jimmy
937ed5f700 Add testing closet to LogReader source (#33242) 
* add testing closet to logreader source

* check source availability

* use internal_source_available
old-commit-hash: 118932acd3
 2024-08-11 15:10:14 -07:00
Hoang Bui
6a2b4dedfc tools/logreader: add desc arg to run_across_segments() (#33226) 
old-commit-hash: 9e648e428d
 2024-08-08 07:51:14 -07:00
Shane Smiskol
7ce33fbe2b process replay: use zst (#33138) 
* test

* update refs to zst

* update
old-commit-hash: dfd387520e
 2024-07-30 01:46:03 -07:00
Shane Smiskol
7b67994a05 use zstandard library (#33110) 
* use zstandard

* run 10

* bye bye

* this was a little overzealous

* clean up
old-commit-hash: 1b3b5ab758
 2024-07-28 02:03:47 -07:00
Shane Smiskol
539c828861 tools: keep re pattern file small (#33084) 
remove bz2 from re file
old-commit-hash: e68bb26e14
 2024-07-25 22:05:10 -07:00
Shane Smiskol
309ccf6482 LogReader: support multiple log extensions in route (#33066) 
* support rlog.zst

* TODO
old-commit-hash: e38a1428aa
 2024-07-24 14:47:33 -07:00
Shane Smiskol
5333e8ba20 LogReader: show total segment count in missing rlog prompt 
old-commit-hash: 3e41b1b287
 2024-07-01 22:43:48 -07:00
StefanLekanic-SYRMIA
5887093eed LogReader: print number of missing rlogs (#32830) 
* count rlogs

* minimization changes

* cache to avoid duplicate checks

* no real point in checking qlogs to just show the warning

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 504dd4543d
 2024-06-26 22:15:13 -07:00
Shane Smiskol
9f50db15c0 LogReader: try zst on internal source (#32751) 
* internal source list files like azure api

* messy but works

* no limit

* simpler

* clean up

* clean up

* clean up

* that's obvious

* better

* we need to unfortunately return a url, so best to take a naive approach for now

* todo

* fix

* clean up
old-commit-hash: b45caf4033
 2024-06-14 01:12:19 -07:00
Shane Smiskol
644156420b LogReader: revert broken check 
it returns files OR None

old-commit-hash: 68e22faa29
 2024-06-14 00:30:14 -07:00
Shane Smiskol
8f402f29a3 SegmentRange: remove timestamp (#32750) 
* add deprecation warning

* remove timestamp

* Update logreader.py
old-commit-hash: d5ce4b3085
 2024-06-13 23:19:32 -07:00
Shane Smiskol
ed80e9357a LogReader: improve error messages (#32747) 
* better error messages

* clean up
old-commit-hash: 3ede1e2a7c
 2024-06-13 22:37:00 -07:00
Shane Smiskol
e88cbb0162 add zstd Python library (#32731) 
* add zstd

* fix

* uploader

* logreader: zst support

* caps

* only logreader
old-commit-hash: 04a93dd849
 2024-06-12 21:14:23 -07:00
Dean Lee
450d012166 logreader.py: optimize event loading and sorting in _LogFileReader (#32729) 
* improve logfilereader

* less changes
old-commit-hash: c64bca97ba
 2024-06-12 14:20:06 -07:00
Justin Newberry
529699f3fe logreader: fix auto source + interactive modes (#31847) 
* interactive modes

* these exceptions don't matter
old-commit-hash: a819292073
 2024-03-12 20:04:17 -07:00
Shane Smiskol
bc7664aa2b LogReader: fix sort by time and union types (#31565) 
* fix :(

* test_sort_by_time

* this isn't required

* not slow, and just compare sorted

* messy

* works

* clean up

* clean up

* not here

* clean up

* clean up

* clean up

* makes network call

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: e964c5944d
 2024-03-08 02:49:24 -08:00
Cameron Clough
9bd90112d0 use pyupgrade to update to new typing syntax (#31580) 
* add pyupgrade hook

* run pyupgrade (pre-commit run -a)

* ruff --fix

* Revert "add pyupgrade hook"

This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.

* revert changes to third_party/

* manual type fixes

* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae49
 2024-02-24 16:41:23 -08:00
Justin Newberry
8dbdd4736c test_logreader: test internal scenarios (#31484) 
* test source scenario

* test source scenario

* fix
old-commit-hash: 16d1339553
 2024-02-15 12:10:56 -08:00
Justin Newberry
42f292b3ce test_logreader: test interactive mode + fix typo in AUTO_INTERACTIVE (#31481) 
* fix spelling

* test interactive

* remove that

* test taht

* move that
old-commit-hash: 8aee0d2af2
 2024-02-15 11:24:27 -08:00
Justin Newberry
edd4649d24 LogReader: add typing hints (#31464) 
logreader typing
old-commit-hash: 33cf6bda9e
 2024-02-14 10:34:17 -08:00
Justin Newberry
38fc6f9dd6 LogReader: retain old behavior for direct parsing of files (#31419) 
* maintain exception

* test that head is not called

* annoying mock

* test with cache
old-commit-hash: 1436f576df
 2024-02-14 10:05:25 -08:00
Shane Smiskol
5c16ae62d8 SegmentRange: simplify slice (#31455) 
* simplify slicing

* rm
old-commit-hash: 8fe9bc7a69
 2024-02-14 04:17:03 -08:00
Shane Smiskol
4c7ad7f3b7 tools/lib: format code (#31454) 
easier to read. pylint used to catch all this stuff, but it's mostly missing in ruff :'(
old-commit-hash: 0846175f44
 2024-02-14 01:56:17 -08:00
Justin Newberry
8b5f1235f4 live_logreader to its own file (#31436) 
own file
old-commit-hash: d6762c3035
 2024-02-13 13:29:52 -08:00
Justin Newberry
fbf5ada69f LogReader: live_logreader helpers (#31416) 
live helper
old-commit-hash: 0a92c5bf96
 2024-02-12 12:58:25 -08:00
Greg Hogan
1384c10fb0 logreader: skip internal source if not available (#31400) 
* logreader: skip internal source if not available

* raise exception

* but only when appropriate
old-commit-hash: 667693b8c2
 2024-02-10 00:20:43 -08:00
Justin Newberry
9b4b5d3cf5 LogReader: skip file_exists check for comma_api (#31251) 
* valid file

* missed this one
old-commit-hash: 6010a39bf8
 2024-01-31 16:43:58 -08:00
Justin Newberry
16e0907129 LogReader: add ``.filter`` (#31222) 
add filter
old-commit-hash: c782fbc34e
 2024-01-29 17:33:47 -08:00
Justin Newberry
223b1ed065 LogReader: add tqdm back to run_across_segments (#31216) 
tqdm
old-commit-hash: 7835f9cce0
 2024-01-29 13:18:49 -08:00