Dean Lee
33849245d8
uploader.py: fix empty string handing in AthenadRecentlyViewedRoutes parameter ( #35139 )
...
Fix empty strings in AthenadRecentlyViewedRoutes parameter
2025-05-08 16:31:06 +01:00
Dean Lee
27c11eb597
athena upload: streaming file upload ( #34559 )
...
streaming file upload
2025-02-10 13:57:31 -08:00
Adeeb Shihadeh
9bc35c0919
Revert "athena upload: reduce memory usage and improve efficiency with streaming ( #34528 )"
...
This reverts commit 4c65f51a55
2025-02-06 11:50:04 -08:00
Dean Lee
4c65f51a55
athena upload: reduce memory usage and improve efficiency with streaming ( #34528 )
...
* reduce memory usage and improve upload efficiency with streaming
* space
2025-02-04 13:10:13 -08:00
Adeeb Shihadeh
c287232374
uploader: increase max qlog size ( #34106 )
2024-11-25 09:40:23 -08:00
Adeeb Shihadeh
139f4034a8
Revert "uploader.py: simplify clear_locks by using rglob ( #32974 )"
...
This reverts commit 870f5cbeb39fa63829e6f3a64fa8df57e82d52c3.
old-commit-hash: 9474523add
2024-08-03 12:12:42 -07:00
Dean Lee
7fcba71725
uploader.py: simplify clear_locks by using rglob ( #32974 )
...
simplify clear_locks
old-commit-hash: cafca4f891
2024-08-02 13:40:57 -07:00
Dean Lee
3c1af12490
uploader.py: ensure proper resource management with io.BytesIO ( #33108 )
...
* ensure proper resource management with io.BytesIO
* improve
old-commit-hash: 2728c95b0d
2024-07-29 13:43:31 -07:00
Shane Smiskol
7b67994a05
use zstandard library ( #33110 )
...
* use zstandard
* run 10
* bye bye
* this was a little overzealous
* clean up
old-commit-hash: 1b3b5ab758
2024-07-28 02:03:47 -07:00
Shane Smiskol
b0a8b3bc9d
uploader: compress with zstd ( #32736 )
...
* zstd uploader
* fix that
* fix name of function
* comment
* log failed
* fix comma_api_source for routes with both bz2 and zst rlogs
* TODO
* 10-14 achieves almost no benefit on qlogs in a few cases, but takes 2x the time
* these aren't written out
* regen: specify any list of sources
ooh this is pretty nice
* regen and process replay
* damn, actually we don't need all this (cool tho)
Revert "regen: specify any list of sources"
This reverts commit ceb0b4abed9ad463a9fe98d9b98a05875a52806f.
* just let it auto resolve
* fix athenad/uploader tests
* zst here too
* TODOs
* yes
* Revert "TODOs"
This reverts commit 8c7da1dbd0340c72290b5eb5563b642080ddc131.
* Revert "zst here too"
This reverts commit 23b0023ddfd22c8090be7a7caa09e7026a12aa5c.
* Revert "just let it auto resolve"
This reverts commit f296d62424227ad05facc62abc18a6f81b474e84.
* Revert "regen and process replay"
This reverts commit 0768330e96974a42616d229d159780619d049cd0.
* revert readme
* not in save_log either
* lfg
* Revert "lfg"
This reverts commit 3718559c6c4de7d1f0c80dc9f1a1d335fe679a89.
old-commit-hash: 7dec7c39be
2024-07-26 19:33:35 -07:00
Adeeb Shihadeh
b73a5fba4c
fix old route sorting ( #31787 )
...
* fix old route sorting
* cleanup
* Update system/loggerd/uploader.py
---------
Co-authored-by: Comma Device <device@comma.ai >
old-commit-hash: 158e36976b
2024-03-07 19:22:06 -08:00
Cameron Clough
a09300385b
mypy: use implicit-optional ( #31590 )
...
* mypy: set implicit-optional = true
* find and replace '| None = None' -> '= None' in function args
old-commit-hash: 80da3aee14
2024-02-25 13:29:18 -08:00
Cameron Clough
9bd90112d0
use pyupgrade to update to new typing syntax ( #31580 )
...
* add pyupgrade hook
* run pyupgrade (pre-commit run -a)
* ruff --fix
* Revert "add pyupgrade hook"
This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.
* revert changes to third_party/
* manual type fixes
* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae49
2024-02-24 16:41:23 -08:00
Adeeb Shihadeh
dd3b005456
uploader: sleep more when no files in queue ( #31523 )
...
* uploader: sleep more when no files in queue
* fix test
old-commit-hash: 1c794b710c
2024-02-20 17:25:12 -08:00
Adeeb Shihadeh
6124ecf1c1
uploader: restrict qcam upload on metered connctions ( #31047 )
...
* uploader: restrict qcam upload on metered connctions
* route requesting
* rename
* str
* cleanup
* strip dongle
* upload bookmarked segments
---------
Co-authored-by: Comma Device <device@comma.ai >
old-commit-hash: 2280ac50b7
2024-01-17 21:41:58 -08:00
Adeeb Shihadeh
3d85d3d857
uploader: delay uploading crash and boot logs on metered connections ( #31045 )
...
* uploader: delay uploading some files on metered connections
* just crash and boot for now
old-commit-hash: 19630c5af9
2024-01-17 17:43:41 -08:00
Adeeb Shihadeh
d6a8213c80
uploader cleanup ( #31035 )
...
* Reapply "uploader cleanup" (#31033 )
This reverts commit a2723989bced503a4506684a4a6fdc6c8101f7b2.
* always sleep
* more cleanup
* little more
* fix linter
* little more
old-commit-hash: 0b5fd7287e
2024-01-17 14:24:09 -08:00
Justin Newberry
7f09117bf0
Revert "uploader cleanup" ( #31033 )
...
old-commit-hash: 5e4df41b2f
2024-01-17 11:07:38 -08:00
Adeeb Shihadeh
4580cd1eb9
uploader cleanup ( #31029 )
...
* cleanup
* startup cleanup
* step
* Update test_uploader.py
old-commit-hash: c6c6a441eb
2024-01-16 19:54:07 -08:00
Dean Lee
16266338c6
uploader.py/listdir_by_creation: list directories only ( #30806 )
...
list only directories
Co-authored-by: Comma Device <device@comma.ai >
old-commit-hash: a1f53c70e6
2023-12-26 16:05:45 -08:00
Adeeb Shihadeh
c0c898d83a
uploader: reduce max qlog/qcam size ( #30746 )
...
old-commit-hash: 37351cbdc1
2023-12-14 22:30:46 -08:00
Adeeb Shihadeh
59eb2988de
swaglog.py goes in common/ ( #30631 )
...
* swaglog.py goes in common/
* all of these go in release
* we'll even include the pyx
old-commit-hash: 35f819c823
2023-12-06 17:27:51 -08:00
Adeeb Shihadeh
1cc4f1a634
new msgs default to invalid ( #30587 )
...
* new msgs default to invalid
* fix lm
* set more valid
* update rest
* update refs
* fix logMessage
* more valids
* cleanup
* fix llk test
* pigeond is also valid
* more valids
old-commit-hash: a1d36961cf
2023-12-03 10:50:17 -08:00
Justin Newberry
c7029ea15a
System: cleanup paths to use a common class ( #29816 )
...
* use OP prefix for logmessage
* cleanup paths too
* cleanup the paths too
* add hw.py to release
* fix those issues
* fix unittests
* fix unittests
* fix unittests
* do swaglog_ipc properly across all the files
* fix that
* fix swaglog in c++
* review suggestions
old-commit-hash: bfe990b112
2023-09-07 11:32:47 -07:00
Justin Newberry
3b91610c0b
Precommit: ensure executable bit is set ( #29784 )
...
* precommit ensure executable
* exclude tinygrad
* bump submodules
* exclude tinygrad globally
old-commit-hash: 4e69937d0d
2023-09-05 16:33:26 -07:00
Adeeb Shihadeh
16ec56398c
add openpilot prefix to imports ( #29498 )
...
* add openpilot prefix to imports
* more
* more
* fix docs
* fix linter
* bump submodules
* fix patched tests
* update dynamic imports
* debug
* Revert "debug"
This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.
* fix pm test
old-commit-hash: a9626f95b6
2023-08-20 20:49:55 -07:00
Justin Newberry
a14f162bc7
Ruff: comprehensions conventions ( #29317 )
...
ignore c408
old-commit-hash: 4a9afd7554
2023-08-10 16:40:30 -07:00
Justin Newberry
7b677910df
Linter: Fix >160 line length test ( #29287 )
...
* test
* fix line length
* fix line length
* fix file size
* revert poetry
* cleanup for PR
old-commit-hash: cf64b27168
2023-08-08 17:13:35 -07:00
Shane Smiskol
8a7283d315
uploader: check upload key value ( #28206 )
...
* fix check
* tests
* fix test
* fix
* xattr truthy and falsy tests
---------
Co-authored-by: Cameron Clough <cameronjclough@gmail.com >
old-commit-hash: 7f7637f3e5
2023-05-16 21:55:24 -07:00
Cameron Clough
c0fa2f3ea2
uploader: log actual uploaded file size ( #28000 )
...
* uploader: log actual uploaded file size
The size will be less if compressed before uploading.
* cleanup
* fix return type
* add request to FakeResponse
* cleanup
* fix
old-commit-hash: 7126363093
2023-04-26 20:45:58 -07:00
Cameron Clough
3bb5167aa8
uploader: add missing types
...
old-commit-hash: e03efa723b
2023-04-21 23:06:47 -07:00
Cameron Clough
6626f3036f
uploader: add typing ( #28001 )
...
* uploader: add typing
* fix types
old-commit-hash: a27ce83b28
2023-04-21 22:03:26 -07:00
Adeeb Shihadeh
3a968c67e1
loggerd: move to system/ ( #27534 )
...
old-commit-hash: 94eb215980
2023-03-08 11:20:49 -08:00
