Dean Lee
541bd4d4d9
ui: switch spinner and text window back to standalone process ( #35470 )
...
switch spinner and text window back to standalone process
2025-06-06 13:20:05 -07:00
Adeeb Shihadeh
f1ba6c389b
quick repo garbage collection ( #35274 )
...
* start gc
* lil more
* debug
* no sound
* add back
2025-05-18 16:59:53 -07:00
Maxime Desroches
47ed90c6cf
Reapply "Mypy: Got passing on macos ( #34591 )" ( #35126 ) ( #35153 )
...
* Mypy: Got passing on macos (#34591 )
* Mypy: Got mypy passing on macos
* common/realtime.py refactor
* Mypy: mypy passing on darwin
* Refactor: Removed else: pass statement
* Refactor: Removed unnecessary check
* added xattr to pyproject
* loggerd: switched to xatter module
* loggerd: removed unused module in xattr_cache.py
* UV: update uv.lock
* Update system/athena/athenad.py
* athenad: fixed blank lines
* loggerd: refactor of xattr_cache
* cleanup
---------
* fix getxattr no attribute on macOS
* try fixing missing ENOATTR on Linux
---------
Co-authored-by: Andrei Radulescu <andi.radulescu@gmail.com >
Co-authored-by: BrainLess <116778989+BrainLessPea@users.noreply.github.com >
2025-05-07 19:11:37 -07:00
Maxime Desroches
aa1b790708
revert agnos 12
2025-05-06 21:38:50 -07:00
Andrei Radulescu
bbf37ae5c7
Reapply "Mypy: Got passing on macos ( #34591 )" ( #35126 )
...
* Mypy: Got passing on macos (#34591 )
* Mypy: Got mypy passing on macos
* common/realtime.py refactor
* Mypy: mypy passing on darwin
* Refactor: Removed else: pass statement
* Refactor: Removed unnecessary check
* added xattr to pyproject
* loggerd: switched to xatter module
* loggerd: removed unused module in xattr_cache.py
* UV: update uv.lock
* Update system/athena/athenad.py
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
* athenad: fixed blank lines
* loggerd: refactor of xattr_cache
* cleanup
---------
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
* fix getxattr no attribute on macOS
* try fixing missing ENOATTR on Linux
---------
Co-authored-by: BrainLess <116778989+BrainLessPea@users.noreply.github.com >
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
2025-05-06 21:09:59 -07:00
Cameron Clough
9d17c73f6b
avoid importing Raylib UI ( #35078 )
2025-04-25 19:09:19 -07:00
Cameron Clough
3ee2882093
ui: replace qt spinner with raylib ( #35059 )
...
* use raylib spinner
* remove qt spinner
* use wrapper, render spinner in thread
* english
* fix?
* match Qt font size more closely
2025-04-24 21:16:38 +01:00
Trey Moen
e785026a98
feat(athenad): priority uploads ( #34856 )
...
Today, Firehose uploads and user-requested uploads from Connect are treated the same. This is not ideal behavior if the user wants to upload routes immediately for a bug report and the queue is full of uploads for Firehose. The workaround is to clear the queue and retry the upload from Connect.
This PR adds an optional `priority` to requested file uploads in `athenad`. By default, all requests are marked `Low`. However, if the caller wishes to mark their uploads as "more important," then the upload queue will prioritize those requests when uploading.
The only caveat to this PR is that we won't reorder files currently being uploaded. Most connections are quick enough to finish uploading max 4 files before polling the new high-priority items in the queue.
Closes https://github.com/commaai/openpilot/issues/34836
**Verification**
Added test case to insert upload tasks with differing priorities. Polling the queue produces items in the correct order (smallest # to largest).
PR to mark Connect uploads as high priority: https://github.com/commaai/connect/pull/557
---------
Co-authored-by: Cameron Clough <cameronjclough@gmail.com >
2025-03-19 19:22:54 +00:00
Adeeb Shihadeh
701868d5cd
firehose auto start + stats ( #34747 )
...
* don't need that
* stats
* lil more
* lil more
* cleanup
* tweaks
---------
Co-authored-by: Comma Device <device@comma.ai >
2025-03-03 18:21:02 -08:00
Adeeb Shihadeh
27b5a727e3
Revert "Mypy: Got passing on macos ( #34591 )"
...
This reverts commit b09b48130e .
2025-02-16 08:50:46 -08:00
BrainLess
b09b48130e
Mypy: Got passing on macos ( #34591 )
...
* Mypy: Got mypy passing on macos
* common/realtime.py refactor
* Mypy: mypy passing on darwin
* Refactor: Removed else: pass statement
* Refactor: Removed unnecessary check
* added xattr to pyproject
* loggerd: switched to xatter module
* loggerd: removed unused module in xattr_cache.py
* UV: update uv.lock
* Update system/athena/athenad.py
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
* athenad: fixed blank lines
* loggerd: refactor of xattr_cache
* cleanup
---------
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
2025-02-16 00:05:52 -08:00
Dean Lee
27c11eb597
athena upload: streaming file upload ( #34559 )
...
streaming file upload
2025-02-10 13:57:31 -08:00
Dean Lee
9ec54f59c6
loggerd: switch logging from raw file format to zstd compressed files ( #34549 )
...
* switch logging from raw file format to zstd compressed files
* more zst suffix
* compress bootlog
* remove class RawFile
* Optimize ZstdFileWriter by adding input caching
* use ZSTD_compressStream2
* cleanup
* LOG_COMPRESSION_LEVEL=10
* space
* add zst suffix to LOGS_SIZE_RATE
2025-02-08 12:55:31 -08:00
Adeeb Shihadeh
9bc35c0919
Revert "athena upload: reduce memory usage and improve efficiency with streaming ( #34528 )"
...
This reverts commit 4c65f51a55 .
2025-02-06 11:50:04 -08:00
Dean Lee
4c65f51a55
athena upload: reduce memory usage and improve efficiency with streaming ( #34528 )
...
* reduce memory usage and improve upload efficiency with streaming
* space
2025-02-04 13:10:13 -08:00
Adeeb Shihadeh
a5dd2ab2aa
athena: upload logs in parallel ( #34509 )
...
Co-authored-by: Comma Device <device@comma.ai >
2025-01-30 20:12:00 -08:00
Adeeb Shihadeh
79e52ea83d
FIREHOSE mode v0.1 ( #34496 )
...
* add toggle
* athena
* fix mypy
2025-01-29 15:31:15 -08:00
Adeeb Shihadeh
9cf02ca8db
Skip registration on newer devices ( #34316 )
2024-12-26 20:22:44 -08:00
Shane Smiskol
cd6d9fee3f
Revert "athenad: fix thread safety issues in upload handing" ( #34224 )
...
Revert "athenad: fix thread safety issues in upload handing (#34199 )"
This reverts commit dcb3113c4b .
2024-12-12 13:32:36 -08:00
Dean Lee
dcb3113c4b
athenad: fix thread safety issues in upload handing ( #34199 )
...
* fix thread safety issues in upload handing
* remove cancelled_uploads
* remove None from current upload items & atomic updates
2024-12-10 01:18:43 -08:00
Shane Smiskol
015aadd48c
Revert "athenad: fix thread safety issues in upload handing" ( #34198 )
...
Revert "athenad: fix thread safety issues in upload handing (#34084 )"
This reverts commit 7c101a40c8 .
2024-12-09 17:22:28 -08:00
Dean Lee
7ec6a47c1e
athenad: optimize network state check in upload callback ( #34185 )
...
optimize network state check in upload callback
2024-12-09 12:14:45 -08:00
Dean Lee
7c101a40c8
athenad: fix thread safety issues in upload handing ( #34084 )
...
* fix thread safety issues in upload handing
* remove cancelled_uploads
2024-12-06 20:14:12 -08:00
Dean Lee
43807746ff
athenad: fix memory leak by closing Response objects ( #34101 )
...
* fix memory leak by closing Response
* use with
2024-12-02 21:30:54 -08:00
Dean Lee
adb9560870
athenad: move last_scan outside the loop ( #34099 )
...
move last_scan outside the loop
2024-11-27 11:44:32 +01:00
Dean Lee
b737e8472f
athenad: explicitly delete socket in getMessage ( #34098 )
...
explicitly delete socket in getMessage
2024-11-27 00:35:36 -08:00
Dean Lee
957d39a5b6
athenad: close websocket before starting next loop iteration ( #34085 )
...
explicitly close websocket before starting next loop iteration
2024-11-23 20:35:42 -08:00
Adeeb Shihadeh
9ad2333546
athena: migrate old termux SSH port to new one
2024-10-16 10:59:58 -07:00
Adeeb Shihadeh
53f514cbd3
athena: update SSH port ( #33804 )
2024-10-16 09:57:57 -07:00
Adeeb Shihadeh
e04455cbaa
selfdrived: controlsd only does controls ( #33485 )
...
* selfdrived
* process replay
* lil more
* set the valids
* rename that
2024-09-06 17:16:32 -07:00
Adeeb Shihadeh
333c666a80
athena: use dongle ID on /persist/ when available ( #33259 )
...
* athena: use dongle ID on /persist/ when available
* comment
* test
* cleanup
old-commit-hash: 91db49b9bfb5ca27068bd13378feb602e8c63f4b
2024-09-01 10:39:49 -07:00
Maxime Desroches
df13b36638
fix all codespell issues ( #33194 )
...
* bring back
* remove 2020 dictionary
old-commit-hash: 44c7144e1c
2024-08-05 15:25:02 -07:00
Shane Smiskol
7b67994a05
use zstandard library ( #33110 )
...
* use zstandard
* run 10
* bye bye
* this was a little overzealous
* clean up
old-commit-hash: 1b3b5ab758
2024-07-28 02:03:47 -07:00
Shane Smiskol
b0a8b3bc9d
uploader: compress with zstd ( #32736 )
...
* zstd uploader
* fix that
* fix name of function
* comment
* log failed
* fix comma_api_source for routes with both bz2 and zst rlogs
* TODO
* 10-14 achieves almost no benefit on qlogs in a few cases, but takes 2x the time
* these aren't written out
* regen: specify any list of sources
ooh this is pretty nice
* regen and process replay
* damn, actually we don't need all this (cool tho)
Revert "regen: specify any list of sources"
This reverts commit ceb0b4abed9ad463a9fe98d9b98a05875a52806f.
* just let it auto resolve
* fix athenad/uploader tests
* zst here too
* TODOs
* yes
* Revert "TODOs"
This reverts commit 8c7da1dbd0340c72290b5eb5563b642080ddc131.
* Revert "zst here too"
This reverts commit 23b0023ddfd22c8090be7a7caa09e7026a12aa5c.
* Revert "just let it auto resolve"
This reverts commit f296d62424227ad05facc62abc18a6f81b474e84.
* Revert "regen and process replay"
This reverts commit 0768330e96974a42616d229d159780619d049cd0.
* revert readme
* not in save_log either
* lfg
* Revert "lfg"
This reverts commit 3718559c6c4de7d1f0c80dc9f1a1d335fe679a89.
old-commit-hash: 7dec7c39be
2024-07-26 19:33:35 -07:00
Shane Smiskol
5fdbd73ba3
athenad: log failed upload items
...
old-commit-hash: add958bb08
2024-07-24 14:48:33 -07:00
Adeeb Shihadeh
3b8ed67aa3
remove navigation ( #32773 )
...
* remove navigation
* lil more
* fix cabana
* revert for now
* fix docs
* move that over
* lock
* rm assets
old-commit-hash: e491c9d964
2024-06-19 19:14:13 -07:00
Mauricio Alvarez Leon
008008b03a
Update to Python 3.12 ( #32548 )
...
* 3.12
* pprofile is broken
* use modified metadrivepy3-12
* 0.3.0 metadrive
* add metadrive/commaai git dependency
* metadrive git set
* pin sounddevice 0.4.6
* datetime.utcnow() deprecation
* poetry lock
* make datetime not aware
* poetry lock
* pin pytools
* google_crc32c wheel
* unpin sounddevice
* clean metadrive
* use python crc
* mypy
* 3.12.4
* allow python3.11
* test
* no pip
* poetry
* better
* better
* merge
* remove
* try
* test
* try this
* snok
* python
* simpler
* setuptools
* lower
* try
* try
* work?
* ubuntu deps
* ubuntu
* try
* remove
* move
* remove this
* names
* Update .github/workflows/tools_tests.yaml
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
* python<4
* <3.13
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
old-commit-hash: 148eaf8fa6
2024-06-11 20:45:15 +00:00
Adeeb Shihadeh
86b6103657
test_*.py files are no longer executable ( #32610 )
...
* test_*.py files are no longer executable
* cleanup empty lines
* update precommit
* fixme
* enable
* build
old-commit-hash: c59887c810
2024-06-03 15:48:56 -07:00
Adeeb Shihadeh
cc4f53e6c6
manager: move to system/ ( #32538 )
...
* manager: move to system/
* one more
old-commit-hash: accdade4cf
2024-05-25 12:41:17 -07:00
Adeeb Shihadeh
3358745103
ruff: enable UP
...
old-commit-hash: b2e4c64cf8
2024-05-20 17:43:54 -07:00
Adeeb Shihadeh
18970afb0e
athena: move to system/ ( #32488 )
...
* athena: move to system/
* slash
old-commit-hash: 3279dbeff7
2024-05-20 14:59:43 -07:00