Commit Graph

169 Commits

Author SHA1 Message Date
Adeeb Shihadeh
954b567b9b merge a bunch of misc stuff into common.utils (#36463)
just utils
2025-10-24 20:45:56 -07:00
Shane Smiskol
3ef5037c16 uploader: fix env var parsing 2025-10-20 11:40:03 -07:00
kostas.pats
dcc5afa8fa improve webrtc stack for use in camera focusing (#36268)
* made LiveStreamVideoStreamTrack use system time to calculate pts

* fixes as requested

* Align panda submodule with master (panda@615009c)

* made loggerd accept a run time env variable to pick stream bitrate

* added /notify endpoint to send json to all session's data channel

* fixed static analysis error

* adapted webrtc stream test to new pts calculation method

* fixed static erro

* fixed wrong indent

* fixed import order

* delete accidental newline

* remove excess spaces

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* remove excess spaces

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* changed exeption handling based on review

* fixed typo on exception handling

---------

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-10-08 22:30:32 -07:00
Adeeb Shihadeh
cf5b743de6 build system cleanups (#36202)
* it's all common

* never getting fixed

* it's just tici

* reorders

* qcom2 -> tici

* Revert "qcom2 -> tici"

This reverts commit f4d849b2952cb0e662975805db6a1d32511ed392.

* Reapply "qcom2 -> tici"

This reverts commit 58b193cb8de872830f8a7821a339edca14e4a337.

* is tici

* lil more

* Revert "is tici"

This reverts commit a169be18d3fdcb3ef8317a63a89d8becadabfad8.

* Revert "Reapply "qcom2 -> tici""

This reverts commit 26f9c0e7d068fc8a1a5f07383b3616e619cd4e8c.

* qcom2 -> __tici__

* lil more

* mv lenv

* clean that up

* lil more]

* fix

* lil more
2025-09-25 20:55:14 -07:00
Shane Smiskol
c7a37c06d8 Revert "Capnp memoryview (#36163)"
This reverts commit 6ed8f07cb6.

bump
2025-09-19 17:18:08 -07:00
Shane Smiskol
6ed8f07cb6 Capnp memoryview (#36163)
* lock

* bump opendbc

* fix one

* and

* Add a memoryview fallback in webrtcd

* fix

* revert

* rerevert

* bump to master

---------

Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
2025-09-19 16:46:23 -07:00
Mitchell Goff
086e33dd6e Revert "minimal ffmpeg build (#36138)"
This reverts commit 347b23055d.
2025-09-16 14:25:18 -07:00
Adeeb Shihadeh
347b23055d minimal ffmpeg build (#36138)
* min ffmpeg

* remove avfilter

* x264

* merge x264

* simpler

* pin x264

* mac

* rm that

* lil more

* move includes to lfs

* try this

* cleanup

* larch

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-09-12 18:59:15 -07:00
Comma Device
b501ad4d51 nice encoder debugging script 2025-08-24 00:48:54 +00:00
ZwX1616
8b90c210f8 encoderd: more efficient compression for low res frames (#35924)
* shein says inline

* Update system/loggerd/loggerd.h

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Revert "Update system/loggerd/loggerd.h"

This reverts commit 3602523cefdeb2a46d77946f7f2cc7fc21bd5a4f.

* Revert "shein says inline"

This reverts commit d3c079e137c5d98068501df636975c5fbf8810ee.

* EncoderSettings

* getter

* update test_encoder

* def

---------

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
2025-08-06 21:17:10 -07:00
ZwX1616
a84089c6e5 EncoderInfo: encoder setting factorys (#35940) 2025-08-06 16:53:16 -07:00
Jimmy
d7b0a5fa7e Record feedback with LKAS button (#35888)
* record feedback with LKAS button

* fix alert test

* slightly simplify feedbackd

* "Audio Feedback Saved" upon time expiration or early stop

* earlySend --> earlyStop

* userFlag --> userBookmark

* RecordAudioFeedback param/toggle

* add audioFeedback test

* simplify feedbackd

* send bookmark regardless of toggle, show feedback event with higher priority

* add userBookmark to selfdrived sm

* fix mispelled param name

* default off and move to main

* segmentNum --> blockNum, earlyStop --> lastBlock

* preserve audioFeedback

* get rid of lastBlock and just send bookmark saved at the end

* update raylib side

* update toggle description and add raylib toggle

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-08-05 13:41:41 -07:00
pencilpusher
be0626f7e3 improved safe_ioctl (#35908)
* improved safe_ioctl

* readability

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* use correct ioctl command

* ameliorated api

* use try/catch to prevent spi_fd leak

* Update common/util.h

* use correct ioctl command

* error log message is more readable

---------

Co-authored-by: Test User <test@example.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-08-04 13:25:24 -07:00
Maxime Desroches
e7b80b78cc params: auto type cast on put (#35810)
* start

* fix

* fix

* more

* more

* more

* fix

* fix

* []

* f

* f

* fix

* lint

* back

* fix

* yep

* better msg

* fix

* fix

* fix

* fix

* more

* more
2025-07-25 19:20:49 -07:00
Maxime Desroches
bc5336d805 params: auto decode based on type (#35794)
* type

* test

* more

* might as well use this

* one more

* live

* athena

* b

* also

* more

* now

* ah

* pigeon
2025-07-22 21:58:06 -07:00
Maxime Desroches
58d3038241 auto type cast for Params (#35786)
* first

* fix

* fix

* this

* real

* time

* time

* more

* more

* fix

* subclass

* fix

* fix

* test

* fix

* ruff

* revert

* default

* json

* more

* test

* test

* d

* better

* better

* test

* one

* test

* space
2025-07-22 19:30:47 -07:00
Adeeb Shihadeh
a8fd55740b Remove NVMe logging (#35740) 2025-07-17 10:05:27 -07:00
Dean Lee
006c06027b loggerd: fix audio truncation by processing remaining partial frames (#35718)
fix audio truncation by processing remaining partial frames
2025-07-15 09:28:15 -07:00
Dean Lee
203b2e75d3 loggerd: remove deprecated av_init_packet usage (#35733)
remove deprecated av_init_packet
2025-07-15 09:27:12 -07:00
Jimmy
c284edcd33 test: fix test_rotation in test_loggerd.py (#35658)
fix test_rotation
2025-07-08 14:20:34 -07:00
Jimmy
de16c6fbe1 test: RecordFront and RecordAudio (fixed) (#35654)
* record front and record audio tests

* set tests to same group so they are sequential

* add comment
2025-07-08 13:20:00 -07:00
Jimmy
684f770435 Revert "test: RecordFront and RecordAudio" (#35652)
Revert "test: RecordFront and RecordAudio (#35650)"

This reverts commit 1b92dbb46f.
2025-07-07 21:14:22 -07:00
Jimmy
1b92dbb46f test: RecordFront and RecordAudio (#35650)
record front and record audio tests
2025-07-07 21:10:37 -07:00
Jimmy
a4e4a8afef include audio in qcamera.ts (#35608)
* encode/store audio as part of video file

* better match write_audio() with write()

* handle different FFmpeg versions, flush audio encoder, suppress encoder QAvg/info messages

* use audio_buffer.size() instead of keeping track of size separately

* no more for loops

* save to qcam and rlog

* assert audio support check

* microphone --> soundPressure, audioData --> rawAudioData

* deque much more efficient if buffer ever >> frame_size, ~ same performance for defaults

* cleanup and fix time scaling

* initialize audio separately and pass sample_rate in

* update comments

* ensure header is written before writing audio

* buffer audio frame but do not process before header written

* handle buffer overflow now that we are using as an actual buffer

* spelling
2025-07-05 11:03:13 -07:00
Jimmy
dcd56ae09a store mic audio with toggle (#35595)
* store/send mic audio with toggle

* script to extract audio from logs

* change description and add translation placeholders

* microphone icon

* apply toggle in loggerd

* add legnth and counter

* startFrameIdx counter

* Revert "change description and add translation placeholders"

This reverts commit 7baa1f6de99c6ebe9f9906193da7e83dad79511a.

* send mic data first and then calc

* restore changed description/icon after revert

* adjust fft samples to keep old time window

* remove extract_audio.py since audio is now stored in qcam isntead of rlog

* qt microphone recording icon

* Revert "remove extract_audio.py since audio is now stored in qcam isntead of rlog"

This reverts commit 7a3a75bd8db5376d1e442a3ba931c67550b5f132.

* move extract_audio script and output file by default

* remove length field

* recording indicator swaps sides based on lhd/rhd

* use record icon from comma body

* Update toggle description

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* update raylib toggle desc cause I did earlier

* microphone --> soundPressure, audioData --> rawAudioData

* cleanup unused var

* update README

* sidebar mic indicator instead of annotated camera

* improve logic readability

* remove startFrameIdx and sequenceNum

* use Q_PROPERTY/setProperty so that update() is actually called on value change

* specify old id for SoundPressure

* fix typo

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-06-30 13:42:21 -07:00
Dean Lee
33849245d8 uploader.py: fix empty string handing in AthenadRecentlyViewedRoutes parameter (#35139)
Fix empty strings in AthenadRecentlyViewedRoutes parameter
2025-05-08 16:31:06 +01:00
Maxime Desroches
47ed90c6cf Reapply "Mypy: Got passing on macos (#34591)" (#35126) (#35153)
* Mypy: Got passing on macos (#34591)

* Mypy: Got mypy passing on macos

* common/realtime.py refactor

* Mypy: mypy passing on darwin

* Refactor: Removed else: pass statement

* Refactor: Removed unnecessary check

* added xattr to pyproject

* loggerd: switched to xatter module

* loggerd: removed unused module in xattr_cache.py

* UV: update uv.lock

* Update system/athena/athenad.py



* athenad: fixed blank lines

* loggerd: refactor of xattr_cache

* cleanup

---------



* fix getxattr no attribute on macOS

* try fixing missing ENOATTR on Linux

---------

Co-authored-by: Andrei Radulescu <andi.radulescu@gmail.com>
Co-authored-by: BrainLess <116778989+BrainLessPea@users.noreply.github.com>
2025-05-07 19:11:37 -07:00
Dean Lee
7c16e65347 loggerd: remove redundant Params Construction (#35138)
remove redundant Params Construction
2025-05-07 16:39:05 +01:00
Maxime Desroches
aa1b790708 revert agnos 12 2025-05-06 21:38:50 -07:00
Andrei Radulescu
bbf37ae5c7 Reapply "Mypy: Got passing on macos (#34591)" (#35126)
* Mypy: Got passing on macos (#34591)

* Mypy: Got mypy passing on macos

* common/realtime.py refactor

* Mypy: mypy passing on darwin

* Refactor: Removed else: pass statement

* Refactor: Removed unnecessary check

* added xattr to pyproject

* loggerd: switched to xatter module

* loggerd: removed unused module in xattr_cache.py

* UV: update uv.lock

* Update system/athena/athenad.py

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* athenad: fixed blank lines

* loggerd: refactor of xattr_cache

* cleanup

---------

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* fix getxattr no attribute on macOS

* try fixing missing ENOATTR on Linux

---------

Co-authored-by: BrainLess <116778989+BrainLessPea@users.noreply.github.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-05-06 21:09:59 -07:00
Adeeb Shihadeh
b33342c18f loggerd: limit encoded queue size (#34837)
* loggerd: limit encoded queue size

* less spammy
2025-03-11 10:22:49 -07:00
Dean Lee
a0a9df2d55 loggerd: enhance ZstdFileWriter test coverage with variable data sizes (#34812)
enhance ZstdFileWriter test coverage
2025-03-07 10:23:46 -08:00
Dean Lee
2c2c6e6437 deleter: fix duplicate segments in get_preserved_segments return value (#34795)
fix duplicate segments in get_preserved_segments return value
2025-03-06 10:16:56 -08:00
Dean Lee
c52112ad4e loggerd: eliminate recursion from handle_encoder_msg (#33453)
* refactor handle_encoder_msg

* removing the recursion only

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-03-02 20:01:21 -08:00
Adeeb Shihadeh
54f1b00447 Log git source commit (#34749)
* Log git source commit

* strip

* fix trim

* no trim
2025-03-02 13:11:38 -08:00
Dean Lee
0619e9a581 v4l_encoder: used encoder_info.fps instead of hardcoded 20 (#34701)
Used encoder_info.fps instead of hardcoded 20
2025-02-24 07:06:38 -08:00
Adeeb Shihadeh
e00b5337d2 misc encoder cleanups 2025-02-21 10:41:30 -08:00
Adeeb Shihadeh
6f5f8e7746 loggerd: reduce debug log frequency 2025-02-17 15:06:16 -08:00
Adeeb Shihadeh
27b5a727e3 Revert "Mypy: Got passing on macos (#34591)"
This reverts commit b09b48130e.
2025-02-16 08:50:46 -08:00
BrainLess
b09b48130e Mypy: Got passing on macos (#34591)
* Mypy: Got mypy passing on macos

* common/realtime.py refactor

* Mypy: mypy passing on darwin

* Refactor: Removed else: pass statement

* Refactor: Removed unnecessary check

* added xattr to pyproject

* loggerd: switched to xatter module

* loggerd: removed unused module in xattr_cache.py

* UV: update uv.lock

* Update system/athena/athenad.py

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* athenad: fixed blank lines

* loggerd: refactor of xattr_cache

* cleanup

---------

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-02-16 00:05:52 -08:00
Dean Lee
644130ca3d encoderd: use AV_CODEC_ID_H264 codec for qcamera encoding (#34498) 2025-02-11 12:46:43 +00:00
Dean Lee
3c0990e716 move thumbnail generation from camerad to encoderd (#34554)
Co-authored-by: Cameron Clough <cameronjclough@gmail.com>
2025-02-10 23:43:46 +00:00
Dean Lee
27c11eb597 athena upload: streaming file upload (#34559)
streaming file upload
2025-02-10 13:57:31 -08:00
Dean Lee
4b7a25efc3 loggerd: remove unused zlib and libswscale linkage (#34552)
remove unused link
2025-02-09 09:13:58 -08:00
Dean Lee
9ec54f59c6 loggerd: switch logging from raw file format to zstd compressed files (#34549)
* switch logging from raw file format to zstd compressed files

* more zst suffix

* compress bootlog

* remove class RawFile

* Optimize ZstdFileWriter by adding input caching

* use ZSTD_compressStream2

* cleanup

* LOG_COMPRESSION_LEVEL=10

* space

* add zst suffix to LOGS_SIZE_RATE
2025-02-08 12:55:31 -08:00
Adeeb Shihadeh
9bc35c0919 Revert "athena upload: reduce memory usage and improve efficiency with streaming (#34528)"
This reverts commit 4c65f51a55.
2025-02-06 11:50:04 -08:00
Dean Lee
4c65f51a55 athena upload: reduce memory usage and improve efficiency with streaming (#34528)
* reduce memory usage and improve upload efficiency with streaming

* space
2025-02-04 13:10:13 -08:00
Dean Lee
e7b63dce65 encoderd: enhance robustness of open() with HANDLE_EINTR Macro (#34478)
HANDLE_EINTR
2025-01-27 19:46:43 -08:00
Dean Lee
3219fe1472 encoder: optimize MessageBuilder data handling by adopting encoded data (#34391)
optimize data handing by adopting external data
2025-01-19 16:22:40 -08:00
Dean Lee
47b13f54f6 encoderd: refactor VideoEncoder::publisher_publish to standardize member variable access (#34342)
remove redundant pointer parameter
2025-01-12 11:38:20 -08:00