Commit Graph

305 Commits

Author SHA1 Message Date
Jake Poznanski
c782e4d796 loggerd: fix length of ArrayPtr in handle_encoder_msg (#26077) 2022-10-17 17:22:47 -07:00
Kurt Nistelberger
dc63245b89 reset LaikadEphemeris after loggerd test 2022-10-03 20:47:00 -07:00
Dean Lee
f4a4ec8fa2 bootlog: rename bz_file to file (#25881) 2022-09-23 13:28:25 -07:00
Dean Lee
d4404511bc encoderd: build v4l_encoder.cc on PC (#25872) 2022-09-22 10:41:09 -07:00
Dean Lee
3b9add5602 V4LEncoder: remove unused member variable buf_in (#25870) 2022-09-22 10:40:41 -07:00
Adeeb Shihadeh
8b741261cf loggerd: add params test cases (#25843) 2022-09-19 14:06:03 -07:00
George Hotz
aea1a0d5e6 port OX03C10 (#25161)
* start porting OS04C10

* something

* is the pll why this doesn't work

* understand plls, still doesn't work

* work

* weee something

* working?

* timing

* encoderd works with less cameras

* 1928

* flip image

* color balance

* adjust black level

* extra height maybe

* cleanups

* clean up sensor i2c

* sensors

* approx 20fps

* remove mirror

* 50ms, no fsin yet

* remove imx390 support

* some attempt at AE

* linearize the response

* OX changes (#25697)

* organize

* wb

* best effort but probably messed up by artifacts

* multicam sync

Co-authored-by: Comma Device <device@comma.ai>

* clean up

* more

* fix multiexposure

* start porting OS04C10

* something

* is the pll why this doesn't work

* understand plls, still doesn't work

* work

* weee something

* working?

* timing

* encoderd works with less cameras

* 1928

* flip image

* color balance

* adjust black level

* extra height maybe

* cleanups

* clean up sensor i2c

* sensors

* approx 20fps

* remove mirror

* 50ms, no fsin yet

* remove imx390 support

* some attempt at AE

* linearize the response

* OX changes (#25697)

* organize

* wb

* best effort but probably messed up by artifacts

* multicam sync

Co-authored-by: Comma Device <device@comma.ai>

* clean up

* more

* fix multiexposure

* unbroken AE

* oops

* sort of good AE

* both should work 1

* open then init

* fix debayer

* rename

* fix test build

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: ZwX1616 <zwx1616@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-09-09 22:38:00 -07:00
Jeroen
502a392199 bootlog: only do nvme checks on tici (#25634) 2022-09-01 19:05:15 -07:00
Adeeb Shihadeh
6590fb2b93 pre-commit: add codespell (#25571) 2022-08-30 11:20:55 -07:00
Dean Lee
aece21c7cb V4LEncoder: set bytesused to buf->len (#25050) 2022-08-25 16:57:34 -07:00
Dean Lee
ed70a9ab1d v4l_encoder: free buf_out in destructor (#25044) 2022-08-21 14:03:44 -07:00
Adeeb Shihadeh
aa32ea0f64 loggerd: log all disk space usage in initData (#25455)
* bootlog: log all disk space usage

* not just agnos

* move to initData

* cleanup
2022-08-16 16:37:58 -07:00
Adeeb Shihadeh
df251ef50e Fix master-ci dirty working tree + CI test (#25087)
* check

* test for dirty working tree

* swap order

* fix diff
2022-07-08 13:00:43 -07:00
Dean Lee
568cc0f892 loggerd: remove 'enable' from struct LogCameraInfo (#25052)
remove enalbe
2022-07-07 10:54:56 -07:00
Dean Lee
fd5b3d7603 move replay from selfdrive/ui/replay to tools/replay (#24971)
* mv to tools/replay

* change folder

* add .gitignore

* fix build doc

* disable warning

* enable warning after build

* build qt/util.cc qt/api.cc to library

* cleanup
2022-06-28 16:12:42 +02:00
Dean Lee
0aa9ae21d4 FfmpegEncoder: free codec_ctx in encoder_close (#24967)
free context
2022-06-27 11:20:13 +02:00
Dean Lee
748bbac344 loggerd: remove rotate_lock (#24969)
remove lock
2022-06-27 11:19:46 +02:00
Dean Lee
461f747247 logger.cc: remove unused function append_property (#24966)
remove append_property
2022-06-27 11:18:56 +02:00
Adeeb Shihadeh
6123ab3d1c Move camerad to system/ (#24836)
* mv camerad

* add hardware symlink

* fix unit tests
2022-06-19 14:43:49 -07:00
Adeeb Shihadeh
0fce5d9045 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
2022-06-11 23:19:27 -07:00
Adeeb Shihadeh
1139fe507b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
2022-06-11 16:38:24 -07:00
Willem Melching
f49a9c9fd2 less TICI when not needed (#24698)
* less TICI when not needed

* fix process replay

* move reading voltages into hw abstraction layer

* Update selfdrive/hardware/tici/hardware.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/hardware/hw.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/hardware/base.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* rename init function

* Update selfdrive/athena/athenad.py

Co-authored-by: Robbe Derks <robbe.derks@gmail.com>

* Update selfdrive/boardd/boardd.cc

* Apply suggestions from code review

* Update selfdrive/thermald/thermald.py

* update ref

* fix alert width if all cameras are bad

* add ecam to test_loggerd

* bump cereal

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Robbe Derks <robbe.derks@gmail.com>
2022-06-02 15:20:51 +02:00
George Hotz
ea5b8cdfb1 nv12: encoderd avoids a full frame copy (#24519)
* rgb to nv12

* nv12 works (w memcpy)

* correct now

* no copy

* fix nv12 with fast debayer

* reverts of unused stuff

* ui use nv12

* comment out thumbnails for now

* rebase fix

* dm read nv12

* model read nv12

* fix ffmpeg encoder

* thumbnails from nv12

* replay to nv12

* python framereader support nv12

* remove hardcoded frame/buffer sizes

* fix build

* ffmpeg encoder fix buffers

* small cleanup

* reduce power usage test

* fix cpu usage test

* fix snapshot

* fix loggerd test

* bump cereal

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
2022-06-01 17:18:28 +02:00
Adeeb Shihadeh
32a9ccfc37 visionipc python shim (#24660) 2022-05-25 20:49:56 -07:00
Adeeb Shihadeh
eca265e07b uploader: increase max uncompressed qlog size to 100MB 2022-05-23 13:43:14 -07:00
Willem Melching
1cc652602f Small fixes for macOS build (#24620)
* Small fixes for macOS build

* remove gpu_libs
2022-05-23 13:46:48 +02:00
George Hotz
0000ad2ac0 loggerd: Fix loggerd encode packet drops (#24599)
* hmm, try this

* hmm, try this

* rewrite handle_encoder_msg

* fix new logic

* add comments and an assert

* handle startup condition better

* handle restarts of encoderd

Co-authored-by: Comma Device <device@comma.ai>
2022-05-20 00:02:50 -07:00
Adeeb Shihadeh
d87f30ea06 encoder test speedup (#24601)
Co-authored-by: Comma Device <device@comma.ai>
2022-05-19 22:33:17 -07:00
Adeeb Shihadeh
cb8885cffb Merge common/ and selfdrive/common (#24556)
* Merge common/ and selfdrive/common

* fix that

* fix version

* fix unit tests
2022-05-18 14:11:57 -07:00
George Hotz
c2279ef39d add unixTimestampNanos to encoderd (#24583)
Co-authored-by: Comma Device <device@comma.ai>
2022-05-18 12:58:52 -07:00
George Hotz
3718f9a6e0 encoderd: fix bytesused (#24553)
fix bytesused

Co-authored-by: Comma Device <device@comma.ai>
2022-05-16 16:58:19 -07:00
Adeeb Shihadeh
81dacbedca common: replace custom xattr implementation with os module's (#24448)
* use os module's xattr function

* fix that

* handle in helper
2022-05-15 17:13:50 -07:00
Adeeb Shihadeh
5a58ebf183 loggerd: adjust qcamera threshold, seemed to be tuned for comma two 2022-05-14 01:04:46 -07:00
George Hotz
94b9972eb7 encoderd: enable (#24492)
* enable encoderd

* correct enable line

* fix loggerd tests

* fix power draw and cpu tests

* correct cpu for encoderd

* fix a bug, video_writer is shared

* fix issue with not recording dcam

* add recording state

* wooo tests pass. encode id keeps counting

* core 3

* loggerd then encoderd

* stop loggerd first

* core 3 always online

* see the camera when we see encoder packet

* encoderd on small core uses 37%

* remove encoder logic from loggerd

* delete unit test that doesn't really make sense anymore

Co-authored-by: Comma Device <device@comma.ai>
2022-05-13 17:56:02 -07:00
grekiki
17e33978cd Mypy fixes for --check-untyped-defs (#24372)
more type fixes
2022-05-12 14:08:11 +02:00
Willem Melching
75f2149267 encoderd: update deprecated ffmpeg functions (#24501) 2022-05-12 12:43:07 +02:00
George Hotz
bd0cc655fb encoderd: support remote encoder behind env var (#24490)
* remote encoder

* remote encoder on PC, fix type

Co-authored-by: Comma Device <device@comma.ai>
2022-05-10 16:53:47 -07:00
George Hotz
2008db47c2 encoderd: make work on PC (#24483)
* don't use the codec in video_writer

* this produces broken videos for some reason

* bugfix

* refactor on the class

* works on device

* fix codec

* no codec enum

* fix pc

* move into dirs

* these includes also

* rename it ffmpegencoder

* add avcodec_close

Co-authored-by: Comma Device <device@comma.ai>
2022-05-10 15:20:51 -07:00
Willem Melching
ba89faa5f7 uploader: ensure requests.put gets file like object (#24462) 2022-05-07 15:42:47 +02:00
George Hotz
ac343433a2 sync qcam p frames (#24459)
Co-authored-by: Comma Device <device@comma.ai>
2022-05-06 21:47:58 -07:00
Joost Wooning
df8f024e19 uploader: size limit for automatic uploading of qlogs/qcams (#24403)
* uploader: size limit for automatic uploading of qlogs/qcams

* move check to add logging

* use constant

* mark as uploaded

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-05-06 11:47:58 +02:00
Adeeb Shihadeh
66fbce638f uploader: compress rlogs (#24447) 2022-05-05 19:13:06 -07:00
Willem Melching
77a6f3d034 delay bz2 compression from logging to uploading (#24392)
* remove log_name

* log without compression

* fix tests

* remove extension for bootlog

* another test fix

* uploader compresses

* also compress in athena

* only compress qlog

* more generic check in do_upload

* fix bootlog compression

* lower loggerd cpu usage

* dont link against bz2

* set core affinity to little cluster

* handle old files
2022-05-03 14:09:17 +02:00
George Hotz
0baa4c3e2a loggerd: switch to v4l encoder try 2 (#24380)
* start v4l encoder

* v4l encoder starts

* start and stop

* fill in proper controls

* it dequeued a buffer

* getting bytes

* it made a video

* it does make files

* getting close

* ahh, so that's how dequeue works

* qcam works (no remuxing)

* remuxing works

* we just need to make shutdown and rollover graceful

* graceful destruction

* switch to polling

* should work now

* fix pc build

* refactors, stop properly

* touchups, remove a copy

* add v4l encoder to release

* inlcude file

* move writing to it's own thread

* fix minor memory leak

* block instead of dropping frames

* add counter, fix tests maybe

* better debugging and test print

* print file path in assert

* format string in test

* no more oversized qlogs

* match qcam

* touchups, remove omx encoder

* remove omx include files

* checked ioctl, better debugging, open by name

* unused import

* move linux includes to third_party/linux/include

* simple encoderd

* full packet

* encoderd should be complete

* lagging print

* updates

* name dq thread

* subset idx

* video file writing works

* debug

* potential bugfix

* rotation works

* iframe

* keep writing support

* ci should pass

* loggerd, not encoderd

* remote encoder code

* support remote encoder

* cereal to master, add encoderd

* header no longer required

* put that back there

* realtime

* lower decoder latency

* don't use queue for VisionIpcBufExtra, disable realtime again

* assert all written

* hmm simpler

* only push to to_write if we are writing

* assert timestamp is right

* use at and remove assert

* revert to queue

Co-authored-by: Comma Device <device@comma.ai>
2022-04-30 09:22:52 -07:00
George Hotz
8d24655787 loggerd: Separate writer from encoder (#24257)
* seperate writer from encoder

* video writer in release

* confirm loggerd on PC works, add YUV option to compressed_vipc

* make raw_logger use video_writer

* put this back to master

* close codec

* put this back

* use unique_ptr

Co-authored-by: Comma Device <device@comma.ai>
2022-04-20 14:51:24 -07:00
George Hotz
1795a2ac03 C2: remove some dead code (#24256)
* remove some dead code

* still needed

Co-authored-by: Comma Device <device@comma.ai>
2022-04-18 20:38:03 -07:00
George Hotz
0ac35a656a loggerd: prereqs for deanlee loggerd omx encoder (try 2) (#24252)
* refactor encoders

* fix pc build

* buf_info

Co-authored-by: Comma Device <device@comma.ai>
2022-04-18 18:21:46 -07:00
Adeeb Shihadeh
5c48e7bc86 remove comma two support (#24248)
* remove comma two support

* cleanup release files

* little more

* more libs

* no more gralloc

* add snpe back
2022-04-18 17:55:23 -07:00
Comma Device
df7275f1f7 encodeData: add flags support 2022-04-16 15:54:22 -07:00
George Hotz
35e776d2fd loggerd: add qcam broadcast support, don't recreate pubmaster (#24226)
* add qcam broadcast support, don't recreate pubmaster

* fixed

Co-authored-by: Comma Device <device@comma.ai>
2022-04-16 14:42:36 -07:00