Commit Graph

126 Commits

Author SHA1 Message Date
Jake Poznanski
e708055ca2 loggerd: fix length of ArrayPtr in handle_encoder_msg (#26077)
old-commit-hash: c782e4d796
2022-10-17 17:22:47 -07:00
Dean Lee
e9f2954ee5 loggerd: remove 'enable' from struct LogCameraInfo (#25052)
remove enalbe
old-commit-hash: 568cc0f892
2022-07-07 10:54:56 -07:00
Dean Lee
1b725b5005 loggerd: remove rotate_lock (#24969)
remove lock
old-commit-hash: 748bbac344
2022-06-27 11:19:46 +02:00
Willem Melching
753045c73d less TICI when not needed (#24698)
* less TICI when not needed

* fix process replay

* move reading voltages into hw abstraction layer

* Update selfdrive/hardware/tici/hardware.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/hardware/hw.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/hardware/base.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* rename init function

* Update selfdrive/athena/athenad.py

Co-authored-by: Robbe Derks <robbe.derks@gmail.com>

* Update selfdrive/boardd/boardd.cc

* Apply suggestions from code review

* Update selfdrive/thermald/thermald.py

* update ref

* fix alert width if all cameras are bad

* add ecam to test_loggerd

* bump cereal

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Robbe Derks <robbe.derks@gmail.com>
old-commit-hash: f49a9c9fd2
2022-06-02 15:20:51 +02:00
George Hotz
109916df80 loggerd: Fix loggerd encode packet drops (#24599)
* hmm, try this

* hmm, try this

* rewrite handle_encoder_msg

* fix new logic

* add comments and an assert

* handle startup condition better

* handle restarts of encoderd

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 0000ad2ac0
2022-05-20 00:02:50 -07:00
George Hotz
a053d070be encoderd: enable (#24492)
* enable encoderd

* correct enable line

* fix loggerd tests

* fix power draw and cpu tests

* correct cpu for encoderd

* fix a bug, video_writer is shared

* fix issue with not recording dcam

* add recording state

* wooo tests pass. encode id keeps counting

* core 3

* loggerd then encoderd

* stop loggerd first

* core 3 always online

* see the camera when we see encoder packet

* encoderd on small core uses 37%

* remove encoder logic from loggerd

* delete unit test that doesn't really make sense anymore

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 94b9972eb7
2022-05-13 17:56:02 -07:00
George Hotz
eb9e57e070 encoderd: support remote encoder behind env var (#24490)
* remote encoder

* remote encoder on PC, fix type

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: bd0cc655fb
2022-05-10 16:53:47 -07:00
George Hotz
1f4628bad5 encoderd: make work on PC (#24483)
* don't use the codec in video_writer

* this produces broken videos for some reason

* bugfix

* refactor on the class

* works on device

* fix codec

* no codec enum

* fix pc

* move into dirs

* these includes also

* rename it ffmpegencoder

* add avcodec_close

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 2008db47c2
2022-05-10 15:20:51 -07:00
Willem Melching
be08693d5c delay bz2 compression from logging to uploading (#24392)
* remove log_name

* log without compression

* fix tests

* remove extension for bootlog

* another test fix

* uploader compresses

* also compress in athena

* only compress qlog

* more generic check in do_upload

* fix bootlog compression

* lower loggerd cpu usage

* dont link against bz2

* set core affinity to little cluster

* handle old files
old-commit-hash: 77a6f3d034
2022-05-03 14:09:17 +02:00
George Hotz
7119a98414 loggerd: switch to v4l encoder try 2 (#24380)
* start v4l encoder

* v4l encoder starts

* start and stop

* fill in proper controls

* it dequeued a buffer

* getting bytes

* it made a video

* it does make files

* getting close

* ahh, so that's how dequeue works

* qcam works (no remuxing)

* remuxing works

* we just need to make shutdown and rollover graceful

* graceful destruction

* switch to polling

* should work now

* fix pc build

* refactors, stop properly

* touchups, remove a copy

* add v4l encoder to release

* inlcude file

* move writing to it's own thread

* fix minor memory leak

* block instead of dropping frames

* add counter, fix tests maybe

* better debugging and test print

* print file path in assert

* format string in test

* no more oversized qlogs

* match qcam

* touchups, remove omx encoder

* remove omx include files

* checked ioctl, better debugging, open by name

* unused import

* move linux includes to third_party/linux/include

* simple encoderd

* full packet

* encoderd should be complete

* lagging print

* updates

* name dq thread

* subset idx

* video file writing works

* debug

* potential bugfix

* rotation works

* iframe

* keep writing support

* ci should pass

* loggerd, not encoderd

* remote encoder code

* support remote encoder

* cereal to master, add encoderd

* header no longer required

* put that back there

* realtime

* lower decoder latency

* don't use queue for VisionIpcBufExtra, disable realtime again

* assert all written

* hmm simpler

* only push to to_write if we are writing

* assert timestamp is right

* use at and remove assert

* revert to queue

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 0baa4c3e2a
2022-04-30 09:22:52 -07:00
George Hotz
6dde52671b loggerd: prereqs for deanlee loggerd omx encoder (try 2) (#24252)
* refactor encoders

* fix pc build

* buf_info

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 0ac35a656a
2022-04-18 18:21:46 -07:00
George Hotz
9720bb37b5 loggerd: broadcast encoder data (#24177)
* encode data try 1

* fix pc build

* low quality compressed streamer with pyav

* nvidia streamer

* timestamp support

* fix latency measurement

* refactor

* camerastream updates

* fix linter

* compressed to vipc, clean

* remove print

* touchups

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: dab978083b
2022-04-12 12:08:50 -07:00
George Hotz
0681a415e9 broadcast encode idx packets (#24176)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: cc8430b668
2022-04-10 15:12:58 -07:00
Adeeb Shihadeh
1c4b6e38e9 loggerd: less error logging
old-commit-hash: 652ef27a72
2022-01-24 21:15:14 -08:00
Dean Lee
3e72a81dac Use std::unique_ptr to manage memory (#23168)
* use unique_ptr

* trigger ci

* move down loggerdstate
old-commit-hash: 4aa291c406
2021-12-13 14:16:53 +01:00
Dean Lee
dd7c226097 util: move all functions into util namespace (#23203)
old-commit-hash: 9decd3d8a2
2021-12-12 14:42:23 -08:00
Adeeb Shihadeh
916c2ed588 loggerd: fix delayed exit from main thread
old-commit-hash: 5f36b48199
2021-12-07 18:22:00 -08:00
Adeeb Shihadeh
1ca1d7a673 increase loggerd per-cycle service limit to 200
old-commit-hash: f71a03f967
2021-12-06 22:08:17 -08:00
Adeeb Shihadeh
f8146db9b9 loggerd: don't let a single service clog up the main thread
old-commit-hash: baf1a77ca2
2021-12-06 19:31:11 -08:00
Willem Melching
3b8daea061 Clear loggerd locks once on boot (#23060)
* add test for clear_locks

* move to bootlog

* simplify test
old-commit-hash: a2f32fd3e1
2021-11-29 12:30:28 +01:00
Dean Lee
7f0beef092 loggerd: add test case for trigger_rotate (#23038)
* test rotate

* remove global LoggerdState
old-commit-hash: 1d323e0fd6
2021-11-26 14:53:09 +01:00
Dean Lee
987f87cc0e loggerd: split loggerd.cc into three files
old-commit-hash: 75dd0d6296
2021-11-23 17:53:49 -08:00
Adeeb Shihadeh
35bfdce477 loggerd: trigger rotate on frame id instead of frame count (#22848)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: ea761cbbd5
2021-11-13 15:38:10 -08:00
Mayfield
43e1903073 enable wshadow (#22756)
* enable wshadow (#22714)

* fix replay

* more build fixes

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 5246f0231e
2021-11-02 17:08:53 +01:00
Dean Lee
c401d7aff3 loggerd: fix concurrency issues in sync encoders (#22590)
* sync_encoders

* add variable camera_synced

* cleanup
old-commit-hash: 18caf3535b
2021-10-29 12:21:02 +02:00
Adeeb Shihadeh
bb7751fdb0 don't set core affinity on PC (#22706)
old-commit-hash: 5b331fd6f5
2021-10-26 21:51:46 -07:00
Willem Melching
0f77dd4506 loggerd: set encodeIdx.valid if frame id is correct (#22634)
* loggerd: set encodeIdx.valid if frame id is correct

* check valid flag in test

* test cleanup

* bump cereal
old-commit-hash: 89f311714c
2021-10-26 16:00:43 +02:00
Adeeb Shihadeh
b773f6711c loggerd: always run encoders (#22649)
* loggerd: always run encoders

* fix raw logger

* bump loggerd cpu usage
old-commit-hash: 76bd932cf5
2021-10-21 16:37:03 -07:00
Adeeb Shihadeh
25cfa2b65a tici: affine loggerd to efficiency cores (#22581)
old-commit-hash: 84de248fa7
2021-10-18 13:21:24 -07:00
Willem Melching
4844f8196d remove loggerd todo about encodeIdx decimation
old-commit-hash: 377fe84948
2021-09-08 12:16:16 -07:00
Willem Melching
e49dd50f98 Disable encoder sync logic if only one rotate camera present (#22107)
* disable sync logic if only one rotate camera

* consistent brackets
old-commit-hash: 48035e9535
2021-09-01 23:08:06 -07:00
Willem Melching
60e7e09127 Ensure that first segment starts with all cameras on the same frameId (#22096)
* test that segment starts with the same frame id

* improve test

* add sync logic

* atomic max

* review comments
old-commit-hash: a93271a518
2021-09-01 14:28:50 -07:00
Dean Lee
ecf3ee7603 loggerd: create encoder threads from cameras_logged (#21875)
old-commit-hash: d9baaf7eb9
2021-08-30 19:33:00 -07:00
Adeeb Shihadeh
277bd43b15 Revert "tici: give loggerd rt priority (#21503)"
This reverts commit 0a50cf851e79342cf3db01033bc91516c504cbea.

old-commit-hash: 54a4eeb0fc
2021-08-22 16:57:53 -07:00
Adeeb Shihadeh
5c1d1ebe76 tici: give loggerd rt priority (#21503)
* give loggerd rt priority

* assert

* little more

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 7e2752c878
2021-08-20 21:27:31 -07:00
Willem Melching
ae4f3c98d5 refactor loggerd: trigger rotate in encoder thread (#21860)
* trigger rotate in eoncode thread

* rotate in time

* lgtm

* dcam trigger rotate on C3

* check trigger rotate field

* Use >=

* add rotator thread

* set cnt to 0

* test encodeIdx is increasing across segments

* test both segmentId and encodeId

* fix encodeIdx

* no thread needed

* add log for failed to encode

* stricter C3 test, dont check start encodeId on C2 dcam

* only update last_camera_seen_tms when considered for rotate

Co-authored-by: deanlee <deanlee3@gmail.com>
old-commit-hash: a39873872d
2021-08-06 11:10:33 +02:00
Willem Melching
f8edcd64c4 loggerd: power failure print sync done
old-commit-hash: dd128f5916
2021-07-12 12:28:07 +02:00
Adeeb Shihadeh
d31397b84b segment regen (#21148)
* start seg regen

* send vipc

* fix up some paths

* set fp

* fix no entries

* engages

* seperate camera procs

* send real frames

* regen test routes

* nice argparser

* fix valgrind test

* move that

* fix that
old-commit-hash: 4a1aec27ef
2021-06-28 21:44:40 -07:00
Dean Lee
147b355dde add space after function name and if statement (#21225)
old-commit-hash: e4e669bb0f
2021-06-11 10:17:52 +02:00
Dean Lee
f70a79b838 Use C++ header files instead of C header files. (#21192)
* use cstring instead of string.h

* use cstdio instead of stdio.h

* remove inttypes.h

* use cstdlib instead of stdlib.h

* use cstdint instead of stdint.h

* #include <cstddef>

* cstdlib

* use cmath

* remove stddef.h

* use cassert

* use csignal

* use ctime

* use cerror

* rebase master
old-commit-hash: c53cb5d570
2021-06-08 13:46:22 -07:00
Willem Melching
80ec32bcf3 loggerd: put signal in end of route sentinel (#21025)
old-commit-hash: 6e22f16ddb
2021-05-25 17:34:40 +02:00
Dean Lee
e333e4f189 Cleanup selfdrive/ includes (#20822)
* cleanup include path

* continue

* format includes

* fix testraw.cc

* remove include path from SConstruct

* regroup

* rebase master

* almost done

* apply review

* rename FileReader.xx to filereader.xx

* rename Unlogger.x->unlogger.x

* rename FrameReader.xx -> framereader.xx

* apply reviews

* ui.h

* continue

* fix framebuffer.cc build error:mv util.h up

* full path to msm_media_info

* fix qcom2 camerad

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 7222d0f20d
2021-05-08 22:15:17 -07:00
Willem Melching
78bf4f5867 Replace ifdef with hw abstraction layer (#20843)
* Replace ifdefs with hardware abstraction layer (#20801)

* add type to class hadwareXXX

* replace ifdefs with hardware layer

* continue

* continue

* new function get_driver_view_transform

* full path to hw.h

* fix build error setup.cc

* apply review

* fix typo

* fix deprecated error:replace deprecated fromPath with new

* fix build error

* Fixes after ifdef clenaup (#20842)

* inheritance doesnt work with static

* fix debayer

* small cleanup

* Update selfdrive/camerad/cameras/camera_common.cc

* Update selfdrive/ui/qt/offroad/settings.cc

* Update selfdrive/common/modeldata.h

* flip conditions

* fix comment

Co-authored-by: Dean Lee <deanlee3@gmail.com>
old-commit-hash: ab319d4f54
2021-05-06 14:39:05 +02:00
Willem Melching
39c18d4c5a Revert "Replace ifdefs with hardware abstraction layer (#20801)"
This reverts commit 9f2ad462f39f4c880f283b71bffbc9248a2b577d.

old-commit-hash: 3a95d3ccaa
2021-05-06 12:29:30 +02:00
Dean Lee
e0282ced49 Replace ifdefs with hardware abstraction layer (#20801)
* add type to class hadwareXXX

* replace ifdefs with hardware layer

* continue

* continue

* new function get_driver_view_transform

* full path to hw.h

* fix build error setup.cc

* apply review

* fix typo

* fix deprecated error:replace deprecated fromPath with new

* fix build error
old-commit-hash: 757d2923d2
2021-05-06 11:41:46 +02:00
Adeeb Shihadeh
55a9ec8a93 set nice values for non RT processes (#20812)
* set nice values for non RT processes

* builds

* fix qcom build

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: ec414e2eb1
2021-05-04 22:00:11 -07:00
Adeeb Shihadeh
532d19427c bump cereal
old-commit-hash: 8a280fbb8e
2021-05-03 20:47:11 -07:00
Adeeb Shihadeh
467811edf1 log encodeIdx packets in qlog (#20738)
old-commit-hash: 9813c0ddbd
2021-04-23 16:14:45 -07:00
Dean Lee
405df8b165 Params: python-like interface (#20506)
* rebase master

* delete outdated test_params.c

* putBool & more robust getBool

* putBool(SshEnabled)
old-commit-hash: 98e55996f6
2021-03-30 12:54:59 +02:00
Willem Melching
c64d818341 use class AlignedBuffer (#20388)
* use class AlignedBuffer from cereal (#20343)

* AlignedBuffer

* move AlignedBuffer to cereal/messaging.hpp

* rename function aligned to get

* use overloaded function AlignedBuffer:: get(Message *)

* use AlignedBuffer

Co-authored-by: Dean Lee <deanlee3@gmail.com>
old-commit-hash: 0a88304e8d
2021-03-17 14:43:23 +01:00