Files
dragonpilot/selfdrive/loggerd
Dean Lee 2b4a477fbc replay: improve segment download and merge (#22654)
* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
2021-11-01 11:55:56 +01:00
..
2020-09-22 11:37:24 +02:00
2021-08-30 17:12:09 -07:00
2021-09-24 10:23:41 +02:00
2021-09-10 12:05:12 -07:00
2020-09-22 11:37:24 +02:00

loggerd

openpilot records routes in one minute chunks called segments. A route starts on the rising edge of ignition and ends on the falling edge.

Check out our python library for reading openpilot logs. Also checkout our tools to replay and view your data. These are the same tools we use to debug and develop openpilot.

log types

For each segment, openpilot records the following log types:

rlog.bz2

rlogs contain all the messages passed amongst openpilot's processes. See cereal/services.py for a list of all the logged services. They're a bzip2 archive of the serialized capnproto messages.

{f,e,d}camera.hevc

Each camera stream is H.265 encoded and written to its respective file.

  • fcamera.hevc is the road camera
  • ecamera.hevc is the wide road camera
  • dcamera.hevc is the driver camera

qlog.bz2 & qcamera.ts

qlogs are a decimated subset of the rlogs. Check out cereal/services.py for the decimation.

qcameras are H.264 encoded, lower res versions of the fcamera.hevc. The video shown in comma connect is from the qcameras.

qlogs and qcameras are designed to be small enough to upload instantly on slow internet and store forever, yet useful enough for most analysis and debugging.