Files
sunnypilot/system/loggerd
Shane Smiskol b0a8b3bc9d uploader: compress with zstd (#32736)
* zstd uploader

* fix that

* fix name of function

* comment

* log failed

* fix comma_api_source for routes with both bz2 and zst rlogs

* TODO

* 10-14 achieves almost no benefit on qlogs in a few cases, but takes 2x the time

* these aren't written out

* regen: specify any list of sources

ooh this is pretty nice

* regen and process replay

* damn, actually we don't need all this (cool tho)

Revert "regen: specify any list of sources"

This reverts commit ceb0b4abed9ad463a9fe98d9b98a05875a52806f.

* just let it auto resolve

* fix athenad/uploader tests

* zst here too

* TODOs

* yes

* Revert "TODOs"

This reverts commit 8c7da1dbd0340c72290b5eb5563b642080ddc131.

* Revert "zst here too"

This reverts commit 23b0023ddfd22c8090be7a7caa09e7026a12aa5c.

* Revert "just let it auto resolve"

This reverts commit f296d62424227ad05facc62abc18a6f81b474e84.

* Revert "regen and process replay"

This reverts commit 0768330e96974a42616d229d159780619d049cd0.

* revert readme

* not in save_log either

* lfg

* Revert "lfg"

This reverts commit 3718559c6c4de7d1f0c80dc9f1a1d335fe679a89.
old-commit-hash: 7dec7c39be
2024-07-26 19:33:35 -07:00
..
2023-03-08 11:20:49 -08:00
2023-03-08 11:20:49 -08:00
2024-03-05 20:43:33 -08:00
2023-03-08 11:20:49 -08:00
2024-06-09 17:44:34 -07:00

loggerd

openpilot records routes in one minute chunks called segments. A route starts on the rising edge of ignition and ends on the falling edge.

Check out our python library for reading openpilot logs. Also checkout our tools to replay and view your data. These are the same tools we use to debug and develop openpilot.

log types

For each segment, openpilot records the following log types:

rlog.bz2

rlogs contain all the messages passed amongst openpilot's processes. See cereal/services.py for a list of all the logged services. They're a bzip2 archive of the serialized capnproto messages.

{f,e,d}camera.hevc

Each camera stream is H.265 encoded and written to its respective file.

  • fcamera.hevc is the road camera
  • ecamera.hevc is the wide road camera
  • dcamera.hevc is the driver camera

qlog.bz2 & qcamera.ts

qlogs are a decimated subset of the rlogs. Check out cereal/services.py for the decimation.

qcameras are H.264 encoded, lower res versions of the fcamera.hevc. The video shown in comma connect is from the qcameras.

qlogs and qcameras are designed to be small enough to upload instantly on slow internet and store forever, yet useful enough for most analysis and debugging.