capnp struct definitions and messaging used in comma ecosystem
Go to file
Kacper Rączy 4e063ca166
FakeSubSocket and FakePubSocket for IPC synchronization (#439)
* Implementation of FakeSubSocket and FakePubSocket using eventfd with support for one-in/one-out synchronization

* Expose FakeEvent to Python

* Add demo showcasing synchronization between processes

* Fix linter errors

* Expose more FakeEvent APIs in Python bindings

* Add FakePoller implementation

* Remove suffix from poll env vars

* Set poller timeout to zero when events are enabled

* Replace poll with ppoll. Add invalidation methods

* Fix lint issues

* Fix comment indent

* Remove fake_demo

* Remove FakePubSocket. Simpler FakePoller implementation. Ability to wait for multiple events

* Rename FakeEvent to Event and move it to event.cc

* Rename event purpose constants in py

* Add support for timeout in wait methods

* Add tests for events and fake sockets

* Fix lint errors

* Add zmq sleeps

* Temporarly disable TestFakeSockets on ZMQ

* Add exception type specifiers to test_fake

* Event Manager implementation

* Fix fake sockets tests

* Update EventManager API

* Add test for enable/disable

* Add tests for cereal prefix

* Remove EventPurpose from python bindings

* Fix lint issues

* event_state_shm_mmap implementation shared by EventManager and FakeSubSocket

* Rename EventManager to SocketEventHandle

* More renames
2023-05-23 19:38:52 +02:00
.github/workflows fix names 2023-04-28 15:11:43 -07:00
include remove java.capnp 2021-10-11 14:35:46 -07:00
logger fix suppression for older cppcheck 2022-03-13 22:22:32 -07:00
messaging FakeSubSocket and FakePubSocket for IPC synchronization (#439) 2023-05-23 19:38:52 +02:00
site_scons/site_tools cython dependency scanner 2021-01-11 14:15:29 -08:00
visionipc add function available_streams to visionipc_pyx.pyx (#424) 2023-03-21 10:52:27 -07:00
.dockerignore Run scons in CI (#14) 2019-11-20 16:32:42 -08:00
.gitignore include from project root (#402) 2022-12-31 14:19:29 -08:00
.pre-commit-config.yaml pre-commit: autoupdate hooks (#446) 2023-05-16 11:49:47 -07:00
Dockerfile include from project root (#402) 2022-12-31 14:19:29 -08:00
LICENSE Add the same license on this repo as openpilot (#78) 2020-11-25 11:12:13 +01:00
README.md fix CI badge 2022-11-30 11:37:35 -08:00
SConscript FakeSubSocket and FakePubSocket for IPC synchronization (#439) 2023-05-23 19:38:52 +02:00
SConstruct CI: run tests with sanitizers (#438) 2023-04-28 15:08:57 -07:00
__init__.py add pre-commit static analysis (#48) 2020-05-28 16:04:53 -07:00
car.capnp Add engineRPM to CarState struct (#448) 2023-05-23 19:03:37 +02:00
codecov.yml Coverage analysis in CI (#75) 2020-07-29 00:33:42 -07:00
generate_javascript.sh initial commit, internal from 6/13/19 2019-06-13 12:06:15 -07:00
legacy.capnp bring qcomgnss back from deprecation 2022-03-23 15:50:02 -07:00
log.capnp Calibration enum (#444) 2023-05-15 10:06:26 -07:00
maptile.capnp get rid of the java (#199) 2021-09-21 14:48:35 -07:00
services.py Uiplan (#413) 2023-02-17 19:35:13 -08:00

README.md

What is cereal? cereal tests codecov

cereal is both a messaging spec for robotics systems as well as generic high performance IPC pub sub messaging with a single publisher and multiple subscribers.

Imagine this use case:

  • A sensor process reads gyro measurements directly from an IMU and publishes a sensorEvents packet
  • A calibration process subscribes to the sensorEvents packet to use the IMU
  • A localization process subscribes to the sensorEvents packet to use the IMU also

Messaging Spec

You'll find the message types in log.capnp. It uses Cap'n proto and defines one struct called Event.

All Events have a logMonoTime and a valid. Then a big union defines the packet type.

Message definition Best Practices

  • All fields must describe quantities in SI units, unless otherwise specified in the field name.

  • In the context of the message they are in, field names should be completely unambiguous.

  • All values should be easy to plot and be human-readable with minimal parsing.

Pub Sub Backends

cereal supports two backends, one based on zmq and another called msgq, a custom pub sub based on shared memory that doesn't require the bytes to pass through the kernel.

Example

import cereal.messaging as messaging

# in subscriber
sm = messaging.SubMaster(['sensorEvents'])
while 1:
  sm.update()
  print(sm['sensorEvents'])

# in publisher
pm = messaging.PubMaster(['sensorEvents'])
dat = messaging.new_message('sensorEvents', size=1)
dat.sensorEvents[0] = {"gyro": {"v": [0.1, -0.1, 0.1]}}
pm.send('sensorEvents', dat)