Commit Graph

527 Commits

Author SHA1 Message Date
Adeeb Shihadeh
484aaf4307 no PC anymore
old-commit-hash: 4e2624783f
2024-08-27 12:05:05 -07:00
Adeeb Shihadeh
854d03a819 manager: remove LastUpdateTime default
old-commit-hash: 39c64d44a1
2024-08-27 12:03:55 -07:00
Dean Lee
965aea8eab scons: remove duplicate linker flags for -ljson11, -lzmq (#33361)
fix duplicate link
old-commit-hash: 908aa525ec
2024-08-23 10:42:57 -07:00
Kacper Rączy
e154f03a12 timed: use gps location packet for time (#33306)
* GPS helpers

* Use GPS socket in timed

* Use helper in controlsd

* Use updated and logMonoTime

* Helper type annot

* Not updated

* Increase time threashold
old-commit-hash: cf355ecf20
2024-08-22 23:30:29 -07:00
Andrei Radulescu
24c18d2c4b agnos updater: set decompress max_length (#33320)
* agnos: decompress max_length

* flash last chunk after eof

* don't decompress more than length

* cleanup

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: ba098f509b
2024-08-17 16:31:22 -07:00
Shane Smiskol
4f019b5f60 move selfdrive/car to opendbc (#32630)
* move most of /car

* move some car tests

move some car tests

* fix selfdrive/car/tests

* fix selfdrive/controls tests

* fix the rest of the selfdrive tests

* bump opendbc

* fix all tests

* few more non-test references

* remove opcar and move docs to car

fix these debugging scripts

fix docs

* bump opendbc and panda

forgot panda
old-commit-hash: e735a7f379
2024-08-17 00:54:51 -07:00
Shane Smiskol
95224db413 selfdrive/car: ban cereal and capnp (#33208)
* ban cereal and msgq

* common too

* do toyota/values.py

* do all fingerprints

* example without builder

* this still works, but no type checking anymore

* stash

* wtf, how does this work

* okay actually not bad

* safe

* epic!

* stash data_structures.py

* some clean up

* hell yeah

* clean up old file

* add to delete

* delete

This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf.

* switch more CarParams stuff over

remove unused

* fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :(

* fix this too

* fix this too

* remove more cereal and add some good hyundai tests

* bunch more typing

* override default with 20hz radar

* temp capnp converter helper

* more lateralTuning

* small union replicator is better than what i was trying, and fixes mypy dynamic typing issues

* can keep all this the same now!

* type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!)

* revert more

* get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(

would have been `CarParams.NetworkLocation()`

* Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :("

This reverts commit bb28b228becba932052d2fc5a4389784027435b1.

* remove cereal from car_helpers (TODO: caching)

* remove a bunch of temp lines

* use dataclass_transform!

* remove some car.CarParams from the interfaces

* remove rest of car.CarParams from the interfaces

* same which() API

* sort

* from cereal/cache from fingerprinting!

* more typing

* dataclass to capnp helper for CarParams, cached it since it's kinda slow

* (partial) fix process replay fingerprintig for new API

* latcontrollers take capnp

* forgot this

* fix test_models

* fix unit tests

* not here

* VehicleModel and controller still takes capnp CP since they get it from Params()

* fix modeld test

* more fix

* need to namespace to structs, since CarState is both class and struct

* this was never in the base class?!

* clean that up again

* fix import error

fix import error

* cmts and more structs

* remove some more cereal from toyota + convert CarState to capnp

* bruh this was wrong

* replace more cereal

* EventName is one of the last things...

* replace a bunch more cereal.car

* missing imports

* more

* can fix this typing now

* proper toyota+others CS typing!

* mypy can detect return type of CS.update() now

* fix redeclaration of cruise_buttons type

* mypy is only complaining about events now

* temp fix

* add carControl struct

* replace CarControl

i hope there's no circular imports in hyundai's CC

* fine now

* lol this was wrong too

* fix crash

* include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( )

but attrs does, maybe will switch in the future

* clean up

* try out attr.s for its converter (doesn't work recursively yet, but interesting!)

* Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)"

This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9.

* test processes doesn't fail anymore (on toyota)!

* fix honda crash

* stash

* Revert "stash"

This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0.

* remove a bunch more cereal!

* LET'S GOOO

* fix these tests

* and these

* and that

* stash, something is wrong with hyundai enable

* Revert "stash, something is wrong with hyundai enable"

This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03.

* forgot these

* remove cereal from fw_versions

* Revert "remove cereal from fw_versions"

This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56.

* remove rest of the cereal exceptions!

* fix that

* add typing to radard since I didn't realize RI.update() switched from cereal to structs

* and here too!

* add TODO for slots

* needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( )

* remove the struct converter

* save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy

btw pickle.loads(pickle.dumps()) is faster by ~1% CPU

* deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone

saves ~1% CPU

* add non-copying asdict function

* should save ~3% CPU (still 4% above baseline)

* fix that, no dict support

* ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s)

* give a better name

* fix

* dont support none, capitalize

* sheesh, this called type() on every field

* remove CS.events, clean up

* bump card %

* this was a bug on master!

* add a which enum

* default to pid

* revert

* update refs

* not needed, but consistent

* just Ecu

* don't need to do this in this pr

* clean up

* no cast

* consistent typing

* rm

* fix

* can do this if we're desperate for the last few %

* Revert "can do this if we're desperate for the last few %"

This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299.

* type this

* don't need to convert carControl

* i guess don't support set either

* fix CP type hint

* simplify that
old-commit-hash: 6a15c42143
2024-08-16 15:13:00 -07:00
Maxime Desroches
c551f1d239 fix bash + python3 shebangs (#33287)
fix shebangs
old-commit-hash: 1dcd660815
2024-08-13 21:22:13 -07:00
Kacper Rączy
80bc61dc6c controlsd: use livePose (#33283)
* Pose calibrator

* Fix static analysis

* Fix static

* Fix test_latcontrol

* Fix test_latcontrol

* Update services in process replay

* Fix static

* Matmul not mul

* Add assertion

* Move pose calibration to data_sample

* Update ref commit

* Remove llk from cycle alerts

* Deprecated nogps event

* Switch power_draw to lp

* Bring back noGps alert

* Add handling code back

* get_bool

* Bring inputsok back
old-commit-hash: 9734015bbb
2024-08-13 21:11:16 -07:00
Adeeb Shihadeh
ac6956a651 tici: new updater build (#33246)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 1b554b488e
2024-08-09 14:30:28 -07:00
Adeeb Shihadeh
7b5d3dbbca sensord: error -> warning for expected failures
old-commit-hash: 16c36b0d8b
2024-08-06 17:25:03 -07:00
Shane Smiskol
9e4a7484be remove unnecessary debugging function from car_helpers.py (#33199)
* format

* better better

* huh
old-commit-hash: 5c1e111d8a
2024-08-05 21:27:43 -07:00
Adeeb Shihadeh
fc469e5f80 Revert "Remove setproctitle (#32716)" (#33195)
* Revert "Remove `setproctitle` (#32716)"

This reverts commit 9020d1931f4cd72f8162e4402278bdf76e3746f2.

* uv lock

* old name
old-commit-hash: 29b58d4f2f
2024-08-05 16:42:22 -07:00
Maxime Desroches
df13b36638 fix all codespell issues (#33194)
* bring back

* remove 2020 dictionary
old-commit-hash: 44c7144e1c
2024-08-05 15:25:02 -07:00
Dean Lee
ad42551437 camerad: encapsulate cl_command_queue in ImgProc (#33178)
move cl queue to imgproc
old-commit-hash: 8f9b165de8
2024-08-03 14:57:26 -07:00
Adeeb Shihadeh
139f4034a8 Revert "uploader.py: simplify clear_locks by using rglob (#32974)"
This reverts commit 870f5cbeb39fa63829e6f3a64fa8df57e82d52c3.

old-commit-hash: 9474523add
2024-08-03 12:12:42 -07:00
Dean Lee
7fcba71725 uploader.py: simplify clear_locks by using rglob (#32974)
simplify clear_locks
old-commit-hash: cafca4f891
2024-08-02 13:40:57 -07:00
Dean Lee
7adc4219f0 camerad: update frame msg related variable types to uint64_t for consistency (#33113)
Update frame_msg related variable types to uint64_t for  type safety
old-commit-hash: c40d129f70
2024-08-02 12:57:29 -07:00
Dean Lee
2a16a305b3 camerad: move do_exit to camera_qcom2.cc (#33086)
move do_exit to camera_qcom2.cc
old-commit-hash: 8ab587d30b
2024-08-02 12:18:10 -07:00
Dean Lee
b12b30c328 camerad: encapsulating event management within the ImgProc Class (#33119)
Encapsulating Event Management Within the ImgProc Class
old-commit-hash: 2189164d45
2024-08-02 12:16:04 -07:00
Dean Lee
d802a423a6 loggerd: Initialize atomic variables to zero (#33130)
Initialize atomic variables in LoggerdState to zero
old-commit-hash: 8f3fc699d4
2024-08-02 12:01:56 -07:00
Hoang Bui
cb306fd7a8 CI: fix recent test_rotation skip (#33167)
old-commit-hash: 5b4b7a08fc
2024-08-01 16:28:26 -07:00
Adeeb Shihadeh
d59f9262c8 disable test_rotation until it's fixed
old-commit-hash: bfa8e0b2ec
2024-08-01 15:36:44 -07:00
ZwX1616
03fa24ee82 camerad: use os binning (#33161)
* all

* right sz

* what is this

* Revert "what is this"

This reverts commit 3ff771926f109452a055dfb61cee06b34e498164.

* ok

* big

* update intrs

* fl

* more specifuc

* fish

* revert

* revert

---------

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 3ccc63deec
2024-07-31 19:10:44 -07:00
Shane Smiskol
5cc7d1c36b deprecate busTime (#33146)
* deprecate busTime

* bump

* do car can + more

* forgot some

* bump opendbc

* fix that too

* bump
old-commit-hash: 42f2601416
2024-07-30 21:26:58 -07:00
Dean Lee
3c1af12490 uploader.py: ensure proper resource management with io.BytesIO (#33108)
* ensure proper resource management with io.BytesIO

* improve
old-commit-hash: 2728c95b0d
2024-07-29 13:43:31 -07:00
Adeeb Shihadeh
8cc6b66cde docs: add concepts section
old-commit-hash: 5ebc65f254
2024-07-28 16:38:41 -07:00
Shane Smiskol
7b67994a05 use zstandard library (#33110)
* use zstandard

* run 10

* bye bye

* this was a little overzealous

* clean up
old-commit-hash: 1b3b5ab758
2024-07-28 02:03:47 -07:00
Shane Smiskol
b0a8b3bc9d uploader: compress with zstd (#32736)
* zstd uploader

* fix that

* fix name of function

* comment

* log failed

* fix comma_api_source for routes with both bz2 and zst rlogs

* TODO

* 10-14 achieves almost no benefit on qlogs in a few cases, but takes 2x the time

* these aren't written out

* regen: specify any list of sources

ooh this is pretty nice

* regen and process replay

* damn, actually we don't need all this (cool tho)

Revert "regen: specify any list of sources"

This reverts commit ceb0b4abed9ad463a9fe98d9b98a05875a52806f.

* just let it auto resolve

* fix athenad/uploader tests

* zst here too

* TODOs

* yes

* Revert "TODOs"

This reverts commit 8c7da1dbd0340c72290b5eb5563b642080ddc131.

* Revert "zst here too"

This reverts commit 23b0023ddfd22c8090be7a7caa09e7026a12aa5c.

* Revert "just let it auto resolve"

This reverts commit f296d62424227ad05facc62abc18a6f81b474e84.

* Revert "regen and process replay"

This reverts commit 0768330e96974a42616d229d159780619d049cd0.

* revert readme

* not in save_log either

* lfg

* Revert "lfg"

This reverts commit 3718559c6c4de7d1f0c80dc9f1a1d335fe679a89.
old-commit-hash: 7dec7c39be
2024-07-26 19:33:35 -07:00
Dean Lee
01ac526046 camerad: merge thread functions into one (#33025)
merge multiple thread functions into one
old-commit-hash: ccd369ee96
2024-07-25 15:05:14 -07:00
Dean Lee
757cb2eb6c mcid.py: optimize FFT and A-weighting calculations (#33057)
* Precomputing weighting

* add comments back

* use cache

* spacing

spacing

* clean up

* lower by diff

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 313a2826c2
2024-07-25 13:36:44 -07:00
Dean Lee
5b5e3ef053 camerad: add CameraConfig struct for initializing CameraState in constructor (#33065)
* Add CameraConfig struct for initializing CameraState in constructor

* init member variables
old-commit-hash: 75f49f84f3
2024-07-25 13:09:03 -07:00
Shane Smiskol
5fdbd73ba3 athenad: log failed upload items
old-commit-hash: add958bb08
2024-07-24 14:48:33 -07:00
Dean Lee
4d6d229820 camerad: refactor camera_open() into separate functions for clarity (#33056)
split function
old-commit-hash: 0fa6745a67
2024-07-24 10:07:10 -07:00
Adeeb Shihadeh
5025140f37 Revert "camerad: add CameraConfig struct for initializing CameraState in constructor (#33034)"
This reverts commit 8ef650da47eec8b64e2aedcba5114ca1a2428a89.

old-commit-hash: cefe00c964
2024-07-21 20:36:45 -07:00
Dean Lee
3f327ef4d3 camerad: add CameraConfig struct for initializing CameraState in constructor (#33034)
* Add CameraConfig struct for initializing CameraState in constructor

* Update system/camerad/cameras/camera_qcom2.h

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: dc886e195f
2024-07-21 19:19:34 -07:00
Dean Lee
cc3f9090bf cereal: deprecate FrameData.frameType (#33019)
deprecate FrameData.frameType
old-commit-hash: 917ea5699f
2024-07-19 10:14:04 -07:00
Kacper Rączy
4efe8d8118 Fix msgq context overrides (#33014)
* Fix msgq context overrides

* Remove unneccesary changes

* add reset_context helper

* Remove redundant import
old-commit-hash: c96dbd5a0b
2024-07-18 20:19:39 -07:00
Adeeb Shihadeh
acdbe2440f tici: set STM_BOOT0 in reset (#33015)
* tici: reset panda boot0

* ooops
old-commit-hash: 560e37cd83
2024-07-18 18:48:54 -07:00
Shane Smiskol
1d480761bd hardwared: don't ignore quick ignition cycles (#32938)
* check on either edge

* clean up

* clean up

* good thing I tested this first :P
old-commit-hash: 11db7b683b
2024-07-11 02:49:42 -07:00
Adeeb Shihadeh
1490a714a0 mvp of new docs.comma.ai (#32926)
* ssh docs

* strict

* need an index

* push to prod
old-commit-hash: f6189568ea
2024-07-08 16:55:16 -07:00
Dean Lee
9f6dae8f44 logmessaged: remove the redundant nested print (#32872)
remove the redundant nested print
old-commit-hash: cec8d5a305
2024-06-29 17:34:46 -07:00
Maxime Desroches
8adfc992e3 fix pytest tests (#32867)
* fix tests

* no dot
old-commit-hash: 6f10b8a209
2024-06-28 23:58:19 +00:00
Shane Smiskol
82ebea3d12 deleter: preserve two previous segments on user flag (#32822)
preserve 2 segs
old-commit-hash: dc41848597
2024-06-24 15:11:20 -07:00
Mauricio Alvarez Leon
8d8c2a29c9 remove timezone from timed.py (#32665)
* use timezone from tilequery mapbox api

* add timezone into cereal

* better clean

* more

* one line

* poetry lock
old-commit-hash: 7013eed50c
2024-06-19 20:34:57 -07:00
Adeeb Shihadeh
3b8ed67aa3 remove navigation (#32773)
* remove navigation

* lil more

* fix cabana

* revert for now

* fix docs

* move that over

* lock

* rm assets
old-commit-hash: e491c9d964
2024-06-19 19:14:13 -07:00
Adeeb Shihadeh
9cfdd6200d Remove setproctitle (#32716)
* Custom setproctitle (#32667)

* add custom setproctitle

* add test

* Update poetry.lock

* fix lint

* support only Linux

* test only Linux

* final lint

* Update test_setproctitle.py

* Update setproctitle.py

* convert to threadnames

* delete proctitles

* Check str len and use PR_GET_NAME

* fix poetry.lock

* lint fix

* Update common/threadname.py

---------

Co-authored-by: reddyn12 <nikhilr.ssm@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* revert that for now

* use last 15

* fix

* use name

* update those

* and modeld

* rm

---------

Co-authored-by: schlimeszn <138847413+schlimeszn@users.noreply.github.com>
Co-authored-by: reddyn12 <nikhilr.ssm@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 83ac80c6c8
2024-06-11 21:57:32 -07:00
Mauricio Alvarez Leon
008008b03a Update to Python 3.12 (#32548)
* 3.12

* pprofile is broken

* use modified metadrivepy3-12

* 0.3.0 metadrive

* add metadrive/commaai git dependency

* metadrive git set

* pin sounddevice 0.4.6

* datetime.utcnow() deprecation

* poetry lock

* make datetime not aware

* poetry lock

* pin pytools

* google_crc32c wheel

* unpin sounddevice

* clean metadrive

* use python crc

* mypy

* 3.12.4

* allow python3.11

* test

* no pip

* poetry

* better

* better

* merge

* remove

* try

* test

* try  this

* snok

* python

* simpler

* setuptools

* lower

* try

* try

* work?

* ubuntu deps

* ubuntu

* try

* remove

* move

* remove this

* names

* Update .github/workflows/tools_tests.yaml

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* python<4

* <3.13

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
old-commit-hash: 148eaf8fa6
2024-06-11 20:45:15 +00:00
Adeeb Shihadeh
e920e57530 Revert "Custom setproctitle (#32667)"
This reverts commit 1f45e163559baa2a6127d5a5e8deb7067e32f813.

old-commit-hash: 5b51f03967
2024-06-10 19:55:41 -07:00
schlimeszn
afea57ae2a Custom setproctitle (#32667)
* add custom setproctitle

* add test

* Update poetry.lock

* fix lint

* support only Linux

* test only Linux

* final lint

* Update test_setproctitle.py

* Update setproctitle.py

* convert to threadnames

* delete proctitles

* Check str len and use PR_GET_NAME

* fix poetry.lock

* lint fix

* Update common/threadname.py

---------

Co-authored-by: reddyn12 <nikhilr.ssm@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 3365ed5eff
2024-06-10 18:59:01 -07:00