Commit Graph

98 Commits

Author SHA1 Message Date
Kacper Rączy
f058b5d64e webrtcd: webrtc streaming server (audio/video/cereal) (#30186)
* WebRTCClient and WebRTCServer abstractions

* webrtc client implementation

* Interactive test scripts

* Send localDescriptions as offer/asnwer, as they are different

* Tracks need to be added after setting remote description for multi-cam streaming to work

* Remove WebRTCStreamingMetadata

* Wait for tracks

* Move stuff to separate files, rename some things

* Refactor everything, create WebRTCStreamBuilder for both offer and answers

* ta flight done time to grind

* wait for incoming tracks and channels

* Dummy track and frame reader track. Fix timing.

* dt based on camera type

* first trial of the new api

* Fix audio track

* methods for checking for incoming tracks

* Web migration part 2

* Fixes for stream api

* use rtc description for web.py

* experimental cereal proxy

* remove old code from bodyav

* fix is_started

* serialize session description

* fix audio

* messaging channel wrapper

* fix audiotrack

* h264 codec preference

* Add codec preference to tracks

* override sdp codecs

* add logging

* Move cli stuff to separate file

* slight cleanup

* Fix audio track

* create codec_mime inside force_codec function

* fix incoming media estimation

* move builders to __init__

* stream updates following builders

* Update example script

* web.py support for new builder

* web speaker fixes

* StreamingMediaInfo API

* Move things around

* should_add_data_channel rename

* is_connected_and_ready

* fix linter errors

* make cli executable

* remove dumb comments

* logging support

* fix parse_info_from_offer

* improve type annotations

* satisfy linters

* Support for waiting for disconnection

* Split device tracks into video/audio files. Move audio speaker to audio.py

* default dt for dummy video track

* Fix cli

* new speaker fixes

* Remove almost all functionality from web.py

* webrtcd

* continue refactoring web.py

* after handling joystick reset in controlsd with #30409, controls are not necessary anymore

* ping endpoint

* Update js files to at least support what worked previously

* Fixes after some tests on the body

* Streaming fixes

* Remove the use of WebRTCStreamBuilder. Subclass use is now required

* Add todo

* delete all streams on shutdown

* Replace lastPing with lastChannelMessageTime

* Update ping text only if rtc is still on

* That should affect the chart too

* Fix paths in web

* use protocol in SSLContext

* remove warnings since aiortc is not used directly anymore

* check if task is done in stop

* remove channel handler wrapper, since theres only one channel

* Move things around

* Moved webrtc abstractions to separate repository

* Moved webrtcd to tools/webrtc

* Update imports

* Add bodyrtc as dependency

* Add webrtcd to process_config

* Remove usage of DummyVideoStreamTrack

* Add main to webrtcd

* Move webrtcd to system

* Fix imports

* Move cereal proxy logic outside of runner

* Incoming proxy abstractions

* Add some tests

* Make it executable

* Fix process config

* Fix imports

* Additional tests. Add tests to pyproject.toml

* Update poetry lock

* New line

* Bump aiortc to 1.6.0

* Added teleoprtc_repo as submodule, and linked its source dir

* Add init file to webrtc module

* Handle aiortc warnings

* Ignore deprecation warnings

* Ignore resource warning too

* Ignore the warnings

* find free port for test_webrtcd

* Start process inside the test case

* random sleep test

* test 2

* Test endpoint function instead

* Update comment

* Add system/webrtc to release

* default arguments for body fields

* Add teleoprtc to release

* Bump teleoprtc

* Exclude teleoprtc from static analysis

* Use separate event loop for stream session tests
2023-12-01 21:13:37 -08:00
Justin Newberry
8d3bf1f41a Pytest: add cpp harness (#30471)
cpp harness
2023-11-22 13:45:49 -08:00
Justin Newberry
ba3067a560 Revert "Pytest: run tests in random order" (#30469)
Revert "Pytest: run tests in random order (#30134)"

This reverts commit a5d99e0ebb.
2023-11-15 10:41:11 -08:00
Justin Newberry
a5d99e0ebb Pytest: run tests in random order (#30134)
* enable random order

* logprint info

* revert that

* better logging

* go to dfu

* try to flash every panda if not up to date

* revert panda debugging
2023-11-15 10:16:51 -08:00
Kacper Rączy
2de13bea91 Remove laika submodule (#30453)
* Remove laika submodule

* Remove laika check for minimal builds

* Fix minimal build check

* Bring back exclude for opendbc in codespell

* Fix typo

* Copy GNSS indices from laika

* Delete laika dependencies
2023-11-15 10:06:37 +01:00
Justin Newberry
31c682d407 Pytest: load by group by default (#30466)
* wip

* use a class marker

* this one should be grouped too
2023-11-14 17:33:47 -08:00
Justin Newberry
2ad82cbfb0 Pytest: default settings on PC improvements (#30441)
* always multithreaded

* multithreaded

* pkill correct process

* escape more

* load for interfaces
2023-11-13 10:45:24 -08:00
Justin Newberry
2629417320 CI: codecov for multiprocessing (#30432) 2023-11-10 10:02:05 -08:00
Justin Newberry
863fdec50e Revert "CI: codecov for multiprocessing" (#30431)
Revert "CI: codecov for multiprocessing (#30428)"

This reverts commit 1e91cf92a1.
2023-11-09 15:52:28 -08:00
Justin Newberry
1e91cf92a1 CI: codecov for multiprocessing (#30428)
* concurrencty codecov

* its a list

* retrigger ci

* increase delay
2023-11-09 15:19:10 -08:00
Justin Newberry
aec7cea30d Pytest: tici mark for skipping on-device tests (#30412)
* mark tici

* not those ones

* missed that one

* add those

* add pypoetry

* oops
2023-11-07 17:35:44 -08:00
Dean Lee
61288dfe06 athenad: fix memory leak in _do_upload() (#30237)
* fix memory leak

* test: stash

* clean up

* clean up

* ruff

* rm

* add py memory profiler

* test compress and no compress

* proper test

* comment

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2023-10-29 07:08:28 -10:00
Adeeb Shihadeh
13bf8c26f3 Update Python packages and pre-commit hooks (#30269)
* Update Python packages and pre-commit hooks

* fix pyqt

---------

Co-authored-by: adeebshihadeh <adeebshihadeh@users.noreply.github.com>
2023-10-18 20:18:08 -07:00
Justin Newberry
2048808827 add random order plugin (#30166)
add random order
2023-10-04 11:51:24 -07:00
Adeeb Shihadeh
0694d3b8f6 remove some python packages (#30135)
* less dependencies

* rm that

* add that back for now

* revert for now
2023-10-02 17:13:45 -07:00
Justin Newberry
3c55f76790 remove carla dependency (#30139)
* only docs and default

* remove carla dependency

* remove carla dependency
2023-10-02 16:11:56 -07:00
Justin Newberry
f32294772d Update metadrive (#30128)
update metadrive
2023-10-02 13:18:03 -07:00
Justin Newberry
2dd13c60cc Revert "run unit tests in a random order (#29799)" (#30116)
This reverts commit b483efe764.
2023-09-30 17:23:38 -07:00
Adeeb Shihadeh
b483efe764 run unit tests in a random order (#29799)
* run unit tests in a random order by default

* bump panda

* fix startup

* re-add random order plugin

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
2023-09-30 15:56:23 -07:00
Kacper Rączy
bfddcdaf89 aarch64: onnxruntime (#30095)
base onnxruntime for aarch64
2023-09-28 22:32:21 -07:00
Justin Newberry
0251742079 CI: use "slow" marker for pytest (#30063)
* skip to mark

* fix quotes
2023-09-26 13:44:18 -07:00
Shane Smiskol
fb8f7ba847 hypothesis: fix too slow health check (#30001)
* update hypothesis

* change less
2023-09-21 17:40:14 -07:00
Adeeb Shihadeh
844b2815d5 add ruff package 2023-09-17 15:44:34 -07:00
Shane Smiskol
5cb4e83528 ruff: check mutable default values for data class attributes (#29945)
* Update pyproject.toml

* Update pyproject.toml
2023-09-16 15:11:09 -07:00
Justin Newberry
ad7432b45d update metadrive (#29938) 2023-09-15 14:55:08 -07:00
Adeeb Shihadeh
2c1dfc2f34 use pytest for cpp tests (#29833)
* add pytest-cpp package (#29828)

* add pytest-cpp package

* disable for now

* use in ci

* use pytest cpp

* use my branch

* add missing tests

* that doesn't make sense

* it was merged upstream

* replace TEST_RLOG_URL with a publicly accessible url

* PR Cleanup

* assert instead

* update poetry lock

* add comment about  pytest-cpp

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
Co-authored-by: deanlee <deanlee3@gmail.com>
2023-09-11 14:01:50 -07:00
Adeeb Shihadeh
d036148550 jenkins speedup (#29869)
* jenkins speedup

* cleanup
2023-09-10 16:47:07 -07:00
Justin Newberry
110cfef1c2 Ruff: disallow non-openpilot imports (#29841) 2023-09-08 10:43:02 -07:00
Adeeb Shihadeh
0d042624ae Revert "add pytest-cpp package (#29828)"
This reverts commit 773d0944d8.
2023-09-07 20:40:25 -07:00
Adeeb Shihadeh
773d0944d8 add pytest-cpp package (#29828)
* add pytest-cpp package

* disable for now
2023-09-07 19:42:30 -07:00
Kacper Rączy
2487db14b0 disable metadrive on linux/aarch64 (#29802) 2023-09-06 21:29:36 +02:00
Adeeb Shihadeh
7b6afbc162 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
2023-09-05 18:52:40 -07:00
Justin Newberry
16c99e3e7f Pytest: don't rerun cereal unittests (#29781)
pytest don't rerun cereal
2023-09-05 14:06:03 -07:00
Adeeb Shihadeh
d344ed9be9 add metadrive sim package (#29736) 2023-08-31 13:38:30 -07:00
Justin Newberry
d3befb2e04 CI: prepare for pytest by adding marker and avoiding duplicated tests (#29682)
prepare for pytest
2023-08-29 17:04:42 -07:00
Adeeb Shihadeh
17bd792eee pycurl is no longer just a dev dependency (#29684) 2023-08-28 14:49:59 -07:00
Adeeb Shihadeh
72e2e2c9ec pytest: strict config (#29676)
* pytest: strict config

* collection works

* fixes
2023-08-28 10:26:19 -07:00
Adeeb Shihadeh
c9e227a9c0 update azure-storage-blob (#29411)
Co-authored-by: Cameron Clough <cameronjclough@gmail.com>
2023-08-25 22:01:10 +01:00
Justin Newberry
979c96167d CI: enable missing tests (#29538) 2023-08-22 16:48:02 -07:00
Adeeb Shihadeh
2cd311da29 add pytest-subtests package (#29536) 2023-08-22 14:31:03 -07:00
Justin Newberry
3492e80d19 CI: use xdist for car model tests (#29514)
* bump panda

* try multithreaded for car models

* need pytest-cov

* upgraded to poetry 1.5.1 and regenerated
2023-08-21 20:51:27 -07:00
Adeeb Shihadeh
ff068dd722 move mypy config into pyproject.toml (#29501)
* move mypy config into pyproject.toml

* rm reference
2023-08-20 21:08:04 -07:00
Kacper Rączy
62a88e504d Unpin numpy (#29421)
* Unpin numpy

* Update lock file

* leave acados comment

* Fix warnings

* Fix more paramsd warnings
2023-08-17 15:36:14 -07:00
Kacper Rączy
766b5e6e8b Disable pyqt5 on aach64 (#29430)
* Make pyqt5 x64 only

* Add comment regarding pyqt5 wheels
2023-08-16 14:11:45 -07:00
Adeeb Shihadeh
6c3e5c4362 add PyQt5 package (#29400) 2023-08-14 15:13:42 -07:00
Harald Schäfer
0dacba96eb Rm poetry from packages (#29361)
no poetry dep needed
2023-08-14 09:50:39 -07:00
Adeeb Shihadeh
d6857996d3 cleanup unused python packages (#29353)
* rm unused python packages

* lock
2023-08-11 21:34:47 -07:00
Adeeb Shihadeh
00a11a1a2b remove fastcluster (#29352)
* remove fastcluster

* lock

* rm there

* and from release files
2023-08-11 21:08:49 -07:00
Adeeb Shihadeh
0ced56b2ea remove xx dependencies (#29263)
* rm xx deps

* mv that

* fix

* fix a path

* relock

* relock

* update

* update sentry

* update

* revert sentry

* Update lock

* Dummy pip package

* Not needed with poetry

* dont need dummy

---------

Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
2023-08-11 20:05:44 -07:00
Justin Newberry
e4ead4f183 Ruff: b905 (strict zip) (#29336)
* added mutable default args

* most of the Bs

* add comment about lrucache

* b905
2023-08-11 16:13:51 -07:00