Commit Graph

64 Commits

Author SHA1 Message Date
Willem Melching
2e0c73fc0c Remove Raw Upload toggle (#23143)
* Remove Raw Upload toggle

* fix tests
2021-12-07 11:25:40 +01:00
grekiki
6951b3271d Python: Replace more lists with generators (#23116)
* Replace lists with generators v2

* Replace set with {}

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Replace more set() with {}

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-12-03 22:57:19 -08:00
Willem Melching
f7c46c6949 Do not run code on version.py import (#23063)
* Do not run code on version.py import

* fix athena
2021-11-29 19:38:55 +01:00
Willem Melching
a2f32fd3e1 Clear loggerd locks once on boot (#23060)
* add test for clear_locks

* move to bootlog

* simplify test
2021-11-29 12:30:28 +01:00
Dean Lee
1d323e0fd6 loggerd: add test case for trigger_rotate (#23038)
* test rotate

* remove global LoggerdState
2021-11-26 14:53:09 +01:00
Dean Lee
b5eb02181c loggerd: add test case for sync_encoders (#23020) 2021-11-24 13:34:16 -08:00
Adeeb Shihadeh
e13630dfee faster jenkins (#22973)
Co-authored-by: Comma Device <device@comma.ai>
2021-11-18 23:43:09 -08:00
Willem Melching
77b334a698 swaglog.py: respect LOGPRINT env variable (#22950)
* swaglog.py: respect LOGPRINT env variable

* only change handler that prints

* keep usefull manager prints

* only print running list

* revert process changes

* fix test
2021-11-18 16:05:06 +01:00
Adeeb Shihadeh
ea761cbbd5 loggerd: trigger rotate on frame id instead of frame count (#22848)
Co-authored-by: Comma Device <device@comma.ai>
2021-11-13 15:38:10 -08:00
Mayfield
5246f0231e enable wshadow (#22756)
* enable wshadow (#22714)

* fix replay

* more build fixes

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-11-02 17:08:53 +01:00
Dean Lee
2b4a477fbc replay: improve segment download and merge (#22654)
* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
2021-11-01 11:55:56 +01:00
Willem Melching
89f311714c loggerd: set encodeIdx.valid if frame id is correct (#22634)
* loggerd: set encodeIdx.valid if frame id is correct

* check valid flag in test

* test cleanup

* bump cereal
2021-10-26 16:00:43 +02:00
Dean Lee
d28b98c602 replay : move utility functions into separate file (#22414)
* move functions into util

* read bz2 into stream

* pre-decompress log in the download thread

* cleanup logreader

* cache sha256 path

* use readBZ2file in test_logger

* Revert "cache sha256 path"

This reverts commit 60459d3ea09a2c80f4560cf95b1ce7d6af59f06d.

* use macro

* use ostringstream

* cleanup readBZ2File

* move precise_nano_sleep into util
2021-10-04 15:39:59 +02:00
Dean Lee
5a8ca8b449 loggerd: remove obsolete testraw.cc (#22138) 2021-09-06 12:10:12 -07:00
Willem Melching
a93271a518 Ensure that first segment starts with all cameras on the same frameId (#22096)
* test that segment starts with the same frame id

* improve test

* add sync logic

* atomic max

* review comments
2021-09-01 14:28:50 -07:00
Dean Lee
935cbd3139 LoggerState: test cases (#21855)
* test multiple threads writing to log

* fix typo

* use util::getenv

* check INIT_DATA and SENTINEL

* test END_OF_SEGMENT

* add comment

* test multiple threads logging and rotation

* cleanup

* update

* LoggerHandle:quick fix

* cleanup

* revert test_loggerd.py

* refactor test case

* check lock file

* check refcnt after close

* test_runner.cc

d

* int eixt_signal
2021-08-30 17:12:09 -07:00
Willem Melching
a39873872d refactor loggerd: trigger rotate in encoder thread (#21860)
* trigger rotate in eoncode thread

* rotate in time

* lgtm

* dcam trigger rotate on C3

* check trigger rotate field

* Use >=

* add rotator thread

* set cnt to 0

* test encodeIdx is increasing across segments

* test both segmentId and encodeId

* fix encodeIdx

* no thread needed

* add log for failed to encode

* stricter C3 test, dont check start encodeId on C2 dcam

* only update last_camera_seen_tms when considered for rotate

Co-authored-by: deanlee <deanlee3@gmail.com>
2021-08-06 11:10:33 +02:00
Adeeb Shihadeh
d748b44f48 fix loggerd test failure when camerad is slow to start 2021-08-02 09:10:29 -07:00
Adeeb Shihadeh
48652db706 debug msg for loggerd tests 2021-07-28 14:01:27 -07:00
Adeeb Shihadeh
d8411e7a62 fix qlog decimation calculation in loggerd tests 2021-07-10 23:20:15 -07:00
Adeeb Shihadeh
0bfd994f73 fast jenkins (#21472)
* speed up setup

* speedup onroad test

* .git doesn't like inplace

* tici2

* no inplace

* revert

* fix

* that one too
2021-07-02 16:29:57 -07:00
Adeeb Shihadeh
c76c30fa2d disable uploading raw by default (#21128) 2021-06-03 15:24:30 -07:00
Willem Melching
4742f55749 store all files in /sys/fs/pstore in bootlog (#20875)
* store all files in /sys/fs/pstore in bootlog

* whitespace

* fix bootlog test
2021-05-12 11:15:54 +02:00
Dean Lee
7222d0f20d Cleanup selfdrive/ includes (#20822)
* cleanup include path

* continue

* format includes

* fix testraw.cc

* remove include path from SConstruct

* regroup

* rebase master

* almost done

* apply review

* rename FileReader.xx to filereader.xx

* rename Unlogger.x->unlogger.x

* rename FrameReader.xx -> framereader.xx

* apply reviews

* ui.h

* continue

* fix framebuffer.cc build error:mv util.h up

* full path to msm_media_info

* fix qcom2 camerad

Co-authored-by: Comma Device <device@comma.ai>
2021-05-08 22:15:17 -07:00
Adeeb Shihadeh
1786d04239 handle unregistered devices (#20728)
* handle unregistered devices

* cleanup

* no uploader or athena

* clean up
2021-04-22 21:03:57 -07:00
Willem Melching
c4e2d374c9 More encoder test cleanup (#20623)
* more encode test cleanup

* fix check at end of function
2021-04-08 17:48:57 +02:00
Willem Melching
ec5465ffd5 check encodeIdx in loggerd test (#20621) 2021-04-08 16:50:48 +02:00
Willem Melching
ae094042ad add putBool/getBool wrappers to cython params class (#20611)
* add putBool/getBool wrappers to cython class

* use new API

* some more puts

* fix mockparams arguments

* add get_bool to MockParams

* typo
2021-04-07 15:36:37 +02:00
Willem Melching
5a3b511306 Manager cleanup (#20231)
* move manager in folder

* inital refactor

* call start

* small cleanup

* add comments

* use self.signal()

* order shouldnt matter

* newlines

* add helpers

* newlines

* add process config

* split out build part of manager

* this should fix most tests

* no sensord on pc

* dont start athena

* remove comment

* fix old athena test

* fix inject model

* fix test car models

* should be not none

* fix helpers exitcode

* ignore manage_athenad

* Use time.monotonic()

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* combine init, remove spinner

* move manager test

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-03-05 11:03:23 +01:00
Willem Melching
6e26403cee test loggerd: limit samples to number of services 2021-02-23 16:45:08 +01:00
ZwX1616
e2044ca5a2 up tici hevc bitrate (#20093)
* 2x tici bitrate

* update test

* clean up
2021-02-22 09:15:56 -08:00
Greg Hogan
1cd9105e96 adjust qcamera quality (#20066)
* better ts

* fix test
2021-02-11 17:55:00 -08:00
Adeeb Shihadeh
0411015103 fix up encoder test for tici rotation (#19941)
* fix encoder test

* revert for c2

Co-authored-by: Comma Device <device@comma.ai>
2021-01-27 14:20:48 -08:00
Willem Melching
87119aebe2 Put bootlog in own folder (#19939)
* put bootlog in own folder

* fix uploader

* remove print statements

* update test

* remove comment

* also write initddata

* remove sentinels from test
2021-01-27 13:36:40 -08:00
Adeeb Shihadeh
9f2088e2c3 make loggerd rotation test more reliable 2021-01-24 19:38:20 -08:00
Adeeb Shihadeh
810ed30cab more tici tests in jenkins (#19908)
* run loggerd tests on tici

* boardd loopback

* fix up seg length

* this doesn't add more coverage
2021-01-24 17:54:03 -08:00
Adeeb Shihadeh
d4489fbf91 OMX encoder stability (#19758)
* cleanup test first

* wait for other encoder threads

* fixes initial crash

* no print

* unused

* fix up tests

* need that

Co-authored-by: Comma Device <device@comma.ai>
2021-01-21 10:40:08 -08:00
Dean Lee
f973c56a36 loggerd: split bootlog to a separate program (#19831)
* bootlog

* blank lines

* move file_exists to util.h

* src = ['loggerd.cc']

* log_init_data

* Revert "src = ['loggerd.cc']"

This reverts commit 18a587023d75b3a3a54f1ceaf9cb31a51640a067.

* add bootlog to .gitignore

* use s->has_qlog instead of true

* add blank lines & remove extern C
2021-01-19 12:16:12 -08:00
Adeeb Shihadeh
5221450f2d loggerd: another rotation test (#19742)
* another rotation test

* make encoder test fast

* fix pc

Co-authored-by: Comma Device <device@comma.ai>
2021-01-12 19:40:57 -08:00
Adeeb Shihadeh
92e4f04873 Loggerd for PC (#19730) 2021-01-12 16:28:22 -08:00
Dean Lee
595830135b merge utilpp.h into util.h (#19710) 2021-01-10 14:57:32 -08:00
Willem Melching
fb496c692a VisionIPC 2.0 (#19641)
* Squashed vipc

* Update release files

* Remove else

* add visionipc to release files

* use poller in vipc receive

* opencl framework instead of lib on macos

* Fix camera webcam

* Fix opencl on mac in ui

* more webcam fixes

* typo in ui sconsfile

* Use cur_yuv_buf

* visionbuf c++ class

* Camera qcom was still using visionbuf_allocate

* Turn loggerd back on

* fix snapshot

* No build needed

* update test camerad

* no more release callback

* make encoder c++

* Revert "no more release callback"

This reverts commit e5707b07002fee665d0483d90713154efc2d70d4.

* fix exit handlers

* No need to check errno

* move release callback call

* s/VIPCBufExtra/VisionIpcBufExtra/g

* use non blocking connect

* ui use non blocking connect

* Lower condition variable wait time

* Snapshot cleanup

* bump cereal

* bump cereal
2021-01-08 14:54:25 +01:00
Adeeb Shihadeh
206d072bb4 loggerd cleanup (#19668) 2021-01-07 23:08:40 -08:00
Adeeb Shihadeh
af81190cfc loggerd tests (#19671) 2021-01-06 19:18:35 -08:00
Dean Lee
c1cf0882bd Replace usleep with util::sleep_for (#19643)
* replace usleep with sleep_for

* sleep microseconds

* sleep nanoseconds

* Trigger Build
2021-01-04 18:03:11 +01:00
Adeeb Shihadeh
59d7c5e64a move to github container registry + CI speedup (#19621) 2020-12-31 14:57:01 -08:00
Adeeb Shihadeh
ffa7e0cbdb manager tests + make all processes exit cleanly (#19595)
* manager tests

* logcatd exits cleanly

* sigint

* boardd

* multiple dbus connections hangs for some reason

* clocksd proclogd

* network type from thermal

* fix tests

* fix android logcatd

* fix mac

* fix mac proclogd

* move on device athena tests

* build first

* build first

Co-authored-by: Comma Device <device@comma.ai>
2020-12-29 22:32:03 -08:00
Adeeb Shihadeh
8674b023ba HW abstraction layer (#19530)
* start hw refactor

* move that

* pins

* put that back
2020-12-16 21:30:23 -08:00
ZwX1616
8d986c82d8 fix loggerd unittest 2020-11-22 15:40:51 -08:00
Willem Melching
339d1967cb --std=c++17 (#2330)
* switch to c++17

* more 17

* 1z

* use std::clamp

* Fix types in clamp

* bump submodules

* bump laika

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2020-10-17 13:33:12 -07:00