Commit Graph

113 Commits

Author SHA1 Message Date
Dean Lee
768cfdb92a framereader: use the same alignement as visionbuf (#23208)
* aligned rgb buffer

* fix yuv stride
2021-12-15 16:19:07 -08:00
Dean Lee
8a84263c2d replay: use std::unique_ptr to manage memory, reuse events buffer. (#23049)
* reuse events buffer

* Trigger Build
2021-12-13 15:22:31 +01:00
Dean Lee
348d2d2b0d rename yuv streams (#23071)
* rename yuv streams

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-11-29 18:14:59 -08:00
Dean Lee
dbec761941 logreader: support reading from corrupt log (#23050)
* catch exception outside loop

* print decompress error

* add test case for corrupt log

* fix decompressbz2 stuck if log is corrupt

* recovered from corrupt data

* add output

* ass space

* std::endl

* override load(), load from buffer

* override FrameReader::load to load from the buffer

* replace NULL with nullptr

* fix test case for corrupt log

* Trigger Build

* check bzerror too

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-11-29 14:10:24 +01:00
Dean Lee
296c4076a2 framereader: refactor seeking (#22926)
* refactor seeking

* ckeck packets.empty()

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-11-29 12:09:14 +01:00
Dean Lee
3fd0264960 refactor httprequest: emit single signals (#23039)
* refactor httprequest

* Trigger Build
2021-11-29 11:19:08 +01:00
Dean Lee
f5e4867864 replay: refactor http download (#23052)
* refactor http download

* use chunk_size instead of parts

testcase:set chunksize to 5mb

* use template

space

* cleanup

* remove unused include

* check buffer overfllow

* simplify print download speed
2021-11-29 11:13:30 +01:00
Dean Lee
a449b856ca framereader: fix crash after fallback to cpu decoding (#23055) 2021-11-28 13:29:40 -08:00
Dean Lee
5ae5174509 framereader: replace swscale with libyuv, reduce cpu usage by half (#22992)
* use libyuv

* cleanup
2021-11-26 14:41:14 +01:00
Dean Lee
1d2dfab81c replay: add test to check dcam and ecam (#22993) 2021-11-20 18:56:27 -08:00
Dean Lee
9cb46cc4a1 logreader: fix memory leak in Event (#22987) 2021-11-20 11:12:52 +01:00
Dean Lee
ddea21e648 replay: fix swsscale crash on nv12->yuv conversion (#22961) 2021-11-17 20:04:45 -08:00
Dean Lee
dc8e23c94b framereader: remove memory copy overhead for AVIO (#22894)
* no memory copy

* merge master

better
2021-11-17 13:25:40 +01:00
Dean Lee
20d3cf1fcb framereader: support hardware decoding (#22906)
* support hardware accelerated video decoding

* support cuda only

* print decode info

* add yuv flag back

* fix height

* default is cuda

* fallback to CPU decoding

fix memory leak&cleanup

add output
2021-11-17 11:17:59 +01:00
Dean Lee
c0b9c1f3af framereader: remove deprecated ffmpeg functions (#22895) 2021-11-13 11:55:04 -08:00
Dean Lee
fb8ba34f31 replay: add qcam and YUV flags (#22788)
* common flags

* cleanup

* remove double semicolon

* camera

* apply reviews
2021-11-09 14:08:24 -08:00
Dean Lee
ff33ca3413 util: remove function to_hex (#22792) 2021-11-04 16:18:01 +01:00
Mayfield
5246f0231e enable wshadow (#22756)
* enable wshadow (#22714)

* fix replay

* more build fixes

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-11-02 17:08:53 +01:00
Dean Lee
79651b6c49 replay: fix qcam download error (#22777)
* fix qcam download error

* test loading qlog/qcamera
2021-11-02 11:24:59 +01:00
Dean Lee
09858f7f6f replay: print colored text (#22771) 2021-11-02 11:24:45 +01:00
Dean Lee
0a5ca71603 replay: fix frame lag when replaying to the next segment (#22773) 2021-11-02 11:24:12 +01:00
Willem Melching
f8d050298e framereader.cc: better avformat_open_input error printing 2021-11-02 09:56:49 +01:00
Dean Lee
6f78cc77fc replay: fix hanging on exit if stream thread has not been started yet (#22770) 2021-11-01 16:06:39 -07:00
Dean Lee
2b4a477fbc replay: improve segment download and merge (#22654)
* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
2021-11-01 11:55:56 +01:00
Dean Lee
41e5e76596 replay: fixed the alignment issue when replaying from C3 qcamera (#22734)
* fix qcamera replay

* fix alignment in replay

* upscale yuv if needed

* remove space
2021-11-01 11:06:00 +01:00
Dean Lee
d70e49dca5 replay: build on devices (#22632)
* there is no QThread::create on device

* add files to files_common

* modify SConscript

* no memory_resouce on device

* #define HAS_MEMORY_RESOURCE

* fix hangs on shutdown

* build on device with the --extra flag

* delete later

* setup -> extras

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-10-29 12:21:57 +02:00
Dean Lee
921600b95c framereader: use swscale to handle the YUV conversion (#22710) 2021-10-29 12:14:14 +02:00
Dean Lee
af7dba9250 replay: use an enum for flags (#22635)
* enum flags

* continue

* remove yuv flag for now
2021-10-22 23:28:53 -07:00
Dean Lee
317deeae84 replay: improve seeking, queuing segments (#22625)
* improve seeking, queuing segments

* cleanup,init current_segment_ to 0

* use isSegmentLoaded

* remove to isSegmentMerged
2021-10-19 22:23:53 -07:00
Dean Lee
702fc5850c replay: fix replay looping (#22618) 2021-10-19 12:59:06 -07:00
Dean Lee
107ed35d1e replay: parse route string, replay from the specified segment (#22503)
* parse segment name

* return 0

* update regexp

* with |

* dd

* use struct

* move to class Route

* refactor loadFromLoacal

* simplify loadFromJson

* cleanup load from server

* const ref
2021-10-19 14:45:07 +02:00
Dean Lee
4678903a61 replay/camera: publish each camera in a separate thread (#22591)
* publish frames in a seperate thread for each camera

* cleanup

* cleanup

* prefetch next frame

* cleanup
2021-10-19 14:39:47 +02:00
Willem Melching
e5e90aeae5 replay: improve download prints 2021-10-19 12:48:38 +02:00
Dean Lee
8d1d0c2cd7 replay: improve segment downloading (#22504)
* improve segment downloading

dd

* continue

* log retrying

* check aborting_ in loop

* std::endl

* log download information

* cleanup

* continue

* dd

* move download to seperate funciton

* simplify logging

* continue

* guard ts

* cleanup

* typo
2021-10-19 12:33:12 +02:00
Dean Lee
0189a19b8e replay: refactor FrameReader (#22438)
* decode from the previous keyframe after seek

* less memory

* some stream seems to contian no keyframes

* test random seek

* merge master

* continue

update test_cases

use fr

* merge master
2021-10-18 14:19:23 -07:00
Dean Lee
ccc07db1ee replay: just load one segment to start replaying (#22598) 2021-10-18 12:03:34 -07:00
Dean Lee
94b3d4d267 replay: allocate events in contiguous memory pool (#22535)
* memory pool

* reserve events too

* remove dctor

* cleanup

* move pool into class LogReader

* cleanup include
2021-10-18 11:10:08 +02:00
Dean Lee
f6de10b55a replay: start streaming after segment loaded (#22575)
* start streaming  after segment loaded

dd

* loop from beginning if reaches the end

* isSegmentLoaded

* one loop

* move to ctor

* delete stream_thread_ on exit

* pause streaming while testing seek

* Revert "one loop"

This reverts commit f029cd118f7ac876dee1dbf2b91478403211ad47.

* test:dummy stream thread

* cleanup

* start thread after vipcserver
2021-10-18 11:05:03 +02:00
Dean Lee
c73d9ddaa6 replay: fix hanging on shutdown while downloading (#22592) 2021-10-18 11:03:30 +02:00
Dean Lee
085c424cf9 framereader: remove swscale (#22578) 2021-10-16 13:56:04 -07:00
Dean Lee
ae5eedb0b1 Replay/framereader: fix c3 qcamera padding problem (#22572) 2021-10-16 12:24:34 -07:00
Dean Lee
47ce0b4cd2 replay: added a missing #include <array> on Mac (#22573) 2021-10-16 09:02:19 -07:00
Dean Lee
20b0ae0e65 replay: fix possible segfault in CameraServer (#22556) 2021-10-15 15:34:54 -07:00
Dean Lee
acc52ece20 replay: refactor Route and Segment (#22531)
* new functions

* fix wrong call to qUrl::isLocalFile

* cleanup

* keep extension in cached files

* cleanup

* simplify segment

* delete thread

* add output

* pre-decompress

* remove suffix

* revert remove suffix

* 1 connection for log file

* cleanup

* segment may not be continuous,use map

* don't emit finish if aborting_

* use QFuture and thread pool

* cleanup

* fix segfault in LoadFromLocal

* cleanup

* handle segment failed to load

* output info

* continue error handling

* Remove redundant testSeekTo

* cleanup

* always return true

* keep time is ok now

change to 1s

write 1 byte at the end of the sparse file

* log loading segment

* merge #22476

* Update selfdrive/ui/replay/logreader.cc

* correct connect

* pub message in function

* typo

* Update selfdrive/ui/replay/replay.cc

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-10-15 14:35:17 -07:00
Dean Lee
43a846843f c++ replay: support local routes (#22288)
* support local route

* kind of works

* works

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-10-12 00:51:22 -07:00
Adeeb Shihadeh
f1fcf17c4c replay: support old, pre-pandaStates routes (#22525)
* replay: support old, pre-pandaStates routes

* cleanup
2021-10-11 15:58:44 -07:00
Dean Lee
c801c65b9d replay: get EncoderIndex from capnp::AnyStruct (#22518)
* get EncoderIndex from capnp::AnyStruct

* const
2021-10-11 09:57:35 -07:00
Dean Lee
537dff3ae0 replay/camera: fix incorrect console output (#22491) 2021-10-08 19:51:23 -07:00
Dean Lee
5527736df6 replay: simplify seek&merge (#22463)
* simplify seek&merge

* update test cases

update test cases

* cleanup test cases

* new function currentSeconds

* add TODO

* thread safe publishFrame

* cleanup

* fix 'at x S' not printed if seek back to old time

* exit replay if failed to load route

* move out setCurrentSegment from if statement

* cleanup

* use std::find

* const variables
2021-10-07 17:32:16 +02:00
Greg Hogan
188b634222 replay: print current time when paused (#22468) 2021-10-07 10:41:49 +02:00