Commit Graph

125 Commits

Author SHA1 Message Date
Kacper Rączy
c27e977475 URLFile: exception type for failed requests (#30330)
URLFileException for URLFile request errors
2023-10-25 16:20:12 -07:00
Greg Hogan
68acb26aa2 framereader: eliminate tempfile usage (#30289) 2023-10-18 19:48:26 -07:00
Greg Hogan
59e947dc6c fix ffprobe on device (#30286)
* fix ffprobe on device

* use FileReader

* no more requests
2023-10-18 18:31:14 -07:00
Justin Newberry
1820b5f259 Proc Replay: cleanup iterable log typing (#30261)
* cleanup log typing

* use iterable

* fix those

* Cleanup

* cleanup

* missed that one
2023-10-17 12:58:28 -07:00
Greg Hogan
6f98a987af vidindex improvements (#30196)
* vidindex improvements

* fix spelling
2023-10-05 22:48:50 -07:00
Greg Hogan
86b90a8ba2 filereader cleanup (#30191)
* filereader cleanup

* make name return url

* remove unused imports
2023-10-05 19:45:46 -07:00
Greg Hogan
f8e488f881 convert vidindex to python (#30176)
* convert vidindex to python

* fix whitespace

* corrupt file option

* fix up typings

* fix return type

* update framereader

* change length delimiter to uint32 value

* change length to uint32 value

* move url_file changes to separate PR

* cleanup caching

* revert whitespace change

* fix frame type param type
2023-10-05 18:40:24 -07:00
Justin Newberry
fa51bbc236 URLFile: don't cache non-existent file's lengths (#30071)
* urlfile empty

* simplify with mock

* better test name

* PR cleanup

* cleanup the length file
2023-10-04 12:57:56 -07:00
YassineYousfi
880157f5a6 framereader: add cache_dir argument (#29904)
* framereader: add cache_dir argument

* make it an env var
2023-09-14 14:36:32 -07:00
Greg Hogan
140a4ba558 importing auth_config.py should not have side effects (#29893)
import should not create directory
2023-09-12 18:00:18 +00:00
Justin Newberry
73eda51a11 System: use paths for download folders too (#29818)
* use paths for download folders too

* trailing slash

* reset that
2023-09-07 12:49:03 -07:00
Justin Newberry
4e69937d0d Precommit: ensure executable bit is set (#29784)
* precommit ensure executable

* exclude tinygrad

* bump submodules

* exclude tinygrad globally
2023-09-05 16:33:26 -07:00
Justin Newberry
405d09714c Tests: expand temporary directory helpers for more directories (#29735)
* create helpers for testing

* document it

* fix other tests
2023-08-31 14:00:31 -07:00
Justin Newberry
5dc7028f91 Tests: logmessaged reduce global state (#29680)
* reduce global state

* use a decorator here too

* use that one

* use base imports

* ipchandler in setup

* move to common dir

* move to helpers

* wip

* fix the decorator
2023-08-29 13:36:26 -07:00
Justin Newberry
dd9b0231c6 Tests: test_caching reduce global state (#29681) 2023-08-29 10:06:32 -07:00
Justin Newberry
1ee6ed4b16 Linter: remove pylint (#29611)
remove pylint
2023-08-24 16:30:54 -07:00
Dean Lee
21da5a216f cpplint: add filter build/include_subdir (#29585) 2023-08-24 15:42:17 +01:00
Adeeb Shihadeh
a9626f95b6 add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
2023-08-20 20:49:55 -07:00
Adeeb Shihadeh
b11b14aff8 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
2023-08-12 17:15:16 -07:00
Justin Newberry
62c1e65924 Ruff: enable most of bugbear (#29320)
* added mutable default args

* most of the Bs

* add comment about lrucache
2023-08-11 15:33:49 -07:00
Justin Newberry
8793cbff40 Ruff: enable flake8-builtins (#29315)
* enable flake8-builtins

* replace any with contains

* fix typo in pack

* fix type

* format is from the parent module, has to be enabled

* item_id

* fix item_id

* disable for id since that's what the remote server returns
2023-08-10 18:58:22 -07:00
Justin Newberry
af3ed37c5c Ruff: flake8 pie (misc lints) (#29318)
flake8 pie
2023-08-10 15:28:51 -07:00
Shane Smiskol
acc732efbb replay: new demo route (#29215)
* new demo route

* replace other references (except gps test)

* Apply suggestions from code review

* it's 13 segments
2023-08-03 13:18:44 -07:00
Kacper Rączy
64377a88f9 process_replay: helpers (#28367)
* process_replay helper

* Refactor

* refactor check_enabled uses

* fix __init__

* noqa in __init__ imports

* Move compare_logs outside of process_replay

* Move save_logs to tools.lib.helpers

* Remove save_log from compare_logs

* fix lint issues
2023-06-06 10:31:13 -07:00
Adeeb Shihadeh
94eb215980 loggerd: move to system/ (#27534) 2023-03-08 11:20:49 -08:00
Adeeb Shihadeh
285fd56a1d bootlog: add helper to get a bootlog by segment id 2022-10-04 17:47:37 -07:00
Adeeb Shihadeh
c6b749fb96 add pyside2 package (#25602) 2022-08-30 15:10:52 -07:00
Adeeb Shihadeh
6590fb2b93 pre-commit: add codespell (#25571) 2022-08-30 11:20:55 -07:00
Mitchell Goff
acbb230e0a Update DATA_ENDPOINT default to data-raw.comma.internal (#25597) 2022-08-29 13:31:43 -07:00
Adeeb Shihadeh
e6e8607306 implement string representation for Bootlog 2022-07-21 14:14:34 -07:00
Adeeb Shihadeh
1139fe507b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
2022-06-11 16:38:24 -07:00
Lukas Petersson
86f73a507e process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-06-01 22:02:42 -07:00
George Hotz
ea5b8cdfb1 nv12: encoderd avoids a full frame copy (#24519)
* rgb to nv12

* nv12 works (w memcpy)

* correct now

* no copy

* fix nv12 with fast debayer

* reverts of unused stuff

* ui use nv12

* comment out thumbnails for now

* rebase fix

* dm read nv12

* model read nv12

* fix ffmpeg encoder

* thumbnails from nv12

* replay to nv12

* python framereader support nv12

* remove hardcoded frame/buffer sizes

* fix build

* ffmpeg encoder fix buffers

* small cleanup

* reduce power usage test

* fix cpu usage test

* fix snapshot

* fix loggerd test

* bump cereal

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
2022-06-01 17:18:28 +02:00
Willem Melching
194e5fdf1c make normal logreader more robust (#24577) 2022-05-18 12:21:01 +02:00
ntegan1
07f7570840 Tools: allow using plotjuggler on local data (#24473)
local juggler
2022-05-17 19:50:23 +02:00
Dylan Herman
d2eef1955c add mypy check for return-any (#24379)
* add mypy check for return-any

* remove unused import

* typing

* remove unnecessary variable typing

* cleanup

* cleanup

* bump submodules

* small fixes

* only a problem on mac

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-05-12 14:59:00 +02:00
grekiki
17e33978cd Mypy fixes for --check-untyped-defs (#24372)
more type fixes
2022-05-12 14:08:11 +02:00
Shane Smiskol
6163dd5ca0 URLFile: raise exception if remote URL doesn't exist when using cache (#24432)
* URLFile returns empty bytes if using cache and remote file doesn't exist

* better exception

* assert on cached files
2022-05-09 13:31:55 -07:00
ntegan1
bf269bd883 Tools: allow uncompressed logs (#24471) 2022-05-09 12:07:19 -07:00
Lukas Petersson
1439867d16 Latency logger bad data robustness (#24343)
* robustness against bad data

* sort by time logreader

* loop until last frameid
2022-04-29 13:32:21 -07:00
grekiki
049b163ea0 mypy: enable --warn-unused-ignores (#24287)
* this is the easy part to check

* Not sure what this fixes, 8aaf00c6a4

* bump laika

* works here

* cleanup

* add types-certifi

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-04-27 14:26:15 -07:00
Lukas Petersson
65fca83abe Latency logging 2 (#24058)
* msg_order and gantt

* frameId in long/lat planner

* track frame id

* controls frame id

* graph tracked events

* graph json

* cloudlog timestamp

* c++ cloudlog

* add frame id

* bug fixes

* bug fixes

* frame id visionicp

* bug fixes and debug level

* timestamp log placement

* print timestamps in table

* translate events

* more logging

* bug fixes

* daemon boardd

* print logs with boardd

* more timestamp logs

* cleanup

* remove publish logs

* bug fix

* timestamp received

* timestamp received

* bug fixes

* use json lib

* ignore driver camera

* prep for new timestamp pipeline

* bug fix

* read new pipeline unfinnished

* read new pipeline

* bug fix

* add frame to controlsstate

* remove controlsstate

* print

* cleanup

* more cleanup + bug fix

* clock build issue

* remove unused imports

* format durations

* increase speed

* pr comments fixes

* conflicts

* set MANAGER_DAEMON for boardd

* clean script code

* bug fix + argparse

* remove rcv time

* bug fixes

* print without tabulate

* fix pre-commits

* plot gnatt

* color bug fix

* read without timestampextra

* bump panda

* mono time instead of frame id

* finnish script

* clean unused

* clean unused logging

* monotonic + json fixes

* del test

* remove whilelines

* bump laika

* cleanup

* remove deps

* logs nicer strings

* remove plotting from scirpt

* reset pipfile

* reset pipfile

* nicer strings

* bug fix

* bug fix

* pr comments cleaning

* remove plotting

* bug fix

* new demo route

* bump opendbc and panda

* cereal master

* cereal master

* script less komplex

* assertions

* matplotlib

* readme

* Update README.md

* graph html

* design fixes

* more code design

* Update common/logging_extra.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* whitespace

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update tools/latency_logger/latency_logger.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* pr comments

* bug fix

* readme + env once

* clean swaglog

* bug fix

* Update tools/latencylogger/README.md

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* revert

* revert

* clean swaglog with error

* remove typo file

* revert graph

* cereal

* submodules

* whitespaces

* update refs

Co-authored-by: Bruce Wayne <batman@workstation-openpilot2.internal>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-04-05 21:05:45 -07:00
Adeeb Shihadeh
c95ec211e1 MultiLogIterator: add function to reset 2022-03-25 19:56:22 -07:00
Adeeb Shihadeh
8af20af66d pylint: enforce indentation (W0311) (#24039)
* pylint: enforce indentation (W0311)

* few more
2022-03-24 23:23:29 -07:00
mitchellgoffpc
5c8c6faa9a Ignore BrokenPipeError 2022-03-22 12:52:25 -07:00
Adeeb Shihadeh
1680f1e251 can_print_changes improvements 2022-03-10 14:32:19 -08:00
xx979xx
da50930450 tools: add MultiLogIterator example to readme (#23917)
* update LogReader doc

use MultiLogIterator to read the entire route, get timestamps and deal with exceptions

* bring back the old example

* clean f-strings

* simplify

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-03-07 10:04:41 -08:00
Greg Hogan
a6b46c311e filereader better name for cd:/ replacement 2022-01-13 18:21:19 -08:00
Greg Hogan
df35ebdceb fix filereader cd:/ replacement 2022-01-13 16:09:56 -08:00
Greg Hogan
d5f4fdb1db eliminate xx filereader (#23514)
* eliminate xx filereader

* framereader cleanup
2022-01-13 15:56:54 -08:00