Commit Graph

125 Commits

Author SHA1 Message Date
Kacper Rączy
6f51ee8267 URLFile: exception type for failed requests (#30330)
URLFileException for URLFile request errors
old-commit-hash: c27e977475
2023-10-25 16:20:12 -07:00
Greg Hogan
a544533476 framereader: eliminate tempfile usage (#30289)
old-commit-hash: 68acb26aa2
2023-10-18 19:48:26 -07:00
Greg Hogan
99d094a006 fix ffprobe on device (#30286)
* fix ffprobe on device

* use FileReader

* no more requests
old-commit-hash: 59e947dc6c
2023-10-18 18:31:14 -07:00
Justin Newberry
ef1acd4e6c Proc Replay: cleanup iterable log typing (#30261)
* cleanup log typing

* use iterable

* fix those

* Cleanup

* cleanup

* missed that one
old-commit-hash: 1820b5f259
2023-10-17 12:58:28 -07:00
Greg Hogan
a62749f527 vidindex improvements (#30196)
* vidindex improvements

* fix spelling
old-commit-hash: 6f98a987af
2023-10-05 22:48:50 -07:00
Greg Hogan
e0469bdde6 filereader cleanup (#30191)
* filereader cleanup

* make name return url

* remove unused imports
old-commit-hash: 86b90a8ba2
2023-10-05 19:45:46 -07:00
Greg Hogan
29bb2cf867 convert vidindex to python (#30176)
* convert vidindex to python

* fix whitespace

* corrupt file option

* fix up typings

* fix return type

* update framereader

* change length delimiter to uint32 value

* change length to uint32 value

* move url_file changes to separate PR

* cleanup caching

* revert whitespace change

* fix frame type param type
old-commit-hash: f8e488f881
2023-10-05 18:40:24 -07:00
Justin Newberry
a9153302a3 URLFile: don't cache non-existent file's lengths (#30071)
* urlfile empty

* simplify with mock

* better test name

* PR cleanup

* cleanup the length file
old-commit-hash: fa51bbc236
2023-10-04 12:57:56 -07:00
YassineYousfi
eb7cad40dd framereader: add cache_dir argument (#29904)
* framereader: add cache_dir argument

* make it an env var
old-commit-hash: 880157f5a6
2023-09-14 14:36:32 -07:00
Greg Hogan
b5375cc82d importing auth_config.py should not have side effects (#29893)
import should not create directory
old-commit-hash: 140a4ba558
2023-09-12 18:00:18 +00:00
Justin Newberry
dc6078a6c3 System: use paths for download folders too (#29818)
* use paths for download folders too

* trailing slash

* reset that
old-commit-hash: 73eda51a11
2023-09-07 12:49:03 -07:00
Justin Newberry
3b91610c0b Precommit: ensure executable bit is set (#29784)
* precommit ensure executable

* exclude tinygrad

* bump submodules

* exclude tinygrad globally
old-commit-hash: 4e69937d0d
2023-09-05 16:33:26 -07:00
Justin Newberry
1cc555c37b Tests: expand temporary directory helpers for more directories (#29735)
* create helpers for testing

* document it

* fix other tests
old-commit-hash: 405d09714c
2023-08-31 14:00:31 -07:00
Justin Newberry
279bd020dc Tests: logmessaged reduce global state (#29680)
* reduce global state

* use a decorator here too

* use that one

* use base imports

* ipchandler in setup

* move to common dir

* move to helpers

* wip

* fix the decorator
old-commit-hash: 5dc7028f91
2023-08-29 13:36:26 -07:00
Justin Newberry
b08c5d7517 Tests: test_caching reduce global state (#29681)
old-commit-hash: dd9b0231c6
2023-08-29 10:06:32 -07:00
Justin Newberry
c4f322ce2d Linter: remove pylint (#29611)
remove pylint
old-commit-hash: 1ee6ed4b16
2023-08-24 16:30:54 -07:00
Dean Lee
0984736972 cpplint: add filter build/include_subdir (#29585)
old-commit-hash: 21da5a216f
2023-08-24 15:42:17 +01:00
Adeeb Shihadeh
16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b6
2023-08-20 20:49:55 -07:00
Adeeb Shihadeh
e76af64065 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
old-commit-hash: b11b14aff8
2023-08-12 17:15:16 -07:00
Justin Newberry
d487c0501f Ruff: enable most of bugbear (#29320)
* added mutable default args

* most of the Bs

* add comment about lrucache
old-commit-hash: 62c1e65924
2023-08-11 15:33:49 -07:00
Justin Newberry
715f028a0e Ruff: enable flake8-builtins (#29315)
* enable flake8-builtins

* replace any with contains

* fix typo in pack

* fix type

* format is from the parent module, has to be enabled

* item_id

* fix item_id

* disable for id since that's what the remote server returns
old-commit-hash: 8793cbff40
2023-08-10 18:58:22 -07:00
Justin Newberry
7d6d3dc9ea Ruff: flake8 pie (misc lints) (#29318)
flake8 pie
old-commit-hash: af3ed37c5c
2023-08-10 15:28:51 -07:00
Shane Smiskol
d78ebfdc45 replay: new demo route (#29215)
* new demo route

* replace other references (except gps test)

* Apply suggestions from code review

* it's 13 segments
old-commit-hash: acc732efbb
2023-08-03 13:18:44 -07:00
Kacper Rączy
2edd65ad4e process_replay: helpers (#28367)
* process_replay helper

* Refactor

* refactor check_enabled uses

* fix __init__

* noqa in __init__ imports

* Move compare_logs outside of process_replay

* Move save_logs to tools.lib.helpers

* Remove save_log from compare_logs

* fix lint issues
old-commit-hash: 64377a88f9
2023-06-06 10:31:13 -07:00
Adeeb Shihadeh
3a968c67e1 loggerd: move to system/ (#27534)
old-commit-hash: 94eb215980
2023-03-08 11:20:49 -08:00
Adeeb Shihadeh
20de7c4b55 bootlog: add helper to get a bootlog by segment id
old-commit-hash: 285fd56a1d
2022-10-04 17:47:37 -07:00
Adeeb Shihadeh
0325a33752 add pyside2 package (#25602)
old-commit-hash: c6b749fb96
2022-08-30 15:10:52 -07:00
Adeeb Shihadeh
35ff65121e pre-commit: add codespell (#25571)
old-commit-hash: 6590fb2b93
2022-08-30 11:20:55 -07:00
Mitchell Goff
a2d87101c7 Update DATA_ENDPOINT default to data-raw.comma.internal (#25597)
old-commit-hash: acbb230e0a
2022-08-29 13:31:43 -07:00
Adeeb Shihadeh
8a7454eec3 implement string representation for Bootlog
old-commit-hash: e6e8607306
2022-07-21 14:14:34 -07:00
Adeeb Shihadeh
9a45a3b99b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
old-commit-hash: 1139fe507b
2022-06-11 16:38:24 -07:00
Lukas Petersson
a16a20005b process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 86f73a507e
2022-06-01 22:02:42 -07:00
George Hotz
2a6ffaf71d nv12: encoderd avoids a full frame copy (#24519)
* rgb to nv12

* nv12 works (w memcpy)

* correct now

* no copy

* fix nv12 with fast debayer

* reverts of unused stuff

* ui use nv12

* comment out thumbnails for now

* rebase fix

* dm read nv12

* model read nv12

* fix ffmpeg encoder

* thumbnails from nv12

* replay to nv12

* python framereader support nv12

* remove hardcoded frame/buffer sizes

* fix build

* ffmpeg encoder fix buffers

* small cleanup

* reduce power usage test

* fix cpu usage test

* fix snapshot

* fix loggerd test

* bump cereal

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
old-commit-hash: ea5b8cdfb1
2022-06-01 17:18:28 +02:00
Willem Melching
f957c16a7b make normal logreader more robust (#24577)
old-commit-hash: 194e5fdf1c
2022-05-18 12:21:01 +02:00
ntegan1
0a3fde5334 Tools: allow using plotjuggler on local data (#24473)
local juggler
old-commit-hash: 07f7570840
2022-05-17 19:50:23 +02:00
Dylan Herman
2dfba97175 add mypy check for return-any (#24379)
* add mypy check for return-any

* remove unused import

* typing

* remove unnecessary variable typing

* cleanup

* cleanup

* bump submodules

* small fixes

* only a problem on mac

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: d2eef1955c
2022-05-12 14:59:00 +02:00
grekiki
7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cd
2022-05-12 14:08:11 +02:00
Shane Smiskol
cf0c8aac08 URLFile: raise exception if remote URL doesn't exist when using cache (#24432)
* URLFile returns empty bytes if using cache and remote file doesn't exist

* better exception

* assert on cached files
old-commit-hash: 6163dd5ca0
2022-05-09 13:31:55 -07:00
ntegan1
d4d72ab938 Tools: allow uncompressed logs (#24471)
old-commit-hash: bf269bd883
2022-05-09 12:07:19 -07:00
Lukas Petersson
2d6ae65ea7 Latency logger bad data robustness (#24343)
* robustness against bad data

* sort by time logreader

* loop until last frameid
old-commit-hash: 1439867d16
2022-04-29 13:32:21 -07:00
grekiki
77d6d9a083 mypy: enable --warn-unused-ignores (#24287)
* this is the easy part to check

* Not sure what this fixes, 8aaf00c6a4

* bump laika

* works here

* cleanup

* add types-certifi

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 049b163ea0
2022-04-27 14:26:15 -07:00
Lukas Petersson
df8476118b Latency logging 2 (#24058)
* msg_order and gantt

* frameId in long/lat planner

* track frame id

* controls frame id

* graph tracked events

* graph json

* cloudlog timestamp

* c++ cloudlog

* add frame id

* bug fixes

* bug fixes

* frame id visionicp

* bug fixes and debug level

* timestamp log placement

* print timestamps in table

* translate events

* more logging

* bug fixes

* daemon boardd

* print logs with boardd

* more timestamp logs

* cleanup

* remove publish logs

* bug fix

* timestamp received

* timestamp received

* bug fixes

* use json lib

* ignore driver camera

* prep for new timestamp pipeline

* bug fix

* read new pipeline unfinnished

* read new pipeline

* bug fix

* add frame to controlsstate

* remove controlsstate

* print

* cleanup

* more cleanup + bug fix

* clock build issue

* remove unused imports

* format durations

* increase speed

* pr comments fixes

* conflicts

* set MANAGER_DAEMON for boardd

* clean script code

* bug fix + argparse

* remove rcv time

* bug fixes

* print without tabulate

* fix pre-commits

* plot gnatt

* color bug fix

* read without timestampextra

* bump panda

* mono time instead of frame id

* finnish script

* clean unused

* clean unused logging

* monotonic + json fixes

* del test

* remove whilelines

* bump laika

* cleanup

* remove deps

* logs nicer strings

* remove plotting from scirpt

* reset pipfile

* reset pipfile

* nicer strings

* bug fix

* bug fix

* pr comments cleaning

* remove plotting

* bug fix

* new demo route

* bump opendbc and panda

* cereal master

* cereal master

* script less komplex

* assertions

* matplotlib

* readme

* Update README.md

* graph html

* design fixes

* more code design

* Update common/logging_extra.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* whitespace

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update tools/latency_logger/latency_logger.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* pr comments

* bug fix

* readme + env once

* clean swaglog

* bug fix

* Update tools/latencylogger/README.md

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* revert

* revert

* clean swaglog with error

* remove typo file

* revert graph

* cereal

* submodules

* whitespaces

* update refs

Co-authored-by: Bruce Wayne <batman@workstation-openpilot2.internal>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 65fca83abe
2022-04-05 21:05:45 -07:00
Adeeb Shihadeh
6f88ca1c6f MultiLogIterator: add function to reset
old-commit-hash: c95ec211e1
2022-03-25 19:56:22 -07:00
Adeeb Shihadeh
bd71c594a2 pylint: enforce indentation (W0311) (#24039)
* pylint: enforce indentation (W0311)

* few more
old-commit-hash: 8af20af66d
2022-03-24 23:23:29 -07:00
mitchellgoffpc
b7e99b3f45 Ignore BrokenPipeError
old-commit-hash: 5c8c6faa9a
2022-03-22 12:52:25 -07:00
Adeeb Shihadeh
8a682597ca can_print_changes improvements
old-commit-hash: 1680f1e251
2022-03-10 14:32:19 -08:00
xx979xx
663579dd78 tools: add MultiLogIterator example to readme (#23917)
* update LogReader doc

use MultiLogIterator to read the entire route, get timestamps and deal with exceptions

* bring back the old example

* clean f-strings

* simplify

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: da50930450
2022-03-07 10:04:41 -08:00
Greg Hogan
20670f0740 filereader better name for cd:/ replacement
old-commit-hash: a6b46c311e
2022-01-13 18:21:19 -08:00
Greg Hogan
c0e265b859 fix filereader cd:/ replacement
old-commit-hash: df35ebdceb
2022-01-13 16:09:56 -08:00
Greg Hogan
0b659ce9b6 eliminate xx filereader (#23514)
* eliminate xx filereader

* framereader cleanup
old-commit-hash: d5f4fdb1db
2022-01-13 15:56:54 -08:00