Commit Graph

117 Commits

Author SHA1 Message Date
YassineYousfi
eb7cad40dd framereader: add cache_dir argument (#29904)
* framereader: add cache_dir argument

* make it an env var
old-commit-hash: 880157f5a6
2023-09-14 14:36:32 -07:00
Greg Hogan
b5375cc82d importing auth_config.py should not have side effects (#29893)
import should not create directory
old-commit-hash: 140a4ba558
2023-09-12 18:00:18 +00:00
Justin Newberry
dc6078a6c3 System: use paths for download folders too (#29818)
* use paths for download folders too

* trailing slash

* reset that
old-commit-hash: 73eda51a11
2023-09-07 12:49:03 -07:00
Justin Newberry
3b91610c0b Precommit: ensure executable bit is set (#29784)
* precommit ensure executable

* exclude tinygrad

* bump submodules

* exclude tinygrad globally
old-commit-hash: 4e69937d0d
2023-09-05 16:33:26 -07:00
Justin Newberry
1cc555c37b Tests: expand temporary directory helpers for more directories (#29735)
* create helpers for testing

* document it

* fix other tests
old-commit-hash: 405d09714c
2023-08-31 14:00:31 -07:00
Justin Newberry
279bd020dc Tests: logmessaged reduce global state (#29680)
* reduce global state

* use a decorator here too

* use that one

* use base imports

* ipchandler in setup

* move to common dir

* move to helpers

* wip

* fix the decorator
old-commit-hash: 5dc7028f91
2023-08-29 13:36:26 -07:00
Justin Newberry
b08c5d7517 Tests: test_caching reduce global state (#29681)
old-commit-hash: dd9b0231c6
2023-08-29 10:06:32 -07:00
Justin Newberry
c4f322ce2d Linter: remove pylint (#29611)
remove pylint
old-commit-hash: 1ee6ed4b16
2023-08-24 16:30:54 -07:00
Dean Lee
0984736972 cpplint: add filter build/include_subdir (#29585)
old-commit-hash: 21da5a216f
2023-08-24 15:42:17 +01:00
Adeeb Shihadeh
16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b6
2023-08-20 20:49:55 -07:00
Adeeb Shihadeh
e76af64065 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
old-commit-hash: b11b14aff8
2023-08-12 17:15:16 -07:00
Justin Newberry
d487c0501f Ruff: enable most of bugbear (#29320)
* added mutable default args

* most of the Bs

* add comment about lrucache
old-commit-hash: 62c1e65924
2023-08-11 15:33:49 -07:00
Justin Newberry
715f028a0e Ruff: enable flake8-builtins (#29315)
* enable flake8-builtins

* replace any with contains

* fix typo in pack

* fix type

* format is from the parent module, has to be enabled

* item_id

* fix item_id

* disable for id since that's what the remote server returns
old-commit-hash: 8793cbff40
2023-08-10 18:58:22 -07:00
Justin Newberry
7d6d3dc9ea Ruff: flake8 pie (misc lints) (#29318)
flake8 pie
old-commit-hash: af3ed37c5c
2023-08-10 15:28:51 -07:00
Shane Smiskol
d78ebfdc45 replay: new demo route (#29215)
* new demo route

* replace other references (except gps test)

* Apply suggestions from code review

* it's 13 segments
old-commit-hash: acc732efbb
2023-08-03 13:18:44 -07:00
Kacper Rączy
2edd65ad4e process_replay: helpers (#28367)
* process_replay helper

* Refactor

* refactor check_enabled uses

* fix __init__

* noqa in __init__ imports

* Move compare_logs outside of process_replay

* Move save_logs to tools.lib.helpers

* Remove save_log from compare_logs

* fix lint issues
old-commit-hash: 64377a88f9
2023-06-06 10:31:13 -07:00
Adeeb Shihadeh
3a968c67e1 loggerd: move to system/ (#27534)
old-commit-hash: 94eb215980
2023-03-08 11:20:49 -08:00
Adeeb Shihadeh
20de7c4b55 bootlog: add helper to get a bootlog by segment id
old-commit-hash: 285fd56a1d
2022-10-04 17:47:37 -07:00
Adeeb Shihadeh
0325a33752 add pyside2 package (#25602)
old-commit-hash: c6b749fb96
2022-08-30 15:10:52 -07:00
Adeeb Shihadeh
35ff65121e pre-commit: add codespell (#25571)
old-commit-hash: 6590fb2b93
2022-08-30 11:20:55 -07:00
Mitchell Goff
a2d87101c7 Update DATA_ENDPOINT default to data-raw.comma.internal (#25597)
old-commit-hash: acbb230e0a
2022-08-29 13:31:43 -07:00
Adeeb Shihadeh
8a7454eec3 implement string representation for Bootlog
old-commit-hash: e6e8607306
2022-07-21 14:14:34 -07:00
Adeeb Shihadeh
9a45a3b99b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
old-commit-hash: 1139fe507b
2022-06-11 16:38:24 -07:00
Lukas Petersson
a16a20005b process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 86f73a507e
2022-06-01 22:02:42 -07:00
George Hotz
2a6ffaf71d nv12: encoderd avoids a full frame copy (#24519)
* rgb to nv12

* nv12 works (w memcpy)

* correct now

* no copy

* fix nv12 with fast debayer

* reverts of unused stuff

* ui use nv12

* comment out thumbnails for now

* rebase fix

* dm read nv12

* model read nv12

* fix ffmpeg encoder

* thumbnails from nv12

* replay to nv12

* python framereader support nv12

* remove hardcoded frame/buffer sizes

* fix build

* ffmpeg encoder fix buffers

* small cleanup

* reduce power usage test

* fix cpu usage test

* fix snapshot

* fix loggerd test

* bump cereal

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
old-commit-hash: ea5b8cdfb1
2022-06-01 17:18:28 +02:00
Willem Melching
f957c16a7b make normal logreader more robust (#24577)
old-commit-hash: 194e5fdf1c
2022-05-18 12:21:01 +02:00
ntegan1
0a3fde5334 Tools: allow using plotjuggler on local data (#24473)
local juggler
old-commit-hash: 07f7570840
2022-05-17 19:50:23 +02:00
Dylan Herman
2dfba97175 add mypy check for return-any (#24379)
* add mypy check for return-any

* remove unused import

* typing

* remove unnecessary variable typing

* cleanup

* cleanup

* bump submodules

* small fixes

* only a problem on mac

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: d2eef1955c
2022-05-12 14:59:00 +02:00
grekiki
7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cd
2022-05-12 14:08:11 +02:00
Shane Smiskol
cf0c8aac08 URLFile: raise exception if remote URL doesn't exist when using cache (#24432)
* URLFile returns empty bytes if using cache and remote file doesn't exist

* better exception

* assert on cached files
old-commit-hash: 6163dd5ca0
2022-05-09 13:31:55 -07:00
ntegan1
d4d72ab938 Tools: allow uncompressed logs (#24471)
old-commit-hash: bf269bd883
2022-05-09 12:07:19 -07:00
Lukas Petersson
2d6ae65ea7 Latency logger bad data robustness (#24343)
* robustness against bad data

* sort by time logreader

* loop until last frameid
old-commit-hash: 1439867d16
2022-04-29 13:32:21 -07:00
grekiki
77d6d9a083 mypy: enable --warn-unused-ignores (#24287)
* this is the easy part to check

* Not sure what this fixes, 8aaf00c6a4

* bump laika

* works here

* cleanup

* add types-certifi

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 049b163ea0
2022-04-27 14:26:15 -07:00
Lukas Petersson
df8476118b Latency logging 2 (#24058)
* msg_order and gantt

* frameId in long/lat planner

* track frame id

* controls frame id

* graph tracked events

* graph json

* cloudlog timestamp

* c++ cloudlog

* add frame id

* bug fixes

* bug fixes

* frame id visionicp

* bug fixes and debug level

* timestamp log placement

* print timestamps in table

* translate events

* more logging

* bug fixes

* daemon boardd

* print logs with boardd

* more timestamp logs

* cleanup

* remove publish logs

* bug fix

* timestamp received

* timestamp received

* bug fixes

* use json lib

* ignore driver camera

* prep for new timestamp pipeline

* bug fix

* read new pipeline unfinnished

* read new pipeline

* bug fix

* add frame to controlsstate

* remove controlsstate

* print

* cleanup

* more cleanup + bug fix

* clock build issue

* remove unused imports

* format durations

* increase speed

* pr comments fixes

* conflicts

* set MANAGER_DAEMON for boardd

* clean script code

* bug fix + argparse

* remove rcv time

* bug fixes

* print without tabulate

* fix pre-commits

* plot gnatt

* color bug fix

* read without timestampextra

* bump panda

* mono time instead of frame id

* finnish script

* clean unused

* clean unused logging

* monotonic + json fixes

* del test

* remove whilelines

* bump laika

* cleanup

* remove deps

* logs nicer strings

* remove plotting from scirpt

* reset pipfile

* reset pipfile

* nicer strings

* bug fix

* bug fix

* pr comments cleaning

* remove plotting

* bug fix

* new demo route

* bump opendbc and panda

* cereal master

* cereal master

* script less komplex

* assertions

* matplotlib

* readme

* Update README.md

* graph html

* design fixes

* more code design

* Update common/logging_extra.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* whitespace

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update tools/latency_logger/latency_logger.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* pr comments

* bug fix

* readme + env once

* clean swaglog

* bug fix

* Update tools/latencylogger/README.md

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* revert

* revert

* clean swaglog with error

* remove typo file

* revert graph

* cereal

* submodules

* whitespaces

* update refs

Co-authored-by: Bruce Wayne <batman@workstation-openpilot2.internal>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 65fca83abe
2022-04-05 21:05:45 -07:00
Adeeb Shihadeh
6f88ca1c6f MultiLogIterator: add function to reset
old-commit-hash: c95ec211e1
2022-03-25 19:56:22 -07:00
Adeeb Shihadeh
bd71c594a2 pylint: enforce indentation (W0311) (#24039)
* pylint: enforce indentation (W0311)

* few more
old-commit-hash: 8af20af66d
2022-03-24 23:23:29 -07:00
mitchellgoffpc
b7e99b3f45 Ignore BrokenPipeError
old-commit-hash: 5c8c6faa9a
2022-03-22 12:52:25 -07:00
Adeeb Shihadeh
8a682597ca can_print_changes improvements
old-commit-hash: 1680f1e251
2022-03-10 14:32:19 -08:00
xx979xx
663579dd78 tools: add MultiLogIterator example to readme (#23917)
* update LogReader doc

use MultiLogIterator to read the entire route, get timestamps and deal with exceptions

* bring back the old example

* clean f-strings

* simplify

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: da50930450
2022-03-07 10:04:41 -08:00
Greg Hogan
20670f0740 filereader better name for cd:/ replacement
old-commit-hash: a6b46c311e
2022-01-13 18:21:19 -08:00
Greg Hogan
c0e265b859 fix filereader cd:/ replacement
old-commit-hash: df35ebdceb
2022-01-13 16:09:56 -08:00
Greg Hogan
0b659ce9b6 eliminate xx filereader (#23514)
* eliminate xx filereader

* framereader cleanup
old-commit-hash: d5f4fdb1db
2022-01-13 15:56:54 -08:00
Adeeb Shihadeh
8f9a6b06df tools: add bootlog helpers (#23504)
* tools: add bootlog helpers

* this is nice

* types
old-commit-hash: 7d7c0ff35f
2022-01-12 21:37:43 -08:00
Greg Hogan
bb3402b54d plotjuggler: support segment names (#23263)
* plotjuggler: support segment names

* update docs

* generic parser

* convert segment number to int when parsing

* add SegmentName

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 71132edf17
2022-01-04 15:55:46 +01:00
Adeeb Shihadeh
ba30137b2a MultiLogIterator: fix after adding sort_by_time
old-commit-hash: bbb47dc860
2022-01-02 20:08:57 -08:00
Adeeb Shihadeh
c036e64c86 MultiLogIterator: remove wraparound mode (#23360)
old-commit-hash: 92f194decc
2022-01-02 15:16:44 -08:00
Greg Hogan
2f843c3023 LogReader: add arg to sort by time (#23346)
* logreader optional sort by time

* robust logreader sort by time option
old-commit-hash: 9992ea1ae8
2022-01-02 11:56:53 -08:00
Ewout ter Hoeven
f206ebd054 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
old-commit-hash: 332f568a82
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven
a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh
0827af77e5 MultiLogIterator: default wraparound to false
old-commit-hash: c8356d3b8f
2021-12-02 08:14:48 -08:00