Commit Graph

125 Commits

Author SHA1 Message Date
Adeeb Shihadeh
8f9a6b06df tools: add bootlog helpers (#23504)
* tools: add bootlog helpers

* this is nice

* types
old-commit-hash: 7d7c0ff35f
2022-01-12 21:37:43 -08:00
Greg Hogan
bb3402b54d plotjuggler: support segment names (#23263)
* plotjuggler: support segment names

* update docs

* generic parser

* convert segment number to int when parsing

* add SegmentName

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 71132edf17
2022-01-04 15:55:46 +01:00
Adeeb Shihadeh
ba30137b2a MultiLogIterator: fix after adding sort_by_time
old-commit-hash: bbb47dc860
2022-01-02 20:08:57 -08:00
Adeeb Shihadeh
c036e64c86 MultiLogIterator: remove wraparound mode (#23360)
old-commit-hash: 92f194decc
2022-01-02 15:16:44 -08:00
Greg Hogan
2f843c3023 LogReader: add arg to sort by time (#23346)
* logreader optional sort by time

* robust logreader sort by time option
old-commit-hash: 9992ea1ae8
2022-01-02 11:56:53 -08:00
Ewout ter Hoeven
f206ebd054 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
old-commit-hash: 332f568a82
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven
a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh
0827af77e5 MultiLogIterator: default wraparound to false
old-commit-hash: c8356d3b8f
2021-12-02 08:14:48 -08:00
Adeeb Shihadeh
6c90a5d445 logreader: remove log path print
old-commit-hash: 01b99eae35
2021-11-29 13:44:20 -08:00
Adeeb Shihadeh
973a48acef remove dead + duplicate python code
old-commit-hash: 08f9316b56
2021-11-28 14:02:06 -08:00
Willem Melching
fd3140fd6f cleanup unused pip packages (#22947)
* cleanup unused pip packages

* fix linter
old-commit-hash: f1c77f97b9
2021-11-17 16:23:03 +01:00
Willem Melching
c11eeb6c80 RobustLogReader that can recover corrupted bz2 files (#22835)
* LogReader with bzip2 recovery

* only rlogs

* add comment

* plotjuggler should also use robust logreader
old-commit-hash: 6d6f989b7f
2021-11-10 16:41:00 +01:00
Adeeb Shihadeh
605c84634a tools/lib/route: expose raw file paths
old-commit-hash: ee33f8a8cd
2021-11-04 20:32:02 -07:00
Willem Melching
2dbecc8a15 tools/lib/auth.py: support github/apple login (#22766)
* tools/lib/auth.py: support github/apple login

* print some info about logged in user

* add docstring
old-commit-hash: 252f3c1c87
2021-11-01 15:00:00 +01:00
Willem Melching
67f41f32f1 Sphinx docs generation (#22697)
* add sphinx

* switch theme

* Experiment: sphinx docs generation updated (#22708)

* moved build to root gitignore, added .gitkeep

* Improved makefile doc build process

- Removed auto-generated docs from source control
- Moved apidoc.sh into Makefile
- Removed make.bat (can add back if Windows support desired)
- Added sphinx viewcode and markdown extensions
- Added feature to source /docs in build, so any .rst file in /docs will
  override the respective file during the build process
- Added feature to copy all markdown/rst files from source into /build/
  during build process so they can be easily referenced while writing
  docs (see examples in index.md)
- Wrote basic starter index.md file

TODO: Add new dependencies to Pipfile [dev-packages]

* Revert accidental modification to Pipfile

* fix command substitution

* exclude xx

* improve docs

* dont include all docs in release build

* Add dockerfile

* update title

* include normal readme

* build container in CI

* use buildkit

* add login

Co-authored-by: Chad Bailey <chadbailey904@gmail.com>
old-commit-hash: b816b5b644
2021-10-28 15:14:37 +02:00
Adeeb Shihadeh
2fcc8323dc docs for tools/lib
old-commit-hash: 431f6eac95
2021-10-12 20:58:04 -07:00
Adeeb Shihadeh
b5ebe63878 lib/route: support for e/d cams (#22529)
* lib/route: support for e/d cams

* comment
old-commit-hash: b6514a5cb3
2021-10-12 00:34:53 -07:00
Dean Lee
67677a4539 remove tools/lib/route_framereader.py (#22486)
old-commit-hash: c43d35a04d
2021-10-08 11:10:44 +02:00
Willem Melching
f4ac70557b tools: make auth optional for public routes (#22386)
* tools: make auth optional for public routes

* handle 403

* show warning from c++ replay
old-commit-hash: a31c6ce3f9
2021-09-30 20:13:46 +02:00
George Hotz
a6c4c2768a Fix test relying on SAFETY.md being in the root
old-commit-hash: 3a6c8d4678
2021-09-14 16:02:36 -07:00
Willem Melching
6f92abc815 Less open file permissions (#21922)
* Less open file permissions

* add test back

* remove params test for permissions

* remove umask

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: dddab597bc
2021-08-28 18:25:05 -07:00
arfy slowy
63b3d6024b fix: spelling typos (#21861)
* fix: typo spelling grammar

* revert

* Update selfdrive/locationd/calibrationd.py

* more revert

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d74def61f8
2021-08-05 12:05:49 -07:00
Greg Hogan
eca265120f FrameIterator that uses 1+ GB less RAM (#21687)
old-commit-hash: 8f1806602c
2021-07-22 12:08:56 -07:00
Adeeb Shihadeh
ee9ac35ca5 fix route_framereader when segments are missing
old-commit-hash: 594877e4a8
2021-06-10 23:12:56 -07:00
Adeeb Shihadeh
0c1c0ddcb1 unlogger: put YUV vipc send behind flag
old-commit-hash: 4b6ec3bca6
2021-06-03 21:53:16 -07:00
Josh Smith
2cae3a3799 Add type hints, small cleanups (#21080)
* improve tools.lib.kbhit and tools.sim.lib.keyboard_ctrl

* unpack more efficiently

* minor improvements

* agnos.py match spec better

* manual_ctrl test missing queue arg

* fix incorrect type annotation

* queues are generic

* varname reuse resulting in incorrect type inference

* bytes().hex() rather than bytes.hex(bytes())

* a bit of type hinting stuff
old-commit-hash: 77321dbac4
2021-06-03 12:21:04 +02:00
Willem Melching
a7ddd6ecc7 fix route framereader when using signed urls (#21029)
old-commit-hash: 476ee91845
2021-05-26 11:36:53 +02:00
Mitchell Goff
7eed124138 Updated RouteFrameReader to work on routes with missing segments (#20971)
old-commit-hash: 0dc27e3d6d
2021-05-23 14:15:48 -07:00
Willem Melching
3725aeca25 tools/lib/auth.py update oauth redirect url
old-commit-hash: 0252072729
2021-05-17 14:51:29 +02:00
Willem Melching
3ec4a7916b add qcamera support to Route (#20521)
old-commit-hash: 3755081145
2021-03-30 11:48:27 +02:00
Greg Hogan
44f03a9401 fix log values with non-utf8 characters and remove index_log (#20173)
* remove index_log

* handle non-unicode characters in values

* dump.py better handling of printing non-unicode characters

* only replace errors in main
old-commit-hash: 78808961c5
2021-03-01 13:03:44 -08:00
iejMac
b97a2a3dd3 plotjuggler falling back to qlog (#20105)
* fallback to qlogs

* minor fixes

* removed extra line

* help on no args

* Update tools/plotjuggler/juggle.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d4feecd803
2021-02-17 22:59:31 -08:00
Adeeb Shihadeh
c64a8356fe move to github container registry + CI speedup (#19621)
old-commit-hash: 59d7c5e64a
2020-12-31 14:57:01 -08:00
Adeeb Shihadeh
b950cab874 HW abstraction layer (#19530)
* start hw refactor

* move that

* pins

* put that back
old-commit-hash: 8674b023ba
2020-12-16 21:30:23 -08:00
Adeeb Shihadeh
df208e441a bump c++ version for index_log
old-commit-hash: 9a605d2617
2020-10-23 22:12:30 -07:00
Willem Melching
1a2893345a add check for partial response code (#2282)
old-commit-hash: 529e9a23ea
2020-10-07 12:14:22 +02:00
Greg Hogan
3ea69f614e eliminate round-trip reading entire file (#2275)
* eliminate round-trip reading entire file

* only if reading entire file

* print headers in errors

* fix end
old-commit-hash: 2293a6a36c
2020-10-05 16:59:19 -07:00
Willem Melching
d849706bc9 remove parallel url downloader
old-commit-hash: 927ef086f7
2020-09-21 10:32:08 +02:00
grekiki
398796a010 FileReader: cache files locally (#2170)
* framework for updates

* File caching

* update .gitignore

* update .gitignore

* Store in home tmp directory

* fix bugs in old code, add tests

* add test

* revert bug

* Fixed some bugs, more testing

* doing some style fixes

* Update caching with lenghts

* doing some style fixes

* doing some style fixes

* doing some style fixes

* subrepositories

* try to fix the tests

* change submodules back

* doing some style fixes

* if we don't cache, we don't cache length either

* fix curls

* make sure tests run in ci

* Cleanup tests

* Use int for cache flag

* fix linter

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 05b64257df
2020-09-17 13:03:34 +02:00
Greg Hogan
d200f048b7 fix pylint errors
old-commit-hash: 1e6c7c94e4
2020-09-03 18:46:13 -07:00
Willem Melching
16fe1bb2ad Hardware abstraction class (#2080)
* hardware abstraction class

* De Morgan

* Rename pc hardware class

* Fix sound card in controlsd

* Pc get sim info

* fix hardware in test

* two more

* No more random imei on android

* no randomness on android

* Need to return something that looks like imei for registration to work

* Return proper network strength

* Unused import

* Bug fixes + gpsd is only android
old-commit-hash: c7152d5419
2020-08-26 14:57:17 +02:00
Willem Melching
9c291968f6 Speedup URLFile (#1888)
* add parallel download support to URLFile

* make python 3.8 happy

* Fix chunk size

* Automatic number of threads

* No daemon threads in unlogger

* Cache length

* dont touch old filereader

* Remove debug info

* remove debug script

* Ignore type
old-commit-hash: c70700758d
2020-07-20 17:10:08 +02:00
Greg Hogan
b518934573 ffmpeg hwaccel is better
old-commit-hash: 32c4367ffb
2020-06-12 16:11:40 -07:00
Adeeb Shihadeh
f69483e094 quick LGTM fixes
old-commit-hash: 3463f1165e
2020-06-05 11:29:09 -07:00
Willem Melching
ce6dd8d4e9 Model + camerad test (#1622)
* kind of works

* move that

* hack to get camerad to reliably terminate

* not sure why SIGTERM wasn't working before

* compare bytes

* clean up some hacks

* gitignore

* fix that

* WIP

* no reboot

* comparison works

* pretty print

* fix build

* run in jenkins

* python path

* space

* raise timeout

* new eon

* skip the copy

* spinner

* spin less

* update model ref commit

* reenable that

* clean up

* fix jenkinsfile

* parallel

* wrap it in a stage

* fix linter

* better progress

* lower timeout

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 63c8e8439b
2020-06-04 18:57:25 -07:00
Adeeb
9027018368 Update Flake8 config (#1624)
* update flake8 checks

* add E502

* no whitespace warnings

* fix violations

* no W391

Co-authored-by: Jason Young <jason@comma.ai>
old-commit-hash: 367155168a
2020-06-03 16:13:34 -07:00
Adeeb
d035394ce7 Add pre-commit hooks (#1629)
old-commit-hash: ab83e48ec4
2020-06-03 12:54:49 -07:00
Willem Melching
a56e2b01f8 Make pylint more strict (#1626)
* make pylint more strict

* cleanup in progress

* done cleaning up

* no opendbc
old-commit-hash: 843a64c72f
2020-06-02 16:29:32 -07:00
Adeeb Shihadeh
9823f1164c enable flake8 E303: too many blank lines
old-commit-hash: f3dcf861c7
2020-05-31 17:21:11 -07:00
Adeeb Shihadeh
68a3061c9d enable E261 in flake8: two spaces before inline comment
old-commit-hash: 27754a277c
2020-05-31 14:07:29 -07:00