Commit Graph

214 Commits

Author SHA1 Message Date
Shane Smiskol
3e41b1b287 LogReader: show total segment count in missing rlog prompt 2024-07-01 22:43:48 -07:00
Maxime Desroches
6f10b8a209 fix pytest tests (#32867)
* fix tests

* no dot
2024-06-28 23:58:19 +00:00
StefanLekanic-SYRMIA
504dd4543d LogReader: print number of missing rlogs (#32830)
* count rlogs

* minimization changes

* cache to avoid duplicate checks

* no real point in checking qlogs to just show the warning

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2024-06-26 22:15:13 -07:00
Shane Smiskol
b45caf4033 LogReader: try zst on internal source (#32751)
* internal source list files like azure api

* messy but works

* no limit

* simpler

* clean up

* clean up

* clean up

* that's obvious

* better

* we need to unfortunately return a url, so best to take a naive approach for now

* todo

* fix

* clean up
2024-06-14 01:12:19 -07:00
Shane Smiskol
68e22faa29 LogReader: revert broken check
it returns files OR None
2024-06-14 00:30:14 -07:00
Shane Smiskol
d5ce4b3085 SegmentRange: remove timestamp (#32750)
* add deprecation warning

* remove timestamp

* Update logreader.py
2024-06-13 23:19:32 -07:00
Shane Smiskol
a0bbc005b6 LogReader: support reading zst logs from Azure (#32745)
* staging api

* other stuff

* ugh this should really not be duplicated, we already check the max segnum in Route class

* better error message!!

* more better

* clean up

* clean up

* breaks again
2024-06-13 22:43:24 -07:00
Shane Smiskol
3ede1e2a7c LogReader: improve error messages (#32747)
* better error messages

* clean up
2024-06-13 22:37:00 -07:00
Shane Smiskol
04a93dd849 add zstd Python library (#32731)
* add zstd

* fix

* uploader

* logreader: zst support

* caps

* only logreader
2024-06-12 21:14:23 -07:00
Dean Lee
c64bca97ba logreader.py: optimize event loading and sorting in _LogFileReader (#32729)
* improve logfilereader

* less changes
2024-06-12 14:20:06 -07:00
Mauricio Alvarez Leon
148eaf8fa6 Update to Python 3.12 (#32548)
* 3.12

* pprofile is broken

* use modified metadrivepy3-12

* 0.3.0 metadrive

* add metadrive/commaai git dependency

* metadrive git set

* pin sounddevice 0.4.6

* datetime.utcnow() deprecation

* poetry lock

* make datetime not aware

* poetry lock

* pin pytools

* google_crc32c wheel

* unpin sounddevice

* clean metadrive

* use python crc

* mypy

* 3.12.4

* allow python3.11

* test

* no pip

* poetry

* better

* better

* merge

* remove

* try

* test

* try  this

* snok

* python

* simpler

* setuptools

* lower

* try

* try

* work?

* ubuntu deps

* ubuntu

* try

* remove

* move

* remove this

* names

* Update .github/workflows/tools_tests.yaml

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* python<4

* <3.13

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2024-06-11 20:45:15 +00:00
Adeeb Shihadeh
c59887c810 test_*.py files are no longer executable (#32610)
* test_*.py files are no longer executable

* cleanup empty lines

* update precommit

* fixme

* enable

* build
2024-06-03 15:48:56 -07:00
Adeeb Shihadeh
b2e4c64cf8 ruff: enable UP 2024-05-20 17:43:54 -07:00
cl0cks4fe
dd9d5d4528 Unittest to pytest (#32366)
* add pytest-asyncio

* switch common

* switch selfdrive

* switch system

* switch tools

* small fixes

* fix setUp and valgrind pytest

* switch to setup

* fix random

* switch mock

* switch test_lateral_limits

* revert test_ui

* fix poetry.lock

* add unittest to banned-api

* add inline ignores to remaining unittest imports

* revert test_models

* revert check_can_parser_performance

* one more skip

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2024-05-17 11:01:44 -07:00
Greg Hogan
2a46d71fc8 logreader: skip internal source if connection refused (#32418)
* logreader: skip internal source if connection refused

* fix indentation

* fix spacing

* explicit ipv4 and tcp
2024-05-13 16:02:28 -07:00
Justin Newberry
8364cd2f2c jenkins: publish casync releases from device (#32142)
* publish in ci

* overwrite

* publish in ci

* fix

* test it

* Revert "test it"

This reverts commit b3de51dc693df79b980d7dffc18bcc3c88d83375.

* use right token

* cleanup after uploading

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-04-09 11:38:18 -07:00
Justin Newberry
fc9f50c90d fix logreader after segment_numbers api removal (#32073)
fix
2024-04-01 10:01:50 -07:00
Cameron Clough
27d2a6066d tools: remove unused timestamp_to_datetime (#31950) 2024-03-21 10:58:35 -07:00
Justin Newberry
b0eb3ba4f4 cars: platform enums -> prepend brand name (#31927)
* with brand name

* migrate

* Fix

* fixes

* more

* passes

* fix

* fix the doc

* collects

* these too

* more stuff

* body exception :/

* more

* hardcode i guess

* update ref

* toyota

* more toyota

* and here

* final!

* fix notebooks and ccs

* move this here
2024-03-19 17:29:50 -07:00
thenhnn
489528dcae PlatformConfig: automatically get platform_str from the enum name (#31868)
* get platform_str from the enum name

* fix tests

* add migration table

* remove impossible todo

* Add link to PR in MIGRATION table

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Remove useless brand name comments and rename RAM_1500 to RAM_1500_5TH_GEN

* rename RAM_HD to RAM_HD_5TH_GEN

* rename references to RAM_HD and RAM_1500

* change "mock" to "MOCK" and rename torque data of Nissan Leaf 2018 IC

* remove MOCK from fingerprints.py

* change hard-coded car model in test_can_fingerprint.py/test_timing

* migration

* update ref

* space

* prius

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: justin newberry <justin@comma.ai>
Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
2024-03-19 11:16:02 -07:00
Justin Newberry
a819292073 logreader: fix auto source + interactive modes (#31847)
* interactive modes

* these exceptions don't matter
2024-03-12 20:04:17 -07:00
Shane Smiskol
e964c5944d LogReader: fix sort by time and union types (#31565)
* fix :(

* test_sort_by_time

* this isn't required

* not slow, and just compare sorted

* messy

* works

* clean up

* clean up

* not here

* clean up

* clean up

* clean up

* makes network call

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
2024-03-08 02:49:24 -08:00
Adeeb Shihadeh
7cfc571f56 timeless routes (#31119)
* timeless route

* update sort

* update test

* fix param name
2024-03-05 20:43:33 -08:00
Justin Newberry
f4a7e8eae6 disable commaCarSegments tests (#31632)
disable this test too
2024-02-28 16:47:08 -08:00
Justin Newberry
9d1cafd0fc move with_http_server to selfdrive/test/helpers (#31597)
* move

* fix
2024-02-26 12:53:17 -08:00
Cameron Clough
995250ae49 use pyupgrade to update to new typing syntax (#31580)
* add pyupgrade hook

* run pyupgrade (pre-commit run -a)

* ruff --fix

* Revert "add pyupgrade hook"

This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.

* revert changes to third_party/

* manual type fixes

* explicit Optional wrapping capnp objects
2024-02-24 16:41:23 -08:00
Justin Newberry
556f973896 segment range docs update (#31560)
* the format

* cleaner
2024-02-23 11:50:33 -08:00
Greg Hogan
43f64ec500 URLFile: default value test (#31544)
* URLFile: default value test

* cleanup

* fix env

* improvements

* fix GET response

* only delete cache dir if it exists

* env pop

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2024-02-22 13:17:02 -08:00
Shane Smiskol
c5f1f4c676 test_fw_query_on_routes: get first qlog (#31496)
* fast

* this isn't internal

* see
2024-02-16 17:55:15 -08:00
Justin Newberry
16d1339553 test_logreader: test internal scenarios (#31484)
* test source scenario

* test source scenario

* fix
2024-02-15 12:10:56 -08:00
Justin Newberry
8aee0d2af2 test_logreader: test interactive mode + fix typo in AUTO_INTERACTIVE (#31481)
* fix spelling

* test interactive

* remove that

* test taht

* move that
2024-02-15 11:24:27 -08:00
Greg Hogan
e59fe0014a URLFile: add typing and internalize pool manager (#31466)
* URLFile: add typing and internalize pool manager

* cleanup
2024-02-14 14:09:01 -08:00
Justin Newberry
33cf6bda9e LogReader: add typing hints (#31464)
logreader typing
2024-02-14 10:34:17 -08:00
Justin Newberry
1436f576df LogReader: retain old behavior for direct parsing of files (#31419)
* maintain exception

* test that head is not called

* annoying mock

* test with cache
2024-02-14 10:05:25 -08:00
Shane Smiskol
796671fe1a SegmentRange: more explicit slice matching (#31451)
* more explicit

* fix it

* use non capturing group

* only needed for regex101

* make test_logreader.py executable

* fix

* stash

* passes test

* none

* type anotate

* test

* fix

* nice for syntax highlighting
2024-02-14 04:36:58 -08:00
Shane Smiskol
8fe9bc7a69 SegmentRange: simplify slice (#31455)
* simplify slicing

* rm
2024-02-14 04:17:03 -08:00
Shane Smiskol
c4f7991bb6 SegmentRange: test API call (#31456)
* test

* better

* better
2024-02-14 03:29:08 -08:00
Shane Smiskol
8276371009 SegmentRange: type annotations (#31453)
* type annotate SegmentRange

* proper formatting

* oops

* numpy?

format test too

* draft

* fixed

* clean up

* rm

* more

* clean up

* clean up

* rm

* not here

* revert
2024-02-14 02:05:55 -08:00
Shane Smiskol
0846175f44 tools/lib: format code (#31454)
easier to read. pylint used to catch all this stuff, but it's mostly missing in ruff :'(
2024-02-14 01:56:17 -08:00
Shane Smiskol
884bd5c7f6 SegmentRange: define __repr__ 2024-02-13 22:58:44 -08:00
Justin Newberry
d6762c3035 live_logreader to its own file (#31436)
own file
2024-02-13 13:29:52 -08:00
Adeeb Shihadeh
e9071f1198 dongle id is only hex (#31426)
* dongle id is only hex

* so is the count
2024-02-13 13:01:25 -08:00
Justin Newberry
0a92c5bf96 LogReader: live_logreader helpers (#31416)
live helper
2024-02-12 12:58:25 -08:00
Greg Hogan
667693b8c2 logreader: skip internal source if not available (#31400)
* logreader: skip internal source if not available

* raise exception

* but only when appropriate
2024-02-10 00:20:43 -08:00
Justin Newberry
664a3c86ba test_caching: use with_http_server (#31393)
use http server
2024-02-09 20:06:11 -08:00
Justin Newberry
f873224235 LogReader: test auto mode fallback (#31390)
* test auto mode

* better

* slow

* better
2024-02-09 15:22:43 -08:00
Greg Hogan
ec9f3dcef3 simplify URLFile (#31365)
* simplify URLFile

* more space
2024-02-08 10:24:45 -08:00
Justin Newberry
b17f24d68e test_logreader: test run_across_segments (#31305)
* more logreader tests

* not in ci for now

* enable cache
2024-02-05 18:35:12 -08:00
Greg Hogan
a8aa04e6bd make URLFile safe after fork() (#31309)
* make URLFile safe after fork()

* cache the pool manager in each instance

* type hints
2024-02-05 18:07:48 -08:00
Justin Newberry
6010a39bf8 LogReader: skip file_exists check for comma_api (#31251)
* valid file

* missed this one
2024-01-31 16:43:58 -08:00