Commit Graph

88 Commits

Author SHA1 Message Date
Dylan Herman
d2eef1955c add mypy check for return-any (#24379)
* add mypy check for return-any

* remove unused import

* typing

* remove unnecessary variable typing

* cleanup

* cleanup

* bump submodules

* small fixes

* only a problem on mac

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-05-12 14:59:00 +02:00
grekiki
17e33978cd Mypy fixes for --check-untyped-defs (#24372)
more type fixes
2022-05-12 14:08:11 +02:00
Willem Melching
77a6f3d034 delay bz2 compression from logging to uploading (#24392)
* remove log_name

* log without compression

* fix tests

* remove extension for bootlog

* another test fix

* uploader compresses

* also compress in athena

* only compress qlog

* more generic check in do_upload

* fix bootlog compression

* lower loggerd cpu usage

* dont link against bz2

* set core affinity to little cluster

* handle old files
2022-05-03 14:09:17 +02:00
grekiki
b8b15d5cf1 mypy: enable --warn-unreachable (#24282)
* Optional solves mypy assuming None is always None

* oops

* spaces

* Update registration.py

* fix local error

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-04-21 18:27:45 +02:00
Adeeb Shihadeh
0fbb7a4673 athena: fix random test failure (#24278) 2022-04-20 13:20:50 -07:00
Willem Melching
e7f23ceca2 uploader: azure returns 401 for files already uploaded (#24141) 2022-04-05 20:09:05 -07:00
Willem Melching
714efe828a athena.upload_handler.upload_start log retry count (#24139) 2022-04-05 17:57:01 -07:00
Willem Melching
da5a0c41a0 C3: detect metered Android hotspot (#23734)
* C3: detect metered networks

* show in ui

* fix text layout

* bump cereal

* revert ui changes

* set networkMetered

* add athena method

* add metered logging to uploader

* use in athena uploader

* remove param

* use networkmanager properties to set cell to unmetered

* fix indentation

* no need to check

* bump cereal

* review

* bump cereal
2022-03-09 11:36:52 +01:00
Robbe Derks
a6214ff3b1 Fix unmetered upload over cell (fix #23811) (#23820) 2022-02-22 13:01:53 +01:00
Robbe Derks
5f64f1c089 Param to allow upload over cellular (#23811)
* param to allow upload over cellular

* brackets to be consistent

* move outside loop

* fix tests

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-02-21 12:17:47 +01:00
Willem Melching
f03549c276 athena: expire items after 31 days (#23751)
* athena: expire items after 31 days

* add test
2022-02-11 13:52:44 +01:00
Willem Melching
7765bc2166 improve logging around uploading (#23750)
* improve logging around uploading

* use int

* use raw

* duplicate
2022-02-11 12:58:41 +01:00
Willem Melching
e9153fdb4b athena: specify network type for file uploads (#23687)
* athena: specify network type for file uploads

* add comment

* catch abort transfer

* fix tests

* put athena upload args in dict

* fix defaults

Co-authored-by: Joost Wooning <jwooning@gmail.com>
2022-02-09 17:28:49 +01:00
Willem Melching
0d099e07aa athena: add suport for download bandwith setting (#23727)
* athena: add suport for download bandwith setting

* clean ingress rules
2022-02-08 12:07:11 +01:00
Willem Melching
4c406cdbe5 Add setUploadLimit method to athena (#23693)
* Add setUploadLimit method to athena

* add comments

* move to hw abstraction layer

* move to hw

* better errors
2022-02-07 16:55:16 +01:00
Adeeb Shihadeh
52d4189e96 don't show registration error on pc 2022-01-29 18:10:41 -08:00
Adeeb Shihadeh
8b5b0ae341 sentry improvements (#23627)
* sentry: filter out unregistered devices and PC

* add daemon name
2022-01-26 11:44:10 -08:00
Ryan
1d4191956b athena: unsuccessful upload handler requests. (#23620)
* Retry unsuccessful upload handler requests.

* test both cases

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-01-26 15:23:59 +01:00
Robbe Derks
1b49ce6ec4 Statsd (#23413)
* device side of statsd

* need to start it

* enable in manager

* add sleep

* cleanup

* remove aggregates for now and standardize on industry terms

* manager needs main

* need to have a try/except

* atomic_write_on_fs_tmp does not work

* cleaner

* use dump

Co-authored-by: Willem Melching <willem.melching@gmail.com>

* one file at a time

* limit amount of files

* move to influx line protocol and cleanup

* needs to be a list

* fix timezone bug

* actually rate limit

* add to release

* normalized origin

* also log deviceType

* more stats

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-01-10 15:21:48 +01:00
Cameron Clough
1de0757c5e fix typo in athena/registration.py (#23434) 2022-01-06 16:11:32 -08:00
Joost Wooning
3ffebf4df5 athena: uploads, show which items fail on error (#23412)
* athena: uploads, show which items fail on error

* fix upload-id

* no more 404

* Update selfdrive/athena/athenad.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2022-01-05 18:18:11 +01:00
Joost Wooning
47bb62b875 athena: methods for adding and cancelling multiple upload requests (#23366)
* multiple upload cancel

* multiple uploads athena method

* cleanup

* cleanup

* more cleanup

* isnt used

* fix test

* actually fix test
2022-01-04 17:01:33 +01:00
Ewout ter Hoeven
332f568a82 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
2021-12-24 11:18:39 -08:00
Willem Melching
d09cc197d1 manage_athenad.py: add daemon argument to launcher 2021-12-15 17:03:12 +01:00
Devin Leamy
8cb83b29a6 athena: persist upload queue to disk (#22659)
* revert submodules

* lowercase

* addressed comments

* add test for cancelled uploads

* formatting

* catch all exceptions

* handle empty param

Co-authored-by: Willem Melching <willem.melching@gmail.com>
2021-12-14 13:29:17 +01:00
Shane Smiskol
b745a14ff7 version.py: make function names more clear (#23216)
* rename a few version functions

* and is_prebuilt

* and some formatting 😊
2021-12-14 12:15:13 +01:00
Joost Wooning
4a73445ee5 nav: athena destination with name and details (#23155) 2021-12-08 14:40:31 +01:00
Adeeb Shihadeh
9ceb1a15eb registration: remove key generation (#23111)
* registration: remove key generation

* add test
2021-12-03 22:58:34 -08:00
Willem Melching
f7c46c6949 Do not run code on version.py import (#23063)
* Do not run code on version.py import

* fix athena
2021-11-29 19:38:55 +01:00
Willem Melching
0e78502181 remove broken athena test 2021-11-11 17:32:26 +01:00
Willem Melching
cca07c0c73 navigation: use mapbox proxy server (#22660)
* use mapbox proxy

* remove MapboxToken param

* do not log jwt

* remove import

* move to staging url

* put has_prime in ui state

* logic was flipped

* Update selfdrive/ui/qt/maps/map.h
2021-10-26 18:59:17 +02:00
Willem Melching
bde568205e athena: listUploadQueue, check None first 2021-10-21 11:15:09 +02:00
Devin Leamy
12165c8865 athena: Do not show canceled upload items in listUploadQueue (#22627)
* do not show canceled upload items in listUploadQueue

* return item._asdict()

* athena: updated test for listUploadQueue
2021-10-20 16:16:05 +02:00
Adeeb Shihadeh
de7a9e0116 move athena tests to github actions (#22475)
* move athena tests to actions

* run

* fix timeout
2021-10-07 13:22:14 -07:00
Adeeb Shihadeh
f7bd264db8 need a jwt for those tests 2021-08-19 15:48:26 -07:00
Adeeb Shihadeh
7e3d002066 athena: hardcode jwt for tests 2021-08-19 14:19:14 -07:00
Greg Hogan
1d64c18e49 athena: change test tearDown to setUp (#21935) 2021-08-14 22:27:22 -07:00
Greg Hogan
a6debb999c athena: split data into multiple frames (#21918) 2021-08-13 08:38:36 -07:00
Greg Hogan
2dff84121e athena: add prefix param to listDataDirectory (#21906)
* athena: add prefix param to listDataDirectory

* only traverse directories that match prefix

* tests
2021-08-13 12:14:20 +02:00
Willem Melching
28823917ea athena: return current upload status in listUploadQueue (#21903)
* return current upload item in listUploadQueue

* update progress on read

* move to file helpers

* use dict instead of global
2021-08-13 11:51:08 +02:00
Willem Melching
267a0b343b api_get: keep same api (#21838) 2021-08-03 15:33:41 +02:00
Dean Lee
543e019f71 api: use API_HOST env variable everywhere (#21814)
* api base url to global constant

* update api/__init__.py
2021-08-03 13:49:49 +02:00
Willem Melching
d5b6746ac5 athenad: retry failed and timed out uploads (#21745)
* retry failed uploads

* test cleanup

* update comment

* also catch SSL error

* use defaults

* sleep in chunks
2021-07-29 11:13:59 +02:00
Adeeb Shihadeh
7b94cf0ca7 athena: detect no prime (#21718)
* detect no prime

* show in sidebar

* revert timeout

* clear ping param

* fix typo

Co-authored-by: Comma Device <device@comma.ai>
2021-07-26 13:50:58 -07:00
Greg Hogan
0964871239 athenad: fix log sort (#21703)
* athenad: fix log sort

* add test for logs to send

* use temp dir for logs

* fix changing SWAGLOG_DIR

* better way to patch SWAGLOG_DIR

* fix grammar
2021-07-24 20:11:29 -07:00
Joost Wooning
d4e0e8c6cf athena: primeActivated response (#21670) 2021-07-22 13:48:27 +02:00
Adeeb Shihadeh
987abb31dd show registration debug info after 60s (#21679)
* show registration debug info after 60s

* little nicer

* larch

* aarch

Co-authored-by: Comma Device <device@comma.ai>
2021-07-21 23:30:02 -07:00
Willem Melching
51ee3be91a athena: primeActivated endpoint (#21623) 2021-07-16 14:38:46 +02:00
Willem Melching
a7aa22253b Athena getNetworks method (#21597)
* add iwlist scan

* return using athena

* add lte

* add last one too

* unused

* add release files

* more compact

* typo

* remove debug code

* different file

* array

* rebase mistake
2021-07-14 16:16:45 +02:00
Adeeb Shihadeh
49e316a34b more athena logging (#21558) 2021-07-11 21:18:28 -07:00