Greg Hogan
e59fe0014a
URLFile: add typing and internalize pool manager ( #31466 )
...
* URLFile: add typing and internalize pool manager
* cleanup
2024-02-14 14:09:01 -08:00
Greg Hogan
ec9f3dcef3
simplify URLFile ( #31365 )
...
* simplify URLFile
* more space
2024-02-08 10:24:45 -08:00
Greg Hogan
a8aa04e6bd
make URLFile safe after fork() ( #31309 )
...
* make URLFile safe after fork()
* cache the pool manager in each instance
* type hints
2024-02-05 18:07:48 -08:00
Justin Newberry
0234cadec3
LogReader: remove redirect message for internal data ( #31088 )
...
fix logging
2024-01-23 12:55:50 -08:00
Justin Newberry
fba521ecc6
url_file: fix non-200 files being cached ( #30886 )
...
* fix + test
* fix unclosed
* easier to read
Co-authored-by: Shane Smiskol <shane@smiskol.com >
* fix that
---------
Co-authored-by: Shane Smiskol <shane@smiskol.com >
2024-01-02 12:42:08 -08:00
Adeeb Shihadeh
51a93cf9aa
fix unused
2023-12-17 16:32:56 -08:00
royjr
70624ffc81
dependency: remove pycurl package ( #30771 )
...
* Update qcomgpsd.py
* Update url_file.py
* remove pycurl
* requests -> urllib3
* unused
* redundant
* fix import
* Revert "requests -> urllib3"
This reverts commit 7ca39e618917986f50356519d2795a432e3a5280.
* headless
* fix trail
* use requests.exceptions.RequestException
* use fp.tell
* fix indents
* reorder imports
* change timeout
* fix debug timing
* remove exception
* add timeout
* missing headers
* move to constructor
* move import
* unused import
* fix debug
* try
* no retries
2023-12-17 16:25:25 -08:00
Greg Hogan
db35dcd0b5
replace common.file_helpers.mkdirs_exists_ok with python os.makedirs funtion ( #30618 )
...
replace common.file_helpers.mkdirs_exists_ok with python os.makedirs function
2023-12-06 09:55:29 -08:00
Kacper Rączy
c27e977475
URLFile: exception type for failed requests ( #30330 )
...
URLFileException for URLFile request errors
2023-10-25 16:20:12 -07:00
Greg Hogan
86b90a8ba2
filereader cleanup ( #30191 )
...
* filereader cleanup
* make name return url
* remove unused imports
2023-10-05 19:45:46 -07:00
Justin Newberry
fa51bbc236
URLFile: don't cache non-existent file's lengths ( #30071 )
...
* urlfile empty
* simplify with mock
* better test name
* PR cleanup
* cleanup the length file
2023-10-04 12:57:56 -07:00
Justin Newberry
73eda51a11
System: use paths for download folders too ( #29818 )
...
* use paths for download folders too
* trailing slash
* reset that
2023-09-07 12:49:03 -07:00
Justin Newberry
1ee6ed4b16
Linter: remove pylint ( #29611 )
...
remove pylint
2023-08-24 16:30:54 -07:00
Adeeb Shihadeh
a9626f95b6
add openpilot prefix to imports ( #29498 )
...
* add openpilot prefix to imports
* more
* more
* fix docs
* fix linter
* bump submodules
* fix patched tests
* update dynamic imports
* debug
* Revert "debug"
This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.
* fix pm test
2023-08-20 20:49:55 -07:00
Shane Smiskol
6163dd5ca0
URLFile: raise exception if remote URL doesn't exist when using cache ( #24432 )
...
* URLFile returns empty bytes if using cache and remote file doesn't exist
* better exception
* assert on cached files
2022-05-09 13:31:55 -07:00
Ewout ter Hoeven
332f568a82
Pyupgrade 3.6: Update syntax with Python 3.6+ features ( #23305 )
...
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default
Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com >
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com >
Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com >
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com >
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven
55390d273f
Convert format strings strings to f-strings ( #23241 )
...
* Convert all text strings to f-strings
Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.
Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.
F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale
* revert pyextra changes
* revert ublox.py
Co-authored-by: Willem Melching <willem.melching@gmail.com >
2021-12-16 14:58:17 +01:00
Willem Melching
dddab597bc
Less open file permissions ( #21922 )
...
* Less open file permissions
* add test back
* remove params test for permissions
* remove umask
* bump cereal
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
2021-08-28 18:25:05 -07:00
arfy slowy
d74def61f8
fix: spelling typos ( #21861 )
...
* fix: typo spelling grammar
* revert
* Update selfdrive/locationd/calibrationd.py
* more revert
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
2021-08-05 12:05:49 -07:00
Greg Hogan
8f1806602c
FrameIterator that uses 1+ GB less RAM ( #21687 )
2021-07-22 12:08:56 -07:00
Adeeb Shihadeh
59d7c5e64a
move to github container registry + CI speedup ( #19621 )
2020-12-31 14:57:01 -08:00
Willem Melching
529e9a23ea
add check for partial response code ( #2282 )
2020-10-07 12:14:22 +02:00
Greg Hogan
2293a6a36c
eliminate round-trip reading entire file ( #2275 )
...
* eliminate round-trip reading entire file
* only if reading entire file
* print headers in errors
* fix end
2020-10-05 16:59:19 -07:00
grekiki
05b64257df
FileReader: cache files locally ( #2170 )
...
* framework for updates
* File caching
* update .gitignore
* update .gitignore
* Store in home tmp directory
* fix bugs in old code, add tests
* add test
* revert bug
* Fixed some bugs, more testing
* doing some style fixes
* Update caching with lenghts
* doing some style fixes
* doing some style fixes
* doing some style fixes
* subrepositories
* try to fix the tests
* change submodules back
* doing some style fixes
* if we don't cache, we don't cache length either
* fix curls
* make sure tests run in ci
* Cleanup tests
* Use int for cache flag
* fix linter
Co-authored-by: Willem Melching <willem.melching@gmail.com >
2020-09-17 13:03:34 +02:00
Willem Melching
c70700758d
Speedup URLFile ( #1888 )
...
* add parallel download support to URLFile
* make python 3.8 happy
* Fix chunk size
* Automatic number of threads
* No daemon threads in unlogger
* Cache length
* dont touch old filereader
* Remove debug info
* remove debug script
* Ignore type
2020-07-20 17:10:08 +02:00
