Commit Graph

16 Commits

Author SHA1 Message Date
Justin Newberry 0c143101c2 conftest: cleanup environment cleaner (#31486)
* clean env

* no self
old-commit-hash: 0cb206cb95
2024-02-15 13:38:42 -08:00
Justin Newberry 1ade9aa8b2 pytest: fix tici setup fixture (#31379)
cleanup

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 88d0231095
2024-02-08 13:32:40 -08:00
Justin Newberry 378c5cb50b Pytest: reenable gc in pytest fixture + re-combine jenkins car tests (#31351)
* Revert "jenkins: spilt car tests in two steps (#31346)"

This reverts commit 03bf426f569151598070a073f658a323eb03b9db.

* enable gc

* codespell

* useless comment
old-commit-hash: 6933d0560a
2024-02-07 08:35:32 -08:00
Justin Newberry fd799bc05a Pytest: consistent hardware state for tici tests (#31279)
* consistent hardware

* consistent hardware

* moved

* this too

* ruff

* s

* duplicated

---------

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 0277fc5548
2024-02-02 14:18:01 -08:00
Justin Newberry 4080bec11d pytest: add marker for sharing the download cache (#31082)
* fix cache

* with a marker
old-commit-hash: 3846130d8e
2024-01-22 16:30:59 -08:00
Adeeb Shihadeh 5eeedf6b95 pytest: cleanup manager procs on teardown (#31101)
old-commit-hash: b1f457bdc8
2024-01-21 16:39:52 -08:00
Shane Smiskol f65ccca8e0 test_models: fuzz test panda and CarState (#30443)
* pre-hypothesis

* some hypothesis junk

* this kinda works but is really slow due to counter check

* choose addrs from fingerprint

* stash

* honda nidec brake pressed mismatches fixed

* bump panda

* stash

* tesla: use DI_torque2 (panda msg)

* run

* run

* ah this honda mismatch too

* no more multi can msgs

* clean up, remove old file

* add todo

* prob can remove urandom

* stash, huge examples

* fix pq standstill mismatch

* yuge

* yup there's a leak somewhere

* try to find leak

* skip dashcam (pq and tesla)

* PR comments

* bump

* draft stash

* fix alt brake hondas

* bump

* bump

* bump

* some clean up

* minor clean up

* more clean up

* stash

* fix honda bug

* more

* 100 examples

* revert tesla

* no memory leak any more?

* bring back tests with skips

* parameterize max_examples

* skip interceptor

* is jenkins on my branch?

* ooh that's fast

* 50 is not bad for GH CI

* 300 might be better with rest of test_models

* no more detection

* bump

* need CS_prev to catch bugs where openpilot changes and panda doesn't (eg. not setting interceptor safety mode)

* need to simplify all this

* need a warm up first, since some signals are 1 by default (toyota's gas_released!=1)

* changes

* set honda safety param

* set toyota safety param

* bump panda

* clean up honda

* rm interceptor

* thought interleaving addrs might help, but we can fine tune later

* Revert "thought interleaving addrs might help, but we can fine tune later"

This reverts commit 153301384b.

* get size from dict

* what

* add nocapture marker

* clean up

* try to raise logging level

* need to run last as pytest_runtest_call, since it starts capturing

* get capman conditionally

* mark

* type fingerprint

* should use gen_empty_fingerprint

* no longer needed

* draft

* no longer need gc

* clean that up

* test everything!

* more clean up

* more

* no point

* fix that

* fix errors

* bump

* nice even 300 examples for 300 segs

* final bump :fingers_crossed:

* better import order

* remove debugging prints

* warm up kinda works

* Revert "warm up kinda works"

This reverts commit 7fc77b07d5.

* random seed

* revert

* strat

strat

* add expl comment

* cmt

* check controls allowed

* Revert "check controls allowed"

This reverts commit e82a0e5396.

* not unittests

* run tests!

* run tests 2!

* run tests 3!

* seed unused

* revert

* add shrink phase, and remove health check suppression

* hello

* oncemore

* Update selfdrive/car/tests/test_models.py
old-commit-hash: 5052b55c44
2023-12-19 03:18:54 -06:00
Shane Smiskol e46c20ac61 test_models: randomize internal segment list (#30653)
* randomize internal seg list segments

* random

* pytest-randomly sets random.seed to a consistent value for all the workers/processes

* noeol

* update

* Revert "update"

This reverts commit aff9a69c4e.

* lock

* don't randomize by default

* remove random-order

* strict

* random

* one fix

* test

* does nothing

* rm tests

* Revert "rm tests"

This reverts commit b548e3fcd4.

* (can't repro locally) just athena should be fine

* bs1

* bs2

* bs3

* bs4

* bs5

* wrong way

* no controls

* no car

* no board

* controls?

* crazy -common

* Revert "crazy -common"

This reverts commit 02365d712b.

* test athena

* test athena 2

* test athena 3

* test athena 4

* test athena 5

* test athena 6

* test athena 7

* test athena 8

* test athena 9

* ??

* in one commit

* common?

* car and board

* -controls -board

* random-order

* no board

* revert

* car/tests

* least likely

* try

* try 2

* draft

* draft

* so much better

* cmt

* use randomly

* not needed here

* directly modify option works

* bb

* test time

* Revert "test time"

This reverts commit 2c5caabe2b.

* tmut

* i concur

* revert
old-commit-hash: ac83318ac4
2023-12-09 14:28:32 -08:00
Shane Smiskol f5d1c4f371 ruff: check newline (#30668)
* check newline

* nothing catches this
old-commit-hash: 97da129e11
2023-12-09 13:24:18 -08:00
Justin Newberry 62d0647ab3 pytest: group test_models by route instead of car name (#30625)
* group by route

* dont need those
old-commit-hash: f16df8e4b7
2023-12-06 16:00:59 -08:00
Justin Newberry 337b156518 Pytest: load by group by default (#30466)
* wip

* use a class marker

* this one should be grouped too
old-commit-hash: 31c682d407
2023-11-14 17:33:47 -08:00
Justin Newberry b4992fefed Pytest: consistent codecov (#30408)
* seed only

* random seed

* ignore version.py

* increase max examples

* increase default max examples
old-commit-hash: d02dd50749
2023-11-08 13:43:32 -08:00
Justin Newberry 3027a428cb Pytest: tici mark for skipping on-device tests (#30412)
* mark tici

* not those ones

* missed that one

* add those

* add pypoetry

* oops
old-commit-hash: aec7cea30d
2023-11-07 17:35:44 -08:00
Justin Newberry fe14ca751f Pytest: fix local params overriden (#30312)
* fix local params being overriden

* just reset prefix after completion
old-commit-hash: 412f4cbc1a
2023-10-23 17:41:19 -07:00
Justin Newberry 26d822b661 cleanup environment variables between tests (#30167)
old-commit-hash: 786f13b0e3
2023-10-04 11:51:44 -07:00
Adeeb Shihadeh 53e0f5eb49 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: 7b6afbc162
2023-09-05 18:52:40 -07:00