Files
sunnypilot/selfdrive/test/openpilotci.py
Shane Smiskol d2d3b7b823 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-05-11 15:03:05 -07:00

35 lines
1.3 KiB
Python
Executable File

#!/usr/bin/env python3
import os
import sys
import subprocess
BASE_URL = "https://commadataci.blob.core.windows.net/openpilotci/"
TOKEN_PATH = "/data/azure_token"
def get_url(route_name, segment_num, log_type="rlog"):
ext = "hevc" if log_type.endswith('camera') else "bz2"
return BASE_URL + f"{route_name.replace('|', '/')}/{segment_num}/{log_type}.{ext}"
def upload_file(path, name):
from azure.storage.blob import BlockBlobService # pylint: disable=import-error
sas_token = os.environ.get("AZURE_TOKEN", None)
if os.path.isfile(TOKEN_PATH):
sas_token = open(TOKEN_PATH).read().strip()
if sas_token is None:
sas_token = subprocess.check_output("az storage container generate-sas --account-name commadataci --name openpilotci --https-only --permissions lrw \
--expiry $(date -u '+%Y-%m-%dT%H:%M:%SZ' -d '+1 hour') --auth-mode login --as-user --output tsv", shell=True).decode().strip("\n")
service = BlockBlobService(account_name="commadataci", sas_token=sas_token)
service.create_blob_from_path("openpilotci", name, path)
return "https://commadataci.blob.core.windows.net/openpilotci/" + name
if __name__ == "__main__":
for f in sys.argv[1:]:
name = os.path.basename(f)
url = upload_file(f, name)
print(url)