1 Commits

Author	SHA1	Message	Date
DevTekVE	3c36374bed	ci: build upstream (tinnygrad) models from our pipeline (#498) ... * Add job to fetch and upload model artifacts Introduced a new `get_model` job to retrieve and upload model artifacts as part of the workflow. Updated the `build_model` job to download and clean up model files before building, ensuring an isolated and streamlined process. These changes improve modularity and artifact handling in the CI pipeline. * Refine model file handling in build workflow Restrict deletion and download commands to ".onnx" files to prevent unintended removal or overwriting of non-ONNX files. This ensures a more targeted and safer workflow for handling model artifacts. * Update model artifact handling in build workflow Changed artifact upload path to include only .onnx files and commented out the deletion of existing models. Also added a dependency for the build stage to ensure proper execution sequence. * Update build workflow to refine model artifact handling Removed unused code for deleting models and fixed the artifact path. Added a reference file logging custom name and upstream branch for better traceability. These changes streamline the workflow and improve clarity. * Set dynamic run-name for sunnypilot build workflow Added a customizable `run-name` field in the sunnypilot build model workflow. This allows more informative and dynamic naming based on input parameters such as `custom_name` or `upstream_branch`. Improves clarity and traceability in workflow runs. * Add customization options and metadata generation to build Introduce new inputs for file name prefix and 20Hz model specification in the workflow. Enable renaming of model files, generation of file hashes, and creation of a `metadata.json` file containing build details. This improves flexibility and enhances artifact information management. * remove EOF * No need for this hack anymore * Add support for compiling ONNX models to Tinygrad format * Add model directory environment variable and metadata generation script * Fix quoting in model generator script command * Refactor conditional flag for 20Hz input in build model script * meh * Add powersave management to build model workflow * Add environment variable setup to build model workflow * make it path call it a day * kill me * kill me x2 * mmm * mmm * Generate metadata.pkl * Generate metadata.pkl * ignore failure on getting model metadata * Cleanup the existing onnxs * Fix rm command syntax in build model script * Cleaning a little * Add optional file name parameter to generate_metadata function * update * stg * linter * more formatting * Update sunnypilot model build workflow: improve custom naming, handle 20Hz default, and add commit date as output. * Update commit date format in workflow script Changed the commit date format in sunnypilot-build-model.yaml to a more human-readable format ('Month DD, YYYY'). This improves readability and consistency in the workflow outputs. * Fix output naming in sunnypilot-build-model workflow Corrected the output name from 'model_date' to 'commit_date' to match its usage. This ensures consistency and prevents potential errors in the workflow. Add debug echo for model date in GitHub workflow This change adds a debug echo statement to display the model date in the sunnypilot-build-model workflow. It helps verify the correct commit date is being set during the workflow process. * Update BUILD_DIR path in sunnypilot workflow Replaced hardcoded BUILD_DIR with a dynamic path using `github.workspace`. This enhances flexibility and ensures compatibility with different environments or runners. * Revert "Update BUILD_DIR path in sunnypilot workflow" This reverts commit 376971b616. * Refactor model metadata generation to use short names Replaced `file_name` with generated `short_name` for metadata and file naming. Introduced `create_short_name` function to simplify short name creation from custom names. Updated scripts and workflow to reflect the new parameter and improve metadata handling. * `Use timezone-aware UTC timestamp for build_time` Updated the code to use `datetime.now(timezone.utc)` instead of `datetime.utcnow()`. This ensures that the generated timestamps are timezone-aware, improving consistency and correctness in the output. * Refactor short name generation logic in model metadata Enhanced the `create_short_name` function to better handle name conversion by incorporating new rules for handling parentheses, alphanumeric words, and version pairs. Updated related variable names in metadata functions for improved clarity and consistency. * Fix syntax error in assigning 'models' key in metadata Corrected a typo where 'models' was missing quotes, causing it to be interpreted as a variable instead of a string key. This ensures proper metadata dictionary structure. Update model metadata structure and JSON generation Revised metadata to include detailed download URIs and a type field for artifacts. Enhanced the JSON generator with new fields like environment, runner, and overrides for better configurability. These changes aim to improve metadata clarity and compatibility with downstream processes. * Refactor variable name from "escaped_name" to "short_name" Renamed "escaped_name" to "short_name" across function parameters and internal logic for clarity and consistency. This improves readability and better aligns with its use case in the metadata generation process. No functional changes were introduced. * Refactor variable names and improve type annotations. Updated parameter names to align with Python naming conventions, enhancing readability and consistency. Added type annotations for function parameters to improve code clarity and maintainability. * Refactor type annotations and clarify comments. Updated type annotations to use "Str" for consistency with custom typing conventions. Improved clarity of a comment describing the normal case logic in the shortening function. * Fix dict formatting in model_generator.py Moved the "overrides" key for better alignment and consistency in the dictionary format. This improves readability without changing functionality. Refactor model short name generation logic Simplify and clarify the `create_short_name` function by improving readability and condensing comments. Adjust casing and truncation logic for single-word cases and handle versioned names more intuitively. * Ensure custom file names are converted to lowercase. When renaming files using a custom short name, enforce lowercase conversion to maintain consistency and prevent potential issues with case-sensitive file systems. This change applies to both the tinygrad and metadata file names. * Use UTC alias for timezone in datetime imports Replaced `timezone.utc` with `UTC` from the `datetime` module for clarity and consistency. This simplifies the representation of time zones and aligns with Python's updated best practices. --------- Co-authored-by: Jason Wen <haibin.wen3@gmail.com>	2025-05-25 10:50:09 +02:00