mirror of
https://github.com/sunnypilot/sunnypilot.git
synced 2026-02-18 20:03:53 +08:00
* Create a GitHub Actions workflow for synchronizing the repository to GitLab and enhance GitLab CI settings A new GitHub Actions workflow for mirroring the current repository to GitLab is added. This workflow is triggered by both push and delete events in addition to manual triggering. The role of GitHub actions runner is defined through a series of steps. A new '.gitlab-ci.yml' build configuration file is also introduced, with a more comprehensive definition of variables, jobs, and pipeline rules for its utilization in the GitLab CI/CD pipeline. Further, other changes include the addition of scripts for installing and uninstalling GitLab CI runner, along with modifying the SCons build system configuration file to include custom cache directory. Moreover, 'release_files.py' has been revised to include additional blacklisted and whitelisted files specific to Sunnypilot, ensuring suitable settings for the CI flow. The improvements facilitate a smoother integration between GitHub and GitLab, powerfully harnessing the capabilities of both platforms for more efficient and effective CI/CD pipelines and version control management. * not needed for this * Update workflow to build model from upstream repository Revised the CI workflow to build directly from the upstream `commaai/openpilot` repository. Simplified configuration, removed unused steps, and added support for specifying the upstream branch dynamically via inputs. * Update SConstruct to allow passing arbitrary cache_dir Modified the SConstruct file to enable setting a custom cache directory via arguments. This enhances flexibility in configuring cache paths during the build process. * Refactor build workflow to improve branch handling logic. Reorganized conditions for setting environment variables, replacing repository_dispatch with workflow_dispatch for prebuilt builds. Added a fallback error message for unsupported triggers to improve robustness. This enhances clarity and ensures compatibility with the intended workflow triggers. * test test test * test test test * test test test * Enable publication flag during build configurations Added `SHOULD_PUBLISH=true` to all relevant build configuration steps to ensure proper handling of publishing logic. Updated environment variables to include this flag for downstream usage. Removed the error-check step for unsupported configurations. * Simplify publish condition in workflow logic Replaced the previous condition for publishing with a single output variable, `should_publish`, to streamline logic and improve maintainability. This change reduces redundancy and makes the workflow more adaptable to future updates. * Simplify restore key patterns in build workflow. Removed unnecessary trailing dashes in SCons cache restore keys to streamline and slightly improve key matching logic. This ensures better consistency with the current workflow setup. * Update cache key usage in build workflow Replaced `github.ref_name` with `github.head_ref` for cache keys to ensure accurate branch-specific caching. Added fallback restore-keys for master branches to improve cache efficiency and reduce redundant builds. * Improved debug logging in GitHub actions workflow This commit refines the debug logging in our GitHub Actions workflow for the Sunnypilot build. This change provides more granularity, enabling logging only during debug mode, which helps to keep the runtime logs less cluttered during normal operations. This includes the conditionally displaying of environmental variables, GitHub output contents, and directory listings. Additionally, debug mode verbosity was added to rsync commands to aid troubleshooting file transfers during the build process. Blank lines were also reduced for better readability and cleaner code presentation. * test diff path * Refactor SCons cache handling in build workflow Replaced hardcoded cache directory paths with an environment variable (`SCONS_CACHE_DIR`) for better maintainability and flexibility. Updated related workflow steps to utilize the new variable and adjusted cache key usage. Removed unused `BASE_BUILD_NUMBER` variable to clean up the configuration. * Update cache key to include commit SHA in workflow This change adds the commit SHA to the cache key in the GitHub Actions workflow. It ensures more precise caching by differentiating builds based on the specific commit, reducing potential conflicts. * clean Update GitHub runner service to set environment variables Updated the ExecStart command to explicitly set HOME, USER, LOGNAME, and MAIL environment variables. This ensures the runner operates with the correct environment configuration, improving reliability and compatibility. Refactor GitHub runner service ExecStart command. Replaced direct command execution with running the service as a specific user using `su`. This improves compatibility and aligns with best practices for user-based execution. No functional changes are expected. Update GitHub runner service to set environment variables The ExecStart command now sets HOME, USER, LOGNAME, and MAIL environment variables for the runner process. This ensures proper environment initialization for the designated user, improving compatibility and reliability during execution. Refactor GitHub runner service template handling. Revised the `modify_service_template` function to create a properly structured service template for the GitHub runner. Updated service permissions, execution parameters, and enabled the function call to ensure usage during runner setup. * Refactor GitHub Runner installer to improve argument parsing. Reworked the script to implement flexible and explicit command-line argument parsing using flags like `--token`, `--repo`, and `--start-at-boot`. Added support for setting default values and enabling/disabling auto-start based on the `--start-at-boot` flag. Improved error handling and usage messaging for better user experience. * Update cache keys in sunnypilot build workflow Modified the cache keys to include `github.ref_name` for more precise caching and restore behavior. This improves build consistency by better differentiating between branches and refs. No changes to the overall workflow logic. * Remove 'tinygrad/*' from release file exclusions This change modifies the release file exclusions by removing 'tinygrad/*'. The adjustment ensures that files in the 'tinygrad' directory are now included in the release process, aligning with updated packaging requirements. * Simplify release file exclusions list. Removed redundant and unnecessary file patterns from the exclusions list in `release_files.py`. This streamlines the file handling process and reduces maintenance overhead. * Refactor SCons cache path and key structure. Updated the SCons cache directory path to `SCONS_CACHE_DIR` for clarity and consistency. Improved the caching key structure to include `github.head_ref` for better cache differentiation and restore hierarchy. Adjusted related build instructions to reflect the new environment variable. * Refactor branch configuration in CI workflow Standardize branch name handling by replacing hardcoded values with environment variables. This improves maintainability and simplifies updates to branch names across the workflow. Updated references to use the new dynamic environment variable approach.
141 lines
4.0 KiB
Bash
Executable File
141 lines
4.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -e
|
|
|
|
# Default values
|
|
DEFAULT_REPO_URL="https://github.com/sunnypilot"
|
|
START_AT_BOOT=false
|
|
|
|
# Parse command line arguments
|
|
while [[ $# -gt 0 ]]; do
|
|
case $1 in
|
|
--start-at-boot)
|
|
START_AT_BOOT=true
|
|
shift
|
|
;;
|
|
--token)
|
|
GITHUB_TOKEN="$2"
|
|
shift 2
|
|
;;
|
|
--repo)
|
|
REPO_URL="$2"
|
|
shift 2
|
|
;;
|
|
*)
|
|
if [ -z "$GITHUB_TOKEN" ]; then
|
|
GITHUB_TOKEN="$1"
|
|
elif [ -z "$REPO_URL" ]; then
|
|
REPO_URL="$1"
|
|
fi
|
|
shift
|
|
;;
|
|
esac
|
|
done
|
|
|
|
# Check required arguments
|
|
if [ -z "$GITHUB_TOKEN" ]; then
|
|
echo "Usage: $0 [--start-at-boot] [--token <github_token>] [--repo <repository_url>]"
|
|
echo "Required argument: github_token"
|
|
echo "Optional arguments:"
|
|
echo " --start-at-boot Enable auto-start at boot (default: false)"
|
|
echo " --repo Repository URL (default: ${DEFAULT_REPO_URL})"
|
|
exit 1
|
|
fi
|
|
|
|
# Set repository URL if not provided
|
|
REPO_URL="${REPO_URL:-$DEFAULT_REPO_URL}"
|
|
|
|
# Constants
|
|
RUNNER_USER="github-runner"
|
|
USER_GROUPS="comma,gpu,gpio,sudo"
|
|
BASE_DIR="/data/github"
|
|
RUNNER_DIR="${BASE_DIR}/runner"
|
|
BUILDS_DIR="${BASE_DIR}/builds"
|
|
LOGS_DIR="${BASE_DIR}/logs"
|
|
CACHE_DIR="${BASE_DIR}/cache"
|
|
OPENPILOT_DIR="${BASE_DIR}/openpilot"
|
|
|
|
create_directories() {
|
|
sudo mkdir -p "$RUNNER_DIR" "$BUILDS_DIR" "$LOGS_DIR" "$CACHE_DIR" "$OPENPILOT_DIR"
|
|
mkdir -p "/data/openpilot"
|
|
sudo chown -R comma:comma "/data/openpilot"
|
|
}
|
|
|
|
download_and_setup_runner() {
|
|
cd "$RUNNER_DIR"
|
|
curl -o actions-runner-linux-arm64-2.321.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-arm64-2.321.0.tar.gz
|
|
tar xzf ./actions-runner-linux-arm64-2.321.0.tar.gz
|
|
rm ./actions-runner-linux-arm64-2.321.0.tar.gz
|
|
chmod +x ./config.sh
|
|
}
|
|
|
|
setup_runner_user() {
|
|
sudo useradd --comment 'GitHub Runner' --create-home --home-dir ${BASE_DIR} ${RUNNER_USER} --shell /bin/bash -G ${USER_GROUPS} || sudo usermod -aG ${USER_GROUPS} ${RUNNER_USER}
|
|
export BASE_DIR
|
|
sudo -u ${RUNNER_USER} bash -c "truncate -s 0 '${BASE_DIR}/.bash_logout'"
|
|
}
|
|
|
|
create_sudoers_entry() {
|
|
sudo grep -qxF "${RUNNER_USER} ALL=(ALL) NOPASSWD: ALL" /etc/sudoers || echo "${RUNNER_USER} ALL=(ALL) NOPASSWD: ALL" | sudo tee -a /etc/sudoers
|
|
}
|
|
|
|
configure_runner() {
|
|
cd "$RUNNER_DIR"
|
|
sudo -u ${RUNNER_USER} ./config.sh --url "$REPO_URL" --token "$GITHUB_TOKEN" --name $(hostname) --runnergroup "tici-tizi" --labels "tici" --work "$BUILDS_DIR" --unattended
|
|
}
|
|
|
|
set_directory_permissions() {
|
|
sudo chown -R ${RUNNER_USER}:comma "$BASE_DIR"
|
|
sudo chmod g+rwx "$BASE_DIR"
|
|
sudo chmod g+s "$BASE_DIR"
|
|
}
|
|
|
|
modify_service_template() {
|
|
cat <<EOL > "$RUNNER_DIR/bin/actions.runner.service.template"
|
|
[Unit]
|
|
Description={{Description}}
|
|
After=network-online.target nss-lookup.target time-sync.target
|
|
Wants=network-online.target nss-lookup.target time-sync.target
|
|
StartLimitInterval=5
|
|
StartLimitBurst=10
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=root
|
|
ExecStart=/usr/bin/unshare -m -- /bin/bash -c 'mount --bind ${OPENPILOT_DIR} /data/openpilot && setpriv --reuid={{User}} --regid={{User}} --init-groups env HOME=${BASE_DIR} USER={{User}} LOGNAME={{User}} MAIL=/var/mail/{{User}} {{RunnerRoot}}/runsvc.sh'
|
|
WorkingDirectory={{RunnerRoot}}
|
|
KillMode=process
|
|
KillSignal=SIGTERM
|
|
TimeoutStopSec=5min
|
|
Restart=always
|
|
RestartSec=120
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
EOL
|
|
}
|
|
|
|
# Make filesystem writable
|
|
sudo mount -o remount,rw /
|
|
|
|
# Ensure filesystem is remounted as read-only on script exit
|
|
trap "sudo mount -o remount,ro /" EXIT
|
|
|
|
# Execute installation steps
|
|
setup_runner_user
|
|
create_sudoers_entry
|
|
create_directories
|
|
download_and_setup_runner
|
|
modify_service_template
|
|
configure_runner
|
|
set_directory_permissions
|
|
|
|
# Install and start service using built-in installer
|
|
cd "$RUNNER_DIR"
|
|
sudo ./svc.sh install $RUNNER_USER
|
|
|
|
# Handle auto-start configuration
|
|
if [ "$START_AT_BOOT" = false ]; then
|
|
sudo systemctl disable actions.runner.sunnypilot.$(uname -n)
|
|
fi
|
|
|
|
sudo ./svc.sh start |