From eb4aad643b8075d768fcbb48c3859af1d4600eab Mon Sep 17 00:00:00 2001 From: Colwyn Fritze-Moor <80352425+colwynlegitscript@users.noreply.github.com> Date: Wed, 9 Feb 2022 13:24:57 -0800 Subject: [PATCH] feat: add SSH key support (#2454) * feat: add support for ssh keys and github.com connections * refactor: allow github.com setup and update docs * docs: add note about using ssh_key * fix: run shfmt * fix: add language to ssh key fence * fix: make ssh setup script executable * fix: gitleaks wins, openssh example removed * notes * docs: make the docs a little more clear Co-authored-by: Admiral Awkbar --- Dockerfile | 1 + README.md | 74 +++++++++++++++++++++++++++++++++++++++ lib/functions/setupSSH.sh | 44 +++++++++++++++++++++++ lib/linter.sh | 11 ++++++ 4 files changed, 130 insertions(+) create mode 100755 lib/functions/setupSSH.sh diff --git a/Dockerfile b/Dockerfile index cb59c9e4..d3559184 100644 --- a/Dockerfile +++ b/Dockerfile @@ -78,6 +78,7 @@ RUN apk add --no-cache \ net-snmp-dev \ npm nodejs-current \ openjdk11-jre \ + openssh-client \ openssl-dev \ perl perl-dev \ py3-setuptools python3-dev \ diff --git a/README.md b/README.md index eba4afec..9e9321f7 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ It is a simple combination of various linters, written in `bash`, to help valida - [Template rules files](#template-rules-files) - [Using your own rules files](#using-your-own-rules-files) - [Disabling rules](#disabling-rules) + - [Using your own SSH key](#using-your-own-ssh-key) - [Filter linted files](#filter-linted-files) - [Docker Hub](#docker-hub) - [Run Super-Linter outside GitHub Actions](#run-super-linter-outside-github-actions) @@ -315,6 +316,9 @@ But if you wish to select or exclude specific linters, we give you full control | **SCALAFMT_CONFIG_FILE** | `.scalafmt.conf` | Filename for [scalafmt configuration](https://scalameta.org/scalafmt/docs/configuration.html) (ex: `.scalafmt.conf`) | | **SNAKEMAKE_SNAKEFMT_CONFIG_FILE** | `.snakefmt.toml` | Filename for [Snakemake configuration](https://github.com/snakemake/snakefmt#configuration) (ex: `pyproject.toml`, `.snakefmt.toml`) | | **SSL_CERT_SECRET** | `none` | SSL cert to add to the **Super-Linter** trust store. This is needed for users on `self-hosted` runners or need to inject the cert for security standards (ex. ${{ secrets.SSL_CERT }}) | +| **SSH_KEY** | `none` | SSH key that has access to your private repositories | +| **SSH_SETUP_GITHUB** | `false` | If set to `true`, adds the `github.com` SSH key to `known_hosts`. This is ignored if `SSH_KEY` is provided - i.e. the `github.com` SSH key is always added if `SSH_KEY` is provided | +| **SSH_INSECURE_NO_VERIFY_GITHUB_KEY** | `false` | **INSECURE -** If set to `true`, does not verify the fingerprint of the github.com SSH key before adding this. This is not recommended! | | **SQL_CONFIG_FILE** | `.sql-config.json` | Filename for [SQL-Lint configuration](https://sql-lint.readthedocs.io/en/latest/files/configuration.html) (ex: `sql-config.json` , `.config.json`) | | **SQLFLUFF_CONFIG_FILE** | `/.sqlfluff` | Filename for [SQLFLUFF configuration](https://docs.sqlfluff.com/en/stable/configuration.html) (ex: `/.sqlfluff`, `pyproject.toml`) | | **SUPPRESS_FILE_TYPE_WARN** | `false` | If set to `true`, will hide warning messages about files without their proper extensions. Default is `false` | @@ -416,6 +420,76 @@ If your repository contains your own rules files that live outside of a `.github If you need to disable certain _rules_ and _functionality_, you can view [Disable Rules](https://github.com/github/super-linter/blob/main/docs/disabling-linters.md) +### Using your own SSH key + +If you need to add your own SSH key to the linter because of private dependencies, you can use the `SSH_KEY` environment +variable. The value of that environment variable should be an SSH private key that has access to your private +repositories. + +You should add this key as an [Encrypted Secret](https://docs.github.com/en/actions/security-guides/encrypted-secrets) +and access it with the `secrets` parameter. + +Example workflow: + +```yml +--- +################################# +################################# +## Super Linter GitHub Actions ## +################################# +################################# +name: Lint Code Base + +# +# Documentation: +# https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions +# + +############################# +# Start the job on all push # +############################# +on: + push: + branches-ignore: [master, main] + # Remove the line above to run when pushing to master + pull_request: + branches: [master, main] + +############### +# Set the Job # +############### +jobs: + build: + # Name the Job + name: Lint Code Base + # Set the agent to run on + runs-on: ubuntu-latest + + ################## + # Load all steps # + ################## + steps: + ########################## + # Checkout the code base # + ########################## + - name: Checkout Code + uses: actions/checkout@v2 + with: + # Full git history is needed to get a proper list of changed files within `super-linter` + fetch-depth: 0 + + ################################ + # Run Linter against code base # + ################################ + - name: Lint Code Base + uses: github/super-linter@v4 + env: + VALIDATE_ALL_CODEBASE: false + DEFAULT_BRANCH: master + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + SSH_KEY: ${{ secrets.SSH_PRIVATE_KEY }} +``` + ## Filter linted files If you need to lint only a folder or exclude some files from linting, you can use optional environment parameters `FILTER_REGEX_INCLUDE` and `FILTER_REGEX_EXCLUDE` diff --git a/lib/functions/setupSSH.sh b/lib/functions/setupSSH.sh new file mode 100755 index 00000000..c06acc3a --- /dev/null +++ b/lib/functions/setupSSH.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash + +################################################################################ +################################################################################ +########### Super-Linter linting Functions ##################################### +################################################################################ +################################################################################ +########################## FUNCTION CALLS BELOW ################################ +################################################################################ +################################################################################ +#### Function SetupSshAgent #################################################### +function SetupSshAgent() { + # Check to see if a SSH_KEY_SECRET was passed + if [ -n "${SSH_KEY}" ]; then + info "--------------------------------------------" + info "SSH key found, setting up agent..." + export SSH_AUTH_SOCK=/tmp/ssh_agent.sock + ssh-agent -a "${SSH_AUTH_SOCK}" >/dev/null + ssh-add - <<<"${SSH_KEY}" 2>/dev/null + fi +} +################################################################################ +#### Function SetupGithubComSshKeys ############################################ +function SetupGithubComSshKeys() { + if [[ -n "${SSH_KEY}" || "${SSH_SETUP_GITHUB}" == "true" ]]; then + info "Adding github.com SSH keys" + # Fetched out of band from + # https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints + GITHUB_RSA_FINGERPRINT="SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8" + ssh-keyscan -t rsa github.com >/tmp/github.pub 2>/dev/null + if [[ "${SSH_INSECURE_NO_VERIFY_GITHUB_KEY}" == "true" ]]; then + warn "Skipping github.com key verification and adding without checking fingerprint" + mkdir -p ~/.ssh + cat /tmp/github.pub >>~/.ssh/known_hosts + elif [[ "$(ssh-keygen -lf /tmp/github.pub)" == "2048 ${GITHUB_RSA_FINGERPRINT} github.com (RSA)" ]]; then + info "Successfully verified github.com key" + mkdir -p ~/.ssh + cat /tmp/github.pub >>~/.ssh/known_hosts + else + error "Could not verify github.com key. SSH requests to github.com will likely fail." + fi + fi +} +################################################################################ diff --git a/lib/linter.sh b/lib/linter.sh index 3c0b1ae4..70c1038b 100755 --- a/lib/linter.sh +++ b/lib/linter.sh @@ -60,6 +60,8 @@ source /action/lib/functions/updateSSL.sh # Source the function script(s) source /action/lib/functions/validation.sh # Source the function script(s) # shellcheck source=/dev/null source /action/lib/functions/worker.sh # Source the function script(s) +# shellcheck source=/dev/null +source /action/lib/functions/setupSSH.sh # Source the function script(s) ########### # GLOBALS # @@ -167,6 +169,9 @@ SNAKEMAKE_SNAKEFMT_FILE_NAME="${SNAKEMAKE_SNAKEFMT_CONFIG_FILE:-.snakefmt.toml}" SUPPRESS_FILE_TYPE_WARN="${SUPPRESS_FILE_TYPE_WARN:-false}" # shellcheck disable=SC2034 # Variable is referenced indirectly SUPPRESS_POSSUM="${SUPPRESS_POSSUM:-false}" +# SSH_KEY="${SSH_KEY}" +SSH_SETUP_GITHUB="${SSH_SETUP_GITHUB:-false}" +SSH_INSECURE_NO_VERIFY_GITHUB_KEY="${SSH_INSECURE_NO_VERIFY_GITHUB_KEY:-false}" # shellcheck disable=SC2034 # Variable is referenced indirectly # SSL_CERT_SECRET="${SSL_CERT_SECRET}" # shellcheck disable=SC2034 # Variable is referenced indirectly @@ -806,6 +811,12 @@ trap 'cleanup' 0 1 2 3 6 14 15 ########## Header +############################################ +# Create SSH agent and add key if provided # +############################################ +SetupSshAgent +SetupGithubComSshKeys + ################################################ # Need to update the loops for the image style # ################################################