When running super-linter on in a self hosted runner based on Action
Runner Controller in K8s container mode the /github/home directory is
not neccesarily mounted. But since we expect this directory to exist
super-linter fails to set /github/workspace as a Git safe directory:
2024-10-04 17:58:37 [INFO] Successfully validated GITHUB_WORKSPACE: /github/workspace
2024-10-04 17:58:37 [DEBUG] Configuring Git safe directories
2024-10-04 17:58:37 [DEBUG] Set /github/workspace as a Git safe directory
error: could not lock config file /github/home/.gitconfig: No such file or directory
2024-10-04 17:58:37 [FATAL] Cannot configure /github/workspace as a Git safe directory.
Making sure this directory exists in container image ensures that under
no circumstances we can fail - if the action runner provides
/github/home it will be mounted over, and if not we provide a directory
where git can actually create the new .gitconfig.
This also defaults the HOME environment variable to /github/home as it
is the default in most places. So if somebody decides to override HOME
they are also expected to ensure that this directory exists (or is a
volume mount)
The https://github.com/super-linter/super-linter/pull/6113 introduced
new Git merge conflicts linter check, that is error prone in some
conditions (see screenshot).
This PR adjusts this check to be more precise and exact in a way that
all three markers should be found to identify check as failed.
Additionally improve Git merge conflict markers matchers.
Co-authored-by: Marco Ferrari <ferrari.marco@gmail.com>
Define functions in linterCommands.sh to avoid the need of exporting
them. Without this fix, a command that sourced linterCommands.sh would
not be able to find these functions, such as AddOptionsToCommand.
Fix#6372
Add support to run Commitlint against commit messages. It supports the
current modes:
- Lint the commit message of the last commit
- Lint the commit messages of the pushed commits in case there is more
than one pushed commit
This commit also removes stuff that we used to run commitlint as a
standalone tool because we can now use the commitlint instance that
Super-linter ships:
- lint-commit steps in lint-commit the GitHub Actions workflow
- lint-commit Make target
- commitlint and its dependencies in package.json and package-lock.json
- Update devcontainer by setting Prettier as a formatter only for
supported languages.
- Install only the VS Code extensions that we need for Super-linter
development.
- Remove the devcontainer as soon as it's not needed to avoid leaving
leftovers behind.
- Enable proseWrap in Prettier configuration.
- Update documentation about how to configure new linters.
- Fix linting issues.
If REMOVE_ANSI_COLOR_CODES_FROM_OUTPUT is set to true, remove ANSI color
codes from linters stdout and stderr files, and from the Super-linter
log file.
Close#5540
Define the RUST_CLIPPY_COMMAND_OPTIONS variable to allow passing
arbitrary options to the command that runs RUST_CLIPPY.
Fix the ARM test case that just failed because of its apiVersion just
expired.
Close#4001
Linters affected:
- actionlint
- Kubeconform
- Perlcritic
Also:
- Update package-lock.json to update some vulnerable dependencies.
- Mount dependency files to where the dependency manager expects them so
it's easier to update vulnerable dependencies.
Fix#6084
Reduce the number of processes to 1 if at least one FIX_xxxx variable is
set to true. This avoids that Parallel runs multiple processes that
might edit the same file at the same time.
- Remove test leftovers before initializing the workspace
against which fix mode tests run. This prevents ownership issues.
- Pass container image build metadata as outputs of the container image
build job so we can include the validate-container-image-labels target
as other test target when we build the test matrix.
- Manually include the 'test' target when building the test suite matrix
so we run it as part of the test suite to ensure that tests don't
pollute each other's working directory.
Fix#5994
- Add missing fix mode options for: CLANG_FORMAT, ENV,
GOOGLE_JAVA_FORMAT, NATURAL_LANGUAGE, PYTHON_ISORT, RUST_CLIPPY.
- Refactor linter tests to make them shorter because there's no need to
have big test files.
- Refactor 'bad' linter tests for linters that support fix mode so they
contain only automatically fixable issues. This is needed to avoid
adding another set of 'bad' linter tests for fix mode.
- Provide configuration files for linters that support fix mode and for
which the default configuration is not suitable to enable fix mode:
ansible-lint, ESLint, golangci-lint.
- Add a test case for linter commands options for linters that support
fix mode, to ensure that fix mode and check-only mode options have
been defined.
- Refactor the fix mode test to check if linters actually applied
modifications to files.
- Update documentation about adding test cases for linters that support
fix mode.
- Don't exit with a fatal error if VALIDATE_xxx is false when testing
fix mode because not all linters support fix mode. To enable this, set
the new FIX_MODE_TEST_CASE_RUN variable to true.
Renovate has an optional requirement for the RE2 package, and will warn if
it is not present.
Setting env variable RENOVATE_X_IGNORE_RE2=true will hide this warning.
(see https://github.com/renovatebot/renovate/pull/21391)
Certain linters and formatters support fixing linting and formatting
issues (fix mode). Before this change, Super-linter runs linters and
formatters in a mode that doesn't modify the source code in any way
(check only mode).
With this change, Super-linter supports running linters and formatters
in fix mode if explicitly requested by the configuration. If the
configuration includes a variable named FIX_<language_name>,
Super-linters modifies the command to run the linter or formatter for
<language_name> to enable fix mode.
The modifications to the linter or formatter command that Super-linter
applies depend on what is the default for a particular linter: it either
removes or adds options to the command to run the linter or formatter.
Add the --gitignore option to LINTER_COMMANDS_ARRAY_JSCPD when
IGNORE_GITIGNORED_FILES=true. Users can also set the gitignore options
in the Jscpd configuration file, but with this change we make Jscpd
automatically respecting that.
Remove clippy.sh script and run cargo-clippy directly. The script was
needed because when we introduced cargo-clippy, Super-linter didn't have
a way to customize the working directory when running linters and
formatters. Now, we can use GNU Parallel to handle that case.
This will unblock the work to do to implement #4001 (pass custom options
to cargo-clippy).
- When a commit is a merge commit, adjust the revision so that it
selects the correct parent.
- Add relevant tests.
- Extract the logic to initialize a Git repository for tests in a
dedicated function.
Fix#5941
Move LANGUAGE_ARRAY definition and initialization to a dedicated file
(globals/languages.sh) so we can source it in different places without
having to source the whole linter.sh
-re-instates previous super-linter pylint functionality
-ignoring import errors is desired as super-linter does not
support installing dependencies as it runs
-creates a new test case file for confirming
import-errors are disabled this consists of a 3rd
party import and a basic usage
- Remove the previous summary file is present to avoid showing stale
contents from old summaries.
- Extract the logic to write summary heading and lines to dedicated
functions in output.sh to make it easier adding new formats in the
future.
- Store Super-linter outputs in the main output directory.
- Update README with a few fixes and details about outputs.
- Add missing test cases for when the configuration didn't enable any
output.
- Allow using both Prettier and StandardJS at the same time by removing
JAVASCRIPT_DEFAULT_STYLE and TYPESCRIPT_DEFAULT_STYLE
configuration variables. They create a confusing corner case that
Super-linter already handles with its core logic that's in place to
enable and disable linters.
- Document VALIDATE_JAVASCRIPT_PRETTIER and VALIDATE_TYPESCRIPT_PRETTIER
because they were missing from the README.
- Add missing TYPESCRIPT_PRETTIER tests.
- Move USE_FIND_ALGORITHM and VALIDATE_ALL_CODEBASE validation in a
dedicated function (ValidateFindMode).
- Move ANSIBLE_DIRECTORY validation to a dedicated function
(ValidateAnsibleDirectory).
- Move VALIDATE_xxxx variables validation to a dedicated function
(ValidateValidationVariables).
- Mark ANY_SET, ANY_TRUE, ANY_FALSE as local because we don't need to
reference them anywhere outside ValidateValidationVariables.
- Add some debug statements in validation functions.
- Merge the loops to initialize VALIDATE_xxx variables and to print
enable/disable language debug messages.
- Add tests for these validation functions.
- Add test start message for all tests.
Set up dotenv-linter for the slim image, and not only for the standard
image because it doesn't have dependencies on the Rust SDK, and weights
less than 10MB.
- New SAVE_SUPER_LINTER_OUTPUT variable. When set to true,
saves super-linter output to ${DEFAULT_WORKSPACE}/${SUPER_LINTER_OUTPUT_DIRECTORY_NAME}
- New SUPER_LINTER_OUTPUT_DIRECTORY_NAME variable to set the output
directory name instide the default workspace.
Close#5774
Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.
Fix#5789
Other related changes
- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
deleted when we moved log variables to log.sh
