Move 'npm audit' execution to a dedicated target (and corresponding
step) so that we can modularize it, and avoid that it blocks that whole
test suite.
Add support to run Commitlint against commit messages. It supports the
current modes:
- Lint the commit message of the last commit
- Lint the commit messages of the pushed commits in case there is more
than one pushed commit
This commit also removes stuff that we used to run commitlint as a
standalone tool because we can now use the commitlint instance that
Super-linter ships:
- lint-commit steps in lint-commit the GitHub Actions workflow
- lint-commit Make target
- commitlint and its dependencies in package.json and package-lock.json
- Update devcontainer by setting Prettier as a formatter only for
supported languages.
- Install only the VS Code extensions that we need for Super-linter
development.
- Remove the devcontainer as soon as it's not needed to avoid leaving
leftovers behind.
- Enable proseWrap in Prettier configuration.
- Update documentation about how to configure new linters.
- Fix linting issues.
Define the RUST_CLIPPY_COMMAND_OPTIONS variable to allow passing
arbitrary options to the command that runs RUST_CLIPPY.
Fix the ARM test case that just failed because of its apiVersion just
expired.
Close#4001
Linters affected:
- actionlint
- Kubeconform
- Perlcritic
Also:
- Update package-lock.json to update some vulnerable dependencies.
- Mount dependency files to where the dependency manager expects them so
it's easier to update vulnerable dependencies.
Fix#6084
Set BUILD_VERSION to the content of the version descriptor (version.txt)
if it changed in the last commit, assuming that the last commit was a
release preparation commit that updated the version descriptor.
Close#4928
- Add missing fix mode options for: CLANG_FORMAT, ENV,
GOOGLE_JAVA_FORMAT, NATURAL_LANGUAGE, PYTHON_ISORT, RUST_CLIPPY.
- Refactor linter tests to make them shorter because there's no need to
have big test files.
- Refactor 'bad' linter tests for linters that support fix mode so they
contain only automatically fixable issues. This is needed to avoid
adding another set of 'bad' linter tests for fix mode.
- Provide configuration files for linters that support fix mode and for
which the default configuration is not suitable to enable fix mode:
ansible-lint, ESLint, golangci-lint.
- Add a test case for linter commands options for linters that support
fix mode, to ensure that fix mode and check-only mode options have
been defined.
- Refactor the fix mode test to check if linters actually applied
modifications to files.
- Update documentation about adding test cases for linters that support
fix mode.
- Don't exit with a fatal error if VALIDATE_xxx is false when testing
fix mode because not all linters support fix mode. To enable this, set
the new FIX_MODE_TEST_CASE_RUN variable to true.
Certain linters and formatters support fixing linting and formatting
issues (fix mode). Before this change, Super-linter runs linters and
formatters in a mode that doesn't modify the source code in any way
(check only mode).
With this change, Super-linter supports running linters and formatters
in fix mode if explicitly requested by the configuration. If the
configuration includes a variable named FIX_<language_name>,
Super-linters modifies the command to run the linter or formatter for
<language_name> to enable fix mode.
The modifications to the linter or formatter command that Super-linter
applies depend on what is the default for a particular linter: it either
removes or adds options to the command to run the linter or formatter.
- When a commit is a merge commit, adjust the revision so that it
selects the correct parent.
- Add relevant tests.
- Extract the logic to initialize a Git repository for tests in a
dedicated function.
Fix#5941
Move LANGUAGE_ARRAY definition and initialization to a dedicated file
(globals/languages.sh) so we can source it in different places without
having to source the whole linter.sh
- Remove the previous summary file is present to avoid showing stale
contents from old summaries.
- Extract the logic to write summary heading and lines to dedicated
functions in output.sh to make it easier adding new formats in the
future.
- Store Super-linter outputs in the main output directory.
- Update README with a few fixes and details about outputs.
- Add missing test cases for when the configuration didn't enable any
output.
- Run Docker build checks when building the dev-dependencies container
image and when running the test target.
- Run Docker build checks when building the Super-linter container, and
not just when running the test target.
- Use the correct variable when setting the dev-dependencies container
user.
- New SAVE_SUPER_LINTER_OUTPUT variable. When set to true,
saves super-linter output to ${DEFAULT_WORKSPACE}/${SUPER_LINTER_OUTPUT_DIRECTORY_NAME}
- New SUPER_LINTER_OUTPUT_DIRECTORY_NAME variable to set the output
directory name instide the default workspace.
Close#5774
Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.
Fix#5789
Other related changes
- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
deleted when we moved log variables to log.sh
- Don't require Git SHA and branch validation when
IGNORE_GITIGNORED_FILES=true because we only need to validate that the
workspace is a Git repository in this case.
Fix#5383
- Don't require Git SHA and branch validation when
IGNORE_GITIGNORED_FILES=true because we only need to validate that the
workspace is a Git repository in this case.
Fix#5383
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
Enable debug logging when one (or more) of the following conditions is
true:
- ACTIONS_RUNNER_DEBUG is set to true
- ACTIONS_STEPS_DEBUG is set to true
- RUNNER_DEBUG is set to 1
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
- Initialize GITHUB_DOMAIN as documented in the README
- Configure other URLs according to GITHUB_DOMAIN
- Automatically fetch SSH key fingerprints for GITHUB_DOMAINS
- Super-linter uses the LOG_LEVEL variable to let the user
configure the desired log level. Checkov and Renovate use a variable
with the same name for the same purpose, but accept a
different set of values, and exit with an error if it gets an unknown
value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
Configuration validation will warn users if they use VERBOSE and
instruct them to use INFO instead. This is not a breaking change
because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
configuration validation will warn the user. Fall back to DEBUG if the
user configured LOG_LEVEL to VERBOSE.
Close#5217
CHANGELOG.md is automatically generated by the release workflow.
Ignore it when linting the codebase because it might not pass textlint
validation because of lowercase terms.
Deprecate the ERROR_ON_MISSING_EXEC_BIT variable to remove a confusing
corner case, and to align the BASH_EXEC check to how super-linter
handles other linters.
Move the following tasks to dedicated stages so we can run these steps
in parallel with other stages:
- Build Python virtual environments
- Install npm packages
- Build clang-format
- Download and install TFlint plugins
Add missing target stage when building the container image
Add a default (empty) configuration file for Checkov and a smoke test to
ensure that we catch the case where a default configuration file is not
present, and it should be.
- Run jscpd, gitleaks, textlint against the entire workspace instead of
running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
- Add support to run Checkov against infrastructure as code descriptors
that are in a given (configurable) directory. Defaults to lint the
whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
issues right away with this change.
