- New SAVE_SUPER_LINTER_OUTPUT variable. When set to true,
saves super-linter output to ${DEFAULT_WORKSPACE}/${SUPER_LINTER_OUTPUT_DIRECTORY_NAME}
- New SUPER_LINTER_OUTPUT_DIRECTORY_NAME variable to set the output
directory name instide the default workspace.
Close#5774
Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.
Fix#5789
Other related changes
- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
deleted when we moved log variables to log.sh
- Support passing an arbitrary path to the shellcheck configuration file
with the BASH_CONFIG_FILE_NAME variable. This brings shellcheck in
line with other linters that support configuration files.
- Enable shellcheck external sources using a directive in the
configuration file, so users can eventually override it if not needed.
- Export command configuration variables that subprocesses might need.
Close#5414
- Write log messages in the log file according to the LOG_LEVEL that the
user configured (or the default), instead of printing all the messages
regardless of LOG_LEVEL to the log file.
- Don't emit colors if there is no terminal
Close#5337
feat: configure ruff
feat: update the orchestration scripts
feat: update the test suite
docs: update README
feat: add test cases for ruff
fix: CI error
chore: del .github/linters/.ruff.toml
fix: CI error
fix: README
update: LINTER_NAMES_ARRAY
fix: Dockerfile
fix: .github/linters/.jscpd.json
fix: test files
fix: del version_command
- Export the TEST_CASE_RUN variable because subprocesses reference it
when building the file list.
- Remove a duplicate file that we didn't catch because Jscpd was not
running due to the TEST_CASE_RUN variable not being exported.
Fix#5483
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
Emit a hopefully more helpful error message when the Git ref doesn't
exist. The error message is the same we use when getting changed files
only: it suggests checking that repository clone is not shallow, and
that the full history is available. Additionally, when running on
GitHub Actions, it also suggests checking the fetch-depth option of the
actions/checkout step.
Fix#5315
- Move log variables in log.sh to shorten linter.sh
- Source log.sh as soon as possible, so we can use log functions and
variables as soon as possible.
- Validate 'boolean' log variables: LOG_DEBUG, LOG_VERBOSE, LOG_NOTICE,
LOG_WARN, LOG_ERROR.
- Move foreground color markers from an associative array (that we
cannot export), inside the log function. This fixes an issue that we
discovered while working on #5254 where color markers were not
available to subprocesses.
- Remove background color markers because we don't use them.
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
- Initialize GITHUB_DOMAIN as documented in the README
- Configure other URLs according to GITHUB_DOMAIN
- Automatically fetch SSH key fingerprints for GITHUB_DOMAINS
- Super-linter uses the LOG_LEVEL variable to let the user
configure the desired log level. Checkov and Renovate use a variable
with the same name for the same purpose, but accept a
different set of values, and exit with an error if it gets an unknown
value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
Configuration validation will warn users if they use VERBOSE and
instruct them to use INFO instead. This is not a breaking change
because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
configuration validation will warn the user. Fall back to DEBUG if the
user configured LOG_LEVEL to VERBOSE.
Close#5217
- Check if Stdout and Stderr have elements before printing them.
- Run the super-linter action in a dedicated step using default logging to
inspect how the output looks during CI.
Move the logic to build the linter versions file outside the main
runtime. There's no need to include it there because it's used only when
building the image.
Move the list of linters by name in linterVersions.sh because we don't
need it in any other place.
Exclude the default docker bake files in HCL format from the list of files to
lint with Terragrunt.
Note that Docker bake supports defining arbitrary names for
configuration files, so this commit doesn't cover the scenario that a
user defined an arbitrary name for a Docker bake configuration file in
HCL format. Example: example-bake-config-file.hcl will be included in
the list of files to lint with Terragrunt.
Ref (default bake names):
https://docs.docker.com/build/bake/reference/#file-format
- Remove the SKIP_FLAG variable and check for the length of the arrays
of files to lint directly.
- Remove the LIST_FILES variable, and use the FILE_ARRAY variable
directly.
- Remove the corner case for RENOVATE because renovate-config-validator
supports passing the path to the file to lint using an argument as the
default case does.
- Remove the corner case for ANSIBLE not having 'bad' tests because it
has them now.
- Set TF_DATA_DIR to avoid any modification to any existing Terraform
data directory that users might have in their workspace.
- Aggregate GO_MODULES and ANSIBLE corner cases because they are the
same.
- Remove the corner case for ANSIBLE to add a trailing slash to
TEST_CASE_FOLDER (similar reason as the previous point about ANSIBLE
corner case).
- Simplify log messages by removing color markers because they are
already handled in log.sh.
- Simplify linterVersions by removing redundant checks and functions.
- Avoid printing debug logs in the versions file.
- Validate variables representing boolean values.
- Group global variables in the same sections.
- Declare variables as lowercase with the 'declare -l' shell builtin for
more clarity.
- Honor SUPPRESS_FILE_TYPE_WARN when printing messages in the
CheckFileType function.
- Reduce duplication when handling log messages in the CheckFileType
function.
- Don't add files to the array of files to lint with JSCPD because we
lint the whole codebase with JSCPD anyway.
Deprecate the ERROR_ON_MISSING_EXEC_BIT variable to remove a confusing
corner case, and to align the BASH_EXEC check to how super-linter
handles other linters.
- Simplify file status checks to assume that a file should be linted
correctly unless we are running in test mode.
- Avoid the corner case of files having the 'bad' string as part of
their file name to be wrongly assumed to fail linting.
- Move FILE_STATUS initialization where it's needed, after running the
linters.
textlint was expensive to run because we added every file in the list of
files to lint to FILE_ARRAY_NATURAL_LANGUAGE. In #5041, we mitigated
this issue but lost the ability to run textlint on changed files only.
Given that textlint ignore files for which it doesn't have a plugin
installed, and that we don't currently install additional plugins
besides the default ones to lint markdown files and text files, we let
textlint run on these files only, so we can have the feature to lint
only changed files with this linter as well, back.
- Fix GITHUB_BEFORE_SHA initialization on push events by setting the
correct key.
- Add an additional check against setting GITHUB_BEFORE_SHA to null.
- Run the CI workflow on push events to trigger required status checks
when using the merge queue.
- Fail if the configuration contains USE_FIND_ALGORITHM=true and
VALIDATE_ALL_CODEBASE=false because this is an unsupported
configuration. Before this change, super-linter silently ignored
VALIDATE_ALL_CODEBASE, leading to potentially surprising behavior.
- Run jscpd, gitleaks, textlint against the entire workspace instead of
running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
- Add support to run Checkov against infrastructure as code descriptors
that are in a given (configurable) directory. Defaults to lint the
whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
issues right away with this change.
- Fail if the installation of a R package fails.
- Install the remotes package once during the image build, and not when we scan
files at runtime.
- Reuse the default R library directory instead of moving it to /home/r-library
* Lint Go projects by directory
* Search for Go modules
* Fix test path
* Add test cases and fix command
* Change workdir
* Add a warning about false positives
* fatal instead of warn
* Move tests to the test directory
* Close group on fatal
* Don't fail in test mode
