Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.
Fix#5789
Other related changes
- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
deleted when we moved log variables to log.sh
- Run a job after all the jobs in the dynamically built test matrix run
to completion. This job is useful for branch protection rules that
that need the whole test suite to run successfully.
- Fix linting issues.
Fix#5686
- Don't require Git SHA and branch validation when
IGNORE_GITIGNORED_FILES=true because we only need to validate that the
workspace is a Git repository in this case.
Fix#5383
- Don't require Git SHA and branch validation when
IGNORE_GITIGNORED_FILES=true because we only need to validate that the
workspace is a Git repository in this case.
Fix#5383
- Support passing an arbitrary path to the shellcheck configuration file
with the BASH_CONFIG_FILE_NAME variable. This brings shellcheck in
line with other linters that support configuration files.
- Enable shellcheck external sources using a directive in the
configuration file, so users can eventually override it if not needed.
- Export command configuration variables that subprocesses might need.
Close#5414
- Configure Ruff to store its cache in a temporary directory inside the
container by default. Users can still override this by providing a
configuration file for Ruff.
- Add tests to ensure that super-linter deletes temporary files and
directories.
Close#5543
- Write log messages in the log file according to the LOG_LEVEL that the
user configured (or the default), instead of printing all the messages
regardless of LOG_LEVEL to the log file.
- Don't emit colors if there is no terminal
Close#5337
feat: configure ruff
feat: update the orchestration scripts
feat: update the test suite
docs: update README
feat: add test cases for ruff
fix: CI error
chore: del .github/linters/.ruff.toml
fix: CI error
fix: README
update: LINTER_NAMES_ARRAY
fix: Dockerfile
fix: .github/linters/.jscpd.json
fix: test files
fix: del version_command
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
Install the .NET SDK and PowerShell from their container images so that
we avoid spending time running their installers, and we can control
their updates automatically.
Close#5243
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
- Initialize GITHUB_DOMAIN as documented in the README
- Configure other URLs according to GITHUB_DOMAIN
- Automatically fetch SSH key fingerprints for GITHUB_DOMAINS
- Super-linter uses the LOG_LEVEL variable to let the user
configure the desired log level. Checkov and Renovate use a variable
with the same name for the same purpose, but accept a
different set of values, and exit with an error if it gets an unknown
value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
Configuration validation will warn users if they use VERBOSE and
instruct them to use INFO instead. This is not a breaking change
because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
configuration validation will warn the user. Fall back to DEBUG if the
user configured LOG_LEVEL to VERBOSE.
Close#5217
Move the logic to build the linter versions file outside the main
runtime. There's no need to include it there because it's used only when
building the image.
Move the list of linters by name in linterVersions.sh because we don't
need it in any other place.
- Update @react-native-community dependencies to their @react-native
upgrades because the former namespace is deprecated (React release
0.72)
- Don't install @react-native/eslint-plugin directly because it's a
dependency of @react-native/eslint-config
Move the following tasks to dedicated stages so we can run these steps
in parallel with other stages:
- Build Python virtual environments
- Install npm packages
- Build clang-format
- Download and install TFlint plugins
Add missing target stage when building the container image
textlint was expensive to run because we added every file in the list of
files to lint to FILE_ARRAY_NATURAL_LANGUAGE. In #5041, we mitigated
this issue but lost the ability to run textlint on changed files only.
Given that textlint ignore files for which it doesn't have a plugin
installed, and that we don't currently install additional plugins
besides the default ones to lint markdown files and text files, we let
textlint run on these files only, so we can have the feature to lint
only changed files with this linter as well, back.
- Remove axios, immer, ini, lodash, node-fetch that were installed to
run Trivy reports. We can remove them because we use the Trivy action.
- Remove the Trivy workflow that was disabled anyway.
Add a default (empty) configuration file for Checkov and a smoke test to
ensure that we catch the case where a default configuration file is not
present, and it should be.
- Remove build-time dependencies
- Remove cached NPM packages
- Remove cached PyPi packages
- Remove dependency descriptors. These still count against the total space,
although it's a few KBs
- Install rust-clippy and rust-fmt using the OS package manager instead of
maintaining our own installation script
- Add tests for build time dependencies that are not supposed to be installed
Recent versions of dart analyzer don't support the --options
configuration option anymore, so we cannot pass an arbitrary
configuration file. This commit removes the default dart analyzer
configuration file as well, because it's not needed anymore.
- Run jscpd, gitleaks, textlint against the entire workspace instead of
running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
- Add support to run Checkov against infrastructure as code descriptors
that are in a given (configurable) directory. Defaults to lint the
whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
issues right away with this change.
- Fail if the installation of a R package fails.
- Install the remotes package once during the image build, and not when we scan
files at runtime.
- Reuse the default R library directory instead of moving it to /home/r-library
* Lint Go projects by directory
* Search for Go modules
* Fix test path
* Add test cases and fix command
* Change workdir
* Add a warning about false positives
* fatal instead of warn
* Move tests to the test directory
* Close group on fatal
* Don't fail in test mode
* Update documentation
* Fix typos
* Update security policy
* Remove outdated instructions
* Fix list
* Add more info to config load step
* Don't test linter.yml
* Point to the cd workflow in README
* Move badge up
* Add info about Make help
---------
Co-authored-by: Zack Koppert <zkoppert@github.com>
* chore: add test case for csharp language which should pass
it shows that the current version of linter for csharp language reports false positive issues
* feat: use built-in linter for csharp language
https://github.com/dotnet/format/issues/1268
* chore: remove dotnet-format installation from install-dotnet.sh
dotnet format is a part of .NET 6 SDK and shouldn't be installed separately anymore
* docs: update linter name for dotnet
instead of dotnet-format built-in format command from dotnet is used (points to the same repository as before)
* chore: update linter name for dotnet in tests
---------
Co-authored-by: Zack Koppert <zkoppert@github.com>
The PSScriptAnalyzer expects a text file, not a PowerShell extension
on the filename. The latest version of powershell now respects this
rule and breaks testing as the settings file doesn't meet the requirements
of the linter.
Signed-off-by: Brett Logan <lindluni@github.com>
* Fix issues with the container image build
* Fix asl-validator version check
* Disable filename rule when running ktlint tests
* Bump asl-validator from 2.2.1 to 3.0.8 in /dependencies
Bumps [asl-validator](https://github.com/ChristopheBougere/asl-validator) from 2.2.1 to 3.0.8.
- [Release notes](https://github.com/ChristopheBougere/asl-validator/releases)
- [Commits](https://github.com/ChristopheBougere/asl-validator/compare/2.2.1...3.0.8)
---
updated-dependencies:
- dependency-name: asl-validator
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Show errors when using curl to call the status API
* Make the build more reproducible
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Build static python depenencies
* Address linting
* Fix copy path
* cleaner
* Stage virtual environments
* Update Dockerfile to support virtual environments
* Remove old python builds
* Remove unnecessary RUN step
* Fix merge conflicts
* Remove test checking for PIP packages
We use virtual environments and no longer install the packages
via pip directly in the image. It should be enough that the version
tests check for the existence already and that the version
comes back correctly.
* Remove binary installation of black
* cleaner
* Remove pip
* pretty
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Add support for google-java-format
* adding tests and array
* fix comma
* Update linter.sh
* get it in order
* fix hava version
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Ignore files marked with @generated marker
`@generated` marker is used by certain tools to understand that the
file is generated, so it should be treated differently than a file
written by a human:
* these files do not need to be reformatted,
* diffs in these files are less important,
* and linters should not be invoked on these files.
This PR proposes builtin support for `@generated` marker (and
`@not-generated` marker to mark file as not generated when it
contains `@generated` marker, like `README.md`).
I have not found a standard for a generated file marker, but:
* Facebook [uses `@generated` marker](https://tinyurl.com/fb-generated)
* Phabricator tool which was spawned from Facebook internal tool
[also understands `@generated` marker](https://git.io/JnVHa)
* Cargo inserts `@generated` marker into [generated Cargo.lock files](https://git.io/JnVHP)
Super-linter supports regex includes and excludes, but they are
harder to maintain (each repository needs to be configured) than
patching the tools which generate the files.
My personal story is that I maintain rust-protobuf crate, which
started emitting `@generated` markers [six years ago](https://git.io/JnV5h)
after a request of a Phabricator user.
Test Plan:
Create a test file `test.sh`:
```
echo $a
```
Run:
```
docker run -e RUN_LOCAL=true -v $HOME/tmp/g:/tmp/lint super-linter-test
```
Result is:
```
In /tmp/lint/test.sh line 1:
echo $a
^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
^-- SC2154: a is referenced but not assigned.
^-- SC2086: Double quote to prevent globbing and word splitting.
...
2021-06-22 23:46:16 [ERROR] ERRORS FOUND in BASH:[1]
```
Now add `@generated` to the file and run again:
```
2021-06-22 23:47:13 [NOTICE] All file(s) linted successfully with no errors detected
```
Additionally, add `@not-generated` in addition to `@generated`, and
linter error pops up again.
* cleanup
* remove space
* fix non utf return
* fix non utf return
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
* adding new one...
* adding automation
* linter
* pull apart for faster builds
* maybe
* update automation
* fix that
* prime is smart
* fixed it
* quotes
* fix build image
* inpec
* typo
* forgot backslash
* fixed hairbrain logic
* this should help cleanup
* i hate r
* order
* maybe fix r
* maybe fix r
* update readme
* remove spaces
* update
* adding it
* fix typo
* fix tests
* adding back jsonlint
* adding back jsonlint
* fix spacing
* found the typo
* add tests
* adding rules for json
* add to local
* Generate a code coverage report
* make linters happy
* adding action and badge
* no report for me
* Upload the test report
* adding folders
* spaces not tabs
* makefiles love tabs
* spacing is important
* rawr
* Fix makefile
* Add diagnostic info
* Set UID and GID
* Make info
* Add docker images in diagnostic info
* Move info before testing the local action
* Add missing -C to git diff-tree
* Reset ownership
* Add missing quotes
* Sudo
* Attempt to fix ARM test
* Attempt to fix go test
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
