Commit graph

89 commits

Author SHA1 Message Date
Marco Ferrari
c99ec7784a
fix: don't skip processing ansible_directory pwd (#5790)
Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.

Fix #5789

Other related changes

- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
  contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
  already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
  deleted when we moved log variables to log.sh
2024-06-19 16:58:11 +00:00
Andrew Kreuzer
5b5d2f7ef0
feat: checkov scans for helm charts (#5631) 2024-06-07 15:06:55 +00:00
Marco Ferrari
a86fbaf65e
ci: run a job on test suite success (#5687)
- Run a job after all the jobs in the dynamically built test matrix run
  to completion. This job is useful for branch protection rules that
  that need the whole test suite to run successfully.
- Fix linting issues.

Fix #5686
2024-05-24 23:09:39 +02:00
Masaya Suzuki
03b4aa0798
fix: fix a shadowing setting in .golangci.yml (#5654) 2024-05-16 17:15:23 +00:00
Marco Ferrari
2bb8a0a3e7
fix: no full git validation when ignoring files (#5599)
- Don't require Git SHA and branch validation when
  IGNORE_GITIGNORED_FILES=true because we only need to validate that the
  workspace is a Git repository in this case.

Fix #5383
2024-04-30 17:58:11 +02:00
Marco Ferrari
ff425b9a7d Revert "fix: no full git validation when ignoring files"
This reverts commit 5b0c248f9c.
2024-04-30 10:14:12 +00:00
Marco Ferrari
5b0c248f9c fix: no full git validation when ignoring files
- Don't require Git SHA and branch validation when
  IGNORE_GITIGNORED_FILES=true because we only need to validate that the
  workspace is a Git repository in this case.

Fix #5383
2024-04-30 10:11:14 +00:00
Marco Ferrari
c26430f868
feat: support arbitrary shellcheck config paths (#5571)
- Support passing an arbitrary path to the shellcheck configuration file
  with the BASH_CONFIG_FILE_NAME variable. This brings shellcheck in
  line with other linters that support configuration files.
- Enable shellcheck external sources using a directive in the
  configuration file, so users can eventually override it if not needed.
- Export command configuration variables that subprocesses might need.

Close #5414
2024-04-25 19:47:55 +00:00
Marco Ferrari
56e675bd33
fix: configure ruff with a temp cache (#5548)
- Configure Ruff to store its cache in a temporary directory inside the
  container by default. Users can still override this by providing a
  configuration file for Ruff.
- Add tests to ensure that super-linter deletes temporary files and
  directories.

Close #5543
2024-04-22 11:40:23 +02:00
Marco Ferrari
49001a2405
fix: respect log level when writing to the log (#5546)
- Write log messages in the log file according to the LOG_LEVEL that the
  user configured (or the default), instead of printing all the messages
  regardless of LOG_LEVEL to the log file.
- Don't emit colors if there is no terminal

Close #5337
2024-04-20 09:18:14 +00:00
dependabot[bot]
2baa96f9a9
build(deps): bump golang.org/x/net (#5544)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-19 17:44:32 +00:00
dependabot[bot]
7cbf4f8662
build(deps): bump golang.org/x/net (#5545)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-19 17:37:25 +00:00
Marco Ferrari
8f405c1a9c
fix: handle initial commit (#5534)
Close #5453
2024-04-18 06:48:55 +00:00
Masaya Suzuki
69249882f3
feat: support GoReleaser (#5505) 2024-04-15 12:38:25 +00:00
Junya Okabe
ca2a4167b8
fix: test/linters/python_*/** (#5502)
fix: revert python_isort/python_bad_1.py
2024-04-15 12:01:10 +00:00
Ümit Büyükulcay
0ae4572874
feat: add clang-format style configuration (#5424) 2024-04-11 14:59:47 +02:00
Junya Okabe
e71a37d49d
feat: add depndency (#5456)
feat: configure ruff

feat: update the orchestration scripts

feat: update the test suite

docs: update README

feat: add test cases for ruff

fix: CI error

chore: del .github/linters/.ruff.toml

fix: CI error

fix: README

update: LINTER_NAMES_ARRAY

fix: Dockerfile

fix: .github/linters/.jscpd.json

fix: test files

fix: del version_command
2024-04-10 12:02:28 +00:00
Chongyi Zheng
252a980961
build: add glibc via gcompat layer (#5334) 2024-02-29 17:12:01 +00:00
Benjamin Wuethrich
95aabd4cfa
feat(bash-exec): add option to ignore shell library files (#5254)
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
2024-02-27 18:17:22 +00:00
Marco Ferrari
0967cd29d0
feat: enable shell error checks (#5126)
Enable error checks to:

- Exit on errors
- Disallow empty variables
- Fail when a piped command errors
2024-02-20 19:05:39 +00:00
Marco Ferrari
091eaa71e3
feat: show error output when info is disabled (#5251)
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
2024-02-10 10:16:31 +01:00
Marco Ferrari
49320c834b
build: install dotnet and powershell from images (#5245)
Install the .NET SDK and PowerShell from their container images so that
we avoid spending time running their installers, and we can control
their updates automatically.

Close #5243
2024-02-10 08:51:09 +00:00
Marco Ferrari
5b5e54ad5c
fix: initialize terrascan at runtime (#5246)
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
2024-02-09 22:57:01 +00:00
Marco Ferrari
6fd6830fb4
fix: initialize GitHub domain variable (#5216)
- Initialize GITHUB_DOMAIN as documented in the README
- Configure other URLs according to GITHUB_DOMAIN
- Automatically fetch SSH key fingerprints for GITHUB_DOMAINS
2024-02-09 18:45:44 +00:00
Marco Ferrari
83eca1df43
fix: unset the log_level variable (#5249)
- Super-linter uses the LOG_LEVEL variable to let the user
  configure the desired log level. Checkov and Renovate use a variable
  with the same name for the same purpose, but accept a
  different set of values, and exit with an error if it gets an unknown
  value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
  Configuration validation will warn users if they use VERBOSE and
  instruct them to use INFO instead. This is not a breaking change
  because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
  configuration validation will warn the user. Fall back to DEBUG if the
  user configured LOG_LEVEL to VERBOSE.

Close #5217
2024-02-09 18:43:58 +00:00
Marco Ferrari
a26db6d34d
feat: lint xsd files (#5250)
Lint XSD files with xmllint

Close #5248
2024-02-09 17:44:30 +00:00
Marco Ferrari
fe6e29b685
feat: automatically set the default branch (#5242)
Get the default branch from the GitHub Actions event payload when
running on GitHub Actions. Default to 'master', as before, otherwise.
2024-02-08 08:41:07 +00:00
Marco Ferrari
d7790e4f1c
build: move linter verions logic outside runtime (#5197)
Move the logic to build the linter versions file outside the main
runtime. There's no need to include it there because it's used only when
building the image.

Move the list of linters by name in linterVersions.sh because we don't
need it in any other place.
2024-02-05 13:31:36 +01:00
Marco Ferrari
eded42747b
fix: don't add unnecessary empty lines (#5221) 2024-02-05 11:49:22 +01:00
Marco Ferrari
99e41ce451
feat: run linters in parallel (#5177) 2024-01-30 19:24:55 +00:00
Marco Ferrari
9bab4a90e8
chore: simplify updateSSL (#5130)
Run command directly instead of checking their exit code afterwards.
2024-01-30 09:05:47 +01:00
Marco Ferrari
e162b950f4
build: update tekton-lint to use its new namespace (#5176)
tekton-lint is now published under the @ibm namespace
2024-01-24 07:14:55 +00:00
Marco Ferrari
f3d1590cd4
build: update react native dependencies (#5152)
- Update @react-native-community dependencies to their @react-native
  upgrades because the former namespace is deprecated (React release
  0.72)
- Don't install @react-native/eslint-plugin directly because it's a
  dependency of @react-native/eslint-config
2024-01-18 17:39:35 +00:00
Marco Ferrari
df911171c4
build: python venvs and npm in dedicated stages (#5078)
Move the following tasks to dedicated stages so we can run these steps
in parallel with other stages:

- Build Python virtual environments
- Install npm packages
- Build clang-format
- Download and install TFlint plugins

Add missing target stage when building the container image
2024-01-10 11:09:53 +01:00
Marco Ferrari
6f70adee89
fix: enable linting changed files with textlint (#5100)
textlint was expensive to run because we added every file in the list of
files to lint to FILE_ARRAY_NATURAL_LANGUAGE. In #5041, we mitigated
this issue but lost the ability to run textlint on changed files only.
Given that textlint ignore files for which it doesn't have a plugin
installed, and that we don't currently install additional plugins
besides the default ones to lint markdown files and text files, we let
textlint run on these files only, so we can have the feature to lint
only changed files with this linter as well, back.
2024-01-10 09:49:04 +01:00
Marco Ferrari
cf2038d903
fix: fix GITHUB_BEFORE_SHA initalization for push (#5098) 2024-01-05 23:07:39 +01:00
Marco Ferrari
3847309eca
build: remove unneeded Node dependencies (#5093)
- Remove axios, immer, ini, lodash, node-fetch that were installed to
  run Trivy reports. We can remove them because we use the Trivy action.
- Remove the Trivy workflow that was disabled anyway.
2024-01-03 16:54:06 +01:00
Marco Ferrari
65aae17a26
build!: uninstall eslint-config-airbnb-typescript (#5077)
eslint-config-airbnb-typescript is unmaintained. Having it installed is
blocking @typescript-eslint/eslint-plugin updates
2024-01-03 10:45:39 +01:00
Marco Ferrari
9257ba8af3
build: use embedded checkstyle configuration files (#5089)
checkstyle embeds sun_checks.xml and google_checks.xml so we can use
them instead of shipping our own.
2024-01-02 18:47:45 +01:00
Marco Ferrari
901a901655
fix: add missing checkov configuration file (#5090)
Add a default (empty) configuration file for Checkov and a smoke test to
ensure that we catch the case where a default configuration file is not
present, and it should be.
2024-01-02 18:03:30 +01:00
Marco Ferrari
19e39e211e
build: install clang-format from OS package repo (#5071)
Build and install clang-format instead of pulling it from a (potentially)
unmaintained repository
2024-01-01 11:39:59 +01:00
Marco Ferrari
1ca3ebccd6
build: reduce container image size (#5072)
- Remove build-time dependencies
- Remove cached NPM packages
- Remove cached PyPi packages
- Remove dependency descriptors. These still count against the total space,
  although it's a few KBs
- Install rust-clippy and rust-fmt using the OS package manager instead of
  maintaining our own installation script
- Add tests for build time dependencies that are not supposed to be installed
2023-12-29 19:30:58 +01:00
Marco Ferrari
4d9eaa5c54
build: update Dart, dart analyzer to 3.2.4 (#5065)
Recent versions of dart analyzer don't support the --options
configuration option anymore, so we cannot pass an arbitrary
configuration file. This commit removes the default dart analyzer
configuration file as well, because it's not needed anymore.
2023-12-26 22:14:57 +01:00
dependabot[bot]
20d12b4c7a
deps(java): bump com.pinterest.ktlint:ktlint-cli in /dependencies/ktlint (#5050)
Remove KOTLIN_ANDROID because ktlint handles that case by reading
a configuration setting in .editorconfig.

Bumps [com.pinterest.ktlint:ktlint-cli](https://github.com/pinterest/ktlint) from 0.47.1 to 1.1.0.
- [Release notes](https://github.com/pinterest/ktlint/releases)
- [Changelog](https://github.com/pinterest/ktlint/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pinterest/ktlint/compare/0.47.1...1.1.0)

---
updated-dependencies:
- dependency-name: com.pinterest.ktlint:ktlint-cli
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-24 19:33:08 +01:00
Marco Ferrari
11b70102c3
feat!: run linters against the workspace (#5041)
- Run jscpd, gitleaks, textlint  against the entire workspace instead of
  running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
  ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
  buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
  not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
2023-12-24 17:56:15 +01:00
Marco Ferrari
b214a59ca7
fix: fix file list when looking for changes (#5044)
- Fix the file diff function on push events.
- Implement a test for the file diff function
2023-12-23 19:33:53 +01:00
Marco Ferrari
9d7268fb99
feat: add support for checkov to lint iac files (#4925)
- Add support to run Checkov against infrastructure as code descriptors
  that are in a given (configurable) directory. Defaults to lint the
  whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
  issues right away with this change.
2023-12-22 13:22:15 +01:00
Marco Ferrari
60983d395f
fix: fail if r package installation fails (#4994)
- Fail if the installation of a R package fails.
- Install the remotes package once during the image build, and not when we scan
  files at runtime.
- Reuse the default R library directory instead of moving it to /home/r-library
2023-12-21 20:52:57 +01:00
dependabot[bot]
088bfe8e7e
Bump golang.org/x/crypto in /test/linters/go_modules/go_modules_good (#5018)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-19 09:08:19 +00:00
dependabot[bot]
c33189a3d3
Bump golang.org/x/crypto in /test/linters/go_modules/go_modules_bad (#5017)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-19 09:08:10 +00:00