Commit graph

204 commits

Author SHA1 Message Date
Marco Ferrari
11d4aac68c
chore: format super-linter summaries (#6063)
Run Prettier to format the Super-linter summary file to produce
summaries with consistent formatting.
2024-08-25 19:43:33 +02:00
Marco Ferrari
9f193cc3c0
feat: format JSONC and JSON5 with prettier (#6041) 2024-08-16 17:15:05 +02:00
Marco Ferrari
8ceae0f304
feat: lint and format dotnet solutions (#6039)
Close #736
2024-08-16 16:23:52 +02:00
Marco Ferrari
1d12a31fce
feat: prettier for additional languages (#6025)
- CSS, Sass, SCSS
- HTML
- JSON
- JSX
- GraphQL
- Vue
- YAML
2024-08-15 11:46:02 +02:00
Marco Ferrari
2f0ac20566
fix: avoid concurrent edits to the same file (#6027)
Reduce the number of processes to 1 if at least one FIX_xxxx variable is
set to true. This avoids that Parallel runs multiple processes that
might edit the same file at the same time.
2024-08-14 22:20:56 +02:00
Marco Ferrari
9131ef7278
fix!: remove undocumented secondary extension case (#5983)
Remove an undocumented corner case that might cause Super-linter to
consider a configuration value for a configuration file that points to a
non-existing YAML file as valid. Before this change, Super-linter falls
back to a yml (yaml) extension if the configuration value points to a
non-existing file with a yaml (yml) extension. This workaround dates
back to when Super-linter didn't support customizing configuration file
names.

Now, if the configuration file doesn't exist, Super-linter throws a
fatal error so users can fix their configuration instead.
2024-08-14 18:50:27 +02:00
Marco Ferrari
3d3274d18a
fix: remove intro text from stdout and stderr (#6020)
- Remove introductory text from stdout and stderr files.
- Always create stdout and stderr files, not only when there are errors.
2024-08-14 18:16:44 +02:00
Marco Ferrari
48d5391b7b
fix!: remove sql-lint (#5991)
sql-lint is unmaintained, and its dependencies are impacted by several
vulnerabilities.
2024-08-14 17:30:46 +02:00
Marco Ferrari
d9bf945fd2
chore: remove file_array leftovers (#6018)
Remove the initialization of FILE_ARRAY_xxx variables because worker.sh
dynamically initializes them already.
2024-08-14 17:26:55 +02:00
Marco Ferrari
8dc1475a3a
feat: allow customizing gitleaks log level (#5993) 2024-08-14 14:54:16 +02:00
Marco Ferrari
91dc6d7234
fix: add missing fix mode options and test cases (#5987)
- Add missing fix mode options for: CLANG_FORMAT, ENV,
  GOOGLE_JAVA_FORMAT, NATURAL_LANGUAGE, PYTHON_ISORT, RUST_CLIPPY.
- Refactor linter tests to make them shorter because there's no need to
  have big test files.
- Refactor 'bad' linter tests for linters that support fix mode so they
  contain only automatically fixable issues. This is needed to avoid
  adding another set of 'bad' linter tests for fix mode.
- Provide configuration files for linters that support fix mode and for
  which the default configuration is not suitable to enable fix mode:
  ansible-lint, ESLint, golangci-lint.
- Add a test case for linter commands options for linters that support
  fix mode, to ensure that fix mode and check-only mode options have
  been defined.
- Refactor the fix mode test to check if linters actually applied
  modifications to files.
- Update documentation about adding test cases for linters that support
  fix mode.
- Don't exit with a fatal error if VALIDATE_xxx is false when testing
  fix mode because not all linters support fix mode. To enable this, set
  the new FIX_MODE_TEST_CASE_RUN variable to true.
2024-08-12 12:31:38 +02:00
Marco Ferrari
40410c2d60
fix: use cargo clippy instead of cargo-clippy (#5988)
The cargo-clippy command has been depreacted since Cargo
v1.78.0. This is needed to unblock the implementation of fix mode.
2024-08-11 23:18:33 +02:00
Marco Ferrari
6fdc091361
feat: local fix mode (#5978)
Certain linters and formatters support fixing linting and formatting
issues (fix mode). Before this change, Super-linter runs linters and
formatters in a mode that doesn't modify the source code in any way
(check only mode).

With this change, Super-linter supports running linters and formatters
in fix mode if explicitly requested by the configuration. If the
configuration includes a variable named FIX_<language_name>,
Super-linters modifies the command to run the linter or formatter for
<language_name> to enable fix mode.

The modifications to the linter or formatter command that Super-linter
applies depend on what is the default for a particular linter: it either
removes or adds options to the command to run the linter or formatter.
2024-08-07 15:36:16 +02:00
Marco Ferrari
0dbe144336
feat: support ignore_gitignored_files with jscpd (#5958)
Add the --gitignore option to LINTER_COMMANDS_ARRAY_JSCPD when
IGNORE_GITIGNORED_FILES=true. Users can also set the gitignore options
in the Jscpd configuration file, but with this change we make Jscpd
automatically respecting that.
2024-08-05 10:15:40 +02:00
Marco Ferrari
c46346c774
chore: remove clippy script, run clippy directly (#5953)
Remove clippy.sh script and run cargo-clippy directly. The script was
needed because when we introduced cargo-clippy, Super-linter didn't have
a way to customize the working directory when running linters and
formatters. Now, we can use GNU Parallel to handle that case.

This will unblock the work to do to implement #4001 (pass custom options
to cargo-clippy).
2024-08-02 13:24:43 +02:00
Marco Ferrari
b09ac00c68
docs: add info about standard and prettier (#5948)
Document how to update the configuration when it includes
JAVASCRIPT_DEFAULT_STYLE or TYPESCRIPT_DEFAULT_STYLE.
2024-08-01 12:39:48 +02:00
Marco Ferrari
ef57e132e1
fix: avoid duplicated content in summary (#5939)
- Remove the previous summary file is present to avoid showing stale
  contents from old summaries.
- Extract the logic to write summary heading and lines to dedicated
  functions in output.sh to make it easier adding new formats in the
  future.
2024-07-30 08:35:14 +02:00
Marco Ferrari
78ed3ef5fc
fix: store outputs in the main output directory (#5899)
- Store Super-linter outputs in the main output directory.
- Update README with a few fixes and details about outputs.
- Add missing test cases for when the configuration didn't enable any
  output.
2024-07-28 21:34:40 +02:00
Marco Ferrari
2daf461143
feat: allow using both prettier and standardjs (#5679)
- Allow using both Prettier and StandardJS at the same time by removing
  JAVASCRIPT_DEFAULT_STYLE and TYPESCRIPT_DEFAULT_STYLE
  configuration variables. They create a confusing corner case that
  Super-linter already handles with its core logic that's in place to
  enable and disable linters.
- Document VALIDATE_JAVASCRIPT_PRETTIER and VALIDATE_TYPESCRIPT_PRETTIER
  because they were missing from the README.
- Add missing TYPESCRIPT_PRETTIER tests.
2024-07-19 05:04:43 +00:00
Marco Ferrari
d2d73347d3
chore: split validation logic in smaller functions (#5892)
- Move USE_FIND_ALGORITHM and VALIDATE_ALL_CODEBASE validation in a
  dedicated function (ValidateFindMode).
- Move ANSIBLE_DIRECTORY validation to a dedicated function
  (ValidateAnsibleDirectory).
- Move VALIDATE_xxxx variables validation to a dedicated function
  (ValidateValidationVariables).
- Mark ANY_SET, ANY_TRUE, ANY_FALSE as local because we don't need to
  reference them anywhere outside ValidateValidationVariables.
- Add some debug statements in validation functions.
- Merge the loops to initialize VALIDATE_xxx variables and to print
  enable/disable language debug messages.
- Add tests for these validation functions.
- Add test start message for all tests.
2024-07-16 08:22:45 +00:00
Marco Ferrari
5be4926633
feat: remove no-eslintrc and simplify eslint conf (#5809)
- Remove --no-eslintrc to simplify the upgrade path to ESlint >= v9,
  because v9 defaults to aonther configuration file format.
- Simplify the default ESlint configuration removing options that have
  been deprecated, or that are equal to the default ones.

Close #5688
2024-07-11 12:54:14 +02:00
Marco Ferrari
57c86588c3
feat: write github actions step summary (#5867)
This also removes an unneeded textlint configuration file because it
matches with the default one.

Close #5650
2024-07-11 08:01:11 +00:00
Marco Ferrari
94bb3f5563
feat: save super-linter output if requested (#5806)
- New SAVE_SUPER_LINTER_OUTPUT variable. When set to true,
  saves super-linter output to ${DEFAULT_WORKSPACE}/${SUPER_LINTER_OUTPUT_DIRECTORY_NAME}
- New SUPER_LINTER_OUTPUT_DIRECTORY_NAME variable to set the output
  directory name instide the default workspace.

Close #5774
2024-07-01 14:50:52 +02:00
Marco Ferrari
cef17760de
feat: configure github server url (#5792)
Allow the configuration of the GitHub server URL, and add some
validation rules that check for common misconfigurations.

Close #5572
2024-06-21 08:54:19 +02:00
Marco Ferrari
c99ec7784a
fix: don't skip processing ansible_directory pwd (#5790)
Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.

Fix #5789

Other related changes

- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
  contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
  already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
  deleted when we moved log variables to log.sh
2024-06-19 16:58:11 +00:00
Marco Ferrari
6047e3f732
feat: ignore avif files when building file list (#5604)
Close #5382
2024-05-06 10:47:11 +02:00
Chongyi Zheng
f84508a9c4
feat: find go.mod based on Go files (#5345) 2024-04-30 14:44:42 +02:00
Marco Ferrari
c26430f868
feat: support arbitrary shellcheck config paths (#5571)
- Support passing an arbitrary path to the shellcheck configuration file
  with the BASH_CONFIG_FILE_NAME variable. This brings shellcheck in
  line with other linters that support configuration files.
- Enable shellcheck external sources using a directive in the
  configuration file, so users can eventually override it if not needed.
- Export command configuration variables that subprocesses might need.

Close #5414
2024-04-25 19:47:55 +00:00
Marco Ferrari
49001a2405
fix: respect log level when writing to the log (#5546)
- Write log messages in the log file according to the LOG_LEVEL that the
  user configured (or the default), instead of printing all the messages
  regardless of LOG_LEVEL to the log file.
- Don't emit colors if there is no terminal

Close #5337
2024-04-20 09:18:14 +00:00
Marco Ferrari
8f405c1a9c
fix: handle initial commit (#5534)
Close #5453
2024-04-18 06:48:55 +00:00
Masaya Suzuki
69249882f3
feat: support GoReleaser (#5505) 2024-04-15 12:38:25 +00:00
Ümit Büyükulcay
0ae4572874
feat: add clang-format style configuration (#5424) 2024-04-11 14:59:47 +02:00
Junya Okabe
e71a37d49d
feat: add depndency (#5456)
feat: configure ruff

feat: update the orchestration scripts

feat: update the test suite

docs: update README

feat: add test cases for ruff

fix: CI error

chore: del .github/linters/.ruff.toml

fix: CI error

fix: README

update: LINTER_NAMES_ARRAY

fix: Dockerfile

fix: .github/linters/.jscpd.json

fix: test files

fix: del version_command
2024-04-10 12:02:28 +00:00
Marco Ferrari
cb109f45f1
fix: export test_case_run (#5499)
- Export the TEST_CASE_RUN variable because subprocesses reference it
  when building the file list.
- Remove a duplicate file that we didn't catch because Jscpd was not
  running due to the TEST_CASE_RUN variable not being exported.

Fix #5483
2024-04-08 20:48:30 +02:00
Benjamin Wuethrich
95aabd4cfa
feat(bash-exec): add option to ignore shell library files (#5254)
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
2024-02-27 18:17:22 +00:00
Marco Ferrari
608bd502d8
docs: suggest setting fetch-depth on ref errors (#5316)
Emit a hopefully more helpful error message when the Git ref doesn't
exist. The error message is the same we use when getting changed files
only: it suggests checking that repository clone is not shallow, and
that the full history is available. Additionally, when running on
GitHub Actions, it also suggests checking the fetch-depth option of the
actions/checkout step.

Fix #5315
2024-02-27 13:25:18 +01:00
Marco Ferrari
52b082b1f6
fix: fix log level variable assignment (#5319)
Correctly declare log level variables when initializing the logging
infrastructure.
2024-02-23 21:16:09 +00:00
Marco Ferrari
0f91a56f21
fix: simplify log infrastructure (#5312)
- Move log variables in log.sh to shorten linter.sh
- Source log.sh as soon as possible, so we can use log functions and
  variables as soon as possible.
- Validate 'boolean' log variables: LOG_DEBUG, LOG_VERBOSE, LOG_NOTICE,
  LOG_WARN, LOG_ERROR.
- Move foreground color markers from an associative array (that we
  cannot export), inside the log function. This fixes an issue that we
  discovered while working on #5254 where color markers were not
  available to subprocesses.
- Remove background color markers because we don't use them.
2024-02-21 17:51:47 +00:00
Marco Ferrari
0967cd29d0
feat: enable shell error checks (#5126)
Enable error checks to:

- Exit on errors
- Disallow empty variables
- Fail when a piped command errors
2024-02-20 19:05:39 +00:00
Marco Ferrari
091eaa71e3
feat: show error output when info is disabled (#5251)
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
2024-02-10 10:16:31 +01:00
Marco Ferrari
5b5e54ad5c
fix: initialize terrascan at runtime (#5246)
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
2024-02-09 22:57:01 +00:00
Marco Ferrari
6fd6830fb4
fix: initialize GitHub domain variable (#5216)
- Initialize GITHUB_DOMAIN as documented in the README
- Configure other URLs according to GITHUB_DOMAIN
- Automatically fetch SSH key fingerprints for GITHUB_DOMAINS
2024-02-09 18:45:44 +00:00
Marco Ferrari
83eca1df43
fix: unset the log_level variable (#5249)
- Super-linter uses the LOG_LEVEL variable to let the user
  configure the desired log level. Checkov and Renovate use a variable
  with the same name for the same purpose, but accept a
  different set of values, and exit with an error if it gets an unknown
  value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
  Configuration validation will warn users if they use VERBOSE and
  instruct them to use INFO instead. This is not a breaking change
  because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
  configuration validation will warn the user. Fall back to DEBUG if the
  user configured LOG_LEVEL to VERBOSE.

Close #5217
2024-02-09 18:43:58 +00:00
Marco Ferrari
787b63ddb2
fix: write hint about fetch-depth (#5241)
Point users at the fetch-depth option when not running locally.
2024-02-09 17:45:00 +00:00
Marco Ferrari
a26db6d34d
feat: lint xsd files (#5250)
Lint XSD files with xmllint

Close #5248
2024-02-09 17:44:30 +00:00
Marco Ferrari
fe6e29b685
feat: automatically set the default branch (#5242)
Get the default branch from the GitHub Actions event payload when
running on GitHub Actions. Default to 'master', as before, otherwise.
2024-02-08 08:41:07 +00:00
Marco Ferrari
38edbe557a
fix: create mypy cache directory (#5240)
MyPy expects the cache directory to be there. We now remove it after
each super-linter run to avoid leftovers, so we need to create it before
running MyPy.

See https://github.com/python/mypy/issues/10768
and https://github.com/python/mypy/issues/10863
2024-02-06 15:53:30 +01:00
Marco Ferrari
20ded7178b
fix: don't print empty lines with default logging (#5238)
- Check if Stdout and Stderr have elements before printing them.
- Run the super-linter action in a dedicated step using default logging to
  inspect how the output looks during CI.
2024-02-05 18:24:07 +01:00
Marco Ferrari
d7790e4f1c
build: move linter verions logic outside runtime (#5197)
Move the logic to build the linter versions file outside the main
runtime. There's no need to include it there because it's used only when
building the image.

Move the list of linters by name in linterVersions.sh because we don't
need it in any other place.
2024-02-05 13:31:36 +01:00
Marco Ferrari
eded42747b
fix: don't add unnecessary empty lines (#5221) 2024-02-05 11:49:22 +01:00