superlint/test/linters
Marco Ferrari 91dc6d7234
fix: add missing fix mode options and test cases (#5987)
- Add missing fix mode options for: CLANG_FORMAT, ENV,
  GOOGLE_JAVA_FORMAT, NATURAL_LANGUAGE, PYTHON_ISORT, RUST_CLIPPY.
- Refactor linter tests to make them shorter because there's no need to
  have big test files.
- Refactor 'bad' linter tests for linters that support fix mode so they
  contain only automatically fixable issues. This is needed to avoid
  adding another set of 'bad' linter tests for fix mode.
- Provide configuration files for linters that support fix mode and for
  which the default configuration is not suitable to enable fix mode:
  ansible-lint, ESLint, golangci-lint.
- Add a test case for linter commands options for linters that support
  fix mode, to ensure that fix mode and check-only mode options have
  been defined.
- Refactor the fix mode test to check if linters actually applied
  modifications to files.
- Update documentation about adding test cases for linters that support
  fix mode.
- Don't exit with a fatal error if VALIDATE_xxx is false when testing
  fix mode because not all linters support fix mode. To enable this, set
  the new FIX_MODE_TEST_CASE_RUN variable to true.
2024-08-12 12:31:38 +02:00
..
ansible fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
arm Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
bash Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
bash_exec feat(bash-exec): add option to ignore shell library files (#5254) 2024-02-27 18:17:22 +00:00
checkov feat: add the kustomize binary for checkov (#5763) 2024-06-27 19:03:16 +02:00
clang_format Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
clojure Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
cloudformation chore: fix linting errors (#5843) 2024-07-05 10:18:22 +02:00
coffeescript Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
cpp Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
csharp Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
css Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
dart build: update Dart, dart analyzer to 3.2.4 (#5065) 2023-12-26 22:14:57 +01:00
dockerfile_hadolint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
editorconfig Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
env Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
gherkin Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
github_actions ci: run a job on test suite success (#5687) 2024-05-24 23:09:39 +02:00
gitleaks feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
go fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
go_modules fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
go_releaser feat: support GoReleaser (#5505) 2024-04-15 12:38:25 +00:00
google_java_format Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
groovy feat: local fix mode (#5978) 2024-08-07 15:36:16 +02:00
html Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
java build: use embedded checkstyle configuration files (#5089) 2024-01-02 18:47:45 +01:00
javascript_es fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
javascript_prettier fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
javascript_standard fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
jscpd feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
json fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
jsonc fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
jsx fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
kotlin deps(java): bump com.pinterest.ktlint:ktlint-cli in /dependencies/ktlint (#5050) 2023-12-24 19:33:08 +01:00
kubernetes_kubeconform Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
latex Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
lua Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
markdown fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
natural_language fix: enable linting changed files with textlint (#5100) 2024-01-10 09:49:04 +01:00
openapi Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
perl Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_builtin Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_phpcs Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_phpstan Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_psalm Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
powershell Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
protobuf fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
python_black fix: test/linters/python_*/** (#5502) 2024-04-15 12:01:10 +00:00
python_flake8 fix: test/linters/python_*/** (#5502) 2024-04-15 12:01:10 +00:00
python_isort fix: test/linters/python_*/** (#5502) 2024-04-15 12:01:10 +00:00
python_mypy fix: test/linters/python_*/** (#5502) 2024-04-15 12:01:10 +00:00
python_pylint fix: update pylint to ignore import-errors (#5927) 2024-07-31 09:27:11 +02:00
python_ruff fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
r Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
raku Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
renovate Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
ruby Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_2015 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_2018 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_2021 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_clippy fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
scalafmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
shell_shfmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
snakemake_lint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
snakemake_snakefmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
sql Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
sqlfluff fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
states Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
tekton Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terraform_fmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terraform_terrascan fix: initialize terrascan at runtime (#5246) 2024-02-09 22:57:01 +00:00
terraform_tflint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terragrunt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
tsx fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
typescript_es fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
typescript_prettier fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
typescript_standard fix: add missing fix mode options and test cases (#5987) 2024-08-12 12:31:38 +02:00
xml feat: lint xsd files (#5250) 2024-02-09 17:44:30 +00:00
yaml Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
README.md feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00

Test Cases

This directory contains test cases that super-linter uses to validate if a particular linter is working.

These test cases focus on how super-linter invokes each linter and their exit codes. We deliberately avoid to verify if the output of a given linter matches the expectations because it's the responsibility of each linter to do so.

Test case format

Each super-linter language should have its own directory, named after the language they refer to.

The name of each test case denotes its nature:

  • Test cases that are expected to pass validation contain the good string in their filename, or path. Example: markdown_good_5.md
  • Test cases that are expected to fail validation contain the bad string in their filename, or path. Example: markdown_bad_5.md

Notes about specific tests

In this section, we explain the peculiarities of certain test cases.

SQL Fluff test cases

From version 0.12.0 SQLFluff requires a dialect to be set, and no longer sets a default. This can be provided as a command-line argument, or a .sqlfluff config file (either in the usualy place for SQLFluff config files, or within the folder containg the SQL).

For SQLFluff we have added a default .sqlfluff config file in its test directory.

OpenAPI test cases

The _bad_ tests are valid .yml/.json but invalid OpenAPI specs. The test extensions used are .ymlopenapi/.jsonopenapi instead of .yml/.json. This is to prevent the YAML and JSON tests from picking them up.

ARM test cases

apiVersions older than 2 years (730 days) are treated as errors by the ARM linter if there is a newer version available.

Ansible test cases

roles/ghe-initialize is a valid Ansible role