lint/test/linters
Marco Ferrari 11b70102c3
feat!: run linters against the workspace (#5041)
- Run jscpd, gitleaks, textlint  against the entire workspace instead of
  running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
  ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
  buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
  not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
2023-12-24 17:56:15 +01:00
..
ansible feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
arm Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
bash Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
bash_exec Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
checkov feat: add support for checkov to lint iac files (#4925) 2023-12-22 13:22:15 +01:00
clang_format Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
clojure Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
cloudformation Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
coffeescript Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
cpp Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
csharp Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
css Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
dart Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
dockerfile_hadolint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
editorconfig Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
env Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
gherkin Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
github_actions Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
gitleaks feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
go Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
go_modules Bump golang.org/x/crypto in /test/linters/go_modules/go_modules_good (#5018) 2023-12-19 09:08:19 +00:00
google_java_format Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
groovy feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
html Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
java feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
javascript_es Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
javascript_prettier Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
javascript_standard Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
jscpd feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
json Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
jsonc Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
jsx Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
kotlin Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
kotlin_android Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
kubernetes_kubeconform Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
latex Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
lua Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
markdown Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
natural_language feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00
openapi Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
perl Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_builtin Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_phpcs Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_phpstan Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
php_psalm Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
powershell Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
protobuf Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
python_black Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
python_flake8 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
python_isort Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
python_mypy Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
python_pylint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
r Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
raku Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
renovate Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
ruby Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_2015 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_2018 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_2021 Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
rust_clippy Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
scalafmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
shell_shfmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
snakemake_lint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
snakemake_snakefmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
sql Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
sqlfluff Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
states Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
tekton Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terraform_fmt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terraform_terrascan Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terraform_tflint Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
terragrunt Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
tsx Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
typescript_es Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
typescript_standard Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
xml Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
yaml Move tests to the test directory (#4985) 2023-12-15 08:50:35 +00:00
README.md feat!: run linters against the workspace (#5041) 2023-12-24 17:56:15 +01:00

Test Cases

This directory contains test cases that super-linter uses to validate if a particular linter is working.

These test cases focus on how super-linter invokes each linter and their exit codes. We deliberately avoid to verify if the output of a given linter matches the expectations because it's the responsibility of each linter to do so.

Test case format

Each super-linter language should have its own directory, named after the language they refer to.

The name of each test case denotes its nature:

  • Test cases that are expected to pass validation contain the good string in their filename, or path. Example: markdown_good_5.md
  • Test cases that are expected to fail validation contain the bad string in their filename, or path. Example: markdown_bad_5.md

Notes about specific tests

In this section, we explain the peculiarities of certain test cases.

SQL Fluff test cases

From version 0.12.0 SQLFluff requires a dialect to be set, and no longer sets a default. This can be provided as a command-line argument, or a .sqlfluff config file (either in the usualy place for SQLFluff config files, or within the folder containg the SQL).

For SQLFluff we have added a default .sqlfluff config file in its test directory.

OpenAPI test cases

The _bad_ tests are valid .yml/.json but invalid OpenAPI specs. The test extensions used are .ymlopenapi/.jsonopenapi instead of .yml/.json. This is to prevent the YAML and JSON tests from picking them up.

ARM test cases

apiVersions older than 2 years (730 days) are treated as errors by the ARM linter if there is a newer version available.

Ansible test cases

roles/ghe-initialize is a valid Ansible role