Commit graph

153 commits

Author SHA1 Message Date
Marco Ferrari
9bab4a90e8
chore: simplify updateSSL (#5130)
Run command directly instead of checking their exit code afterwards.
2024-01-30 09:05:47 +01:00
Marco Ferrari
05009f2816
chore: simplify error code checks (#5131)
Instead of loading the error code and checking it, shorten checks by
running the command directly.
2024-01-30 08:48:24 +01:00
Marco Ferrari
f1873b0374
feat: don't check bake files with terragrunt (#5178)
Exclude the default docker bake files in HCL format from the list of files to
lint with Terragrunt.

Note that Docker bake supports defining arbitrary names for
configuration files, so this commit doesn't cover the scenario that a
user defined an arbitrary name for a Docker bake configuration file in
HCL format. Example: example-bake-config-file.hcl will be included in
the list of files to lint with Terragrunt.

Ref (default bake names):
https://docs.docker.com/build/bake/reference/#file-format
2024-01-29 13:51:28 +01:00
Marco Ferrari
5219feefab
fix: simplify worker and linterVersions (#5123)
- Remove the SKIP_FLAG variable and check for the length of the arrays
  of files to lint directly.
- Remove the LIST_FILES variable, and use the FILE_ARRAY variable
  directly.
- Remove the corner case for RENOVATE because renovate-config-validator
  supports passing the path to the file to lint using an argument as the
  default case does.
- Remove the corner case for ANSIBLE not having 'bad' tests because it
  has them now.
- Set TF_DATA_DIR to avoid any modification to any existing Terraform
  data directory that users might have in their workspace.
- Aggregate GO_MODULES and ANSIBLE corner cases because they are the
  same.
- Remove the corner case for ANSIBLE to add a trailing slash to
  TEST_CASE_FOLDER (similar reason as the previous point about ANSIBLE
  corner case).
- Simplify log messages by removing color markers because they are
  already handled in log.sh.
- Simplify linterVersions by removing redundant checks and functions.
- Avoid printing debug logs in the versions file.
2024-01-15 19:37:45 +01:00
Marco Ferrari
4a28fc5e73
feat: validate variables and simplify lowercase (#5128)
- Validate variables representing boolean values.
- Group global variables in the same sections.
- Declare variables as lowercase with the 'declare -l' shell builtin for
  more clarity.
2024-01-11 20:30:00 +00:00
Marco Ferrari
5a2056d77a
fix: handle log messages in CheckFileType (#5117)
- Honor SUPPRESS_FILE_TYPE_WARN when printing messages in the
  CheckFileType function.
- Reduce duplication when handling log messages in the CheckFileType
  function.
- Don't add files to the array of files to lint with JSCPD because we
  lint the whole codebase with JSCPD anyway.
2024-01-11 20:44:16 +01:00
Marco Ferrari
3a5617235c
feat!: deprecate error_on_missing_exec_bit (#5120)
Deprecate the ERROR_ON_MISSING_EXEC_BIT variable to remove a confusing
corner case, and to align the BASH_EXEC check to how super-linter
handles other linters.
2024-01-10 22:54:13 +00:00
Marco Ferrari
3a784fcfd6
fix: simplify file status checks (#5119)
- Simplify file status checks to assume that a file should be linted
  correctly unless we are running in test mode.
- Avoid the corner case of files having the 'bad' string as part of
  their file name to be wrongly assumed to fail linting.
- Move FILE_STATUS initialization where it's needed, after running the
  linters.
2024-01-10 12:35:05 +01:00
Marco Ferrari
6f70adee89
fix: enable linting changed files with textlint (#5100)
textlint was expensive to run because we added every file in the list of
files to lint to FILE_ARRAY_NATURAL_LANGUAGE. In #5041, we mitigated
this issue but lost the ability to run textlint on changed files only.
Given that textlint ignore files for which it doesn't have a plugin
installed, and that we don't currently install additional plugins
besides the default ones to lint markdown files and text files, we let
textlint run on these files only, so we can have the feature to lint
only changed files with this linter as well, back.
2024-01-10 09:49:04 +01:00
Marco Ferrari
e62b382bf0
feat: don't inspect files if not needed (#5094)
Don't run potentially expensive I/O operations to check file types if
we're not going to analyze them anyway.
2024-01-06 18:39:39 +01:00
Marco Ferrari
cf2038d903
fix: fix GITHUB_BEFORE_SHA initalization for push (#5098) 2024-01-05 23:07:39 +01:00
Marco Ferrari
1d5ed2c386
fix: fix GITHUB_BEFORE_SHA diff on push events (#5096)
- Fix GITHUB_BEFORE_SHA initialization on push events by setting the
  correct key.
- Add an additional check against setting GITHUB_BEFORE_SHA to null.
- Run the CI workflow on push events to trigger required status checks
  when using the merge queue.
2024-01-04 22:54:47 +01:00
Marco Ferrari
9257ba8af3
build: use embedded checkstyle configuration files (#5089)
checkstyle embeds sun_checks.xml and google_checks.xml so we can use
them instead of shipping our own.
2024-01-02 18:47:45 +01:00
Marco Ferrari
ededa44d36
fix: fail when validating as expected (#5076)
Fail when USE_FIND_ALGORITHM is true and VALIDATE_ALL_CODEBASE is false
as expected.
2023-12-30 23:02:25 +01:00
Marco Ferrari
69a45e022d
feat!: validate configuration when using find (#5045)
- Fail if the configuration contains USE_FIND_ALGORITHM=true and
  VALIDATE_ALL_CODEBASE=false because this is an unsupported
  configuration. Before this change, super-linter silently ignored
  VALIDATE_ALL_CODEBASE, leading to potentially surprising behavior.
2023-12-25 11:15:42 +01:00
dependabot[bot]
20d12b4c7a
deps(java): bump com.pinterest.ktlint:ktlint-cli in /dependencies/ktlint (#5050)
Remove KOTLIN_ANDROID because ktlint handles that case by reading
a configuration setting in .editorconfig.

Bumps [com.pinterest.ktlint:ktlint-cli](https://github.com/pinterest/ktlint) from 0.47.1 to 1.1.0.
- [Release notes](https://github.com/pinterest/ktlint/releases)
- [Changelog](https://github.com/pinterest/ktlint/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pinterest/ktlint/compare/0.47.1...1.1.0)

---
updated-dependencies:
- dependency-name: com.pinterest.ktlint:ktlint-cli
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-24 19:33:08 +01:00
Marco Ferrari
11b70102c3
feat!: run linters against the workspace (#5041)
- Run jscpd, gitleaks, textlint  against the entire workspace instead of
  running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
  ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
  buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
  not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
2023-12-24 17:56:15 +01:00
Marco Ferrari
b214a59ca7
fix: fix file list when looking for changes (#5044)
- Fix the file diff function on push events.
- Implement a test for the file diff function
2023-12-23 19:33:53 +01:00
Marco Ferrari
9d7268fb99
feat: add support for checkov to lint iac files (#4925)
- Add support to run Checkov against infrastructure as code descriptors
  that are in a given (configurable) directory. Defaults to lint the
  whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
  issues right away with this change.
2023-12-22 13:22:15 +01:00
Marco Ferrari
60983d395f
fix: fail if r package installation fails (#4994)
- Fail if the installation of a R package fails.
- Install the remotes package once during the image build, and not when we scan
  files at runtime.
- Reuse the default R library directory instead of moving it to /home/r-library
2023-12-21 20:52:57 +01:00
Marco Ferrari
30317804b1
Lint Go modules (#4984)
* Lint Go projects by directory

* Search for Go modules

* Fix test path

* Add test cases and fix command

* Change workdir

* Add a warning about false positives

* fatal instead of warn

* Move tests to the test directory

* Close group on fatal

* Don't fail in test mode
2023-12-16 08:30:33 +00:00
Marco Ferrari
22564fb65c
Switch to tflint image because tflint-bundle is deprecated (#4990)
* Switch to tflint image because tflint-bundle is deprecated

* Fix version string

* Don't copy plugins

* Don't copy plugins

* Set terraform log vars globally

* Fix tflint error
2023-12-15 09:29:34 +00:00
Marco Ferrari
e6cf8d3845
Move tests to the test directory (#4985)
* Move tests to the test directory

* Fix linting errors

* Add states back

* Add xml back
2023-12-15 08:50:35 +00:00
Marco Ferrari
7150e1f8b0
Group log output on GitHub Actions (#4961) 2023-12-12 20:57:15 +01:00
Marco Ferrari
7a21f934b4
Don't forcefully validate Git repos if not needed (#4953)
* Fix find when linting non-git repos

* Remove uses

* Move safe.directory config after we set GITHUB_WORKSPACE

* Fix Git validation check

* Move 'find' test runner to makefile

* Validate vars before validating Git repo

* Validate Git repo even when VALIDATE_ALL_CODEBASE=false

* Initialize GITHUB_SHA when running locally

* Initialize safe git dirs

* Check git safe dirs error code

* Fix log level color marker

* Fix linting errors

* Fix change dir command

* Fix linting errors

* Set default branch
2023-12-07 19:07:22 +00:00
Marco Ferrari
eb688a090c
Change directory when checking ignored files (#4933)
* Change dir when checking ignored files

* Set to GITHUB_WORKSPACE
2023-12-06 07:32:46 +00:00
Marco Ferrari
879672e936
Don't write colors and logs on disk if not necessary (#4934)
* Don't write colors and logs on disk if not necessary

* Set color markers

* Fix colors

* Fix linting errors

* Fix linting errors

* Use sudo to access logs
2023-12-05 08:04:13 +00:00
Marco Ferrari
5a8805dc4f
Exit on errors when running Git (#4889)
* Exit on errors when running Git

* Skip pulling changes entirely

* Enable pipefail when generating diffs

* Cleanup

* Shallow repo check

* Echo GITHUB_SHA update

* Check if GITHUB_SHA exists before using it

* Move GITHUB_SHA validation to validation script

* Rely on cat-file return code

* Check if DEFAULT_BRANCH exists

* Change dir when checking DEFAULT_BRANCH

* Show git branches

* Don't switch branches

* Check GITHUB_SHA only when needed

* Ensure we have permissions before interacting with the repo

* Remove the DIFF_CMD variable

* Move TEST_CASE_RUN and RUN_LOCAL init up

* Validate if Git repo and if SHA exists

* Move validation function

* Change dir when getting branch names

* Move debug messages up to be less verbose

* Move branch validation in a function

* Fix linting errors
2023-12-04 09:47:49 +00:00
Marco Ferrari
a8150b40c8
Fix ts-standard configuration (#4932)
* Fix ts-standard configuration

* Fix default TYPESCRIPT_STANDARD_TSCONFIG_FILE assignment
2023-12-02 08:35:41 +00:00
Chris McIntosh
6c7dc51cf8
Fixes Regex to match only container files more strictly (#4867)
* #4846 fixes regex to be more strict with filenames that container Dockerfile but that are not actually dockerfiles

* Debug print for ANSIBLE_DIRECTORY to see if tests exist

* Trying out the =~ operator

* rm echo cmd

* catch the i-am-containerfile case

* rm extra space

* Remove quotes for shell format linter

---------

Co-authored-by: Zack Koppert <zkoppert@github.com>
2023-11-15 05:44:46 +00:00
Zack Koppert
35c3fa445c
update GitHub SSH fingerprints (#4826) 2023-11-01 12:09:09 -07:00
Kin Fai Tse
c3ac3aa5d9
Batched & parallel support for cfn-lint, eslint, gitleaks (#4088)
* faster linter for cfn-lint and eslint

* workaround shfmt error

* fix xargs interleave large outputs

* parallel gitleaks

* fix exec bit, shfmt, bash linter

* show parallel --citation

* refactor a common interface using named pipe

* add readme for the experimental impl

* fix readme format

* minimize change in worker.sh

* will cite, showed once

* remove junk comment

* explicitly set EXPERIMENTAL_BATCH_WORKER=false

* fix: errors from github/super-linter:v5
2023-10-31 00:13:33 +00:00
cicdguy
94279fc4ed
No fatal error on R package installation (#4705)
Co-authored-by: Zack Koppert <zkoppert@github.com>
2023-10-15 05:10:37 +00:00
Masaya Suzuki
50d462e0ec
renovate-config-validator: validate shareable config preset (#4674)
* renovate-config-validator: validate default.json

* Add RENOVATE_SHAREABLE_CONFIG_PRESET_FILE

* Allow specifying multiple additional files with a environment variable

* Add break
2023-09-29 20:25:37 +00:00
Masaya Suzuki
3703f392e8
Add renovate-config-validator (#4643) 2023-09-12 16:58:09 +00:00
Zack Koppert
da4dd08cd6
Follow up with several more documentation and automation renames for the org move from github to super-linter (#4543)
* move from github org to super-linter org

* rename to super-linter org

* rename to super-linter org

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter and remove dockerhub reference

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rm dockerhub section link
2023-08-17 15:12:11 +00:00
Jan Hentschel
ec17c93135
Handle tflint deprecations (#4451)
Solve the deprecations introduced in tflint, which are getting removed
in v0.47.0.

Co-authored-by: Philip Mallegol-Hansen <philip@mallegolhansen.com>
2023-07-09 09:56:47 -07:00
Stephen Abbene
35d5a3e9fb
Running superlinter as a user not defined in /etc/passwd causes error messages (#4351)
* no need to run git check-ignore on every file if you are not setting IGNORE_GITIGNORED_FILES

* use id instead of whoami for determining running and discard stderr

* accidently include change from other PR

* remove commented code

---------

Co-authored-by: Zack Koppert <zkoppert@github.com>
2023-06-20 19:57:38 +00:00
EJ Etherington
dea6e50722
Update Kuberentes File Detection (#4375)
Co-authored-by: Zack Koppert <zkoppert@github.com>
2023-06-20 12:27:03 -07:00
Stephen Abbene
6c1fcb284a
no need to run git check-ignore on every file if you are not setting IGNORE_GITIGNORED_FILES (#4348) 2023-06-09 09:33:05 -07:00
Bert Roos
ef76367358
Issue hint when failing to switch back to branch (#4107)
* Issue hint when failing to switch back to branch

Switching back to the branch fails with the below error on a shallow checkout:

2023-04-18 21:34:51 [ERROR]   Failed to switch back to branch!
2023-04-18 21:34:51 [FATAL]   [fatal: reference is not a tree: 3afa21b5f9bef8e81396e3572a598d089e710b96]

With this change, it will look as follows:

2023-04-18 21:34:51 [ERROR]   Failed to switch back to branch!
2023-04-18 21:34:51 [INFO]   Check that you have the full git history, the checkout is not shallow, etc
2023-04-18 21:34:51 [INFO]  See https://github.com/github/super-linter#example-connecting-github-action-workflow
2023-04-18 21:34:51 [FATAL]   [fatal: reference is not a tree: 3afa21b5f9bef8e81396e3572a598d089e710b96]

* Update lib/functions/buildFileList.sh

---------

Co-authored-by: Philip Mallegol-Hansen <philip@mallegolhansen.com>
2023-05-23 00:30:16 +00:00
Chris Kuhl
6bde96b30c
fix: Correct typo in env. var. name (#4153) 2023-05-22 23:08:35 +00:00
Janis Voigtländer
92ee250684 Textlint should check every file 2023-04-16 22:08:23 -04:00
tonyk7440
516b3f81b1 #3501 edit code to debug from review 2023-04-10 19:29:18 -04:00
Tony Kenny
e8d8e662c8 #3473 redirect more output 2023-04-10 19:29:18 -04:00
tonyk7440
c525f0b39f #3473 try alternate install cmd 2023-04-10 19:29:18 -04:00
Brett Logan
c681b09ebe Specify version flag for kubeconform
Signed-off-by: Brett Logan <lindluni@github.com>
2023-01-16 21:14:13 -05:00
Rui Chen
e6445c358f replace kubeval with kubeconform
Signed-off-by: Rui Chen <rui@chenrui.dev>
Signed-off-by: Brett Logan <lindluni@github.com>
2023-01-16 21:14:13 -05:00
Marco Ferrari
81c370a611
Validate if we the test suite runs both good and bad tests (#3811)
Also, remove the ansible-lint configuration file because we can rely on the default one
2023-01-11 17:24:48 +01:00
Jon Hamm
aad87c569b add jenkinsfile to groovy files 2023-01-05 09:52:29 -05:00