textlint was expensive to run because we added every file in the list of
files to lint to FILE_ARRAY_NATURAL_LANGUAGE. In #5041, we mitigated
this issue but lost the ability to run textlint on changed files only.
Given that textlint ignore files for which it doesn't have a plugin
installed, and that we don't currently install additional plugins
besides the default ones to lint markdown files and text files, we let
textlint run on these files only, so we can have the feature to lint
only changed files with this linter as well, back.
- Fix GITHUB_BEFORE_SHA initialization on push events by setting the
correct key.
- Add an additional check against setting GITHUB_BEFORE_SHA to null.
- Run the CI workflow on push events to trigger required status checks
when using the merge queue.
- Fail if the configuration contains USE_FIND_ALGORITHM=true and
VALIDATE_ALL_CODEBASE=false because this is an unsupported
configuration. Before this change, super-linter silently ignored
VALIDATE_ALL_CODEBASE, leading to potentially surprising behavior.
- Run jscpd, gitleaks, textlint against the entire workspace instead of
running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
- Add support to run Checkov against infrastructure as code descriptors
that are in a given (configurable) directory. Defaults to lint the
whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
issues right away with this change.
- Fail if the installation of a R package fails.
- Install the remotes package once during the image build, and not when we scan
files at runtime.
- Reuse the default R library directory instead of moving it to /home/r-library
* Lint Go projects by directory
* Search for Go modules
* Fix test path
* Add test cases and fix command
* Change workdir
* Add a warning about false positives
* fatal instead of warn
* Move tests to the test directory
* Close group on fatal
* Don't fail in test mode
* Don't write colors and logs on disk if not necessary
* Set color markers
* Fix colors
* Fix linting errors
* Fix linting errors
* Use sudo to access logs
* Exit on errors when running Git
* Skip pulling changes entirely
* Enable pipefail when generating diffs
* Cleanup
* Shallow repo check
* Echo GITHUB_SHA update
* Check if GITHUB_SHA exists before using it
* Move GITHUB_SHA validation to validation script
* Rely on cat-file return code
* Check if DEFAULT_BRANCH exists
* Change dir when checking DEFAULT_BRANCH
* Show git branches
* Don't switch branches
* Check GITHUB_SHA only when needed
* Ensure we have permissions before interacting with the repo
* Remove the DIFF_CMD variable
* Move TEST_CASE_RUN and RUN_LOCAL init up
* Validate if Git repo and if SHA exists
* Move validation function
* Change dir when getting branch names
* Move debug messages up to be less verbose
* Move branch validation in a function
* Fix linting errors
* #4846 fixes regex to be more strict with filenames that container Dockerfile but that are not actually dockerfiles
* Debug print for ANSIBLE_DIRECTORY to see if tests exist
* Trying out the =~ operator
* rm echo cmd
* catch the i-am-containerfile case
* rm extra space
* Remove quotes for shell format linter
---------
Co-authored-by: Zack Koppert <zkoppert@github.com>
* faster linter for cfn-lint and eslint
* workaround shfmt error
* fix xargs interleave large outputs
* parallel gitleaks
* fix exec bit, shfmt, bash linter
* show parallel --citation
* refactor a common interface using named pipe
* add readme for the experimental impl
* fix readme format
* minimize change in worker.sh
* will cite, showed once
* remove junk comment
* explicitly set EXPERIMENTAL_BATCH_WORKER=false
* fix: errors from github/super-linter:v5
* move from github org to super-linter org
* rename to super-linter org
* rename to super-linter org
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter and remove dockerhub reference
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rm dockerhub section link
* no need to run git check-ignore on every file if you are not setting IGNORE_GITIGNORED_FILES
* use id instead of whoami for determining running and discard stderr
* accidently include change from other PR
* remove commented code
---------
Co-authored-by: Zack Koppert <zkoppert@github.com>
* Issue hint when failing to switch back to branch
Switching back to the branch fails with the below error on a shallow checkout:
2023-04-18 21:34:51 [ERROR] Failed to switch back to branch!
2023-04-18 21:34:51 [FATAL] [fatal: reference is not a tree: 3afa21b5f9bef8e81396e3572a598d089e710b96]
With this change, it will look as follows:
2023-04-18 21:34:51 [ERROR] Failed to switch back to branch!
2023-04-18 21:34:51 [INFO] Check that you have the full git history, the checkout is not shallow, etc
2023-04-18 21:34:51 [INFO] See https://github.com/github/super-linter#example-connecting-github-action-workflow
2023-04-18 21:34:51 [FATAL] [fatal: reference is not a tree: 3afa21b5f9bef8e81396e3572a598d089e710b96]
* Update lib/functions/buildFileList.sh
---------
Co-authored-by: Philip Mallegol-Hansen <philip@mallegolhansen.com>
The list-based method breaks down in case of ignored folders:
If .gitignore contains foo/, foo/bar.py is still checked.
Signed-off-by: Christoph Höger <christoph.hoeger@piano.io>
* Fix issues with the container image build
* Fix asl-validator version check
* Disable filename rule when running ktlint tests
* Bump asl-validator from 2.2.1 to 3.0.8 in /dependencies
Bumps [asl-validator](https://github.com/ChristopheBougere/asl-validator) from 2.2.1 to 3.0.8.
- [Release notes](https://github.com/ChristopheBougere/asl-validator/releases)
- [Commits](https://github.com/ChristopheBougere/asl-validator/compare/2.2.1...3.0.8)
---
updated-dependencies:
- dependency-name: asl-validator
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Show errors when using curl to call the status API
* Make the build more reproducible
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat: add support for ssh keys and github.com connections
* refactor: allow github.com setup and update docs
* docs: add note about using ssh_key
* fix: run shfmt
* fix: add language to ssh key fence
* fix: make ssh setup script executable
* fix: gitleaks wins, openssh example removed
* notes
* docs: make the docs a little more clear
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* fix: support tflint relative module references
* chore: add test for relative module imports
* chore: add terraform binary
* chore: move tests
* chore: add newlines to tests
* chore: add newlines to tests
* refactor: move terraform get
* refactor: put terraform get back where it was
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
* Lint the whole code base with JSCPD if VALIDATE_ALL_CODEBASE is true
* Use VALIDATE_JSCPD_ALL_CODEBASE
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
* fix(R linting): try installing the R package before linting R language
* the tool used to lint the R language gives false positives for files inside an R library, which is not installed
* this change tries to naively install the package in the linted directory
Resolves#1910
* fix code
* fixed it
* fixed it
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Add support for google-java-format
* adding tests and array
* fix comma
* Update linter.sh
* get it in order
* fix hava version
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Match AWS states file using "States" key
Matching only on `"Resource": "arn` is too wide and will match also aws json policy files
* Update detectFiles.sh
* spacing
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Ignore files marked with @generated marker
`@generated` marker is used by certain tools to understand that the
file is generated, so it should be treated differently than a file
written by a human:
* these files do not need to be reformatted,
* diffs in these files are less important,
* and linters should not be invoked on these files.
This PR proposes builtin support for `@generated` marker (and
`@not-generated` marker to mark file as not generated when it
contains `@generated` marker, like `README.md`).
I have not found a standard for a generated file marker, but:
* Facebook [uses `@generated` marker](https://tinyurl.com/fb-generated)
* Phabricator tool which was spawned from Facebook internal tool
[also understands `@generated` marker](https://git.io/JnVHa)
* Cargo inserts `@generated` marker into [generated Cargo.lock files](https://git.io/JnVHP)
Super-linter supports regex includes and excludes, but they are
harder to maintain (each repository needs to be configured) than
patching the tools which generate the files.
My personal story is that I maintain rust-protobuf crate, which
started emitting `@generated` markers [six years ago](https://git.io/JnV5h)
after a request of a Phabricator user.
Test Plan:
Create a test file `test.sh`:
```
echo $a
```
Run:
```
docker run -e RUN_LOCAL=true -v $HOME/tmp/g:/tmp/lint super-linter-test
```
Result is:
```
In /tmp/lint/test.sh line 1:
echo $a
^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
^-- SC2154: a is referenced but not assigned.
^-- SC2086: Double quote to prevent globbing and word splitting.
...
2021-06-22 23:46:16 [ERROR] ERRORS FOUND in BASH:[1]
```
Now add `@generated` to the file and run again:
```
2021-06-22 23:47:13 [NOTICE] All file(s) linted successfully with no errors detected
```
Additionally, add `@not-generated` in addition to `@generated`, and
linter error pops up again.
* cleanup
* remove space
* fix non utf return
* fix non utf return
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
