textlint was expensive to run because we added every file in the list of
files to lint to FILE_ARRAY_NATURAL_LANGUAGE. In #5041, we mitigated
this issue but lost the ability to run textlint on changed files only.
Given that textlint ignore files for which it doesn't have a plugin
installed, and that we don't currently install additional plugins
besides the default ones to lint markdown files and text files, we let
textlint run on these files only, so we can have the feature to lint
only changed files with this linter as well, back.
- Fix GITHUB_BEFORE_SHA initialization on push events by setting the
correct key.
- Add an additional check against setting GITHUB_BEFORE_SHA to null.
- Run the CI workflow on push events to trigger required status checks
when using the merge queue.
- Run jscpd, gitleaks, textlint against the entire workspace instead of
running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
- Add support to run Checkov against infrastructure as code descriptors
that are in a given (configurable) directory. Defaults to lint the
whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
issues right away with this change.
* Lint Go projects by directory
* Search for Go modules
* Fix test path
* Add test cases and fix command
* Change workdir
* Add a warning about false positives
* fatal instead of warn
* Move tests to the test directory
* Close group on fatal
* Don't fail in test mode
* Exit on errors when running Git
* Skip pulling changes entirely
* Enable pipefail when generating diffs
* Cleanup
* Shallow repo check
* Echo GITHUB_SHA update
* Check if GITHUB_SHA exists before using it
* Move GITHUB_SHA validation to validation script
* Rely on cat-file return code
* Check if DEFAULT_BRANCH exists
* Change dir when checking DEFAULT_BRANCH
* Show git branches
* Don't switch branches
* Check GITHUB_SHA only when needed
* Ensure we have permissions before interacting with the repo
* Remove the DIFF_CMD variable
* Move TEST_CASE_RUN and RUN_LOCAL init up
* Validate if Git repo and if SHA exists
* Move validation function
* Change dir when getting branch names
* Move debug messages up to be less verbose
* Move branch validation in a function
* Fix linting errors
* #4846 fixes regex to be more strict with filenames that container Dockerfile but that are not actually dockerfiles
* Debug print for ANSIBLE_DIRECTORY to see if tests exist
* Trying out the =~ operator
* rm echo cmd
* catch the i-am-containerfile case
* rm extra space
* Remove quotes for shell format linter
---------
Co-authored-by: Zack Koppert <zkoppert@github.com>
* move from github org to super-linter org
* rename to super-linter org
* rename to super-linter org
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter and remove dockerhub reference
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rename org to super-linter
* rm dockerhub section link
* Issue hint when failing to switch back to branch
Switching back to the branch fails with the below error on a shallow checkout:
2023-04-18 21:34:51 [ERROR] Failed to switch back to branch!
2023-04-18 21:34:51 [FATAL] [fatal: reference is not a tree: 3afa21b5f9bef8e81396e3572a598d089e710b96]
With this change, it will look as follows:
2023-04-18 21:34:51 [ERROR] Failed to switch back to branch!
2023-04-18 21:34:51 [INFO] Check that you have the full git history, the checkout is not shallow, etc
2023-04-18 21:34:51 [INFO] See https://github.com/github/super-linter#example-connecting-github-action-workflow
2023-04-18 21:34:51 [FATAL] [fatal: reference is not a tree: 3afa21b5f9bef8e81396e3572a598d089e710b96]
* Update lib/functions/buildFileList.sh
---------
Co-authored-by: Philip Mallegol-Hansen <philip@mallegolhansen.com>
The list-based method breaks down in case of ignored folders:
If .gitignore contains foo/, foo/bar.py is still checked.
Signed-off-by: Christoph Höger <christoph.hoeger@piano.io>
* Lint the whole code base with JSCPD if VALIDATE_ALL_CODEBASE is true
* Use VALIDATE_JSCPD_ALL_CODEBASE
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
* Add support for google-java-format
* adding tests and array
* fix comma
* Update linter.sh
* get it in order
* fix hava version
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Ignore files marked with @generated marker
`@generated` marker is used by certain tools to understand that the
file is generated, so it should be treated differently than a file
written by a human:
* these files do not need to be reformatted,
* diffs in these files are less important,
* and linters should not be invoked on these files.
This PR proposes builtin support for `@generated` marker (and
`@not-generated` marker to mark file as not generated when it
contains `@generated` marker, like `README.md`).
I have not found a standard for a generated file marker, but:
* Facebook [uses `@generated` marker](https://tinyurl.com/fb-generated)
* Phabricator tool which was spawned from Facebook internal tool
[also understands `@generated` marker](https://git.io/JnVHa)
* Cargo inserts `@generated` marker into [generated Cargo.lock files](https://git.io/JnVHP)
Super-linter supports regex includes and excludes, but they are
harder to maintain (each repository needs to be configured) than
patching the tools which generate the files.
My personal story is that I maintain rust-protobuf crate, which
started emitting `@generated` markers [six years ago](https://git.io/JnV5h)
after a request of a Phabricator user.
Test Plan:
Create a test file `test.sh`:
```
echo $a
```
Run:
```
docker run -e RUN_LOCAL=true -v $HOME/tmp/g:/tmp/lint super-linter-test
```
Result is:
```
In /tmp/lint/test.sh line 1:
echo $a
^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
^-- SC2154: a is referenced but not assigned.
^-- SC2086: Double quote to prevent globbing and word splitting.
...
2021-06-22 23:46:16 [ERROR] ERRORS FOUND in BASH:[1]
```
Now add `@generated` to the file and run again:
```
2021-06-22 23:47:13 [NOTICE] All file(s) linted successfully with no errors detected
```
Additionally, add `@not-generated` in addition to `@generated`, and
linter error pops up again.
* cleanup
* remove space
* fix non utf return
* fix non utf return
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
* Allow ansible-lint with git diffs
super-linter checks the path of the file in lib/functions/detectFiles.sh to determine whether the file is an ansible playbook. When VALIDATE_ALL_CODEBASE=true, a list of absolute paths is generated which matches the expected path but when VALIDATE_ALL_CODEBASE=false, a list of relative paths is generated so no files are "detected" as ansible playbooks. This change outputs everything as an absolute path so diffs will also work.
* fix quotes
* Include eval to interpret pipe in cmd substitution
* more quotes
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
