Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.
Fix#5789
Other related changes
- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
deleted when we moved log variables to log.sh
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
- Super-linter uses the LOG_LEVEL variable to let the user
configure the desired log level. Checkov and Renovate use a variable
with the same name for the same purpose, but accept a
different set of values, and exit with an error if it gets an unknown
value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
Configuration validation will warn users if they use VERBOSE and
instruct them to use INFO instead. This is not a breaking change
because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
configuration validation will warn the user. Fall back to DEBUG if the
user configured LOG_LEVEL to VERBOSE.
Close#5217
- Validate variables representing boolean values.
- Group global variables in the same sections.
- Declare variables as lowercase with the 'declare -l' shell builtin for
more clarity.
- Honor SUPPRESS_FILE_TYPE_WARN when printing messages in the
CheckFileType function.
- Reduce duplication when handling log messages in the CheckFileType
function.
- Don't add files to the array of files to lint with JSCPD because we
lint the whole codebase with JSCPD anyway.
- Fail if the installation of a R package fails.
- Install the remotes package once during the image build, and not when we scan
files at runtime.
- Reuse the default R library directory instead of moving it to /home/r-library
* fix(R linting): try installing the R package before linting R language
* the tool used to lint the R language gives false positives for files inside an R library, which is not installed
* this change tries to naively install the package in the linted directory
Resolves#1910
* fix code
* fixed it
* fixed it
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Match AWS states file using "States" key
Matching only on `"Resource": "arn` is too wide and will match also aws json policy files
* Update detectFiles.sh
* spacing
Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
* Ignore files marked with @generated marker
`@generated` marker is used by certain tools to understand that the
file is generated, so it should be treated differently than a file
written by a human:
* these files do not need to be reformatted,
* diffs in these files are less important,
* and linters should not be invoked on these files.
This PR proposes builtin support for `@generated` marker (and
`@not-generated` marker to mark file as not generated when it
contains `@generated` marker, like `README.md`).
I have not found a standard for a generated file marker, but:
* Facebook [uses `@generated` marker](https://tinyurl.com/fb-generated)
* Phabricator tool which was spawned from Facebook internal tool
[also understands `@generated` marker](https://git.io/JnVHa)
* Cargo inserts `@generated` marker into [generated Cargo.lock files](https://git.io/JnVHP)
Super-linter supports regex includes and excludes, but they are
harder to maintain (each repository needs to be configured) than
patching the tools which generate the files.
My personal story is that I maintain rust-protobuf crate, which
started emitting `@generated` markers [six years ago](https://git.io/JnV5h)
after a request of a Phabricator user.
Test Plan:
Create a test file `test.sh`:
```
echo $a
```
Run:
```
docker run -e RUN_LOCAL=true -v $HOME/tmp/g:/tmp/lint super-linter-test
```
Result is:
```
In /tmp/lint/test.sh line 1:
echo $a
^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
^-- SC2154: a is referenced but not assigned.
^-- SC2086: Double quote to prevent globbing and word splitting.
...
2021-06-22 23:46:16 [ERROR] ERRORS FOUND in BASH:[1]
```
Now add `@generated` to the file and run again:
```
2021-06-22 23:47:13 [NOTICE] All file(s) linted successfully with no errors detected
```
Additionally, add `@not-generated` in addition to `@generated`, and
linter error pops up again.
* cleanup
* remove space
* fix non utf return
* fix non utf return
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
* Change log level from warn to debug
* Move 'Linting x file' to be printed only if files are found
Co-authored-by: Lukas Gravley <admiralawkbar@github.com>
