Commit graph

51 commits

Author SHA1 Message Date
Marco Ferrari
78ed3ef5fc
fix: store outputs in the main output directory (#5899)
- Store Super-linter outputs in the main output directory.
- Update README with a few fixes and details about outputs.
- Add missing test cases for when the configuration didn't enable any
  output.
2024-07-28 21:34:40 +02:00
Marco Ferrari
57c86588c3
feat: write github actions step summary (#5867)
This also removes an unneeded textlint configuration file because it
matches with the default one.

Close #5650
2024-07-11 08:01:11 +00:00
Marco Ferrari
12da4973c6
ci: enable dev-dependencies docker build checks (#5871)
- Run Docker build checks when building the dev-dependencies container
  image and when running the test target.
- Run Docker build checks when building the Super-linter container, and
  not just when running the test target.
- Use the correct variable when setting the dev-dependencies container
  user.
2024-07-11 08:08:23 +02:00
Marco Ferrari
bcf8ca82ad
ci: print info about the environment, image size (#5869)
- Add information about container image layers size when printing info
  about the environment.
- Print info about the environment during the CI job.
2024-07-10 14:45:32 +02:00
Marco Ferrari
ce59f5c323
ci: run docker build checks (#5864)
Run docker build checks as part of the test suite.

Ref https://docs.docker.com/reference/build-checks/
2024-07-09 11:22:29 +02:00
Marco Ferrari
94bb3f5563
feat: save super-linter output if requested (#5806)
- New SAVE_SUPER_LINTER_OUTPUT variable. When set to true,
  saves super-linter output to ${DEFAULT_WORKSPACE}/${SUPER_LINTER_OUTPUT_DIRECTORY_NAME}
- New SUPER_LINTER_OUTPUT_DIRECTORY_NAME variable to set the output
  directory name instide the default workspace.

Close #5774
2024-07-01 14:50:52 +02:00
Marco Ferrari
c99ec7784a
fix: don't skip processing ansible_directory pwd (#5790)
Don't skip processing the current item (FILE) before we give
BuildFileArrays the chance to process it as an item to eventually add to
the list of directories to lint with ansible-lint.

Fix #5789

Other related changes

- Add a new make target to open a shell in a Super-linter container.
- Use a fixed path for FILE_ARRAYS_DIRECTORY_PATH so we can verify its
  contents in tests
- Remove redundant ValidateBooleanVariable in buildFileList because we
  already check those variables in valudation.
- Move Ansible directory detection to a function so we can reuse it.
- Add missing exports for global configuration variables.
- Remove unused LOG_XXXX variables from tests. These should have been
  deleted when we moved log variables to log.sh
2024-06-19 16:58:11 +00:00
Marco Ferrari
2bb8a0a3e7
fix: no full git validation when ignoring files (#5599)
- Don't require Git SHA and branch validation when
  IGNORE_GITIGNORED_FILES=true because we only need to validate that the
  workspace is a Git repository in this case.

Fix #5383
2024-04-30 17:58:11 +02:00
Marco Ferrari
ff425b9a7d Revert "fix: no full git validation when ignoring files"
This reverts commit 5b0c248f9c.
2024-04-30 10:14:12 +00:00
Marco Ferrari
5b0c248f9c fix: no full git validation when ignoring files
- Don't require Git SHA and branch validation when
  IGNORE_GITIGNORED_FILES=true because we only need to validate that the
  workspace is a Git repository in this case.

Fix #5383
2024-04-30 10:11:14 +00:00
Marco Ferrari
8f405c1a9c
fix: handle initial commit (#5534)
Close #5453
2024-04-18 06:48:55 +00:00
Benjamin Wuethrich
95aabd4cfa
feat(bash-exec): add option to ignore shell library files (#5254)
Introduce a new configuration variable, BASH_EXEC_IGNORE_LIBRARIES. If
set to true, the behaviour of bash-exec is modified: if a shell file has
a file extension and no shebang line, it is ignored, i.e., allowed to be
non-executable. This allows files that are only every sourced from other
shell files, acting as libraries and not executables, to have no
executable bit set without failing the bash-exec linter.
2024-02-27 18:17:22 +00:00
Marco Ferrari
ed27c0146d
fix: github actions debug logging (#5288)
Enable debug logging when one (or more) of the following conditions is
true:

- ACTIONS_RUNNER_DEBUG is set to true
- ACTIONS_STEPS_DEBUG is set to true
- RUNNER_DEBUG is set to 1
2024-02-20 17:27:06 +00:00
Marco Ferrari
091eaa71e3
feat: show error output when info is disabled (#5251)
In case of linting errors, print stdout and stderr (if present)
at the ERROR level if users set LOG_LEVEL to NOTICE to avoid
failures without any explanation.
2024-02-10 10:16:31 +01:00
Marco Ferrari
5b5e54ad5c
fix: initialize terrascan at runtime (#5246)
Terrascan runs initialization anyway when scanning files, so there's no
point in running it at build time. Also, this works around a Terrascan
bug that caused it to fail its initialization if $HOME/.terrascan
directory is not present. This happens on GitHub Actions because it
configures a $HOME directory that is different from ours.
2024-02-09 22:57:01 +00:00
Marco Ferrari
6fd6830fb4
fix: initialize GitHub domain variable (#5216)
- Initialize GITHUB_DOMAIN as documented in the README
- Configure other URLs according to GITHUB_DOMAIN
- Automatically fetch SSH key fingerprints for GITHUB_DOMAINS
2024-02-09 18:45:44 +00:00
Marco Ferrari
83eca1df43
fix: unset the log_level variable (#5249)
- Super-linter uses the LOG_LEVEL variable to let the user
  configure the desired log level. Checkov and Renovate use a variable
  with the same name for the same purpose, but accept a
  different set of values, and exit with an error if it gets an unknown
  value for that variable.
- Refactor the VERBOSE log level to the more commonly used INFO.
  Configuration validation will warn users if they use VERBOSE and
  instruct them to use INFO instead. This is not a breaking change
  because super-linter falls back on INFO if VERBOSE is set.
- Remove the TRACE log level because we rarely used it. As with VERBOSE,
  configuration validation will warn the user. Fall back to DEBUG if the
  user configured LOG_LEVEL to VERBOSE.

Close #5217
2024-02-09 18:43:58 +00:00
Marco Ferrari
ace79ca403
build: ignore changelog when linting codebase (#5205)
CHANGELOG.md is automatically generated by the release workflow.
Ignore it when linting the codebase because it might not pass textlint
validation because of lowercase terms.
2024-01-31 10:31:06 +01:00
Marco Ferrari
99e41ce451
feat: run linters in parallel (#5177) 2024-01-30 19:24:55 +00:00
Marco Ferrari
9bab4a90e8
chore: simplify updateSSL (#5130)
Run command directly instead of checking their exit code afterwards.
2024-01-30 09:05:47 +01:00
Kin Fai Tse
3a21ed5bdf
ci: implement more smoke tests (#4955)
- Non-default workspace
- Git-related flags
2024-01-16 10:23:17 +01:00
Marco Ferrari
3a5617235c
feat!: deprecate error_on_missing_exec_bit (#5120)
Deprecate the ERROR_ON_MISSING_EXEC_BIT variable to remove a confusing
corner case, and to align the BASH_EXEC check to how super-linter
handles other linters.
2024-01-10 22:54:13 +00:00
Marco Ferrari
df911171c4
build: python venvs and npm in dedicated stages (#5078)
Move the following tasks to dedicated stages so we can run these steps
in parallel with other stages:

- Build Python virtual environments
- Install npm packages
- Build clang-format
- Download and install TFlint plugins

Add missing target stage when building the container image
2024-01-10 11:09:53 +01:00
Marco Ferrari
e62b382bf0
feat: don't inspect files if not needed (#5094)
Don't run potentially expensive I/O operations to check file types if
we're not going to analyze them anyway.
2024-01-06 18:39:39 +01:00
Marco Ferrari
cf2038d903
fix: fix GITHUB_BEFORE_SHA initalization for push (#5098) 2024-01-05 23:07:39 +01:00
Marco Ferrari
901a901655
fix: add missing checkov configuration file (#5090)
Add a default (empty) configuration file for Checkov and a smoke test to
ensure that we catch the case where a default configuration file is not
present, and it should be.
2024-01-02 18:03:30 +01:00
Marco Ferrari
11b70102c3
feat!: run linters against the workspace (#5041)
- Run jscpd, gitleaks, textlint  against the entire workspace instead of
  running them over single files, one by one.
- Implement a warning function for deprecated variables.
- Deprecate the VALIDATE_JSCPD_ALL_CODEBASE variable.
- Remove duplicate configuration files when they are the same as the
  ones we provide in TEMPLATES.
- Add a missing tests for ansible-lint.
- Move ANSIBLE_DIRECTORY configuration when running tests in
  buildFileList, where similar configs are.
- Simplify ansible-lint test cases to include only what's necessary, and
  not an entire set of roles, playbooks, and inventory.
- Write instructions about major upgrades in the upgrade guide.
2023-12-24 17:56:15 +01:00
Marco Ferrari
b214a59ca7
fix: fix file list when looking for changes (#5044)
- Fix the file diff function on push events.
- Implement a test for the file diff function
2023-12-23 19:33:53 +01:00
Marco Ferrari
9d7268fb99
feat: add support for checkov to lint iac files (#4925)
- Add support to run Checkov against infrastructure as code descriptors
  that are in a given (configurable) directory. Defaults to lint the
  whole workspace.
- Establish a baseline for our own codebase so we don't have to fix
  issues right away with this change.
2023-12-22 13:22:15 +01:00
Marco Ferrari
641c65a8c4
ci: configure release-please dry-run and changelog (#5039)
- Implement a job to preview the release notes
- Include build, ci, and dependency updates
- Add emoji to section headings to match the existing release notes
- Add documentation about how to run release-please from the CLI
2023-12-21 15:03:14 +01:00
Marco Ferrari
93b5ede1e8
ci: configure release-please (#5016)
- Configure release-please to automatically create pull requests and
  releases.
- Run release-please and tag update in the CD workflow.
- Update Git tags pointing to latest, major.minor, and major versions as
  part of the CD workflow.
- Remove workflows (draft-release, release) that are not necessary
  anymore, and related configuration files.
- Handle automatic updates to README.md, action.yml, and action.yaml
- Mount .github to /tmp/lint/.github so super-linter finds config files,
  and the GitHub Actions to lint.
2023-12-20 14:58:25 +01:00
Marco Ferrari
9db632f0e1
ci: configure commitlint (#5014)
- Check if the PR contains a single commit, and fail otherwise.
- Enable commitlint to check if commits adhere to the
  conventialcommits.org spec.
- Update the the pull request template to point to the conventional
  commit spec.
- Update the dependabot configuration to add the "build(...)" prefix to
  commits.
2023-12-19 19:37:16 +01:00
Marco Ferrari
22b8624f61
Simplify container image build (#4962) 2023-12-15 11:59:36 +01:00
Marco Ferrari
e6cf8d3845
Move tests to the test directory (#4985)
* Move tests to the test directory

* Fix linting errors

* Add states back

* Add xml back
2023-12-15 08:50:35 +00:00
Marco Ferrari
7150e1f8b0
Group log output on GitHub Actions (#4961) 2023-12-12 20:57:15 +01:00
Marco Ferrari
ac4b767bd7
Reduce duplication in CI and CD workflows (#4982)
* Reduce duplication in CI and CD workflows

* Fix indentation in README

* Load token from file

* Fix instructions

* Ignore test leftovers
2023-12-12 18:53:48 +00:00
Marco Ferrari
7a21f934b4
Don't forcefully validate Git repos if not needed (#4953)
* Fix find when linting non-git repos

* Remove uses

* Move safe.directory config after we set GITHUB_WORKSPACE

* Fix Git validation check

* Move 'find' test runner to makefile

* Validate vars before validating Git repo

* Validate Git repo even when VALIDATE_ALL_CODEBASE=false

* Initialize GITHUB_SHA when running locally

* Initialize safe git dirs

* Check git safe dirs error code

* Fix log level color marker

* Fix linting errors

* Fix change dir command

* Fix linting errors

* Set default branch
2023-12-07 19:07:22 +00:00
Marco Ferrari
9869638131
Validate container image labels (#4926)
* Validate labels and avoid busting the cache

* Fix validation

* Validate non-empty labels

* Add build date back

* Don't set build date

* Simplify validation script

* Enable build cache

* Setup buildx

* Dynamically set build revision and version

* Remove leftover

* Disable cache

* Add build date back

* Add build date back

* Fix linting errors

* Add checks

* Get head SHA

* Fix linting errors

* Handle merge_group
2023-12-07 14:18:47 +00:00
Marco Ferrari
bb8ba1ad7b
Run the info target when running the test suite (#4799) 2023-10-31 17:45:19 +00:00
Marco Ferrari
dfedd306f3
Use the same image tags in CI and CD (#4778)
* Use the same image tags in CI and CD

* Define image tag once per workflow

* Fix yq tag

* Don't quote yq to avoid literal interpretation

* yq version

* Fix yq tag

* Reduce duplication

* Fix workflow env var

* Don't remove an already removed container
2023-10-19 21:03:14 +00:00
Zack Koppert
da4dd08cd6
Follow up with several more documentation and automation renames for the org move from github to super-linter (#4543)
* move from github org to super-linter org

* rename to super-linter org

* rename to super-linter org

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter and remove dockerhub reference

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rename org to super-linter

* rm dockerhub section link
2023-08-17 15:12:11 +00:00
Brett Logan
4d34146719 Fix container image names in InSpec tests
Signed-off-by: Brett Logan <lindluni@github.com>
2023-01-05 09:46:51 -05:00
Brett Logan
3ed738059a Update build command
Signed-off-by: Brett Logan <lindluni@github.com>
2023-01-04 20:17:46 -05:00
Brett Logan
2b8f626566 Fix deps and secret injection
Signed-off-by: Brett Logan <lindluni@github.com>
2023-01-04 20:17:46 -05:00
Brett Logan
5224656969 Push scripts into standalone files
Pushes inline scripts in the Dockerfile into standalone
scripts and authenticates requests to GitHub using a
Personal Access Token to reduce build flakiness due
to GitHub's abuse and ratelimiting due to unauthenticated
reuests.

Signed-off-by: Brett Logan <lindluni@github.com>
2023-01-04 20:17:46 -05:00
Marco Ferrari
c0f47d0eff
De-duplicate workflows (#2220)
* De-duplicate deploy-DEV workflows

* Add $

* Fix build target

* Fix standard target

* Consolidate workflows in a single one

* more clean

* add changes

* fix spaces

* fix release

Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
2022-01-26 09:17:59 -06:00
Lukas Gravley
2c135d4a17
adding rules file (#1644)
* adding rules file

* fix naming

* cleanup

* make smart

* fixing the make

* adding template

* white space

* not sure how i got windows

* found the space
2021-06-09 11:53:11 -05:00
Lukas Gravley
83badbc49c
Image 4.0 (#1565)
* adding new one...

* adding automation

* linter

* pull apart for faster builds

* maybe

* update automation

* fix that

* prime is smart

* fixed it

* quotes

* fix build image

* inpec

* typo

* forgot backslash

* fixed hairbrain logic

* this should help cleanup

* i hate r

* order

* maybe fix r

* maybe fix r

* update readme

* remove spaces

* update
2021-05-25 10:14:43 -05:00
Lukas Gravley
5d2ea81f00
Cpp (#1492)
* adding cpp

* adding cpp

* tests

* make test better

* fix test
2021-05-04 13:24:41 -05:00
Marco Ferrari
0c8db849aa
Implement a test suite with InSpec (#1466)
* Implement a test suite with InSpec

* make ruby happy

* adding file

* Fix inspec and add make target

* Run inspec

* adding binaries

* make ruby happy

* fix linter order

* cleanup

* adding version check

* adding notes

* cleanup

* cleanup

* fixed r test

* fixed copy paste

* dynamic tests

* fix hash

* fix notation

* docker ps

* Fix makefile

* Fix makefile

Co-authored-by: Admiral Awkbar <admiralawkbar@github.com>
2021-04-30 11:29:15 -05:00