Add more expected reports validations

This commit is contained in:
Victor Martinez 2020-07-19 20:38:18 +01:00
parent 9d68381696
commit 0bc699a72f
No known key found for this signature in database
GPG key ID: 4058B656AD58C4F5
15 changed files with 123 additions and 2 deletions

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - arm_bad_1.json
---
message: \nValidating arm\arm_bad_1.json\n deploymentTemplate\n [+] adminUsername Should Not Be A Literal (109 ms)\n [+] apiVersions Should Be Recent (121 ms)\n [+] artifacts parameter (13 ms)\n [+] DependsOn Best Practices (7 ms)\n [+] Deployment Resources Must Not Be Debug (6 ms)\n [+] DeploymentTemplate Schema Is Correct (4 ms)\n [+] Dynamic Variable References Should Not Use Concat (8 ms)\n [+] IDs Should Be Derived From ResourceIDs (40 ms)\n [?] Location Should Not Be Hardcoded (5 ms)\n Skipped because IsMainTemplate was missing\n\n [+] ManagedIdentityExtension must not be used (54 ms)\n [+] Min And Max Value Are Numbers (6 ms)\n [+] Outputs Must Not Contain Secrets (15 ms)\n [+] Parameters Must Be Referenced (9 ms)\n [+] Parameters Property Must Exist (2 ms)\n [+] providers apiVersions Is Not Permitted (2 ms)\n [+] ResourceIds should not contain (14 ms)\n [+] Resources Should Have Location (3 ms)\n [+] Secure String Parameters Cannot Have Default (4 ms)\n [-] Template Should Not Contain Blanks (33 ms)\n Empty property [] found on line 28 Index 1331\n Empty property "" found on line 3 Index 121\n\n [-] Variables Must Be Referenced (35 ms)\n Unreferenced variable Network.Location\n Unreferenced variable Test.Blank.Variable\n\n [+] Virtual Machines Should Not Be Preview (17 ms)\n [+] VM Images Should Use Latest Version (1 ms)\n [+] VM Size Should Be A Parameter (15 ms)\n \n
...
ok 2 - arm_good_1.json

View file

@ -0,0 +1,22 @@
TAP version 13
1..8
not ok 1 - cfn_bad_1.json
---
message: E3001 Invalid or unsupported Type AWS Route53 HostedZonee for resource DNS in us-east-1\n/tmp/lint/.automation/test/cfn/cfn_bad_1.json 6 10\n
...
not ok 2 - cfn_bad_2.yaml
---
message: E3001 Invalid or unsupported Type AWS Route53 HostedZonee for resource DNS in us-east-1\n/tmp/lint/.automation/test/cfn/cfn_bad_2.yaml 5 5\n
...
not ok 3 - cfn_bad_3.json
---
message: E3002 Invalid Property Resources/myDNSRecord/Properties/Ttl\n/tmp/lint/.automation/test/cfn/cfn_bad_3.json 11 13\n
...
not ok 4 - cfn_bad_4.yaml
---
message: E3002 Invalid Property Resources/myDNSRecord/Properties/Ttl\n/tmp/lint/.automation/test/cfn/cfn_bad_4.yaml 9 7\n
...
ok 5 - cfn_good_1.json
ok 6 - cfn_good_2.yaml
ok 7 - cfn_good_3.json
ok 8 - cfn_good_4.yaml

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - clojure_bad_1.clj
---
message: /tmp/lint/.automation/test/clojure/clojure_bad_1.clj 5 5 warning namespace clojure.set is required but never used\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 8 2 error unresolved symbol but-last\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 12 1 error clojure.string/join is called with 0 args but expects 1 or 2\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 15 17 warning unused binding y\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 17 3 warning redundant do\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 19 5 warning inline def\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 22 7 warning Redundant let expression.\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 22 13 warning unused binding z\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 24 9 error y is called with 1 arg but expects 0\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 26 9 error recur argument count mismatch (expected 2, got 0)\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 29 9 error h is called with 1 arg but expects 0\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 30 9 error f is called with 1 arg but expects 0\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 34 1 warning redefined var #'foo/private-fn\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 34 8 warning Unused private var foo/private-fn\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 38 6 error Expected number, received keyword.\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 42 1 error java.lang.Thread/sleep is called with 3 args but expects 1 or 2\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 48 1 error foo/foo-fn is called with 0 args but expects 2\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 51 1 error #'foo/private-fn is private\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 54 7 error duplicate key a\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 56 5 error duplicate set element 1\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 58 7 error missing value for key b\n/tmp/lint/.automation/test/clojure/clojure_bad_1.clj 64 3 warning missing test assertion\nlinting took 18ms, errors 13, warnings 9\n
...
ok 2 - clojure_good_1.clj

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - css_bad_01.css
---
message: \ncss/css_bad_01.css\n 2 1 ✖ Expected empty line before comment comment-empty-line-before\n 3 1 ✖ Expected empty line before comment comment-empty-line-before\n 5 5 ✖ Expected indentation of 2 spaces indentation \n 5 33 ✖ Expected "#FFFFFF" to be "#ffffff" color-hex-case \n 5 33 ✖ Expected "#FFFFFF" to be "#FFF" color-hex-length \n 6 5 ✖ Expected indentation of 2 spaces indentation \n 7 5 ✖ Expected indentation of 2 spaces indentation \n 8 5 ✖ Expected indentation of 2 spaces indentation \n 8 12 ✖ Expected "#AAAAAA" to be "#aaaaaa" color-hex-case \n 8 12 ✖ Expected "#AAAAAA" to be "#AAA" color-hex-length\n
...
ok 2 - css_good_01.css

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - dart_bad_1.dart
---
message: Analyzing /tmp/lint/.automation/test/dart/dart_bad_1.dart...\n lint • Use `;` instead of `{}` for empty constructor bodies. • dart/dart_bad_1.dart 4 25 • empty_constructor_bodies\n1 lint found.\n
...
ok 2 - dart_good_1.dart

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - env_bad_1.env
---
message: env/env_bad_1.env 1 LeadingCharacter Invalid leading character detected\nenv/env_bad_1.env 2 KeyWithoutValue The MY_ENV key should be with a value or have an equal sign\nenv/env_bad_1.env 3 IncorrectDelimiter The DB-NAME key has incorrect delimiter\nenv/env_bad_1.env 3 UnorderedKey The DB-NAME key should go before the LOGGER_LEVEL key\nenv/env_bad_1.env 4 LowercaseKey The DEbUG_hTTP key should be in uppercase\nenv/env_bad_1.env 4 UnorderedKey The DEbUG_hTTP key should go before the LOGGER_LEVEL key\nenv/env_bad_1.env 5 UnorderedKey The DB_NAME key should go before the DEbUG_hTTP key\n
...
ok 2 - env_good_1.env

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - golang_bad_01.go
---
message: level=error msg="[linters context] typechecking error golang/golang_bad_01.go 1 1 expected 'package', found 'if'"\nlevel=warning msg="[runner] Can't run linter goanalysis_metalinter S1005 failed prerequisites [(inspect@command-line-arguments, isgenerated@command-line-arguments) analysis skipped errors in package [- golang/golang_bad_01.go 1 1 expected 'package', found 'if']]"\nlevel=warning msg="[runner] Can't run linter unused buildir analysis skipped errors in package [- golang/golang_bad_01.go 1 1 expected 'package', found 'if']"\nlevel=error msg="Running error buildir analysis skipped errors in package [- golang/golang_bad_01.go 1 1 expected 'package', found 'if']"\n
...
ok 2 - golang_good_01.go

File diff suppressed because one or more lines are too long

View file

@ -2,6 +2,6 @@ TAP version 13
1..2 1..2
not ok 1 - json_bad_1.json not ok 1 - json_bad_1.json
--- ---
message: Error Parse error on line 6 \n...ng" { "level" 'ignore', "space\n----------------------^\nExpecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'\n at Object.parseError (/usr/lib/node_modules/jsonlint/lib/jsonlint.js 55 11)\n at Object.parse (/usr/lib/node_modules/jsonlint/lib/jsonlint.js 132 22)\n at parse (/usr/lib/node_modules/jsonlint/lib/cli.js 82 14)\n at main (/usr/lib/node_modules/jsonlint/lib/cli.js 135 14)\n at Object.<anonymous> (/usr/lib/node_modules/jsonlint/lib/cli.js 179 1)\n at Module._compile (internal/modules/cjs/loader.js 1138 30)\n at Object.Module._extensions..js (internal/modules/cjs/loader.js 1158 10)\n at Module.load (internal/modules/cjs/loader.js 986 32)\n at Function.Module._load (internal/modules/cjs/loader.js 879 14)\n at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js 71 12)\n message: Error Parse error on line 6 \n...ng" { "level" 'ignore', "space\n----------------------^\nExpecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'\n at Object.parseError (/node_modules/jsonlint/lib/jsonlint.js 55 11)\n at Object.parse (/node_modules/jsonlint/lib/jsonlint.js 132 22)\n at parse (/node_modules/jsonlint/lib/cli.js 82 14)\n at main (/node_modules/jsonlint/lib/cli.js 135 14)\n at Object.<anonymous> (/node_modules/jsonlint/lib/cli.js 179 1)\n at Module._compile (internal/modules/cjs/loader.js 1138 30)\n at Object.Module._extensions..js (internal/modules/cjs/loader.js 1158 10)\n at Module.load (internal/modules/cjs/loader.js 986 32)\n at Function.Module._load (internal/modules/cjs/loader.js 879 14)\n at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js 71 12)\n
... ...
ok 2 - json_good_1.json ok 2 - json_good_1.json

View file

@ -0,0 +1,12 @@
TAP version 13
1..4
not ok 1 - openapi_bad_1.ymlopenapi
---
message: OpenAPI 3.x detected\n\n/tmp/lint/.automation/test/openapi/openapi_bad_1.ymlopenapi\n 1 1 warning info-contact Info object should contain `contact` object.\n 1 1 warning info-description OpenAPI object info `description` must be present and non-empty string.\n 1 1 warning oas3-api-servers OpenAPI `servers` must be present and non-empty array.\n 1 1 error oas3-schema Object should have required property `info`.\n 1 1 warning openapi-tags OpenAPI object should have non-empty `tags` array.\n\n✖ 5 problems (1 error, 4 warnings, 0 infos, 0 hints)\n
...
not ok 2 - openapi_bad_2.jsonopenapi
---
message: OpenAPI 3.x detected\n\n/tmp/lint/.automation/test/openapi/openapi_bad_2.jsonopenapi\n 1 1 warning info-contact Info object should contain `contact` object.\n 1 1 warning info-description OpenAPI object info `description` must be present and non-empty string.\n 1 1 warning oas3-api-servers OpenAPI `servers` must be present and non-empty array.\n 1 1 error oas3-schema Object should have required property `info`.\n 1 1 warning openapi-tags OpenAPI object should have non-empty `tags` array.\n\n✖ 5 problems (1 error, 4 warnings, 0 infos, 0 hints)\n
...
ok 3 - openapi_good_1.ymlopenapi
ok 4 - openapi_good_2.jsonopenapi

View file

@ -0,0 +1,17 @@
TAP version 13
1..6
not ok 1 - powershell_bad_1.ps1
---
message: \nRuleName Severity ScriptName Line Message\n-------- -------- ---------- ---- -------\nPSAvoidUsingEmptyCatchBlock Warning powershell 12 Empty catch b\n _bad_1.ps1 lock is used.\n Please use W\n rite-Error or\n throw statem\n ents in catch\n blocks.\nPSAvoidUsingUsernameAndPasswordPara Error powershell 4 Function 'Bad\nms _bad_1.ps1 Function' has\n both Usernam\n e and Passwor\n d parameters.\n Either set t\n he type of th\n e Password pa\n rameter to Se\n cureString or\n replace the\n Username and\n Password para\n meters with a\n Credential p\n arameter of t\n ype PSCredent\n ial. If using\n a Credential\n parameter in\n PowerShell 4\n .0 or earlier\n , please defi\n ne a credenti\n al transforma\n tion attribut\n e after the P\n SCredential t\n ype attribute\n .\nPSAvoidTrailingWhitespace Information powershell 11 Line has trai\n _bad_1.ps1 ling whitespa\n ce\nPSAvoidUsingPlainTextForPassword Warning powershell 5 Parameter '$P\n _bad_1.ps1 assword' shou\n ld use Secure\n String, other\n wise this wil\n l expose sens\n itive informa\n tion. See Con\n vertTo-Secure\n String for mo\n re informatio\n n.\nPSUseDeclaredVarsMoreThanAssignment Warning powershell 9 The variable\ns _bad_1.ps1 'VariableThat\n IsNotUsedLate\n r' is assigne\n d but never u\n sed.\n
...
not ok 2 - powershell_bad_1.psd1
---
message: \nRuleName Severity ScriptName Line Message\n-------- -------- ---------- ---- -------\nTerminatorExpectedAtEndOfString ParseError powershell 2 The string is\n _bad_1.psd missing the\n 1 terminator '\n .\nMissingEndCurlyBrace ParseError powershell 1 Missing closi\n _bad_1.psd ng '}' in sta\n 1 tement block\n or type defin\n ition.\n
...
not ok 3 - powershell_bad_1.psm1
---
message: \nRuleName Severity ScriptName Line Message\n-------- -------- ---------- ---- -------\nTerminatorExpectedAtEndOfString ParseError powershell 1 The string is\n _bad_1.psm missing the\n 1 terminator "\n .\n
...
ok 4 - powershell_good_1.ps1
ok 5 - powershell_good_1.psd1
ok 6 - powershell_good_1.psm1

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - protobuf_bad_1.proto
---
message: [protobuf/protobuf_bad_1.proto 8 1] The order of Import is invalid. Check if the file is ordered in the correct manner.\n[protobuf/protobuf_bad_1.proto 18 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 19 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 20 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 21 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 24 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 26 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 28 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 25 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 27 7] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 29 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 30 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 31 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 34 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 47 5] Found an incorrect indentation style " ". " " is correct.\n[protobuf/protobuf_bad_1.proto 3 1] The line length is 91, but it must be shorter than 80\n[protobuf/protobuf_bad_1.proto 4 1] Package name "examplePb" must only contains lowercase letters, digits and/or periods.\n[protobuf/protobuf_bad_1.proto 8 1] Imports are not sorted.\n[protobuf/protobuf_bad_1.proto 9 1] Imports are not sorted.\n[protobuf/protobuf_bad_1.proto 14 1] Imports are not sorted.\n[protobuf/protobuf_bad_1.proto 15 1] Imports are not sorted.\n[protobuf/protobuf_bad_1.proto 19 5] EnumField name "UNKNOWN" with zero value should have the suffix "UNSPECIFIED"\n[protobuf/protobuf_bad_1.proto 17 1] Enum name "enumAllowingAlias" must be UpperCamelCase\n[protobuf/protobuf_bad_1.proto 36 3] Field "inner_message" should avoid required for proto3\n[protobuf/protobuf_bad_1.proto 37 3] Group "Result" should be avoided for proto3\n[protobuf/protobuf_bad_1.proto 40 3] Group "Result" should be avoided for proto3\n[protobuf/protobuf_bad_1.proto 43 3] Group "Regular" should be avoided for proto3\n[protobuf/protobuf_bad_1.proto 29 5] Repeated field name "inner_message" must be pluralized name "inner_messages"\n[protobuf/protobuf_bad_1.proto 40 3] Repeated group name "Result" must be pluralized name "Results"\n[protobuf/protobuf_bad_1.proto 42 3] Repeated field name "paper" must be pluralized name "papers"\n[protobuf/protobuf_bad_1.proto 43 3] Repeated group name "Regular" must be pluralized name "Regulars"\n[protobuf/protobuf_bad_1.proto 23 1] Message name "outer" must be UpperCamelCase\n[protobuf/protobuf_bad_1.proto 26 5] Message name "inner" must be UpperCamelCase\n[protobuf/protobuf_bad_1.proto 47 5] RPC name "search" must be UpperCamelCase\n
...
ok 2 - protobuf_good_1.proto

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - raku_bad_1.raku
---
message: ===SORRY!=== Error while compiling /tmp/lint/.automation/test/raku/raku_bad_1.raku\nVariable '$foo' is not declared. Did you mean '@foo'?\nat /tmp/lint/.automation/test/raku/raku_bad_1.raku 4\n------> <BOL>⏏$foo[1] = 42\n
...
ok 2 - raku_good_1.raku

View file

@ -2,6 +2,6 @@ TAP version 13
1..2 1..2
not ok 1 - ruby_bad_1.rb not ok 1 - ruby_bad_1.rb
--- ---
message: /usr/lib/ruby/gems/2.7.0/gems/rubocop-0.74.0/lib/rubocop/config_loader_resolver.rb 76 warning Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call\n/usr/lib/ruby/gems/2.7.0/gems/rubocop-0.74.0/lib/rubocop/config_loader_resolver.rb 84 warning The called method `merge' is defined here\nInspecting 1 file\nC\n\nOffenses \n\nruby/ruby_bad_1.rb 1 1 C Style/FrozenStringLiteralComment Missing magic comment # frozen_string_literal true.\nruby/ruby_bad_1.rb 11 11 C Style/StringLiterals Prefer double-quoted strings unless you need single quotes to avoid extra backslashes for escaping.\nFile.open('/tmp/urls.txt', " w" ) do | file|\n ^^^^^^^^^^^^^^^\nruby/ruby_bad_1.rb 11 32 C Layout/SpaceInsideParens Space inside parentheses detected.\nFile.open('/tmp/urls.txt', " w" ) do | file|\n ^\nruby/ruby_bad_1.rb 11 39 C Layout/SpaceAroundBlockParameters Space before first block parameter detected.\nFile.open('/tmp/urls.txt', " w" ) do | file|\n ^\nruby/ruby_bad_1.rb 12 24 C Layout/SpaceAroundBlockParameters Space after last block parameter detected.\n Hook.active.map do |h |\n ^\nruby/ruby_bad_1.rb 13 13 C Layout/SpaceInsideArrayLiteralBrackets Do not use space inside array brackets.\n urls = [ ARRAY_OF_URLS_CALLING_INSTANCE]\n ^\n\n1 file inspected, 6 offenses detected\n message: Inspecting 1 file\nC\n\nOffenses \n\nruby/ruby_bad_1.rb 1 1 C Style/FrozenStringLiteralComment Missing frozen string literal comment.\nruby/ruby_bad_1.rb 11 11 C Style/StringLiterals Prefer double-quoted strings unless you need single quotes to avoid extra backslashes for escaping.\nFile.open('/tmp/urls.txt', " w" ) do | file|\n ^^^^^^^^^^^^^^^\nruby/ruby_bad_1.rb 11 32 C Layout/SpaceInsideParens Space inside parentheses detected.\nFile.open('/tmp/urls.txt', " w" ) do | file|\n ^\nruby/ruby_bad_1.rb 11 39 C Layout/SpaceAroundBlockParameters Space before first block parameter detected.\nFile.open('/tmp/urls.txt', " w" ) do | file|\n ^\nruby/ruby_bad_1.rb 12 24 C Layout/SpaceAroundBlockParameters Space after last block parameter detected.\n Hook.active.map do |h |\n ^\nruby/ruby_bad_1.rb 13 13 C Layout/SpaceInsideArrayLiteralBrackets Do not use space inside array brackets.\n urls = [ ARRAY_OF_URLS_CALLING_INSTANCE]\n ^\n\n1 file inspected, 6 offenses detected\n
... ...
ok 2 - ruby_good_1.rb ok 2 - ruby_good_1.rb

View file

@ -0,0 +1,7 @@
TAP version 13
1..2
not ok 1 - terraform_bad_1.tf
---
message: Failed to load configurations. 1 error(s) occurred \n\nError Invalid expression\n\n on /tmp/lint/.automation/test/terraform/bad/terraform_bad_1.tf line 3, in resource "aws_instance" "bad" \n 3 instance_type = # invalid type!\n 4 }\n\nExpected the start of an expression, but found an invalid expression token.\n
...
ok 2 - terraform_good_1.tf