Commit graph

880 commits

Author SHA1 Message Date
Daz DeBoer
6084a4eb65
Cache artifacts with single entry per type
When caching is too fine-grained, an excessive number of cache
requests can result in HTTP 429 errors due to rate limiting.
By caching all artifacts of a particular type in a single entry
we hope to mitigate this, at the expense of some reduction in
cache space optimization.

This change also adds caching for all dependency jars, as well as
instrumented jars in the 'caches/jars-X' directory.
2021-09-15 17:45:47 -06:00
Daz DeBoer
dbb485d80d
Only perform post-restore and pre-save cache actions when required 2021-09-15 17:45:47 -06:00
Daz DeBoer
9bfa003014
Build outputs 2021-09-14 18:11:31 -06:00
Daz DeBoer
fe64d05f86
Do not cache dependency jars individually
Unfortunately, doing this overloads the GitHub actions cache infrastructure
leading to failures and unpredictable results.
A later solution may re-implement artifact sharing for dependency jars
as well as jars within the `caches/jars-9` directory. But for now these
will be duplicated across each Gradle User Home cache entry.
2021-09-14 18:07:28 -06:00
Daz DeBoer
decca791c5
Consolidate error handling for cache restore/save 2021-09-14 13:39:37 -06:00
Daz DeBoer
bd08e7b7cd
Do not fail build on error restoring artifact for marker
Instead, catch and report these errors before continuing.
2021-09-14 13:38:48 -06:00
Daz DeBoer
cca55d0890
Fail action execution on unhandled errors
Without this, the error logs contain an "UnhandledPromiseRejectionError"
but the action is reported as succeeding.
2021-09-14 07:48:06 -06:00
Daz DeBoer
a802a3c0ce
Generate source-map files when compiling
This will make reported errors easier to link back to Typescript sources.
2021-09-14 05:46:11 -06:00
Daz DeBoer
d06e19f862
Build outputs 2021-09-13 11:50:07 -06:00
Daz DeBoer
bbe1574290
Add some TODOs to the code for future reference 2021-09-13 11:50:07 -06:00
Daz DeBoer
4264cda558
Group log messages generated when provisioning Gradle 2021-09-13 11:50:07 -06:00
Daz DeBoer
3390540145
Simplify setting caches to disabled or read-only 2021-09-13 11:04:42 -06:00
Daz DeBoer
1c72a31463
Allow cache debug logging to be selectively enabled
- Move more messages out of main 'info' log
- Only log cache entry size when debugging
- Process cache entries sequentially when debugging
2021-09-13 10:52:09 -06:00
Daz DeBoer
7dfbe33bba
Improve formatting for cache entry size reporting 2021-09-13 10:51:24 -06:00
Daz DeBoer
e63ddf9c00
Avoid warning for cache reservation errors
We have no way of knowing if another build has already cached these
artifacts, so CacheReservationError is expected.
2021-09-13 10:50:47 -06:00
Daz DeBoer
d5cd9d86a1
Split the 'prod' workflow up into different isolated workflows
This is required to ensure that cache entries are namespaced correctly
for the integration tests.
2021-09-12 12:17:05 -06:00
Daz DeBoer
cae99bf6d9
Tidy up cache entry de-duplication
- Rename feature to "common artifact caching"
- Cleanup logging
- Refactor/rename for clarity
2021-09-12 10:31:52 -06:00
Daz DeBoer
5a90152b1f
Restore/cache deduplicated files in parallel 2021-09-12 10:31:52 -06:00
Daz DeBoer
4b92b8d013
Cache downloaded dependency jars separately
This will further remove common files out the the Gradle User Home
cache entries, reducing cache usage by removing redundancy.
2021-09-12 10:31:51 -06:00
Daz DeBoer
693293c29a
Consolidate logic for separate artifact file caching
Making this functionality more general will make it easier to add other
artifacts to the set that we cache separately.
2021-09-12 10:31:51 -06:00
Daz DeBoer
ac5d8920dd
Cache generated-gradle-jars separately from Gradle User Home
Similar to wrapper distributions, these large files are common
to many Gradle User Home cache entries. Storing them separately removes
this redundancy from the Gradle User Home cache.
2021-09-12 10:31:51 -06:00
Daz DeBoer
e833360307
Cache downloaded wrapper dists separately
In the current model, each cached Gradle User Home could contain
a copy of one or more downloaded wrapper distributions. This results
in large cache entries which could easily lead to premature eviction.

With this change, wrapper dists are cached separately from the rest
of the Gradle User Home directory. The artifact file is replaced by
a marker file which allows the action to restore the artifact from
cache when the Gradle user Home cache is restored.
2021-09-12 10:31:51 -06:00
Daz DeBoer
b5a08466b4
Add kotlin-dsl sample to prod workflow tests 2021-09-12 10:30:49 -06:00
Daz DeBoer
4032438d2b
Build outputs 2021-09-07 16:26:37 -06:00
Daz DeBoer
9b3abaad52
Use current matrix values for workflow job context 2021-09-07 16:26:28 -06:00
Daz DeBoer
d20d631365
Cleanup code to resolve Gradle wrapper script 2021-09-07 15:17:39 -06:00
Daz DeBoer
378bd0b6f8
Save and restore caches in parallel 2021-09-07 15:17:39 -06:00
Daz DeBoer
6d1455a33e
Tidy-up caching code
- Extracted common code for Gradle User Home and Project .gradle caches
  into abstract supertype.
- Improve error handling by checking error types
2021-09-07 15:17:39 -06:00
Daz DeBoer
c44ebadf6f
Add build scan link as a notice annotation
- Use 'notice' method introduced in @actions/core v1.5.0
2021-09-05 21:38:34 -06:00
Daz DeBoer
4d37378696
Consolidate cache-enabled options 2021-09-05 21:38:33 -06:00
Daz DeBoer
777a6fc967
Generate cache key based on Job invocation
Attempt to capture as much context as possible about the job run
to generate a unique cache key. Unfortunately much of the matrix context
is not available to the action implementation.
2021-09-05 21:35:17 -06:00
Daz DeBoer
d7ed6d7e8d
Extract cache key generation into common function 2021-09-05 17:11:50 -06:00
Daz DeBoer
0ecbac99f3
Group log messsages for restore/save cache 2021-09-03 11:34:22 -06:00
Daz DeBoer
436390bd4e
Test coverage: Verify configuration-cache is restored with project-dot-gradle 2021-08-27 12:53:02 -06:00
Daz DeBoer
a587e93714
Test coverage: Verify local build cache is restored with gradle-user-home 2021-08-27 12:53:02 -06:00
Daz DeBoer
75e00ee3d1
Test coverage: Verify dependency caching by running offline build 2021-08-27 12:53:02 -06:00
Daz DeBoer
c01af7a6f6
Test coverage: verify correct Gradle version is being used 2021-08-27 12:53:02 -06:00
Daz DeBoer
c79d4172e0
Build outputs 2021-08-27 12:53:01 -06:00
Daz DeBoer
b85ac67c9a
Seed the cache key with workflow id in CI
This will eliminate cache entries from previous workflow runs, allowing
us to test cache functionality in isolation. If the `CACHE_KEY_SEED` environment
variable is not set, this will have no impact.
2021-08-27 12:52:01 -06:00
Daz DeBoer
fa0c026e07
Always execute Gradle with --no-daemon
The Gradle daemon is not useful for ephemeral builds, and the process
can hold file locks which interfere with cache entry generation.

In the case where multiple Gradle invocations occur in the same job,
we could provide a way for users to override this behaviour, taking care
of stopping any daemon process at the end of the job.
2021-08-27 12:21:46 -06:00
Daz DeBoer
986024f0b7
Adapt README and workflow for new caching 2021-08-26 17:25:50 -06:00
Daz DeBoer
6fca6b3929
Add support for read-only caching in v2 2021-08-26 17:25:49 -06:00
Daz DeBoer
d9cc0aeccf
Remove old caching code 2021-08-24 12:54:21 -06:00
Daz DeBoer
5340f6e816
Add cache for project .gradle dir
- For now, this is limited to configuration-cache directory
2021-08-24 12:52:51 -06:00
Daz DeBoer
c211be411e
Use monolithic cache for Gradle User Home
- Do not restore cache when GUH exists
- Include RUNNER_OS in the cache key
- Do not save cache on exact hit
- Only save cache in the final post action
- Log before saving cache
2021-08-24 12:52:51 -06:00
Daz DeBoer
b3afdc78a7
Truncate Gradle args when constructing cache key (#71)
Cache keys have a hard limit of 512 characters, so we need to ensure that we don't generate a key longer than this.

- Remove excess whitespace
- Truncate to 400 characters

Fixes #70
2021-08-24 12:46:48 -06:00
Daz DeBoer
e0c2736e35
Include Gradle invocation arguments in cache keys (#69)
This permits a new cache entry to be persisted when a subsequent Gradle invocation does more work that an earlier invocation.

Fixes #68
2021-08-22 17:11:05 -06:00
Daz DeBoer
a63892c289
Log cache entry on save 2021-08-22 14:58:36 -06:00
Daz DeBoer
d432f2086c
Provide a better description for the action 2021-08-22 14:48:38 -06:00
Daz DeBoer
eaad2cd2bb
Merge pull request #67 from gradle/dd/v15
Allow caching to be enabled read-only
2021-08-22 14:45:57 -06:00