Conversation
|
|
When init links a provider from the global plugin cache and the dependency lock file already has checksums for it, the installer computed the same h1 hash of the cached package three times: to decide whether the cache entry matches the lock file, again inside LinkFromOtherCache, and once more for the lock file update. Each hash reads the whole package, which for large providers such as hashicorp/aws (around 770MB unpacked) takes most of a second. The hash is now computed at most once per cache entry and reused, and still only when something needs it. The lock file checks are unchanged.
Adds a test hook to the CachedProvider hashing methods and a test that counts how often EnsureProviderVersions reads a package from the global plugin cache: once when the lock file has checksums for it or the cache may break the lock file, and not at all when the entry isn't eligible.
Dutchy-
force-pushed
the
init-hash-cached-provider-once
branch
from
September 21, 2026 17:56
c364d42 to
83fd380
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When
terraform initlinks a provider from the global plugin cache and the dependency lock file already has checksums for it, the installer read and hashed the entire cached package three times. It now hashes it once and reuses the result. The lock file checks themselves are unchanged.For a configuration that only requires
hashicorp/aws6.44.0 (770MB unpacked), with a lock file and a warm plugin cache,terraform initon an Apple silicon Mac went from 1.28s to 0.50s wall time and from 0.99s to 0.35s CPU time (median of 8 runs).The existing
TestEnsureProviderVersionscases cover each branch of this path: no lock entry, matching and mismatching checksums, andplugin_cache_may_break_dependency_lock_file. The second commit adds a test that counts how often the cached package is hashed. It fails onmain(3 reads instead of 1). It relies on a test hook incached_provider.go, and I'm unsure whether that's acceptable here, even with precedent such astestChecksumHookin the s3 and oss backends. I'm happy to drop that commit if you'd rather not have it.Fixes #39252
AI disclosure
I used Claude Code (Claude Opus 5) for this PR. It found the problem while profiling
terraform init, wrote the change and the test, ran the benchmarks, and drafted the commit messages and this description. I reviewed the change and the test and can explain both.Target Release
1.18.x
Rollback Plan
Changes to Security Controls
None. Cache entries are still verified against the lock file before use. The second check before linking is skipped only after the first one has passed on the same cache entry.
CHANGELOG entry