Skip to main content
Private secrets and keyserver delivery are in beta because they use V3 attestation, which is still being deployed.

​
Overview

Standard secrets are stored by Tinfoil and injected at deploy time, which means Tinfoil infrastructure can access their values. With private secrets, you store the values yourself and run a keyserver to control access. The keyserver releases them directly to your secure enclave only after verifying hardware attestation for the exact release you approved. Tinfoil never handles the value. It transits an attested TLS connection from your keyserver into the enclave, and exists in plaintext only inside enclave memory.

​
How it works

Your tinfoil-config.yml sets keyserver-url to your keyserver’s address. Because the config is measured as part of attestation, approving a release also pins the keyserver URL. Tinfoil cannot redirect a running deployment to another keyserver. If you publish a release with a different URL, your keyserver refuses it until you add approval for the new tag. Your enclave gets secrets only from your keyserver, and your keyserver gives them only to code you approved. On every boot:
  1. The enclave requests a single-use nonce from your keyserver.
  2. It builds a fresh hardware attestation document over that nonce. The document embeds everything needed to verify it: the CPU and GPU evidence, and the Sigstore artifacts that authenticate the running code as a specific release of your repo.
  3. It sends the document to your keyserver over mutual TLS.
  4. The keyserver uses the Tinfoil SDK to verify the document offline, just as Tinfoil clients do. It also checks the document against your policy, including the pinned repository, release tag, and deployment domain. It answers only over the TLS connection that proves possession of the key inside the attestation, so a copied or relayed attestation document is useless.
  5. Only if every check passes, the keyserver returns the secrets, and the enclave injects them into your container as environment variables.
The release decision is yours alone: your keyserver, your policy, your secret store. Authorizing a new release requires an explicit policy approval. Enclaves that fail verification (wrong repo, wrong tag, debug mode enabled, stale or replayed evidence) receive nothing.

​
Requirements

  • Use the current keyserver release.
  • Every release needs a freshness witness before an enclave can boot from it. Tinfoil publishes one automatically for the latest release of each repository it deploys. If a brand-new release fails to start, wait a minute and deploy the failed instance again. If only a blue/green update failed while the current version is still running, cancel the failed update and retry Update.
  • Your config repo must be public, like all Tinfoil Containers config repos. The Docker image and your secrets stay private (see Private images).
  • The keyserver must be reachable from the enclave at every boot, serve /challenge and /fetch directly at the keyserver-url (redirects are refused), and present a TLS certificate from a publicly trusted CA (Let’s Encrypt works; self-signed certificates are rejected).
  • keyserver-url must be HTTPS with a host, without URL credentials, a query string, or a fragment. A base path is allowed; an explicit port must be from 1 through 65535. These syntax checks also apply in debug mode.
  • Non-debug private delivery requires hardware attestation. Non-confidential instances and dummy attestation are rejected rather than using managed secrets.

​
Run a keyserver

The reference keyserver is a small, auditable Go server: tinfoilsh/keyserver. It fronts HashiCorp Vault (KV v2), AWS Secrets Manager, or a local file for isolated deployments. Its policy pins which build may receive which secrets:
policy.yaml
Each workload entry approves one repository, tag, and domain. To authorize another release while retaining this approval, add a differently named workload entry with the new tag, its exact domain, and its required secret mappings. Keep the original entry while that release may still need to boot, including during rollback. domain is required; the keyserver refuses to load a policy without it. Config repos are public, so anyone can deploy your exact repo and tag and produce an identical, validly-attested measurement. Without a domain pin, that copy would also pass verification and receive your secrets. The domain pin requires the caller to hold a CA-issued certificate for your deployment’s domain, which only your deployment can obtain, so a copy is refused. See the keyserver README for running the server and connecting each backend.

​
Wire your deployment

Declare the secrets on your container and point keyserver-url at your keyserver. Replace <CVM_VERSION> with the current CVM release version, without the v prefix:
tinfoil-config.yml
With keyserver-url set and debug mode off, the enclave requests the full set of names declared in containers[].secrets, models[].key-secret, and volumes[].key-secret from your keyserver at boot. Names are deduplicated; environment-variable interpolation and API secret selections do not add private names. Workload secrets become environment variables; model and volume keys serve their declared uses. Do not create these values in Tinfoil’s organization or project secret store, select them with --secret, or supply them through the API’s variables map. The API’s secrets field selects Tinfoil-managed secrets, not private names. For a new private instance, omit that field or send "secrets": []. Nonempty managed selections are rejected, including names not declared in the config. Private declarations cannot collide with saved/request variables or operator bootstrap credentials. There is no fallback to a same-named managed secret. If the keyserver cannot return every declared value and pass verification, the workload does not boot. Never upload a private secret or volume key to Tinfoil to work around a validation, keyserver, or unlock failure.
Debug mode intentionally ignores keyserver-url and uses managed secret delivery. This is not private delivery and does not provide confidentiality. Use disposable test secrets and test disks only, never production secrets or the keys to production volumes. The keyserver would refuse a debug enclave’s attestation.
Registry credentials and certificate authorization remain host-supplied bootstrap inputs, separate from private workload values. Debug SSH credentials also retain their managed delivery path.

​
Switching an existing instance to private delivery

  1. Publish a release containing keyserver-url and the required secret names, and authorize its repository, tag, and instance domain in your keyserver.
  2. Explicitly clear the instance’s managed secret selection in the same deploy/update request. Omitting secrets or sending null preserves the saved selection; it does not clear it. Remove any saved variable bindings that collide with private names, preserving other intended variables.
  3. For a running instance, inspect a read-only update plan before submitting the update. Verify the guest’s secret delivery and any volume unlock after boot; a successful plan cannot verify either.
For example, suppose v1.1.0 is your published private release, DATABASE_URL is now a declared private secret, and LOG_LEVEL=info is the only saved per-instance variable you want to retain. This complete replacement map removes any saved DATABASE_URL variable while preserving LOG_LEVEL. Use this JSON body with POST /api/containers/:id/update/plan, then with POST /api/containers/:id/update after reviewing it:
Adapt variables to include every non-conflicting per-instance variable you still need. A supplied map replaces the whole saved map; omitting it or sending null preserves collisions. Use "variables": {} only when no saved variables should remain. This does not replace measured env in the release config. The equivalent CLI update for this example is:
A single --secret="" sends "secrets": []; do not combine it with named selections or repeat it. Each --variable contributes to the full replacement map, so repeat it for all non-conflicting values you are retaining. The CLI shows the update plan before execution, including external secret names and the warning that authorization and unlock are not verified. Replace :id with the instance UUID. Updates still inherit the project’s hold default. If the plan requires replacement, explicitly disable holding for that operation and acknowledge downtime as described in Updating. For a stopped or failed instance, send the clearing body to POST /api/containers/:id/deploy; deploys cannot be held. Clearing the selection removes instance bindings, not the managed secret values from the organization’s store. Project updates cannot clear per-instance selections; transition affected instances individually first. API callers must send [], not [""]; the CLI’s empty flag is translated to the empty array.

​
Operating notes

  • Fetches happen at every enclave boot: Create, Deploy, and Update all create a new enclave that must reach your keyserver. A keyserver outage doesn’t affect running enclaves, but blocks new ones from starting.
  • Rotation of application credentials: update the value in your secret store, then update the container. The new enclave fetches the current value. This does not apply to persistent-volume keys: keep the original key unless you perform an explicit data migration or supported re-encryption procedure. Changing a secret value alone does not re-encrypt a disk; see volume-key rotation.
  • New releases: add approval for the exact repository, tag, and domain before deploying the new version, or the new enclave will be refused. Retain previous approvals while those releases may still need to boot, including during rollback. The keyserver reads its policy at startup, so restart it after editing.
  • Audit: the keyserver logs every release with the verified repo and tag, and every denial with its reason, giving you a complete record of who received which secrets.
  • Diagnosing a refused boot: inspect the instance’s boot stages and your keyserver logs for the refusal reason.
For a minimal end-to-end example, see tinfoilsh/confidential-secrets-demo.