Secret sources

A SecretSource is the adapter contract for backends that resolve ${secret:<scheme>:<path>} references. ADR-0002 ships two Phase 2 adapters:

• EnvSecretSource — mandatory in-process adapter for the env scheme. Always auto-registered; you do not have to install anything.
• VaultSource — optional opt-in adapter for HashiCorp Vault KV v2. Distributed as an extra / peer dependency / sub-module so the Vault SDK does not leak into binaries that only use file sources.

This page covers the install command, the public construction API, and the operational boundaries per binding. For the user-facing syntax (${secret:vault:...}, lazy vs eager, refresh) see Secrets.

EnvSecretSource — mandatory in-process

EnvSecretSource is auto-registered by the loader if the consumer does not pass one explicitly. The env scheme requires no configuration and no extra install — ${secret:env:OPENAI_API_KEY} works out of the box.

You construct one explicitly only when you need a custom env lookup (typically tests):

Python

from dagstack.config import (
    Config,
    EnvSecretSource,
    YamlFileSource,
)

# Override the getenv callable for a deterministic test.
env_table = {"OPENAI_API_KEY": "sk-test-...", "DB_PASSWORD": "pw-test"}
cfg = Config.load_from([
    YamlFileSource("app-config.yaml"),
    EnvSecretSource(getenv=env_table.get),
])

TypeScript

import {
  Config,
  EnvSecretSource,
  YamlFileSource,
} from "@dagstack/config";

const envTable: Record<string, string> = {
  OPENAI_API_KEY: "sk-test-...",
  DB_PASSWORD: "pw-test",
};
const cfg = await Config.loadFrom([
  new YamlFileSource("app-config.yaml"),
  new EnvSecretSource({ lookup: (name) => envTable[name] }),
]);

Go

import (
    "context"

    "go.dagstack.dev/config"
)

envTable := map[string]string{
    "OPENAI_API_KEY": "sk-test-...",
    "DB_PASSWORD":    "pw-test",
}
envSrc := &config.EnvSecretSource{
    Lookup: func(name string) (string, bool) {
        v, ok := envTable[name]
        return v, ok
    },
}
cfg, err := config.LoadFrom(ctx, []config.Source{
    config.YamlFileSource("app-config.yaml"),
    envSrc,
})

The env scheme does not support ?query parameters or #field projection — env values are opaque single-value strings. If you need structured secrets, switch to a backend that returns JSON (such as Vault).

VaultSource — HashiCorp Vault KV v2

Phase 2 ships a pilot adapter for HashiCorp Vault KV v2. The adapter resolves ${secret:vault:...} references against a Vault server, with caching for the lifetime of the Config object.

Install

The Vault SDK is a transitive — not required — dependency of dagstack/config. You opt in per binding:

Python

pip install 'dagstack-config[vault]'

The [vault] extra pulls hvac>=2.0,<3.0 (the maintained Python client). Without the extra, import dagstack.config.vault raises ImportError with a hint to install the extra.

TypeScript

npm install @dagstack/config node-vault

node-vault is declared as an optional peer dependency (peerDependenciesMeta.optional: true) — it is not installed transitively when you install @dagstack/config. Add it explicitly when you need VaultSource.

Go

go get go.dagstack.dev/config/vault

The Vault adapter is a separate Go module (go.dagstack.dev/config/vault) so that the official github.com/hashicorp/vault/api dependency does not leak into binaries that use only file sources.

Construct the source

Python

import os

from dagstack.config import Config, YamlFileSource
from dagstack.config.vault import (
    AppRoleAuth,
    KubernetesAuth,
    TokenAuth,
    VaultSource,
)

src = VaultSource(
    addr="https://vault.example.com",
    auth=TokenAuth(token=os.environ["VAULT_TOKEN"]),
    namespace="dagstack/prod",  # optional Vault Enterprise namespace
    verify=True,                # True / path to CA bundle / False (dev-only)
    timeout=30.0,
)

cfg = Config.load_from([
    YamlFileSource("app-config.yaml"),
    src,
], eager_secrets=True)

TypeScript

import { Config, YamlFileSource, VaultSource } from "@dagstack/config";

const src = new VaultSource({
  addr: "https://vault.example.com",
  auth: { kind: "token", token: process.env.VAULT_TOKEN! },
  namespace: "dagstack/prod",
});

const cfg = await Config.loadFrom([
  new YamlFileSource("app-config.yaml"),
  src,
]);

Go

import (
    "context"
    "os"

    "go.dagstack.dev/config"
    "go.dagstack.dev/config/vault"
)

src, err := vault.NewSource(
    "https://vault.example.com",
    vault.TokenAuth{Token: os.Getenv("VAULT_TOKEN")},
    vault.WithNamespace("dagstack/prod"),
)
if err != nil {
    return err
}

cfg, err := config.LoadFrom(ctx, []config.Source{
    config.YamlFileSource("app-config.yaml"),
    src,
})

Auth methods

The Phase 2 normative auth methods are Token and AppRole. Kubernetes ServiceAccount is also supported across all three bindings. AWS IAM, JWT/OIDC, TLS client certificate and others are deferred to Phase 3.

Token

The simplest case. The operator supplies a token directly — typically injected via init-container or developer action.

Python

from dagstack.config.vault import TokenAuth, VaultSource

src = VaultSource(
    addr="https://vault.example.com",
    auth=TokenAuth(token=os.environ["VAULT_TOKEN"]),
)

TypeScript

new VaultSource({
  addr: "https://vault.example.com",
  auth: { kind: "token", token: process.env.VAULT_TOKEN! },
});

Go

vault.NewSource(
    "https://vault.example.com",
    vault.TokenAuth{Token: os.Getenv("VAULT_TOKEN")},
)

AppRole

The production CI/CD default. The application bootstraps with a role_id (well-known, baked into the deployment) and a secret_id (short-lived, delivered by a trusted broker).

Python

from dagstack.config.vault import AppRoleAuth, VaultSource

src = VaultSource(
    addr="https://vault.example.com",
    auth=AppRoleAuth(
        role_id=os.environ["VAULT_ROLE_ID"],
        secret_id=os.environ["VAULT_SECRET_ID"],
        mount_point="approle",  # default
    ),
)

TypeScript

new VaultSource({
  addr: "https://vault.example.com",
  auth: {
    kind: "approle",
    roleId: process.env.VAULT_ROLE_ID!,
    secretId: process.env.VAULT_SECRET_ID!,
    // mountPoint defaults to "approle"
  },
});

Go

vault.NewSource(
    "https://vault.example.com",
    vault.AppRoleAuth{
        RoleID:     os.Getenv("VAULT_ROLE_ID"),
        SecretID:   os.Getenv("VAULT_SECRET_ID"),
        MountPoint: "approle", // default
    },
)

Kubernetes ServiceAccount

Reads the projected ServiceAccount JWT from /var/run/secrets/kubernetes.io/serviceaccount/token (override via jwt_path) and exchanges it for a Vault token via auth/kubernetes/login. Ergonomic for Kubernetes deployments — no extra secret injection needed.

Python

from dagstack.config.vault import KubernetesAuth, VaultSource

src = VaultSource(
    addr="https://vault.example.com",
    auth=KubernetesAuth(
        role="dagstack-prod",
        jwt_path="/var/run/secrets/kubernetes.io/serviceaccount/token",  # default
        mount_point="kubernetes",                                         # default
    ),
)

TypeScript

new VaultSource({
  addr: "https://vault.example.com",
  auth: {
    kind: "kubernetes",
    role: "dagstack-prod",
    // jwtPath defaults to "/var/run/secrets/kubernetes.io/serviceaccount/token"
    // mountPoint defaults to "kubernetes"
  },
});

Go

vault.NewSource(
    "https://vault.example.com",
    vault.KubernetesAuth{
        Role:       "dagstack-prod",
        JWTPath:    "/var/run/secrets/kubernetes.io/serviceaccount/token", // default
        MountPoint: "kubernetes",                                          // default
    },
)

Namespace, KV path layout, versioning

• Vault Enterprise namespaces are configured at construction time (namespace="dagstack/prod"); the adapter prepends them automatically. The YAML reference stays namespace-free for portability across environments.
• KV v2 path layout. The user-visible path is what vault kv get accepts (secret/dagstack/prod/openai). The first segment is the KV v2 mount point (default install uses secret); the remainder is the logical key path. The Vault HTTP API expects <mount>/data/<path> — the adapter rewrites it internally.
• Versioning. Append ?version=N to pin a specific KV v2 version (${secret:vault:secret/dagstack/prod/db?version=3#password}). Without ?version=, the adapter reads the latest. If the requested version is destroyed or deleted, the adapter raises secret_unresolved.
• Sub-key projection. A Vault secret with multiple fields is addressed via #field (secret/dagstack/prod/db#password vs secret/dagstack/prod/db#username). The cache key strips the projection, so two references with different #field share one Vault read.

Token renewal — Phase 2 boundary

Vault tokens issued by AppRole / Kubernetes login have a TTL. The Phase 2 adapter authenticates once at construction time and uses that token until the process exits. There is no in-flight background renewal goroutine / asyncio task / setInterval watchdog.

This is a deliberate Phase 2 scope cut — adding renewal requires a subscription model that ADR-0001 §7 only partially specifies. The push-rotation ADR-0003 (candidate) is the design discussion for renewal background tasks; until it lands, the operator's options are:

Pattern	When it fits	Trade-off
Long-TTL Vault tokens — issue a 30-day token, restart the process every 28 days.	Stateless services with rolling restarts.	Restart adds load; long-lived tokens widen the impact of a leak.
AppRole with frequent restarts — short-TTL token (1h), Kubernetes liveness probe forces restart.	Fits k8s deployments with horizontal autoscaling.	Restart loops if Vault is briefly unavailable; mitigate with a longer probe period.
Kubernetes ServiceAccount auth — the SA JWT is rotated by k8s automatically, but the Vault token from the login exchange still has its own TTL.	k8s-native deployments.	Same TTL trade-off as the other methods; only the JWT auth path is renewal-free.
External rotation (refresh_secrets) — call cfg.refresh_secrets() on a schedule. The next backend read fails when the Vault token has expired, surfacing as secret_backend_unavailable.	Lower-traffic services where you can tolerate a 5xx burst on rotation day.	The constructor's authenticated client cannot itself re-auth, so this only buys time until the underlying token expires. For long-running services, plan for restart.

A renewal API will land in Phase 3 as part of ADR-0003 candidate. The expected shape: a background task in the adapter that calls Vault auth/token/renew-self before expiry and auth/token/login when renewal fails. Detail is deferred until the operator data justifies the design.

Diagnostics

• VaultSource.id is "vault:<addr>", with a ?namespace=... suffix if a namespace was configured. The id appears in SecretValue.source_id and in ConfigError.source_id so that you can correlate failures with the specific Vault cluster.
• Auth failures map to secret_permission_denied (Vault returned 403) or secret_backend_unavailable (network, timeout, unauthenticated transport).
• Read failures map to secret_unresolved (key missing, version destroyed) or secret_permission_denied (Vault policy denies the read).

Multiple Vault clusters

The loader rule "one source per scheme" allows operator-extensible scheme names. If you need two Vault clusters in the same config (production + DR), construct two VaultSource instances with different scheme:

Python

# Python — subclass to override the scheme.
class VaultDR(VaultSource):
    scheme: str = "vault-dr"

cfg = Config.load_from([
    YamlFileSource("app-config.yaml"),
    VaultSource(addr="https://vault-prod.example.com", auth=...),
    VaultDR(addr="https://vault-dr.example.com",       auth=...),
])

YAML reference: ${secret:vault-dr:secret/dagstack/prod/db#password}.

TypeScript

// TypeScript — subclass and override the readonly scheme field.
class VaultDR extends VaultSource {
  override readonly scheme = "vault-dr";
}

const cfg = await Config.loadFrom([
  new YamlFileSource("app-config.yaml"),
  new VaultSource({ addr: "https://vault-prod.example.com", auth: ... }),
  new VaultDR({   addr: "https://vault-dr.example.com",   auth: ... }),
]);

YAML reference: ${secret:vault-dr:secret/dagstack/prod/db#password}.

Go

// Go — wrap vault.Source to override Scheme().
type vaultDR struct{ *vault.Source }

func (vaultDR) Scheme() string { return "vault-dr" }

prod, _ := vault.NewSource("https://vault-prod.example.com", ...)
dr,   _ := vault.NewSource("https://vault-dr.example.com",   ...)

cfg, err := config.LoadFrom(ctx, []config.Source{
    config.YamlFileSource("app-config.yaml"),
    prod,
    vaultDR{dr},
})

YAML reference: ${secret:vault-dr:secret/dagstack/prod/db#password}.

The vault-dr scheme is operator-defined; only env, vault, awssm, gcpsm and k8ssecret are reserved by the spec.

What is deferred

Out of Phase 2 scope and tracked under ADR-0003 candidate:

• AwsSecretsManagerSource, GcpSecretManagerSource, K8sSecretSource — cloud and in-cluster adapters.
• Push-based rotation — on_secret_change subscription, Vault lease renewal callbacks, AWS-SM EventBridge integration.
• Token self-renewal for Vault auth — background renewal tasks for AppRole / Kubernetes / IAM auth methods.
• KV v1 adapter — Vault deployments still on KV v1 land in Phase 3 if operator demand justifies it.
• Dynamic secrets with leases (Vault database/creds/...) — needs the lease lifecycle, which depends on the subscription model.

See also

• Secrets — the user-facing reference syntax, resolution timing, error taxonomy.
• Sources — ConfigSource (tree-shaped, distinct from SecretSource).
• ADR-0002 — Phase 2 normative spec.
• ADR-0003 candidate — Phase 3 design discussion.
• Per-binding ADRs for Vault SDK choice: Python · TypeScript · Go.