> ## Documentation Index
> Fetch the complete documentation index at: https://docs.goldsky.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Secrets

> Create and manage credentials for pipeline sinks, transforms, and webhooks with the goldsky secret CLI.

## Overview

Secrets store the credentials Goldsky needs to connect to your infrastructure: database connection details, message queue credentials, cloud provider keys, and HTTP auth headers. They are stored securely in your Goldsky project and referenced by name from your pipeline configuration, so credentials never appear in your YAML.

## Managing secrets

Create and manage secrets with the `goldsky secret` command:

```bash theme={"dark"}
# Create a secret (the CLI prompts for the type and value)
goldsky secret create MY_POSTGRES_SECRET

# List existing secrets
goldsky secret list

# Show a secret's stored value
goldsky secret reveal MY_POSTGRES_SECRET

# Delete a secret (add -f to skip the confirmation prompt in scripts)
goldsky secret delete MY_POSTGRES_SECRET
```

Run `goldsky secret -h` for the full list of commands.

<Warning>
  Deleting a secret that a running pipeline references causes that pipeline to fail on its next connection. Check for references before deleting.
</Warning>

### Secret types and the interactive prompt

Running `goldsky secret create` with no `--value` starts an interactive prompt that first asks you to pick a secret type, then collects the fields that type requires (host, credentials, region, etc.) and prints the permissions the credentials need before you paste them in. Every stored secret is tagged with one of these types:

| Type            | Used for                                                                                |
| --------------- | --------------------------------------------------------------------------------------- |
| `jdbc`          | Postgres and MySQL sinks. Enter a connection string or type in host/port/user/pass etc. |
| `clickHouse`    | ClickHouse sinks. URL, username, password, database name                                |
| `elasticSearch` | Elasticsearch sinks (Mirror legacy). Host URL, username, password                       |
| `opensearch`    | OpenSearch sinks. Host URL, username, password                                          |
| `kafka`         | Kafka sinks, with `PLAINTEXT`, `SASL_PLAINTEXT`, or `SASL_SSL` protocols                |
| `s3`            | S3 and S3-compatible object storage sinks. Access key, secret key, region               |
| `sqs`           | AWS SQS sinks. Access key, secret key, region                                           |
| `pubsub`        | Google Cloud Pub/Sub sinks. GCP project ID plus a service-account JSON key              |
| `dynamodb`      | AWS DynamoDB sinks. Access key, secret key, region                                      |
| `httpauth`      | Webhook sinks. Stores a single auth header as a `secretKey` / `secretValue` pair        |

Compose secrets are created through the [`goldsky compose` CLI](/compose/cli-reference#secrets).

<Note>
  Turbo webhook sinks accept an `httpauth` secret via `secret_name`, but the header it injects cannot also appear in the sink's inline `headers:` field. Pick one or the other. See the [Turbo webhook sink](/turbo-pipelines/sinks/webhook#secret-creation) for details.
</Note>

### Naming secrets

Secret names can only contain alphanumeric characters, underscores (`_`), and hyphens (`-`). Use descriptive uppercase names that encode environment and purpose (`PROD_POSTGRES_MAIN`, `STAGING_CLICKHOUSE`) rather than `secret1` or `postgres`, so a `goldsky secret list` stays legible as your project grows.

### Rotating credentials

Update a secret in place to rotate credentials without touching any pipeline YAML:

```bash theme={"dark"}
goldsky secret update MY_POSTGRES_SECRET --value 'postgres://admin:NEW_PASSWORD@db.example.com:5432/mydb'
```

Running pipelines pick up the new value on their next connection.

## Referencing a secret in a pipeline

Reference a secret by name in the `secret_name` field of a sink or transform:

```yaml theme={"dark"}
sinks:
  postgres_output:
    type: postgres
    from: filtered_transfers
    schema: public
    table: erc20_transfers
    secret_name: MY_POSTGRES_SECRET # References the stored secret
    primary_key: id
```

## Secret formats

Each sink or transform type expects its own secret format. For [Turbo pipelines](/turbo-pipelines/introduction), database secrets are created from a connection string; queue and storage secrets are JSON objects; HTTP secrets store a header name and value.

| Used by                                                                                                              | Format                                                              | Details                                                                            |
| -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| [PostgreSQL sink](/turbo-pipelines/sinks/postgres), [dynamic tables](/turbo-pipelines/transforms/dynamic-tables)     | `postgres://user:password@host:port/database`                       | [Postgres sink](/turbo-pipelines/sinks/postgres)                                   |
| [ClickHouse sink](/turbo-pipelines/sinks/clickhouse)                                                                 | `https://user:password@host:port/database` (`clickHouse` type)      | [ClickHouse sink § Secret format](/turbo-pipelines/sinks/clickhouse#secret-format) |
| [MySQL sink](/turbo-pipelines/sinks/mysql)                                                                           | `mysql://user:password@host:3306/database` or individual fields     | [MySQL sink § Secret format](/turbo-pipelines/sinks/mysql#secret-format)           |
| [Kafka sink](/turbo-pipelines/sinks/kafka)                                                                           | JSON object (`bootstrapServers`, SASL credentials, schema registry) | [Kafka sink § Secret structure](/turbo-pipelines/sinks/kafka#secret-structure)     |
| [SQS sink](/turbo-pipelines/sinks/sqs)                                                                               | JSON object (`accessKeyId`, `secretAccessKey`, `region`)            | [SQS sink § Secret format](/turbo-pipelines/sinks/sqs#secret-format)               |
| [S3 sink](/turbo-pipelines/sinks/s3)                                                                                 | JSON object (`accessKeyId`, `secretAccessKey`, `region`)            | [S3 sink § Secret format](/turbo-pipelines/sinks/s3#secret-format)                 |
| [Pub/Sub sink](/turbo-pipelines/sinks/pubsub)                                                                        | `pubsub` type (GCP project id + service account key)                | [Pub/Sub sink § Secret format](/turbo-pipelines/sinks/pubsub#secret-format)        |
| [Webhook sinks](/turbo-pipelines/sinks/webhook), [HTTP handler transforms](/turbo-pipelines/transforms/http-handler) | `httpauth` type (header name + value)                               | [Webhook sink § Secret creation](/turbo-pipelines/sinks/webhook#secret-creation)   |

### Example: PostgreSQL

Postgres is the most common case. Create the secret and paste a standard connection string when prompted:

```bash theme={"dark"}
goldsky secret create MY_POSTGRES_SECRET
```

```
postgres://goldsky_writer:your_secure_password@db.example.com:5432/your_database?sslmode=require
```

See the [Postgres sink](/turbo-pipelines/sinks/postgres) page for database role setup, provider-specific notes, and the inline JSON alternative.

<Tip>
  If you use [Goldsky-hosted Postgres](/turbo-pipelines/sinks/postgres), provisioning the database automatically registers a secret in your project. There is no separate `goldsky secret create` step.
</Tip>

## Mirror (legacy) secrets

[Mirror](/mirror/introduction) pipelines use a different format for database secrets: a JSON object with individual connection fields instead of a connection string. For example, a Mirror Postgres secret looks like:

```bash theme={"dark"}
goldsky secret create --name MY_POSTGRES_SECRET --value '{
  "type": "jdbc",
  "protocol": "postgresql",
  "host": "db.host.com",
  "port": 5432,
  "databaseName": "myDatabase",
  "user": "myUser",
  "password": "myPassword"
}'
```

When migrating a Mirror pipeline to Turbo, recreate database secrets in the Turbo connection-string format; see the [migration guide](/turbo-pipelines/migrate-from-mirror). Formats for Mirror-only sinks are documented on the surviving legacy pages, for example [Elasticsearch](/mirror/sinks/elasticsearch).

Mirror pipelines: the guided `goldsky pipeline create <pipeline-name>` flow lists your existing secrets and offers to create a new one as part of pipeline creation, so you don't need to create a secret beforehand.

## Troubleshooting

| Symptom                                     | Cause and fix                                                                                                                                                                                                                                              |
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Secret 'MY_SECRET' not found`              | The name doesn't exist or is misspelled. Run `goldsky secret list` and check the exact name.                                                                                                                                                               |
| `Secret 'MY_SECRET' already exists`         | The name is taken. Use `goldsky secret update MY_SECRET --value "..."` to change its value, or pick a different name.                                                                                                                                      |
| `Invalid JSON in secret value`              | Syntax error in a JSON-typed secret. Validate first with `echo '{...}' \| jq .` before creating.                                                                                                                                                           |
| Pipeline fails with `connection refused`    | Credentials are wrong or the database is unreachable. Verify outside Goldsky (`psql "postgresql://..."`), check the stored value with `goldsky secret reveal`, and make sure your database accepts connections from [Goldsky's IPs](/platform/static-ips). |
| Pipeline fails with `authentication failed` | Wrong username or password. Update the secret with corrected credentials.                                                                                                                                                                                  |
| Password contains special characters        | In connection strings, URL-encode special characters. In JSON-typed secrets the password is its own field, so most characters work as-is; escape backslashes (`\\`), quotes (`\"`), and newlines (`\n`).                                                   |


## Related topics

- [Wallets and Secrets](/compose/secrets.md)
- [Turbo CLI Reference](/turbo-pipelines/cli-reference.md)
- [Compose CLI Reference](/compose/cli-reference.md)
- [Turbo pipeline YAML configuration reference](/turbo-pipelines/pipeline-config.md)
- [CLI Reference](/reference/cli.md)