> ## Documentation Index
> Fetch the complete documentation index at: https://docs.goldsky.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Turbo - Supported sinks

> Write Turbo pipeline output to PostgreSQL, ClickHouse, Kafka, S3, SQS, webhooks, and more, with shared batch_size and batch_flush_interval controls.

## Overview

Sinks are the final destination for data in your Turbo pipelines. They write processed data to external systems like databases, data warehouses, or HTTP endpoints.

## Available Sinks

<CardGroup cols={3}>
  <Card title="PostgreSQL" icon="database" href="/turbo-pipelines/sinks/postgres">
    Write to PostgreSQL databases
  </Card>

  <Card title="PostgreSQL aggregation" icon="calculator" href="/turbo-pipelines/sinks/postgres-aggregate">
    Real-time aggregations in PostgreSQL
  </Card>

  <Card title="ClickHouse" icon="chart-line" href="/turbo-pipelines/sinks/clickhouse">
    Write to ClickHouse for analytics
  </Card>

  <Card title="MySQL" icon="database" href="/turbo-pipelines/sinks/mysql">
    Write to MySQL databases
  </Card>

  <Card title="Webhook" icon="webhook" href="/turbo-pipelines/sinks/webhook">
    Send data to HTTP endpoints
  </Card>

  <Card title="Kafka" icon="cloud" href="/turbo-pipelines/sinks/kafka">
    Publish to Kafka topics
  </Card>

  <Card title="S3" icon="archive" href="/turbo-pipelines/sinks/s3">
    Write to S3-compatible storage
  </Card>

  <Card title="SQS" icon="aws" href="/turbo-pipelines/sinks/sqs">
    Send to Amazon SQS queues
  </Card>

  <Card title="Google Cloud Pub/Sub" icon="google" href="/turbo-pipelines/sinks/pubsub">
    Publish to Google Cloud Pub/Sub topics
  </Card>

  <Card title="S2" icon="waves" href="/turbo-pipelines/sinks/s2">
    Publish to S2.dev streams
  </Card>

  <Card title="Blackhole" icon="ban" href="/turbo-pipelines/sinks/blackhole">
    Discard data for testing purposes
  </Card>
</CardGroup>

## Common Parameters

Every sink accepts at least these parameters. See each sink's page for
type-specific fields.

<ParamField path="type" type="string" required>
  The sink type — for example `postgres`, `clickhouse`, `kafka`, `webhook`,
  `mysql_sink`, `s3_sink`, `sqs_sink`, `pubsub`, `s2_sink`, `blackhole`.
</ParamField>

<ParamField path="from" type="string" required>
  Name of the transform or source to read data from.
</ParamField>

<ParamField path="secret_name" type="string">
  Name of the Goldsky secret containing connection credentials. Recommended for
  any sink that connects to an external system (databases, Kafka, S3, SQS,
  webhooks, etc.). Some sinks also accept inline credentials — see each sink's
  page for details.
</ParamField>

<ParamField path="primary_key" type="string">
  Column (or comma-separated list of columns) used to identify unique rows.
  Required for upserts in database sinks and required (not optional) on the
  ClickHouse sink.
</ParamField>

<ParamField path="batch_size" type="integer">
  Maximum number of rows the Turbo runtime buffers before flushing a batch to
  the sink. Applies to every sink — batching is a property of the runtime, not
  of the individual sink. Individual sink pages may document defaults or upper
  limits that make sense for that destination (for example the Kafka sink also
  exposes librdkafka-level `batch_size`).
</ParamField>

<ParamField path="batch_flush_interval" type="string">
  Maximum time the Turbo runtime waits before flushing a batch, even if
  `batch_size` has not been reached. Applies to every sink. Accepts humantime
  durations such as `500ms`, `1s`, or `30s`. A bare number is interpreted as
  milliseconds (for example `"1000"` is equivalent to `1s`).
</ParamField>

## Batching

Every Turbo sink accepts `batch_size` and `batch_flush_interval` — they are
handled by the Turbo runtime, so they work on every sink even when a specific
sink's page does not list them explicitly. Rows accumulate in an in-memory
buffer and flush to the sink whenever either threshold is reached, whichever
comes first.

* **Larger batches** — increase throughput and reduce write amplification, at
  the cost of higher end-to-end latency.
* **Smaller batches / shorter intervals** — reduce latency, useful when a
  downstream system needs to react to rows quickly.
* **Defaults** — `batch_size` defaults to `1000` rows and `batch_flush_interval`
  defaults to `1s`. Individual sinks may override these; see each sink's page
  for destination-specific guidance.

## Multiple Sinks

You can write the same data to multiple destinations:

```yaml theme={null}
transforms:
  processed_data:
    type: sql
    primary_key: id
    sql: SELECT * FROM source

sinks:
  # Write to PostgreSQL
  postgres_archive:
    type: postgres
    from: processed_data
    schema: public
    table: archive
    secret_name: MY_POSTGRES

  # Send to webhook
  webhook_notification:
    type: webhook
    from: processed_data
    url: https://api.example.com/notify
    secret_name: NOTIFY_API_SECRET

  # Publish to Kafka
  kafka_downstream:
    type: kafka
    from: processed_data
    topic: processed.events
```

Each sink writes its own copy of the data. Because all sinks share the same
checkpoint barrier, a sink that is slow or stalled will apply backpressure to
the whole pipeline (see below).

## Sink Behavior

### Checkpointing

All sinks participate in Turbo's checkpointing system:

* Each sink buffers writes and acknowledges only after it has flushed them
* Sources only commit their position after every sink has acknowledged
* This gives at-least-once delivery — duplicates are possible after a restart,
  but no committed data is lost

### Backpressure

Sinks apply backpressure to the pipeline:

* If any sink can't keep up, the entire pipeline slows down
* Prevents data loss and memory overflow
* Monitor sink performance to identify bottlenecks

### Error Handling

Sink errors are retried with exponential backoff. Transient errors (network
blips, connection resets) recover automatically. Persistent errors (bad
credentials, schema mismatches, unreachable destination) will keep retrying and
show up as stalled checkpoints — check the pipeline logs for the underlying
error message.

## Best Practices

<AccordionGroup>
  <Accordion title="Choose the right sink for your use case">
    * **PostgreSQL**: Transactional data, updates, relational queries
    * **PostgreSQL aggregation**: Real-time aggregations like balances, totals, counts
    * **ClickHouse**: High-volume analytics, aggregations, time-series
    * **MySQL**: Transactional data, updates, relational queries
    * **Webhook**: Real-time notifications, integrations with external systems
    * **Kafka**: Downstream processing, event sourcing, decoupling systems
    * **SQS**: Event-driven architectures, decoupled integrations, message queuing
    * **Pub/Sub**: Fan-out to many GCP consumers, event-driven workloads on Google Cloud
    * **S2**: Decoupled processing, large number of readers, serverless architectures
  </Accordion>

  <Accordion title="Use appropriate primary keys">
    For upsert behavior in databases, choose a stable primary key:

    ```yaml theme={null}
    sinks:
      postgres_sink:
        primary_key: id  # Use a unique, stable identifier
    ```
  </Accordion>

  <Accordion title="Monitor sink performance">
    Use logs and metrics to track:

    * Write throughput
    * Error rates
    * Latency

    ```bash theme={null}
    goldsky turbo logs my-pipeline
    ```
  </Accordion>

  <Accordion title="Secure credentials">
    Always use secrets for database credentials:

    ```bash theme={null}
    # Create secret
    goldsky secret create MY_DB_SECRET

    # Reference in pipeline
    sinks:
      my_sink:
        secret_name: MY_DB_SECRET
    ```
  </Accordion>
</AccordionGroup>


## Related topics

- [Supported Mirror pipeline sinks and destinations](/mirror/sinks/supported-sinks.md)
- [PostgreSQL sink for Turbo pipelines](/turbo-pipelines/sinks/postgres.md)
- [Mirror vs. Turbo pipelines](/mirror-vs-turbo.md)
- [Supported Mirror pipeline sources](/mirror/sources/supported-sources.md)
- [Supported networks](/chains/supported-networks.md)
