> ## Documentation Index > Fetch the complete documentation index at: https://docs.goldsky.com/llms.txt > Use this file to discover all available pages before exploring further. # Throttle transform > Cap streaming throughput with the throttle transform: configure max_batch_size and min_batch_interval defaults, pacing algorithm, and backpressure. ## Overview The throttle transform caps the throughput of a stream by buffering records into batches and emitting each batch on a fixed minimum interval. Use it to: * Stay under rate limits of downstream sinks or external APIs * Smooth out bursty sources into a steady, predictable rate * Test sink behavior at a controlled records-per-second rate * Reduce pressure on small resource sizes during development Throttle does not modify the data: every input record passes through unchanged. It only controls *when* records are emitted. ## Configuration ```yaml theme={"dark"} transforms: my_throttle: type: throttle from: max_batch_size: 100 min_batch_interval: 10s ``` ### Parameters Must be `throttle` The source or transform to read data from Row budget per `min_batch_interval`. Must be greater than `0`. Defaults to `100`. Minimum time between emissions, in `humantime` format (e.g., `500ms`, `1s`, `2m`). Must be greater than `0`. Defaults to `1s`. Invalid values (`max_batch_size: 0`, an unparseable duration, etc.) fail at pipeline start with a configuration error. ## How throttling works Batches arrive from the upstream source or transform and pass through unchanged. The throttle paces *when* each batch is released so that the long-run row rate does not exceed `max_batch_size` rows per `min_batch_interval`. For each incoming batch of `rows` rows: 1. If the schedule says the throttle is not yet eligible to emit, it sleeps until it is. 2. The batch is emitted downstream, unchanged. 3. The next-eligible time advances by: ``` cost = max(min_batch_interval, ceil(rows / max_batch_size) * min_batch_interval) ``` Batches are never split and never dropped. An oversized batch passes through whole and then "pays" for itself on the schedule, so the long-run rate is preserved. The effective maximum throughput is approximately: ``` max_batch_size / min_batch_interval = rows per second ``` Examples with `max_batch_size: 100`, `min_batch_interval: 1s`: | Incoming batch sizes | Behavior | | ---------------------------- | ----------------------------------------------------- | | `50, 50, 50, …` | One batch per second. Average rate: 50 rows/s. | | `100, 100, 100, …` | One batch per second. Average rate: 100 rows/s. | | `1000, 1, 1, …` | First emits immediately; the next is delayed **10s**. | | `200, 1, 1, …` | First emits immediately; the next is delayed 2s. | | Idle for 1 minute, then `50` | Emits immediately (a long idle resets the schedule). | "Rows per second" here is the **average throughput** the downstream system needs to handle, not literal requests or messages per second. The throttle emits one batch per interval; sinks consume that batch in whatever way is natural for them. For example, an S3 sink writes one file per interval at the configured batch size. Throttle limits the *maximum* rate, not the minimum. If the upstream is slow, batches will be smaller and arrive less frequently. Empty batches pass through immediately and do not advance the schedule. ### Backpressure When the throttle is sleeping, its input slot fills up and the upstream stage's send blocks. That stall walks all the way back to the source, so no batches are dropped and in-flight memory stays bounded. Shutdown is not delayed by a long `min_batch_interval`; an in-flight sleep is cancelled when the pipeline terminates. ## Example Throttle a high-volume ERC-20 transfer stream down to \~10 rps before sending it to a sink: ```yaml theme={"dark"} name: throttle_example resource_size: s use_dedicated_ip: false job: false sources: erc20s: type: dataset dataset_name: matic.erc20_transfers version: 1.2.0 start_at: latest transforms: throttled_erc20s: type: throttle from: erc20s max_batch_size: 100 # ~10 rps with a 10s interval min_batch_interval: 10s sinks: sink_1: type: blackhole from: throttled_erc20s ``` ## When to use throttle * **Rate-limited sinks**: Stay under per-second write quotas on downstream APIs or databases. * **External handler protection**: Pace records into an [HTTP handler](/turbo-pipelines/transforms/http-handler) so the receiving service is not overwhelmed. * **Cost control during development**: Slow down processing while iterating on a pipeline against a live source. * **Testing**: Reproduce sink behavior under a known, fixed input rate. ## Best Practices Throttle the stream just before the rate-limited sink or handler so upstream transforms still process at full speed. Larger `max_batch_size` reduces per-batch overhead but increases latency per record. Pick a size that matches your sink's preferred batch size. Throttle caps throughput by design. Once rate-limit concerns are addressed, remove the transform to let the pipeline run at full speed. ## Related topics - [Pipeline cookbook](/turbo-pipelines/reference/pipeline-cookbook.md) - [Turbo transforms overview](/turbo-pipelines/transforms/overview.md) - [Migrate from Mirror to Turbo](/turbo-pipelines/migrate-from-mirror.md) - [Proof-of-reserves & treasury intelligence](/solutions/treasury-and-reserves.md) - [SQL transforms](/turbo-pipelines/transforms/sql.md)