Skip to main content

Pipeline management

Apply a pipeline

Deploy a pipeline from a YAML configuration file:
This command will:
  • Validate your pipeline configuration
  • Create or update the pipeline in your current project
  • Start processing data according to your configuration
If a pipeline with the same name already exists, apply performs a server-side upsert that preserves checkpoints.

Options

To have a pipeline restart from scratch, you will have to rename the pipeline or rename the specific source node to avoid using the existing checkpoints.

List pipelines

View all pipelines in your current project:

Options

Get a pipeline

Fetch details for a single pipeline by name or YAML file:

Options

Delete a pipeline

Remove a pipeline by name:
Or delete using the YAML file:

Options

Validate a pipeline

Check your pipeline configuration without deploying:
This is useful for:
  • Catching syntax errors before deployment
  • Verifying source and sink configurations
  • Testing transform logic

Pause a pipeline

Temporarily stop a running pipeline:
Or pause using the YAML file:
This command sets deployment replicas to 0 or suspends jobs, preserving the pipeline state for later resumption.

Options

Resume a pipeline

Resume a previously paused pipeline:
Or resume using the YAML file:
This command restores the pipeline to its running state:
  • For deployments: restores the replica count from the original pipeline configuration (defaults to 1 if not specified)
  • For jobs: sets suspend to false

Options

You can only resume a paused pipeline. Attempting to resume an already running pipeline will return an error.

Restart a pipeline

Restart a running or paused pipeline by triggering a pod restart:
Or restart using the YAML file:
To clear all state data and start fresh from the beginning:

Options

This command triggers a pod restart for the pipeline. If the pipeline is paused, it will automatically resume before restarting.
Restart vs Resume: Use resume to restore a paused pipeline without restarting pods. Use restart when you need to trigger a fresh pod restart, such as after configuration changes or to recover from issues. The --clear-state flag allows you to discard all checkpoints and reprocess data from the beginning.
Restart is not supported for Job-mode pipelines. Use delete and apply to recreate the job instead.

Viewing logs

Monitor your pipeline’s execution with the logs command:

Options

Use -f / --follow to monitor your pipeline in real-time, especially useful when debugging or watching data flow through transforms. --follow is supported for streaming deployments; Job-mode pipelines emit their logs up to completion and then terminate (the pipeline is auto-deleted 1 hour after termination).

Inspecting live data

Open an interactive TUI to view live data flowing through your pipeline:

Options

The positional argument accepts either a pipeline name or a path to a YAML file (the name is read from the file’s name field).

Keyboard shortcuts

The TUI automatically reconnects if the pipeline is updated or temporarily unavailable, with a 30-minute timeout for persistent connection failures.
For detailed information about Live Inspect, including all keyboard shortcuts and features, see the Live Inspect guide.

Inspecting pipeline state

List checkpoint state entries for a pipeline:
This is primarily useful when debugging checkpoint-related behavior or confirming what state would be cleared by --clear-state.

Managing projects and secrets

The Turbo Pipelines CLI integrates with the Goldsky CLI for project management. Any command that is not a Turbo pipeline subcommand is forwarded to the goldsky CLI on your PATH.

Projects

To log into a project you will need to generate an API key and run the goldsky login command.

Secrets

Secrets are used to store sensitive configuration like database credentials:
Secrets are referenced in your pipeline YAML using the secret_name field. See the Pipeline Configuration guide for examples.