Skip to main content

​
Pipeline management

​
Apply a pipeline

Deploy a pipeline from a YAML configuration file: 
goldsky turbo apply my-pipeline.yaml
This command will:
  • Validate your pipeline configuration
  • Create or update the pipeline in your current project
  • Start processing data according to your configuration
If there were existing pipelines with the same name, the checkpoints from those pipelines will be reused.

​
Options

FlagDescription
-i, --inspectOpen the inspect TUI after successful deployment to monitor live data
# Deploy and immediately start inspecting live data
goldsky turbo apply my-pipeline.yaml -i
To have a pipeline restart from scratch, you will have to rename the pipeline or rename the specific source node to avoid using the existing checkpoints.

​
List pipelines

View all pipelines in your current project: 
goldsky turbo list

​
Delete a pipeline

Remove a pipeline by name: 
goldsky turbo delete my-pipeline
Or delete using the YAML file: 
goldsky turbo delete -f my-pipeline.yaml

​
Validate a pipeline

Check your pipeline configuration without deploying: 
goldsky turbo validate my-pipeline.yaml
This is useful for:
  • Catching syntax errors before deployment
  • Verifying source and sink configurations
  • Testing transform logic

​
Pause a pipeline

Temporarily stop a running pipeline: 
goldsky turbo pause my-pipeline
Or pause using the YAML file: 
goldsky turbo pause -f my-pipeline.yaml
This command sets deployment replicas to 0 or suspends jobs, preserving the pipeline state for later resumption.

​
Resume a pipeline

Resume a previously paused pipeline: 
goldsky turbo resume my-pipeline
Or resume using the YAML file: 
goldsky turbo resume -f my-pipeline.yaml
This command restores the pipeline to its running state:
  • For deployments: restores the replica count from the original pipeline configuration (defaults to 1 if not specified)
  • For jobs: sets suspend to false
You can only resume a paused pipeline. Attempting to resume an already running pipeline will return an error.

​
Restart a pipeline

Restart a running or paused pipeline by triggering a pod restart: 
goldsky turbo restart my-pipeline
Or restart using the YAML file: 
goldsky turbo restart -f my-pipeline.yaml
To clear all state data and start fresh from the beginning: 
goldsky turbo restart my-pipeline --clear-state

​
Options

FlagDescription
--clear-stateClear all state data to start processing from the beginning (default: false)
This command triggers a pod restart for the pipeline. If the pipeline is paused, it will automatically resume before restarting.
Restart vs Resume: Use resume to restore a paused pipeline without restarting pods. Use restart when you need to trigger a fresh pod restart, such as after configuration changes or to recover from issues. The --clear-state flag allows you to discard all checkpoints and reprocess data from the beginning.
Restart is not supported for Job-mode pipelines. Use delete and apply to recreate the job instead.

​
Viewing logs

Monitor your pipeline’s execution with the logs command: 
# View recent logs (default: last 10 lines)
goldsky turbo logs my-pipeline

# Show last 50 lines
goldsky turbo logs my-pipeline --tail 50

# Show logs from last hour
goldsky turbo logs my-pipeline --since 3600

# Include timestamps
goldsky turbo logs my-pipeline --timestamps
Use --follow to monitor your pipeline in real-time, especially useful when debugging or watching data flow through transforms.

​
Inspecting live data

Open an interactive TUI to view live data flowing through your pipeline: 
# Inspect all nodes in a pipeline
goldsky turbo inspect my-pipeline

# Filter to specific topology nodes
goldsky turbo inspect my-pipeline -n source1,transform1

# Adjust buffer size for more history
goldsky turbo inspect my-pipeline -b 50000

​
Options

FlagDescription
-n, --topology-node-keysComma-separated list of nodes to filter
-b, --buffer-sizeMaximum records to keep in buffer (default: 10000)

​
Keyboard shortcuts

KeyAction
Tab / ← →Switch between topology node tabs
j k / ↑ ↓Scroll up/down
g / GJump to top/bottom
/Search records
n / NNext/previous search match
dToggle pipeline definition view
wOpen in web dashboard
qQuit
The TUI automatically reconnects if the pipeline is updated or temporarily unavailable, with a 30-minute timeout for persistent connection failures.
For detailed information about Live Inspect, including all keyboard shortcuts and features, see the Live Inspect guide.

​
Managing projects and secrets

The Turbo Pipelines CLI integrates with the Goldsky CLI for project management

​
Projects

# List all projects you are a member of
goldsky project list
To log into a project you will need to generate an API key and run the golsky login command

​
Secrets

Secrets are used to store sensitive configuration like database credentials: 
# Create a secret
goldsky secret create MY_POSTGRES_SECRET

# List secrets
goldsky secret list

# Delete a secret
goldsky secret delete MY_POSTGRES_SECRET
Secrets are referenced in your pipeline YAML using the secret_name field. See the Pipeline Configuration guide for examples.