Skip to main content

Overview

Live Inspect provides an interactive terminal user interface (TUI) for monitoring and debugging your pipelines. View live data samples as they flow through each topology node (sources, transforms, and sinks) with features like multi-tab navigation, search, and scrolling.
Use Live Inspect when debugging pipeline logic, verifying transform outputs, or understanding data flow through complex pipelines.

Basic usage

Inspect live data from a running pipeline:
Or use a pipeline configuration file:
This opens an interactive TUI that displays live data organized by topology node, with real-time updates as new records arrive.

Command options

string
required
Pipeline name or path to YAML configuration file
string
Comma-separated list of topology node keys to filter. Only shows data from the specified nodes. Example: -n source1,transform1
integer
default:"10000"
Maximum number of records to keep in the TUI buffer. Older records are removed when this limit is reached.
boolean
default:"false"
Print records to stdout instead of opening the TUI. Useful for piping data to other tools or redirecting to files.

TUI interface

The inspect command opens an interactive terminal interface with the following features:

Multi-tab navigation

Data is organized into tabs by topology node. The All tab shows records from all nodes, while individual tabs show records from specific sources, transforms, or sinks. Tabs appear dynamically as data arrives from each node.

Scrolling

Navigate through records using keyboard or mouse:
Hold Shift while using the mouse to select and copy text from the TUI.
Search across all visible records with highlighted matches:

Pipeline definition

Press d to toggle the pipeline definition view. On the All tab, this shows the complete pipeline YAML. On individual node tabs, it shows only that node’s configuration.

Additional shortcuts

Filtering topology nodes

By default, Live Inspect shows data from all nodes in your pipeline. Filter to specific nodes using the -n flag:
Use filtering to focus on specific parts of your pipeline when debugging complex data flows.

Auto-reconnection

The TUI automatically reconnects when:
  • The pipeline is updated, paused, or resumed
  • The connection is temporarily lost
  • The pipeline is stopped and restarted
While waiting to reconnect, the TUI displays a “Reconnecting…” status and preserves previously received data. The status bar shows the time since the last record was received.
The TUI has a 30-minute connection timeout. If the pipeline remains unreachable for 30 minutes, the TUI automatically closes with a message suggesting you check the pipeline status.

Buffer management

The TUI maintains a buffer of recent records for scrolling and searching. By default, it keeps the last 10,000 records. Adjust this with the -b flag:
When the buffer is full, older records are removed as new ones arrive. Use the --print flag to stream records directly to stdout instead of opening the TUI. This is useful for:
  • Piping data to other tools like jq for filtering or transformation
  • Redirecting output to files for later analysis
  • Integrating with shell scripts or automation workflows
In print mode:
  • Records are pretty-printed as JSON to stdout
  • Status messages (like “Streaming live data…”) are sent to stderr
  • Press Ctrl+C to stop streaming
  • Each record is separated by a line of dashes for readability
Combine --print with -n to filter to specific topology nodes before piping to other tools.

Example

Inspect a pipeline and filter to a specific transform:
The TUI displays:
  • Pipeline name and status in the header
  • Tabs for each topology node receiving data
  • Live records with JSON pretty-printing
  • Status bar with last record time and keyboard shortcuts

Use cases

Use Live Inspect to verify that your SQL transforms are filtering and transforming data correctly. Switch between tabs to compare input and output of each transform.
Quickly verify that your sources are producing data in the expected format. Use search to find specific records or patterns.
After deploying with goldsky turbo apply -i, the TUI opens automatically so you can verify data is flowing correctly.

Troubleshooting

If you don’t see any data:
  • Verify your pipeline is running: goldsky turbo list
  • Check that data is flowing: goldsky turbo logs my-pipeline
  • The TUI shows “Waiting for records…” until data arrives
  • If the pipeline is starting, the TUI will auto-reconnect when it becomes active
If you see “Pipeline not found”, verify the pipeline name with goldsky turbo list or check that your YAML file path is correct.