Overview
Turbo pipelines are built on Streamling, Goldsky’s next generation data pipeline engine, offering significant performance improvements and new capabilities, including inter-batch parallelism and startups under 5 seconds. Every team can now have the same efficiencies as highly tuned and single-purpose pipelines built by specialist teams, working with a simple set of SQL and Typescript transforms to define their own logic.Solana datasets are exclusively available for Turbo pipelines and not available for Standard pipelines (Mirror v1 pipelines).
Key Improvements Over Mirror
Live Data Inspection
Use our
inspect command to see data flowing through your live pipeline with no performance penalty.Solana Support
Access Solana datasets including transactions, instructions, and token transfers. Live and historical data available.
Infinite Filters
Filter on unlimited addresses, updated from on-chain data or backend updates.
Really Fast Decoding
Create your own real-time streams with our decoding functions, using your own ABI or IDL.
Pipeline Structure
Like the previous engine, Turbo pipelines consist of three components:- Sources: Where data comes from (datasets)
- Transforms: How data is processed (SQL, HTTP handlers, TypeScript)
- Sinks: Where data is written (PostgreSQL, ClickHouse, webhooks, Kafka, etc.)
Use Cases
- Real-time Data Pipelines: Stream blockchain data to databases and warehouses
- Data Transformation: Apply SQL or custom transformations to data before it hits your sink
- Event Processing: Process and route events based on business logic
- Data Integration: Connect multiple data sources and sinks to get the data you need in the shape you need it