Overview
Goldsky is a high-performance data indexing provider for Fogo that makes it easy to extract, transform, and load on-chain data to power both application and analytics use cases via Turbo Pipelines (real-time data replication pipelines). Fogo is a purpose-built Layer 1 blockchain designed for high-performance trading with sub-40ms blocks, sub-second confirmation, and SVM L1 compatibility. Built on a custom Firedancer client, Fogo delivers the speed and reliability demanded by modern finance.Getting started
To use Goldsky, you’ll need to create an account, install the CLI, and log in.Install Goldsky's CLI and log in
Install Goldsky's CLI and log in
-
Install the Goldsky CLI:
For macOS/Linux:
For Windows:Windows users need to have Node.js and npm installed first. Download from nodejs.org if not already installed.
- Go to your Project Settings page and create an API key.
-
Back in your Goldsky CLI, log into your Project by running the command
goldsky loginand paste your API key. -
Now that you are logged in, run
goldskyto get started:
Turbo Pipelines
Turbo pipelines allow users to replicate data into their own infrastructure (any of the supported sinks) in real time. For a complete overview of how to deploy Turbo pipelines, including a video walkthrough, check the Quickstart guide.Working with Fogo datasets
Goldsky provides real-time streaming of Fogo datasets, including all historical data, for both mainnet and testnet. The following datasets are currently available:| Dataset | Description |
|---|---|
| Transactions with Instructions | Enriched transaction data including instructions, accounts, balance changes, and metadata for the block. |
| Rewards | Records of rewards distributed to validators for securing and validating the network. |
| Blocks | Metadata for each block on the chain including hashes, transaction count, slot and leader rewards. |
Deploying Fogo pipelines
Turbo pipelines are defined using YAML configuration files and deployed via the Goldsky CLI. Here’s the workflow:- Create a pipeline configuration file - Define your sources, transforms, and sinks in a YAML file
- Validate your configuration - Run
goldsky turbo validate fogo-pipeline.yamlto check for errors - Deploy the pipeline - Run
goldsky turbo apply -f fogo-pipeline.yamlto deploy - Monitor your pipeline - Use
goldsky turbo logs fogo-pipeline.yamlto view logs andgoldsky turbo inspect fogo-pipeline.yamlto see live data
Remember to first create a Secret in order for Turbo Pipelines to be able to write the data into the database of your choice.
Example pipeline configuration
Here’s an example configuration file for streaming Fogo transactions with instructions:fogo-transactions.yaml
start_block parameter allows you to specify exactly where your pipeline should begin processing data:
- Omit
start_blockto start from the latest block (recommended for new pipelines) - Set a specific block number (e.g.,
start_block: 1000000) to process from that block forward - Use for historical processing when you need data from a specific point in time
goldsky turbo apply -f fogo-transactions.yaml to deploy the pipeline.