How to stream data from any EVM chain over RPC

Overview

Edge EVM datasets (edge.evm.*) stream blockchain data directly from an EVM JSON-RPC endpoint while your pipeline runs. Instead of reading from Goldsky’s pre-indexed datasets, the pipeline runs its own fetcher that pulls block bodies, receipts, and traces over batched JSON-RPC and reshapes them into consistent, typed datasets. This makes Edge EVM sources the right choice when you want to:

Index a chain that Goldsky doesn’t have a curated dataset for yet — any EVM-compatible chain works.
Use your own RPC node, or a specific provider you already pay for.
Get data at the chain tip with minimal indexing infrastructure between the node and your pipeline.

You connect the source to a chain in one of two ways:

Goldsky Edge endpoints — set chain_id (or chain_slug) and Goldsky routes the source through Edge RPC, our managed multi-provider RPC infrastructure. No endpoint to manage.
Bring your own RPC — set rpc_url to any HTTP(S) JSON-RPC endpoint.

Edge EVM datasets are distinct from the chain-prefixed datasets like ethereum.raw_logs documented on the EVM Sources page. Chain-prefixed datasets read from Goldsky’s pre-indexed data and support fast scan filters; Edge EVM datasets fetch from an RPC at pipeline runtime and work on any EVM chain. See Edge EVM vs curated EVM datasets below for guidance.

Quick start

Stream Ethereum mainnet logs through a Goldsky Edge endpoint:

name: edge-evm-quickstart
resource_size: s

sources:
  eth_logs:
    type: dataset
    dataset_name: edge.evm.logs
    version: 1.0.0
    start_at: latest
    chain_id: 1 # Ethereum mainnet, served by Goldsky Edge

sinks:
  postgres_logs:
    type: postgres
    from: eth_logs
    schema: public
    table: eth_logs
    secret_name: MY_POSTGRES
    primary_key: id

Connecting to a chain

Every Edge EVM source requires exactly one of the following options. Setting none, or more than one, fails validation at deploy time.

Option	Example	Behavior
`chain_id`	`chain_id: 1`	Route through a Goldsky-provided Edge endpoint, selected by numeric chain ID.
`chain_slug`	`chain_slug: mainnet`	Route through a Goldsky-provided Edge endpoint, selected by chain alias (`mainnet` is Ethereum mainnet’s alias).
`rpc_url`	`rpc_url: https://eth.example.com`	Bring your own RPC: fetch from the HTTP(S) JSON-RPC endpoint you provide.

Goldsky Edge endpoints

When you set chain_id or chain_slug, the source fetches through Goldsky’s Edge RPC infrastructure — multi-provider failover, archive access, and request batching tuned for indexing workloads, with nothing for you to operate. The chain selector is validated against the Edge API when the pipeline starts, so a typo’d chain ID or slug fails loudly with the list of available chains instead of silently producing no data. See the supported networks page for the chains Edge serves, or fetch edge.goldsky.com directly for a programmatic JSON list of every chain with its ID and alias.

sources:
  eth_blocks:
    type: dataset
    dataset_name: edge.evm.blocks
    version: 1.0.0
    start_at: latest
    chain_id: 1 # or equivalently: chain_slug: mainnet

Bring your own RPC

When you set rpc_url, the source fetches from your endpoint instead. Any HTTP(S) JSON-RPC endpoint works — a node you run yourself or a third-party provider.

sources:
  eth_transactions:
    type: dataset
    dataset_name: edge.evm.receipt_transactions
    version: 1.0.0
    start_at: latest
    rpc_url: https://your-rpc-endpoint.example.com

Your endpoint needs to support the following methods:

Datasets	Required RPC methods
`blocks`, `full_block`, `logs`, `receipt_transactions`	`eth_getBlockByNumber`, plus `eth_getBlockReceipts` or `eth_getTransactionReceipt`
`traces`, `traces_by_transaction`, `traces_by_block`	`debug_traceBlockByNumber` or `debug_traceTransaction` (Geth `callTracer`)

Method detection is automatic. If the endpoint doesn’t implement eth_getBlockReceipts, the source transparently falls back to per-transaction eth_getTransactionReceipt calls; the trace datasets likewise fall back from per-block to per-transaction tracing. No configuration is needed, but note that per-transaction fallbacks issue one sub-request per transaction instead of one per block — size your provider quota accordingly on chains with high transaction counts.

Historical backfills (start_at: earliest or an old block number) require an archive node. The trace datasets additionally require a node with the debug API enabled — many public endpoints don’t expose it.

Available datasets

All Edge EVM datasets are version 1.0.0 and share consistent column names and types on every chain.

Dataset	One row per	Primary key
`edge.evm.blocks`	Block	`number`
`edge.evm.full_block`	Block, with all receipts (transactions + logs) nested	`block_number`
`edge.evm.receipt_transactions`	Transaction, merged with its receipt	`id` = `<block_hash>_<transaction_index>`
`edge.evm.logs`	Event log	`id` = `<block_hash>_<log_index>`
`edge.evm.traces`	Trace (internal call), flat	`id` = `<block_hash>_<trace_index>`
`edge.evm.traces_by_transaction`	Transaction, with its traces nested	`id` = `<block_hash>_<transaction_index>`
`edge.evm.traces_by_block`	Block, with transactions and their traces nested	`id` = `<block_hash>`

Sources in the same pipeline that point at the same chain and block range share one underlying fetcher, so adding a second or third edge.evm.* dataset does not multiply your RPC usage. In particular, edge.evm.blocks, edge.evm.logs, edge.evm.receipt_transactions, and edge.evm.full_block are all served by the same block + receipts fetch. The trace datasets use a separate trace fetcher, which the three trace variants likewise share.

Dataset schemas

Every dataset also carries the _gs_op (string) operation column used for delivery guarantees: c for inserts and d for deletes issued when a reorg orphans previously emitted rows. Timestamps (block_timestamp / timestamp) are Unix seconds. uint256 columns hold full 256-bit values (wei amounts) without precision loss.

edge.evm.blocks

Flat block headers, one row per block.

Column	Type
`number`	long
`hash`	string
`parent_hash`	string
`nonce`	string
`sha3_uncles`	string
`logs_bloom`	string
`transactions_root`	string
`state_root`	string
`receipts_root`	string
`miner`	string
`difficulty`	uint256
`total_difficulty`	uint256
`size`	long
`extra_data`	string
`gas_limit`	long
`gas_used`	long
`timestamp`	long
`transaction_count`	long
`base_fee_per_gas`	long
`withdrawals_root`	string
`blob_gas_used`	long
`excess_blob_gas`	long

edge.evm.full_block

One row per block: every column from edge.evm.blocks (with block_number / block_hash / block_timestamp naming), plus a nested receipts array. Each receipt struct merges the transaction fields (from, to, value, gas, input, nonce, fee fields) with its receipt fields (status, gas_used, cumulative_gas_used, contract_address, effective_gas_price, blob and OP Stack L1 fee fields) and nests the transaction’s logs (each with address, topics, data, log_index, …).Use this dataset when you want one self-contained row per block to shred yourself in a transform; it costs no additional RPC requests over edge.evm.blocks.

edge.evm.receipt_transactions

One row per transaction, merging transaction and receipt fields.

Column	Type
`id`	string
`block_number`	long
`block_hash`	string
`block_timestamp`	long
`hash`	string
`transaction_index`	long
`from_address`	string
`to_address`	string
`receipt_gas_used`	long
`receipt_cumulative_gas_used`	long
`receipt_contract_address`	string
`receipt_status`	long
`transaction_type`	long
`receipt_effective_gas_price`	long
`logs_bloom`	string
`blob_gas_used`	long
`blob_gas_price`	long
`receipt_root_hash`	string
`nonce`	long
`value`	uint256
`gas`	long
`gas_price`	long
`input`	string
`max_fee_per_gas`	long
`max_priority_fee_per_gas`	long
`blob_versioned_hashes`	array<string>
`max_fee_per_blob_gas`	long
`receipt_l1_fee`	uint256
`receipt_l1_gas_used`	uint256
`receipt_l1_gas_price`	uint256
`receipt_l1_fee_scalar`	uint256
`receipt_l1_blob_base_fee`	uint256
`receipt_l1_blob_base_fee_scalar`	long
`receipt_l1_block_number`	long
`receipt_l1_base_fee_scalar`	long

The receipt_l1_* columns carry OP Stack L1 fee data and are null on chains that don’t emit them.

edge.evm.logs

One row per event log.

Column	Type
`id`	string
`block_number`	long
`block_hash`	string
`block_timestamp`	long
`transaction_hash`	string
`transaction_index`	long
`log_index`	long
`address`	string
`data`	string
`topics`	array<string>
`removed`	boolean
`from_address`	string
`to_address`	string
`transaction_from_address`	string
`transaction_to_address`	string
`input`	string

from_address / to_address (and the transaction_* aliases) are the sender and recipient of the transaction that emitted the log; input is that transaction’s calldata.

Unlike the curated raw_logs datasets, where topics is a single comma-separated string, topics here is a proper array — access the event signature with topics[1] instead of SPLIT_INDEX(topics, ',', 0).

edge.evm.traces

One row per trace (internal call), in the flat ethereum-etl style.

Column	Type
`id`	string
`type`	string
`transaction_index`	long
`from_address`	string
`to_address`	string
`value`	uint256
`input`	string
`output`	string
`trace_type`	string
`call_type`	string
`reward_type`	string
`gas`	long
`gas_used`	long
`subtraces`	long
`trace_address`	array<long>
`error`	string
`status`	long
`transaction_hash`	string
`block_number`	long
`trace_index`	long
`block_timestamp`	long
`block_hash`	string
`tx_from_address`	string
`tx_to_address`	string

status is 1/0 and follows the ethereum-etl failure cascade: a trace is 0 if its own frame failed or any ancestor in the same transaction failed (a reverted transaction marks its whole tree 0). Blocks with zero traces (e.g. empty blocks) produce no rows.

edge.evm.traces_by_transaction

One row per transaction with its traces nested.

Column	Type
`id`	string
`block_number`	long
`block_hash`	string
`block_timestamp`	long
`transaction_hash`	string
`transaction_index`	long
`tx_from_address`	string
`tx_to_address`	string
`success`	boolean
`traces`	array<struct>

The top-level success is the transaction outcome. Each nested trace struct carries type, from_address, to_address, value (uint256), input, output, trace_type, call_type, reward_type, gas, gas_used, subtraces, trace_address, error, trace_index, and its own per-frame success (no cascade applied — a successful sub-call of a reverted transaction stays true, mirroring Dune’s ethereum.traces).

edge.evm.traces_by_block

One row per block with a nested transactions array.

Column	Type
`id`	string
`block_number`	long
`block_hash`	string
`block_timestamp`	long
`transactions`	array<struct>

Each transaction struct carries transaction_hash, transaction_index, tx_from_address, tx_to_address, the transaction outcome success, and its own nested traces array (same trace struct as edge.evm.traces_by_transaction). Blocks with zero traces produce no rows.

Controlling the block range

start_at accepts latest, earliest, or a specific block number:

sources:
  eth_logs:
    type: dataset
    dataset_name: edge.evm.logs
    version: 1.0.0
    start_at: "18000000" # latest | earliest | a specific block number
    chain_id: 1

To bound the range for a backfill, pair start_at with end_at (a numeric block number). Once the source passes end_at it stops fetching and drains its remaining output to the sinks:

name: eth-logs-backfill
resource_size: m

sources:
  eth_logs:
    type: dataset
    dataset_name: edge.evm.logs
    version: 1.0.0
    start_at: "18000000"
    end_at: 18100000 # inclusive upper bound
    chain_id: 1

sinks:
  s3_logs:
    type: s3
    from: eth_logs
    bucket: my-data-lake
    prefix: eth/logs/
    secret_name: MY_S3

end_at must be a numeric block number — unlike start_at, it does not accept latest or earliest. end_block is not accepted on Edge EVM dataset sources; the deploy fails with a pointer to end_at if you use it.

A bounded Edge EVM pipeline does not support job mode proper: there is no automatic shutdown. After the final block is processed, the pipeline stays alive in an idle state so downstream sinks can flush their final batches. Check goldsky turbo logs <pipeline-name> for the “bounded backfill complete” message to confirm the backfill has finished, verify the data landed, then run goldsky turbo delete <pipeline-name> to release resources.

Reorg handling

The source tracks the chain’s hash-linked history in a rewind buffer (the most recent rewind_buffer_depth blocks, 6 by default). When a reorg orphans blocks it already emitted, the pipeline issues deletes (_gs_op = 'd') for every row of each orphaned block and re-emits the canonical replacement rows — sinks keyed on the primary key converge to the canonical chain automatically. If you’d rather avoid reorg churn entirely, set confirmation_depth to trail the chain tip by a fixed number of blocks:

sources:
  eth_blocks:
    type: dataset
    dataset_name: edge.evm.blocks
    version: 1.0.0
    start_at: latest
    chain_id: 1
    confirmation_depth: 12 # only emit blocks 12 behind the tip

Tuning options

All options sit directly on the source alongside the chain selector. Defaults are tuned for chain-tip streaming; raise the fetch options for faster backfills.

Option	Default	Description
`confirmation_depth`	`0`	Emit blocks only once they are this many blocks behind the tip.
`polling_interval_ms`	`1000`	How often to poll for new blocks at the chain tip.
`fetch_batch_size`	`10`	Blocks fetched per batched JSON-RPC request during backfill.
`fetch_parallelism`	`1`	Concurrent fetch requests during backfill.
`max_blocks_per_batch`	`1`	Blocks emitted per output batch.
`batch_interval_ms`	`100`	Maximum wait before flushing a partial output batch.
`rewind_buffer_depth`	`6`	Recent blocks tracked for reorg detection and deletes.
`request_timeout_secs`	`10`	Per-request RPC timeout.
`ws_url`	—	Optional WebSocket endpoint; the source subscribes to `newHeads` for lower-latency tip detection instead of relying on polling alone.
`verify_consensus`	`false`	Verify block-header hashes and transaction/receipt trie roots against the returned data — useful when streaming from an RPC provider you don’t fully trust.

RPC usage: the block datasets cost roughly two batched sub-requests per block (one eth_getBlockByNumber, one eth_getBlockReceipts); the trace datasets add one debug_traceBlockByNumber per block. Provider fallbacks to per-transaction methods multiply sub-requests by the block’s transaction count. If you bring your own RPC, size rate limits with this in mind.

Guide: multi-chain monitoring with mixed endpoints

Combine a Goldsky Edge endpoint and your own RPC in one pipeline:

name: multi-chain-edge-blocks
resource_size: s

sources:
  ethereum_blocks:
    type: dataset
    dataset_name: edge.evm.blocks
    version: 1.0.0
    start_at: latest
    chain_id: 1 # Ethereum via Goldsky Edge

  base_blocks:
    type: dataset
    dataset_name: edge.evm.blocks
    version: 1.0.0
    start_at: latest
    rpc_url: https://base-node.internal.example.com # your own Base node

transforms:
  all_blocks:
    type: sql
    primary_key: id
    sql: |
      SELECT CONCAT('ethereum_', hash) AS id, 'ethereum' AS chain, number, hash, timestamp, gas_used, _gs_op
      FROM ethereum_blocks
      UNION ALL
      SELECT CONCAT('base_', hash) AS id, 'base' AS chain, number, hash, timestamp, gas_used, _gs_op
      FROM base_blocks

sinks:
  clickhouse_blocks:
    type: clickhouse
    from: all_blocks
    table: multi_chain_blocks
    primary_key: id
    secret_name: MY_CLICKHOUSE

Edge EVM vs curated EVM datasets

	Edge EVM (`edge.evm.*`)	Curated (`<chain>.<dataset>`)
Chains	Any EVM chain	Supported chains only
Data path	Fetched from an RPC at runtime	Goldsky’s pre-indexed datasets
RPC endpoint	Goldsky Edge (`chain_id` / `chain_slug`) or your own (`rpc_url`)	None needed
Backfill speed	Bounded by RPC throughput; archive node required	Fast; fast scan filters supported
Traces	Requires a `debug`-enabled node	Included where `raw_traces` exists
Curated datasets (e.g. `erc20_transfers`)	Not available — decode in transforms	Available
Bounded backfills	`start_at` + `end_at` (no automatic shutdown; delete manually)	`filter` on `block_number` + job mode

Rule of thumb: if the chain has a curated dataset and you need deep history, start with the curated datasets. Reach for Edge EVM when the chain isn’t indexed yet, when you need to control the RPC path, or when you want RPC-direct freshness at the tip.

​Overview

​Quick start

​Connecting to a chain

​Goldsky Edge endpoints

​Bring your own RPC

​Available datasets

​Dataset schemas

​Controlling the block range

​Reorg handling

​Tuning options

​Guide: multi-chain monitoring with mixed endpoints

​Edge EVM vs curated EVM datasets

Overview

Quick start

Connecting to a chain

Goldsky Edge endpoints

Bring your own RPC

Available datasets

Dataset schemas

Controlling the block range

Reorg handling

Tuning options

Guide: multi-chain monitoring with mixed endpoints

Edge EVM vs curated EVM datasets