Sync dataset to Postgres

This pipeline is named decoded-logs-pipeline. It pulls data from a curated goldsky dataset, without performing any transformations, and stores the result into a PostgreSQL sink, in a table called eth_logs in the goldsky schema.

decoded-logs-pipeline.yaml

name: decoded-logs-pipeline
apiVersion: 3
sources:
  my_ethereum_decoded_logs:
    dataset_name: ethereum.decoded_logs
    version: 1.0.0
    type: dataset
    start_at: latest

transforms:
  logs:
    sql: |
      SELECT
          id,
          address,
          event_signature,
          event_params,
          raw_log.block_number as block_number,
          raw_log.block_hash as block_hash,
          raw_log.transaction_hash as transaction_hash
      FROM
          my_ethereum_decoded_logs
    primary_key: id

sinks:
  logs_sink:
    type: postgres
    table: eth_logs
    schema: goldsky
    secret_name: API_POSTGRES_CREDENTIALS
    from: logs

You can start above pipeline by running:

goldsky pipeline start pipeline.yaml

goldsky pipeline apply pipeline.yaml --status ACTIVE