ERC-721 transfers

• Native Transfers
• ERC-20 Transfers
• ERC-1155 Transfers

​

What you’ll need

1. A Goldky account and the CLI installed

2. A basic understanding of the Mirror product
3. A destination sink to write your data to. In this example, we will use the PostgreSQL Sink

​

Introduction

1. Use the readily available ERC-20 dataset for the chain you are interested in: this is the easiest and quickest method to get you streaming token transfers into your sink of choice with minimum code.
2. Build the ERC-20 Transfers pipeline from scratch using raw or decoded logs: this method takes more code and time to implement but it’s a great way to learn about how you can use decoding functions in case you want to build more customized pipelines.

​

Using the ERC-20 Transfers Source Dataset

name: apex-erc721-pipeline
resource_size: s
apiVersion: 3
sources:
  apex.erc721_transfers:
    dataset_name: apex.erc721_transfers
    version: 1.0.0
    type: dataset
    start_at: earliest
transforms: {}
sinks:
  postgres_apex.erc721_transfers_public_apex_erc721_transfers:
    type: postgres
    table: apex_erc721_transfers
    schema: public
    secret_name: <YOUR_SECRET>
    description: 'Postgres sink for Dataset: apex.erc721_transfers'
    from: apex.erc721_transfers

goldsky pipeline start apex-erc721-pipeline.yaml

goldsky pipeline apply apex-erc721-pipeline.yaml --status ACTIVE

​

Building ERC-721 Transfers from scratch using logs

​

Building ERC-721 Tranfers using Decoding Transform Functions

• ERC-20: event Transfer(address indexed _from, address indexed _to, uint256 _value)
• ERC-721: event Transfer(address indexed _from, address indexed _to, uint256 indexed _tokenId)

​

Pipeline Definition

name: scroll-erc721-transfers
apiVersion: 3
sources:
  my_scroll_mainnet_raw_logs:
    type: dataset
    dataset_name: scroll_mainnet.raw_logs
    version: 1.0.0
transforms:
  scroll_decoded:
    primary_key: id
    # Fetch the ABI from scrollscan for Cosmic Surprise token
    sql: >
      SELECT
        *,
        _gs_log_decode(
            _gs_fetch_abi('https://api.scrollscan.com/api?module=contract&action=getabi&address=0xcf7f37b4916ac5c530c863f8c8bb26ec1e8d2ccb&apikey=YOUR_KEY', 'etherscan'),
            `topics`,
            `data`
        ) AS `decoded`
        WHERE topics LIKE '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef%'
        AND SPLIT_INDEX(topics, ',', 3) IS NOT NULL
        FROM my_scroll_mainnet_raw_logs
  scroll_clean:
    primary_key: id
    # Clean up the previous transform, unnest the values from the `decoded` object
    sql: >
      SELECT
        *,
        decoded.event_params AS `event_params`,
        decoded.event_signature AS `event_name`
        FROM scroll_decoded
        WHERE decoded IS NOT NULL
        AND decoded.event_signature = 'Transfer'
  scroll_721_transfers:
    primary_key: id
    sql: >
      SELECT
        id,
        address AS contract_address,
        lower(event_params[1]) AS sender,
        lower(event_params[2]) AS recipient,
        COALESCE(TRY_CAST(event_params[3] AS NUMERIC), -999) AS token_id,
        event_name,
        block_number,
        block_hash,
        log_index,
        transaction_hash,
        transaction_index
        FROM scroll_clean
sinks:
  scroll_721_sink:
    type: postgres
    table: erc721_transfers
    schema: mirror
    secret_name: <YOUR SECRET>
    description: Postgres sink for ERC721 transfers
    from: scroll_721_transfers

SELECT
  *,
  _gs_log_decode(
      _gs_fetch_abi('https://api.scrollscan.com/api?module=contract&action=getabi&address=0xc7d86908ccf644db7c69437d5852cedbc1ad3f69&apikey=YOUR_KEY', 'etherscan'),
      `topics`,
      `data`
  ) AS `decoded`
  FROM scroll_mainnet.raw_logs
  WHERE topics LIKE '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef%'
  AND SPLIT_INDEX(topics, ',', 3) IS NOT NULL

• topics LIKE '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef%': topics is a comma separated string. Each value in the string is a hash. The first is the hash of the full event_signature (including arguments), in our case Transfer(address,address,uint256) for ERC-721, which is hashed to 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef. We use LIKE to only consider the first signature, with a % at the end, which acts as a wildcard.
• SPLIT_INDEX(topics, ',', 3) IS NOT NULL: as mentioned in the introduction, ERC-20 transfers share the same event_signature as ERC-721 transfers. The difference between them is the number of topics associated with the event. ERC-721 transfers have four topics, and ERC-20 transfers have three.

SELECT
  *,
  decoded.event_params AS `event_params`,
  decoded.event_signature AS `event_name`
  FROM scroll_decoded
  WHERE decoded IS NOT NULL
  AND decoded.event_signature = 'Transfer'

• decoded IS NOT NULL: to leave out potential null results from the decoder
• decoded.event_signature = 'Transfer': the decoder will output the event name as event_signature, excluding its arguments. We use it to filter only for Transfer events.

SELECT
  id,
  address AS contract_address,
  lower(event_params[1]) AS sender,
  lower(event_params[2]) AS recipient,
  COALESCE(TRY_CAST(event_params[3] AS NUMERIC), -999) AS token_id,
  event_name,
  block_number,
  block_hash,
  log_index,
  transaction_hash,
  transaction_index
  FROM scroll_clean

• id: This is the Goldsky provided id, it is a string composed of the dataset name, block hash, and log index, which is unique per event, here’s an example: log_0x60eaf5a2ab37c73cf1f3bbd32fc17f2709953192b530d75aadc521111f476d6c_18
• address AS contract_address: We use the lower function here to lower-case the address to make using this data simpler downstream, we also rename the column to contract_address to make it more explicit.
• lower(event_params[1]) AS sender: Here we continue to lower-case values for consistency. In this case we’re using the first element of the event_params array (using a 1-based index), and renaming it to sender. Each event parameter maps to an argument to the event_signature.
• lower(event_params[2]) AS recipient: Like the previous column, we’re pulling the second element in the event_params array and renaming it to recipient.

1. event_params[3] is the third element of the event_params array, and for ERC-721 this is the token ID. Although not covered in this example, since ERC-20 shares the same signature, this element represents a token balance rather than token ID if you’re decoding ERC-20 transfers.
2. TRY_CAST(event_params[3] AS NUMERIC) is casting the string element event_params[3] to NUMERIC - token IDs can be as large as an unsigned 256 bit integer, so make sure your database can handle that, if not, you can cast it to a different data type that your sink can handle. We use TRY_CAST because it will prevent the pipeline from failing in case the cast fails returning a NULL value instead.
3. COALESCE(TRY_CAST(event_params[3] AS NUMERIC), -999): COALESCE can take an arbitrary number of arguments and returns the first non-NULL value. Since TRY_CAST can return a NULL we’re returning -999 in case it does. This isn’t strictly necessary but is useful to do in case you want to find offending values that were unable to be cast.

event_name,
block_number,
block_hash,
log_index,
transaction_hash,
transaction_index

​

Deploying the pipeline

​

Conclusion