ERC-721 transfers

• Native Transfers
• ERC-20 Transfers
• ERC-1155 Transfers

​

What you’ll need

1. A Goldky account and the CLI installed

2. A basic understanding of the Mirror product
3. A destination sink to write your data to. In this example, we will use the PostgreSQL Sink

​

Introduction

1. Use the readily available ERC-20 dataset for the chain you are interested in: this is the easiest and quickest method to get you streaming token transfers into your sink of choice with minimum code.
2. Build the ERC-20 Transfers pipeline from scratch using raw or decoded logs: this method takes more code and time to implement but it’s a great way to learn about how you can use decoding functions in case you want to build more customized pipelines.

​

Using the ERC-20 Transfers Source Dataset

name: apex-erc721-pipeline
resource_size: s
apiVersion: 3
sources:
  apex.erc721_transfers:
    dataset_name: apex.erc721_transfers
    version: 1.0.0
    type: dataset
    start_at: earliest
transforms: {}
sinks:
  postgres_apex.erc721_transfers_public_apex_erc721_transfers:
    type: postgres
    table: apex_erc721_transfers
    schema: public
    secret_name: <YOUR_SECRET>
    description: 'Postgres sink for Dataset: apex.erc721_transfers'
    from: apex.erc721_transfers

goldsky pipeline start apex-erc721-pipeline.yaml

goldsky pipeline apply apex-erc721-pipeline.yaml --status ACTIVE

​

Building ERC-721 Transfers from scratch using logs

1. Use the raw_logs Direct Indexing dataset for that chain in combination with Decoding Transform Functions using the ABI of a specific ERC-721 Contract.
2. Use the decoded_logs Direct Indexing dataset for that chain in which the decoding process has already been done by Goldsky. This is only available for certain chains as you can check in this list.

​

Building ERC-721 Tranfers using Decoding Transform Functions

• ERC-20: event Transfer(address indexed _from, address indexed _to, uint256 _value)
• ERC-721: event Transfer(address indexed _from, address indexed _to, uint256 indexed _tokenId)

​

Pipeline Definition

name: scroll-erc721-transfers
apiVersion: 3
sources:
  my_scroll_mainnet_raw_logs:
    type: dataset
    dataset_name: scroll_mainnet.raw_logs
    version: 1.0.0
transforms:
  scroll_decoded:
    primary_key: id
    # Fetch the ABI from scrollscan for Cosmic Surprise token
    sql: >
      SELECT
        *,
        _gs_log_decode(
            _gs_fetch_abi('https://api.scrollscan.com/api?module=contract&action=getabi&address=0xcf7f37b4916ac5c530c863f8c8bb26ec1e8d2ccb', 'etherscan'),
            `topics`,
            `data`
        ) AS `decoded`
        WHERE topics LIKE '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef%'
        AND SPLIT_INDEX(topics, ',', 3) IS NOT NULL
        FROM my_scroll_mainnet_raw_logs
  scroll_clean:
    primary_key: id
    # Clean up the previous transform, unnest the values from the `decoded` object
    sql: >
      SELECT
        *,
        decoded.event_params AS `event_params`,
        decoded.event_signature AS `event_name`
        FROM scroll_decoded
        WHERE decoded IS NOT NULL
        AND decoded.event_signature = 'Transfer'
  scroll_721_transfers:
    primary_key: id
    sql: >
      SELECT
        id,
        address AS contract_address,
        lower(event_params[1]) AS sender,
        lower(event_params[2]) AS recipient,
        COALESCE(TRY_CAST(event_params[3] AS NUMERIC), -999) AS token_id,
        event_name,
        block_number,
        block_hash,
        log_index,
        transaction_hash,
        transaction_index
        FROM scroll_clean
sinks:
  scroll_721_sink:
    type: postgres
    table: erc721_transfers
    schema: mirror
    secret_name: <YOUR SECRET>
    description: Postgres sink for ERC721 transfers
    from: scroll_721_transfers

SELECT
  *,
  _gs_log_decode(
      _gs_fetch_abi('https://api.scrollscan.com/api?module=contract&action=getabi&address=0xc7d86908ccf644db7c69437d5852cedbc1ad3f69', 'etherscan'),
      `topics`,
      `data`
  ) AS `decoded`
  FROM scroll_mainnet.raw_logs
  WHERE topics LIKE '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef%'
  AND SPLIT_INDEX(topics, ',', 3) IS NOT NULL

• topics LIKE '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef%': topics is a comma separated string. Each value in the string is a hash. The first is the hash of the full event_signature (including arguments), in our case Transfer(address,address,uint256) for ERC-721, which is hashed to 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef. We use LIKE to only consider the first signature, with a % at the end, which acts as a wildcard.
• SPLIT_INDEX(topics, ',', 3) IS NOT NULL: as mentioned in the introduction, ERC-20 transfers share the same event_signature as ERC-721 transfers. The difference between them is the number of topics associated with the event. ERC-721 transfers have four topics, and ERC-20 transfers have three.

SELECT
  *,
  decoded.event_params AS `event_params`,
  decoded.event_signature AS `event_name`
  FROM scroll_decoded
  WHERE decoded IS NOT NULL
  AND decoded.event_signature = 'Transfer'

• decoded IS NOT NULL: to leave out potential null results from the decoder
• decoded.event_signature = 'Transfer': the decoder will output the event name as event_signature, excluding its arguments. We use it to filter only for Transfer events.

SELECT
  id,
  address AS contract_address,
  lower(event_params[1]) AS sender,
  lower(event_params[2]) AS recipient,
  COALESCE(TRY_CAST(event_params[3] AS NUMERIC), -999) AS token_id,
  event_name,
  block_number,
  block_hash,
  log_index,
  transaction_hash,
  transaction_index
  FROM scroll_clean

• id: This is the Goldsky provided id, it is a string composed of the dataset name, block hash, and log index, which is unique per event, here’s an example: log_0x60eaf5a2ab37c73cf1f3bbd32fc17f2709953192b530d75aadc521111f476d6c_18
• address AS contract_address: We use the lower function here to lower-case the address to make using this data simpler downstream, we also rename the column to contract_address to make it more explicit.
• lower(event_params[1]) AS sender: Here we continue to lower-case values for consistency. In this case we’re using the first element of the event_params array (using a 1-based index), and renaming it to sender. Each event parameter maps to an argument to the event_signature.
• lower(event_params[2]) AS recipient: Like the previous column, we’re pulling the second element in the event_params array and renaming it to recipient.

1. event_params[3] is the third element of the event_params array, and for ERC-721 this is the token ID. Although not covered in this example, since ERC-20 shares the same signature, this element represents a token balance rather than token ID if you’re decoding ERC-20 transfers.
2. TRY_CAST(event_params[3] AS NUMERIC) is casting the string element event_params[3] to NUMERIC - token IDs can be as large as an unsigned 256 bit integer, so make sure your database can handle that, if not, you can cast it to a different data type that your sink can handle. We use TRY_CAST because it will prevent the pipeline from failing in case the cast fails returning a NULL value instead.
3. COALESCE(TRY_CAST(event_params[3] AS NUMERIC), -999): COALESCE can take an arbitrary number of arguments and returns the first non-NULL value. Since TRY_CAST can return a NULL we’re returning -999 in case it does. This isn’t strictly necessary but is useful to do in case you want to find offending values that were unable to be cast.

event_name,
block_number,
block_hash,
log_index,
transaction_hash,
transaction_index

​

Deploying the pipeline

​

ERC-721 Transfers using decoded datasets

​

Conclusion