Sinks define the destination of your data. We support two broad categories of sinks based on their functionality and applicability:

  • Standard Sinks: These sinks are destinations readily available for querying and analysis, such as traditional databases.
  • Channel Sinks: These sinks serve as intermediate storage layers, facilitating further integration into your data stack. Examples: Kafka, AWS S3, or AWS SQS.

​
Standard Sinks

Standard Sinks are the default and most popular type of sinks for Mirror. They are optimized for immediate querying and analysis, providing a seamless experience for real-time data access and operations. These sinks are:

Postgres

Postgres stands out with its advanced features, extensibility, and strong ACID compliance.

MySQL

MySQL stands out with its advanced features, extensibility, and strong ACID compliance.

ClickHouse

ClickHouse delivers exceptional performance for OLAP queries with its columnar storage format.

Elasticsearch

Elasticsearch is a powerful tool for real-time search and analytics on large datasets.

Timescale

Timescale offers powerful time-series data management and analytics with PostgreSQL compatibility.

Goldsky Channels

Goldsky Channels are intermediate storage layers designed to absorb the Goldsky firehose and let you stream the data into alternative sinks. These channels are AWS S3, AWS SQS and Kafka.

​
What should I use?

​
For APIs for apps

For sub-second queries, typically you would choose a database that has row-based storage (i.e. it stores each row as it is instead of applying any sort of compression).

The drawbacks are that they take more space. This means large, non-indexed scans can take longer and storage costs can be higher.

  1. Postgres is the gold standard for application databases. It can scale almost infinitely with some management, and can support very fast point-lookups with proper indexing.

    If you require super fast lookups by transaction_hash or a specific column, Postgres is a very safe choice to start with. It’s great as a backend for live data APIs.

    However, it can be slow for analytics queries with a lot of aggregations. For that, you may want to look for an analytical database.

    Great hosted solutions for Postgres include NeonDB, AWS Aurora, and GCP CloudSQL.

  2. Elasticsearch is a no-sql database that allows for blazing fast lookups and searches. Elasticsearch is built around super-fast non-indexed scanning, meaning it can look at every single record to find the one you want. As a result, you can do queries like fuzzy matches and wildcard lookups with millisecond latency.

    Common applications include search on multiple columns, ‘instant’ auto-complete, and more.

​
For Analytics

  1. ClickHouse is a very efficient choice for storage. You can store the entire Ethereum blockchain and pay around $50 in storage.

    We recommend considering ClickHouse as an alternative to Snowflake or BigQuery - it supports many of the same use cases, and has additional features such as materialized views. We’ve seen our customers save tens of thousands of dollars using Goldsky and ClickHouse as a solution.

    The pricing for managing ClickHouse is based on storage cost, then compute cost. The compute cost is constant and isn’t based on the amount of data scanned, so you can run concurrent queries without increasing cost.

​
Channel Sinks

Channel Sinks act as an extension of the default sinks, providing intermediate storage for more complex data integration scenarios. They are designed to handle high-throughput data streams and enable further processing within your data stack. Examples include:

  • AWS S3: A scalable object storage service.
  • AWS SQS: A fully managed message queue for microservices, distributed systems, and serverless applications.
  • Kafka: A distributed event streaming platform.

For more information on Channel Sinks and how to integrate them, visit our Channels documentation.

Was this page helpful?

NFT datasetsPostgreSQL