Frequently asked questions

Endpoints are by default publicly accessible but you can make your endpoints private so that it’s only accessible by authenticated users, see private endpoints. Regardless of the access type, endpoints are typically rate-limited preventing abuse, and are not publicly indexed or searchable. As a best practice, you may want to proxy your requests to prevent leaking your endpoint URL from your front-end.

No! If Goldsky has already indexed that subgraph (unique subgraphs identified by their IPFS hash), it will sync instantly, though you will be provided your own endpoint with your own rate limits applied. Query away.

By default, the Scale plan is restricted to 50 requests every 10 seconds. However, our Enterprise plans scale horizontally and our highest-use endpoints are seamlessly handling thousands of requests a second at peak. If you need a higher rate limit than what you have enabled on your account, please contact us!

Not at the moment, though similar functionality for “live queries” can be accomplished by polling our querying endpoints. We also do support webhooks, which can be similarly useful for certain push-based use cases.

Deployments with a lot of metadata can sometimes time out the IPFS server. You can try again (right away, and if that isn’t working, a bit later) and eventually one attempt should work. This is a limitation of the IPFS server, but we’re exploring options to workaround this. If you continue to face issues, contact our support team at support@goldsky.com and we’ll help manually port it over.

You may get store error: column "x" specified more than once when using Goldsky’s Instant Subgraphs functionality. Multiple ABIs might be causing name conflicts due to conflicting fields or event names in the ABI. You can try splitting multiple ABIs into multiple subgraphs. There will be a mitigation for this in a future version. If you run into issues deploying or with the subgraph separately, contact our support team at support@goldsky.com.

Resource sizes represent the compute (vCPUs and RAM) available to the pipeline. For jobs with a large backlog of data (eg. a backfill pipeline) or with heavy transformations, larger resource sizes can speed up the job. In many cases, the default (S) pipeline is sufficient.

Mirror pipelines write data from us-west-2 on AWS from a dynamic range of IP addresses. If you need VPC peering / static IPs for your allow list, contact us at support@goldsky.com to discuss your use case.

Yes! Add --resource size <size> to your goldsky pipeline create <name> command, and the resource size will be set prior to deployment of the pipeline, preventing the need for a pipeline update (which restarts the resource).

Yes - if the primary key is the same (which is the default), the pipeline will upsert and not rewrite data. If it’s already there (based on the primary key) it will skip and move to the next record until it identifies data that isn’t already in the destination sink. It’s important to note that this only applies for databases that support upserts, such as Postgres, MySQL, Rockset, and Elasticsearch. This does not work on S3, and duplicate data is written.

Your destination sink and indexes kept will vastly influence how much storage you need for your data. We are working on publishing a record count for raw data tables to serve as a starting point for approximation, but in the meantime feel free to contact support for a better estimate for your specific use case!

API keys are only kept hashed (meaning after it’s displayed for the first time, you need to copy and save it locally in order to access it, we won’t be able to restore it for you!). If your API key is lost, you can reset / generate a new one from the settings page in the web app.

Goldsky can support any EVM-compatible chain. If we don’t support it in our shared indexing infrastructure, contact us to get set up with a dedicated indexer. Once set up, we can add new chains to your instance in a ~1 hour turnarount time or less.

Yes, every version of a subgraph incurs a separate worker fee and storage (in terms of entities) is also counted separately. Be sure to delete old versions of a subgraph you no longer need to query to minimize wasteful spend.