A Webhook sink allows you to send data to an external service via HTTP. This provides considerable flexibility for forwarding pipeline results to your application server, a third-party API, or a bot.
Webhook sinks ensure at least once delivery and manage back-pressure, meaning data delivery adapts based on the responsiveness of your endpoints. The pipeline sends a POST request with a JSON payload to a specified URL, and the receiver only needs to return a 200 status code to confirm successful delivery.
Here is a snippet of YAML that specifies a Webhook sink:
Pipeline configuration
sinks:
my_webhook_sink:
type: webhook
# The webhook url
url: Type.String()
# The object key coming from either a source or transform.
# Example: ethereum.raw_blocks.
from: Type.String()
# The name of a goldsky httpauth secret you created which contains a header that can be used for authentication. More on how to create these in the section below.
secret_name: Type.Optional(Type.String())
# Optional metadata that you want to send on every request.
headers:
SOME-HEADER-KEY: Type.Optional(Type.String())
# Whether to send only one row per http request (better for compatibility with third-party integrations - e.g bots) or to mini-batch it (better for throughput).
one_row_per_request: Type.Optional(Type.Boolean())
sinks:
my_webhook_sink:
type: webhook
# The webhook url
url: Type.String()
# The object key coming from either a source or transform.
# Example: ethereum.raw_blocks.
from: Type.String()
# The name of a goldsky httpauth secret you created which contains a header that can be used for authentication. More on how to create these in the section below.
secret_name: Type.Optional(Type.String())
# Optional metadata that you want to send on every request.
headers:
SOME-HEADER-KEY: Type.Optional(Type.String())
# Whether to send only one row per http request (better for compatibility with third-party integrations - e.g bots) or to mini-batch it (better for throughput).
one_row_per_request: Type.Optional(Type.Boolean())
sinks:
myWebhookSink:
type: webhook
# The webhook url
url: Type.String()
# The object key coming from either a source or transform.
# Example: ethereum.raw_blocks.
from: Type.String()
# The name of a goldsky httpauth secret you created which contains a header that can be used for authentication. More on how to create these in the section below.
secretName: Type.Optional(Type.String())
# Optional metadata that you want to send on every request.
headers:
SOME-HEADER-KEY: Type.Optional(Type.String())
# Whether to send only one row per http request (better for compatibility with third-party integrations - e.g bots) or to mini-batch it (better for throughput).
oneRowPerRequest: Type.Optional(Type.Boolean())
Secret creation
Create a httpauth secret with the following CLI command:
Select httpauth as the secret type and then follow the prompts to finish creating your httpauth secret.
Example Webhook sink configuration
sinks:
my_webhook_sink:
type: webhook
url: https://my-webhook-service.com/webhook-1
from: ethereum.raw_blocks
secret_name: ETH_BLOCKS_SECRET
