Mirror
ClickHouse
Closed Beta
This feature is in closed beta and only available for our enterprise customers.
Please contact us at [email protected] to request access to this feature.
ReplacingMergeTree engine is used for all sink tables by default.
When appendOnlyMode
is true
(default), the sink behaves the following way:
- All updates and deletes are converted to inserts.
is_deleted
column is automatically added to a table. It contains1
in case of deletes,0
otherwise.- If
versionColumnName
is specified, it's used as a version number column for deduplication. If it's not specified,insert_time
column is automatically added to a table. It contains insertion time and is used for deduplication. - Primary key is used in the
ORDER BY
clause.
When appendOnlyMode
is false
:
- All updates and deletes are propagated as is.
- No extra columns are added.
- Primary key is used in the
PRIMARY KEY
clause.
Pipeline configuration
{
"sources": [],
"transforms": [],
"sinks": [
{
"description": "Type.Optional(Type.String())",
"type": "clickHouse",
"sourceStreamName": "Type.String()",
"secretName": "Type.String()",
"table": "Type.String()",
"batchSize": "Type.Optional(Type.Integer())",
"flushInterval": "Type.Optional(Type.String())",
"appendOnlyMode": "Type.Optional(Type.Boolean())",
"versionColumnName": "Type.Optional(Type.String())"
}
]
}
Secrets
goldsky secret create A_CLICKHOUSE_SECRET --type clickHouse --value '{
"url": "Type.String()",
"username": "Type.String()",
"password": "Type.String()",
"databaseName": "Type.String()"
}'