ClickHouse

EventNative supports ClickHouse as a destination. For more information about ClickHouse see docs.
Configuration
ClickHouse destination config consists of following schema:
destinations:
  clickhouse_1:
    type: clickhouse
    clickhouse:
      dsns:
        - "https://username:password@host1:8443/mydb?read_timeout=5m&timeout=5m&enable_http_compression=1&tls_config=maincert"
        - "https://username:password@host2:8443/mydb?read_timeout=5m&timeout=5m&enable_http_compression=1&tls_config=maincert"
        - "https://username:password@host3:8443/mydb?read_timeout=5m&timeout=5m&enable_http_compression=1&tls_config=maincert"
      db: mydb
      cluster: mycluster
      engine:
        raw_statement: 'ENGINE = ReplacingMergeTree(_timestamp) ORDER BY (eventn_ctx_event_id)' #optional
        nullable_fields:
          - middle_name
          - event_description
        partition_fields:
          - function: toYYYYMMDD
            field: _timestamp
          - field: event_type
        order_fields:
          - function: intHash32
            field: id
        primary_keys:
          - eventn_ctx_event_id
      tls:
        maincert: path_to_crt_file
clickhouse
Field (*required)
Type
Description
Default value
dsns*
string array
Array of connection strings. Must contain at least one connection string.
-
db*
string
Database name.
-
cluster*
string
Required. Run SELECT * from system.clusters to get  the list of all available clustes
-
engine
object
Tables engine configuration. 
see below
tls 
object
TLS configuration. Map of cert names and paths to cert files. Cert names will be used in tls_config query parameter in dsns.
-
If engine wasn't provided default one (depends on cluster configuration) will be used:
default values
#if ClickHouse single server (dsns = 1)
ReplacingMergeTree(_timestamp) PARTITION BY toYYYYMM(_timestamp) ORDER BY (eventn_ctx_event_id) 
​
#if ClickHouse cluster (dsns > 1)
ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/$db/$tablename', '{replica}', _timestamp) PARTITION BY toYYYYMM(_timestamp) ORDER BY (eventn_ctx_event_id) 
engine
Field (*required)
Type
Description
Default value
raw_statement
string
Engine raw statement which will be used in 
CREATE TABLE db.name (a type,b type) $raw_statement.
Must begin with 'ENGINE='. Could include ORDER BY, PARTITION BY, etc.
-
nullable_fields
string array
Array of nullable fields. All fields are created as non-null by default.
-
partition_fields
field object array
Array of field objects will be used in PARTITION BY ($partition_fields) statement
toYYYYMM(_timestamp)
order_fields
field object array
Array of field objects will be used in ORDER BY ($order_fields) statement
eventn_ctx_event_id
primary_keys 
string array
Array of field names will be used in PRIMARY KEY ($primary_keys) statement
-
Please note, if raw_statement exists than config keys partition_fields, order_fields, primary_keys will be skipped.
field object
Field (*required)
Type
Description
Default value
function
string
ClickHouse function name which will be applied to the field (e.g. toYYYYMMDD, intHash32)
-
field*
string
Field name
-

BigQuery

Google Analytics

Last updated 2 months ago

Field (*required)	Type	Description	Default value
dsns*	string array	Array of connection strings. Must contain at least one connection string.	-
db*	string	Database name.	-
cluster*	string	Required. Run `SELECT * from system.clusters` to get the list of all available clustes	-
engine	object	Tables engine configuration.	see below
tls	object	TLS configuration. Map of cert names and paths to cert files. Cert names will be used in tls_config query parameter in dsns.	-