Schema and Mappings

EventNative is a "schemaless" by design. Any JSON structure can be accepted. Simultaneously, explicit SQL types can be configured in the mapping section. Explicit SQL types will override types determined from JSON values. They will be used in creating/patching Data Warehouse tables and in cast functions on insert operations. See more about types recognizing and typecasting in Typecast section.
​
Configuration
A special section in the destination configuration is designed to define how JSON is transformed before it's sent to the target. Currently, four mapping actions are supported: move, remove, cast, and constant.
destinations:
  destination_name:
    data_layout:
      mappings:
        keep_unmapped: true # if fields that are not explicitly mapped should be kept or removed
        fields:
        - src: /src/field/path # JSON path
          dst: /dst/field/path # could be just_field_name, without leading. Before inserting all / (except
          # first one) will be replaced wth '_'
          action: move | remove | cast | constant
          type: Lowcardinality(String) # for 'move' (optional) and 'cast' (required) actions - SQL type (depend on destination)
          value: # Value for setting as constant to 'dst'. Required only for 'constant' action. Other actions will ignore this field.
Parameter
Description
keep_unmapped
if true - all not mapped fields will be added to the result JSON as-is.
if false - only fields that were mentioned in mapping rules will be added to the result JSON. Default value: true
fields
An array of mapping objects
fields[N].src
Slash separated source JSON node path.
fields[N].dst
Slash separated or final destination JSON node path
fields[N].action
Currently, move, remove, cast, and constant are supported. 
move - get value with src JSON path and put it to dst JSON path.
remove - remove value from src JSON path
cast - take into account SQL type from type field and apply it to src JSON path node in SQL statements (creates/updated tables and inserts with src field)
constant - put the value from value field into dst JSON path node
fields[N].type
Data Warehouse specific SQL type which will be applied to dst JSON path field. Can be used with move, cast, and constant actions.
fields[N].value
A constant value that will be set into dst JSON path in result object. Can be used only with constant action
​
Example
destinations:
  destination_name:
    data_layout:
      mappings:
        keep_unmapped: false #only fields from configured ruless will be in the result
        fields:
        - src: /employee/id
          dst: /id
          action: move
        - src: /employee/start_at
          dst: /working_period_start
          action: move
          type: timestamp #SQL type
        - src: /employee/salary
          action: remove
        - dst: /postal_code
          action: constant
          type: text #SQL type
          value: 90210
        - dst: /taxes_sum
          action: cast
          type: numeric(38,18) #SQL type         
Input JSON object:
{
  "employee": {
    "id": 19318412,
    "start_at": "2018-12-10 10:06:18.509996",
    "salary": 50000.00,
    "personal_data": {
      "address": "...",
      "phone_number": "..."
    }
  },
  "taxes_sum": "892.32"
}

Result object with SQL types:
{
  "id": 19318412,
  "working_period_start": "2018-12-10 10:06:18.509996" #::timestamp,
  "postal_code": 90210 #::text,
  "taxes_sum": "892.32" #::numeric(38,18)
}
For configuring Segment like schema please see our wiki page.

SQL debug logs

Enrichment Rules

Last updated 1 month ago

Parameter	Description
keep_unmapped	if true - all not mapped fields will be added to the result JSON as-is. if false - only fields that were mentioned in mapping rules will be added to the result JSON. Default value: true
fields	An array of mapping objects
fields[N].src	Slash separated source JSON node path.
fields[N].dst	Slash separated or final destination JSON node path
fields[N].action	Currently, move, remove, cast, and constant are supported. move - get value with src JSON path and put it to dst JSON path. remove - remove value from src JSON path cast - take into account SQL type from type field and apply it to src JSON path node in SQL statements (creates/updated tables and inserts with src field) constant - put the value from value field into dst JSON path node
fields[N].type	Data Warehouse specific SQL type which will be applied to dst JSON path field. Can be used with move, cast, and constant actions.
fields[N].value	A constant value that will be set into dst JSON path in result object. Can be used only with constant action

Schema and Mappings

​

Configuration

​

Example

Input JSON object:

Result object with SQL types: