Directories Structure

EventNative keeps all data in a configurable directory ./events. Here's the layout of sub-directories. File naming convention. Files usually named as ${file_type}.${var1}=${val1}|${var2}=${val2}.${extention}. The structure is very similar to the URL query string, but instead of `?` parameters are delimited by `|` and `.` is used instead of `?` (dots can't be a part of ${file_type})

Sub-dir

Purpose

Data format

Filename pattern

events/incoming

Incoming events (batch mode only)

Original JSON after ContextEnrichment step

incoming.tok=${tok}.log

where {tok} is used API

token id

events/incoming

(status files)

Status of each batch: to which destinations and tables data has been sent successfully

DestinationStatus JSON

incoming.tok=${tok}.log.status

events/archive

Events that have been already processed

Original JSON after ContextEnrichment step

yyyy-mm-dd/incoming.tok=${tok}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz or

yyyy-mm-dd/streaming-archive.dst=${destination_id}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz

or

yyyy-mm-dd/failed.dst=${destination_id}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz

depend on events from batch destination or stream destination or replaying failed events. Where {tok} is used API token id

events/failed

Events that haven't been saved to a destination due to error

Collection of EventError: original JSON (after ContextEnrichment step) wrapped with EventError structure

failed.dst=${destination_id}.log

events/queue.dst=${destination_id}

Stream mode only: persistence for event queue

Binary

${partition_number}.dque and lock.lock