Directories Structure

EventNative keeps all data in a configurable directory ./events. Here's the layout of sub-directories. File naming convention. Files usually named as ${file_type}.${var1}=${val1}|${var2}=${val2}.${extention}. The structure is very similar to the URL query string, but instead of `?` parameters are delimited by `|` and `.` is used instead of `?` (dots can't be a part of ${file_type})
Sub-dir
Purpose
Data format
Filename pattern
events/incoming
Incoming events (batch mode only)
Original JSON after ContextEnrichment step
incoming.tok=${tok}.log
where {tok} is used API 
token id
events/incoming
(status files)
Status of each batch: to which destinations and tables data has been sent successfully
DestinationStatus JSON
incoming.tok=${tok}.log.status
events/archive
Events that have been already processed
Original JSON after ContextEnrichment step
yyyy-mm-dd/incoming.tok=${tok}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz or 
yyyy-mm-dd/streaming-archive.dst=${destination_id}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz 
or 
yyyy-mm-dd/failed.dst=${destination_id}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz 
depend on events from batch destination or stream destination or replaying failed events. Where {tok} is used API token id
events/failed
Events that haven't been saved to a destination due to error
Collection of EventError: original JSON (after ContextEnrichment step) wrapped with EventError structure
failed.dst=${destination_id}.log
events/queue.dst=${destination_id}
Stream mode only: persistence for event queue
Binary
${partition_number}.dque and lock.lock

🚀How it works - Previous

Pipelines

Next - ✈️Sending Data

JavaScript Reference

Last updated 1 month ago

Sub-dir	Purpose	Data format	Filename pattern
`events/incoming`	Incoming events (batch mode only)	Original JSON after ContextEnrichment step	`incoming.tok=${tok}.log` where {tok} is used API token id
`events/incoming` (status files)	Status of each batch: to which destinations and tables data has been sent successfully	DestinationStatus JSON	`incoming.tok=${tok}.log.status`
`events/archive`	Events that have been already processed	Original JSON after ContextEnrichment step	`yyyy-mm-dd/incoming.tok=${tok}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz` or `yyyy-mm-dd/streaming-archive.dst=${destination_id}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz` or `yyyy-mm-dd/failed.dst=${destination_id}-yyyy-mm-ddTHH-mm-ss.SSS.log.gz` depend on events from batch destination or stream destination or replaying failed events. Where {tok} is used API token id
`events/failed`	Events that haven't been saved to a destination due to error	Collection of EventError: original JSON (after ContextEnrichment step) wrapped with EventError structure	`failed.dst=${destination_id}.log`
`events/queue.dst=${destination_id}`	Stream mode only: persistence for event queue	Binary	`${partition_number}.dque` and `lock.lock`