Messages

ERDOS applications send data on streams via messages. Messages wrap data and provide timestamp information used to resolve control loops and track data flow through the system.

Timestamps

Timestamps consist of an array of coordinates. Timestamp semantics are user-defined for now; however, we may eventually formalize their use in the future in order to provide more advanced features in order to scale up stateful operators. Generally, the 0th coordinate is used to track message’s sequence number and subsequent coordinates track the message’s progress in cyclic data flows.

Watermarks

Watermarks in ERDOS signal completion of computation. More concretely, sending a watermark with timestamp t on a stream asserts that all future messages sent on that stream will have timestamps t' > t. ERDOS also introduces a top watermark, which is a watermark with the maximum possible timestamp. Sending a top watermark closes the stream as there is no t' > t_top, so no more messages can be sent.