Streams

Streams are used to send data in ERDOS applications.

ERDOS streams are similar to ROS topics, but have a few additional desirable properties. Streams facilitate one-to-many communication, so only 1 operator sends messages on a stream. ERDOS broadcasts messages sent on a stream to all connected operators. In addition, streams are typed when using the Rust API.

Streams expose 2 classes of interfaces that access the underlying stream:

  1. Read-interfaces expose methods to receive and process data. They allow pulling data by calling read() and try_read(). Often, they also support a push data model accessed by registering callbacks (e.g. add_callback and add_watermark_callback). Structures that implement read interfaces include:

  • ReadStream: used by operators to read data and register callbacks.

  • ExtractStream: used by the driver to read data.

  1. Write-interfaces expose the send method to send data on a stream. Structures that implement write interfaces include:

  • WriteStream: used by operators to send data.

  • IngestStream: used by the driver to send data.

Some applications may want to introduce loops in their dataflow graphs which is possible using the LoopStream.

Sending Messages

Operators use Write Streams to send data.

Receiving Messages

Operators receive data by registering callbacks or manually reading messages from Read Streams.

Callbacks are functions which take an ERDOS message and any necessary write streams as arguments. Generally, callbacks process received messages and publish the results on write streams.

Ingesting and Extracting Data

Some applications have trouble placing all of the data processing logic inside operators. For these applications, ERDOS provides special stream interfaces to ingest and extract data.

A comprehensive example is available here.

Loops

Certain applications require feedback in the dataflow. To support this use case, ERDOS provides the LoopStream interface to support loops in the dataflow.

A comprehensive example is available here.