Data Streaming
Category: Transform
Process large datasets with memory-efficient streaming
What this node does
- Memory-efficient
- Chunk processing
- Large dataset support
- Async streaming
How to use
- In the Agentic Studio, open or create a workflow
- In the node palette on the left, find Data Streaming under the Transform category (or use the search bar)
- Drag the node onto the canvas
- Double-click the node to open its configuration dialog
- Fill in the required parameters (see Configuration below)
- Connect the Input Data (file/stream) input port from an upstream node
- Optionally connect the Chunk Size port if needed
- Connect the Processed Data Stream output to the next node downstream
Inputs
| Port | Type | Required | Description |
|---|---|---|---|
| Input Data (file/stream) | any | ✓ | any data |
| Chunk Size | number | Optional | Numeric value |
Outputs
| Port | Type | Description |
|---|---|---|
| Processed Data Stream | any | any data |
Configuration
Open the configuration dialog by double-clicking the Data Streaming node on the canvas.
| Parameter | What to enter |
|---|---|
chunkSize | Configure chunkSize in the node settings |
processingFunction | Configure processingFunction in the node settings |
concurrency | Configure concurrency in the node settings |
When to use this node
- Process large FHIR bundles
- Stream log files
- Handle big data
Need help configuring this node?
Go to Settings → Connectors to set up the connection this node depends on, then reference the connector ID in the node configuration dialog.