Microservice Pipelines

A microservice pipeline is a pipeline that creates a fine-grained service to perform a specific task.

For example, you might use a microservice pipeline to validate changes to user accounts. You can have the pipeline check for matching accounts to validate account creation requests, account updates, and account deletions. The pipeline can tag requests as actionable or not and send all responses back to the originating system for next steps.

You can create REST-based and message-based microservice pipelines that use standalone execution mode. Other microservice types and platforms are planned for future releases.

A microservice pipeline uses a microservice origin to listen for or read JSON-formatted requests. In the pipeline, you can use any available processor to transform and route the records as needed. You can also use destinations and executors to write data and execute additional actions as needed.

Then, use one or more microservice destinations to send responses back to the origin and write the records to a destination system if necessary. The origin then transmits JSON-formatted responses back to the origin system.

For example, you might use the WebSocket Server origin to listen at a WebSocket endpoint and process all authorized requests. In the pipeline, you use stages to process the data and route records to different destinations based on whether the record is considered a success record or bad request record. You use two Send Response to Origin destinations to return different responses, 200 for OK and 400 for Bad Request. The origin then passes the records with the responses back to the WebSocket endpoint. Or, you might use a different microservice destination to write records to a destination system while sending responses back to the origin from the destination system.

For more information and a descriptive video, see our blog post on microservice pipelines. For a full description of a sample microservice pipeline and a tutorial on creating one, see our GitHub tutorial, "Creating a CRUD Microservice Pipeline."