Sprocket

A distributed application framework for building streaming, event-driven pipelines on commodity serverless infrastructure - Lambda, ECS, Kubernetes, or anything else - while remaining language and platform agnostic. The core idea, drawn from experience with Apache Storm and Flink, is to separate the application logic (tasks invoked over a simple HTTP protocol) from the orchestration concerns (event queueing, dispatch, rate limiting, backpressure), handled by a sidecar layer that runs alongside each execution unit.

The closest analogue is DAPR, a Microsoft Research project that launched around the same time and addresses a similar problem. Sprocket’s emphasis was on the protocol layer: any service that can handle an HTTP request can participate in the graph, regardless of language or runtime.

The Go proof-of-concept, built in late 2019, got as far as a working input→buffer→process→output pipeline with cross-instance routing, acknowledgements, and Serf-based membership for service discovery.