The KoCircuit Experiment
A bold experiment in the simplification of data pipeline logic for developers.
KoCircuit is an open reimplementation of a highly successful internal project developed to unify and rationalize data pipelines at Google. It allows users to build process graphs (DAGs) on top of a fully open source base of plugin containers.
Ko was designed to be syntactically simple, concurrent, functional, and entirely type-safe. The language sits on top of a Go runtime environment, and can thus expose the APIs of Kubernetes, Knative, and other powerful resources. KoCircuit defines fully distributed DAG processes, at any scale. The result is an elegant form of functional “assembly language” for interfacing with the Kubernetes runtime environment.
KoCircuit is the first fruit of our research into robust self-healing data processing environments.