Capy
Capy is a compiled C++20 library providing abstractions, vocabulary types, and idioms needed to write programs with event-driven control flows, such as I/O. It makes those programs efficient and manageable: structured and hard to get wrong.
Coroutines are used to represent the control flow. The program logic is described in a sequential way, familiar to programmers, even though there are tasks being pushed to task queues behind the scenes.
I/O operations are represented via the concept of a stream which consumes and populates buffer sequences. You also get type-erased wrappers for streams, so that programs can compile quickly, without triggering excessive template instantiations.
It offers task synchronization mechanisms at several levels.
Low-level ones include async mutexes and events.
High-level ones include strands, when_all/when_any "joins", and the resume-on-the-same-executor guarantee.
Libraries that want to provide concrete I/O implementations, like those based on epoll or io_uring,
can plug into Capy through the ContextAwaitable protocol.
The protocol describes how tasks can be scheduled, cancelled, and allocated in memory.
You also get the testing tools that allow you to test your asynchronous flows in a deterministic way.
What Capy is Not
While Capy offers algorithms and interfaces for dealing with I/O,
it does not itself provide any I/O backend (like a wrapper over epoll or io_uring).
For that, you need to use Capy in tandem with a Capy-conformant library.
This can be either your home-grown one or Corosio.
Quick Example
#include <boost/capy.hpp>
namespace capy = boost::capy;
capy::task<void> echo(capy::any_stream& stream) (1)
{
char buf[1024];
for(;;)
{
auto [ec, n] = co_await stream.read_some(capy::make_buffer(buf)); (2)
auto [wec, _] = co_await capy::write(stream, capy::const_buffer(buf, n));
if(wec)
throw std::system_error(wec); (3)
if(ec == capy::cond::eof) (4)
co_return;
if(ec)
throw std::system_error(ec);
}
}
int main()
{
// In a real application, you would obtain a stream from Corosio,
// then start the coroutine on its io_context and run it:
//
// corosio::io_context ioc;
// corosio::tcp_socket stream = /* from an acceptor or connect */;
// capy::run_async(ioc.get_executor())(echo(stream));
// ioc.run();
}
| 1 | The task<> return type defines a coroutine that starts suspended. any_stream is a type-erased wrapper that works with any concrete stream implementation. |
| 2 | Each co_await suspends until the I/O operation completes. |
| 3 | You can signal failures by throwing an exception, or by returning a std::error_code. |
| 4 | The condition cond::eof indicates reaching the end of the stream. |
Protocol Anatomy
Capy serves three audiences:
-
Application developers write tasks that do the work of a program. Some of them write protocol libraries, such as HTTP or WebSocket, on top of streams.
-
Framework authors write task types and the functions that start them.
-
I/O library authors write the I/O awaitables that connect Capy to an I/O backend, an event loop, or another async library.
All three share one model of how a task runs. This section shows that model, and then what each audience needs from it.
How a Task Runs
The diagram follows one task from start to finish.
The task, my_task, awaits a child task.
The child awaits an I/O operation, such as a socket read.
-
The caller passes an executor and a task to
run_async. The task does not run yet, because every task starts suspended. -
run_asyncgives the task its environment, anawait_context. The environment holds the executor, a stop token, a frame allocator, and the owner of the chain. The owner unwinds the chain if it is abandoned, as Who Owns a Suspended Coroutine explains. Thenrun_asynccallsdispatchon the executor. -
The executor resumes
my_task. If it is safe to resume the task at once, for example because the call is on the executor’s own thread, the executor does so. If it is not safe, the executor queues the task to resume later. -
my_taskawaitschild(). The child gets the same environment. Control transfers directly to the child, without the executor. This direct transfer is called symmetric transfer. -
The child awaits an I/O awaitable. The I/O awaitable keeps the executor from the environment.
-
The I/O awaitable starts the operation in the I/O backend.
-
The child suspends, and the thread goes back to the executor. The executor can run other work while the operation is pending.
-
Later, the I/O backend reports that the operation is complete. This report can arrive on any thread.
-
The I/O awaitable posts the child to the executor with
post. It never resumes the child directly. -
The executor resumes the child. The
co_awaitexpression gives the result of the operation to the child. -
The child finishes, and control transfers directly back to
my_task.my_taskis on the same executor, because the child used the environment ofmy_task. -
my_taskfinishes, and control goes back torun_async. -
run_asynccalls the completion handler with the result ofmy_task.
Steps 9 and 10 keep the same-executor invariant: a task always resumes on the executor it started with. Cancellation follows the same path, with a stop request in place of the completion. The Cancellation Sequence shows it.
The program below runs this sequence and prints one line for each arrow. It exits with an error if the order differs from the diagram, so the test suite checks the diagram.
A program that runs the sequence
#include <boost/capy.hpp>
#include <cstddef>
#include <functional>
#include <iostream>
#include <queue>
#include <string>
#include <vector>
namespace capy = boost::capy;
std::vector<std::string> trace;
void step(std::string what)
{
std::cout << trace.size() + 1 << ". " << what << "\n";
trace.push_back(std::move(what));
}
// A fake I/O backend: pending operations complete when main() polls.
std::queue<std::function<void()>> backend;
class trace_loop : public capy::execution_context
{
std::queue<capy::continuation> queue_;
public:
class executor_type;
trace_loop()
: execution_context(this)
{
}
// Nothing will run what is still queued: give it up, so its chain is
// unwound. Destroying a service, or unwinding a chain, can queue more
// work, so repeat until a drain finds nothing.
~trace_loop()
{
auto drain = [this]
{
bool any = false;
while (!queue_.empty())
{
auto c = queue_.front();
queue_.pop();
any = true;
capy::abandon(c);
}
return any;
};
shutdown();
drain();
do
{
destroy();
}
while (drain());
}
// A copy keeps the owner, which abandon needs.
void enqueue(capy::continuation const& c)
{
queue_.push(c);
}
// Run queued coroutines. Poll the backend only when the queue is empty.
void run()
{
while (!queue_.empty() || !backend.empty())
{
if (queue_.empty())
{
auto complete = std::move(backend.front());
backend.pop();
complete();
continue;
}
auto c = queue_.front();
queue_.pop();
step("executor: resume");
capy::safe_resume(c.h);
}
}
executor_type get_executor() noexcept;
};
class trace_loop::executor_type
{
trace_loop* loop_ = nullptr;
public:
executor_type() = default;
explicit executor_type(trace_loop& loop) noexcept
: loop_(&loop)
{
}
capy::execution_context& context() const noexcept
{
return *loop_;
}
void on_work_started() const noexcept {}
void on_work_finished() const noexcept {}
std::coroutine_handle<> dispatch(capy::continuation& c) const
{
step("executor: dispatch");
loop_->enqueue(c);
return std::noop_coroutine();
}
void post(capy::continuation& c) const
{
step("executor: post");
loop_->enqueue(c);
}
bool operator==(executor_type const& other) const noexcept
{
return loop_ == other.loop_;
}
};
trace_loop::executor_type
trace_loop::get_executor() noexcept
{
return executor_type{*this};
}
// An I/O awaitable for a read that the fake backend completes with 5 bytes.
struct read_awaitable
{
capy::continuation cont_;
capy::executor_ref ex_;
std::size_t bytes_ = 0;
bool await_ready() const noexcept
{
return false;
}
std::coroutine_handle<> await_suspend(
std::coroutine_handle<> h, capy::await_context const* env)
{
cont_.h = h;
cont_.owner = env->owner;
ex_ = env->executor;
step("I/O awaitable: start the operation");
backend.push([this] {
step("I/O backend: the operation completes");
bytes_ = 5;
ex_.post(cont_); // never cont_.h.resume()
});
step("I/O awaitable: suspend");
return std::noop_coroutine();
}
capy::results<std::size_t> await_resume() const noexcept
{
return {std::error_code(), bytes_};
}
};
capy::task<std::size_t> child()
{
step("child: co_await read_some");
auto [ec, n] = co_await read_awaitable{};
step("child: co_return");
co_return ec ? 0 : n;
}
capy::task<std::size_t> my_task()
{
step("my_task: co_await child()");
std::size_t n = co_await child();
step("my_task: co_return");
co_return n;
}
int main()
{
trace_loop loop;
step("caller: run_async(ex)(my_task())");
capy::run_async(loop.get_executor(), [](std::size_t n) {
step("run_async: call the completion handler with " +
std::to_string(n));
})(my_task());
loop.run();
// One entry per arrow in the diagram, in the same order.
std::vector<std::string> const diagram = {
"caller: run_async(ex)(my_task())", // 1
"executor: dispatch", // 2
"executor: resume", // 3
"my_task: co_await child()", // 4
"child: co_await read_some", // 5
"I/O awaitable: start the operation", // 6
"I/O awaitable: suspend", // 7
"I/O backend: the operation completes", // 8
"executor: post", // 9
"executor: resume", // 10
"child: co_return", // 11
"my_task: co_return", // 12
"run_async: call the completion handler with 5", // 13
};
if (trace != diagram)
{
std::cout << "trace does not match the diagram\n";
return 1;
}
}
Application Developers
You write my_task and child.
The rest of the diagram is Capy, or an I/O library such as Corosio.
-
Code between two
co_awaitexpressions runs like ordinary sequential code. Eachco_awaitis a point where the task can suspend. -
A task resumes on the executor it started with. State that only one task uses needs no lock. State that tasks share on a thread pool needs a strand or an
async_mutex. -
Start top-level work from
mainwithrun_async. Inside a task, await other tasks directly. To run a child task on a different executor, await it throughrun. -
To cancel work, pass a stop token to
run_async. Operations that stop early reportcond::canceled. To check for a stop request inside a task, read the token withco_awaitthis_coro::stop_token. -
An object that a task refers to must stay alive until the task finishes. Lifetime and Ownership gives the rules for parameters, lambda captures, and buffers.
-
Stop work and wait for it to finish before you destroy the execution context that it runs on. Destroying the context abandons the chains that are still suspended in it. Their frames are destroyed without resuming, so they deliver no result and call no completion handler.
Read next:
Protocol Libraries
If you write a protocol library, such as HTTP or WebSocket, your child implements the protocol on a stream.
Your caller supplies the stream, and an I/O library supplies the I/O awaitables.
-
Write the protocol against the
ReadStreamandWriteStreamconcepts. To keep the protocol out of headers, write it against the type-erasedany_streaminstead. -
The executor and the stop token reach your task through
co_await. Your functions do not take them as parameters. -
Each stream operation returns a
resultsthat holds an error code and a byte count. A read can return bytes and an error together. The stream only reports what happened. Your protocol decides whether the result is a success or a failure, for example whether a partial message is usable. -
Compare the error code with the conditions in
cond, for examplecond::eofat the end of the data. -
When a stop is requested, a pending operation fails with an error that compares equal to
cond::canceled. This error means that a caller asked for the stop, not that the protocol failed. Do not retry the operation. Return the error to your caller. -
Test the protocol with the mock streams in
capy::test. The mock streams run without an I/O backend and can inject errors.
Read next:
Framework Authors
You write the task type, such as task<T>, and the functions that start tasks, such as run_async.
In the diagram, you write the run_async participant and the transfers between tasks: steps 2, 4, and 11 to 13.
-
A task type is an
ContextAwaitable, so that other tasks can await it. It is also anContextRunnable, so that functions such asrun_asynccan start it. -
Derive the promise type from
context_awaitable_promise_base. The base allocates the coroutine frame with the current frame allocator and stores the environment. It also answersco_await this_coro::…queries, and passes every other awaitable to thetransform_awaitableof your promise. -
In
transform_awaitable, pass the environment to each I/O awaitable through its two-argumentawait_suspend. -
An awaited child task gets the same environment, and it returns to its parent by symmetric transfer. So the parent continues on the same executor.
-
A function that starts tasks creates the environment: an executor, a stop token, a frame allocator, and an owner. It must set the frame allocator before the task’s frame is allocated. Frame Allocators explains why
run_asyncuses two calls for this. -
The function must also own the chain that it starts, by binding a
chain_owner, so that a holder can abandon the chain. The alternative is to make sure that nothing awaits a coroutine that could be abandoned.run_async,runon another executor, and each branch ofwhen_allandwhen_anyown their chains already. -
To store an executor without its type, use
executor_ref.
Read next:
I/O Library Authors
In the diagram, you write the I/O awaitable, and sometimes the I/O backend and the executor.
-
The I/O awaitable implements the
ContextAwaitableprotocol. Itsawait_suspendreceives the coroutine handle and theawait_context. -
Keep the executor from the environment. When the operation completes, resume the coroutine through that executor. Wrap the handle in a
continuationthat lives in the awaitable, and pass it topostordispatch. Copy theownerof the environment intoowner, so that the chain can be abandoned while the coroutine waits. Never callresume()on the handle directly. -
Register a
std::stop_callbackon the stop token of the environment. The callback runs on the thread that requests the stop. If a stop was already requested, it runs at once, inside the constructor of the callback. It can run at the same time as the completion of the operation. Only one of them may resume the coroutine, so each one must first win an atomic claim. -
An executor implements the
Executorconcept. Itscontext()returns theexecution_contextthat created it. An event loop that runs coroutines resumes them withsafe_resume, so that each coroutine keeps its own frame allocator. -
An execution context that is destroyed with work still queued passes each queued continuation to
abandoninstead of resuming it. -
To start Capy tasks from another event loop, use
run_asyncwith an executor for that loop.
Read next: