Capy

Capy is a compiled C++20 library providing abstractions, vocabulary types, and idioms needed to write programs with event-driven control flows, such as I/O. It makes those programs efficient and manageable: structured and hard to get wrong.

Coroutines are used to represent the control flow. The program logic is described in a sequential way, familiar to programmers, even though there are tasks being pushed to task queues behind the scenes.

I/O operations are represented via the concept of a stream which consumes and populates buffer sequences. You also get type-erased wrappers for streams, so that programs can compile quickly, without triggering excessive template instantiations.

It offers task synchronization mechanisms at several levels. Low-level ones include async mutexes and events. High-level ones include strands, when_all/when_any "joins", and the resume-on-the-same-executor guarantee.

Libraries that want to provide concrete I/O implementations, like those based on epoll or io_uring, can plug into Capy through the ContextAwaitable protocol. The protocol describes how tasks can be scheduled, cancelled, and allocated in memory.

You also get the testing tools that allow you to test your asynchronous flows in a deterministic way.

What Capy is Not

While Capy offers algorithms and interfaces for dealing with I/O, it does not itself provide any I/O backend (like a wrapper over epoll or io_uring). For that, you need to use Capy in tandem with a Capy-conformant library. This can be either your home-grown one or Corosio.

Quick Example

#include <boost/capy.hpp>

namespace capy = boost::capy;

capy::task<void> echo(capy::any_stream& stream)   (1)
{
    char buf[1024];
    for(;;)
    {
        auto [ec, n] = co_await stream.read_some(capy::make_buffer(buf));   (2)

        auto [wec, _] = co_await capy::write(stream, capy::const_buffer(buf, n));

        if(wec)
            throw std::system_error(wec);         (3)

        if(ec == capy::cond::eof)                 (4)
            co_return;

        if(ec)
            throw std::system_error(ec);
    }
}

int main()
{
    // In a real application, you would obtain a stream from Corosio,
    // then start the coroutine on its io_context and run it:
    //
    //   corosio::io_context ioc;
    //   corosio::tcp_socket stream = /* from an acceptor or connect */;
    //   capy::run_async(ioc.get_executor())(echo(stream));
    //   ioc.run();
}
1 The task<> return type defines a coroutine that starts suspended. any_stream is a type-erased wrapper that works with any concrete stream implementation.
2 Each co_await suspends until the I/O operation completes.
3 You can signal failures by throwing an exception, or by returning a std::error_code.
4 The condition cond::eof indicates reaching the end of the stream.

Protocol Anatomy

Capy serves three audiences:

  • Application developers write tasks that do the work of a program. Some of them write protocol libraries, such as HTTP or WebSocket, on top of streams.

  • Framework authors write task types and the functions that start them.

  • I/O library authors write the I/O awaitables that connect Capy to an I/O backend, an event loop, or another async library.

All three share one model of how a task runs. This section shows that model, and then what each audience needs from it.

How a Task Runs

The diagram follows one task from start to finish. The task, my_task, awaits a child task. The child awaits an I/O operation, such as a socket read.

sequenceDiagram autonumber accTitle: How a task runs accDescr: A caller starts my_task through run_async. my_task awaits a child task, which awaits an I/O awaitable. The thread returns to the executor while the operation is pending. When the I/O backend completes the operation, the I/O awaitable posts the child to the executor, which resumes it. The child returns to my_task, and my_task's result goes to the completion handler. participant C as Caller participant R as run_async participant X as Executor participant P as my_task participant K as child participant A as I/O awaitable participant B as I/O backend C->>R: run_async(ex)(my_task()) R->>X: dispatch(my_task) X->>P: resume P->>K: co_await child() K->>A: co_await read_some(buf) A->>B: start the operation A-->>X: suspend, the thread is free B->>A: the operation completes A->>X: post(child) X->>K: resume with the result K->>P: co_return P->>R: co_return R->>R: call the completion handler
  1. The caller passes an executor and a task to run_async. The task does not run yet, because every task starts suspended.

  2. run_async gives the task its environment, an await_context. The environment holds the executor, a stop token, a frame allocator, and the owner of the chain. The owner unwinds the chain if it is abandoned, as Who Owns a Suspended Coroutine explains. Then run_async calls dispatch on the executor.

  3. The executor resumes my_task. If it is safe to resume the task at once, for example because the call is on the executor’s own thread, the executor does so. If it is not safe, the executor queues the task to resume later.

  4. my_task awaits child(). The child gets the same environment. Control transfers directly to the child, without the executor. This direct transfer is called symmetric transfer.

  5. The child awaits an I/O awaitable. The I/O awaitable keeps the executor from the environment.

  6. The I/O awaitable starts the operation in the I/O backend.

  7. The child suspends, and the thread goes back to the executor. The executor can run other work while the operation is pending.

  8. Later, the I/O backend reports that the operation is complete. This report can arrive on any thread.

  9. The I/O awaitable posts the child to the executor with post. It never resumes the child directly.

  10. The executor resumes the child. The co_await expression gives the result of the operation to the child.

  11. The child finishes, and control transfers directly back to my_task. my_task is on the same executor, because the child used the environment of my_task.

  12. my_task finishes, and control goes back to run_async.

  13. run_async calls the completion handler with the result of my_task.

Steps 9 and 10 keep the same-executor invariant: a task always resumes on the executor it started with. Cancellation follows the same path, with a stop request in place of the completion. The Cancellation Sequence shows it.

The program below runs this sequence and prints one line for each arrow. It exits with an error if the order differs from the diagram, so the test suite checks the diagram.

A program that runs the sequence
#include <boost/capy.hpp>
#include <cstddef>
#include <functional>
#include <iostream>
#include <queue>
#include <string>
#include <vector>

namespace capy = boost::capy;

std::vector<std::string> trace;

void step(std::string what)
{
    std::cout << trace.size() + 1 << ". " << what << "\n";
    trace.push_back(std::move(what));
}

// A fake I/O backend: pending operations complete when main() polls.
std::queue<std::function<void()>> backend;

class trace_loop : public capy::execution_context
{
    std::queue<capy::continuation> queue_;

public:
    class executor_type;

    trace_loop()
        : execution_context(this)
    {
    }

    // Nothing will run what is still queued: give it up, so its chain is
    // unwound. Destroying a service, or unwinding a chain, can queue more
    // work, so repeat until a drain finds nothing.
    ~trace_loop()
    {
        auto drain = [this]
        {
            bool any = false;
            while (!queue_.empty())
            {
                auto c = queue_.front();
                queue_.pop();
                any = true;
                capy::abandon(c);
            }
            return any;
        };
        shutdown();
        drain();
        do
        {
            destroy();
        }
        while (drain());
    }

    // A copy keeps the owner, which abandon needs.
    void enqueue(capy::continuation const& c)
    {
        queue_.push(c);
    }

    // Run queued coroutines. Poll the backend only when the queue is empty.
    void run()
    {
        while (!queue_.empty() || !backend.empty())
        {
            if (queue_.empty())
            {
                auto complete = std::move(backend.front());
                backend.pop();
                complete();
                continue;
            }
            auto c = queue_.front();
            queue_.pop();
            step("executor: resume");
            capy::safe_resume(c.h);
        }
    }

    executor_type get_executor() noexcept;
};

class trace_loop::executor_type
{
    trace_loop* loop_ = nullptr;

public:
    executor_type() = default;

    explicit executor_type(trace_loop& loop) noexcept
        : loop_(&loop)
    {
    }

    capy::execution_context& context() const noexcept
    {
        return *loop_;
    }

    void on_work_started() const noexcept {}
    void on_work_finished() const noexcept {}

    std::coroutine_handle<> dispatch(capy::continuation& c) const
    {
        step("executor: dispatch");
        loop_->enqueue(c);
        return std::noop_coroutine();
    }

    void post(capy::continuation& c) const
    {
        step("executor: post");
        loop_->enqueue(c);
    }

    bool operator==(executor_type const& other) const noexcept
    {
        return loop_ == other.loop_;
    }
};

trace_loop::executor_type
trace_loop::get_executor() noexcept
{
    return executor_type{*this};
}

// An I/O awaitable for a read that the fake backend completes with 5 bytes.
struct read_awaitable
{
    capy::continuation cont_;
    capy::executor_ref ex_;
    std::size_t bytes_ = 0;

    bool await_ready() const noexcept
    {
        return false;
    }

    std::coroutine_handle<> await_suspend(
        std::coroutine_handle<> h, capy::await_context const* env)
    {
        cont_.h = h;
        cont_.owner = env->owner;
        ex_ = env->executor;
        step("I/O awaitable: start the operation");
        backend.push([this] {
            step("I/O backend: the operation completes");
            bytes_ = 5;
            ex_.post(cont_);  // never cont_.h.resume()
        });
        step("I/O awaitable: suspend");
        return std::noop_coroutine();
    }

    capy::results<std::size_t> await_resume() const noexcept
    {
        return {std::error_code(), bytes_};
    }
};

capy::task<std::size_t> child()
{
    step("child: co_await read_some");
    auto [ec, n] = co_await read_awaitable{};
    step("child: co_return");
    co_return ec ? 0 : n;
}

capy::task<std::size_t> my_task()
{
    step("my_task: co_await child()");
    std::size_t n = co_await child();
    step("my_task: co_return");
    co_return n;
}

int main()
{
    trace_loop loop;

    step("caller: run_async(ex)(my_task())");
    capy::run_async(loop.get_executor(), [](std::size_t n) {
        step("run_async: call the completion handler with " +
            std::to_string(n));
    })(my_task());

    loop.run();

    // One entry per arrow in the diagram, in the same order.
    std::vector<std::string> const diagram = {
        "caller: run_async(ex)(my_task())",              //  1
        "executor: dispatch",                            //  2
        "executor: resume",                              //  3
        "my_task: co_await child()",                     //  4
        "child: co_await read_some",                     //  5
        "I/O awaitable: start the operation",            //  6
        "I/O awaitable: suspend",                        //  7
        "I/O backend: the operation completes",          //  8
        "executor: post",                                //  9
        "executor: resume",                              // 10
        "child: co_return",                              // 11
        "my_task: co_return",                            // 12
        "run_async: call the completion handler with 5", // 13
    };
    if (trace != diagram)
    {
        std::cout << "trace does not match the diagram\n";
        return 1;
    }
}

Application Developers

You write my_task and child. The rest of the diagram is Capy, or an I/O library such as Corosio.

  • Code between two co_await expressions runs like ordinary sequential code. Each co_await is a point where the task can suspend.

  • A task resumes on the executor it started with. State that only one task uses needs no lock. State that tasks share on a thread pool needs a strand or an async_mutex.

  • Start top-level work from main with run_async. Inside a task, await other tasks directly. To run a child task on a different executor, await it through run.

  • Run tasks concurrently with when_all and when_any.

  • To cancel work, pass a stop token to run_async. Operations that stop early report cond::canceled. To check for a stop request inside a task, read the token with co_await this_coro::stop_token.

  • An object that a task refers to must stay alive until the task finishes. Lifetime and Ownership gives the rules for parameters, lambda captures, and buffers.

  • Stop work and wait for it to finish before you destroy the execution context that it runs on. Destroying the context abandons the chains that are still suspended in it. Their frames are destroyed without resuming, so they deliver no result and call no completion handler.

Read next:

Protocol Libraries

If you write a protocol library, such as HTTP or WebSocket, your child implements the protocol on a stream. Your caller supplies the stream, and an I/O library supplies the I/O awaitables.

  • Write the protocol against the ReadStream and WriteStream concepts. To keep the protocol out of headers, write it against the type-erased any_stream instead.

  • The executor and the stop token reach your task through co_await. Your functions do not take them as parameters.

  • Each stream operation returns a results that holds an error code and a byte count. A read can return bytes and an error together. The stream only reports what happened. Your protocol decides whether the result is a success or a failure, for example whether a partial message is usable.

  • Compare the error code with the conditions in cond, for example cond::eof at the end of the data.

  • When a stop is requested, a pending operation fails with an error that compares equal to cond::canceled. This error means that a caller asked for the stop, not that the protocol failed. Do not retry the operation. Return the error to your caller.

  • Test the protocol with the mock streams in capy::test. The mock streams run without an I/O backend and can inject errors.

Read next:

Framework Authors

You write the task type, such as task<T>, and the functions that start tasks, such as run_async. In the diagram, you write the run_async participant and the transfers between tasks: steps 2, 4, and 11 to 13.

  • A task type is an ContextAwaitable, so that other tasks can await it. It is also an ContextRunnable, so that functions such as run_async can start it.

  • Derive the promise type from context_awaitable_promise_base. The base allocates the coroutine frame with the current frame allocator and stores the environment. It also answers co_await this_coro::…​ queries, and passes every other awaitable to the transform_awaitable of your promise.

  • In transform_awaitable, pass the environment to each I/O awaitable through its two-argument await_suspend.

  • An awaited child task gets the same environment, and it returns to its parent by symmetric transfer. So the parent continues on the same executor.

  • A function that starts tasks creates the environment: an executor, a stop token, a frame allocator, and an owner. It must set the frame allocator before the task’s frame is allocated. Frame Allocators explains why run_async uses two calls for this.

  • The function must also own the chain that it starts, by binding a chain_owner, so that a holder can abandon the chain. The alternative is to make sure that nothing awaits a coroutine that could be abandoned. run_async, run on another executor, and each branch of when_all and when_any own their chains already.

  • To store an executor without its type, use executor_ref.

Read next:

I/O Library Authors

In the diagram, you write the I/O awaitable, and sometimes the I/O backend and the executor.

  • The I/O awaitable implements the ContextAwaitable protocol. Its await_suspend receives the coroutine handle and the await_context.

  • Keep the executor from the environment. When the operation completes, resume the coroutine through that executor. Wrap the handle in a continuation that lives in the awaitable, and pass it to post or dispatch. Copy the owner of the environment into owner, so that the chain can be abandoned while the coroutine waits. Never call resume() on the handle directly.

  • Register a std::stop_callback on the stop token of the environment. The callback runs on the thread that requests the stop. If a stop was already requested, it runs at once, inside the constructor of the callback. It can run at the same time as the completion of the operation. Only one of them may resume the coroutine, so each one must first win an atomic claim.

  • An executor implements the Executor concept. Its context() returns the execution_context that created it. An event loop that runs coroutines resumes them with safe_resume, so that each coroutine keeps its own frame allocator.

  • An execution context that is destroyed with work still queued passes each queued continuation to abandon instead of resuming it.

  • To start Capy tasks from another event loop, use run_async with an executor for that loop.

Read next: