Stop Tokens and Cancellation
The Problem
-
A user clicks "Cancel" on a download dialog
-
A timeout expires while waiting for a network response
-
A connection drops unexpectedly
-
An application is shutting down
The Naive Approach: Boolean Flags
The obvious solution seems to be a boolean flag:
std::atomic<bool> should_cancel{false};
void worker()
{
while (!should_cancel)
{
do_work();
}
}
This approach has problems:
-
No standardization — Every component invents its own cancellation flag
-
Race conditions — Checking the flag and acting on it is not atomic
-
No cleanup notification — The worker just stops; no opportunity for graceful cleanup
-
Polling overhead — Must check the flag repeatedly
C++20 Stop Tokens—A General-Purpose Signaling Mechanism
C++20 introduces std::stop_token, std::stop_source, and std::stop_callback. While named for "stopping," these implement a general-purpose Observer pattern—a thread-safe one-to-many notification system.
The Three Components
std::stop_source-
The Subject/Publisher. Owns the shared state and can trigger notifications. Create one source, then distribute tokens to observers.
std::stop_token-
The Subscriber View. A read-only, copyable, cheap-to-pass-around handle. Multiple tokens can share the same underlying state.
std::stop_callback<F>-
The Observer Registration. An RAII object that registers a callback to run when signaled. Destruction automatically unregisters.
How They Work Together
#include <stop_token>
#include <iostream>
void example()
{
std::stop_source source;
// Create tokens (distribute notification capability)
std::stop_token token1 = source.get_token();
std::stop_token token2 = source.get_token(); // Same underlying state
// Register callbacks (observers)
std::stop_callback cb1(token1, []{ std::cout << "Observer 1 notified\n"; });
std::stop_callback cb2(token2, []{ std::cout << "Observer 2 notified\n"; });
std::cout << "Before signal\n";
source.request_stop(); // Triggers all callbacks
std::cout << "After signal\n";
}
Output:
Before signal
Observer 1 notified
Observer 2 notified
After signal
Immediate Invocation
If a callback is registered after request_stop() was already called, the callback runs immediately in the constructor:
std::stop_source source;
source.request_stop(); // Already signaled
// Callback runs in constructor, not later
std::stop_callback cb(source.get_token(), []{
std::cout << "Runs immediately!\n";
});
This ensures observers never miss the signal, regardless of registration timing.
Type-Erased Polymorphic Observers
Each stop_callback<F> stores a different callable type F. Despite this, all callbacks for a given source can be invoked uniformly. This is equivalent to having vector<function<void()>> but with:
-
No heap allocation per callback
-
No virtual function overhead
-
RAII lifetime management
The One-Shot Nature
|
Critical limitation:
|
Why This Matters
If you design a system that needs to cancel and restart operations, you cannot reuse the same stop_source. Each cycle requires a fresh source and fresh tokens.
The Reset Workaround
To "reset," create an entirely new stop_source:
std::stop_source source;
auto token = source.get_token();
// ... distribute token to workers ...
source.request_stop(); // Triggered, now permanently signaled
// To "reset": create new source
source = std::stop_source{}; // New shared state
// Old tokens still see the old, already-signaled state
// Must redistribute new tokens to ALL holders of the old token
auto new_token = source.get_token();
This is manual and error-prone. Any code still holding the old token does not receive new signals.
Design Implication
If you need repeatable signals, stop_token is the wrong tool. Consider:
-
Condition variables for repeatable wake-ups
-
Atomic flags with explicit reset protocol
-
Custom event types
Beyond Cancellation
The "stop" naming obscures the mechanism’s generality. stop_token implements one-shot broadcast notification, useful for:
-
Starting things — Signal "ready" to trigger initialization
-
Configuration loaded — Notify components when config is available
-
Resource availability — Signal when database connected or cache warmed
-
Any one-shot broadcast scenario
Stop Tokens in Coroutines
Coroutines have a propagation problem: passing a stop token explicitly to every function clutters your APIs.
Capy’s Answer: Automatic Propagation
Capy propagates stop tokens downward through co_await. When you await a task, the ContextAwaitable protocol passes the current stop token to the child:
capy::task<> parent()
{
// Our stop token is automatically passed to child
co_await child();
}
capy::task<> child()
{
// Receives parent's stop token via ContextAwaitable protocol
auto token = co_await capy::this_coro::stop_token; // Access current token
}
Accessing the Stop Token
Inside a task, use co_await this_coro::stop_token to access the current stop token:
capy::task<> cancellable_work()
{
auto token = co_await capy::this_coro::stop_token;
while (!token.stop_requested())
{
co_await do_chunk_of_work();
}
}
Why Not coroutine_handle::destroy()?
std::coroutine_handle::destroy() is the C++20 primitive that frees a coroutine frame. It is not a cancellation mechanism. It has the same flaw as forceful thread interruption. The coroutine is torn down with no opportunity to complete pending I/O, release locks, or run RAII destructors in the expected order.
Capy exposes handle() and handle() so that Capy’s own launchers (run_async, run) and custom integrations can dispatch coroutines through executors. Calling destroy() on such a handle while the coroutine is being awaited by a parent produces undefined behavior: the parent still holds the frame and will destroy it again. A holder that must give up a suspended coroutine for good, such as an execution context being torn down, passes its continuation to abandon instead, which unwinds the chain from the top on its owner’s executor.
The rule:
Responding to Cancellation
Checking the Token
capy::task<> process_items(std::vector<Item> const& items)
{
auto token = co_await capy::this_coro::stop_token;
for (auto const& item : items)
{
if (token.stop_requested())
co_return; // Exit early
co_await process(item);
}
}
Cleanup with RAII
RAII ensures resources are released on early exit:
capy::task<> with_resource()
{
auto resource = acquire_resource(); // RAII wrapper
auto token = co_await capy::this_coro::stop_token;
while (!token.stop_requested())
{
co_await use_resource(resource);
}
// resource destructor runs regardless of how we exit
}
The canceled Convention
When cancellation causes an operation to fail, the conventional error code is error::canceled, which compares equal to the portable condition cond::canceled:
capy::task<std::string> fetch_with_cancel()
{
auto token = co_await capy::this_coro::stop_token;
if (token.stop_requested())
{
throw std::system_error(
capy::make_error_code(capy::error::canceled));
}
co_return co_await do_fetch();
}
OS Integration
When Capy’s I/O is provided by Corosio, requesting stop cancels work in progress rather than waiting for the operation to finish on its own. Corosio cancels the pending operation through whatever backend is active for the platform, and it resolves promptly with std::errc::operation_canceled.
The mechanism depends on the backend. Completion-based backends (Windows IOCP, and io_uring when enabled on Linux) cancel the operation in the kernel. Readiness-based backends (Linux epoll, kqueue on the BSDs and macOS, and the portable select fallback) remove it from the reactor before its system call runs. Either way the operation is reported as cancelled instead of blocking until the I/O would have completed.
The Cancellation Sequence
The diagram follows a stop request from the caller to a suspended task. The task awaits an I/O awaitable that observes the stop token. How a Task Runs shows the same task when it completes normally.
-
The caller owns a
std::stop_source. It passes the token of the source torun_async, which puts the token in the environment of the task. -
The task starts, as in How a Task Runs. Every task that it awaits gets the same stop token.
-
The task awaits an I/O awaitable.
-
The I/O awaitable registers a
std::stop_callbackon the stop token. -
The task suspends, and the thread goes back to the executor.
-
The caller calls
request_stop()on the source. The stop callback runs at once, on the thread of the caller. -
The stop request and the completion of the operation race. The I/O awaitable uses one atomic claim to pick the winner. If the stop request wins, the I/O awaitable posts the task to the executor. It does not resume the task on the thread of the caller.
-
The executor resumes the task. The result of the operation compares equal to
cond::canceled. -
The task finishes. It can return the error, throw, or handle the cancellation itself.
-
run_asynccalls the completion handler with the result of the task.
If the operation completed first, the stop callback does nothing. The completion resumes the task with the result of the operation, as in How a Task Runs.
In both cases the task resumes and returns through its own code. Abandoning a chain is different: its frames are destroyed without resuming. Who Owns a Suspended Coroutine explains when that happens.
|
The stop callback runs on the thread that calls
|
async_waker is a complete I/O awaitable that implements this sequence, including the atomic claim.
It does not make the winner wait for await_suspend, so it requires a single-threaded executor or a strand.
The next section shows the stop callback in detail.
Implementing Stoppable Awaitables
The examples above show polling for cancellation with token.stop_requested(). For awaitables that suspend indefinitely—waiting for I/O, a lock, or an external event—you need a std::stop_callback to wake the coroutine when cancellation arrives.
The Dangerous Pattern
A std::stop_callback fires synchronously on whatever thread calls request_stop().
The callback must not call resume() on the coroutine handle, because every resumption goes through the executor.
Stop Callbacks Must Post, Not Resume explains why.
The next obvious solution, posting the coroutine through the executor, is also wrong:
// WRONG: the completion and the stop callback can both resume the
// coroutine.
struct racy_read
{
struct cancel_fn
{
racy_read* self;
void operator()() const noexcept
{
self->canceled_ = true;
self->ex_.post(self->cont_); // even if the read completed
}
};
capy::continuation cont_;
capy::executor_ref ex_;
bool canceled_ = false;
std::optional<std::stop_callback<cancel_fn>> stop_cb_;
bool await_ready() const noexcept { return false; }
std::coroutine_handle<> await_suspend(
std::coroutine_handle<> h, capy::await_context const* env)
{
cont_.h = h;
cont_.owner = env->owner;
ex_ = env->executor;
stop_cb_.emplace(env->stop_token, cancel_fn{this});
start_read(this); // the backend calls on_complete() later
return std::noop_coroutine();
}
// Called by the I/O backend, on any thread, when the read finishes.
void on_complete() noexcept
{
ex_.post(cont_); // even if the stop callback posted already
}
void await_resume() noexcept { stop_cb_.reset(); }
};
The stop callback posts the coroutine without checking whether the read already completed. The completion posts it without checking whether a stop request already did. When the read completes at about the same time as a stop request, both post, and the coroutine resumes twice. The second resumption runs a coroutine that has already moved on, or whose frame is already destroyed.
This bug is a race. Tests rarely trigger it, and under load it shows up as a crash or as memory corruption in unrelated code.
The Correct Pattern: Claim, Then Post
Posting the coroutine through the executor is necessary, but it is not enough. The stop callback and the completion of the operation race, so each one must first win the claim:
-
read_op::claimis the atomic claim. Only the winner writes the result. -
arrivals_makes the winner wait forawait_suspend. The winner and the end ofawait_suspendboth arrive, and the second to arrive resumes the coroutine. Ifawait_suspendarrives second, it resumes the coroutine directly, because it already runs on the executor. If the winner arrives second, it posts the coroutine through the executor.
A stoppable I/O awaitable that claims, then posts
// An I/O object with one pending read. The I/O backend calls complete()
// from any thread when the read finishes.
class read_op
{
public:
struct awaiter;
awaiter wait() noexcept;
// Called by the I/O backend. Ignored if no wait is armed, or if a
// stop request already claimed the wait.
void complete(std::size_t n) noexcept;
bool waiting() const noexcept
{
return armed_.load(std::memory_order_acquire);
}
private:
// The claim: whoever moves armed_ from true to false owns the
// result. The loser does not touch the waiter.
bool claim() noexcept
{
bool expected = true;
return armed_.compare_exchange_strong(
expected, false, std::memory_order_acq_rel);
}
std::atomic<bool> armed_{false};
awaiter* waiter_ = nullptr;
};
struct read_op::awaiter
{
// Runs on the thread that calls request_stop().
struct cancel_fn
{
awaiter* self;
void operator()() const noexcept
{
if (!self->op_->claim())
return; // complete() won, or nothing is armed yet
self->canceled_ = true;
self->arrive();
}
};
read_op* op_;
capy::continuation cont_;
capy::executor_ref ex_;
std::size_t bytes_ = 0;
bool canceled_ = false;
// The winner and the end of await_suspend both arrive here. The
// second to arrive resumes the coroutine, so it cannot resume while
// await_suspend is still running.
std::atomic<int> arrivals_{0};
std::optional<std::stop_callback<cancel_fn>> stop_cb_;
explicit awaiter(read_op* op) noexcept : op_(op) {}
// co_await can move the awaiter, but only before await_suspend,
// when there is no stop callback yet.
awaiter(awaiter&& other) noexcept : op_(other.op_) {}
// Called by the winner. If await_suspend has returned, post the
// coroutine; otherwise await_suspend resumes it.
void arrive() noexcept
{
if (arrivals_.fetch_add(1, std::memory_order_acq_rel) == 1)
ex_.post(cont_); // last action: *this can be destroyed now
}
bool await_ready() const noexcept { return false; }
std::coroutine_handle<> await_suspend(
std::coroutine_handle<> h, capy::await_context const* env)
{
if (env->stop_token.stop_requested())
{
canceled_ = true;
return h;
}
// 1. Write everything the winner reads, before anyone can win.
// The owner lets a holder abandon the chain while it waits.
cont_.h = h;
cont_.owner = env->owner;
ex_ = env->executor;
op_->waiter_ = this;
// 2. If a stop is requested now, the callback runs here and
// loses, because nothing is armed yet. Step 4 catches it.
stop_cb_.emplace(env->stop_token, cancel_fn{this});
// 3. Arm. complete() or the stop callback can win from here on.
op_->armed_.store(true, std::memory_order_release);
// 4. Claim a stop request that arrived during step 2.
if (env->stop_token.stop_requested() && op_->claim())
{
canceled_ = true;
return h;
}
// 5. Arrive. If the winner came first, resume now, on this
// thread. Otherwise the winner posts the coroutine later.
if (arrivals_.fetch_add(1, std::memory_order_acq_rel) == 1)
return h;
return std::noop_coroutine();
}
capy::results<std::size_t> await_resume() noexcept
{
// Waits for a stop callback that is still running to return.
stop_cb_.reset();
if (canceled_)
return {capy::make_error_code(capy::error::canceled), 0};
return {std::error_code(), bytes_};
}
};
inline read_op::awaiter read_op::wait() noexcept
{
return awaiter{this};
}
inline void read_op::complete(std::size_t n) noexcept
{
if (!claim())
return;
waiter_->bytes_ = n;
waiter_->arrive();
}
| Capy’s built-in I/O awaitables (via Corosio) already use the post-back pattern internally. This guidance applies when writing your own custom awaitables. |
Patterns
Racing a Deadline
A timeout is expressed as a race: the operation you care about against
a deadline. async_waker provides the suspension point for both sides
of that race; a user thread supplies the clock:
struct fetch_channel
{
capy::async_waker fetch_ready;
std::atomic<bool> cancelled{false};
std::string result;
};
// One side of the race: completes when the fetch worker thread
// wakes fetch_ready. If the deadline wins first, when_any's stop
// request cancels the wait; flag the worker so it stops early.
capy::task<capy::results<std::string>> await_fetch(fetch_channel& ch)
{
auto [ec] = co_await ch.fetch_ready.wait();
if (ec)
{
ch.cancelled.store(true);
co_return capy::results<std::string>{ec, {}};
}
co_return capy::results<std::string>{std::error_code(), std::move(ch.result)};
}
// The other side of the race: completes once whatever plays the
// clock wakes the waker.
capy::task<capy::results<>> deadline(capy::async_waker& waker)
{
auto [ec] = co_await waker.wait();
co_return capy::results<>{ec};
}
when_any(await_fetch(ch), deadline(waker)) returns as soon as either side
wakes; the loser’s wait resolves with cond::canceled. See
Timeout with Cancellation
for the full runnable demo, including the two std::thread objects that
play fetch worker and clock.
When the work you are racing is Corosio I/O rather than coroutine-internal
work, Corosio’s own timed operations take the place of the deadline
thread. They schedule against the platform event loop instead of a
std::this_thread::sleep_for.
User Cancellation
Connect UI cancellation to stop tokens. Pass the token through run_async so it propagates automatically via the execution environment. The task accesses it with co_await this_coro::stop_token instead of receiving it as a function argument:
class download_manager
{
capy::executor_ref executor_;
std::stop_source stop_source_;
public:
void start_download(std::string url)
{
// Token propagated via await_context, not as a function argument
capy::run_async(executor_, stop_source_.get_token())(download(url));
}
void cancel()
{
stop_source_.request_stop();
}
};
capy::task<void> download(std::string url)
{
// From run_async's await_context
auto token = co_await capy::this_coro::stop_token;
while (!token.stop_requested())
{
co_await fetch_next_chunk(url);
}
}
Graceful Shutdown
Cancel all pending work during shutdown:
class server
{
std::stop_source shutdown_source_;
public:
void shutdown()
{
shutdown_source_.request_stop();
// All pending operations receive stop request
}
capy::task<> handle_connection(connection conn)
{
auto token = shutdown_source_.get_token();
while (!token.stop_requested())
{
co_await process_request(conn);
}
// Graceful cleanup
co_await send_goodbye(conn);
}
};
when_any Cancellation
when_any uses stop tokens internally to cancel "losing" tasks when the first task completes. This is covered in Concurrent Composition.