Async GRPC with C++20 coroutine support

agrpc

Build an elegant GRPC async interface with C++20 coroutine and libunifex (target for C++23 executor).

Get started

mkdir build && cd build
conan install ..
cmake -DCMAKE_CXX_FLAGS:STRING=-fcoroutines ..
cmake --build .

Work in progress

One glance:

native async grpc agrpc
class CallData {
 public:
  void Proceed() {
    if (status_ == CREATE) {
      status_ = PROCESS;
      service_->RequestSayHello(&ctx_, &request_,
                                &responder_,
                                cq_, cq_, this);
    } else if (status_ == PROCESS) {
      new CallData(service_, cq_);
  
      std::string prefix("Hello ");
      reply_.set_message(prefix + request_.name());

      status_ = FINISH;
      responder_.Finish(reply_, Status::OK, this);
    } else {
      GPR_ASSERT(status_ == FINISH);
      delete this;
    }
  }

 private:
  // ...
  enum CallStatus { CREATE, PROCESS, FINISH };
  CallStatus status_{CREATE};  // The current serving state.
};

void HandleRpcs() {
  new CallData(&service_, cq_.get());
  void* tag;  // uniquely identifies a request.
  bool ok;
  while (true) {
    GPR_ASSERT(cq_->Next(&tag, &ok));
    GPR_ASSERT(ok);
    static_cast<CallData*>(tag)->Proceed();
  }
}
while (true) {
  grpc::ServerContext server_context;
  helloworld::HelloRequest request;
  grpc::ServerAsyncResponseWriter<helloworld::HelloReply> writer{
      &server_context};
  bool request_ok = co_await agrpc::AsyncRequest(
      grpc_context.get_scheduler(),
      &helloworld::Greeter::AsyncService::RequestSayHello,
      service, server_context, request, writer);
  if (!request_ok)
    co_return;
  helloworld::HelloReply response;
  response.set_message("Hello " + request.name());
  co_await agrpc::AsyncFinish(grpc_context.get_scheduler(), writer,
                              response, grpc::Status::OK);
}

Benchmark

1 CPU server

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
cpp_agrpc 35385 26.74 ms 39.48 ms 49.60 ms 64.11 ms 92.43% 28.78 MiB
cpp_grpc_st 35364 26.03 ms 48.25 ms 58.39 ms 72.63 ms 80.95% 21.57 MiB
cpp_grpc_mt 35068 26.23 ms 48.16 ms 58.60 ms 72.72 ms 80.29% 18.05 MiB
rust_tonic_st 33407 25.96 ms 54.54 ms 60.15 ms 68.41 ms 66.22% 15.34 MiB
dotnet_grpc 32125 26.78 ms 53.29 ms 66.82 ms 81.76 ms 95.1% 94.31 MiB
rust_tonic_mt 31428 29.06 ms 90.03 ms 97.46 ms 106.46 ms 69.87% 13.52 MiB
cpp_grpc_callback 12283 80.54 ms 103.30 ms 107.25 ms 178.37 ms 100.27% 406.88 MiB
go_grpc 5600 176.37 ms 306.34 ms 397.40 ms 598.96 ms 100.41% 44.02 MiB

2 CPU server

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
rust_tonic_mt 77236 11.97 ms 30.99 ms 44.78 ms 68.00 ms 191.33% 17.42 MiB
cpp_agrpc 72084 13.50 ms 20.83 ms 23.73 ms 31.08 ms 202.99% 86.8 MiB
cpp_grpc_mt 68099 14.18 ms 34.95 ms 39.29 ms 45.88 ms 203.08% 61.64 MiB
dotnet_grpc 63479 13.98 ms 20.35 ms 26.62 ms 61.63 ms 201.58% 175.97 MiB
rust_tonic_st 61962 15.83 ms 19.73 ms 21.05 ms 24.13 ms 102.46% 16.32 MiB
cpp_grpc_st 50593 19.47 ms 23.59 ms 25.26 ms 29.66 ms 102.15% 19.53 MiB
cpp_grpc_callback 29612 32.49 ms 74.33 ms 78.51 ms 85.74 ms 202.37% 444.44 MiB
go_grpc 12050 80.52 ms 188.05 ms 194.19 ms 286.45 ms 200.78% 45.11 MiB

Goal

Maybe something like C# interface.

public override async Task RouteChat(Grpc.Core.IAsyncStreamReader<RouteNote> requestStream,
    Grpc.Core.IServerStreamWriter<RouteNote> responseStream,
    Grpc.Core.ServerCallContext context)
{
    while (await requestStream.MoveNext())
    {
        var note = requestStream.Current;
        List<RouteNote> prevNotes = AddNoteForLocation(note.Location, note);
        foreach (var prevNote in prevNotes)
        {
            await responseStream.WriteAsync(prevNote);
        }
    }
}

Reference

Owner
Yuchao Zhang
speech synthesis/machine learning
Yuchao Zhang
Similar Resources

Elle - The Elle coroutine-based asynchronous C++ development framework.

Elle - The Elle coroutine-based asynchronous C++ development framework.

Elle, the coroutine-based asynchronous C++ development framework Elle is a collection of libraries, written in modern C++ (C++14). It contains a rich

Jan 1, 2023

A go-style coroutine library in C++11 and more.

A go-style coroutine library in C++11 and more.

cocoyaxi English | 简体中文 A go-style coroutine library in C++11 and more. 0. Introduction cocoyaxi (co for short), is an elegant and efficient cross-pla

Dec 27, 2022

C++14 coroutine-based task library for games

SquidTasks Squid::Tasks is a header-only C++14 coroutine-based task library for games. Full project and source code available at https://github.com/we

Nov 30, 2022

A library OS for Linux multi-process applications, with Intel SGX support

Graphene Library OS with Intel SGX Support A Linux-compatible Library OS for Multi-Process Applications NOTE: We are in the middle of transitioning ou

Jan 4, 2023

EnkiTS - A permissively licensed C and C++ Task Scheduler for creating parallel programs. Requires C++11 support.

EnkiTS - A permissively licensed C and C++ Task Scheduler for creating parallel programs. Requires C++11 support.

Support development of enkiTS through Github Sponsors or Patreon enkiTS Master branch Dev branch enki Task Scheduler A permissively licensed C and C++

Dec 27, 2022

Mini-async-log-c - Mini async log C port. Now with C++ wrappers.

Description A C11/C++11 low-latency wait-free producer (when using Thread Local Storage) asynchronous textual data logger with type-safe strings. Base

Nov 9, 2022

Coroutine - C++11 single .h asymmetric coroutine implementation via ucontext / fiber

C++11 single .h asymmetric coroutine implementation API in namespace coroutine: routine_t create(std::functionvoid() f); void destroy(routine_t id);

Dec 20, 2022

The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

gRPC - An RPC library and framework gRPC is a modern, open source, high-performance remote procedure call (RPC) framework that can run anywhere. gRPC

Dec 30, 2022

Asynchronous gRPC with Boost.Asio executors

asio-grpc This library provides an implementation of boost::asio::execution_context that dispatches work to a grpc::CompletionQueue. Making it possibl

Dec 31, 2022

gRPC - An RPC library and framework Baind Unity 3D Project

Unity 3D Compose for Desktop and Android, a modern UI framework for C ++ , C# that makes building performant and beautiful user interfaces easy and enjoyable.

May 19, 2022

Async++ concurrency framework for C++11

Async++ Async++ is a lightweight concurrency framework for C++11. The concept was inspired by the Microsoft PPL library and the N3428 C++ standard pro

Dec 30, 2022

Mongoose Embedded Web Server Library - a multi-protocol embedded networking library with TCP/UDP, HTTP, WebSocket, MQTT built-in protocols, async DNS resolver, and non-blocking API.

Mongoose Embedded Web Server Library - a multi-protocol embedded networking library with TCP/UDP, HTTP, WebSocket,  MQTT built-in protocols, async DNS resolver, and non-blocking API.

Mongoose - Embedded Web Server / Embedded Networking Library Mongoose is a networking library for C/C++. It implements event-driven non-blocking APIs

Jan 1, 2023

Async & Concurrent Servers implemented in C

Concurrent servers in c Imlementation of concurrent severs in c from scratch using this awesome blog as a tutorial. Project Structure . ├── readme.md

Jun 21, 2021

Example programs for Talking Async videos

Talking Async Example programs for the Talking Async videos, which can be found on YouTube. Episode 1: Why C++20 is the Awesomest Language for Network

Dec 28, 2022

A PostgreSQL extension providing an async networking interface accessible via SQL using a background worker and curl.

pg_net is a PostgreSQL extension exposing a SQL interface for async networking with a focus on scalability and UX.

Dec 14, 2022

An SQLite binding for node.js with built-in encryption, focused on simplicity and (async) performance

Description An SQLite (more accurately SQLite3MultipleCiphers) binding for node.js focused on simplicity and (async) performance. When dealing with en

May 15, 2022

Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Jan 4, 2023

A C++ async HTTP client library to use in asynchronous applications while communicating with REST services.

libashttp An asynchronous HTTP library using Boost.ASIO as the backend. This project is licensed under: Usage Here is a example usage which is taken f

Dec 17, 2022

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

Jan 8, 2023
Comments
  • Why cpp_agrpc is faster than grpc_callback?

    Why cpp_agrpc is faster than grpc_callback?

    Hi, as i know coroutine just change the style of async programming, don't improve performance, but i noticed your benchmark, the coroutine version is much faster than callback version on multiple cores server, why? What optimizations you did?

  • Fix clock_gettime for Apple clang

    Fix clock_gettime for Apple clang

    The PR fixes clock_gettime lack of type conversion error for Apple clang version 13.1.6 (clang-1316.0.21.2.5):

    /Users/genertorg/git/agrpc/agrpc/base/chrono.cc:31:3: error: no matching function for call to 'clock_gettime'
      clock_gettime(type, &ts);
      ^~~~~~~~~~~~~
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include/time.h:178:5: note: candidate function not viable: no known conversion from 'int' to 'clockid_t' for 1st argument
    int clock_gettime(clockid_t __clock_id, struct timespec *__tp);
        ^
    1 error generated.
    make[2]: *** [agrpc/base/CMakeFiles/agrpc_base_chrono.dir/chrono.cc.o] Error 1
    make[1]: *** [agrpc/base/CMakeFiles/agrpc_base_chrono.dir/all] Error 2
    
Async++ concurrency framework for C++11

Async++ Async++ is a lightweight concurrency framework for C++11. The concept was inspired by the Microsoft PPL library and the N3428 C++ standard pro

Dec 30, 2022
Open source PHP extension for Async IO, Coroutines and Fibers

Swoole is an event-driven asynchronous & coroutine-based concurrency networking communication engine with high performance written in C++ for PHP. Ope

Jan 4, 2023
Powerful multi-threaded coroutine dispatcher and parallel execution engine

Quantum Library : A scalable C++ coroutine framework Quantum is a full-featured and powerful C++ framework build on top of the Boost coroutine library

Dec 30, 2022
:copyright: Concurrent Programming Library (Coroutine) for C11

libconcurrent tiny asymmetric-coroutine library. Description asymmetric-coroutine bidirectional communication by yield_value/resume_value native conte

Sep 2, 2022
Single header asymmetric stackful cross-platform coroutine library in pure C.
Single header asymmetric stackful cross-platform coroutine library in pure C.

minicoro Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind. The proje

Dec 29, 2022
A golang-style C++ coroutine library and more.

CO is an elegant and efficient C++ base library that supports Linux, Windows and Mac platforms. It pursues minimalism and efficiency, and does not rely on third-party library such as boost.

Jan 5, 2023
A C++20 coroutine library based off asyncio
A C++20 coroutine library based off asyncio

kuro A C++20 coroutine library, somewhat modelled on Python's asyncio Requirements Kuro requires a C++20 compliant compiler and a Linux OS. Tested on

Nov 9, 2022
C++20 Coroutine-Based Synchronous Parser Combinator Library

This library contains a monadic parser type and associated combinators that can be composed to create parsers using C++20 Coroutines.

Dec 17, 2022
Cppcoro - A library of C++ coroutine abstractions for the coroutines TS

CppCoro - A coroutine library for C++ The 'cppcoro' library provides a large set of general-purpose primitives for making use of the coroutines TS pro

Dec 30, 2022
Mx - C++ coroutine await, yield, channels, i/o events (single header + link to boost)

mx C++11 coroutine await, yield, channels, i/o events (single header + link to boost). This was originally part of my c++ util library kit, but I'm se

Sep 21, 2019