Packio - An asynchronous msgpack-RPC and JSON-RPC library built on top of Boost.Asio.

Header-only | JSON-RPC | msgpack-RPC | asio | coroutines

This library requires C++17 and is designed as an extension to boost.asio. It will let you build asynchronous servers or client for JSON-RPC or msgpack-RPC.

The project is hosted on GitHub and available on Conan Center. Documentation is available on GitHub Pages.

Overview

#include <iostream>

#include <packio/packio.h>

using packio::arg;
using packio::nl_json_rpc::completion_handler;
using packio::nl_json_rpc::make_client;
using packio::nl_json_rpc::make_server;
using packio::nl_json_rpc::rpc;

int main(int, char**)
{
    using namespace packio::arg_literals;

    // Declare a server and a client, sharing the same io_context
    packio::net::io_context io;
    packio::net::ip::tcp::endpoint bind_ep{
        packio::net::ip::make_address("127.0.0.1"), 0};
    auto server = make_server(packio::net::ip::tcp::acceptor{io, bind_ep});
    auto client = make_client(packio::net::ip::tcp::socket{io});

    // Declare a synchronous callback with named arguments
    server->dispatcher()->add(
        "add", {"a", "b"}, [](int a, int b) { return a + b; });
    // Declare an asynchronous callback with named arguments
    server->dispatcher()->add_async(
        "multiply", {"a", "b"}, [&io](completion_handler complete, int a, int b) {
            // Call the completion handler later
            packio::net::post(
                io, [a, b, complete = std::move(complete)]() mutable {
                    complete(a * b);
                });
        });
    // Declare a coroutine with unnamed arguments
    server->dispatcher()->add_coro(
        "pow", io, [](int a, int b) -> packio::net::awaitable<int> {
            co_return std::pow(a, b);
        });

    // Connect the client
    client->socket().connect(server->acceptor().local_endpoint());
    // Accept connections
    server->async_serve_forever();
    // Run the io_context
    std::thread thread{[&] { io.run(); }};

    // Make an asynchronous call with named arguments
    std::promise<int> add1_result, multiply_result;
    client->async_call(
        "add",
        std::tuple{arg("a") = 42, arg("b") = 24},
        [&](packio::error_code, const rpc::response_type& r) {
            add1_result.set_value(r.result.get<int>());
        });
    std::cout << "42 + 24 = " << add1_result.get_future().get() << std::endl;

    // Use packio::net::use_future with named arguments and literals
    auto add_future = client->async_call(
        "multiply",
        std::tuple{"a"_arg = 12, "b"_arg = 23},
        packio::net::use_future);
    std::cout << "12 * 23 = " << add_future.get().result.get<int>() << std::endl;

    // Spawn the coroutine and wait for its completion
    std::promise<int> pow_result;
    packio::net::co_spawn(
        io,
        [&]() -> packio::net::awaitable<void> {
            // Call using an awaitable and positional arguments
            auto res = co_await client->async_call(
                "pow", std::tuple{2, 8}, packio::net::use_awaitable);
            pow_result.set_value(res.result.get<int>());
        },
        packio::net::detached);
    std::cout << "2 ** 8 = " << pow_result.get_future().get() << std::endl;

    io.stop();
    thread.join();

    return 0;
}

Requirements

  • C++17 or C++20
  • msgpack >= 3.2.1
  • nlohmann_json >= 3.9.1
  • boost.asio >= 1.70.0 or asio >= 1.13.0

Older versions of msgpack and nlohmann_json are probably compatible but they are not tested on the CI.

Configurations

Standalone or Boost.Asio

By default, packio uses boost::asio. It is also compatible with standalone asio. To use the standalone version, the preprocessor macro PACKIO_STANDALONE_ASIO=1 must be defined.

If you are using the conan package, you can use the option standalone_asio=True.

Depending on your choice, the namespace packio::net will be an alias for either boost::asio or asio.

RPC components

You can define the following preprocessor macros to either 0 or 1 to force-disable or force-enable components of packio:

  • PACKIO_HAS_MSGPACK
  • PACKIO_HAS_NLOHMANN_JSON
  • PACKIO_HAS_BOOST_JSON

If you're using the conan package, use the associated options instead, conan will define these macros accordingly.

If you're not using the conan package, packio will try to auto-detect whether these components are available on your system. Define the macros to the appropriate value if you encounter any issue.

Boost before 1.75

If you're using the conan package with a boost version older than 1.75, you need to manually disable Boost.Json with the options boost_json=False.

Tested compilers

  • gcc-7
  • gcc-8
  • gcc-9
  • gcc-10
  • clang-6
  • clang-7
  • clang-8
  • clang-9
  • clang-10
  • clang-11
  • Apple clang-12
  • Visual Studio 2019 Version 16.8

Older compilers may be compatible but are not tested.

Install with conan

conan install packio/x.x.x

Coroutines

packio is compatible with C++20 coroutines:

  • calls can use the packio::asio::use_awaitable completion token
  • coroutines can be registered in the server

Coroutines are tested for the following compilers:

  • gcc-10 (with -fcoroutines)
  • clang-11 (with libc++)
  • Apple clang-12

Samples

You will find some samples in test_package/samples/ to help you get a hand on packio.

Bonus

Let's compute fibonacci's numbers recursively over websockets with coroutines on a single thread ... in 65 lines of code.

#include <iostream>

#include <packio/extra/websocket.h>
#include <packio/packio.h>

using packio::msgpack_rpc::make_client;
using packio::msgpack_rpc::make_server;
using packio::net::ip::make_address;

using awaitable_tcp_stream = decltype(packio::net::use_awaitable_t<>::as_default_on(
    std::declval<boost::beast::tcp_stream>()));
using websocket = packio::extra::
    websocket_adapter<boost::beast::websocket::stream<awaitable_tcp_stream>, true>;
using ws_acceptor =
    packio::extra::websocket_acceptor_adapter<packio::net::ip::tcp::acceptor, websocket>;

int main(int argc, char** argv)
{
    if (argc < 2) {
        std::cerr << "I require one argument" << std::endl;
        return 1;
    }
    const int n = std::atoi(argv[1]);

    packio::net::io_context io;
    packio::net::ip::tcp::endpoint bind_ep{make_address("127.0.0.1"), 0};

    auto server = make_server(ws_acceptor{io, bind_ep});
    auto client = make_client(websocket{io});

    server->dispatcher()->add_coro(
        "fibonacci", io, [&](int n) -> packio::net::awaitable<int> {
            if (n <= 1) {
                co_return n;
            }

            auto r1 = co_await client->async_call("fibonacci", std::tuple{n - 1});
            auto r2 = co_await client->async_call("fibonacci", std::tuple{n - 2});

            co_return r1.result.as<int>() + r2.result.as<int>();
        });

    int result = 0;
    packio::net::co_spawn(
        io,
        [&]() -> packio::net::awaitable<void> {
            auto ep = server->acceptor().local_endpoint();
            co_await client->socket().next_layer().async_connect(ep);
            co_await client->socket().async_handshake(
                "127.0.0.1:" + std::to_string(ep.port()), "/");
            auto ret = co_await client->async_call("fibonacci", std::tuple{n});
            result = ret.result.template as<int>();
            io.stop();
        },
        packio::net::detached);

    server->async_serve_forever();
    io.run();

    std::cout << "F{" << n << "} = " << result << std::endl;

    return 0;
}
Comments
  • Separated client and server

    Separated client and server

    Based on my previous request Closed Issue #65 i have a further question. I'm using the nl_json_rpc client and I tried to separate the client and server in two different programs (executable). From the client i can connect to the server socket (in the code below i get "The connection has been established!")

    packio::net::io_context io_context;
    packio::net::ip::tcp::socket socket{io_context};
    packio::net::ip::tcp::resolver resolver{io_context};
    auto endpoints = resolver.resolve("localhost", std::to_string(6543));
    
    packio::error_code error;
    auto client = make_client(std::move(socket));
    client->socket().connect(endpoints->endpoint(), error);
    
    if(not error)
    {
        std::cout << "The connection has been established!";
    }
    else
    {
        std::cerr << "Something went wrong :(";
    }
    

    But for some reason the following RPC async_call does not work anymore, e.g. it seems it waits forever for the add_result.get_future().get()

    std::promise<int> add_result, multiply_result;
    client->async_call(
        "add",
        std::tuple{arg("a") = 42, arg("b") = 24},
        [&](packio::error_code, const rpc::response_type& r) {
            add_result.set_value(r.result.get<int>());
        });
    std::cout << "42 + 24 = " << add_result.get_future().get() << std::endl;
    

    May i ask you if you know what the problem is or you can add an example with separated client & server?

  • RPC client as class member variable

    RPC client as class member variable

    How do i define the rpc client as a class member variable? I cannot use auto. I have to setup a rpc connection to serveral devices and i thought the best to do this is to instantiate a device object from a device class, each holding the rpc client as a member variable.

  • Progress support with WebSocket adapter

    Progress support with WebSocket adapter

    I have WebSocket server over JSON-RPC (MR https://github.com/qchateau/packio/pull/60), that can do long (from 1 to 30 minutes) the API call and can generate progress (%) with extra data (remaining time, sec). Can I do progress somehow?

  • Missing optional header

    Missing optional header

    The optional header isn't included in traits.h. It causes some errors with MSVC:

    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(88): error C2065: 'optional': undeclared identifier
      E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(89): note: see reference to class template instantiation 'packio::traits::AsCallHandler<T,Result>' being compiled
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(88): error C3544: '_Args': parameter pack expects a type template argument
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(88): error C2993: 'unknown-type': is not a valid type for non-type template parameter 'condition'
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(88): error C2955: 'packio::traits::Trait': use of class template requires template argument list
      E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(62): note: see declaration of 'packio::traits::Trait'
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\traits.h(88): error C2143: syntax error: missing ',' before '>'
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\client.h(166): error C2061: syntax error: identifier 'optional'
      E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\client.h(483): note: see reference to class template instantiation 'packio::client<Socket,Map>' being compiled
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\client.h(182): error C2061: syntax error: identifier 'optional'
    E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\client.h(414): error C2061: syntax error: identifier 'optional'
      E:\Users\Yoann\.conan\data\packio\1.3.0\_\_\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\packio\client.h(472): note: see reference to class template instantiation 'packio::client<Socket,Map>::initiate_async_call<Buffer>' being compiled
    
  • C++20 should use coroutines by default

    C++20 should use coroutines by default

    ATM, C++20 and coroutines usage is differenciated in tests because compilers do not properly support coroutines. In a (close?) future, C++20 and coroutines usage should go hand in hand.

    Compiler status:

    • gcc: need gcc10.3 or gcc11 because of a bug in coroutines which has already been fixed upstream
    • clang: need clang-11 for non experimental support
    • msvc: need version 16.8 (released but not yet available in Github Actions)
    • apple-clang: probably in the next version ?
Asynchronous gRPC with Boost.Asio executors

asio-grpc This library provides an implementation of boost::asio::execution_context that dispatches work to a grpc::CompletionQueue. Making it possibl

Sep 18, 2022
HTTP and WebSocket built on Boost.Asio in C++11
HTTP and WebSocket built on Boost.Asio in C++11

HTTP and WebSocket built on Boost.Asio in C++11 Branch Linux/OSX Windows Coverage Documentation Matrix master develop Contents Introduction Appearance

Sep 20, 2022
Aug 29, 2022
A very simple, fast, multithreaded, platform independent HTTP and HTTPS server and client library implemented using C++11 and Boost.Asio.

A very simple, fast, multithreaded, platform independent HTTP and HTTPS server and client library implemented using C++11 and Boost.Asio. Created to be an easy way to make REST resources available from C++ applications.

Sep 17, 2022
C++ peer to peer library, built on the top of boost

Breep What is Breep? Breep is a c++ bridged peer to peer library. What does that mean? It means that even though the network is constructed as a peer

Aug 31, 2022
Boost::ASIO low-level redis client (connector)

bredis Boost::ASIO low-level redis client (connector), github gitee Features header only zero-copy (currently only for received replies from Redis) lo

Sep 15, 2022
An asynchronous web framework for C++ built on top of Qt

!!! I can no longer maintain this project. If you're interessed, please contact me and I can move the projetct to you !!! Tuf√£o - an asynchronous web

Sep 20, 2022
Boost.GIL - Generic Image Library | Requires C++11 since Boost 1.68
Boost.GIL - Generic Image Library | Requires C++11 since Boost 1.68

Documentation GitHub Actions AppVeyor Azure Pipelines CircleCI Regression Codecov Boost.GIL Introduction Documentation Requirements Branches Community

Sep 14, 2022
C++ framework for json-rpc (json remote procedure call)
C++ framework for json-rpc (json remote procedure call)

I am currently working on a new C++17 implementation -> json-rpc-cxx. Master Develop | libjson-rpc-cpp This framework provides cross platform JSON-RPC

Sep 18, 2022
Socket and Networking Library using msgpack.org[C++11]

netLink C++ 11 KISS principle networking library. Features: C++ 11 IPv4, IPv6 Protocols: TCP, UDP Enable/Disable blocking mode Join/Leave UDP-Multicas

Sep 9, 2022
RPC++ is a tool for Discord RPC (Rich Presence) to let your friends know about your Linux system
RPC++ is a tool for Discord RPC (Rich Presence) to let your friends know about your Linux system

RPC++ RPC++ is a tool for Discord RPC (Rich Presence) to let your friends know about your Linux system Installing requirements Arch based systems pacm

Jul 6, 2022
requests-like networking library using boost for C++

cq == C++ Requests cq == C++ Requests is a "Python Requests"-like C++ header-only library for sending HTTP requests. The library is inspired a lot by

Dec 15, 2021
Lightweight, header-only, Boost-based socket pool library

Stream-client This is a lightweight, header-only, Boost-based library providing client-side network primitives to easily organize and implement data t

Aug 5, 2022
Level up your Beat Saber experience on Quest! AnyTweaks provides various tweaks to help boost your experience on Quest, such as Bloom, FPS Counter and more.

Need help/support? Ask in one of BSMG's support channels for Quest, or join my Discord server! AnyTweaks Level up your Beat Saber experience on Quest!

Aug 29, 2022
Boost headers

About This repository contains a set of header files from Boost. Can be useful when using header only libraries. How to use You can easily include the

Oct 16, 2021
Boost.org signals2 module

Signals2, part of collection of the Boost C++ Libraries, is an implementation of a managed signals and slots system. License Distributed under the Boo

Sep 14, 2022
Boost.org property_tree module

Maintainer This library is currently maintained by Richard Hodges with generous support from the C++ Alliance. Build Status Branch Status develop mast

Sep 14, 2022
Super-project for modularized Boost

Boost C++ Libraries The Boost project provides free peer-reviewed portable C++ source libraries. We emphasize libraries that work well with the C++ St

Sep 15, 2022
Cross-platform, efficient, customizable, and robust asynchronous HTTP/WebSocket server C++14 library with the right balance between performance and ease of use

What Is RESTinio? RESTinio is a header-only C++14 library that gives you an embedded HTTP/Websocket server. It is based on standalone version of ASIO

Sep 19, 2022