CppSerdes is a serialization/deserialization library designed with embedded systems in mind

cppserdes_icon

CppSerdes is a serialization/deserialization library designed with embedded systems in mind

Build Status license version

Features

  • Bitpacking support - any bit alignment, any bit padding.
  • Portability - uses bit shifting and masking, with no dynamic memory allocation (avoids most std containers).
  • Serializes to/from 8-bit arrays, AND 16-bit, 32-bit, and 64-bit arrays. With no API changes to your code, to work with hardware drivers directly in their native bit widths.
  • Common Embedded Systems types - floating point, unsigned, signed, classes, enums, arrays, custom types, etc. While avoiding pulling in atypically embedded std types such as std::string, std::vector, etc.
  • Arrays - fixed sized arrays, dynamically sized arrays, and delimited arrays.
  • Memory safety - Bound checking for both serial buffers and array fields.
  • Choose from both high abstraction level (object-oriented & streams) and low level (memcpy-like) APIs.
  • Custom formatter types, such as optional validation checks, attached right to a field.
  • Virtual fields, and pure virtual fields, allowing you to easily change formats at runtime.
  • Header only, and works with C++11 and greater.
  • "constexpr" support for C++14 or greater using bitcpy function.
  • Compiles with high warning levels (pedantic, Wall, etc.).

Documentation

darrenlevine.github.io/cppserdes

Quick Start

  1. Check out the examples:
    • Compile and run "examples/01_simple_example.cpp".
    • Read through the examples/ folder for various usage cases.
  2. Use the library in your project:
    • Manual inclusion: Make sure "cppserdes/include/" is added to your project's path and use as desired.
    • With CMake: Add the following lines to your CmakeLists.txt project file:
add_subdirectory(cppserdes/)
target_link_libraries(your_project_name cppserdes)

Object Oriented Example (using serdes::packet_base)

#include "serdes.h"

struct my_packet : serdes::packet_base {
    int8_t x = 6, y = 7, z = 8;

    // define a format (used for BOTH serialization and deserialization)
    void format(serdes::packet &serdes_process) {
        serdes_process + x + y + serdes::pad(1) + serdes::bitpack(z, 7);
    }
};

int main() {
    uint16_t serial_data[] = {0x0102, 0x7B00};

    my_packet obj1;
    obj1.load(serial_data); // loads serial data into obj1 x == 1, y == 2, z == -5

    my_packet obj2;
    obj2.store(serial_data); // stores {0x0607, 0x0800} into serial_data from obj2
}

Functional Example (using serdes::bitcpy)

#include "bitcpy.h"

int main() {
    uint16_t serial_data[] = {0x0102, 0xFB00};
    int8_t x, y, z;

    // deserialization
    serdes::bitcpy(x, serial_data, 0, 8);
    serdes::bitcpy(y, serial_data, 8, 8);
    serdes::bitcpy(z, serial_data, 16, 8);

    // serialization
    serdes::bitcpy(serial_data, x, 0, 8);
    serdes::bitcpy(serial_data, y, 8, 8);
    serdes::bitcpy(serial_data, z, 16, 8);
}

Streams Example (using serdes::packet)

> x >> y >> z; // take same variables and place it into serial data serdes::packet(serial_data) << x << y << z; } ">
#include "serdes.h"

int main() {
    uint16_t serial_data[] = {0x0102, 0x0304, 0x0506};
    int8_t x, y, z;

    // take some serial data and place it into variables
    serdes::packet(serial_data) >> x >> y >> z;

    // take same variables and place it into serial data
    serdes::packet(serial_data) << x << y << z;
}

Known Limitations

  • Little endian "serialization" is not yet supported (Note: little and big endian "platforms" ARE both supported)
  • Cannot bitcpy more than 0.25GB of data in a single function call (due to a memory-usage/speed design decision)
  • The examples are written using C++14 syntax for succinctness and simplicity, so if you're using an older compiler, templated fields might need to have their templates explicitly specified, for example:
serdes::bitpack(123, serdes::bit_length(2)) // works in >= C++14
serdes::bitpack(123, serdes::bit_length(2)) // needed in < C++14

Planned Features

  • Little endian serialization support (compile time only is planned)
  • The ability to avoid serialization/deserialization by setting pointers to point to the appropriate spot within the serial buffer, instead of making a copy into a new buffer, thus saving time and memory if the serial buffer's lifetime persists longer than the reference, and the memory is aligned appropriately to allow this optimization. Similar to how capnproto and flatbuffers work.

Requirements

  • C++11 or greater
Similar Resources

Fast Binary Encoding is ultra fast and universal serialization solution for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift

Fast Binary Encoding (FBE) Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server request

Dec 1, 2022

Yet Another Serialization

Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

Nov 18, 2022

Binary Serialization

Binn Binn is a binary data serialization format designed to be compact, fast and easy to use. Performance The elements are stored with their sizes to

Nov 24, 2022

An implementation of the MessagePack serialization format in C / msgpack.org[C]

CMP CMP is a C implementation of the MessagePack serialization format. It currently implements version 5 of the MessagePack Spec. CMP's goal is to be

Nov 17, 2022

MPack - A C encoder/decoder for the MessagePack serialization format / msgpack.org[C]

Introduction MPack is a C implementation of an encoder and decoder for the MessagePack serialization format. It is: Simple and easy to use Secure agai

Nov 30, 2022

Serialization framework for Unreal Engine Property System that just works!

DataConfig Serialization framework for Unreal Engine Property System that just works! Unreal Engine features a powerful Property System which implemen

Nov 27, 2022

Yet Another Serialization

Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

Sep 7, 2021

universal serialization engine

A Universal Serialization Engine Based on compile-time Reflection iguana is a modern, universal and easy-to-use serialization engine developed in c++1

Dec 2, 2022

A lightweight C++20 serialization framework

zpp::bits A modern C++20 binary serialization library, with just one header file. This library is a successor to zpp::serializer. The library tries to

Nov 28, 2022
Comments
  • Clang compilation issues

    Clang compilation issues

    Hi. I trying to compile the library with clang, but unfortunately, I'm running into errors.

    1. Reference to 'atomic' is ambiguous at bitcpy_common:75 I was able to "fix" it by commenting out forward declared std::atomic at the beginning of the file.

    2. Variable of non-literal type 'const sized_pointer<const uint8_t>' (aka 'const sized_pointer<const unsigned char>') cannot be defined in a constexpr function Not sure what to do here 🤷

    Any ideas on how to properly compile it with Clang? FWIW, I'm using lasted AppleClang with XCode.

Your binary serialization library

Bitsery Header only C++ binary serialization library. It is designed around the networking requirements for real-time data delivery, especially for ga

Dec 1, 2022
Cap'n Proto serialization/RPC system - core tools and C++ library
Cap'n Proto serialization/RPC system - core tools and C++ library

Cap'n Proto is an insanely fast data interchange format and capability-based RPC system. Think JSON, except binary. Or think Protocol Buffers, except

Dec 1, 2022
A C++11 library for serialization
A C++11 library for serialization

cereal - A C++11 library for serialization cereal is a header-only C++11 serialization library. cereal takes arbitrary data types and reversibly turns

Dec 2, 2022
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Nov 27, 2022
Simple C++ 20 Serialization Library that works out of the box with aggregate types!

BinaryLove3 Simple C++ 20 Serialization Library that works out of the box with aggregate types! Requirements BinaryLove3 is a c++20 only library.

Sep 2, 2022
Header-only library for automatic (de)serialization of C++ types to/from JSON.

fuser 1-file header-only library for automatic (de)serialization of C++ types to/from JSON. how it works The library has a predefined set of (de)seria

Oct 20, 2022
C++17 library for all your binary de-/serialization needs

blobify blobify is a header-only C++17 library to handle binary de-/serialization in your project. Given a user-defined C++ struct, blobify can encode

Oct 20, 2022
Yet another JSON/YAML/BSON serialization library for C++.
Yet another JSON/YAML/BSON serialization library for C++.

ThorsSerializer Support for Json Yaml Bson NEW Benchmark Results Conformance mac linux Performance max linux For details see: JsonBenchmark Yet anothe

Oct 27, 2022
Cista is a simple, high-performance, zero-copy C++ serialization & reflection library.

Simple C++ Serialization & Reflection. Cista++ is a simple, open source (MIT license) C++17 compatible way of (de-)serializing C++ data structures. Si

Dec 1, 2022
Nov 28, 2022