A C++11 or library for parsing and serializing JSON to and from a DOM container in memory.

Boost.JSON

Branch master develop
Azure Build Status Build Status
Docs Documentation Documentation
Drone Build Status Build Status
Matrix Matrix Matrix
Fuzzing --- fuzz
Appveyor Build status Build status
codecov.io codecov codecov

Boost.JSON

Overview

Boost.JSON is a portable C++ library which provides containers and algorithms that implement JavaScript Object Notation, or simply "JSON", a lightweight data-interchange format. This format is easy for humans to read and write, and easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language (Standard ECMA-262). JSON is a text format that is language-independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

This library focuses on a common and popular use-case: parsing and serializing to and from a container called value which holds JSON types. Any value which you build can be serialized and then deserialized, guaranteeing that the result will be equal to the original value. Whatever JSON output you produce with this library will be readable by most common JSON implementations in any language.

The value container is designed to be well suited as a vocabulary type appropriate for use in public interfaces and libraries, allowing them to be composed. The library restricts the representable data types to the ranges which are almost universally accepted by most JSON implementations, especially JavaScript. The parser and serializer are both highly performant, meeting or exceeding the benchmark performance of the best comparable libraries. Allocators are very well supported. Code which uses these types will be easy to understand, flexible, and performant.

Boost.JSON offers these features:

  • Fast compilation
  • Require only C++11
  • Fast streaming parser and serializer
  • Constant-time key lookup for objects
  • Options to allow non-standard JSON
  • Easy and safe modern API with allocator support
  • Compile without Boost, define BOOST_JSON_STANDALONE
  • Optional header-only, without linking to a library

Requirements

The library relies heavily on these well known C++ types in its interfaces (henceforth termed standard types):

  • string_view
  • memory_resource, polymorphic_allocator
  • error_category, error_code, error_condition, system_error

The requirements for Boost.JSON depend on whether the library is used as part of Boost, or in the standalone flavor (without Boost):

Using Boost

  • Requires only C++11
  • The default configuration
  • Aliases for standard types use their Boost equivalents
  • Link to a built static or dynamic Boost library, or use header-only (see below)
  • Supports -fno-exceptions, detected automatically

Without Boost

  • Requires C++17
  • Aliases for standard types use their std equivalents
  • Obtained when defining the macro BOOST_JSON_STANDALONE
  • Link to a built static or dynamic standalone library, or use header-only (see below)
  • Supports -fno-exceptions: define BOOST_NO_EXCEPTIONS and boost::throw_exception manually

When using without Boost, support for <memory_resource> is required. In particular, if using libstdc++ then version 8.3 or later is needed.

Header-Only

To use as header-only; that is, to eliminate the requirement to link a program to a static or dynamic Boost.JSON library, simply place the following line in exactly one new or existing source file in your project.

#include <boost/json/src.hpp>

Standalone Shared Library

To build a standalone shared library, it is necessary to define the macros BOOST_JSON_DECL and BOOST_JSON_CLASS_DECL as appropriate for your toolchain. Example for MSVC:

// When building the DLL
#define BOOST_JSON_DECL       __declspec(dllexport)
#define BOOST_JSON_CLASS_DECL __declspec(dllexport)

// When building the application which uses the DLL
#define BOOST_JSON_DECL       __declspec(dllimport)
#define BOOST_JSON_CLASS_DECL __declspec(dllimport)

Embedded

Boost.JSON works great on embedded devices. The library uses local stack buffers to increase the performance of some operations. On Intel platforms these buffers are large (4KB), while on non-Intel platforms they are small (256 bytes). To adjust the size of the stack buffers for embedded applications define this macro when building the library or including the function definitions:

#define BOOST_JSON_STACK_BUFFER_SIZE 1024
#include <boost/json/src.hpp>

Note

This library uses separate inline namespacing for the standalone
mode to allow libraries which use different modes to compose
without causing link errors. Linking to both modes of Boost.JSON
(Boost and standalone) is possible, but not recommended.

Supported Compilers

Boost.JSON has been tested with the following compilers:

  • clang: 3.8, 4, 5, 6, 7, 8, 9, 10, 11
  • gcc: 4.8, 4.9, 5, 6, 7, 8, 9, 10
  • msvc: 14.0, 14.1, 14.2

Quality Assurance

The development infrastructure for the library includes these per-commit analyses:

  • Coverage reports
  • Benchmark performance comparisons
  • Compilation and tests on Drone.io, Azure Pipelines, Appveyor
  • Fuzzing using clang-llvm and machine learning

Visual Studio Solution

cmake -G "Visual Studio 16 2019" -A Win32 -B bin -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/msvc.cmake
cmake -G "Visual Studio 16 2019" -A x64 -B bin64 -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/msvc.cmake

License

Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at https://www.boost.org/LICENSE_1_0.txt)

Owner
Boost.org
Boost provides free peer-reviewed portable C++ source libraries.
Boost.org
Comments
  • json Conversions to/from C++ objects

    json Conversions to/from C++ objects

    Does Boost.json support conversions from C++ objects to json, and from json to C++ objects? Use case is in the interface between C++ application and calls to the ArangoDB HTTP REST API. I'm using libcurl to generate REST calls (with json bodies) into ArangoDB, and I need to convert C++ object members and their values to json for those calls. Then, on the return route, I need to populate C++ object members with the json data returned from ArangoDB calls. I also need to know if such conversions can be done recursively, or only singly.

    What of the case where my C++ objects exist in a vector or other std container? I'm looking for functionality akin to that noted here: https://stackoverflow.com/questions/17549906/c-json-serialization

    Scroll down to the solution with 85 votes, posted by Guillaume Racicot.

  • Refactoring of conversion functions

    Refactoring of conversion functions

    The list of changes:

    • take advantage of mp11
    • split container_traits into several traits and helpers
    • split map_traits into several traits
    • use value_type more consistently
    • refactor detail::inserter
    • use detail::inserter with map-like types
    • refactor conversion of tuple-like to array
    • remove conversion from value_ref and don't use value_to/from when constructing value from value_ref
    • check for round-trip
    • user-facing conversion traits
    • opt-in null-like conversion
    • same requirements for value_to and value_from
    • stricter is_map_like
    • stricter is_sequence_like
  • tuple-like key_value_pair (for structured bindings)

    tuple-like key_value_pair (for structured bindings)

    At the moment key_value_pair cannot be used for structured bindings without user boilerplate. Since it does model a pair anyway and offers an interface similar to std::pair, it would be nice for it to also specialize std::tuple_size, std::tuple_element and a form of get when using C++17 or above This would make the following code well-formed in C++17 or above:

    object o{{"one", 1}, {"two", 2}};
    const auto it = o.find("one");
    if (it == o.end())
      handle_error();
    const auto& [key, value] = *it;
    

    Note that replacing object with std::map<std::string, int> would produce well-formed C++17 code (see https://godbolt.org/z/fVZUkC)

    If this is something the authors would like too see too, I'd happily make a PR for it

  • Add support for json::string to std::string_view conversion

    Add support for json::string to std::string_view conversion

    When I'm using Boost.JSON along with Boost.Beast, it's common to parse a json frame from a remote http/websocket server and do some processing with it.

    A typical scenario is, I need to pass a json::string to a function where a std::string_view is expected.

    I can use standalone Boost.JSON with macro BOOST_JSON_STANDALONE to replace boost::string_view with std::string_view. But since I also need Boost.Beast, mixing Boost and standalone Boost.JSON would result in a lot of "macro redefinition" error.

    So now I have to tediously do the conversion like this everytime

    void process_sv(std::string_view sv);
    
    auto j_str = j_v.as_string();
    auto j_sv = std::string_view(j_str.data(), j_str.size());
    
    process_sv(j_sv);
    

    It would make it much more convenient if a direct conversion from json::string to std::string_view is available.

    Thank you!

  • Support for numbers which can't be represented by int64_t/uint64_t/double

    Support for numbers which can't be represented by int64_t/uint64_t/double

    I want to interface with a finance service that unfortunately encodes monetary quantities as JSON numbers. To process those correctly, parsing and storage of numbers needs to be customized. Specifically, I want to parse them into boost::multiprecision::cpp_dec_float.

    (I realize that this particular case can be worked around by converting the parsed doubles back to shortest string representation and parsing decimal floats from that. However, my point stands - it's useful to be able to handle numbers that can't be represented by int64_t/uint64_t/double, like big integers or arbitrary precision floats.)

    Well, I thought, I just need a template-based JSON library which would let me supply my custom numeric type and a parser for it. Something like nlohmann::json, perhaps. Except nlohmann::json doesn't let you supply arbitrary numeric types, only built-in ones. In fact, after a few hours spent surveying numerous C++ JSON libs on github, to my great surprise I couldn't find one that would let me do that.

    Of course, I realize boost::json is explicitly designed with different goals in mind. And I'm not advocating converting it to templates or anything like that. Fortunately, that's unnecessary. Consider:

    • To parse_options add bool numbers_as_strings = false; member (perhaps partitioned into ints_as_strings/floats_as_strings).
    • Find a single spare bit somewhere within string representation. Use it to indicate that this string was parsed from JSON number/must be serialized as JSON number. Let's call it raw_number flag. Initialize it to false. Add a getter and a setter for it, both public.
    • When parsing, if numbers_as_strings is set upon encountering a number, create a string consisting of exactly the characters comprising the number in input. Set the raw_number flag on it. Otherwise, parse int64_t/uint64_t/double as usual.
    • When serializing a string, if raw_number flag is set on it, assume the content is a valid JSON number and dump it into the output without quoting/escaping. Otherwise, serialize the string as usual.
    • Optionally, provide functions to convert string to/from int64_t/uint64_t/double using the algorithms currently employed by the parser/serializer.

    This moves number parsing/serialization to user code when necessary, in a way that doesn't complicate usage for people who don't care about this—admittedly quite rare—case. I believe it also fits the design principles of this library as I understand them. You guys have done a great job making common things easy, here I aim to make uncommon things possible.

    Do you consider this a reasonable design? Something you're willing to merge if presented with a PR? (not that I have one at the moment)

  • Update bintray credentials (was: add fuzzing)

    Update bintray credentials (was: add fuzzing)

    This is a ticket to keep track of adding fuzz testing. I promised to help out, so here I am! I wanted to highlight the plan so everyone is happy with the direction.

    The plan is to

    • add a fuzz target (and build support, if necessary)
    • helper scripts to get an initial seed corpus
    • add a github action which runs the fuzzing for a short time (30 seconds or so) to make sure easy to find bugs are detected already at the pull request stage

    Building

    I had problem building the library - I suspect others wanting to try it out also may run into problems. Is there documentation somewhere that I missed? I expected the usual git clone, submodule recursive update, cmake to work out of the box but there seems to be a dependency on boost beast.

    Fuzz target

    There is already a fuzzer in #13 . @msimonsson are you ok that I incorporate your fuzzer code from that ticket under the boost license? I assume this is OK, but I am not a lawyer (well perhaps a C++ language lawyer wannabe :-).

    Github Action

    I have used this for the simdjson project recently, worked fine. I am not sure if it is possible to browse through it unless being a member, but here are some links:

    • the config: https://github.com/lemire/simdjson/blob/master/.github/workflows/fuzzers.yml
    • the jobs: https://github.com/lemire/simdjson/actions
    • partial motivation: https://github.com/lemire/simdjson/issues/370

    For efficiency, it is good if the corpus can be stored somewhere between the runs. Otherwise it has to bootstrap each time which is inefficient. I use bintray for simdjson - @vinniefalco where would you be ok to store the corpus, do you perhaps already have a bintray account? In the meanwhile, I will use my own.

    Implementation

    I develop this in my clone for now. See the Readme over here: https://github.com/pauldreik/json/tree/paul/fuzz/fuzzing

  • Consider using the same atomic ops as boost::shared_ptr` does.

    Consider using the same atomic ops as boost::shared_ptr` does.

    https://github.com/boostorg/json/blob/5098730a87148228f65b2dcb40ecec7f15019c09/include/boost/json/storage_ptr.hpp#L108

    boost::json::storage_ptr uses sequentially consistent atomic ops for the reference count. This is needlessly expensive on most non-x86 platforms, in particular arm and the like. boost::shared_ptr apparently implements more efficient reference counting (usually relaxed inc + acquire release dec). Considering that there's one storage_ptr per value, that's potentially quite a large number of incs and decs for a large json document. I would suggest that storage_ptr reference count operations match those of boost::shared_ptr.

  • Cmake overhaul

    Cmake overhaul

    Some refactoring of root and test CMakeLists.txt

    • Add json tests to tests target (as has been requested by @pdimov in #581)
    • Refactor json usage requirements into a separate CMake target (fixes #562)
    • Overhaul of dealing with CMake use cases (fixes #582)
    • Fix test standalone_with_boost for CMake (fixes #577)
    • Other minor changes
  • Documentation mentions the possibility of linking boost in as a static or dynamic library, but offers practically zero details on how to do so.

    Documentation mentions the possibility of linking boost in as a static or dynamic library, but offers practically zero details on how to do so.

    Problem

    The overview mentions the possibility of linking boost::json in as a static or dynamic library, but offers practically zero details on how to do so. I've heard that you just chuck -lboost_system at your linker when you're building the executable, but this rumor is not confirmed by any of the documentation, and didn't seem to work for me (see error below). Do I have to build the library? Should I have installed boost with my package manager instead of getting the latest version directly from the boost website? Perhaps detailed instructions on how to build and link boost::json are outside the scope of the overview, but it would be helpful if there were a link to something that explained how to do it.

    Steps necessary to reproduce the problem

    Visit the boost::json docs overview.

    All relevant compiler information

    Platform/Architecture: 64-bit Linux Mint 21 (Cinnamon) Compiler: g++ 11.3 Error output: /usr/bin/ld: cannot find -lboost_system: No such file or directory

  • Make boost::json::string implicitly convertible to std::string

    Make boost::json::string implicitly convertible to std::string

    Version of Boost

    1.75

    This is more of a would-be-nice type of issue.

    Because the boost::json::string is not implicitly convertible to std::string you have to always do:

    std::string s = {jv.as_string().data(), jv.as_string().size()}; //ok but unyieldy
    //or
    std::string s = value_to<std::string>(jv); //via tag dispatch. ok but not user friendly
    //ideally
    std::string s = jv.as_string();
    

    Alternatively remove the boost::json::string type altogether, unless it has some special built-in features which I'm not aware of.

    Same goes for boost::string_view.

    I understand that under JSON_STANDALONE these types are aliased to STL types, but from a user perspective is highly undesirable. If I want to switch tomorrow from standalone to boost+json or vice-versa, my code will no longer compile. Ideally consider not using any other boost types unless you have < C++17, in which case you simply use std::string instead of string_view on your public interfaces.

  • Add std::hash specializations for json types

    Add std::hash specializations for json types

    As discussed for the issue #521, this pull request

    • Add std::hash specializations for json::array, json::object, and json::value.
    • Add a new header boost/json/detail/hash_combine.hpp for inline hash functions.
    • Add basic unit tests
  • basic_parser_impl: make sentinel() return a unique pointer

    basic_parser_impl: make sentinel() return a unique pointer

    Right now, sentinel() casts the basic_parser_impl pointer (this) to const char *, but that pointer is not unique if the input buffer happens to be placed right before the basic_parser_impl instance - the end of that buffer then has the same address as basic_parser_impl.

    Example code:

      struct {
        char buffer[8]{"{\"12345\""};
        boost::json::stream_parser p;
      } s;
      s.p.write(s.buffer, sizeof(s.buffer));
      s.p.write(":0}", 3);
    

    This stops parsing at the end of the buffer, and then the incomplete() check in parse_string() will return true; the second write() call will crash with assertion failure:

    boost/json/basic_parser_impl.hpp:1016: const char* boost::json::basic_parser::parse_unescaped(const char*, std::integral_constant<bool, StackEmpty_>, std::integral_constant<bool, AllowComments_>, bool) [with bool StackEmpty_ = true; bool IsKey_ = true; Handler = boost::json::detail::handler]: Assertion `*cs == '\x22'' failed.

    This changes sentinel() to return the address of a static variable instead, which is guaranteed to be unique.

  • Investigate generating into string from user structures directly

    Investigate generating into string from user structures directly

    There is only issue about parsing here: https://github.com/boostorg/json/issues/627 But the generating is also actual question and it is also could potentially gain a lot of performance.

    E.g.

    template<class V, class OI>
    void
    generate_into(
        OI sink,
        const V& v);
    
    std::vector<int> v{1, 2, 3};
    std::string s;
    boost::json::generate_into(std::back_inserter(s), v);
    std::cout << s << std::endl; // [1, 2, 3]
    
  • Wrong value in max_number_chars?

    Wrong value in max_number_chars?

    Shouldn't 19 be 20 here? https://github.com/boostorg/json/blob/develop/include/boost/json/detail/format.hpp#L18

    because 2^64-1 = 18,446,744,073,709,551,615 which is 20 digits not 19

Parsing gigabytes of JSON per second
Parsing gigabytes of JSON per second

simdjson : Parsing gigabytes of JSON per second JSON is everywhere on the Internet. Servers spend a *lot* of time parsing it. We need a fresh approach

Nov 30, 2022
A header only C++11 library for parsing TOML

tinytoml A header only C++11 library for parsing TOML. This parser is based on TOML v0.4.0. This library is distributed under simplified BSD License.

Oct 10, 2022
cpptoml is a header-only library for parsing TOML

cpptoml A header-only library for parsing TOML configuration files. Targets: TOML v0.5.0 as of August 2018. This includes support for the new DateTime

Dec 1, 2022
Use to copy a file from an NTFS partitioned volume by reading the raw volume and parsing the NTFS structures.

ntfsDump Use to copy a file from an NTFS partitioned volume by reading the raw volume and parsing the NTFS structures. Similar to https://github.com/P

Nov 28, 2022
A C++ library for interacting with JSON.

JsonCpp JSON is a lightweight data-interchange format. It can represent numbers, strings, ordered sequences of values, and collections of name/value p

Dec 4, 2022
Header-only library for automatic (de)serialization of C++ types to/from JSON.

fuser 1-file header-only library for automatic (de)serialization of C++ types to/from JSON. how it works The library has a predefined set of (de)seria

Oct 20, 2022
Yet another JSON/YAML/BSON serialization library for C++.
Yet another JSON/YAML/BSON serialization library for C++.

ThorsSerializer Support for Json Yaml Bson NEW Benchmark Results Conformance mac linux Performance max linux For details see: JsonBenchmark Yet anothe

Oct 27, 2022
JSON for Modern C++
JSON for Modern C++

Design goals Sponsors Integration CMake Package Managers Pkg-config Examples JSON as first-class data type Serialization / Deserialization STL-like ac

Nov 29, 2022
Similar to C++ streams, but the stream elements are structured JSON data rather than characters.

JIOS : JSON Input Output Streams Similar to C++ streams, but the stream elements are structured JSON data rather than characters. Contents Features [P

Aug 16, 2019
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Dec 8, 2022
libcluon is a small and efficient, single-file and header-only library written in modern C++ to power microservices.

libcluon Linux & OSX Build (TravisCI) Win64 Build (AppVeyor) Test Coverage Coverity Analysis CII Best Practices libcluon is a small single-file, heade

Nov 19, 2022
Cap'n Proto serialization/RPC system - core tools and C++ library
Cap'n Proto serialization/RPC system - core tools and C++ library

Cap'n Proto is an insanely fast data interchange format and capability-based RPC system. Think JSON, except binary. Or think Protocol Buffers, except

Dec 1, 2022
Header-only C++11 library to encode/decode base64, base64url, base32, base32hex and hex (a.k.a. base16) as specified in RFC 4648, plus Crockford's base32. MIT licensed with consistent, flexible API.

cppcodec Header-only C++11 library to encode/decode base64, base64url, base32, base32hex and hex (a.k.a. base16) as specified in RFC 4648, plus Crockf

Dec 5, 2022
FlatBuffers Compiler and Library in C for C

OS-X & Ubuntu: Windows: The JSON parser may change the interface for parsing union vectors in a future release which requires code generation to match

Dec 4, 2022
Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications

Zmeya Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications. Zmeya is not even a serializ

Nov 2, 2022
A C++11 ASN.1 BER Encoding and Decoding Library

fast_ber A performant ASN.1 BER encoding and decoding library written in C++11 Introduction fast_ber is a small, lightweight library for BER encoding

Dec 1, 2022
A high performance C++14 library for effortlessly reading and writing UBJSON

UbjsonCpp A high performance C++14 library for effortlessly reading and writing UBJSON This library implements UBJSON Draft 12 and Value semmantics Ab

Aug 2, 2022
Nov 29, 2021
Your binary serialization library

Bitsery Header only C++ binary serialization library. It is designed around the networking requirements for real-time data delivery, especially for ga

Dec 1, 2022