A C++11 or library for parsing and serializing JSON to and from a DOM container in memory.

Boost.JSON

Branch master develop
Azure Build Status Build Status
Docs Documentation Documentation
Drone Build Status Build Status
Matrix Matrix Matrix
Fuzzing --- fuzz
Appveyor Build status Build status
codecov.io codecov codecov

Boost.JSON

Overview

Boost.JSON is a portable C++ library which provides containers and algorithms that implement JavaScript Object Notation, or simply "JSON", a lightweight data-interchange format. This format is easy for humans to read and write, and easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language (Standard ECMA-262). JSON is a text format that is language-independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

This library focuses on a common and popular use-case: parsing and serializing to and from a container called value which holds JSON types. Any value which you build can be serialized and then deserialized, guaranteeing that the result will be equal to the original value. Whatever JSON output you produce with this library will be readable by most common JSON implementations in any language.

The value container is designed to be well suited as a vocabulary type appropriate for use in public interfaces and libraries, allowing them to be composed. The library restricts the representable data types to the ranges which are almost universally accepted by most JSON implementations, especially JavaScript. The parser and serializer are both highly performant, meeting or exceeding the benchmark performance of the best comparable libraries. Allocators are very well supported. Code which uses these types will be easy to understand, flexible, and performant.

Boost.JSON offers these features:

  • Fast compilation
  • Require only C++11
  • Fast streaming parser and serializer
  • Constant-time key lookup for objects
  • Options to allow non-standard JSON
  • Easy and safe modern API with allocator support
  • Compile without Boost, define BOOST_JSON_STANDALONE
  • Optional header-only, without linking to a library

Requirements

The library relies heavily on these well known C++ types in its interfaces (henceforth termed standard types):

  • string_view
  • memory_resource, polymorphic_allocator
  • error_category, error_code, error_condition, system_error

The requirements for Boost.JSON depend on whether the library is used as part of Boost, or in the standalone flavor (without Boost):

Using Boost

  • Requires only C++11
  • The default configuration
  • Aliases for standard types use their Boost equivalents
  • Link to a built static or dynamic Boost library, or use header-only (see below)
  • Supports -fno-exceptions, detected automatically

Without Boost

  • Requires C++17
  • Aliases for standard types use their std equivalents
  • Obtained when defining the macro BOOST_JSON_STANDALONE
  • Link to a built static or dynamic standalone library, or use header-only (see below)
  • Supports -fno-exceptions: define BOOST_NO_EXCEPTIONS and boost::throw_exception manually

When using without Boost, support for <memory_resource> is required. In particular, if using libstdc++ then version 8.3 or later is needed.

Header-Only

To use as header-only; that is, to eliminate the requirement to link a program to a static or dynamic Boost.JSON library, simply place the following line in exactly one new or existing source file in your project.

#include <boost/json/src.hpp>

Standalone Shared Library

To build a standalone shared library, it is necessary to define the macros BOOST_JSON_DECL and BOOST_JSON_CLASS_DECL as appropriate for your toolchain. Example for MSVC:

// When building the DLL
#define BOOST_JSON_DECL       __declspec(dllexport)
#define BOOST_JSON_CLASS_DECL __declspec(dllexport)

// When building the application which uses the DLL
#define BOOST_JSON_DECL       __declspec(dllimport)
#define BOOST_JSON_CLASS_DECL __declspec(dllimport)

Embedded

Boost.JSON works great on embedded devices. The library uses local stack buffers to increase the performance of some operations. On Intel platforms these buffers are large (4KB), while on non-Intel platforms they are small (256 bytes). To adjust the size of the stack buffers for embedded applications define this macro when building the library or including the function definitions:

#define BOOST_JSON_STACK_BUFFER_SIZE 1024
#include <boost/json/src.hpp>

Note

This library uses separate inline namespacing for the standalone
mode to allow libraries which use different modes to compose
without causing link errors. Linking to both modes of Boost.JSON
(Boost and standalone) is possible, but not recommended.

Supported Compilers

Boost.JSON has been tested with the following compilers:

  • clang: 3.8, 4, 5, 6, 7, 8, 9, 10, 11
  • gcc: 4.8, 4.9, 5, 6, 7, 8, 9, 10
  • msvc: 14.0, 14.1, 14.2

Quality Assurance

The development infrastructure for the library includes these per-commit analyses:

  • Coverage reports
  • Benchmark performance comparisons
  • Compilation and tests on Drone.io, Azure Pipelines, Appveyor
  • Fuzzing using clang-llvm and machine learning

Visual Studio Solution

cmake -G "Visual Studio 16 2019" -A Win32 -B bin -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/msvc.cmake
cmake -G "Visual Studio 16 2019" -A x64 -B bin64 -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/msvc.cmake

License

Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at https://www.boost.org/LICENSE_1_0.txt)

Owner
Boost.org
Boost provides free peer-reviewed portable C++ source libraries.
Boost.org
Comments
  • json Conversions to/from C++ objects

    json Conversions to/from C++ objects

    Does Boost.json support conversions from C++ objects to json, and from json to C++ objects? Use case is in the interface between C++ application and calls to the ArangoDB HTTP REST API. I'm using libcurl to generate REST calls (with json bodies) into ArangoDB, and I need to convert C++ object members and their values to json for those calls. Then, on the return route, I need to populate C++ object members with the json data returned from ArangoDB calls. I also need to know if such conversions can be done recursively, or only singly.

    What of the case where my C++ objects exist in a vector or other std container? I'm looking for functionality akin to that noted here: https://stackoverflow.com/questions/17549906/c-json-serialization

    Scroll down to the solution with 85 votes, posted by Guillaume Racicot.

  • tuple-like key_value_pair (for structured bindings)

    tuple-like key_value_pair (for structured bindings)

    At the moment key_value_pair cannot be used for structured bindings without user boilerplate. Since it does model a pair anyway and offers an interface similar to std::pair, it would be nice for it to also specialize std::tuple_size, std::tuple_element and a form of get when using C++17 or above This would make the following code well-formed in C++17 or above:

    object o{{"one", 1}, {"two", 2}};
    const auto it = o.find("one");
    if (it == o.end())
      handle_error();
    const auto& [key, value] = *it;
    

    Note that replacing object with std::map<std::string, int> would produce well-formed C++17 code (see https://godbolt.org/z/fVZUkC)

    If this is something the authors would like too see too, I'd happily make a PR for it

  • Add support for json::string to std::string_view conversion

    Add support for json::string to std::string_view conversion

    When I'm using Boost.JSON along with Boost.Beast, it's common to parse a json frame from a remote http/websocket server and do some processing with it.

    A typical scenario is, I need to pass a json::string to a function where a std::string_view is expected.

    I can use standalone Boost.JSON with macro BOOST_JSON_STANDALONE to replace boost::string_view with std::string_view. But since I also need Boost.Beast, mixing Boost and standalone Boost.JSON would result in a lot of "macro redefinition" error.

    So now I have to tediously do the conversion like this everytime

    void process_sv(std::string_view sv);
    
    auto j_str = j_v.as_string();
    auto j_sv = std::string_view(j_str.data(), j_str.size());
    
    process_sv(j_sv);
    

    It would make it much more convenient if a direct conversion from json::string to std::string_view is available.

    Thank you!

  • Support for numbers which can't be represented by int64_t/uint64_t/double

    Support for numbers which can't be represented by int64_t/uint64_t/double

    I want to interface with a finance service that unfortunately encodes monetary quantities as JSON numbers. To process those correctly, parsing and storage of numbers needs to be customized. Specifically, I want to parse them into boost::multiprecision::cpp_dec_float.

    (I realize that this particular case can be worked around by converting the parsed doubles back to shortest string representation and parsing decimal floats from that. However, my point stands - it's useful to be able to handle numbers that can't be represented by int64_t/uint64_t/double, like big integers or arbitrary precision floats.)

    Well, I thought, I just need a template-based JSON library which would let me supply my custom numeric type and a parser for it. Something like nlohmann::json, perhaps. Except nlohmann::json doesn't let you supply arbitrary numeric types, only built-in ones. In fact, after a few hours spent surveying numerous C++ JSON libs on github, to my great surprise I couldn't find one that would let me do that.

    Of course, I realize boost::json is explicitly designed with different goals in mind. And I'm not advocating converting it to templates or anything like that. Fortunately, that's unnecessary. Consider:

    • To parse_options add bool numbers_as_strings = false; member (perhaps partitioned into ints_as_strings/floats_as_strings).
    • Find a single spare bit somewhere within string representation. Use it to indicate that this string was parsed from JSON number/must be serialized as JSON number. Let's call it raw_number flag. Initialize it to false. Add a getter and a setter for it, both public.
    • When parsing, if numbers_as_strings is set upon encountering a number, create a string consisting of exactly the characters comprising the number in input. Set the raw_number flag on it. Otherwise, parse int64_t/uint64_t/double as usual.
    • When serializing a string, if raw_number flag is set on it, assume the content is a valid JSON number and dump it into the output without quoting/escaping. Otherwise, serialize the string as usual.
    • Optionally, provide functions to convert string to/from int64_t/uint64_t/double using the algorithms currently employed by the parser/serializer.

    This moves number parsing/serialization to user code when necessary, in a way that doesn't complicate usage for people who don't care about this—admittedly quite rare—case. I believe it also fits the design principles of this library as I understand them. You guys have done a great job making common things easy, here I aim to make uncommon things possible.

    Do you consider this a reasonable design? Something you're willing to merge if presented with a PR? (not that I have one at the moment)

  • Update bintray credentials (was: add fuzzing)

    Update bintray credentials (was: add fuzzing)

    This is a ticket to keep track of adding fuzz testing. I promised to help out, so here I am! I wanted to highlight the plan so everyone is happy with the direction.

    The plan is to

    • add a fuzz target (and build support, if necessary)
    • helper scripts to get an initial seed corpus
    • add a github action which runs the fuzzing for a short time (30 seconds or so) to make sure easy to find bugs are detected already at the pull request stage

    Building

    I had problem building the library - I suspect others wanting to try it out also may run into problems. Is there documentation somewhere that I missed? I expected the usual git clone, submodule recursive update, cmake to work out of the box but there seems to be a dependency on boost beast.

    Fuzz target

    There is already a fuzzer in #13 . @msimonsson are you ok that I incorporate your fuzzer code from that ticket under the boost license? I assume this is OK, but I am not a lawyer (well perhaps a C++ language lawyer wannabe :-).

    Github Action

    I have used this for the simdjson project recently, worked fine. I am not sure if it is possible to browse through it unless being a member, but here are some links:

    • the config: https://github.com/lemire/simdjson/blob/master/.github/workflows/fuzzers.yml
    • the jobs: https://github.com/lemire/simdjson/actions
    • partial motivation: https://github.com/lemire/simdjson/issues/370

    For efficiency, it is good if the corpus can be stored somewhere between the runs. Otherwise it has to bootstrap each time which is inefficient. I use bintray for simdjson - @vinniefalco where would you be ok to store the corpus, do you perhaps already have a bintray account? In the meanwhile, I will use my own.

    Implementation

    I develop this in my clone for now. See the Readme over here: https://github.com/pauldreik/json/tree/paul/fuzz/fuzzing

  • Consider using the same atomic ops as boost::shared_ptr` does.

    Consider using the same atomic ops as boost::shared_ptr` does.

    https://github.com/boostorg/json/blob/5098730a87148228f65b2dcb40ecec7f15019c09/include/boost/json/storage_ptr.hpp#L108

    boost::json::storage_ptr uses sequentially consistent atomic ops for the reference count. This is needlessly expensive on most non-x86 platforms, in particular arm and the like. boost::shared_ptr apparently implements more efficient reference counting (usually relaxed inc + acquire release dec). Considering that there's one storage_ptr per value, that's potentially quite a large number of incs and decs for a large json document. I would suggest that storage_ptr reference count operations match those of boost::shared_ptr.

  • Cmake overhaul

    Cmake overhaul

    Some refactoring of root and test CMakeLists.txt

    • Add json tests to tests target (as has been requested by @pdimov in #581)
    • Refactor json usage requirements into a separate CMake target (fixes #562)
    • Overhaul of dealing with CMake use cases (fixes #582)
    • Fix test standalone_with_boost for CMake (fixes #577)
    • Other minor changes
  • Make boost::json::string implicitly convertible to std::string

    Make boost::json::string implicitly convertible to std::string

    Version of Boost

    1.75

    This is more of a would-be-nice type of issue.

    Because the boost::json::string is not implicitly convertible to std::string you have to always do:

    std::string s = {jv.as_string().data(), jv.as_string().size()}; //ok but unyieldy
    //or
    std::string s = value_to<std::string>(jv); //via tag dispatch. ok but not user friendly
    //ideally
    std::string s = jv.as_string();
    

    Alternatively remove the boost::json::string type altogether, unless it has some special built-in features which I'm not aware of.

    Same goes for boost::string_view.

    I understand that under JSON_STANDALONE these types are aliased to STL types, but from a user perspective is highly undesirable. If I want to switch tomorrow from standalone to boost+json or vice-versa, my code will no longer compile. Ideally consider not using any other boost types unless you have < C++17, in which case you simply use std::string instead of string_view on your public interfaces.

  • Add std::hash specializations for json types

    Add std::hash specializations for json types

    As discussed for the issue #521, this pull request

    • Add std::hash specializations for json::array, json::object, and json::value.
    • Add a new header boost/json/detail/hash_combine.hpp for inline hash functions.
    • Add basic unit tests
  • Heap-buffer-overflow when fuzzing with libFuzzer

    Heap-buffer-overflow when fuzzing with libFuzzer

    Fuzzing the validate() function in example/validate.cpp results in heap-buffer-overflow:

    ==75832==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000005901 at pc 0x00000031fa46 bp 0x7fffffff8f50 sp 0x7fffffff8f48
    READ of size 1 at 0x603000005901 thread T0
        #0 0x31fa45 in boost::json::basic_parser::write_some(char const*, unsigned long, std::__1::error_code&)
            boost/json/impl/basic_parser.ipp:456:16
        #1 0x30f73c in boost::json::basic_parser::write(char const*, unsigned long, std::__1::error_code&)
            boost/json/impl/basic_parser.ipp:894:9
        #2 0x309cf2 in boost::json::basic_parser::finish(char const*, unsigned long, std::__1::error_code&)
            boost/json/impl/basic_parser.ipp:924:5
        #3 0x30937e in (anonymous namespace)::validate(std::__1::basic_string_view<char, std::__1::char_traits<char> >)
            bjson.cc:73:11
    

    Input:

    \"~QQ36644632   {n
    Base64: In5RUTM2NjQ0NjMyICAge24=
    

    Please let me know if you need more details.

  • JSON pointer implementation

    JSON pointer implementation

    • Class pointer in boost/json/pointer.hpp; stores a string_view that should contain a JSON Pointer string.
    • Member function value& value::at(pointer) that either returns a reference to corresponding child or throws system_error.
    • Member function value* value::find(pointer, error_code&) that either returns a pointer to corresponding child or sets error_code to some error.
    • const versions of those functions.
    • Extend error and error_condition enums with values needed for Pointer.
  • value_ref documentation

    value_ref documentation

    value_ref needs more robust explanation in its javadocs. It should state that it should only be used as a function parameter type and never as an automatic or member variable.

  • weid grammar in value javadoc

    weid grammar in value javadoc

        This is a
        <a href="https://en.cppreference.com/w/cpp/concepts/regular"><em>Regular</em></a>.
        <em>Regular</em>
        type which works like
    

    image

  • set_at_pointer

    set_at_pointer

    Fix #700

    This implementation makes these choices:

    1. Nulls are replaced when creating intermediaries.
    2. When creating intermediaries segments are always treated as object keys, (that is only objects are created as intermediaries).
    3. Arrays are extended by 1 element at most.
  • Explore potential for optimization in array::insert(Iter, InputIt, InputIt)

    Explore potential for optimization in array::insert(Iter, InputIt, InputIt)

    The idea is this:

    1. Fill the buffer up to capacity at the end (needs a revert_insert protection).
    2. Create a temporary buffer with the remaining elements (can throw).
    3. If temporary buffer is not empty
      1. Allocate new buffer with necessary size (can throw).
      2. Relocate elements
        • first original elements, up to insertion point,
        • then new elements (at the end of the original buffer),
        • then new elements from the temporary buffer,
        • then remaining old elements.
    4. If the temporary buffer is empty, rotate elements in the original buffer instead.

    In the end result:

    1. If the original capacity could accommodate the input range, then no new buffer (even a temporary one) is allocated. This is particularly useful when the caller uses input iterators, but does know the input range size, and thus can call reserve.
    2. We should be able to keep the strong guarantee.
Parsing gigabytes of JSON per second
Parsing gigabytes of JSON per second

simdjson : Parsing gigabytes of JSON per second JSON is everywhere on the Internet. Servers spend a *lot* of time parsing it. We need a fresh approach

Jun 14, 2022
A header only C++11 library for parsing TOML

tinytoml A header only C++11 library for parsing TOML. This parser is based on TOML v0.4.0. This library is distributed under simplified BSD License.

Jun 8, 2022
cpptoml is a header-only library for parsing TOML

cpptoml A header-only library for parsing TOML configuration files. Targets: TOML v0.5.0 as of August 2018. This includes support for the new DateTime

Jun 18, 2022
Use to copy a file from an NTFS partitioned volume by reading the raw volume and parsing the NTFS structures.

ntfsDump Use to copy a file from an NTFS partitioned volume by reading the raw volume and parsing the NTFS structures. Similar to https://github.com/P

Apr 14, 2022
A C++ library for interacting with JSON.

JsonCpp JSON is a lightweight data-interchange format. It can represent numbers, strings, ordered sequences of values, and collections of name/value p

Jun 21, 2022
Header-only library for automatic (de)serialization of C++ types to/from JSON.

fuser 1-file header-only library for automatic (de)serialization of C++ types to/from JSON. how it works The library has a predefined set of (de)seria

Jun 4, 2022
Yet another JSON/YAML/BSON serialization library for C++.
Yet another JSON/YAML/BSON serialization library for C++.

ThorsSerializer Support for Json Yaml Bson NEW Benchmark Results Conformance mac linux Performance max linux For details see: JsonBenchmark Yet anothe

Jun 15, 2022
JSON for Modern C++
JSON for Modern C++

Design goals Sponsors Integration CMake Package Managers Pkg-config Examples JSON as first-class data type Serialization / Deserialization STL-like ac

Jun 16, 2022
Similar to C++ streams, but the stream elements are structured JSON data rather than characters.

JIOS : JSON Input Output Streams Similar to C++ streams, but the stream elements are structured JSON data rather than characters. Contents Features [P

Aug 16, 2019
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Jun 20, 2022
libcluon is a small and efficient, single-file and header-only library written in modern C++ to power microservices.

libcluon Linux & OSX Build (TravisCI) Win64 Build (AppVeyor) Test Coverage Coverity Analysis CII Best Practices libcluon is a small single-file, heade

Apr 23, 2022
Cap'n Proto serialization/RPC system - core tools and C++ library
Cap'n Proto serialization/RPC system - core tools and C++ library

Cap'n Proto is an insanely fast data interchange format and capability-based RPC system. Think JSON, except binary. Or think Protocol Buffers, except

Jun 23, 2022
Header-only C++11 library to encode/decode base64, base64url, base32, base32hex and hex (a.k.a. base16) as specified in RFC 4648, plus Crockford's base32. MIT licensed with consistent, flexible API.

cppcodec Header-only C++11 library to encode/decode base64, base64url, base32, base32hex and hex (a.k.a. base16) as specified in RFC 4648, plus Crockf

Jun 8, 2022
FlatBuffers Compiler and Library in C for C

OS-X & Ubuntu: Windows: The JSON parser may change the interface for parsing union vectors in a future release which requires code generation to match

Jun 14, 2022
Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications

Zmeya Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications. Zmeya is not even a serializ

Jun 6, 2022
A C++11 ASN.1 BER Encoding and Decoding Library

fast_ber A performant ASN.1 BER encoding and decoding library written in C++11 Introduction fast_ber is a small, lightweight library for BER encoding

May 22, 2022
A high performance C++14 library for effortlessly reading and writing UBJSON

UbjsonCpp A high performance C++14 library for effortlessly reading and writing UBJSON This library implements UBJSON Draft 12 and Value semmantics Ab

Nov 10, 2021
Nov 29, 2021
Your binary serialization library

Bitsery Header only C++ binary serialization library. It is designed around the networking requirements for real-time data delivery, especially for ga

Jun 15, 2022