A generator of JSON parser & serializer C++ code from structure header files

JSON-CPP-gen

This is a program that parses C++ structures from a header file and automatically generates C++ code capable of serializing said structures into the JSON format and parsing it back.

For example, if provided with the following structure as input,

struct User {
    int id;
    std::string name;
    std::vector<int> friendIds;
};

JSON-CPP-gen can generate a UserParser class that can be used as

User user;
auto error = UserParser::parse(user, jsonString);

to automatically parse a JSON string that matches the stucture, e.g.

{
    "id": 137,
    "name": "John Smith",
    "friendIds": [ 63, 51, 206 ]
}

And analogously, it can generate a UserSerializer class for the inverse operation. Of course, the input may be more complex - see Configuration and the generated ConfigurationParser for a more complex example.

The generated parsers and serializers are highly efficient because no intermediate DOM or other auxiliary data structures are constructed - data is read directly to / from the input structures.

How to use

To build the program, simply use the provided CMake file. JSON-CPP-gen has no dependencies except the C++ standard library.

To run the program, you must provide a JSON configuration file that corresponds to the Configuration structure as its command line argument. For example, to generate the UserParser and UserSerializer classes described above, you could use a configuration file containing:

{
    "inputs": [ "User.h" ],
    "includes": [ ],
    "settings": { },
    "parsers": [ {
        "name": "UserParser",
        "types": [ "User" ],
        "headerOutput": "UserParser.h",
        "sourceOutput": "UserParser.cpp"
    } ],
    "serializers": [ {
        "name": "UserSerializer",
        "types": [ "User" ],
        "headerOutput": "UserSerializer.h",
        "sourceOutput": "UserSerializer.cpp"
    } ],
    "stringType": "std::string"
}

All (non-absolute) file names are relative to the configuration file. includes may contain additional include files that should be present in the output files and settings may contain code generator settings. The includes, settings, and stringType fields could be omitted in this case since they are empty or equal to the default value. Even parsers or serializers could be omitted if you only needed one of the two. Additionally, custom string and container data types can be defined in the configuration file - see below. Another example is the configuration file for the ConfigurationParser class.

Features

  • Supported C++ types:
    • All fundamental integer and floating-point types, bool, and size_t
    • Structures (struct)
    • Enumerations (enum and enum class) - serialized as strings
    • Static arrays, including multi-dimensional
    • Standard library types:
      • std::string for strings
      • std::vector, std::deque, std::list for dynamic-sized arrays
      • std::array for static-sized arrays
      • std::map for objects with arbitrary keys and homogeneous values
      • std::optional as well as std::auto_ptr, std::shared_ptr, and std::unique_ptr for optional values
    • Custom string, array, object, and optional types if defined in the configuration file, see below
  • Namespaces
  • Structure inheritance (basic support)
  • Full UTF-8 & UTF-16 support

Currently NOT supported but planned features:

  • Out of order structure declaration - input header files must currently be in the correct order - WIP
  • Anonymous structures
  • Structure / enum declaration and variable declaration in one statement
  • Omitting specific member variables from parsing and serialization - planned via annotations
  • Structure members under different name in JSON - planned via annotations
  • Classes - currently ignored due to their data being typically private but explicit enablement via annotations is planned - access to private members must be ensured by the user
  • #define, typedef, using aliases - basic support planned

What will (probably) never be supported:

  • Heterogeneous JSON objects - not really representable by static stuctures
  • Unions - impossible to serialize without more advanced logic
  • Raw pointers - no point in serializing memory addresses and unclear memory management otherwise
  • Template structures
  • More complex expression / macro evaluation (e.g. array of length 20*sizeof(User))

Custom types

Even though the standard library string and container types are supported by default, you are not locked into using these exclusively in order for the automatic parser and serializer generation to work. In fact, you may configure JSON-CPP-gen even to produce code where none of the C++ standard library is used.

However, in order to do that, you must provide a set of replacement classes, and for each an API that dictates how it should be used. The API consists of code snippets for specific operations with placeholders. See comments in Configuration.h for a full list of the placeholders. The following are examples for each category of definable custom types and how they should be written in the configuration file. You can specify multiple types for each category.

These are mostly demonstrated on types from the standard library, which however should not be put into the configuration file as they are present implicitly and would cause collisions.

String (dynamic)

"stringTypes": [ {
    "name": "std::string",
    "api": {
        "clear": "$S.clear()",
        "appendChar": "$S.push_back($X)",
        "appendCStr": "$S += $X",
        "iterateChars": "for (char $E : $S) { $F }"
    }
} ]

Constant string

Represents a string type that cannot be constructed incrementally. Does not have an analogue in the standard library. An intermediate (dynamic) string type must be specified for the parser (can be std::string).

"constStringTypes": [ {
    "name": "ConstString",
    "stringType": "std::string",
    "api": {
        "copyFromString": "$S = $X",
        "moveFromString": "$S = std::move($X)",
        "iterateChars": "for (char $E : $S) { $F }"
    }
} ]

Array (dynamic)

"arrayContainerTypes": [ {
    "name": "std::vector<$T>",
    "api": {
        "clear": "$S.clear()",
        "refAppended": "($S.emplace_back(), $S.back())",
        "iterateElements": "for ($T const &$E : $S) { $F }"
    }
} ]

Note: The refAppended operation must add an empty element at the end of the array and return a modifiable reference to the new element.

Fixed-length array

Represents a non-statically fixed-length array that cannot be constructed incrementally. Does not have an analogue in the standard library. An intermediate (dynamic) array type must be specified for the parser (can be std::vector or another implicit array type).

"fixedArrayContainerTypes": [ {
    "name": "FixedArray<$T>",
    "arrayContainerType": "std::vector",
    "api": {
        "copyFromArrayContainer": "$S = $X",
        "moveFromArrayContainer": "$S = std::move($X)",
        "iterateElements": "for ($T const &$E : $S) { $F }"
    }
} ]

Static length array

"staticArrayContainerTypes": [ {
    "name": "std::array<$T, $N>",
    "api": {
        "refByIndex": "$S[$I]"
    }
} ]

Note: $N is array length.

Object (with implicit key type)

"objectContainerTypes": [ {
    "name": "std::map<std::string, $T>",
    "keyType": "std::string",
    "api": {
        "clear": "$S.clear()",
        "refByKey": "$S[$K]",
        "iterateElements": "for (const std::pair<$U, $T> &$I : $S) { $U const &$K = $I.first; $T const &$V = $I.second; $F }"
    }
} ]

Notes: A key type must be specified as keyType (can be std::string). The refByKey operation must create the element if it doesn't already exist and return a modifiable reference to its value. The iterateElements operation must provide keys and values separately as $K and $V.

Object map (with explicit key type)

"objectMapContainerTypes": [ {
    "name": "std::map<$K, $T>",
    "api": {
        "clear": "$S.clear()",
        "refByKey": "$S[$K]",
        "iterateElements": "for (const std::pair<$U, $T> &$I : $S) { $U const &$K = $I.first; $T const &$V = $I.second; $F }"
    }
} ]

Optional value

"optionalContainerTypes": [ {
    "name": "std::optional<$T>",
    "api": {
        "clear": "$S.reset()",
        "refInitialized": "($S = $T()).value()",
        "hasValue": "$S.has_value()",
        "getValue": "$S.value()"
    }
} ]

Note: The refInitialized operation must initialize the container with a default-constructed value and return a modifiable reference to the value.

Comments
  • enums not working as desired?

    enums not working as desired?

    (Linux, gcc 9.4.0)

    The project looks promising, I'd rather use a generator for serialization instead of code annotation and other weird hacks.

    I'm using an enum as a struct field and it is not serializing / parsing as desired. The field is being ignored completely or always parsed as zero.

    The desire is to get this:

    {"_allkeys":[
    {"_act":"NextImage","_keyval":11},
    {"_act":"NextImage","_keyval":12},
    {"_act":"PrevImage","_keyval":13}
    ]}
    

    But I'm getting this:

    {"_allkeys":[
    {"_keyval":11},
    {"_keyval":12},
    {"_keyval":13}
    ]}
    

    Note the act_ field, which is an enum, is missing.

    I attach what I hope is a complete set of files to demo the problem.

    actionKeysTest.zip

  • Incorrect error?

    Incorrect error?

    I had a syntax error in my struct as so:

    struct Foo {
       enum Bar _bar;
    };
    

    and the error code in json-cpp-gen trying to parse this was "INVALID_STRUCT_SYNTAX". Should that be "INVALID_ENUM_SYNTAX"?

    [HeaderParser.cpp, parseEnum(), line 242]

  • Output position when JSON parser fails

    Output position when JSON parser fails

    The generated parser should report the postition within the input string if parsing fails. This is simply cur - jsonString. To achieve this, Error must be a structure containing both the error code and position. The current enumeration can be renamed to ErrorType. Proposed error structure:

    struct Error {
        ErrorType type;
        int position;
    
        inline Error(ErrorType type = OK, int position = -1) : type(type), position(position) { }
        operator ErrorType() const;
        operator bool() const;
    };
    

    For serializers, reporting the source of error would be tricky because it is an element within a structure. Providing a pointer to the faulty element would be possible but probably not too helpful, because it isn't enough to easily find it within the structure tree. Still, serializers' error enumeration should also be renamed to ErrorType for consistency (with a possibility of typedef ErrorType Error;.

  • Add space inbetween multiple closing template brackets

    Add space inbetween multiple closing template brackets

    Some compilers may have problems with expressions such as std::vector<std::vector<int>>(orig) due to potentially interpreting the two closing brackets as a shift operator. To maximize portability of generated code, it should be ensured that a space is added in such scenarios.

  • Add error to string conversion to parsers & serializers

    Add error to string conversion to parsers & serializers

    Add a public

    static const char * errorString(Error error);
    

    to parsers and serializers if enabled in Settings. Also get rid of listing all error types in multiple places while at it, instead putting them in a list or a macro iterator, e.g.

    https://github.com/Chlumsky/json-cpp-gen/blob/bb1a666e834e21da03faf57a28771f516aa48daa/src/ParserGenerator.cpp#L296-L304

  • Improve error reporting

    Improve error reporting

    If the program fails, it is pretty hard to guess why without using a debugger. Try to provide some useful information regarding the cause of the failure.

  • Ignore elements with unrecognized template arguments

    Ignore elements with unrecognized template arguments

    The intended behavior is to skip any structure elements with unrecognized types. I believe this works fine if the unrecognized type is the direct type of the element, but if it is used within a template argument, the header parser fails.

  • Get rid of sscanf / sprintf

    Get rid of sscanf / sprintf

    Not only are these functions archaic and overly complex (format string parsing etc.) but I have found that sscanf in particular is extremely slow. This needs to be done away with ASAP. However, the implementation for floats is very complex. There should also be some way for the user to select an implementation or provide their own functions for number (de)serialization.

  • First official release

    First official release

    Release the first official stable version along with a binary. It should probably also have an icon, at least on Windows. Depending on the state the project is in, it could be version 1.0.0, or something like 0.9.0.

  • Deduplicate code in object map container type

    Deduplicate code in object map container type

    The source code of the following functions is exatly the same between the classes ObjectContainerType and ObjectMapContainerType:

    • generateParserFunctionBody
    • generateSerializerFunctionBody
    • generateClear
    • generateRefByKey
    • generateIterateElements

    However, each class must have a different base class. I think this could be fixed with templates and if that fails, simply convert these to static in one class and expose them for the other to use.

https://github.com/json-c/json-c is the official code repository for json-c. See the wiki for release tarballs for download. API docs at http://json-c.github.io/json-c/

\mainpage json-c Overview and Build Status Building on Unix Prerequisites Build commands CMake options Testing Building with vcpkg Linking to libjson-

Dec 31, 2022
a header-file-only, JSON parser serializer in C++

PicoJSON - a C++ JSON parser / serializer Copyright © 2009-2010 Cybozu Labs, Inc. Copyright © 2011-2015 Kazuho Oku Licensed under 2-clause BSD license

Dec 27, 2022
json-build is a zero-allocation JSON serializer compatible with C89

json-build is a zero-allocation JSON serializer compatible with C89. It is inspired by jsmn, a minimalistic JSON tokenizer.

Nov 16, 2022
JSON parser and generator for C/C++ with scanf/printf like interface. Targeting embedded systems.

JSON parser and emitter for C/C++ Features ISO C and ISO C++ compliant portable code Very small footprint No dependencies json_scanf() scans a string

Dec 30, 2022
A fast JSON parser/generator for C++ with both SAX/DOM style API
A fast JSON parser/generator for C++ with both SAX/DOM style API

A fast JSON parser/generator for C++ with both SAX/DOM style API Tencent is pleased to support the open source community by making RapidJSON available

Dec 30, 2022
RapidJSON is a JSON parser and generator for C++.
RapidJSON is a JSON parser and generator for C++.

A fast JSON parser/generator for C++ with both SAX/DOM style API

Dec 30, 2022
This is a JSON C++ library. It can write and read JSON files with ease and speed.

Json Box JSON (JavaScript Object Notation) is a lightweight data-interchange format. Json Box is a C++ library used to read and write JSON with ease a

Dec 4, 2022
🗄️ single header json parser for C and C++

??️ json.h A simple single header solution to parsing JSON in C and C++. JSON is parsed into a read-only, single allocation buffer. The current suppor

Jan 7, 2023
single-header json parser for c99 and c++

ghh_json.h a single-header ISO-C99 (and C++ compatible) json loader. why? obviously this isn't the first json library written for C, so why would I wr

Dec 1, 2022
Mapping json to and from a c++ structure

StructMapping документация на русском English translation provided by translate.google.com Table of contents Introduction Compatibility Installation U

Dec 14, 2022
Ultralightweight JSON parser in ANSI C

cJSON Ultralightweight JSON parser in ANSI C. Table of contents License Usage Welcome to cJSON Building Copying the source CMake Makefile Vcpkg Includ

Jan 4, 2023
JSON & BSON parser/writer

jbson is a library for building & iterating BSON data, and JSON documents in C++14. \tableofcontents Features # {#features} Header only. Boost license

Sep 14, 2022
Jsmn is a world fastest JSON parser/tokenizer. This is the official repo replacing the old one at Bitbucket

JSMN jsmn (pronounced like 'jasmine') is a minimalistic JSON parser in C. It can be easily integrated into resource-limited or embedded projects. You

Jan 9, 2023
A JSON parser in C++

JSON++ Introduction JSON++ is a light-weight JSON parser, writer and reader written in C++. JSON++ can also convert JSON documents into lossless XML d

Dec 28, 2022
Very low footprint JSON parser written in portable ANSI C

Very low footprint JSON parser written in portable ANSI C. BSD licensed with no dependencies (i.e. just drop the C file into your project) Never recur

Jan 5, 2023
Very simple C++ JSON Parser

Very simple JSON parser for c++ data.json: { "examples": [ { "tag_name": "a", "attr": [ { "key":

Nov 20, 2022
a JSON parser and printer library in C. easy to integrate with any model.

libjson - simple and efficient json parser and printer in C Introduction libjson is a simple library without any dependancies to parse and pretty prin

Nov 21, 2022
Lightweight, extremely high-performance JSON parser for C++11

sajson sajson is an extremely high-performance, in-place, DOM-style JSON parser written in C++. Originally, sajson meant Single Allocation JSON, but i

Dec 16, 2022
🔋 In-place lightweight JSON parser

?? JSON parser for C This is very simple and very powerful JSON parser. It creates DOM-like data structure and allows to iterate and process JSON obje

Dec 10, 2022