Does Boost.json support conversions from C++ objects to json, and from json to C++ objects? Use case is in the interface between C++ application and calls to the ArangoDB HTTP REST API. I'm using libcurl to generate REST calls (with json bodies) into ArangoDB, and I need to convert C++ object members and their values to json for those calls. Then, on the return route, I need to populate C++ object members with the json data returned from ArangoDB calls. I also need to know if such conversions can be done recursively, or only singly.

What of the case where my C++ objects exist in a vector or other std container? I'm looking for functionality akin to that noted here: https://stackoverflow.com/questions/17549906/c-json-serialization

Scroll down to the solution with 85 votes, posted by Guillaume Racicot.

Right now, sentinel() casts the basic_parser_impl pointer (this) to const char *, but that pointer is not unique if the input buffer happens to be placed right before the basic_parser_impl instance - the end of that buffer then has the same address as basic_parser_impl.

Example code:

  struct {
    char buffer[8]{"{\"12345\""};
    boost::json::stream_parser p;
  } s;
  s.p.write(s.buffer, sizeof(s.buffer));
  s.p.write(":0}", 3);

This stops parsing at the end of the buffer, and then the incomplete() check in parse_string() will return true; the second write() call will crash with assertion failure:

boost/json/basic_parser_impl.hpp:1016: const char* boost::json::basic_parser::parse_unescaped(const char*, std::integral_constant<bool, StackEmpty_>, std::integral_constant<bool, AllowComments_>, bool) [with bool StackEmpty_ = true; bool IsKey_ = true; Handler = boost::json::detail::handler]: Assertion `*cs == '\x22'' failed.

This changes sentinel() to return the address of a static variable instead, which is guaranteed to be unique.

The list of changes:

• take advantage of mp11
• split container_traits into several traits and helpers
• split map_traits into several traits
• use value_type more consistently
• refactor detail::inserter
• use detail::inserter with map-like types
• refactor conversion of tuple-like to array
• remove conversion from value_ref and don't use value_to/from when constructing value from value_ref
• check for round-trip
• user-facing conversion traits
• opt-in null-like conversion
• same requirements for value_to and value_from
• stricter is_map_like
• stricter is_sequence_like

Fix #700

This implementation makes these choices:

1. Nulls are replaced when creating intermediaries.
2. When creating intermediaries segments are always treated as object keys, (that is only objects are created as intermediaries).
3. Arrays are extended by 1 element at most.

At the moment key_value_pair cannot be used for structured bindings without user boilerplate. Since it does model a pair anyway and offers an interface similar to std::pair, it would be nice for it to also specialize std::tuple_size, std::tuple_element and a form of get when using C++17 or above This would make the following code well-formed in C++17 or above:

object o{{"one", 1}, {"two", 2}};
const auto it = o.find("one");
if (it == o.end())
  handle_error();
const auto& [key, value] = *it;

Note that replacing object with std::map<std::string, int> would produce well-formed C++17 code (see https://godbolt.org/z/fVZUkC)

If this is something the authors would like too see too, I'd happily make a PR for it

Currently trying out the new JSON pointer implementation and I think this would be very useful in many cases.

One thing I have needed is inserting or assigning arbitrary JSON values at locations specified by JSON pointers. Is it currently possible? How can I achieve it in the current state?

When I'm using Boost.JSON along with Boost.Beast, it's common to parse a json frame from a remote http/websocket server and do some processing with it.

A typical scenario is, I need to pass a json::string to a function where a std::string_view is expected.

I can use standalone Boost.JSON with macro BOOST_JSON_STANDALONE to replace boost::string_view with std::string_view. But since I also need Boost.Beast, mixing Boost and standalone Boost.JSON would result in a lot of "macro redefinition" error.

So now I have to tediously do the conversion like this everytime

void process_sv(std::string_view sv);

auto j_str = j_v.as_string();
auto j_sv = std::string_view(j_str.data(), j_str.size());

process_sv(j_sv);

It would make it much more convenient if a direct conversion from json::string to std::string_view is available.

Thank you!

I want to interface with a finance service that unfortunately encodes monetary quantities as JSON numbers. To process those correctly, parsing and storage of numbers needs to be customized. Specifically, I want to parse them into boost::multiprecision::cpp_dec_float.

(I realize that this particular case can be worked around by converting the parsed doubles back to shortest string representation and parsing decimal floats from that. However, my point stands - it's useful to be able to handle numbers that can't be represented by int64_t/uint64_t/double, like big integers or arbitrary precision floats.)

Well, I thought, I just need a template-based JSON library which would let me supply my custom numeric type and a parser for it. Something like nlohmann::json, perhaps. Except nlohmann::json doesn't let you supply arbitrary numeric types, only built-in ones. In fact, after a few hours spent surveying numerous C++ JSON libs on github, to my great surprise I couldn't find one that would let me do that.

Of course, I realize boost::json is explicitly designed with different goals in mind. And I'm not advocating converting it to templates or anything like that. Fortunately, that's unnecessary. Consider:

• To parse_options add bool numbers_as_strings = false; member (perhaps partitioned into ints_as_strings/floats_as_strings).
• Find a single spare bit somewhere within string representation. Use it to indicate that this string was parsed from JSON number/must be serialized as JSON number. Let's call it raw_number flag. Initialize it to false. Add a getter and a setter for it, both public.
• When parsing, if numbers_as_strings is set upon encountering a number, create a string consisting of exactly the characters comprising the number in input. Set the raw_number flag on it. Otherwise, parse int64_t/uint64_t/double as usual.
• When serializing a string, if raw_number flag is set on it, assume the content is a valid JSON number and dump it into the output without quoting/escaping. Otherwise, serialize the string as usual.
• Optionally, provide functions to convert string to/from int64_t/uint64_t/double using the algorithms currently employed by the parser/serializer.

This moves number parsing/serialization to user code when necessary, in a way that doesn't complicate usage for people who don't care about this—admittedly quite rare—case. I believe it also fits the design principles of this library as I understand them. You guys have done a great job making common things easy, here I aim to make uncommon things possible.

Do you consider this a reasonable design? Something you're willing to merge if presented with a PR? (not that I have one at the moment)

This is a ticket to keep track of adding fuzz testing. I promised to help out, so here I am! I wanted to highlight the plan so everyone is happy with the direction.

The plan is to

• add a fuzz target (and build support, if necessary)
• helper scripts to get an initial seed corpus
• add a github action which runs the fuzzing for a short time (30 seconds or so) to make sure easy to find bugs are detected already at the pull request stage

Building

I had problem building the library - I suspect others wanting to try it out also may run into problems. Is there documentation somewhere that I missed? I expected the usual git clone, submodule recursive update, cmake to work out of the box but there seems to be a dependency on boost beast.

Fuzz target

There is already a fuzzer in #13 . @msimonsson are you ok that I incorporate your fuzzer code from that ticket under the boost license? I assume this is OK, but I am not a lawyer (well perhaps a C++ language lawyer wannabe :-).

Github Action

I have used this for the simdjson project recently, worked fine. I am not sure if it is possible to browse through it unless being a member, but here are some links:

• the config: https://github.com/lemire/simdjson/blob/master/.github/workflows/fuzzers.yml
• the jobs: https://github.com/lemire/simdjson/actions
• partial motivation: https://github.com/lemire/simdjson/issues/370

For efficiency, it is good if the corpus can be stored somewhere between the runs. Otherwise it has to bootstrap each time which is inefficient. I use bintray for simdjson - @vinniefalco where would you be ok to store the corpus, do you perhaps already have a bintray account? In the meanwhile, I will use my own.

Implementation

I develop this in my clone for now. See the Readme over here: https://github.com/pauldreik/json/tree/paul/fuzz/fuzzing

https://github.com/boostorg/json/blob/5098730a87148228f65b2dcb40ecec7f15019c09/include/boost/json/storage_ptr.hpp#L108

boost::json::storage_ptr uses sequentially consistent atomic ops for the reference count. This is needlessly expensive on most non-x86 platforms, in particular arm and the like. boost::shared_ptr apparently implements more efficient reference counting (usually relaxed inc + acquire release dec). Considering that there's one storage_ptr per value, that's potentially quite a large number of incs and decs for a large json document. I would suggest that storage_ptr reference count operations match those of boost::shared_ptr.

Some refactoring of root and test CMakeLists.txt

• Add json tests to tests target (as has been requested by @pdimov in #581)
• Refactor json usage requirements into a separate CMake target (fixes #562)
• Overhaul of dealing with CMake use cases (fixes #582)
• Fix test standalone_with_boost for CMake (fixes #577)
• Other minor changes

#807 introduced a special case for when an struct contains an optional member. But it is special cased for std::optional. Having a similar behaviour for boost::optional is logical, but implementing it the same way introduces a dependency on Boost.Optional and it's also a bit silly to have several special cases. It appears, we need a special coversion category for optionals. We can look at Boost.Url for ideas on how to implement the check for whether something is an optional.

Now that Boost.URL is accepted, Boost.JSON can throw out its older iteration of test_suite.hpp and use the one from URL, which is updated to be almost identical in behavior to Boost lightweight test in terms of diagnostic outputs.

For integration tips look to Boost.Http.Proto: https://github.com/CPPAlliance/http_proto/blob/f795e313bee75ca4310f899375227c23f64e1909/test/unit/Jamfile#L12 and https://github.com/CPPAlliance/http_proto/blob/f795e313bee75ca4310f899375227c23f64e1909/test/unit/CMakeLists.txt#L10 .

The commit https://github.com/boostorg/json/commit/cc019f753bacfbdd449c72176d435b7c81e28c15 changes a bunch of code into enormous macros, presumably so that file and line number information is more correcterer. While this might have been useful in that moment, we now have the problem that the debugger is not capable of getting a breakpoint at code inside the macro and stepping through it. As I have a failure occurring at code inside the macro, I cannot debug it effectively.

This change needs to be reverted. There is an alternate solution for getting better diagnostics from failing tests, which I will open as another issue.

Json could have a binary/blob type that can story raw memory and must/should have a conversion operator to a data type that can be represented as json. If optional that would be a string.

Motivation:

1. json::value as freeform data

In a statically typed language it is often useful to still have free form data for certain things, e.g. for dumping diagnostic information or data in addition to what's typed. json::value does an excellent job at this, and I have used it in exactly this way.

Adding in a binary type will make this more efficient and "more correct" in that we don't need to stringify data that is by nature binary. As an example, consider the following enum:

enum this_idea
{
   great = 1,
   greater,
   the_greatest
};

This enum could be stored as a single byte, yet obviously serialized into a string for json.

2. other serialization format

Some issues, such as #822 and #510 have address the possibility of adding other formats, such as MsgPack & CBOR respectively. Both of those formats know of a binary type. Since they are made to lower in data, the above enum would most likely be serialized as it's binary value in cbor and msgpack.

Rough design idea

The rough design idea is to provide a json::binary type that is either similar to json::string without the .c_str() or holds the raw memory as a void*, size_t pair.

At construction it should get assigned a tag and an optional conversion function. The tag indicates bulit-in conversions, such as are provided by cbor. The conversion function shall be without context and thus a raw function pointer.

With other serializers boost::json can become the basis of advanced RPC libraries to compete with the big ones, such as gRPC.

PLEASE DON'T FORGET TO "STAR" THIS REPOSITORY :) Hello Team, I am having issues when trying to serialize my JSON into a string before sending it. The problem is that the serialize command returns a NULL string. From what I can see the string is not empty it only say unable to read memory.

Version of Boost

1.81.0

Steps necessary to reproduce the problem

All elements in the map are std::string.

	boost::json::object JSONObject;
	JSONObject["type"] = this->PV_DTOMessageType;
	for (auto iter = keyValueMap.begin(); iter != keyValueMap.end(); ++iter) {
		JSONObject[iter->first] = iter->second;
	}
	return boost::json::serialize(JSONObject);