Bond is a cross-platform framework for working with schematized data. It supports cross-language de/serialization and powerful generic mechanisms for efficiently manipulating data. Bond is broadly used at Microsoft in high scale services.

The Bond logo: a stylized glue gun


Build Status Bond.CSharp NuGet package

Bond

Bond is an open-source, cross-platform framework for working with schematized data. It supports cross-language serialization/deserialization and powerful generic mechanisms for efficiently manipulating data. Bond is broadly used at Microsoft in high scale services.

Bond is published on GitHub at https://github.com/microsoft/bond/.

For details, see the User's Manuals:

For a discussion about how Bond compares to similar frameworks see Why Bond.

Dependencies

Bond C++ library requires some C++11 features (currently limited to those supported by Visual C++ 2015); a C++11 compiler is required. Additionally, to build Bond you will need CMake (3.1+), Haskell Stack (1.5.1+) and Boost (1.61+).

Additionally, Bond requires RapidJSON and optionally requires gRPC. The Bond repository primarily uses Git submodules for these two dependencies. It should be cloned with the --recursive flag:

git clone --recursive https://github.com/microsoft/bond.git

If you already have RapidJSON and would like to build against it, add argument -DBOND_FIND_RAPIDJSON=TRUE to the CMake invocation. It will use find_package(RapidJSON). If you do not provide a RapidJSON library, Bond will also install RapidJSON.

If you do not wish to build the gRPC component, add argument -DBOND_ENABLE_GRPC=FALSE to the CMake invocation.

Following are specific instructions for building on various platforms.

Linux

Bond must be built with C++11 compiler. We test with Clang (3.8) and GNU C++ (5.4). We recommend Clang as it's faster with template-heavy code like Bond.

Run the following commands to install the minimal set of packages needed to build the core Bond library on Ubuntu 14.04:

sudo apt-get install \
    clang \
    cmake \
    zlib1g-dev \
    libboost-dev \
    libboost-thread-dev

Additionally, you need the Haskell Tool Stack. If your distro isn't shipping a new enough version of it, you may encounter some non-obvious build failures, so we recommend installing the latest Stack outside of package management:

curl -sSL https://get.haskellstack.org/ | sh

In the root bond directory run:

mkdir build
cd build
cmake -DBOND_ENABLE_GRPC=FALSE ..
make
sudo make install

The build directory is just an example. Any directory can be used as the build destination.

To build the Bond Python module, all the C++/Python tests and examples, and Bond-over-gRPC, a few more packages are needed.

sudo apt-get install \
    autoconf \
    build-essential \
    golang \
    libboost-date-time-dev \
    libboost-python-dev \
    libboost-test-dev \
    libtool \
    python2.7-dev

CMake needs to be re-run with different options. This can be done after building just the core libraries: the build tree will simply be updated with the new options.

cd build # or wherever you ran CMake before
cmake -DBOND_ENABLE_GRPC=TRUE -DgRPC_ZLIB_PROVIDER=package ..

Running the following command in the build directory will build and execute all the tests and examples:

make --jobs 8 check
sudo make install # To install the other libraries just built

(The unit tests are large so you may want to run 4-8 build jobs in parallel, assuming you have enough memory.)

macOS

Install Xcode and then run the following command to install the required packages using Homebrew (http://brew.sh/):

brew install \
    cmake \
    haskell-stack \
    boost \
    boost-python

(boost-python is optional and only needed for Python support.)

Bond can be built on macOS using either standard *nix makefiles or Xcode. In order to generate and build from makefiles, in the root bond directory run:

mkdir build
cd build
cmake -DBOND_ENABLE_GRPC=FALSE ..
make
sudo make install

Alternatively, you can generate Xcode projects by passing the -G Xcode option to cmake:

cmake -DBOND_ENABLE_GRPC=FALSE -G Xcode ..

You can build and run unit tests by building the check target in Xcode or by running make in the build directory:

make --jobs 8 check

Note that if you are using Homebrew's Python, you'll need to build boost-python from source:

brew install --build-from-source boost-python

and tell cmake the location of Homebrew's libpython by setting the PYTHON_LIBRARY variable, e.g.:

cmake .. \
    -DPYTHON_LIBRARY=/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib

Windows

Build Status

Install the following tools:

If you are building on a network behind a proxy, set the environment variable HTTP_PROXY, e.g.:

set HTTP_PROXY=http://your-proxy-name:80

Now you are ready to build the C# version of Bond. Open the solution file cs\cs.sln in Visual Studio and build as usual. The C# unit tests can also be run from within the solution.

To build using the .NET Core SDK:

dotnet restore cs\cs.sln
dotnet msbuild cs\cs.sln

The C++ and Python versions of Bond additionally require:

You may need to set the environment variables BOOST_ROOT and BOOST_LIBRARYDIR to specify where Boost and its pre-built libraries for your environment (MSVC 12 or MSVC 14) can be found, e.g.:

set BOOST_ROOT=D:\boost_1_61_0
set BOOST_LIBRARYDIR=D:\boost_1_61_0\lib64-msvc-14.0

The core Bond library and most examples only require Boost headers. The pre-built libraries are only needed for unit tests, Python, and gRPC support. If Boost or Python libraries are not found on the system, then some tests and examples will not be built.

To generate a solution to build the Bond Core C++ and Python with Visual Studio 2015 run the following commands from the root bond directory:

mkdir build
cd build
set PreferredToolArchitecture=x64
cmake -DBOND_ENABLE_GRPC=FALSE -G "Visual Studio 14 2015 Win64" ..

Setting PreferredToolArchitecture=x64 selects the 64-bit toolchain which dramatically improves build speed. (The Bond unit tests are too big to build with 32-bit tools.)

Instead of cmake you can also use cmake-gui and specify configuration settings in the UI. This configuration step has to be performed only once. From then on you can use the generated solution build\bond.sln from Visual Studio or build from the command line using cmake:

cmake --build . --target
cmake --build . --target INSTALL

To build and execute the unit tests and examples run:

cmake --build . --target check -- /maxcpucount:8

To build Bond's gRPC++ integration from source, some of gRPC's prerequisites are also needed:

choco install activeperl golang ninja yasm

You will also need to enable gRPC in the cmake configuration step by running the following in the build directory from above and then following the other cmake commands above:

cmake -DBOND_ENABLE_GRPC=TRUE -G "Visual Studio 14 2015 Win64" ..

Alternatively, you can build and install Bond using the vcpkg dependency manager:

git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.bat
./vcpkg integrate install
./vcpkg install bond

The Bond port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request in the vcpkg repository.

Contributing

Interested in contributing to Bond? Take a look at our contribution guidelines to get started.

Owner
Microsoft
Open source projects and samples from Microsoft
Microsoft
Comments
  • Can't compile the sources

    Can't compile the sources

    Linux platform:

    At first I tried to compile on Ubuntu 12.04, cmake 3.10 says -- Could NOT find Boost. I tried to install boost 1.46 and boost 1.48 from the ubuntu packages. Could you please add to the cmake output minimal required version of the Boost?

    Then I ordered 1GB RAM Ubuntu 14.04 VPS and tried to compile it again. cabal install update-install drops with out of memory exception. Please add minimum RAM requirements to the documentation.

    Also, what is a required mono version for building cs.sln? Will it be built with mono 3.10?

    Windows platform:

    I followed the steps in documentation (install cmake, haskell, update cabal, open cs.sln in Visual Studio 2013) and run the build. The build has failed on the step:

    Generating C:/Projects/bond/compiler/cabal.sandbox.config
    Could not find target path 
    

    The path C:\Projects\bond\compiler exists on the machine.

  • Bonded-nested Polymorphic types get lost during deserialization in C#

    Bonded-nested Polymorphic types get lost during deserialization in C#

    The primary problem this bug is trying to address is that the generic type parameter gets lost on a Clone action and so IBonded<T>.Deserialize() behaves differently depending on an object's level of nesting within the tree. For example, even though src.Shapes[0] = new Bonded<Circle>(circle), the following returns false (and ideally it would return true).

    // false; "typeof(ShapeProps) != typeof(CircleProps)"
    Debug.Assert(
        src.Shapes[0].Deserialize().Props.Deserialize().GetType() ==
        circle.Props.Deserialize().GetType());
    

    To illustrate this, I took the polymorphic_container.csproj sample and updated it to included polymorphic types nested within a bonded struct.

    Expected: Polymorphic Bonded types Deserialize as the derived type they were created with where possible

    Actual: Polymorphic Bonded types Deserialize only as the base type when nested inside another Bonded struct

    IDL:

    namespace Examples
    
    enum Type
    {
        Unknown,
        Circle,
        Rectange
    }
    
    struct ShapeProps
    {
    }
    
    struct Shape
    {
        0: Type Type = Unknown;
        1: bonded<ShapeProps> Props;
    }
    
    struct Circle : Shape
    {
        0: double Radius;
    }
    
    struct CircleProps : ShapeProps
    {
        0: string CircleString;
    }
    
    struct Rectangle : Shape
    {
        0: double Width;
        1: double Height;
    }
    
    struct RectangleProps : ShapeProps
    {
        0: string RectangleString;
    }
    

    Program.cs:

    static class Program
    {
        static void Main()
        {
            var circle = new Circle
            {
                Type = Type.Circle,
                Radius = 3.14,
                Props = new Bonded<CircleProps>(new CircleProps
                {
                    CircleString = "I'm a circle",
                }),
            };
    
            var rectangle = new Bonded<RectangleProps>(new Rectangle
            {
                Type = Type.Rectange,
                Width = 10,
                Height = 5.5,
                Props = new RectangleProps
                {
                    RectangleString = "I'm a rectangle",
                },
            });
    
            var src = new Polymorphic
            {
                Shapes = 
                {
                    new Bonded<Circle>(circle),
                    new Bonded<Rectangle>(rectangle)
                }
            };
    
            // ===
            // false; "typeof(ShapeProps) != typeof(CircleProps)"
            Debug.Assert(
                src.Shapes[0].Deserialize().Props.Deserialize().GetType() ==
                circle.Props.Deserialize().GetType());
            // ===
    
            // ... remainder omitted
        }
    }
    
  • [gbc] Upgrading to stack lts-14.4

    [gbc] Upgrading to stack lts-14.4

    stack lts-14.4 uses , among others, version 7 of megaparsec and version 0.8.7 of aeson-pretty.

    These versions of library have introduced some breaking changes in their respective API and this commit makes the corresponding changes in bond library.

    Also, stack lts-14.4 does not have derive package listed. The derive package was used in test cases for deriving instances of Arbitrary for Bond types. This commit replaces the package with quickcheck-arbitrary-template.

    references

    1. Megaparsec Changelog
    2. Megaparsec 7 changes
    3. quickcheck-arbitrary-template
  • Support for unsafe pointer based InputBuffer and OutputBuffer

    Support for unsafe pointer based InputBuffer and OutputBuffer

    The InputPtrBuffer and OutputPtrBuffer will not have the ability to grow because they do not have ownership of the memory involved.

    Added an optimized memory copy implementation that it's even faster than BlockCopy, System.Buffer (.Net 4.6) and looping for small values. That’s 4,8 Gb/sec against 3,6 Gb/sec (fastest in System.Buffer) in Bulk Copy mode in the tested environment. https://gist.github.com/redknightlois/149fdd3479cbd074f115

    I haven't implemented an optimized ReadBytes(). That would require some internal changes that I don't think I can do on my own without further discussion.

    Fixes #92

  • Implement move assignment for the bond::blob type

    Implement move assignment for the bond::blob type

    Pull request for issue #934. The tests are probably overkill, I can remove them if you want. But they showed that blob& operator=(blob&& that) = default; would copy the internal buffer instead of moving it, so I chose the swap implementation.

  • make with errors

    make with errors

    Hello,

    I tried to build bond locally by checking out the source from github. I experienced the following error after running command "make" in the build folder:

    openFile: does not exist (No such file or directory) CMake Error at cabal_build.cmake:33 (message): make[2]: *** [compiler/build/gbc/gbc] Error 1 make[1]: *** [compiler/CMakeFiles/gbc.dir/all] Error 2 make: *** [all] Error 2

    Build log is empty. There is a gbc compiler for Windows. How do i have it installed on Mac?

    Please advise. Thanks.

  • Add CI builds for C# solution

    Add CI builds for C# solution

    Allows to build the csharp solution in Travis CI. Tested only for linux platform, because Travis CI requires to do manual request to enable CI on multiple platforms. Also, beta-support of C# is enabled only for linux in Travis CI, so it may be need to do additional install of mono on os x platform in before_install step.

  • [c++ protocol] Add Read/Write/Skip type_alias methods to protocols

    [c++ protocol] Add Read/Write/Skip type_alias methods to protocols

    Read/Write/Skip methods are present in JSON protocol, but are missed in all other protocols. I copy-pasted them to Compact Binary, Fast Binary, Simple Binary and Random.

  • Add netstandard support to NuGet packages

    Add netstandard support to NuGet packages

    I think this should resolve #171.

    Adding support for NETStandard requires upgrading Newtonsoft.Json from 9.0.1, since that is the lowest version of that library that supports NETStandard.

    Per https://docs.nuget.org/ndocs/schema/target-frameworks, the existing Profile78 is equivalent to netstandard1.0, so that is what I used.

    Per https://github.com/davidfowl/NetStandard/issues/5, the current way to support both PCL and NETStandard projects is to have a copy of the DLL in both lib folders in the NuGet package, so I added a netstandard1.0 lib folder to the nuspec files that already had a portable-net45+wp80+win8+wpa81+dnxcore50 lib folder.

    All unit tests in cs\cs.sln and cs\nuget\nuget.sln passed, but I had to add an assembly binding redirect for cs\nuget\nuget.sln because it references the currently-published 5.0.0 NuGet packages which still reference the 7.x version of Newtonsoft.Json.

    I also created another test solution on my workstation with both a PCL and NETStandard project. I was able to add the modified NuGet packages to both projects, and I was able to serialize and deserialize a simple struct in CompactBinary, SimpleBinary, and Json formats.

  • added solution and projects for compilation with mono

    added solution and projects for compilation with mono

    cs.sln bond solution actively uses msbuild features which do not implemented in mono, so the original solution falis to build using mono. I added simply solution (without files generation steps) with two targets "Release and Debug", which can be compiled using xbuild or monodevelop. This solution makes only libraries, it does not allow to build tests or samples.

    Tested on Ubuntu linux 12.04 with monodevelop 5.5 and mono 3.10.0 (current stable versions from official Xamarin repository)

  • Visual Studio removed newly added .bond files from BodeCodegen

    Visual Studio removed newly added .bond files from BodeCodegen

    In the new csproj, VS automatically adds to the csproj if you add a new file with the .bond extension. It is therefore not included in the Bond code generation target by default.

    I need to explicitly set the build action for the file to "Bond code generation" to resolve this and have it included as expected. Is there some msbuild magic that can be done to resolve this?

  • Deprecating Bond support for EOL platforms and toolchains

    Deprecating Bond support for EOL platforms and toolchains

    Bond is currently supporting Ubuntu 14.04 with Clang 3.8 and g++ 5.4 and VS 2017 on Windows. Ubuntu 14.04 is past support as is VS 2017. Should we update the dependencies to?

    • Ubuntu 20.04 and the versions of Clang & GCC it ships with
    • VS 2019 atop Server 2019 My read is that the versions of gcc/clang/MSVC would support C++ 17. Should we bump up to C++ 17?

    We can also Boost dependency to require version 1.73 or later which was released on Apr 28, 2020 almost two years ago (https://www.boost.org/users/history/).

    For .Net, we can set support surface to

    • .Net Framework 4.8+
    • .Net Core 3.1
    • .Net 5+
  • Update Install-Boost.ps1

    Update Install-Boost.ps1

    I have tried implementing the functionality that if the version of boost is >=1.66 then the .dll and .lib files should be placed together in the -x32- or -x64- folders respectively, but the other functionality i am confused how will the .dll and .lib will get segregated.

  • Bond-over-gRPC will be deprecated February 2022

    Bond-over-gRPC will be deprecated February 2022

    Deprecation plan

    Microsoft is ceasing its direct investment in Bond-over-gRPC as of December 2021.

    The Bond serialization libraries are not impacted.

    Our current plan is to deprecate the Bond-over-gRPC APIs in February 2022 and remove them in May 2022. We have no plans to remove any existing Bond-over-gRPC packages from NuGet.org.

    Development of Bond by Microsoft is driven by the needs of the teams at Microsoft that use Bond, and, at Microsoft, we no longer need the Bond-over-gRPC feature set.

    Request for Help to Lead Development/Maintenance

    If there are members of the Bond community that are interested in leading development and maintenance of Bond-over-gRPC, we want to hear from you. Let us know in a comment on this issue how you can contribute.

    Alternatives

    If you are looking for something similar in spirit to Bond-over-gRPC, we recommend the following.


    What will be deprecated/deleted?

    We plan to deprecate in February 2022 and then remove in May 2022 the APIs under

    • cpp/inc/bond/ext/grpc/
    • cs/src/grpc/
    • things under compiler/ that are related to Bond-over-gRPC codegen
      • gbc will still be able to parse service definitions and emit them in its JSON-based AST

    We will release a version of Bond after the deprecation and after the removal. These will both be major version releases.

    We will clean up references to Bond-over-gRPC in the documentation.

    We will archive the Bond-over-gRPC example repo.

    What happens to the Bond.Grpc.CSharp NuGet package?

    The package will be delisted so it is hidden from search results. It will not be deleted, so all existing use of the Bond.Grpc.CSharp package will continue to work.

    Will I be able to get help with Bond-over-gRPC issues from this repository?

    After May 2022, we will be unable to assist with Bond-over-gRPC issues.

  • gbc does not handle UTF-8 characters in attribute values when generating C++ (and C#)

    gbc does not handle UTF-8 characters in attribute values when generating C++ (and C#)

    gbc successfully parses UTF-8 encoded bond files with attribute values like this one:

    namespace demo
    
    struct Thing
    {
         [DisplayName("∑")]
         0: int32 sum;
    }
    

    It generates the UTF-8 encoded JSON schema file without issues:

    (...)
    {
        "attrName": [
            "DisplayName"
        ],
        "attrValue": "∑"
    }
    

    Unfortunately, gbc fails when generating C++ (or C#) code with an error:

    gbc.exe: ...\demo_types.cpp: commitAndReleaseBuffer: invalid argument (invalid character)
    

    I tried Haskell string escaping but to no avail.

    I also found #687 but that issue relates to C++ identifiers.

    I see no immediate reason (and could not find one by searching through the repo) why UTF-8 encoded attribute values could not be properly generated in C++ code. GCC and Clang should pose no problem. For MSVC this might require setting the byte-order-mark in the generated source file though I'm not 100% sure about that (see Docs ).

  • Referencing both Bond.CSharp and Bond.Compiler.CSharp produces

    Referencing both Bond.CSharp and Bond.Compiler.CSharp produces "Duplicate BondCodegen items were included" error

    If both of the Bond.CSharp and Bond.Compiler.CSharp packages are referenced in the same SDK-style csproj file, an error about "Duplicate BondCodegen items were included" will be emitted.

    This is caused by each package including Common.props and Common.targets, so the <BondCodegen Include="**/*.bond" /> item in Common.props is evaluated twice, which results in the duplicate items in BondCodegen.

    Work-around: only reference Bond.CSharp xor Bond.Compiler.CSharp.

    The codegen targets should detect this and emit an error. Bonus points if they handle this automatically instead.

    Conditional MSBuild imports have, historically, confused Visual Studio, so an approach like <Import Project="Common.props" Condition=" '$(_bond_common_props_imported)' != 'true' " /> is probably not viable.

  • Allow forward declaring a type outside the present namespace

    Allow forward declaring a type outside the present namespace

    Since bond files can only declare a single namespace, forward declaring objects in other namespaces becomes tedious. While it may seem odd to need to forward declare an object in another namespace, when using custom aliases, the type may never be defined in bond. However, to keep the bond compiler happy, the type must exist.

    An alternative would be to allow a scoped namespace declaration.

Related tags
Cista is a simple, high-performance, zero-copy C++ serialization & reflection library.

Simple C++ Serialization & Reflection. Cista++ is a simple, open source (MIT license) C++17 compatible way of (de-)serializing C++ data structures. Si

Jun 11, 2022
Serialization framework for Unreal Engine Property System that just works!

DataConfig Serialization framework for Unreal Engine Property System that just works! Unreal Engine features a powerful Property System which implemen

Jun 12, 2022
A lightweight C++20 serialization framework

zpp::bits A modern C++20 binary serialization library, with just one header file. This library is a successor to zpp::serializer. The library tries to

Jun 9, 2022
Cap'n Proto serialization/RPC system - core tools and C++ library
Cap'n Proto serialization/RPC system - core tools and C++ library

Cap'n Proto is an insanely fast data interchange format and capability-based RPC system. Think JSON, except binary. Or think Protocol Buffers, except

Jun 23, 2022
Fast Binary Encoding is ultra fast and universal serialization solution for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift

Fast Binary Encoding (FBE) Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server request

Jun 16, 2022
Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications

Zmeya Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications. Zmeya is not even a serializ

Jun 6, 2022
Your binary serialization library

Bitsery Header only C++ binary serialization library. It is designed around the networking requirements for real-time data delivery, especially for ga

Jun 15, 2022
A C++11 library for serialization
A C++11 library for serialization

cereal - A C++11 library for serialization cereal is a header-only C++11 serialization library. cereal takes arbitrary data types and reversibly turns

Jun 23, 2022
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Jun 20, 2022
Yet Another Serialization
Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

Jun 20, 2022
Binary Serialization

Binn Binn is a binary data serialization format designed to be compact, fast and easy to use. Performance The elements are stored with their sizes to

Jun 13, 2022
An implementation of the MessagePack serialization format in C / msgpack.org[C]

CMP CMP is a C implementation of the MessagePack serialization format. It currently implements version 5 of the MessagePack Spec. CMP's goal is to be

May 27, 2022
MPack - A C encoder/decoder for the MessagePack serialization format / msgpack.org[C]

Introduction MPack is a C implementation of an encoder and decoder for the MessagePack serialization format. It is: Simple and easy to use Secure agai

May 24, 2022
Simple C++ 20 Serialization Library that works out of the box with aggregate types!

BinaryLove3 Simple C++ 20 Serialization Library that works out of the box with aggregate types! Requirements BinaryLove3 is a c++20 only library.

Dec 23, 2021
CppSerdes is a serialization/deserialization library designed with embedded systems in mind
CppSerdes is a serialization/deserialization library designed with embedded systems in mind

A C++ serialization/deserialization library designed with embedded systems in mind

Jun 3, 2022
Header-only library for automatic (de)serialization of C++ types to/from JSON.

fuser 1-file header-only library for automatic (de)serialization of C++ types to/from JSON. how it works The library has a predefined set of (de)seria

Jun 4, 2022
Yet Another Serialization
Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

Sep 7, 2021
C++17 library for all your binary de-/serialization needs

blobify blobify is a header-only C++17 library to handle binary de-/serialization in your project. Given a user-defined C++ struct, blobify can encode

May 29, 2022
universal serialization engine

A Universal Serialization Engine Based on compile-time Reflection iguana is a modern, universal and easy-to-use serialization engine developed in c++1

Jun 20, 2022