A C++11 library for serialization

cereal - A C++11 library for serialization

cereal is a header-only C++11 serialization library. cereal takes arbitrary data types and reversibly turns them into different representations, such as compact binary encodings, XML, or JSON. cereal was designed to be fast, light-weight, and easy to extend - it has no external dependencies and can be easily bundled with other code or used standalone.

cereal has great documentation

Looking for more information on how cereal works and its documentation? Visit cereal's web page to get the latest information.

cereal is easy to use

Installation and use of of cereal is fully documented on the main web page, but this is a quick and dirty version:

  • Download cereal and place the headers somewhere your code can see them
  • Write serialization functions for your custom types or use the built in support for the standard library cereal provides
  • Use the serialization archives to load and save data
#include <cereal/types/unordered_map.hpp>
#include <cereal/types/memory.hpp>
#include <cereal/archives/binary.hpp>
#include <fstream>
    
struct MyRecord
{
  uint8_t x, y;
  float z;
  
  template <class Archive>
  void serialize( Archive & ar )
  {
    ar( x, y, z );
  }
};
    
struct SomeData
{
  int32_t id;
  std::shared_ptr<std::unordered_map<uint32_t, MyRecord>> data;
  
  template <class Archive>
  void save( Archive & ar ) const
  {
    ar( data );
  }
      
  template <class Archive>
  void load( Archive & ar )
  {
    static int32_t idGen = 0;
    id = idGen++;
    ar( data );
  }
};

int main()
{
  std::ofstream os("out.cereal", std::ios::binary);
  cereal::BinaryOutputArchive archive( os );

  SomeData myData;
  archive( myData );

  return 0;
}

cereal has a mailing list

Either get in touch over email or on the web.

cereal has a permissive license

cereal is licensed under the BSD license.

cereal build status

  • master : Build Status Build status

Were you looking for the Haskell cereal? Go here.

Comments
  • Exposing references to objects created outside the cereal load process.

    Exposing references to objects created outside the cereal load process.

    Concept:

    During load you may have dependencies on something which is already in memory loaded from another aspect of your application. That may be something like a renderer, or a mouse, or a central texture atlas which has its own load/save process unrelated to your existing object hierarchy. You may even need to hook up some loaded/saved items to objects outside of the save/load heirarchy (as in the case with a central texture atlas, if you have textures from a loaded scene coming in, maybe you want to register them with that atlas as they get created.)

    Right now you can construct objects through cereal and any owned resources will be correctly loaded during the save/load(serialize) process. The issue is that handles to in-memory resources need to be reconstituted later.

    For simple cases of flat single type objects this can be trivial to hook up after load.

    RenderObject myRenderer;
    
    JSONInputArchive archive;
    std::shared_ptr<Rectangle> rect;
    archive(cereal::make_nvp("rect", rect));
    rect->setRenderer(myRenderer); //acceptable!  Unless the rectangle needs information about the viewport on load.  Maybe we have to re-design the class a bit, but probably workable.
    

    This becomes more difficult, however when layers are added, and polymorphic types step in. What if I have a clickable element deep inside a scene graph and it needs reference to the mouse wrangler?

    RenderThing myRenderer;
    MouseWrangler myMouse;
    
    JSONInputArchive archive;
    std::shared_ptr<Node> root;
    archive(cereal::make_nvp("root", root));
    root->setRenderer(myRenderer); //Same as above example!
    //root->setMouse(myMouse); //WHOOPSE!  What if setMouse only exists in clickable nodes?  Now the user of cereal has to make a choice as to how to fix the loaded object post-load, or make myMouse globally accessible!
    

    Ideally we would be able to supply these additional requirements which the enclosing scope (enclosing of the archive process) would already have access to. Then propogate that through the load process so objects could grab that information during their creation/loading instead of after.

    RenderThing myRenderer;
    MouseWrangler myMouse;
    Point defaultRotateOrigin; //just adding this for a "value" example of something we want copied.
    
    JSONInputArchive archive;
    
    archive.store( cereal::make_nvp("mouse", &myMouse), cereal::make_nvp("renderer", &myRenderer), cereal::make_nvp("defaultRotateOrigin", defaultRotateOrigin));
    
    std::shared_ptr<Node> root;
    archive(cereal::make_nvp("root", root));
    //Oh, and hey, all the items are now responsible for their own loading.  If they try to access a value that doesn't exist in the archive we can throw or something as usual.
    

    the following is some pseudo code:

    class Clickable : public Node {
    Clickable(const std::string &name, RenderThing* renderer, MouseWrangler &mouse, Point defaultRotateOrigin);
    ...
    
    template <class Archive>
    static void load_and_allocate( Archive & ar, cereal::allocate<Clickable > & allocate )
        {
          std::string name;
          Point position;
    
          ar( cereal::make_nvp("name", name), cereal::make_nvp("position", position) );
    
          RenderThing* renderer;
          MouseWrangler* mouse;
          Point defaultRotateOrigin;
          ar.extract(cereal::make_nvp("renderer", renderer), cereal::make_nvp("mouse", mouse), cereal::make_nvp("defaultRotateOrigin", defaultRotateOrigin));
          assert(mouse != null);
    
          allocate( name, renderer, mouse, defaultRotateOrigin );
          allocate.object().position = position;
    }
    ...
    }
    
  • PolymorphicVirtualCaster StaticObject instantiation takes a very long time at app startup

    PolymorphicVirtualCaster StaticObject instantiation takes a very long time at app startup

    I recently updated to a newer version of cereal and it seems like some things have changed.

    On startup Cereal creates these "StaticObject" instances now for every polymorphic registration. My app now takes around 60 seconds to startup, compared to 1 second previously. I'm on iOS with an iPad Pro.

    Why does this happen now? What's the benefit of these StaticObjects? Can I turn this off somehow?

  • Add serialization support for C++17 std::optional and std::variant

    Add serialization support for C++17 std::optional and std::variant

    The test and code has been written in similar style to that of the existing support for boost_variant, I hope this is fine. For the tests to run one shall pass -DCMAKE_CXX_STANDARD=17 to cmake.

  • Primitive types

    Primitive types

    I am missing the equivalent of BOOST_CLASS_IMPLEMENTATION(MyClass, primitive_type)

    Example: I have classes that already know how to convert to/from a string. so I use

    template<class Archive>                                 
    void save( Archive& archive ) const                         
    {                                                                           
        archive( ToString() );              
    }          
    

    resulting in the JSON

    "testobject": {
        "value0": "testobjectAsString"
    }
    

    but WHAT I WANT is just a single line

    "testobject": "testobjectAsString"
    

    Remark: the same thing happens for ENUMs in your implementation. They are always saved as a 2-level object, which seems unnecessary.


  • Nested circular references using load_and_allocate

    Nested circular references using load_and_allocate

    I'm opening this as a separate issue as it was introduced by the most recent couple check-ins on the develop branch. You should be using Visual Studio 2013 to reproduce this (I am currently on the November Compiler Prerelease as well, but I suspect it's a problem with the standard install too.)

    #include <iostream>
    #include <sstream>
    #include <string>
    #include <map>
    
    #include "cereal/cereal.hpp"
    #include "cereal/types/map.hpp"
    #include "cereal/types/vector.hpp"
    #include "cereal/types/memory.hpp"
    #include "cereal/types/string.hpp"
    #include "cereal/types/base_class.hpp"
    
    #include "cereal/archives/json.hpp"
    #include <cereal/types/polymorphic.hpp>
    
    class BaseClass : public std::enable_shared_from_this<BaseClass> {
    public:
        virtual ~BaseClass(){}
    
        template <class Archive>
        void serialize(Archive & archive){
            archive(CEREAL_NVP(name), CEREAL_NVP(baseMember));
        }
    protected:
        BaseClass(const std::string &a_name):
            name(a_name){
        }
    
        std::string name;
        int baseMember; //let this have random junk so we can see if it saves right.
    };
    
    class DerivedClass : public BaseClass {
        friend cereal::access;
    public:
        static std::shared_ptr<DerivedClass> make(const std::string &a_name, int a_derivedMember){
            return std::shared_ptr<DerivedClass>(new DerivedClass(a_name, a_derivedMember));
        }
    
        template <class Archive>
        void serialize(Archive & archive){
            archive(CEREAL_NVP(derivedMember), cereal::make_nvp("base", cereal::base_class<BaseClass>(this)));
        }
    private:
        DerivedClass(const std::string &a_name, int a_derivedMember):
            BaseClass(a_name),
            derivedMember(a_derivedMember){
        }
    
        template <class Archive>
        static DerivedClass * load_and_allocate(Archive &archive){
            int derivedMember;
            archive(CEREAL_NVP(derivedMember));
            DerivedClass* object = new DerivedClass("", derivedMember);
            archive(cereal::make_nvp("base", cereal::base_class<BaseClass>(object)));
            return object;
        }
    
        int derivedMember;
    };
    
    CEREAL_REGISTER_TYPE(DerivedClass);
    
    void saveTest(){
        std::stringstream stream;
        {
            cereal::JSONOutputArchive archive(stream);
            auto testSave = DerivedClass::make("TestName", 4);
            archive(cereal::make_nvp("test", testSave));
        }
        std::cout << stream.str() << std::endl;
        std::shared_ptr<DerivedClass> loaded;
        {
            cereal::JSONInputArchive archive(stream);
            archive(cereal::make_nvp("test", loaded));
        }
        std::stringstream stream2;
        {
            cereal::JSONOutputArchive archive(stream2);
            archive(cereal::make_nvp("test", loaded));
        }
        std::cout << stream2.str() << std::endl;
        std::cout << "TA-DA!" << std::endl;
    }
    
    int main(){
        saveTest();
    }
    

    The error I get is related to the line: if( ar.isSharedPointerValid ) in memory.hpp:

    The following is what I get from my LIVE CODE, not from the above test. I'll go ahead and get the error for the above code this evening.

    1>  textures.cpp
    1>C:\git\external\cereal\include\cereal/types/memory.hpp(171): error C3867: 'cereal::InputArchive<cereal::JSONInputArchive,0>::isSharedPointerValid': function call missing argument list; use '&cereal::InputArchive<cereal::JSONInputArchive,0>::isSharedPointerValid' to create a pointer to member
    1>          C:\git\external\cereal\include\cereal/cereal.hpp(767) : see reference to function template instantiation 'void cereal::load<AA,MV::FileTextureDefinition>(Archive &,cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &> &)' being compiled
    1>          with
    1>          [
    1>              AA=cereal::JSONInputArchive
    1>  ,            Archive=cereal::JSONInputArchive
    1>          ]
    1>          C:\git\external\cereal\include\cereal/cereal.hpp(692) : see reference to function template instantiation 'cereal::JSONInputArchive &cereal::InputArchive<cereal::JSONInputArchive,0>::processImpl<T>(T &)' being compiled
    1>          with
    1>          [
    1>              T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
    1>          ]
    1>          C:\git\external\cereal\include\cereal/cereal.hpp(692) : see reference to function template instantiation 'cereal::JSONInputArchive &cereal::InputArchive<cereal::JSONInputArchive,0>::processImpl<T>(T &)' being compiled
    1>          with
    1>          [
    1>              T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
    1>          ]
    1>          C:\git\external\cereal\include\cereal/cereal.hpp(558) : see reference to function template instantiation 'void cereal::InputArchive<cereal::JSONInputArchive,0>::process<_Ty>(T &&)' being compiled
    1>          with
    1>          [
    1>              _Ty=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
    1>  ,            T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
    1>          ]
    1>          C:\git\external\cereal\include\cereal/cereal.hpp(558) : see reference to function template instantiation 'void cereal::InputArchive<cereal::JSONInputArchive,0>::process<_Ty>(T &&)' being compiled
    1>          with
    1>          [
    1>              _Ty=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
    1>  ,            T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(160) : see reference to function template instantiation 'ArchiveType &cereal::InputArchive<ArchiveType,0>::operator ()<cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>>(cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &> &&)' being compiled
    1>          with
    1>          [
    1>              ArchiveType=cereal::JSONInputArchive
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(160) : see reference to function template instantiation 'ArchiveType &cereal::InputArchive<ArchiveType,0>::operator ()<cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>>(cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &> &&)' being compiled
    1>          with
    1>          [
    1>              ArchiveType=cereal::JSONInputArchive
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(150) : while compiling class template member function 'cereal::detail::InputBindingCreator<Archive,T>::InputBindingCreator(void)'
    1>          with
    1>          [
    1>              Archive=cereal::JSONInputArchive
    1>  ,            T=MV::FileTextureDefinition
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/static_object.hpp(56) : see reference to function template instantiation 'cereal::detail::InputBindingCreator<Archive,T>::InputBindingCreator(void)' being compiled
    1>          with
    1>          [
    1>              Archive=cereal::JSONInputArchive
    1>  ,            T=MV::FileTextureDefinition
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(293) : see reference to class template instantiation 'cereal::detail::InputBindingCreator<Archive,T>' being compiled
    1>          with
    1>          [
    1>              Archive=cereal::JSONInputArchive
    1>  ,            T=MV::FileTextureDefinition
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(290) : while compiling class template member function 'void cereal::detail::polymorphic_serialization_support<cereal::JSONInputArchive,T>::instantiate(void)'
    1>          with
    1>          [
    1>              T=MV::FileTextureDefinition
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(307) : see reference to class template instantiation 'cereal::detail::polymorphic_serialization_support<cereal::JSONInputArchive,T>' being compiled
    1>          with
    1>          [
    1>              T=MV::FileTextureDefinition
    1>          ]
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(306) : while compiling class template member function 'void cereal::detail::bind_to_archives<MV::FileTextureDefinition>::bind(std::false_type) const'
    1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(321) : see reference to function template instantiation 'void cereal::detail::bind_to_archives<MV::FileTextureDefinition>::bind(std::false_type) const' being compiled
    1>          Source\Render\textures.cpp(13) : see reference to class template instantiation 'cereal::detail::bind_to_archives<MV::FileTextureDefinition>' being compiled
    ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
    
  • Error With Nested Circular shared_ptr and weak_ptr References

    Error With Nested Circular shared_ptr and weak_ptr References

    I'm having issues with circular references. I'm not sure if this is another bug or if there's something I'm doing wrong. It seems like creating a nested pointer_wrapper to a node higher up the chain fails and simply returns 1. I'm getting an exception throwing on load because of this:

    throw Exception("Error while trying to deserialize a smart pointer. Could not find id " + std::to_string(id));
    

    Also, another oddity is that shared_ptr objects no matter if they are polymorphic or not seem to need to need CEREAL_REGISTER_TYPE which means I need to give them a virtual method as well. Trying to remove the CEREAL_REGISTER_TYPE and virtual destructor for ChildWithPointerToParent fails at compile time.

    #include <iostream>
    #include <sstream>
    #include <string>
    #include <map>
    
    #include "cereal/cereal.hpp"
    #include "cereal/types/map.hpp"
    #include "cereal/types/vector.hpp"
    #include "cereal/types/memory.hpp"
    #include "cereal/types/string.hpp"
    #include "cereal/types/base_class.hpp"
    
    #include "cereal/archives/json.hpp"
    #include <cereal/types/polymorphic.hpp>
    
    class ChildWithPointerToParent;
    
    class BaseClass : public std::enable_shared_from_this<BaseClass> {
    public:
        virtual ~BaseClass(){}
    
        template <class Archive>
        void serialize(Archive & archive){
            archive(CEREAL_NVP(name), CEREAL_NVP(baseMember), CEREAL_NVP(child));
        }
    
        void addChild(std::shared_ptr<ChildWithPointerToParent> a_child){
            child = a_child;
        }
    protected:
        BaseClass(const std::string &a_name):
            name(a_name){
        }
    
        std::weak_ptr<ChildWithPointerToParent> child;
        std::string name;
        int baseMember; //let this have random junk so we can see if it saves right.
    };
    
    class DerivedClass : public BaseClass {
        friend cereal::access;
    public:
        static std::shared_ptr<DerivedClass> make(const std::string &a_name, int a_derivedMember){
            return std::shared_ptr<DerivedClass>(new DerivedClass(a_name, a_derivedMember));
        }
    
        template <class Archive>
        void serialize(Archive & archive){
            archive(CEREAL_NVP(derivedMember), cereal::make_nvp("base", cereal::base_class<BaseClass>(this)));
        }
    private:
        DerivedClass(const std::string &a_name, int a_derivedMember):
            BaseClass(a_name),
            derivedMember(a_derivedMember){
        }
    
        template <class Archive>
        static DerivedClass * load_and_allocate(Archive &archive){
            return new DerivedClass("", 0); //values loaded in serialize (using work-around in memory.hpp)
        }
    
        int derivedMember;
    };
    
    class ChildWithPointerToParent {
        friend cereal::access;
    public:
        virtual ~ChildWithPointerToParent(){} //Why do I have to do this?  I get an error if I don't.
        static std::shared_ptr<ChildWithPointerToParent> make(std::shared_ptr<BaseClass> parent){
            return std::shared_ptr<ChildWithPointerToParent>(new ChildWithPointerToParent(parent));
        }
    
        template <class Archive>
        void serialize(Archive & archive){
            archive(CEREAL_NVP(parent));
        }
    
    private:
        ChildWithPointerToParent(std::shared_ptr<BaseClass> a_parent):
            parent(a_parent){
        }
    
        template <class Archive>
        static ChildWithPointerToParent * load_and_allocate(Archive &archive){
            return new ChildWithPointerToParent(nullptr); //values loaded in serialize (using work-around in memory.hpp)
        }
        std::shared_ptr<BaseClass> parent;
    };
    
    CEREAL_REGISTER_TYPE(DerivedClass);
    CEREAL_REGISTER_TYPE(ChildWithPointerToParent); //Why do I have to do this?  I get an error if I don't.
    
    void saveTest(){
        std::stringstream stream;
        {
            cereal::JSONOutputArchive archive(stream);
            std::shared_ptr<BaseClass> testSave = DerivedClass::make("TestName", 4);
            std::shared_ptr<ChildWithPointerToParent> child = ChildWithPointerToParent::make(testSave);
            testSave->addChild(child);
            archive(cereal::make_nvp("test", testSave));
        }
        std::cout << stream.str() << std::endl;
        std::shared_ptr<BaseClass> loaded;
        {
            cereal::JSONInputArchive archive(stream);
            archive(cereal::make_nvp("test", loaded));
        }
        std::stringstream stream2;
        {
            cereal::JSONOutputArchive archive(stream2);
            archive(cereal::make_nvp("test", loaded));
        }
        std::cout << stream2.str() << std::endl;
        std::cout << "TA-DA!" << std::endl;
    }
    
    int main(){
        saveTest
    }
    

    EXAMPLE OF REAL WORLD USE:


    I have a class of type TextureDefinition. A TextureDefinition is always created via shared_ptr. TextureDefinitions are typically stored in a map keyed by their "name" so that multiple identical TextureDefinitions don't exist, but optionally a TextureDefinition might be owned by a specific rendering object like a text character which manages its own dynamic texture. It's also possible that I only care about the TextureDefinition when loading a scene and won't want to have it owned by a central node at all, but just let it be owned by its TextureHandles (more on this later). Basically I don't have one draconic TextureManager class that deals with their lifespan.

    I have another class of type TextureHandle. A TextureHandle is created by a TextureDefinition always of type shared_ptr. When the first TextureHandle is created the TextureDefinition associated with it loads into memory. When the all TextureHandles are destroyed the TextureDefinition unloads, but if something still owns the TextureDefinition it remains possible to reload it by creating new handles.

    TextureDefinition maintains a list of weak_ptr's to TextureHandles (because they do not manage a TextureHandle's lifespan, something like Shape::Rectangle would have a shared_ptr).

    TextureHandle maintains a shared_ptr because it would be an error to have the TextureDefinition explode while there are any active TextureHandles.

    In this way, we can plot the object composition like this:

    ----> is weak_ptr
    ====> is shared_ptr
    TextureDefinition1|---->TextureHandle1===>TextureDefinition1
                      |---->TextureHandle2===>TextureDefinition1
                      |---->TextureHandle3===>TextureDefinition1
    

    The objects owning these could be a different shape for each TextureHandle and a scene manager owning the TextureDefinition so it can spawn new handles. If we don't care about spawning new handles (let's say we've loaded the whole scene already) that scene manager owned handle is optional, but the objects owning TextureHandles must still keep the TextureDefinition alive.

  • shared_from_this and members containing shared_ptr can cause issues with the destructor after saving.

    shared_from_this and members containing shared_ptr can cause issues with the destructor after saving.

    Seems to have an issue with the fact that we save drawListVector in the serialize method. If I comment out ar(CEREAL_NVP(drawListVector)); it stops crashing.

    It was not doing this prior to pulling the latest save_from_this fix for saving.

    #include <strstream>
    
    #include "cereal/cereal.hpp"
    #include "cereal/types/map.hpp"
    #include "cereal/types/vector.hpp"
    #include "cereal/types/memory.hpp"
    #include "cereal/types/string.hpp"
    #include "cereal/types/base_class.hpp"
    
    #include "cereal/archives/json.hpp"
    #include "cereal/types/polymorphic.hpp"
    
    #include <memory>
    
    struct A : std::enable_shared_from_this<A>
    {
        typedef std::vector<std::shared_ptr<A>> DrawListVectorType;
        DrawListVectorType drawListVector;
    
        template <class Archive>
        void serialize(Archive & ar)
        {
            ar(CEREAL_NVP(drawListVector)); //If we don't actually save the drawListVector it's fine.  If we do, we'll get a crash in the destructor.
        }
        virtual ~A(){
        }
    };
    
    struct B : A
    {
        template <class Archive>
        void serialize(Archive & ar)
        { }
    };
    
    CEREAL_REGISTER_TYPE(B);
    
    void saveTest(){
        {
            cereal::JSONOutputArchive ar(std::cout);
            std::shared_ptr<A> a = std::make_shared<A>();
            std::shared_ptr<A> b = std::make_shared<B>();
            std::shared_ptr<A> c = std::make_shared<A>();
            a->drawListVector.push_back(b);
    
            b->drawListVector.push_back(c);
    
            ar(a);
    
            auto x = a->shared_from_this();
            b.reset();
            a->drawListVector.clear();
            std::cout << "grood";
        }
        std::cout << "grood";
    }
    
    int main(){
        saveTest();
    }
    
  • Added a 'getNodeName' API to JSON/XML InputArchives

    Added a 'getNodeName' API to JSON/XML InputArchives

    If you are doing some pretty custom serialization/deserialization it can really help being able to know the current node key/tag.

    I couldn't really figure out where to put this into the unit tests - any hints? Would you want tests for this?

  • using type name attribute in XMLOutputArchive leaves undesired characters in type

    using type name attribute in XMLOutputArchive leaves undesired characters in type

    The problem is in this segment of code (in xml.hpp, at or around line 261)

      template <class T> inline
      void insertType()
      {
        if( !itsOutputType )
          return;
    
        // generate a name for this new node
        const auto nameString = util::demangledName<T>();
    
        // allocate strings for all of the data in the XML object
        auto namePtr = itsXML.allocate_string( nameString.data(), nameString.size() );
    
        itsNodes.top().node->append_attribute( itsXML.allocate_attribute( "type", namePtr ) );
      }
    

    Since the nameString parameter is mapped to a char ptr of length 99 issues arise. If the string is shorter, say 47 characters long, then what trails will be binary trash. If you open the resulting XML in your favorite browser it should fail to load, if you open it in an editor the trash should be obvious. To fix this problem you should explicitly let allocate_attribute know the size of "type" and "namePTR". The fix can be seen below.

    I have not checked to see if something similar has occurred elsewhere in the code.

      template <class T> inline
      void insertType()
      {
        if( !itsOutputType )
          return;
    
        // generate a name for this new node
        const auto nameString = util::demangledName<T>();
    
        // allocate strings for all of the data in the XML object
        auto namePtr = itsXML.allocate_string( nameString.data(), nameString.size() );
    
        itsNodes.top().node->append_attribute( itsXML.allocate_attribute( "type", namePtr , 4 , nameString.size() ) );
      }
    
  • JSON: unsigned long are base64 encoded

    JSON: unsigned long are base64 encoded

    When serializing an unsigned long to JSON, cereal outputs some non-human readable data which seems to be in base64. Indeed, looking at the source code (https://github.com/USCiLab/cereal/blob/master/include/cereal/archives/json.hpp#L110), any type with a size >= sizeof(long long) is considered 'exotic' and thus base64-encoded. Furthermore, the XML output is perfectly fine (by that, I mean readable).

    I fail to see why longs are base64 encoded. Is this a desired thing? Also, it means that when compiling the code in 32bits or 64 bits, we have a different output. Finally, it makes unnecessarily difficult to read the JSON output in another language (Python in my case)

  • Allow capitalized `Serialize` functions

    Allow capitalized `Serialize` functions

    Currently cereal searches for serialize, load and save. If they could be made #defines, the user of the library could specify the name of the functions that cereal should be looking for.

    #ifndef CEREAL_SERIALIZE_FUNCTION_NAME
    #define CEREAL_SERIALIZE_FUNCTION_NAME serialize
    #endif
    #ifndef CEREAL_LOAD_FUNCTION_NAME
    #define CEREAL_LOAD_FUNCTION_NAME load
    #endif
    #ifndef CEREAL_SAVE_FUNCTION_NAME
    #define CEREAL_SAVE_FUNCTION_NAME save
    #endif
    

    This is because some styling guides require functions that are Capitalized and having Serialize, Load, Save would conform.

    How do you guys feel about this? Is this something that would be acceptable to create a pull request for?

  • cereal::JSONOutputArchive serialize 1M large string cost more than 100ms

    cereal::JSONOutputArchive serialize 1M large string cost more than 100ms

    I use cereal::JSONOutputArchive serialize 1M large string, it takes more than 100ms. Is it a normal behavior ? Is there a solution to reduce the serialization time ?

    ` std::string str1 = std::string(100 * 1024, 'a'); oStr = std::ostringstream(); // 创建新ss对象 StopWatch sw; { cereal::JSONOutputArchive archive(oStr); archive(cereal::make_nvp("Test", str1)); } std::cout << "cereal::JSONOutputArchive 100k string cost time:" << sw.elapsed().count() << std::endl;

    std::string str2 = std::string(1024 * 1024, 'a');
    oStr = std::ostringstream();  // 创建新ss对象
    sw.reset();
    {
        cereal::JSONOutputArchive archive(oStr);
        archive(cereal::make_nvp("Test", str2));
    }
    std::cout << "cereal::JSONOutputArchive 1M string cost time:" << sw.elapsed().count() << std::endl;
    
    std::string str3 = std::string(10 * 1024 * 1024, 'a');
    oStr = std::ostringstream();  // 创建新ss对象
    sw.reset();
    {
        cereal::JSONOutputArchive archive(oStr);
        archive(cereal::make_nvp("Test", str3));
    }
    std::cout << "cereal::JSONOutputArchive 10M string cost time:" << sw.elapsed().count() << std::endl;
    

    `

    output log: cereal::JSONOutputArchive 100k string cost time:0.01547 cereal::JSONOutputArchive 1M string cost time:0.147921 cereal::JSONOutputArchive 10M string cost time:1.36517

  • How to name the outermost name tag in output XML?

    How to name the outermost name tag in output XML?

    Is there a way to change the outermost name tag of an XML output file? I have an XML output of a class like this: <?xml version="1.0" encoding="utf-8"?> <cereal> <Item> <ItemID>Item 1</ItemID> </Item> </cereal>

    Here instead of 'cereal', I would like to have a custom name for it.

  • deserialize json with cereal

    deserialize json with cereal

    I am trying to deserialize json file, the problem is that ifstream is not supported from what I have seen in another issue and tried myself. so this:

    std::ifstream input("config.json");
    cereal::JSONInputArchive archive(input);
    

    won't work.

    I have tried to read each line of the file into stringstream , which also isn't working because from what I have seen cereal::JSONInputArchive expect the format of the json to have quotation mark with escape characters like this - \" and stringstream isn't doing that while reading reading the file line by line.

    I have searched the net and Cereal docs but couldn't find a way to deserialize json.

  • handle trivially-copyable types correctly

    handle trivially-copyable types correctly

    Cereal has two implementations for contiguous standard library containers:

    1. serialise element-wise (default)
    2. serialise en-bloc

    The second implementation is only chosen for arithmetic element types when, in fact, it should be chosen for all trivially-copyable types; this is exactly what the trivially-copyable trait was made for 🙂

    This PR changes the behaviour. It reduces the deserialisation time in one of our applications by more than 5x!

    Note that the on-disk representation is not guaranteed to stay the same, although in many cases it will.

  • Fixed support for boost variant.

    Fixed support for boost variant.

    When boost variant is created using boost::make_variant_over, the template parameter list is not the list of variant types. The list has to be extracted from variant<...>::types fixes#490

  • test_cmake_config_module fails for in-source builds

    test_cmake_config_module fails for in-source builds

    To reproduce:

    cd cereal
    cmake .
    make
    ctest
    

    fails with:

    38: Test command: /usr/bin/cmake "-P" "/tmp/cereal-1.3.2/unittests/cmake-config-module.cmake"
    38: Test timeout computed to be: 10000000
    38: -- Configuring done
    38: -- Generating done
    38: -- Build files have been written to: /tmp/cereal-1.3.2
    38: Error: could not load cache
    38: CMake Error at cmake-config-module.cmake:30 (message):
    38:   Cereal cmake install-step failed
    

    Found in https://bugzilla.redhat.com/show_bug.cgi?id=2137999

A C++11 library for serialization
A C++11 library for serialization

cereal - A C++11 library for serialization cereal is a header-only C++11 serialization library. cereal takes arbitrary data types and reversibly turns

Dec 2, 2022
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Nov 27, 2022
Simple C++ 20 Serialization Library that works out of the box with aggregate types!

BinaryLove3 Simple C++ 20 Serialization Library that works out of the box with aggregate types! Requirements BinaryLove3 is a c++20 only library.

Sep 2, 2022
Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications

Zmeya Zmeya is a header-only C++11 binary serialization library designed for games and performance-critical applications. Zmeya is not even a serializ

Nov 2, 2022
CppSerdes is a serialization/deserialization library designed with embedded systems in mind
CppSerdes is a serialization/deserialization library designed with embedded systems in mind

A C++ serialization/deserialization library designed with embedded systems in mind

Nov 5, 2022
Header-only library for automatic (de)serialization of C++ types to/from JSON.

fuser 1-file header-only library for automatic (de)serialization of C++ types to/from JSON. how it works The library has a predefined set of (de)seria

Oct 20, 2022
C++17 library for all your binary de-/serialization needs

blobify blobify is a header-only C++17 library to handle binary de-/serialization in your project. Given a user-defined C++ struct, blobify can encode

Oct 20, 2022
Yet another JSON/YAML/BSON serialization library for C++.
Yet another JSON/YAML/BSON serialization library for C++.

ThorsSerializer Support for Json Yaml Bson NEW Benchmark Results Conformance mac linux Performance max linux For details see: JsonBenchmark Yet anothe

Oct 27, 2022
Cista is a simple, high-performance, zero-copy C++ serialization & reflection library.

Simple C++ Serialization & Reflection. Cista++ is a simple, open source (MIT license) C++17 compatible way of (de-)serializing C++ data structures. Si

Dec 1, 2022
Nov 28, 2022
Fast Binary Encoding is ultra fast and universal serialization solution for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift

Fast Binary Encoding (FBE) Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server request

Dec 1, 2022
Yet Another Serialization
Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

Nov 18, 2022
Binary Serialization

Binn Binn is a binary data serialization format designed to be compact, fast and easy to use. Performance The elements are stored with their sizes to

Nov 24, 2022
An implementation of the MessagePack serialization format in C / msgpack.org[C]

CMP CMP is a C implementation of the MessagePack serialization format. It currently implements version 5 of the MessagePack Spec. CMP's goal is to be

Nov 17, 2022
MPack - A C encoder/decoder for the MessagePack serialization format / msgpack.org[C]

Introduction MPack is a C implementation of an encoder and decoder for the MessagePack serialization format. It is: Simple and easy to use Secure agai

Nov 30, 2022
Serialization framework for Unreal Engine Property System that just works!

DataConfig Serialization framework for Unreal Engine Property System that just works! Unreal Engine features a powerful Property System which implemen

Nov 27, 2022
Yet Another Serialization
Yet Another Serialization

YAS Yet Another Serialization - YAS is created as a replacement of boost.serialization because of its insufficient speed of serialization (benchmark 1

Sep 7, 2021
universal serialization engine

A Universal Serialization Engine Based on compile-time Reflection iguana is a modern, universal and easy-to-use serialization engine developed in c++1

Dec 2, 2022
A lightweight C++20 serialization framework

zpp::bits A modern C++20 binary serialization library, with just one header file. This library is a successor to zpp::serializer. The library tries to

Nov 28, 2022