Protocol Buffers with small code size

Nanopb - Protocol Buffers for Embedded Systems

Build Status

Nanopb is a small code-size Protocol Buffers implementation in ansi C. It is especially suitable for use in microcontrollers, but fits any memory restricted system.

Using the nanopb library

To use the nanopb library, you need to do two things:

  1. Compile your .proto files for nanopb, using protoc.
  2. Include pb_encode.c, pb_decode.c and pb_common.c in your project.

The easiest way to get started is to study the project in "examples/simple". It contains a Makefile, which should work directly under most Linux systems. However, for any other kind of build system, see the manual steps in README.txt in that folder.

Generating the headers

Protocol Buffers messages are defined in a .proto file, which follows a standard format that is compatible with all Protocol Buffers libraries. To use it with nanopb, you need to generate .pb.c and .pb.h files from it:

python generator/nanopb_generator.py myprotocol.proto  # For source checkout
generator-bin/nanopb_generator myprotocol.proto        # For binary package

(Note: For instructions for nanopb-0.3.9.x and older, see the documentation of that particular version here)

The binary packages for Windows, Linux and Mac OS X should contain all necessary dependencies, including Python, python-protobuf library and protoc. If you are using a git checkout or a plain source distribution, you will need to install Python separately. Once you have Python, you can install the other dependencies with pip install protobuf grpcio-tools.

You can further customize the header generation by creating an .options file. See documentation for details.

Running the tests

If you want to perform further development of the nanopb core, or to verify its functionality using your compiler and platform, you'll want to run the test suite. The build rules for the test suite are implemented using Scons, so you need to have that installed (ex: sudo apt install scons or pip install scons). To run the tests:

cd tests
scons

This will show the progress of various test cases. If the output does not end in an error, the test cases were successful.

Note: Mac OS X by default aliases 'clang' as 'gcc', while not actually supporting the same command line options as gcc does. To run tests on Mac OS X, use: scons CC=clang CXX=clang. Same way can be used to run tests with different compilers on any platform.

For embedded platforms, there is currently support for running the tests on STM32 discovery board and simavr AVR simulator. Use scons PLATFORM=STM32 and scons PLATFORM=AVR to run these tests.

Build systems and integration

Nanopb C code itself is designed to be portable and easy to build on any platform. Often the bigger hurdle is running the generator which takes in the .proto files and outputs .pb.c definitions.

There exist build rules for several systems:

And also integration to platform interfaces:

Building nanopb - Using vcpkg

You can download and install nanopb using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install nanopb

The nanopb port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Owner
Comments
  • running with -q still outputs to stderr if unmatched fields in the options file

    running with -q still outputs to stderr if unmatched fields in the options file

    The line here, when there are items in the options files that don't match, will print to stderr even in quiet mode.

    Seems like a trivial fix, on my machine I indented that if statement once so that it is inside the elif not options.quiet: branch. But I am not sure that is the intended semantics.

    This is actually causing major issues because at least for my project Ninja is crashing if a command it runs prints to stderr! (I can't explain that, but have shown it though trial and error).

  • Option package prefix also msg from other .proto

    Option package prefix also msg from other .proto

    Hi, I'm using protobuf and nanopb in some projects and I follow the google recommendations for the directories architectures ////<proto_files>

    As this path is really long it generate struct like <orga>_<project>_<package>_<version>_<MSG_NAME>. So I use the option (nanopb_fileopt).package = "<project>_<package>" to reduce message name to <project>_<package>_<MSG_NAME>

    Now I just wanted to import a proto (A) into an other (B) and use it's msg into.

    A.proto

    package <orga>/<project>/<B_packages>/<version>/<A_proto_file>
    option (nanopb_fileopt).package = "<project>_<A_package>"
    
    message Tata {  ...  }
    

    B.proto

    package <orga>/<project>/<B_packages>/<version>/<B_proto_file>
    option (nanopb_fileopt).package = "<project>_<B_package>"
    
    Import <orga>/<project>/<A_packages>/<version>/<A_proto_file>
    
    message Toto {
       <orga>.<project>.<A_packages>.<version>.Tata tata = 1;
    }
    

    The issue is that the generated code for B.proto contains a struct <project>_<B_package>_Toto that have a field tata prefixed by <project>_<B_package> and <orga>/<project>/<A_packages>/<version> Like:

    struct <project>_<B_package>_Toto {
       <project>_<B_package>_<orga>_<project>_<A_packages>_<version>_Tata tata;
    }
    

    Of course no one declare this type tata. My only solution now is to NOT now use option (nanopb_fileopt).package and keep really long names....

    The first part of the issue can be reproduced easily by adding the following into examples/cmake_relpath/proto/simple.proto

    import "nanopb.proto";
    option (nanopb_fileopt).package = "simple";
    

    If we look at the generated simple.pb.c

    typedef struct _simple_SimpleMessage {
        int32_t lucky_number;
        simpleUnluckyNumber unlucky;
    } simple_SimpleMessage;
    

    I hope this is clear. I'm doing something wrong or is it a bug ? Thanks

  • PlatformIO include issue after scanning for other libraries

    PlatformIO include issue after scanning for other libraries

    I'm building with PlatformIO, and I've got a strange issue where after the library manager installs other libraries (in my case rBase64), the platformio_generator.py script runs twice (see #734) and for some reason when trying to build the generated files, the build command is missing -I.pio/libdeps/quail/Nanopb and so the build fails with fatal error: pb.h: No such file or directory.

    If I build again, it works fine, but if I delete rBase64/ from my libdeps, then on the next build when the library manager downloads it anew, the build fails again. This is causing CI builds to fail because they are always fresh builds.

    I've added a manual -I.pio/libdeps/quail/Nanopb to my platformio.ini, but I'm sure that's not the best way.

    Perhaps the issue is that the generator is calling env.BuildSources() before the env actually contains the library itself? I'm don't fully understand the inner workings, so I'm not sure.

    (cc @valeros)

  • [Bazel] cc_nanopb_proto_library() --library-include-format incorrect when providing additional options.

    [Bazel] cc_nanopb_proto_library() --library-include-format incorrect when providing additional options.

    When passing options to cc_nanopb_proto_library (e.g. via extra_protoc_args), the library include format option below is interpreted incorrectly. The single quotes are taken as part of the format template.

    https://github.com/nanopb/nanopb/blob/404b2f072d46d53c12b87818b86b34ebb6ad1184/BUILD.bazel#L52

    When compiling the generated C files this error message is thrown:

    error: expected identifier or '(' before '\x622e6822'
        6 | '#include"pb.h"'
          | ^~~~~~~~~~~~~~~~
    

    This bug does not surface if there aren't additional options needs. There is a work around for this right now and that's to override the format when supplying additional args. e.g.

        extra_protoc_args = [
            "--nanopb_plugin_opt=...,
            "--nanopb_plugin_opt=-L#include\"%s\"",  # Override --library-include-format with this.
        ],
    
  • Macros *_init_zero and *_init_default may produce strange initializers in C code

    Macros *_init_zero and *_init_default may produce strange initializers in C code

    I downloaded nanopb 0.4.6 and studied nanopb/examples/network_server. After generating fileproto.pb.h, I saw these two macros there:

    #define ListFilesResponse_init_default           {false, false, {{NULL}, NULL}}
    #define ListFilesResponse_init_zero              {false, 0, {{NULL}, NULL}}
    

    These are not reasonable, as initializers for a C struct. Namely the struct to initialize is like this:

    typedef struct _ListFilesResponse { 
        bool has_path_error;
        bool path_error;
        DIR* file;
    } ListFilesResponse;
    

    So initializer should be something like {false,false,NULL}. GCC compiler tends to react to such initializers with messages like "warning: braces around scalar initializer".

  • [Bazel] cc_nanopb_proto_library() does not find .options file

    [Bazel] cc_nanopb_proto_library() does not find .options file

    cc_nanopb_proto_library does not find the .options file for my proto (it lives in the same directory and has the same name as the .proto file). I've found that I can point it to the options file using an absolute path via extra_protoc_args, but it seems to only take an absolute path. If cc_nanopb_proto_library cannot find options files automatically, is there at least a way to pass options files via relative paths?

    What I'm doing now:

    cc_nanopb_proto_library(
        name = "myproto_nanopb",
        protos = [":myproto_proto"],
        extra_protoc_args = [
            "--nanopb_plugin_opt=-f/asolute/path/to/myproto.options",
        ],
    )
    
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Dec 9, 2022
Protocol Buffers implementation in C

Overview This is protobuf-c, a C implementation of the Google Protocol Buffers data serialization format. It includes libprotobuf-c, a pure C library

Nov 30, 2022
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Dec 3, 2022
a small protobuf implementation in C

μpb - a small protobuf implementation in C Platform Build Status macOS ubuntu μpb (often written 'upb') is a small protobuf implementation written in

Dec 2, 2022
International obfuscated contest: Small C program to minify HTML sources and generate a minified HTML output.
International obfuscated contest: Small C program to minify HTML sources and generate a minified HTML output.

HTML Minifier C International obfuscated contest: Just a small C program to minify HTML sources and generate a minified HTML output. Using $ gcc html-

Oct 27, 2022
libcluon is a small and efficient, single-file and header-only library written in modern C++ to power microservices.

libcluon Linux & OSX Build (TravisCI) Win64 Build (AppVeyor) Test Coverage Coverity Analysis CII Best Practices libcluon is a small single-file, heade

Nov 19, 2022
Macesuted's Code

Macesuted's Code 这里存放了自 2021-5-1 以来我在学习 OI 过程中于各大 OJ 所做题目的 AC 代码。 仅供个人学习参考使用,请不要直接复制这些代码以 AC 对应题目,转载请注明出处。 我的 个人博客 中包含部分题目的题解。 我在这些 OJ 上的帐号: AtCoder:

Nov 16, 2022
A fast, byte-code interpreted language

Minima Minima is a small, portable, and fast programming language written in C. The Syntax Minima's syntax is optimized for a fast byte-code translati

Aug 16, 2022
Morse code decoding library
Morse code decoding library

ggmorse Morse code decoding library ggmorse2.mp4 ggmorse0.mp4 ggmorse1.mp4 Try it out You can easily test the library using the free GGMorse applicati

Nov 20, 2022
Google Protocol Buffers tools (C code generator).

About Google Protocol Buffers tools in Python 3.6+. C source code generator. Rust source code generator ( ?? ?? ?? under construction ?? ?? ?? ). prot

Nov 29, 2022
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Dec 9, 2022
Protocol Buffers implementation in C

Overview This is protobuf-c, a C implementation of the Google Protocol Buffers data serialization format. It includes libprotobuf-c, a pure C library

Nov 30, 2022
A protocol buffers library for C

PBC PBC is a google protocol buffers library for C without code generation. Quick Example package tutorial; message Person { required string name =

Dec 4, 2022
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Dec 3, 2022
Simple and fast C library implementing a thread-safe API to manage hash-tables, linked lists, lock-free ring buffers and queues

libhl C library implementing a set of APIs to efficiently manage some basic data structures such as : hashtables, linked lists, queues, trees, ringbuf

Nov 19, 2022
CUDA Custom Buffers and example blocks
CUDA Custom Buffers and example blocks

gr-cuda CUDA Support for GNU Radio using the custom buffer changes introduced in GR 3.10. Custom buffers for CUDA-enabled hardware are provided that c

Aug 17, 2022
CodeCompactor is an open source program designed for reducing the size of your code!

CodeCompacter An exciting, new and open source program for reducing the length of your code! Usage: ./CodeCompacter {ARGUMENTS} Arguments: -L {languag

Nov 28, 2021
A library of type safe sets over fixed size collections of types or values, including methods for accessing, modifying, visiting and iterating over those.

cpp_enum_set A library of type safe sets over fixed size collections of types or values, including methods for accessing, modifying, visiting and iter

Jun 16, 2022