Structy is an irresponsibly dumb and simple struct serialization/deserialization library for C, Python, and vanilla JavaScript.

Structy

Structy is an irresponsibly dumb and simple struct serialization/deserialization library for C, Python, and vanilla JavaScript. You can think of it like protobuf, thrift, flatbuffers, etc. but imagine that instead of a team of engineers maintaining it, it's instead written by a single moron.

Structy was created to exchange data between C-based firmware on embedded devices and Python- and JavaScript-based programming, test, and calibration scripts running on a big-girl computer. As such, it's C implementation is designed specifically for embedded devices: it doesn't do any dynamic allocation and it doesn't have any fancy code for optimizations.

Structy's design goals:

Be small, be simple, be useful

Explicit non-goals:

Be fast, be clever

If you want something far more thought out and comprehensive I'd suggest checking out things like Kaitai Struct.

Using structy

Like protobuf and other complicated data exchange libraries, structy uses a schema. Structy's schemas use Python's syntax (so it can be lazy and re-use Python's parser):

class UserSettings:
    brightness : uint8 = 127
    dark_mode: bool = False
    user_id : uint32

With this nonsense you can run structy's generator to generate C, JavaScript, and Python code for this "struct".

$ python3 -m pip install structy
$ structy user_settings.schema --c generated

For C, you can import and use this struct just like a normal struct:

#import "generated/user_settings.h"

struct UserSettings settings = {
    .brightness = 127,
    .dark_mode = true,
    .user_id = 6,
};

But it also creates the initialization (UserSettings_init), serialization (UserSettings_pack), and deserialization (UserSettings_unpack) functions needed for the struct.

C implementation

Structy's C implementation is written specifically with microcontrollers in mind. Notably:

  • The runtime is tiny, coming in at right at ~250 lines of code and compiles to less than 1kB of thumb code.
  • Never uses the heap
  • Uses very very little stack space
  • Compiles cleanly with -Werror -Wall -Wextra -Wpedantic -std=c17
  • It includes optional support for libfixmath's fix16_t.

Including Structy's runtime

You must include the files in runtimes/c in your project to use structy-generated c code.

Using generated code

The Structy generates the struct definition and four functions for use with the struct:

Initialization

void StructName_init(struct StructName* inst);

Initializes an instance of the struct with the default values specified in the struct's schema.

Pack

struct StructyResult StructName_pack(const struct StructName* inst, uint8_t* buf);

Packs an instance of the struct into the given buffer. The buffer must have at least STRUCTNAME_PACKED_SIZE bytes. Example:

uint8_t output_buffer[STRUCTNAME_PACKED_SIZE];
StructName_pack(&instance, output_buffer);

Unpack

struct StructyResult StructName_unpack(struct StructName* inst, const uint8_t* buf);

Unpacks the buffer into the given instance. The buffer must have at least STRUCTNAME_PACKED_SIZE bytes.

Print

void StructName_print(const struct StructName* inst);

Prints out the struct in a nicely formatted way, for example:

struct UserSettings @ 0x0B00B135
- brightness: 127
- dark_mode: 1
- user_id: 6

Since the runtime is designed for embedded systems, Structy doesn't hardcode printf here. You can override the print function used by creating a structy_config.h file and setting the STRUCTY_PRINTF macro:

#include "super_cool_printf.h"

#define STRUCTY_PRINTF(...) super_cool_printf(__VA_ARGS__)

By default, Structy will try to use 's printf if you're on a big girl computer. If you're on a 32-bit ARM system, Structy will check and see if you have mpland's embedded-friendly printf and use that if you have it. Otherwise, it'll disable printing.

Python implementation

Structy's Python implementation is written to be simple, not fast or "powerful" or whatever. Some notes:

  • The runtime depends on Python 3.7+
  • The runtime has complete type annotations.
  • The runtime supports converting floats between libfixmath's fix16 during packing & unpacking.
  • It's built on top of Python's excellent struct module.

Including Structy's runtime

The runtime can be installed by installing the structy package:

$ python3 -m pip install structy

Using generated code

The Structy generates the struct as a dataclass in its own module.

Initialization

# Default values.
inst = StructName()
# Override default value
inst = StructName(field_name=something)

Since it's a dataclass, an empty constructor gives the default specified in the .structy definition file, but you can pass keyword arguments to override them.

Pack

result: bytes = inst.pack()

Packs an instance and returns a bytes object with the data. len(result) will be StructName.PACKED_SIZE.

Unpack

inst = StructName.unpack(data)

Unpacks the bytes-like buffer into a new instance. The buffer must be at least StructName.PACKED_SIZE long.

Print

Since a Structy Struct is just a dataclass, it uses the dataclass __str__ and __repr__.

JavaScript implementation

Structy's JavaScript implementation, like the others, is intended to be simple. Here's some notes on it:

Including Structy's runtime

The runtime is a single file located at runtimes/js/structy.js. There's no Node/npm package because the last time I used npm my nose suddenly starting bleeding and I passed out and woke up with several mysterious lesions, and the last time I used Node.js I managed to somehow unleash a 10,000 year-old minor demon who's currently causing minor chaos by removing stop signs in low-traffic rural areas.

So just copy it into your project next to wherever you're going to place the generated code, like we did when the web was young. If you're lazy (and you probably are), just run this:

$ wget https://raw.githubusercontent.com/theacodes/structy/runtimes/js/structy.js
$ wget https://raw.githubusercontent.com/theacodes/structy/third_party/struct.js/struct.mjs

Those commands also copy in the one dependency: the excellent and tiny struct.js module- yes, I know it's confusing to have structy.js and stuct.js, but get over it. struct.js implements Python struct module in JavaScript which is great because I can write less code.

Using generated code

The Structy generates the struct as a nice class in its own module. The module's only export is the class, so you can import it like this:

<script type="module">
import StructName from "./structname.js";
script>

Initialization

// Default values.
inst = new StructName();
// Override a default value by passing in an object
inst = new StructName({field_name: something});

It's a JavaScript class so just use new to make an instance. It uses the pattern of passing in an object with field values so you can update fields without needing to specify all of them.

Pack

result = inst.pack();

Packs an instance and returns a Uint8Array object with the data. result.length() will be StructName.packed_size.

Unpack

inst = StructName.unpack(data);

Unpacks the Uint8Array or ArrayBuffer into a new instance. The buffer must be at least StructName.packed_size long.

FAQ

Does Structy support arrays/lists?

No, Struct structs must be a deterministic length when encoded.

How about bitfields?

No yet, but it could. I'm just too lazy.

Little-endian? Mixed-endianess?

No. Supporting anything other than big-endian would complicate the C runtime and I don't want to do that. Idk, maybe I could be talked into accepting a PR for it.

Custom types?

No, but it's possible in the future. There's a little bit of this thought out because Struct supports Q16.16 fixed-point values.

Nested structs?

No, but it could be added. I just haven't needed it yet.

Why does structy use snake_case for properties and methods even in C and JS where it's common to use lowerCamelCase?

Mostly because I want data access to be identical in all languages, but also because I deeply, deeply, hate lowerCamelCase. I find it hard to read.

Support / issues / etc

Unlike some of my other open-source projects, this project was made with one user in mind: me. I made it open source in case someone else finds it useful but I am not in the business of supporting this project. If you run into issues or need help feel free to submit an issue on GitHub, however, please know that I do not feel any obligation to support this project.

License

The Structy generator and runtime components are all published under the MIT license. See LICENSE for the full text.

Owner
Stargirl Flowers
Open-source advocate ✨ Synth crafter ✨ PSF Fellow ✨ She/her
Stargirl Flowers
Similar Resources

Libraries and examples to support Pimoroni Pico add-ons in C++ and MicroPython.

Pimoroni Pico Libraries and Examples Welcome to the brave new world of Pico! This repository contains the C/C++ and MicroPython libraries for our rang

Nov 24, 2022

Tools and libraries to glue C/C++ APIs to high-level languages

CppSharp is a tool and set of libraries which facilitates the usage of native C/C++ code with the .NET ecosystem. It consumes C/C++ header and library

Nov 23, 2022

A tool for generating cross-language type declarations and interface bindings.

Djinni Djinni is a tool for generating cross-language type declarations and interface bindings. It's designed to connect C++ with either Java or Objec

Nov 19, 2022

The missing bridge between Java and native C++

JavaCPP Commercial support: Introduction JavaCPP provides efficient access to native C++ inside Java, not unlike the way some C/C++ compilers interact

Nov 26, 2022

SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages.

SWIG (Simplified Wrapper and Interface Generator) Version: 4.1.0 (in progress) Tagline: SWIG is a compiler that integrates C and C++ with languages

Nov 25, 2022

A minimalist and mundane scripting language.

Drift Script A minimalist and mundane scripting language. I like all simple things, simple and beautiful, simple and strong. I know that all developme

Nov 14, 2022

CppSerdes is a serialization/deserialization library designed with embedded systems in mind

CppSerdes is a serialization/deserialization library designed with embedded systems in mind

A C++ serialization/deserialization library designed with embedded systems in mind

Nov 5, 2022

Fast Binary Encoding is ultra fast and universal serialization solution for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift

Fast Binary Encoding (FBE) Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server request

Nov 21, 2022

matrix-effect This is a dumb matrix effect type thing

matrix-effect This is a dumb matrix effect type thing

matrix-effect This is a dumb matrix effect type thing. It's only like one source file which should compile... Define __POSIX or __WIN though, for posi

Sep 23, 2022

Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and deploy without Python.

Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and deploy without Python.

Python Inference Script(PyIS) Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and d

Nov 4, 2022

An Arduino library with additions to vanilla Serial.print(). Chainable methods and verbosity levels. Suitable for debug messages.

advancedSerial This library provides some additions to vanilla Serial.print(): 1. Chainable print() and println() methods: // you can chain print() a

Oct 7, 2022

Freelancer: HD Edition is a mod that aims to improve every visual, aural, and gameplay aspect of the game Freelancer (2003) while keeping the look and feel as close to vanilla as possible.

Freelancer: HD Edition Freelancer: HD Edition is a mod that aims to improve every visual, aural, and gameplay aspect of the game Freelancer (2003) whi

Nov 17, 2022

Speed Running and Competition Doom. For strictly vanilla speed runs and competitions - forked from CNDoom

Speed Running and Competition Doom Speed Running and Competition Doom is based on Chocolate Doom and aims to accurately reproduce the original DOS ver

May 24, 2022

1.12.1 (5875) Vanilla Emulator (Core & DB)

RaspyWoW - Vanilla Welcome to the new generation of interconnected WoW realms — all hosted on Raspberry PI. Raspy WoW aims to create an efficient, che

Aug 16, 2022

is a c++20 compile and runtime Struct Reflections header only library.

is a c++20 compile and runtime Struct Reflections header only library. It allows you to iterate over aggregate type's member variables.

Apr 18, 2022

A miniature library for struct-field reflection in C++

visit_struct A header-only library providing structure visitors for C++11 and C++14. Motivation In C++ there is no built-in way to iterate over the me

Nov 25, 2022

A miniature library for struct-field reflection in C++

visit_struct A header-only library providing structure visitors for C++11 and C++14. Motivation In C++ there is no built-in way to iterate over the me

Nov 17, 2022

🎵 Music notation engraving library for MEI with MusicXML and Humdrum support and various toolkits (JavaScript, Python)

🎵 Music notation engraving library for MEI with MusicXML and Humdrum support and various toolkits (JavaScript, Python)

Verovio is a fast, portable and lightweight library for engraving Music Encoding Initiative (MEI) digital scores into SVG images. Verovio also contain

Nov 18, 2022

Source code and raw content for NEON STRUCT: Desperation Column

NEON STRUCT: Desperation Column copyright © 2021 Minor Key Games, LLC. Source code and raw content for NEON STRUCT: Desperation Column are release

Aug 11, 2022
Comments
  • Fix tty detection

    Fix tty detection

    os.isatty() takes a file descriptor argument which was not being provided here. This raised a TypeError exception on my installation as it expected one argument.

Related tags
Duktape - embeddable Javascript engine with a focus on portability and compact footprint

Duktape ⚠️ Master branch is undergoing incompatible changes for Duktape 3.x. To track Duktape 2.x, follow the v2-maintenance branch. Introduction Dukt

Nov 16, 2022
Embedded JavaScript engine for C/C++

V7: Embedded JavaScript engine NOTE: this project is deprecated in favor of https://github.com/cesanta/mjs V7 is the smallest JavaScript engine writte

Nov 20, 2022
ChakraCore is an open source Javascript engine with a C API.

ChakraCore ChakraCore is a Javascript engine with a C API you can use to add support for Javascript to any C or C compatible project. It can be compil

Nov 19, 2022
Seamless operability between C++11 and Python
Seamless operability between C++11 and Python

pybind11 — Seamless operability between C++11 and Python Setuptools example • Scikit-build example • CMake example Warning Combining older versions of

Nov 25, 2022
Import C++ files directly from Python!

If you've used cppimport version 0.0.*, some new features for you! Compiler arguments, multiple source files, bug fixes! Read on. Import C or C++ file

Nov 16, 2022
The most widely used Python to C compiler

Welcome to Cython! Cython is a language that makes writing C extensions for Python as easy as Python itself. Cython is based on Pyrex, but supports mo

Nov 16, 2022
A portable foreign-function interface library.

Status libffi-3.4 was released on TBD. Check the libffi web page for updates: URL:http://sourceware.org/libffi/. What is libffi? Compilers for high le

Nov 19, 2022
A lightweight, dependency-free library for binding Lua to C++
A lightweight, dependency-free library for binding Lua to C++

LuaBridge 2.6 LuaBridge is a lightweight and dependency-free library for mapping data, functions, and classes back and forth between C++ and Lua (a po

Nov 16, 2022
Library to build PHP extensions with C++

PHP-CPP The PHP-CPP library is a C++ library for developing PHP extensions. It offers a collection of well documented and easy-to-use classes that can

Nov 16, 2022
Sol3 (sol2 v3.0) - a C++ <-> Lua API wrapper with advanced features and top notch performance - is here, and it's great! Documentation:

sol2 sol2 is a C++ library binding to Lua. It currently supports all Lua versions 5.1+ (LuaJIT 2.0+ and MoonJIT included). sol2 aims to be easy to use

Nov 25, 2022