jluna: A modern julia ⭤ C++ Wrapper

jluna: A modern julia ⭤ C++ Wrapper (v0.5)

Julia is a beautiful language, it is well-designed and well-documented. Julias C-API is also well-designed, less beautiful and much less... documented.

Heavily inspired in design and syntax by (but in no way affiliated with) the excellent Lua⭤C++ wrapper sol2, jluna aims to fully wrap the official julia C-API and replace it in usage in C++ projects, by making accessing julias unique strengths through C++ safe, hassle-free and just as beautiful.


Table of Contents

  1. Introduction
  2. Showcase
  3. Features
  4. Planned Features
  5. Documentation
    4.1 🔗 Manual
    4.2 🔗 Installation
    4.3 🔗 Troubleshooting
  6. Dependencies
    5.1 julia 1.7.0+
    5.2 g++10
    5.3 cmake 3.19+
    5.4 Linux / Mac OS
  7. License

Showcase

Access julia-Side Values/Functions

// execute arbitrary strings with exception forwarding
State::safe_script(R"(
    f(x) = x*x*x
    
    mutable struct MyStruct
        _field
        MyStruct() = new(123)
    end

    instance = MyStruct();
)");

// access and modify variables
Main["instance"]["_field"] = 456;
State::script(R"(println("instance._field is now: ", instance._field))");

// call julia-side functions with C++-side values
int result = Main["f"](12);
Base["println"](result);
instance._field is now: 456
1728

Multi-Dimensional Array Interface

cpp_array = Main["array"]; // julia style list indexing auto sub_array = cpp_array[{6, 5, 4, 3, 2}]; Base["println"]((Any*) sub_array); // iterable and assignable for (auto e : cpp_array) e = e.operator size_t() + 10; State::script("println(array)");">
State::script("array = collect(1:9)");
Array<size_t, 1> cpp_array = Main["array"];

// julia style list indexing
auto sub_array = cpp_array[{6, 5, 4, 3, 2}];
Base["println"]((Any*) sub_array);

// iterable and assignable
for (auto e : cpp_array)
    e = e.operator size_t() + 10;

State::script("println(array)");
[7, 6, 5, 4, 3]
[11, 12, 13, 14, 15, 16, 17, 18, 19]

Call C++ Functions from julia

Any* { auto as_string = unbox (x); std::cout << "cpp prints " << as_string << " and returns: " << std::endl; auto as_set = unbox >(y); size_t out = 0; for (size_t x : as_set) out += x; return box(out); }; // now callable from julia State::safe_script(R"( println(Main.lambda("what julia handed it", Set([1, 2, 3, 3, 4]))) # non-c-types work! )");">
/// register lambda and bind to julia-side variable
State::new_named_undef("lambda") = [](Any* x, Any* y) -> Any*
{
    auto as_string = unbox
      (x);
    std::cout << 
      "cpp prints " << as_string << 
      " and returns: " << std::endl;
    
      auto as_set = unbox
      
       <
       size_t>>(y);

    
       size_t out = 
       0;
    
       for (
       size_t x : as_set)
        out += x;

    
       return 
       box(out);
};


       // now callable from julia

       State::safe_script(
       R"(

           println(Main.lambda("what julia handed it", Set([1, 2, 3, 3, 4])))  # non-c-types work!

       )");
      
     
cpp prints what julia handed it and returns: 
10

Features

Some of the many advantages jluna has over the C-API include:

  • expressive generic syntax
  • call C++ functions from julia using any julia-type
  • assigning C++-side proxies also mutates the corresponding variable with the same name julia-side
  • julia-side values, including temporaries, are kept safe from the garbage collector while they are in use C++-side
  • verbose exception forwarding from julia, compile-time assertions
  • wraps most of the relevant C++ std objects and types
  • multidimensional, iterable array interface with julia-style indexing
  • manual written by a human for beginners
  • inline documentation for IDEs for both C++ and julia code
  • freely mix jluna and the C-API
  • And more!

Planned (but not yet implemented):

  • v0.6-0.7: expression proxy, access to meta features via C++ including C-API-only introspection
  • v0.7-0.8: linear algebra, matrices
  • v0.8-0.9: thread-safety, parallelization
  • v0.9-1.0: 0-overhead performance version of proxies and cppcall
  • v1.0+: multiple julia worlds, save-states: restoring a previous julia state

Documentation

A step-by-step introduction and reference guide intended is available here. Furthermore, all user-facing code has in-line documentation available through most IDEs (or the julia help? command).

Advanced users are encouraged to check the headers (available in jluna/include/) for implementation details. They are formatted specifically to be easily understood by 3rd parties.


Dependencies

jluna aims to be as modern as is practical. It uses C++20 features extensively and aims to support the newest julia version, rather than focusing on backwards compatibility. If you are looking for a C++ library that supports julia 1.5 or lower, consider checking out CxxWrap instead.

For jluna you'll need:

Currently, only g++10 and g++11 are supported, clang support is planned in the future.


Installation & Troubleshooting

A step-by-step tutorial on how to create, compile and link a new C++ Project with jluna can be found here. It is recommended that you follow this guide closely instead of trying to resolve issues on your own.

For Advanced Users Only

Users familiar with C++ and cmake can go through the following steps:

Install:

  • g++-11
  • julia 1.7+
  • cmake 3.16+

Then execute (in the same directory as your CMakeLists.txt):

git clone https://github.com/Clemapfel/jluna.git

export JULIA_PATH=$(julia -e "println(joinpath(Sys.BINDIR, \"..\"))")

mkdir build
cd build
cmake -D CMAKE_CXX_COMPILER=g++-11 ..
make

./JLUNA_TEST

cd ..
rm -r build

Where JULIA_PATH needs to be set at the time of compilation.

Link against jluna/libjluna.so, jluna/libjluna_c_adapter.so and $ENV{JULIA_PATH}/lib/libjulia.so.

Add "${CMAKE_SOURCE_DIR}/jluna" and "$ENV{JULIA_PATH}/include/julia" to your include directories.

Then you can make jluna available to your library using:

#include <julia.h>
#include <jluna.hpp>

If errors appear at any point, head to the step-by-step guide.


License

jluna is supplied under Open Software License 3.0, available here. For collaboration or further questions, feel free to contact the developer.


Owner
Clem Cords
using template meta magic to make C++ syntax pretty since g++-10 released
Clem Cords
Comments
  • CMake and Windows improvements

    CMake and Windows improvements

    This PR makes the CMake build scripts better organized and uses a find module to import Julia. The find module and the imported target it produces are not present in the install interface and requires consumers to do the work themselves. This is because Julia doesn't provide a CMake package.

    There are some fixes for Windows as well and a workaround for bugs in Julia's headers. They include headers directly that are not intended to be included standalone.

    The test and benchmark code are too broken on Windows, so they are disabled there for now.

  • ctest --verbose fails when built with clang++-14

    ctest --verbose fails when built with clang++-14

    Note: This issue occurs for me when compiling with clang++-14, but not with g++-11.

    Working my way through the install instructions, however, ctest --verbose fails after make install.

    The test seems to segfault during unsafe: resize_array: reshape at unsafe::resize_array(arr, 5, 5);

    signal (11): Segmentation fault
    in expression starting at none:0
    ...
    

    "Segmentation fault in expression starting at none:0" are mentioned in the troubleshooting guide, but specifically in a multithreading context.

    Additional details: Ubuntu 20.04 Using -DCMAKE_CXX_COMPILER=clang++-14 (Note: ctests pass when built with g++-11)

    Full ctest --verbose output:

    UpdateCTestConfiguration  from :/home/frivold/Code/jluna/build/DartConfiguration.tcl
    Parse Config file:/home/frivold/Code/jluna/build/DartConfiguration.tcl
    UpdateCTestConfiguration  from :/home/frivold/Code/jluna/build/DartConfiguration.tcl
    Parse Config file:/home/frivold/Code/jluna/build/DartConfiguration.tcl
    Test project /home/frivold/Code/jluna/build
    Constructing a list of tests
    Done constructing a list of tests
    Updating test list for fixtures
    Added 0 tests to meet fixture requirements
    Checking test dependency graph...
    Checking test dependency graph end
    test 1
        Start 1: jluna_test
    
    1: Test command: /home/frivold/Code/jluna/build/jluna_test
    1: Test timeout computed to be: 1500
    1: [JULIA][LOG] initialization successful (1 thread(s)).
    1: starting test...
    1:
    1: c_adapter found: [OK]
    1: unsafe: gc_push / gc_pop: [OK]
    1: unsafe: gc: [OK]
    1: unsafe: _sym: [OK]
    1: as_julia_pointer: [OK]
    1: unsafe: get_function: [OK]
    1: unsafe: Expr & eval: [OK]
    1: unsafe: get/set value: [OK]
    1: unsafe: get_field: [OK]
    1: unsafe: set_field: [OK]
    1: unsafe: call: [OK]
    1: unsafe: new_array: [OK]
    1: unsafe: new_array_from_data: [OK]
    1: unsafe: override_array: [OK]
    1: unsafe: swap_array_data: [OK]
    1: unsafe: set_array_data: [OK]
    1:
    1: signal (11): Segmentation fault
    1: in expression starting at none:0
    1: set_nth_field at /buildworker/worker/package_linux64/build/src/datatype.c:1498 [inlined]
    1: jl_new_struct at /buildworker/worker/package_linux64/build/src/datatype.c:1251
    1: _ZN5jluna6unsafe12resize_arrayEP10jl_array_tmm at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    1: _ZZ4mainENK4$_19clEv at /home/frivold/Code/jluna/build/jluna_test (unknown line)
    1: _ZN5jluna6detail4Test4testIZ4mainE4$_19EEvNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEOT_ at /home/frivold/Code/jluna/build/jluna_test (unknown line)
    1: main at /home/frivold/Code/jluna/build/jluna_test (unknown line)
    1: __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
    1: _start at /home/frivold/Code/jluna/build/jluna_test (unknown line)
    1: Allocations: 1724877 (Pool: 1723947; Big: 930); GC: 3
    1/1 Test #1: jluna_test .......................***Exception: SegFault  0.92 sec
    
    0% tests passed, 1 tests failed out of 1
    
    Total Test time (real) =   0.92 sec
    
    The following tests FAILED:
              1 - jluna_test (SEGFAULT)
    Errors while running CTest
    Output from these tests are in: /home/frivold/Code/jluna/build/Testing/Temporary/LastTest.log
    Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
    
  • various fixes for cmake based find_package

    various fixes for cmake based find_package

    @Clemapfel Thanks for your great work on Jluna! I've been waiting for a package like this. Here are a few changes that I made locally that made it easier to use for my uses cases. Please feel free to merge if these are useful for you.

    • modified #include directives so that they work outside of installed as a subtree of the project

    • modified how jluna.jl is found so that they can be found if they are installed or just simply built, by default, it prefers the local version from the installed versions. I would have loved to use c++ std::format here rather than snprintf, but since the supported compilers don't support it I can't use it here.

    • modified how libjluna_c_adapter.so is found to use the dynamic linker instead of explicit path passing which improves relocatablity

    • rather than always forcing a Release build, allow the user to set it to each of the CMake defaults

    • include CTest to make compiling the unit tests optional in the standard way

    • include GNUInstallDirs which was not provided before

    • use find_library() to locate libjulia.so to make it more robust to different OS layouts

    • use find_path() to locate julia.h to make it more robust to different OS layouts

    • set the target_include_directories and target_link_libraries correctly so that downstream consumers can simply

      find_package(jluna)
      target_link_libraries(foo PRIVATE jluna::jluna)
      

      and everything works

    • simplified the logic to set the library name. Note that using global in the module is safe because globals in julia are local to the module they are defined in

  • Usertype best practices?

    Usertype best practices?

    Hey clem, haven't been working with Jluna for a few weeks, but back at it.

    Got a couple of quick question about usertypes best practices.

    1. How should I be setting up a jluna usertype for something I already have a Julia side type for? Should I be making an equivalent type through the jluna usertype system and doing conversions Julia side (or using multiple dispatch when possible)? Or could I somehow link my jluna usertype to the existing Julia side struct?

    2. Can I use a usertype in another usertype? Is there anything I need to do to make that possible? Currently getting the following error when trying to build the example below: In template: no member named 'type_name' in 'jluna::detail::as_julia_type_aux<JlunaStamp>'

    #ifndef JLUNA_WRAPPER_JLUNA_STAMP_H
    #define JLUNA_WRAPPER_JLUNA_STAMP_H
    #include <jluna.hpp>
    using namespace jluna;
    
    struct JlunaStamp {
        Int32 sec;
        UInt32 nanosec;
    
        JlunaStamp() {}
        JlunaStamp(Int32 sec, UInt32 nanosec)
        : sec(sec), nanosec(nanosec) {}
    
        static void addProperties(jluna::Module m = jl_main_module);
    };
    
    set_usertype_enabled(JlunaStamp);
    
    #endif //JLUNA_WRAPPER_JLUNA_STAMP_H
    
    -------------------
    
    #include <jluna_wrapper/types/jluna_stamp.h>
    
    void JlunaStamp::addProperties(jluna::Module m) {
        jluna::Usertype<JlunaStamp>::add_property<Int32>(
                "sec",
                [](JlunaStamp& in) -> Int32 {return in.sec;},
                [](JlunaStamp& out, Int32 in) -> void {out.sec = in;}
        );
        jluna::Usertype<JlunaStamp>::add_property<UInt32>(
                "nanosec",
                [](JlunaStamp& in) -> UInt32 {return in.nanosec;},
                [](JlunaStamp& out, UInt32 in) -> void {out.nanosec = in;}
        );
    
        Usertype<JlunaStamp>::implement(m);
    }
    
    #ifndef JLUNA_WRAPPER_JLUNA_HEADER_H
    #define JLUNA_WRAPPER_JLUNA_HEADER_H
    #include <jluna.hpp>
    #include <jluna_wrapper/types/jluna_stamp.h>
    using namespace jluna;
    
    struct JlunaHeader {
        JlunaStamp stamp;
        std::string frame_id;
    
        JlunaHeader() {}
        JlunaHeader(JlunaStamp stamp, std::string frame_id)
        : stamp(stamp), frame_id(frame_id) {}
    
        static void addProperties(jluna::Module m = jl_main_module);
    };
    
    set_usertype_enabled(JlunaHeader);
    
    #endif //JLUNA_WRAPPER_JLUNA_HEADER_H
    
    ---------------------------
    
    #include <jluna_wrapper/types/jluna_header.h>
    
    void JlunaHeader::addProperties(jluna::Module m) {
        jluna::Usertype<JlunaHeader>::add_property<JlunaStamp>(
                "stamp",
                [](JlunaHeader& in) -> JlunaStamp {return in.stamp;},
                [](JlunaHeader& out, JlunaStamp in) -> void {out.stamp = in;}
        );
        jluna::Usertype<JlunaHeader>::add_property<std::string>(
                "frame_id",
                [](JlunaHeader& in) -> std::string {return in.frame_id;},
                [](JlunaHeader& out, std::string in) -> void {out.frame_id = in;}
        );
    
    
        Usertype<JlunaHeader>::implement(m);
    }
    
        jluna::initialize();
        jluna::Module m = jluna::Main.safe_eval("return jluna_wrapper");
        JlunaStamp::addProperties(m);
        JlunaHeader::addProperties(m);
    
  • Segfault when original Task object goes out of scope

    Segfault when original Task object goes out of scope

    Hey Clem, sorry to bother you with another issue. Not sure if I'm missing something basic here, but struggling to get multithreading working.

    Quick stats before getting into things:

    • Ubuntu 20.04
    • Julia 1.7.1
    • Same behavior with gcc-11 & clang-14
    • On latest master ce9a1f5
    • ctests --verbose still pass w/ gcc-11 & still fail w/ clang-14
    • unsure if relevant at all, but still happens when preceeded by unsafe::gc_disable()

    Ok, so I've been working my way through the multithreading docs, and things were great until I started trying to manage Task lifetimes.

    Tasks seem to terminate the moment the original task object goes out of scope, even when it's added to an std::vector which is still in scope. If I create a task with auto task = ThreadPool::create(f) then add task to a std::vector, the task terminates when task goes out of scope, despite the vector still being in scope. Even more immediate, when I directly add the task to the std::vector with tasks.push_back(ThreadPool::create(f)) the program segfaults the moment I attempt tasks.back().schedule().

    Not sure if that explanation made any sense, here are a few examples to hopefully illustrate what I'm seeing.

    Basic example

    using namespace jluna;
    
    int main() {
        initialize(8);
    
        std::function<void()> f = []() -> void {
            std::cout << "This is Thread: " << ThreadPool::thread_id << std::endl;
        };
    
        // naive spawn task (this works fine)
        auto task = ThreadPool::create(f);
        task.schedule();
        task.join();
    
        // vector for managing task lifetimes
        std::vector<Task<void>> tasks;
    
        // spawn task into vector
        tasks.push_back(ThreadPool::create(f));
        tasks.back().schedule();
        tasks.back().join();
    }
    
    [JULIA][LOG] initialization successful (8 thread(s)).
    This is Thread: 1
    
    signal (11): Segmentation fault
    in expression starting at none:0
    unknown function (ip: 0x7fd2e8479e40)
    operator() at /home/frivold/Code/jluna/install/include/jluna/.src/multi_threading.inl:252
    __invoke_impl<_jl_value_t*, jluna::ThreadPool::create<>(const std::function<void()>&)::<lambda()>&> at /usr/include/c++/11/bits/invoke.h:61
    __invoke_r<_jl_value_t*, jluna::ThreadPool::create<>(const std::function<void()>&)::<lambda()>&> at /usr/include/c++/11/bits/invoke.h:114
    _M_invoke at /usr/include/c++/11/bits/std_function.h:291
    _ZNKSt8functionIFP11_jl_value_tvEEclEv at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    jluna_invoke_from_task at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    #3 at ./none:813
    unknown function (ip: 0x7fd2eb6ef89f)
    _jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
    jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
    jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
    start_task at /buildworker/worker/package_linux64/build/src/task.c:877
    Allocations: 1626159 (Pool: 1625207; Big: 952); GC: 1
    
    Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
    

    Example with scope block

    #include <jluna.hpp>
    
    using namespace jluna;
    
    int main() {
        initialize(8);
        auto println_jl = Main.safe_eval("return println");
        auto sleep_jl = Main.safe_eval("return sleep");
    
        std::function<void()> five_mississippi = [&]() -> void {
            for (int i=1; i<=5; i++) {
                println_jl(i, " Mississippi");
                sleep_jl(1);
            }
            println_jl("Done");
        };
    
        // vector for managing task lifetimes
        std::vector<Task<void>> tasks;
    
        // scope block
        {
            auto task = ThreadPool::create(five_mississippi);
            tasks.push_back(task);
            tasks.back().schedule();
            sleep_jl(2);
        }
        tasks.back().join();
    }
    
    [JULIA][LOG] initialization successful (8 thread(s)).
    1 Mississippi
    2 Mississippi
    
    signal (11): Segmentation fault
    in expression starting at none:0
    jl_get_nth_field at /buildworker/worker/package_linux64/build/src/datatype.c:1392
    _ZNK5jluna5Proxy10ProxyValue5valueEv at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    safe_call<int&, char const (&)[13]> at /home/frivold/Code/jluna/install/include/jluna/.src/proxy.inl:93
    operator()<int&, char const (&)[13]> at /home/frivold/Code/jluna/install/include/jluna/.src/proxy.inl:115
    operator() at /home/frivold/kef_env/kef_ws/src/jluna_wrapper/src/gcc_test.cpp:12
    __invoke_impl<void, main()::<lambda()>&> at /usr/include/c++/11/bits/invoke.h:61
    __invoke_r<void, main()::<lambda()>&> at /usr/include/c++/11/bits/invoke.h:111
    _M_invoke at /usr/include/c++/11/bits/std_function.h:291
    operator() at /usr/include/c++/11/bits/std_function.h:560
    operator() at /home/frivold/Code/jluna/install/include/jluna/.src/multi_threading.inl:252
    __invoke_impl<_jl_value_t*, jluna::ThreadPool::create<>(const std::function<void()>&)::<lambda()>&> at /usr/include/c++/11/bits/invoke.h:61
    __invoke_r<_jl_value_t*, jluna::ThreadPool::create<>(const std::function<void()>&)::<lambda()>&> at /usr/include/c++/11/bits/invoke.h:114
    _M_invoke at /usr/include/c++/11/bits/std_function.h:291
    _ZNKSt8functionIFP11_jl_value_tvEEclEv at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    jluna_invoke_from_task at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    #3 at ./none:813
    unknown function (ip: 0x7f9540090b9f)
    _jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
    jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
    jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
    start_task at /buildworker/worker/package_linux64/build/src/task.c:877
    Allocations: 1625914 (Pool: 1624963; Big: 951); GC: 1
    
    Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
    

    Example from multithreading docs

    int main() {
        using namespace jluna;
        using namespace std::chrono_literals;
    
        /// in main.cpp
        jluna::initialize(8);
    
        // task storage
        std::vector<Task<void>> tasks;
    
        {
            // declare lambda
            std::function<void()> print_numbers = []() -> void
            {
                for (size_t i = 0; i < 10000; ++i)
                    std::cout << i << std::endl;
            };
    
            // add task to storage
            tasks.push_back(ThreadPool::create(print_numbers));
    
            // start just pushed task
            tasks.back().schedule();
    
            // wait for 1ms
            std::this_thread::sleep_for(1ms);
        }
    
        // wait for another 10ms
        std::this_thread::sleep_for(10ms);
        return 0;
    }
    
    [JULIA][LOG] initialization successful (8 thread(s)).
    
    signal (11): Segmentation fault
    in expression starting at none:0
    unknown function (ip: 0x55cf03eabce0)
    jluna_invoke_from_task at /home/frivold/Code/jluna/install/libjluna.so.0.9.1 (unknown line)
    #3 at ./none:813
    unknown function (ip: 0x7f0ac586291f)
    _jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
    jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
    jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
    start_task at /buildworker/worker/package_linux64/build/src/task.c:877
    Allocations: 1626138 (Pool: 1625189; Big: 949); GC: 1
    
  • Use Julia new foreign thread support

    Use Julia new foreign thread support

    Possibly this is not new news for you, but Julia is about to support foreign threads calling into it: https://github.com/JuliaLang/julia/pull/45447. This will probably be very helpful and simplify jluna.

  • v0.9.0 build issues Ubuntu g++10

    v0.9.0 build issues Ubuntu g++10

    I encounters some build issues with version 0.9.0 when building in Ubuntu 20.04.

    • Got an error that mutex isn't a member of namespace std and had to add a #include directive in safe_utilities.cpp
    • Got an error that reference to ThreadPool is ambiguous in a testing file and had to explicitly resolve namespace in friend class in multi_threading.inl

    I was able to install and ctest did not report any errors so I'm assuming these changes are correct :shrug: . These are small changes so I'll submit a PR quickly and let y'all determine if my errors are one-offs or not.

    OS: Ubuntu 20.04 Compiler: GCC 11 (installed via apt)

  • Merge CMake Rework

    Merge CMake Rework

    Adds improved cmake functionality, along with one-line install init.sh and updated documentation. Furthermore fixed dependency of libjluna_c_adapter and jluna.jl location.

  • jl_set_global going away

    jl_set_global going away

    Jluna uses a function jl_set_global which seems to be going away in an upcoming version https://docs.julialang.org/en/v1.9-dev/manual/embedding/. I wasn't sure what the easy work around was otherwise I would have created a PR to adapt to this change.

  • C-Adapter shared library may not be installed in exactly `CMAKE_INSTALL_PREFIX` in VS2022

    C-Adapter shared library may not be installed in exactly `CMAKE_INSTALL_PREFIX` in VS2022

    In VS 2022, despite specifying

    # in jluna/build
    cmake .. -DCMAKE_INSTALL_PREFIX=C:/path/to/folder`
    

    During cmake, after make install

    jluna_c_adapter.dll may be installed into C:/path/to/folder/Debug or C:/path/to/folder/Release instead. This causes State::initialize() to fail, as it expects jluna_c_adapter.dll to be located in C:/path/to/folder.

  • docs for simple examples, and necessity of unboxing

    docs for simple examples, and necessity of unboxing

    Dear Clem, Thanks for this package. I am very interested in calling Julia from C (or C++ if it must be). Forgive the newbie questions.

    1. docs. I struggled to find a simple "hello world" example - the "showcase" in the readme.md is too complicated for me, advertizing various advanced features that I cannot understand. I only use a bit of C++ and certainly not fancy features of C++20. As a scientific programmer I, and many others, would benefit most from a series of examples of calling simple julia code (first a command, then function, then module...) from C++ (or even from C if possible). So my request is: could you kindly simplify the first examples? (in the readme, and in the manual).

    2. unboxing. In the manual you describe that unboxing is essential. (maybe I misunderstood?). However, one needs to be able to simply pass pointers to existing arrays without forcing a copy each time, for obvious reasons (avoiding slow-down for relatively cheap functions performed on large arrays, and also for saving RAM in large problems). A year ago I set up some demos of C calling Julia, using pointers:

    https://github.com/ahbarnett/multilingual-julia/tree/main/ccallj

    These examples start simple (calling a julia base function, then a julia function, then a julia module). Eg see https://github.com/ahbarnett/multilingual-julia/blob/main/ccallj/cfuncmodarr.c which wraps multi-threaded functions in the simple julia module https://github.com/ahbarnett/multilingual-julia/blob/main/ccallj/ArrMod.jl

    They are incomplete, just a couple of days work, are not as elegant as I'd hoped (but SGJohnson helped), and nothing like the scale of your big project. (I also had trouble compiling/linking, as you will see from comments.) However, they do show what we consider essential in scientific programming --- eg, passing large arrays by pointers, accessing multithreaded high-performance julia code from a low-level langauge --- so I would be curious if/how your project can showcase some similar very simple array-passing and multithreaded examples? Maybe my simple examples could influence some of the documented examples you start with? (ties back to part 1 above). [Sadly I have not had time to use my own examples in projects yet, but plan to.]

    Thanks and best wishes, Alex

  • Add usertype-related macro, fix usertype::implement scoping

    Add usertype-related macro, fix usertype::implement scoping

    Add additional macro for implicit type conversion from a usertype-wrapped type, c.f. #35.

    Fix a typo that called usertype::implement in Main-scope, rather than the specified scope, c.f. #36.

    Thank you to @paulerikf for pointing these out.

  • Confusion about how sleeping main thread affects task execution

    Confusion about how sleeping main thread affects task execution

    The multithreading docs show examples where sleep is called on the main c++ thread, but issues seem to arise in more complicated scenarios (i.e. if any calls to julia functions are made in the task code).

    I've been trying to use jluna in a situation where I'm not fully in charge of the c++ thread. I use it to set up and schedule julia-side tasks, and then it sleeps and occasionally runs callbacks at intervals outside of my direct control.

    In attempting to get this working I've become rather confused about how sleeping the main thread affects task execution.

    Here's an example that hopefully illustrates what I'm seeing:

    initialize(2);
    
    auto lambda = [&]() {
        int i = 0;
        while(true) {
            // somehow print i
            i++;
        }
    };
    
    Task<void> t1 = ThreadPool::create<void()>(lambda);
    t1.schedule();
    
    while(true) {
        // somehow print "main_loop"
        // somehow sleep for 1s
    }
    

    Cases where sleep is done with std::this_thread::sleep_for(1000ms):

    • task and main thread use std::cout
      • task runs nonstop
    • task uses Julia println, main thread uses std::cout
      • task prints 0 then gets stuck forever
    • task and main thread use Julia println
      • task prints one number for every time main thread prints "main_thread"

    Cases where sleep is done using Julia sleep function:

    • task runs nonstop no matter how print is done.

    Extended examples:

    Task and main thread use std::cout:

    initialize(2);
    
    auto lambda = [&]() {
        int i = 0;
        while(true) {
            std::cout << i << std::endl;
            i++;
        }
    };
    
    Task<void> t1 = ThreadPool::create<void()>(lambda);
    t1.schedule();
    
    while(true) {
        std::cout << "main_loop" << std::endl;
        std::this_thread::sleep_for(1000ms);
    }
    
    ...
    3444624
    3444625
    3444626
    3444627
    3444628
    3444629
    3444630
    3444630
    3444631
    3444632
    3444632
    3444633
    ...
    

    Task uses jl println, main thread uses std::cout

        initialize(2);
        auto println_jl = Main.safe_eval("return println");
        
        auto lambda = [&]() {
            int i = 0;
            while(true) {
                println_jl.safe_call<void>(i);
                i++;
            }
        };
    
        Task<void> t1 = ThreadPool::create<void()>(lambda);
        t1.schedule();
    
        while(true) {
            std::cout << "main_loop\n" << std::endl;
            std::this_thread::sleep_for(1000ms);
        }
    
    [JULIA][LOG] initialization successful (4 thread(s)).
    main_loop
    
    0main_loop
    
    main_loop
    
    main_loop
    
    main_loop
    
    ...
    

    Both task and main thread us jl println

        initialize(2);
        auto println_jl = Main.safe_eval("return println");
        
        auto lambda = [&]() {
            int i = 0;
            while(true) {
                println_jl.safe_call<void>(i);
                i++;
            }
        };
    
        Task<void> t1 = ThreadPool::create<void()>(lambda);
        t1.schedule();
    
        while(true) {
            println_jl.safe_call<void>("main_loop");
            std::this_thread::sleep_for(1000ms);
        }
    
    [JULIA][LOG] initialization successful (4 thread(s)).
    main_loop
    0
    1
    main_loop
    2
    main_loop
    3
    main_loop
    4
    main_loop
    5
    ...
    
  • Multithreading crashes: KeyError in get_reference(key::UInt64) / free_reference(key::UInt64) && No method matching create_reference(::UInt64)

    Multithreading crashes: KeyError in get_reference(key::UInt64) / free_reference(key::UInt64) && No method matching create_reference(::UInt64)

    Hey there clem, I'm running into a few different crashes whenever I try multithreading. Here's a minimal example that tends to crash in 5 seconds or less. Am I misunderstanding the docs and doing something unsafe here?

    ctest --verbose output is all fine (except for the resize_array test #25) Ubuntu 20.04, Julia 1.7.1, clang-14

    Minimal Example:

    #include <jluna.hpp>
    
    using namespace jluna;
    
    int main() {
        initialize(4);
    
        auto lambda = [](){
            while(true) {
                Main.safe_eval("@info \"lambda:\" Threads.threadid()");
                Main.safe_eval("sleep(1)");
            }
        };
    
        Task<void> t1 = ThreadPool::create<void()>(lambda);
        t1.schedule();
    
        while(true) {
            Main.safe_eval("@info \"main:\" Threads.threadid()");
            Main.safe_eval("sleep(1)");
        }
    
        return 0;
    }
    

    Most common crash output Worth noting the crash sometimes happens in free_reference(key::UInt64) rather than get_reference(key::UInt64).

    terminate called after throwing an instance of 'jluna::JuliaException'
      what():  [JULIA][EXCEPTION] KeyError: key 0x00007f88f5adc750 not found
    Stacktrace:
     [1] getindex
       @ ./dict.jl:481 [inlined]
     [2] get_reference(key::UInt64)
       @ Main.jluna.memory_handler ./none:594
     [3] safe_call(f::Function, args::UInt64)
       @ Main.jluna ./none:18
     [4] (::Main.jluna.cppcall.var"#3#4"{UInt64})()
       @ Main.jluna.cppcall ./none:813
    
    signal (6): Aborted
    in expression starting at none:1
    gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
    abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
    unknown function (ip: 0x7f89591a9a30)
    unknown function (ip: 0x7f89591b55db)
    _ZSt9terminatev at /lib/x86_64-linux-gnu/libstdc++.so.6 (unknown line)
    __cxa_throw at /lib/x86_64-linux-gnu/libstdc++.so.6 (unknown line)
    safe_call<_jl_value_t *> at /home/frivold/kef_env/warm_dep_ws/install/jluna/include/jluna/.src/safe_utilities.inl:44
    ProxyValue at /home/frivold/kef_env/warm_dep_ws/build/jluna/../../src/jluna/.src/proxy.cpp:31
    Proxy at /home/frivold/kef_env/warm_dep_ws/build/jluna/../../src/jluna/.src/proxy.cpp:106
    safe_eval at /home/frivold/kef_env/warm_dep_ws/build/jluna/../../src/jluna/.src/module.cpp:67
    operator() at /home/frivold/kef_env/kef_ws/src/jluna_wrapper/src/jluna_test.cpp:11
    _M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:300
    operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688
    operator() at /home/frivold/kef_env/warm_dep_ws/install/jluna/include/jluna/.src/multi_threading.inl:327
    _M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:285
    #3 at ./none:813
    unknown function (ip: 0x7f88f8d7453f)
    _jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
    jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
    jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
    start_task at /buildworker/worker/package_linux64/build/src/task.c:877
    Allocations: 1620960 (Pool: 1620032; Big: 928); GC: 1
    

    Less common crash output

    terminate called after throwing an instance of 'jluna::JuliaException'
      what():  [JULIA][EXCEPTION] MethodError: no method matching create_reference(::UInt64)
    Closest candidates are:
      create_reference(!Matched::Ptr{Nothing}) at none:546
      create_reference(!Matched::Nothing) at none:567
    Stacktrace:
     [1] safe_call(f::Function, args::UInt64)
       @ Main.jluna ./none:18
    
    signal (6): Aborted
    in expression starting at none:0
    gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
    abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
    unknown function (ip: 0x7f16feea8a30)
    unknown function (ip: 0x7f16feeb45db)
    _ZSt9terminatev at /lib/x86_64-linux-gnu/libstdc++.so.6 (unknown line)
    __clang_call_terminate at /home/frivold/kef_env/warm_dep_ws/install/jluna/lib/libjluna.so.0.9.1 (unknown line)
    ~ProxyValue at /home/frivold/kef_env/warm_dep_ws/build/jluna/../../src/jluna/.src/proxy.cpp:62
    _M_dispose at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:377
    _M_release at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:155 [inlined]
    ~__shared_count at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:730 [inlined]
    ~__shared_ptr at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:1169 [inlined]
    reset at /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:1287 [inlined]
    ~Proxy at /home/frivold/kef_env/warm_dep_ws/build/jluna/../../src/jluna/.src/proxy.cpp:111
    main at /home/frivold/kef_env/kef_ws/src/jluna_wrapper/src/jluna_test.cpp:31
    __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
    _start at /home/frivold/kef_env/kef_ws/src/jluna_wrapper/cmake-build-debug/jluna_test (unknown line)
    Allocations: 2722 (Pool: 2712; Big: 10); GC: 0
    
  • `make install` deadlocks with `CMAKE_INSTALL_PREFIX=*/jluna`

    `make install` deadlocks with `CMAKE_INSTALL_PREFIX=*/jluna`

    After installing a fresh version of jluna into ~/Desktop/jluna:

    mkdir build
    cd build
    cmake .. -DCMAKE_INSTALL_PREFIX=~/Desktop/jluna/
    make install
    

    Causes a loop:

    -- Installing: /home/clem/Desktop/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/jluna/include/array.hpp
    

    Where jluna/include/jluna/include is appended to itself infinitely. Most likely an issue with cmake/install-rules.cmake

  • README FIRST

    README FIRST

    Welcome to the dark side of the j~~moon~~luna

    Please be patient and understanding when opening issues, we're a very small team, but we'll try our best to get back to you in a timely manner and implement fixes where they are needed.

    If you are posting about a bug / crash / compiler error, please make sure you went through the troubleshooting steps outlined in the installation tutorial first.

    If the issue is a runtime bug, please post the output of the CTest bundled with jluna. You can call it from inside jluna/build using

    # in jluna/build
    ctest --verbose
    

    More information about this also available in the installation guide .

    Thank you for helping jluna evolve!

A single file C++ header-only minizip wrapper library

cpp-zipper A single file C++ header-only minizip wrapper library This code is based on 'Making MiniZip Easier to Use' by John Schember. https://nachti

Sep 10, 2022
🔌 A C++ RAII Pipewire-API Wrapper

A C++ RAII PipeWire-API Wrapper Description Rohrkabel is a RAII wrapper around the pipewire-api that aims to simplify work with it, at the moment Rohr

Sep 4, 2022
Homework repo of Modern Cpp for CV (2020Spring) at UniBonn

Modern C++ Course For CV (2020) source file can be found here. Homework Assignments Homework # Title Homework sheet Files and Data MyStatus Homework_1

Aug 27, 2022
🐛 Pangea Software's Bugdom for modern systems
🐛 Pangea Software's Bugdom for modern systems

Bugdom This is Bugdom running on modern macOS, Windows and Linux! This version, at https://github.com/jorio/Bugdom, is approved by Pangea Software. Ge

Sep 12, 2022
📚 Modern C++ Tutorial: C++11/14/17/20 On the Fly
📚 Modern C++ Tutorial: C++11/14/17/20 On the Fly

The book claims to be "On the Fly". Its intent is to provide a comprehensive introduction to the relevant features regarding modern C++ (before 2020s). Readers can choose interesting content according to the following table of content to learn and quickly familiarize the new features you would like to learn. Readers should be aware that not all of these features are required. Instead, it should be learned when you really need it.

Sep 21, 2022
A cheatsheet of modern C++ language and library features.

C++20/17/14/11 Overview Many of these descriptions and examples come from various resources (see Acknowledgements section), summarized in my own words

Sep 21, 2022
Pangea Software's Mighty Mike (Power Pete) for modern systems
Pangea Software's Mighty Mike (Power Pete) for modern systems

Mighty Mike (a.k.a. Power Pete) This is Pangea Software's Mighty Mike updated to run on modern systems. Set in a toy store, this top-down action game

Sep 9, 2022
Modern C++ Programming Course (C++11/14/17/20)

Modern C++ Programming Course (C++11/14/17/20)

Sep 22, 2022
A comprehensive catalog of modern and classic books on C++ programming language
A comprehensive catalog of modern and classic books on C++ programming language

A comprehensive catalog of modern and classic books on C++ programming language

Sep 18, 2022
A modern dynamically typed programming language that gets compiled to bytecode and is run in a virtual machine called SVM (Strawbry Virtual Machine).

Strawbry A bytecode programming language. Here is what I want Strawbry to look like: var a = 1 var b = 2 var c = a + b print(c) func sqrt(x) { re

Jan 5, 2022
Modern, header-only, compact and cross platform C++ network/sockets library

cpp-net-lib Modern, header-only, compact and cross-platform C++ network/sockets library. Don't mind the crappy name, I suck at naming things. Why? I n

Jul 20, 2022
Linux Network Programming in Modern C++

Linux Network Programming in Modern C++ Starter code for network programming in the Linux environment, providing wrapper classes written in modern C++

Feb 7, 2022
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

Sep 16, 2022
Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

Sep 15, 2022
Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

Sep 21, 2022
Feature-rich C99 library for memory scanning purposes, designed for Windows running machines, meant to work on both 32-bit and 64-bit portable executables. Has a modern C++ wrapper.

memscan Feature-rich C99 library for memory scanning purposes, designed for Windows running machines, meant to work on both 32-bit and 64-bit portable

Aug 24, 2022
A modern day direct port of BOOM 2.02 for modern times. Aiming to tastefully continue the development of BOOM, in the style of TeamTNT.
A modern day direct port of BOOM 2.02 for modern times.  Aiming to tastefully continue the development of BOOM, in the style of TeamTNT.

ReBOOM ReBOOM is a continuation of the BOOM source port, version 2.02. what is it ReBOOM is a source port, directly ported from BOOM 2.02 with additio

Jul 27, 2022
a simple RPC wrapper generator to C/C++ functions

This project initiated from the following practical problem. To control experimental equipment via computers, manufactures provide software drivers wi

Jan 25, 2022
An object oriented C++ wrapper for CURL (libcurl)

curlcpp An object-oriented C++ wrapper for cURL tool If you want to know a bit more about cURL and libcurl, you should go on the official website http

Sep 17, 2022
Openframework wrapper for box2d
Openframework wrapper for box2d

ofxBox2d Introduction This is a simple wrapper for box2d using Openframeworks. The examples below are still in progressive, but should be stable for t

Jul 4, 2022