A easy to use multithreading thread pool library for C. It is a handy stream like job scheduler with an automatic garbage collector. This is a multithreaded job scheduler for non I/O bound computation.

C Thread Stream

A easy to use multithreading thread pool library for C. It is a handy stream-like job scheduler with an automatic garbage collector for non I/O bound computation.

  • scalability
  • easy to integrate
  • easy to use

Use Case

void *my_print(sensor_data_t data){
    fprintf(stderr,"function call %"PRIu16 "- id is called \n", ((sensor_data_t)data).id );
}

void *my_print1(sensor_data_t data){
    fprintf(stderr,"this is second thread function call %"PRIu16 "- id is called \n", ((sensor_data_t)data).id );
}

declare a thread-safe, reentrant function

int main() {
    cstream_t * my_stream;
    cstream_init(&my_stream,2,2);
    stream_function_init(my_stream,my_print,1);
    stream_function_init(my_stream,my_print1,2);
    for(int i = 1; i < 20000; i ++ ){
        sensor_data_t mydata;
        mydata.id    = i;
        mydata.ts    = time(NULL);
        mydata.value = 1232;
        cstream_insert(my_stream,&mydata);
    }
    cstream_free(&my_stream);
}

initiate cstream struct, with 2 thread and 2 jobs. (2 kinds of function.) it is possible to initiate more thread than jobs. If you want multiple thread on one job you can initiate multiple thread with stream_function_init with same job number. The stream will do correct job stealing for you.

    cstream_t * my_stream;
    cstream_init(&my_stream,2,2);

Initiate thread to the stream with stream_function_init( pointer to cstream struct, job function, job identification number). no work is done at this stage. A thread is initialized and attatched to the stream.

job identification number should start from 1 and increase consecutively.

    stream_function_init(my_stream,my_print,1);
    stream_function_init(my_stream,my_print1,2);

insert data to be processed by job function. (function should not write to the data element, writable + dependancy schedulable job initializer comming soon) already initialized job functions will process data automatically. A garbage collector will free resources when all job functions have visited and left the data stream element.

        cstream_insert(my_stream,&mydata);

Example output

./cstream
function call 1 id is called 
function call 2 id is called 
function call 3 id is called 
function call 4 id is called 
function call 5 id is called 
function call 6 id is called 
this is second thread function call 1 id is called 
this is second thread function call 2 id is called 
this is second thread function call 3 id is called 
this is second thread function call 4 id is called 
Garbage Collector deletion started 
Garbage Collector deletion started 
Garbage Collector deletion started 
Garbage Collector deletion started 
this is second thread function call 5 id is called 
this is second thread function call 6 id is called 
Garbage Collector deletion started 
Garbage Collector deletion started 
Garbage Collector deletion started 
GarbageCollector At EOS, Joining All Job thread and Terminating 

Valgrind check after 1,000,000 insertion.

GarbageCollector At EOS, Joining All Job thread and Terminating 
==970== 
==970== HEAP SUMMARY:
==970==     in use at exit: 0 bytes in 0 blocks
==970==   total heap usage: 1,000,011 allocs, 1,000,011 frees, 256,002,926 bytes allocated
==970== 
==970== All heap blocks were freed -- no leaks are possible
==970== 
==970== For lists of detected and suppressed errors, rerun with: -s
==970== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Make

Previously due to tail recursion, -O3 level optimization was needed. However, it is no longer needed.

test:
	gcc -O3 -g sbuffer.h sbuffer.c -lpthread -o sbuffer
	./sbuffer

Remarks

This is unfinished work, A non-side effect functions with dependency will be supported later. Currently dynamically allocated data element is not supported but the functionality will be added soon.

Similar Resources

EOSP ThreadPool is a header-only templated thread pool writtent in c++17.

EOSP Threadpool Description EOSP ThreadPool is a header-only templated thread pool writtent in c++17. It is designed to be easy to use while being abl

Apr 22, 2022

High Performance Linux C++ Network Programming Framework based on IO Multiplexing and Thread Pool

Kingpin is a C++ network programming framework based on TCP/IP + epoll + pthread, aims to implement a library for the high concurrent servers and clie

Jun 19, 2022

Work Stealing Thread Pool

wstpool Work Stealing Thread Pool, Header Only, C++ Threads Consistent with the C++ async/future programming model. Drop-in replacement for 'async' fo

Jul 20, 2021

MAN - Man is Thread Pool in C++17

Introduction MAN is a ThreadPool wrote in C++17. The name is chosen because, at least in France, it is said that men are not able to do several things

Mar 6, 2022

ThreadPool - A fastest, exception-safety and pure C++17 thread pool.

Warnings Since commit 468129863ec65c0b4ede02e8581bea682351a6d2, I move ThreadPool to C++17. (To use std::apply.) In addition, the rule of passing para

Jun 4, 2022

ThreadPool - A simple C++11 Thread Pool implementation

ThreadPool A simple C++11 Thread Pool implementation. Basic usage: // create thread pool with 4 worker threads ThreadPool pool(4); // enqueue and sto

Jun 24, 2022

A modern thread pool implementation based on C++20

A modern thread pool implementation based on C++20

thread-pool A simple, functional thread pool implementation using pure C++20. Features Built entirely with C++20 Enqueue tasks with or without trackin

Jun 7, 2022

Sqrt OS is a simulation of an OS scheduler and memory manager using different scheduling algorithms including Highest Priority First (non-preemptive), Shortest Remaining Time Next, and Round Robin

Sqrt OS is a simulation of an OS scheduler and memory manager using different scheduling algorithms including Highest Priority First (non-preemptive), Shortest Remaining Time Next, and Round Robin

A CPU scheduler determines an order for the execution of its scheduled processes; it decides which process will run according to a certain data structure that keeps track of the processes in the system and their status.

Jul 14, 2021

A hybrid thread / fiber task scheduler written in C++ 11

Marl Marl is a hybrid thread / fiber task scheduler written in C++ 11. About Marl is a C++ 11 library that provides a fluent interface for running tas

Jun 16, 2022
Thread-pool - Thread pool implementation using c++11 threads
Thread-pool - Thread pool implementation using c++11 threads

Table of Contents Introduction Build instructions Thread pool Queue Submit function Thread worker Usage example Use case#1 Use case#2 Use case#3 Futur

Jun 15, 2022
Thread-pool-cpp - High performance C++11 thread pool

thread-pool-cpp It is highly scalable and fast. It is header only. No external dependencies, only standard library needed. It implements both work-ste

May 27, 2022
An easy to use C++ Thread Pool

mvThreadPool (This library is available under a free and permissive license) mvThreadPool is a simple to use header only C++ threadpool based on work

Jun 7, 2022
Lucy job system - Fiber-based job system with extremely simple API

Lucy Job System This is outdated compared to Lumix Engine. Use that instead. Fiber-based job system with extremely simple API. It's a standalone versi

Mar 11, 2022
🧵 Fast and easy multithreading for React Native using JSI
🧵 Fast and easy multithreading for React Native using JSI

react-native-multithreading ?? Fast and easy multithreading for React Native using JSI. Installation npm install react-native-multithreading npx pod-i

Jun 24, 2022
Pool is C++17 memory pool template with different implementations(algorithms)

Object Pool Description Pool is C++17 object(memory) pool template with different implementations(algorithms) The classic object pool pattern is a sof

Feb 14, 2022
A multithreading library implemented in C
A multithreading library implemented in C

CthreAd A multi-threading library for C Programs Report Bug · Request Feature Table of Contents About The Project One-One Many-One Usage Testing Contr

May 6, 2021
CTPL - Modern and efficient C++ Thread Pool Library

CTPL Modern and efficient C++ Thread Pool Library A thread pool is a programming pattern for parallel execution of jobs, http://en.wikipedia.org/wiki/

Jun 20, 2022
A C++17 thread pool for high-performance scientific computing.

We present a modern C++17-compatible thread pool implementation, built from scratch with high-performance scientific computing in mind. The thread pool is implemented as a single lightweight and self-contained class, and does not have any dependencies other than the C++17 standard library, thus allowing a great degree of portability

Jun 18, 2022
An ultra-simple thread pool implementation for running void() functions in multiple worker threads
An ultra-simple thread pool implementation for running void() functions in multiple worker threads

void_thread_pool.cpp © 2021 Dr Sebastien Sikora. [email protected] Updated 06/11/2021. What is it? void_thread_pool.cpp is an ultra-simple

Nov 19, 2021