Low dependency（C++11 STL only）, good portability, header-only, deep neural networks for embedded

LKYDeepNN

LKYDeepNN

  • 可訓練的深度類神經網路 (Deep Neural Network) 函式庫。
  • 輕量，核心部份只依賴 C++11 標準函式庫，低相依性、好移植，方便在嵌入式系統上使用。

Class diagram

this is Class diagram SVG

附有訓練視覺化 demo 程式

  • 訓練視覺化程式以 OpenCV 撰寫，但 LKYDeepNN 本身不依賴 OpenCV。
  • 繪圖功能僅以 function pointer 傳入物件中，在訓練過程中呼叫。
  • 下面這2張圖 ↓ ↓ ↓ ↓ 是 33fps 的 GIF，分別是 classification 和 regression。如果不會動的話，請按 F5 重新整理網頁，或是單獨對圖檔另開新視窗就可以看到動畫了。

    classification_training_demo_Spiral.gif　　regression_training_demo_Cos(2x)+Sin(3x).gif

隱藏層的層數和節點數可以任意設定，簡單又有彈性

  • std::vector<int>(8,7) 8個隱藏層，每層都是7個節點，還可以再高，只要記憶體夠大的話。
  • std::vector<int>{5,5,6,6} 4個隱藏層，每層節點數分別是:5個、5個、6個、6個。
  • LKYDeepNN::LKYDeepNN(9, std::vector<int>{4,8}, 7) 代表輸入點9個、2個隱藏層分別是4節點和8節點，最後輸出層有7個節點。
  • 目前只能 Fully-Connected，未來會考慮實作 Dropout-Connected 或 Fuzzy-Connected

自由設定活化函數

  • LKYDeepNN::SetActivation( 隱藏層 , 輸出層 )
  • LKYDeepNN::SetActivation( new Tanh() ,new Linear() ) //回歸推薦使用
  • LKYDeepNN::SetActivation( new ReLU() ,new Softmax() ) //分類推薦使用
  • 目前已經有: SeLU, ReLU, Tanh, Softmax, Sigmoid, Linear
  • 可以繼承 abstract class Activation 自由實作任意活化函數

訓練

  • void LKYDeepNN::Training(double learningRate, int epochs, std::vector<vector<double>> trainData)
  • 每一筆資料都需要先整理成 std::vector<double>
  • std::vector<vector<double>> 就是很多筆資料，這個才能餵給模型。

未來預計處理issue

  • Xavier initialization
  • He initialization
  • 可保存和讀取的 Weights 和 Biases
  • 更豐富得測試資料集
  • Cross Entropy BP（測試中）
  • Hinge Loss Function（努力中）
  • Early Stopping
  • L1 & L2 Regularization
  • Copy Constructor
  • 資料正規化工具
  • 更多活化函數。
  • 訓練過程中可自適應的動態 Learning Rate
  • Weights 分析工具
  • Convolution Layer

編譯參數

  • 在 .vscode/tasks.json 裡

Reference:

致謝

  • 由 Microprogram 贊助上班時間產出。
Owner
Lin Kao-Yuan
Lin Kao-Yuan
https://github.com/mosdeo/LKYDeepNN
Similar Resources

InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.

InsNet documentation InsNet (documentation) is a powerful neural network library aiming at building instance-dependent computation graphs. It is desig

Jan 3, 2023

A framework for generic hybrid two-party computation and private inference with neural networks

MOTION2NX -- A Framework for Generic Hybrid Two-Party Computation and Private Inference with Neural Networks This software is an extension of the MOTI

Nov 29, 2022

TS-9 guitar pedal clone using neural networks.

TS-9 guitar pedal clone using neural networks.

TS-M1N3 TS-M1N3 is a guitar plugin clone of the TS-9 Tubescreamer overdrive pedal. Machine learning was used to train a model of both the drive and to

Nov 23, 2022

A Tool for Verifying Neural Networks using SMT-Based Model Checking

Project Title QNNVerifier Description A Tool for Verifying Neural Networks using SMT-Based Model Checking. Using Frama-C and ESBMC as the backends. Yo

Dec 11, 2021

CoDi is a cellular automaton model for spiking neural networks

CoDi CoDi is a cellular automaton (CA) model for spiking neural networks (SNNs). CoDi is an acronym for Collect and Distribute, referring to the signa

May 5, 2022

oneAPI Deep Neural Network Library (oneDNN)

oneAPI Deep Neural Network Library (oneDNN) This software was previously known as Intel(R) Math Kernel Library for Deep Neural Networks (Intel(R) MKL-

Jan 6, 2023

Benchmark framework of compute-in-memory based accelerators for deep neural network (inference engine focused)

DNN+NeuroSim V1.3 The DNN+NeuroSim framework was developed by Prof. Shimeng Yu's group (Georgia Institute of Technology). The model is made publicly a

Nov 24, 2022

Implementing Deep Convolutional Neural Network in C without External Libraries for YUV video Super-Resolution

Implementing Deep Convolutional Neural Network in C without External Libraries for YUV video Super-Resolution

DeepC: Implementing Deep Convolutional Neural Network in C without External Libraries for YUV video Super-Resolution This code uses FSRCNN algorithm t

Dec 27, 2022

Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models

Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine DSSTNE (pronounced "Destiny") is an open source software library for training and deploying

Dec 30, 2022
Comments
  • 在 macOS 上發生「NSInternalInconsistencyException」

    在 macOS 上發生「NSInternalInconsistencyException」 

    LKYs-MacBook-Air:LKYDeepNN lky$ ./LKYDeepNN

Completed BackPropagation Layer InitializeWeights()
Completed BackPropagation Layer InitializeWeights()
Completed BackPropagation Layer InitializeWeights()
Completed BackPropagation Layer InitializeWeights()
Activation is SeLU.
Activation is Softmax.
LossFunction is CrossEntropy.
====== LKYDeepNN ======
Layer: 
   Input: 2
  Hidden: 8-8-8-
  Output: 2
Activation Function: 
  Hidden: SeLU
  Output: Softmax
Loss Function: 
  Output: CrossEntropy
=======================

訓練開始
learningRate=0.008333
2019-10-10 18:03:20.550 LKYDeepNN[2436:117418] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
        0   CoreFoundation                      0x00007fff326dad63 __exceptionPreprocess + 250
        1   libobjc.A.dylib                     0x00007fff685c9bd4 objc_exception_throw + 48
        2   CoreFoundation                      0x00007fff326f530c -[NSException raise] + 9
        3   AppKit                              0x00007fff2f8bf7b3 -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 310
        4   AppKit                              0x00007fff2f8bcce1 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1416
        5   AppKit                              0x00007fff2f8bc753 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
        6   AppKit                              0x00007fff2fb50368 -[NSWindow initWithContentRect:styleMask:backing:defer:screen:] + 52
        7   libopencv_highgui.4.1.dylib         0x00000001089db666 cvNamedWindow + 368
        8   libopencv_highgui.4.1.dylib         0x00000001089db174 cvShowImage + 95
        9   libopencv_highgui.4.1.dylib         0x00000001089d8c28 _ZN2cv6imshowERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEERKNS_11_InputArrayE + 360
        10  LKYDeepNN                           0x0000000107a6fb87 _Z24Draw2DClassificationDataNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_6vectorINS6_IdNS3_IdEEEENS3_IS8_EEEEP9LKYDeepNNS5_dddd + 2183
        11  LKYDeepNN                           0x0000000107a71044 _Z12DrawTrainingP9LKYDeepNNiiRKNSt3__16vectorINS2_IdNS1_9allocatorIdEEEENS3_IS5_EEEE + 1108
        12  LKYDeepNN                           0x0000000107a772fc _ZNSt3__114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEPFvP9LKYDeepNNiiRKNS_6vectorINS9_IdNS_9allocatorIdEEEENSA_ISC_EEEEES8_iiSE_EEEEEPvSK_ + 60
        13  libsystem_pthread.dylib             0x00007fff69b34d76 _pthread_start + 125
        14  libsystem_pthread.dylib             0x00007fff69b315d7 thread_start + 15
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6
Related tags
Artificial Intelligence data-science deep-learning data-visualization back-propagation cross-entropy multilayer-perceptron
A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms.
 A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms.

iNeural A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms. What is a Neural Network? Work on

Apr 5, 2022
IBM Z Deep Neural Network Library (zDNN) provides an interface for applications making use of Neural Network Processing Assist Facility (NNPA).

zDNN API Reference Contacts Nicholas Marion ([email protected]) Andreas Krebbel ([email protected]) Version 0.4.0 Table of Contents Overview Envi

Dec 20, 2022
header only, dependency-free deep learning framework in C++14
 header only, dependency-free deep learning framework in C++14

The project may be abandoned since the maintainer(s) are just looking to move on. In the case anyone is interested in continuing the project, let us k

Dec 31, 2022
Tensors and Dynamic neural networks in Python with strong GPU acceleration
 Tensors and Dynamic neural networks in Python with strong GPU acceleration

PyTorch is a Python package that provides two high-level features: Tensor computation (like NumPy) with strong GPU acceleration Deep neural networks b

Jan 4, 2023
A lightweight C library for artificial neural networks

Getting Started # acquire source code and compile git clone https://github.com/attractivechaos/kann cd kann; make # learn unsigned addition (30000 sam

Dec 19, 2022
Convolutional Neural Networks
 Convolutional Neural Networks

Darknet Darknet is an open source neural network framework written in C and CUDA. It is fast, easy to install, and supports CPU and GPU computation. D

Jan 9, 2023
Tiny CUDA Neural Networks
 Tiny CUDA Neural Networks

This is a small, self-contained framework for training and querying neural networks. Most notably, it contains a lightning fast "fully fused" multi-layer perceptron as well as support for various advanced input encodings, losses, and optimizers.

Jan 7, 2023
Raspberry Pi guitar pedal using neural networks to emulate real amps and pedals.
 Raspberry Pi guitar pedal using neural networks to emulate real amps and pedals.

NeuralPi NeuralPi is a guitar pedal using neural networks to emulate real amps and pedals on a Raspberry Pi 4. The NeuralPi software is a VST3 plugin

Jan 5, 2023
An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks
 An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks

AnalyticMesh Analytic Marching is an exact meshing solution from neural networks. Compared to standard methods, it completely avoids geometric and top

Dec 21, 2022
An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks
 An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks

AnalyticMesh Analytic Marching is an exact meshing solution from neural networks. Compared to standard methods, it completely avoids geometric and top

Dec 21, 2022