Deep Learning API and Server in C++11 support for Caffe, Caffe2, PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE

DeepDetect Logo

Open Source Deep Learning Server & API

Join the chat at https://gitter.im/beniz/deepdetect GitHub release (latest SemVer) GitHub Release Date GitHub commits since latest release (by date)

DeepDetect (https://www.deepdetect.com/) is a machine learning API and server written in C++11. It makes state of the art machine learning easy to work with and integrate into existing applications. It has support for both training and inference, with automatic conversion to embedded platforms with TensorRT (NVidia GPU) and NCNN (ARM CPU).

It implements support for supervised and unsupervised deep learning of images, text, time series and other data, with focus on simplicity and ease of use, test and connection into existing applications. It supports classification, object detection, segmentation, regression, autoencoders, ...

And it relies on external machine learning libraries through a very generic and flexible API. At the moment it has support for:

Please join the community on Gitter, where we help users get through with installation, API, neural nets and connection to external applications.


Build type STABLE DEVEL
SOURCE
Docker image CPU
Docker image GPU
Docker image GPU+TORCH
Docker image GPU+TENSORRT

Main features

  • high-level API for machine learning and deep learning
  • support for Caffe, Tensorflow, XGBoost, T-SNE, Caffe2, NCNN, TensorRT, Pytorch
  • classification, regression, autoencoders, object detection, segmentation, time-series
  • JSON communication format
  • remote Python and Javacript clients
  • dedicated server with support for asynchronous training calls
  • high performances, benefit from multicore CPU and GPU
  • built-in similarity search via neural embeddings
  • connector to handle large collections of images with on-the-fly data augmentation (e.g. rotations, mirroring)
  • connector to handle CSV files with preprocessing capabilities
  • connector to handle text files, sentences, and character-based models
  • connector to handle SVM file format for sparse data
  • range of built-in model assessment measures (e.g. F1, multiclass log loss, ...)
  • range of special losses (e.g Dice, contour, ...)
  • no database dependency and sync, all information and model parameters organized and available from the filesystem
  • flexible template output format to simplify connection to external applications
  • templates for the most useful neural architectures (e.g. Googlenet, Alexnet, ResNet, convnet, character-based convnet, mlp, logistic regression, SSD, DeepLab, PSPNet, U-Net, CRNN, ShuffleNet, SqueezeNet, MobileNet, RefineDet, VOVNet, ...)
  • support for sparse features and computations on both GPU and CPU
  • built-in similarity indexing and search of predicted features, images, objects and probability distributions

Machine Learning functionalities per library

Caffe Caffe2 XGBoost TensorRT NCNN Libtorch Tensorflow T-SNE Dlib
Serving
Training (CPU) Y Y Y N/A N/A Y N Y N
Training (GPU) Y Y Y N/A N/A Y N Y N
Inference (CPU) Y Y Y N Y Y Y N/A Y
Inference (GPU) Y Y Y Y N Y Y N/A Y
Models
Classification Y Y Y Y Y Y Y N/A Y
Object Detection Y Y N Y Y N N N/A Y
Segmentation Y N N N N N N N/A N
Regression Y N Y N N Y N N/A N
Autoencoder Y N N/A N N N N N/A N
NLP Y N Y N N Y N Y N
OCR / Seq2Seq Y N N N Y N N N N
Time-Series Y N N N Y Y N N N
Data
CSV Y N Y N N N N Y N
SVM Y N Y N N N N N N
Text words Y N Y N N N N N N
Text characters Y N N N N N N Y N
Images Y Y N Y Y Y Y Y Y
Time-Series Y N N N Y N N N N

Tools and Clients

Models

Caffe Tensorflow Source Top-1 Accuracy (ImageNet)
AlexNet Y N BVLC 57.1%
SqueezeNet Y N DeepScale 59.5%
Inception v1 / GoogleNet Y Y BVLC / Google 67.9%
Inception v2 N Y Google 72.2%
Inception v3 N Y Google 76.9%
Inception v4 N Y Google 80.2%
ResNet 50 Y Y MSR 75.3%
ResNet 101 Y Y MSR 76.4%
ResNet 152 Y Y MSR 77%
Inception-ResNet-v2 N Y Google 79.79%
VGG-16 Y Y Oxford 70.5%
VGG-19 Y Y Oxford 71.3%
ResNext 50 Y N https://github.com/terrychenism/ResNeXt 76.9%
ResNext 101 Y N https://github.com/terrychenism/ResNeXt 77.9%
ResNext 152 Y N https://github.com/terrychenism/ResNeXt 78.7%
DenseNet-121 Y N https://github.com/shicai/DenseNet-Caffe 74.9%
DenseNet-161 Y N https://github.com/shicai/DenseNet-Caffe 77.6%
DenseNet-169 Y N https://github.com/shicai/DenseNet-Caffe 76.1%
DenseNet-201 Y N https://github.com/shicai/DenseNet-Caffe 77.3%
SE-BN-Inception Y N https://github.com/hujie-frank/SENet 76.38%
SE-ResNet-50 Y N https://github.com/hujie-frank/SENet 77.63%
SE-ResNet-101 Y N https://github.com/hujie-frank/SENet 78.25%
SE-ResNet-152 Y N https://github.com/hujie-frank/SENet 78.66%
SE-ResNext-50 Y N https://github.com/hujie-frank/SENet 79.03%
SE-ResNext-101 Y N https://github.com/hujie-frank/SENet 80.19%
SENet Y N https://github.com/hujie-frank/SENet 81.32%
VOC0712 (object detection) Y N https://github.com/weiliu89/caffe/tree/ssd 71.2 mAP
InceptionBN-21k Y N https://github.com/pertusa/InceptionBN-21K-for-Caffe 41.9%
Inception v3 5K N Y https://github.com/openimages/dataset
5-point Face Landmarking Model (face detection) N N http://blog.dlib.net/2017/09/fast-multiclass-object-detection-in.html
Front/Rear vehicle detection (object detection) N N http://blog.dlib.net/2017/09/fast-multiclass-object-detection-in.html

More models:

References

Authors

DeepDetect is designed, implemented and supported by Jolibrain with the help of other contributors.

Owner
JoliBrain
Pretty AI for solving real world problems
JoliBrain
Comments
  • Support for TensorFlow

    Support for TensorFlow

    See short discussion here, https://github.com/tensorflow/tensorflow/issues/85

    In fact, ability to save computation graph models as protobuffers then read them up and providing data at runtime, not too different than with Caffe.

    The API has been designed to absorb more libraries, and this is a nice test case that should allow to expand to more complex models such as seq2seq without breaking compatibility through API.

    Additional relevant doc:

    • http://tensorflow.org/api_docs/cc/index.md
    • http://tensorflow.org/how_tos/new_data_formats/index.md
    • https://codesachin.wordpress.com/2015/11/20/recursively-copying-elements-from-one-graph-to-another-in-tensorflow/
  • Support for Deep Residual Net (ResNet) reference models for ILSVRC

    Support for Deep Residual Net (ResNet) reference models for ILSVRC

    This is support for the state-of-the-art nets just released by https://github.com/KaimingHe/deep-residual-networks. They are implemented as DeepDetect neural net templates: resnet_50,resnet_101 and resnet_152 are now available from the API.

    Note: training successfully tested on resnet_18 and resnet_50

    For using the nets in predict mode:

    • model repository preparation:
    • download models from https://github.com/KaimingHe/deep-residual-networks
    • mkdir path/to/model
    • cp ResNet-50-model.caffemodel path/to/model/
    • cp ResNet_mean.binaryproto path/to/model/mean.binaryproto
    • service creation:
    curl -X PUT "http://localhost:8080/services/imageserv" -d "{\"mllib\":\"caffe\",\"description\":\"image classification service\",\"type\":\"supervised\",\"parameters\":{\"input\":{\"connector\":\"image\"},\"mllib\":{\"template\":\"resnet_50\",\"nclasses\":1000}},\"model\":{\"templates\":\"../templates/caffe/\",\"repository\":\"/path/to/model\"}}"
    

    Note that template is set to resnet_50

    • image classification:
    curl -X POST "http://localhost:8080/predict" -d "{\"service\":\"imageserv\",\"parameters\":{\"input\":{\"width\":224,\"height\":224},\"output\":{\"best\":5}},\"data\":[\"http://i.ytimg.com/vi/0vxOhd4qlnA/maxresdefault.jpg\"]}"
    
  • Amazon Machine Instance (AMI) on EC2

    Amazon Machine Instance (AMI) on EC2

    ~~Providing AMI is a good idea to ease deployment for some users.~~

    ~~Related links of interest:~~ ~~- Deprecated Caffe AMI: https://github.com/BVLC/caffe/wiki/Ubuntu-14.04-ec2-instance~~ ~~- Newly contributed Caffe AMI (that includes Torch): http://blog.titocosta.com/post/110345699197/public-~ec2-ami-with-torch-and-caffe-deep-learning~~ ~~- Another Caffe AMI and Docker: https://github.com/BVLC/caffe/issues/1092#issuecomment-59393030~~ ~~- An AMI for deep learning, and that contains Caffe: https://www.kaggle.com/forums/f/208/getting-started/t/11505/an-aws-ami-mainly-for-deep-learning~~

    After a long wait, the official AMIs with support for Caffe, XGBoost and Tensorflow backends are available for both GPU and CPU:

    See https://deepdetect.com/products/ami/ for thorough documentation.

  • Docker images optimization / CI/CD refactoring

    Docker images optimization / CI/CD refactoring

    Docker images optimization :

    | Image | Uncompressed size | Image efficiency score | Layers | Optimized | | ------ | ------ | ------ | ------ | ------ | | jolibrain/deepdetect_cpu | 7.4 GB (New 919MB)| 57% (new 95%) | 27 (new 18) |:+1: | | jolibrain/deepdetect_gpu | 12 GB (New 2.15GB)| 65% (new 95%) | 33 (new 24) |:+1: |

    CPU and GPU images contains build dependencies which increase Docker images size. To prevent this, we use multi-stage build feature to remove all build dependencies.

    CI/CD refactoring :

    Dockerfiles duplication have been deleted and we add build argument to select cmake profile.

    Build documentation have been updated on README.md file in docker folder.

  • Install deepdetect on CentOS

    Install deepdetect on CentOS

    Hi, I tried to install dependencies on CentOS with similar scripts, however, it did not work: yum install build-essential libgoogle-glog-dev libgflags-dev libeigen3-dev libopencv-dev libcppnetlib-dev libboost-dev libcurlpp-dev libcurl4-openssl-dev protobuf-compiler libopenblas-dev libhdf5-dev libprotobuf-dev libleveldb-dev libsnappy-dev liblmdb-dev libutfcpp-dev Loaded plugins: security Setting up Install Process No package build-essential available. No package libgoogle-glog-dev available. No package libgflags-dev available. No package libeigen3-dev available. No package libopencv-dev available. No package libcppnetlib-dev available. No package libboost-dev available. No package libcurlpp-dev available. No package libcurl4-openssl-dev available. No package libopenblas-dev available. No package libhdf5-dev available. No package libprotobuf-dev available.

    Does it mean, that I have to compiled all those source code one by one for installation?

  • Error creating network

    Error creating network

    I'm trying to use 'Supervised Semantics-preserving Deep Hashing' (48-bit SSDH) model as provided here.

    The following files are used for network initialization (provided by the above repo): men_ssdh.prototxt:

    name: "CaffeNet" input: "data" input_shape { dim: 10 dim: 3 dim: 227 dim: 227 } layer { name: "conv1" type: "Convolution" bottom: "data" top: "conv1" convolution_param { num_output: 96 kernel_size: 11 stride: 4 } } layer { name: "relu1" type: "ReLU" bottom: "conv1" top: "conv1" } layer { name: "pool1" type: "Pooling" bottom: "conv1" top: "pool1" pooling_param { pool: MAX kernel_size: 3 stride: 2 } } layer { name: "norm1" type: "LRN" bottom: "pool1" top: "norm1" lrn_param { local_size: 5 alpha: 0.0001 beta: 0.75 } } layer { name: "conv2" type: "Convolution" bottom: "norm1" top: "conv2" convolution_param { num_output: 256 pad: 2 kernel_size: 5 group: 2 } } layer { name: "relu2" type: "ReLU" bottom: "conv2" top: "conv2" } layer { name: "pool2" type: "Pooling" bottom: "conv2" top: "pool2" pooling_param { pool: MAX kernel_size: 3 stride: 2 } } layer { name: "norm2" type: "LRN" bottom: "pool2" top: "norm2" lrn_param { local_size: 5 alpha: 0.0001 beta: 0.75 } } layer { name: "conv3" type: "Convolution" bottom: "norm2" top: "conv3" convolution_param { num_output: 384 pad: 1 kernel_size: 3 } } layer { name: "relu3" type: "ReLU" bottom: "conv3" top: "conv3" } layer { name: "conv4" type: "Convolution" bottom: "conv3" top: "conv4" convolution_param { num_output: 384 pad: 1 kernel_size: 3 group: 2 } } layer { name: "relu4" type: "ReLU" bottom: "conv4" top: "conv4" } layer { name: "conv5" type: "Convolution" bottom: "conv4" top: "conv5" convolution_param { num_output: 256 pad: 1 kernel_size: 3 group: 2 } } layer { name: "relu5" type: "ReLU" bottom: "conv5" top: "conv5" } layer { name: "pool5" type: "Pooling" bottom: "conv5" top: "pool5" pooling_param { pool: MAX kernel_size: 3 stride: 2 } } layer { name: "fc6" type: "InnerProduct" bottom: "pool5" top: "fc6" inner_product_param { num_output: 4096 } } layer { name: "relu6" type: "ReLU" bottom: "fc6" top: "fc6" } layer { name: "drop6" type: "Dropout" bottom: "fc6" top: "fc6" dropout_param { dropout_ratio: 0.5 } } layer { name: "fc7" type: "InnerProduct" bottom: "fc6" top: "fc7" inner_product_param { num_output: 4096 } } layer { name: "relu7" type: "ReLU" bottom: "fc7" top: "fc7" } layer { name: "latent_layer" type: "InnerProduct" bottom: "fc7" top: "latent_layer" inner_product_param { num_output: 48 } } layer { name: "encode_neuron" type: "Sigmoid" bottom: "latent_layer" top: "encode_neuron" }

    men_ssdh_solver.prototxt

    net: "men_ssdh.prototxt" test_iter: 100 test_interval: 100 base_lr: 0.001 lr_policy: "step" gamma: 0.1 stepsize: 25000 display: 20 max_iter: 50000 momentum: 0.9 weight_decay: 0.0005 snapshot: 10000 snapshot_prefix: "men_ssdh"

    I'm trying to create the service with this call:

    curl -X PUT "http://localhost:8090/services/men_ssdh_orig" -d "{\"mllib\":\"caffe\",\"description\":\"image classification men finetuned\",\"type\":\"supervised\",\"parameters\":{\"input\":{\"connector\":\"image\",\"width\":227,\"height\":227},\"mllib\":{\"nclasses\":10}},\"model\":{\"repository\":\"/home/ubuntu/code/deepdetect/models/men_ssdh\"}}" 
    

    This fails with

    INFO - 11:50:20 - Initializing net from parameters: 
    E0720 11:50:20.006518 12587 caffelib.cc:1228] Error creating network
    
    ERROR - 11:50:20 - service creation call failed
    

    Any clue about what is wrong with the above initialization ?

  • Service creation Error

    Service creation Error

    Hi, @beniz , I tried to create the age classification service with:

    curl -X PUT "http://localhost:8080/services/ageserv" -d "{"mllib":"caffe","description":"age classification service","type":"supervised","parameters":{"input":{"connector":"image"},"mllib":{"nclasses":2,"template":"age_model"}},"model":{"templates":"../templates/caffe/","repository":"/opt/models/age_model/"}}"

    and got these errors: {"status":{"code":400,"msg":"BadRequest","dd_code":1006,"dd_msg":"Service Bad Request Error"}}

    Thanks.

  • CentOS 7 cannot run image prediction example

    CentOS 7 cannot run image prediction example

    Hi Beniz,

    Thanks for doing this great work of making a docker file.

    I am new to this deepdetect package. I am facing a problem running this docker image under CentOS 7, specifically version 3.10.0-327.el7.x86_64.

    I successfully ran the docker image "deepdetect_cpu" in Ubuntu 14.04, and had no problem with the following example lines.

    docker run -d -p 8080:8080 beniz/deepdetect_cpu

    curl http://localhost:8080/info

    curl -X PUT "http://localhost:8080/services/imageserv" -d "{\"mllib\":\"caffe\",\"description\":\"image classification service\",\"type\":\"supervised\",\"parameters\":{\"input\":{\"connector\":\"image\"},\"mllib\":{\"nclasses\":1000,\"template\":\"googlenet\"}},\"model\":{\"templates\":\"../templates/caffe/\",\"repository\":\"/opt/models/ggnet/\"}}"

    curl -X POST "http://localhost:8080/predict" -d "{\"service\":\"imageserv\",\"parameters\":{\"input\":{\"width\":224,\"height\":224},\"output\":{\"best\":3},\"mllib\":{\"gpu\":false}},\"data\":[\"http://i.ytimg.com/vi/0vxOhd4qlnA/maxresdefault.jpg\"]}"

    But in CentOS 7, the service is created successfully. It returns the following as in the example.

    {"status":{"code":201,"msg":"Created"}}

    But if I go on to do the prediction line, I will get

    {"status":{"code":400,"msg":"BadRequest","dd_code":1006,"dd_msg":"Service Bad Request Error"}}

    Could you take a look at this? It seems I am missing something obvious, but I cannot tell where.

    Thanks.

  • GPU Docker image fails on prediction call (GTX1080)

    GPU Docker image fails on prediction call (GTX1080)

    The problem:

    On my Ubuntu 16.04 system, which uses a GTX 1080 graphics card, I seem to be facing issues performing prediction calls on the Deepdetect GPU image. This happens even when I try to use the built-in imageserv-model. The Check failed (custom): (error) == (cudaSuccess) usually points to a memory error, but I have more than 6GB of free video memory according to nvidia-smi, which should be more than enough for the imageserv-model. Therefore, I am considering if this might be a DeepDetect-bug. When I set GPU to False, the call succeeds, albeit after quite some processing time. I am running the image in NVIDIA-Docker.

    Thanks for all of your hard work; DeepDetect is an amazing piece of software. :)

    Configuration:

    • DeepDetect GPU Docker image: beniz/deepdetect_gpu
    • DeepDetect Commit: 11adce0366d367fd16c5ac5233c8a6118a24f8ad
    • GPU: GeForce GTX 1080
    • NVIDIA driver version: 375.39
    • Linux kernel version: 4.8.0-46-generic
    • Docker version: 17.03.1-ce, build c6d412e

    Steps taken:

    • Spin up DeepDetect Docker container:

    sudo nvidia-docker run -d -p 8080:8080 beniz/deepdetect_gpu

    • API PUT call:

    curl -X PUT "http://localhost:8080/services/imageserv" -d "{\"mllib\":\"caffe\",\"description\":\"image classification service\",\"type\":\"supervised\",\"parameters\":{\"input\":{\"connector\":\"image\"},\"mllib\":{\"nclasses\":1000}},\"model\":{\"repository\":\"/opt/models/ggnet/\"}}"

    {"status":{"code":201,"msg":"Created"}}

    • API PRED call:

    curl -X POST "http://localhost:8080/predict" -d "{\"service\":\"imageserv\",\"parameters\":{\"input\":{\"width\":224,\"height\":224},\"output\":{\"best\":3},\"mllib\":{\"gpu\":true}},\"data\":[\"http://i.ytimg.com/vi/0vxOhd4qlnA/maxresdefault.jpg\"]}"

    {"status":{"code":500,"msg":"InternalError","dd_code":1007,"dd_msg":"src/caffe/util/im2col.cu:61 / Check failed (custom): (error) == (cudaSuccess)"}}

    • Server log output:

    INFO - 11:42:41 - Device id: 0 INFO - 11:42:41 - Major revision number: 6 INFO - 11:42:41 - Minor revision number: 1 INFO - 11:42:41 - Name: GeForce GTX 1080 INFO - 11:42:41 - Total global memory: 8491368448 INFO - 11:42:41 - Total shared memory per block: 49152 INFO - 11:42:41 - Total registers per block: 65536 INFO - 11:42:41 - Warp size: 32 INFO - 11:42:41 - Maximum memory pitch: 2147483647 INFO - 11:42:41 - Maximum threads per block: 1024 INFO - 11:42:41 - Maximum dimension of block: 1024, 1024, 64 INFO - 11:42:41 - Maximum dimension of grid: 2147483647, 65535, 65535 INFO - 11:42:41 - Clock rate: 1771000 INFO - 11:42:41 - Total constant memory: 65536 INFO - 11:42:41 - Texture alignment: 512 INFO - 11:42:41 - Concurrent copy and execution: Yes INFO - 11:42:41 - Number of multiprocessors: 20 INFO - 11:42:41 - Kernel execution timeout: Yes[11:42:42] /opt/deepdetect/src/caffelib.cc:1185: Error while proceeding with prediction forward pass, not enough memory?

    ERROR - 11:42:42 - service imageserv prediction call failed

    ERROR - 11:42:42 - Tue Apr 18 11:42:42 2017 UTC - 172.17.0.1 "POST /predict" 500 437

  • Error while running GET service info #2

    Error while running GET service info #2

    Hello, i m using the following command to create the service.

    [email protected]:~/dev/deepdetect/build# curl -X PUT "http://localhost:8080/services/p" -d "{\"mllib\":\"caffe\",\"description\":\"p classification service\",\"type\":\"supervised\",\"parameters\":{\"input\":{\"connector\":\"image\",\"width\":224,\"height\":224},\"mllib\":{\"template\":\"mlp\",\"nclasses\":5,\"layers\":[512,512,512],\"activation\":\"prelu\"}},\"model\":{\"templates\":\"../templates/caffe/\",\"repository\":\"../../../test_images\"}}"
    

    Using the following command to train the service

    curl -X POST "http://localhost:8080/train" -d "{\"service\":\"p\",\"async\":true,\"parameters\":{\"mllib\":{\"gpu\":false,\"net\":{\"batch_size\":32},\"solver\":{\"test_interval\":500,\"iterations\":30000,\"base_lr\":0.001,\"stepsize\":1000,\"gamma\":0.9}},\"input\":{\"connector\":\"image\",\"test_split\":0.1,\"shuffle\":true,\"width\":224,\"height\":224},\"output\":{\"measure\":[\"acc\",\"mcll\",\"f1\"]}},\"data\":[\"../../../test_images\"]}"
    
    

    In the dede logs, i can see that it started working on the images directory(attached is the dede log from console output) dede-errorlog.txt

    And then, if i run the GET training service info, it crashes the Training job with the error:

    curl -X GET "http://localhost:8080/train?service=p&job=1"
    
    INFO - 20:11:47 - Solver scaffolding done.
    INFO - 20:11:47 - Ignoring source layer inputl
    INFO - 20:11:47 - Ignoring source layer loss
    INFO - 20:11:47 - Opened lmdb ../../../test_images/test.lmdb
    
    > ERROR - 20:12:32 - service p training status call failed
    > 
    > ERROR - 20:12:32 - {"code":500,"msg":"InternalError","dd_code":1007,"dd_msg":"src/caffe/data_transformer.cpp:177 / Check failed (custom): (datum_height) == (height)"}
    

    If i don't run the check training service info call, i can see the screen is stuck at the following lines in dede console logs:

    INFO - 20:12:59 - act0 does not need backward computation.
    INFO - 20:12:59 - ip0 does not need backward computation.
    INFO - 20:12:59 - inputlt does not need backward computation.
    INFO - 20:12:59 - This network produces output label
    INFO - 20:12:59 - This network produces output losst
    INFO - 20:12:59 - Network initialization done.
    INFO - 20:12:59 - Solver scaffolding done.
    INFO - 20:12:59 - Ignoring source layer inputl
    INFO - 20:12:59 - Ignoring source layer loss
    

    Does it mean its still running and meanwhile, i should not run any check call? Please assist, thanks

  • Torch v1.12 requires libcupti* but nvidia/cuda:11.6.0-cudnn8-runtime-ubuntu20.04 doesn't include it

    Torch v1.12 requires libcupti* but nvidia/cuda:11.6.0-cudnn8-runtime-ubuntu20.04 doesn't include it

    Creating this to track the problem described here: https://gitter.im/beniz/deepdetect?at=6308cbb5f4d7a323de92e248

    In short, Torch v1.12 requires libcupti* libs but the nvidia/cuda:11.6.0-cudnn8-runtime-ubuntu20.04 docker image doesn't include them (although the nvidia/cuda:11.6.0-cudnn8-devel-ubuntu20.04 docker image does).

    Potentially related issues: https://github.com/pytorch/pytorch/issues/74473 https://github.com/pytorch/vision/issues/5635

  • DeepDetect full rewrite in Pure Java

    DeepDetect full rewrite in Pure Java

    Hi, as you all know, Java is underrated, and it is now in 2022 a reality that C++ is doomed to fail. Therefore I would strongly suggest to give up on this current project and invest a in full rewrite of DeepDetect in the absolute purest Java form.

    For this purpose, I propose using Github copilot AI (https://copilot.github.com/) and fix the code by hand afterward.

    May the beauty of Java bring real-time to your hearts and souls, my brothers and sisters! Thus having freed your minds from C++, your asses will naturally follow, or the opposite.

    image

    Glory to Java!

  • getting error while training, .solverstate

    getting error while training, .solverstate

    (A Markdown syntax reminder is available here: https://guides.github.com/features/mastering-markdown/)

    Before creating a new issue, please make sure that:

    If Ok, please give as many details as possible to help us solve the problem more efficiently.

    Configuration

    • Version of DeepDetect:
      • [ ] Locally compiled on:
        • [ ] Ubuntu 18.04 LTS
        • [ ] Other:
      • [x] Docker CPU
      • [ ] Docker GPU
      • [ ] Amazon AMI
    • Commit (shown by the server when starting):

    Your question / the problem you're facing:

    When I want to train a simple classification (dog_cat example), I face the following error:

    resuming a model requires a .solverstate file in model repository
    

    Error message (if any) / steps to reproduce the problem:

    • [x] list of API calls: as shown in the sample.

    • [x] Server log output:

    4c01f52c6171_cpu_deepdetect_1 | [2021-09-26 13:15:41.201] [dogs_cats] [info] selected solver: SGD
    4c01f52c6171_cpu_deepdetect_1 | [2021-09-26 13:15:41.201] [dogs_cats] [info] solver flavor : rectified
    4c01f52c6171_cpu_deepdetect_1 | [2021-09-26 13:15:41.201] [dogs_cats] [info] detected network type is classification
    4c01f52c6171_cpu_deepdetect_1 | [2021-09-26 13:15:41.203] [dogs_cats] [error] resuming a model requires a .solverstate file in model repository
    4c01f52c6171_cpu_deepdetect_1 | [2021-09-26 13:15:41.219] [dogs_cats] [error] training status call failed: Dynamic exception type: dd::MLLibBadParamException
    4c01f52c6171_cpu_deepdetect_1 | std::exception::what: resuming a model requires a .solverstate file in model repository
    4c01f52c6171_cpu_deepdetect_1 |
    4c01f52c6171_cpu_deepdetect_1 | [2021-09-26 13:15:41.220] [api] [error] {"code":400,"msg":"BadRequest","dd_code":1006,"dd_msg":"Service Bad Request Error: resuming a model requires a .solverstate file in model repository"}
    
  • Different prediction with tensorrt on refinedet model for the version v0.18.0

    Different prediction with tensorrt on refinedet model for the version v0.18.0

    pred_trt_refinedet_issue.zip

    Configuration

    • Version of DeepDetect:
      • [ ] Locally compiled on:
        • [ ] Ubuntu 18.04 LTS
        • [ ] Other:
      • [ ] Docker CPU
      • [X] Docker GPU
      • [ ] Amazon AMI
    • Commit (shown by the server when starting): 23bd913ac180b56eddbf90c71d1f2e8bc2310c54

    Your question / the problem you're facing:

    I am observing weird predictions (with tensorrt and a refinedet model) associated to the last version of DeepDetect. The predictions seem really off.

    I have created a script to replicate. It will launch predictions on dd's version from v0.15.0 to v0.18.0 with and without tensorrt. Then it dumps the predictions and a hash is computed on each prediction file (we keep only the predicions' list). We observe that the v0.18.0 trt is not consistent with its caffe version or with the previous trt models.

    Please fill in the script the following env variables and make sure that you have a gpu available for testing. BASE_PATH=TODO LOGGING_FOLDER=TODO

    and then simply launch the script

    bash  pred_trt_refinedet_issue.sh
    

    You should get the following output at then end (all the docker logs are not shown here):

    Here we compute the sha256sum of the predictions obtained.
    For the caffe models nothing changes however we observe differences for the trt model of the last version of dd v0.18.0.
    Compare deepdetect_gpu
    PATH_LOGS/prediction_deepdetect_gpu_v0.15.0.json: 9e056b235be08f7245bdd324ac8ca756c41353771fcb3004df2f6b6347326d63  -
    PATH_LOGS/prediction_deepdetect_gpu_v0.16.0.json: 9e056b235be08f7245bdd324ac8ca756c41353771fcb3004df2f6b6347326d63  -
    PATH_LOGS/prediction_deepdetect_gpu_v0.17.0.json: 9e056b235be08f7245bdd324ac8ca756c41353771fcb3004df2f6b6347326d63  -
    PATH_LOGS/prediction_deepdetect_gpu_v0.18.0.json: 9e056b235be08f7245bdd324ac8ca756c41353771fcb3004df2f6b6347326d63  -
    
    Compare deepdetect_gpu_tensorrt
    PATH_LOGS/prediction_deepdetect_gpu_tensorrt_v0.15.0.json: 51767470062ecba3d77e765c34bed6000cf175400d5ff59dda9b4727356f49b5  -
    PATH_LOGS/prediction_deepdetect_gpu_tensorrt_v0.16.0.json: 51767470062ecba3d77e765c34bed6000cf175400d5ff59dda9b4727356f49b5  -
    PATH_LOGS/prediction_deepdetect_gpu_tensorrt_v0.17.0.json: 51767470062ecba3d77e765c34bed6000cf175400d5ff59dda9b4727356f49b5  -
    PATH_LOGS/prediction_deepdetect_gpu_tensorrt_v0.18.0.json: 1508b68447819ff281231ad5c757e88f4a651f50570115565438ac9fee88d566  -
    
    Expected predictions
    [
      {
        "classes": [
          {
            "last": true,
            "bbox": {
              "ymax": 350.2694091796875,
              "xmax": 745.9049682617188,
              "ymin": 108.38544464111328,
              "xmin": 528.0482788085938
            },
            "prob": 0.9999849796295166,
            "cat": "1"
          }
        ],
        "uri": "https://icour.fr/ELeveSeconde/ajout/yann_lecum_vidal/images/yann_LeCun.jpg"
      }
    ]
    
    Anormal predictions for trt v0.18.0
    [
      {
        "classes": [
          {
            "last": true,
            "bbox": {
              "ymax": 239.68505859375,
              "xmax": 425.599365234375,
              "ymin": 0,
              "xmin": 211.946044921875
            },
            "prob": 1,
            "cat": "1"
          }
        ],
        "uri": "https://icour.fr/ELeveSeconde/ajout/yann_lecum_vidal/images/yann_LeCun.jpg"
      }
    
    
tutorial on how to train deep learning models with c++ and dlib.

Dlib Deep Learning tutorial on how to train deep learning models with c++ and dlib. usage git clone https://github.com/davisking/dlib.git mkdir build

Dec 21, 2021
Caffe2 is a lightweight, modular, and scalable deep learning framework.

Source code now lives in the PyTorch repository. Caffe2 Caffe2 is a lightweight, modular, and scalable deep learning framework. Building on the origin

Dec 4, 2022
Caffe: a fast open framework for deep learning.

Caffe Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR)/The Berke

Nov 26, 2022
Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models

Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine DSSTNE (pronounced "Destiny") is an open source software library for training and deploying

Nov 30, 2022
Support Yolov4/Yolov3/Centernet/Classify/Unet. use darknet/libtorch/pytorch to onnx to tensorrt
Support Yolov4/Yolov3/Centernet/Classify/Unet. use darknet/libtorch/pytorch to onnx to tensorrt

ONNX-TensorRT Yolov4/Yolov3/CenterNet/Classify/Unet Implementation Yolov4/Yolov3 centernet INTRODUCTION you have the trained model file from the darkn

Nov 29, 2022
KSAI Lite is a deep learning inference framework of kingsoft, based on tensorflow lite

KSAI Lite English | 简体中文 KSAI Lite是一个轻量级、灵活性强、高性能且易于扩展的深度学习推理框架,底层基于tensorflow lite,定位支持包括移动端、嵌入式以及服务器端在内的多硬件平台。 当前KSAI Lite已经应用在金山office内部业务中,并逐步支持金山

Nov 2, 2022
TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.

TensorRT Open Source Software This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. Included are the sources for Tens

Dec 3, 2022
Openvino tensorflow - OpenVINO™ integration with TensorFlow
Openvino tensorflow - OpenVINO™ integration with TensorFlow

English | 简体中文 OpenVINO™ integration with TensorFlow This repository contains the source code of OpenVINO™ integration with TensorFlow, designed for T

Nov 23, 2022
Pose-tensorflow - Human Pose estimation with TensorFlow framework
Pose-tensorflow -  Human Pose estimation with TensorFlow framework

Human Pose Estimation with TensorFlow Here you can find the implementation of the Human Body Pose Estimation algorithm, presented in the DeeperCut and

Nov 23, 2022
PSTensor provides a way to hack the memory management of tensors in TensorFlow and PyTorch by defining your own C++ Tensor Class.

PSTensor : Custimized a Tensor Data Structure Compatible with PyTorch and TensorFlow. You may need this software in the following cases. Manage memory

Feb 12, 2022
This repository is a tensorrt deployment of the onsets and frames model, which is implemented using pytorch.

Onsets and Frames TensorRT inference This repository is a tensorrt deployment of the onsets and frames model, which is implemented using pytorch (http

Jan 13, 2022
A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms.
A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms.

iNeural A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms. What is a Neural Network? Work on

Apr 5, 2022
yolov5 onnx caffe

环境配置 ubuntu:18.04 cuda:10.0 cudnn:7.6.5 caffe: 1.0 OpenCV:3.4.2 Anaconda3:5.2.0 相关的安装包我已经放到百度云盘,可以从如下链接下载: https://pan.baidu.com/s/17bjiU4H5O36psGrHlF

Dec 1, 2022
Eclipse Deeplearning4J (DL4J) ecosystem is a set of projects intended to support all the needs of a JVM based deep learning application
Eclipse Deeplearning4J (DL4J) ecosystem is a set of projects intended to support all the needs of a JVM based deep learning application

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

Nov 27, 2022
This is a sample ncnn android project, it depends on ncnn library and opencv
This is a sample ncnn android project, it depends on ncnn library and opencv

This is a sample ncnn android project, it depends on ncnn library and opencv

Dec 3, 2022
GFPGAN-ncnn - a naive NCNN implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration
GFPGAN-ncnn - a naive NCNN implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration

GFPGAN-ncnn a naive ncnn implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration model support: 1.GFPGANClean

Nov 26, 2022
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.
 RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

Dec 3, 2022
Simple inference deep head pose ncnn version
Simple inference deep head pose ncnn version

ncnn-deep-head-pose Simple implement inference deep head pose ncnn version with high performance and optimized resource. This project based on deep-he

Nov 17, 2022