I split the patch on #85 per class.

All commits in this PR can be applied separately. Tell me if there is a problem in the commit, the commit conflicts with your development or you think it should be checked more carefully. I'll update the PR except that commits.

Hi,

I've been playing around today with DSSTNE with the goal of running the example with MovieLens20M and compare the NN in the example with some state-of-the-art CF algorithms that I have implemented here. From my evaluation (which is by no means exhaustive or perfect) the example provided by DSSTNE does not seem to be competitive with respect to state of the art CF algorithms.

To summarise, I have downloaded the original MovieLens 20M dataset and I have performed a random 80%-20% partition. I have transformed the training subset to the DSSTNE format, with the only difference that I do not include the timestamps of the dataset, but 1's for all movies (is this actually very important??). I have generated recommendations with my CF algorithms (popularity, user-based and matrix factorisation) and, following the steps in the example, the predictions of DSSTNE. Finally, I have evaluated the performance with the testing subset using precision at cutoff 10.

These are the results, the configuration provided in your example does not seem to work very well: pop 0.10974162112149495 ub 0.24097987334078072 mf 0.25135912784469483 dsstne 0.056956854920365056

I am no expert in ANN's so I cannot figure out easily whether I should modify the parameters in the config.json provided in the example to make it work better. Have you compared the performance of the example with similar CF algorithms? If so, could you please share some results/insights?

Cheers Saúl

This PR includes a set of changes that will allow tests for NetCDFhelper.cpp/h to be included in the unit test suite without pulling in CUDA-related dependencies. I recommend reviewing the PR by commit, as each commit neatly captures a set of changes that I have made.

The first and most significant change was to lift the nested enum types from NNDataSet (NNTypes.h) into a separate header file (NNEnum.h). This has two advantages:

• The NNEnum.h has no external dependencies other than the STL, so it can be easily used outside of engine code. As it is, these enum values were the only declarations from NNTypes.h that were used in NetCDFhelper.
• It becomes clear which namespace/class the enum values should be qualified with. Most of the changes in this commit are to correctly qualify usage of those enum values.

The second commit tidies up NetCDFhelper.h. This should all be pretty self-explanatory.

Finally, the third commit updates the unit test build and Travis CI configuration to bring in the necessary NetCDF dependencies to compile NetCDFhelper.cpp.

Hi,

I completed the training then performed the predicting, but got below exception: Do you have any suggestions?

BTW, just one neural network got this error, others is OK.

=========== exception messages ========= Exported gl_input_predict.samplesIndex with 65075 entries. Raw max index is: 65064 Rounded up max index to: 65152 Created NetCDF file gl_input_predict.nc for dataset gl_input Number of network input nodes: 65064 Number of entries to generate predictions for: 65075 LoadNetCDF: Loading UInt data set NNDataSet::NNDataSet: Name of data set: gl_input NNDataSet::NNDataSet: Attributes: Sparse Boolean NNDataSet::NNDataSet: 1-dimensional data comprised of (65152, 1, 1) datapoints. NNDataSet::NNDataSet: 3778407 total datapoints. NNDataSet::NNDataSet: 65075 examples. [snx-dsstne:02608] *** Process received signal *** [snx-dsstne:02608] Signal: Segmentation fault (11) [snx-dsstne:02608] Signal code: Address not mapped (1) [snx-dsstne:02608] Failing at address: 0xb3a1840 [snx-dsstne:02608] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7f02cc834330] [snx-dsstne:02608] [ 1] predict[0x430d26] [snx-dsstne:02608] [ 2] predict[0x453fa0] [snx-dsstne:02608] [ 3] predict[0x42a87b] [snx-dsstne:02608] [ 4] predict[0x408307] [snx-dsstne:02608] [ 5] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f02cc480f45] [snx-dsstne:02608] [ 6] predict[0x40aab1] [snx-dsstne:02608] *** End of error message *** Segmentation fault (core dumped)

Hi, I was playing with the sample data, and now I have 3 questions.

Q1. How to make dataset with multiple feature values? Currently, only one feature has one feature value. Is it possible to a feature has multiple values? If so how can I do that?

Q2. Changing all timestamps to 1 manually giving me a different result. ml20m-all is the dataset of userId and movieId with timestamp.

userId movieId,timestamp: movieId,timestamp: movieId,timestamp…

On Issue#21, Mr.Rejith said “Currently no movie features are taken. Currently only 1/0 signals are supported from the wrapper script even though the Engine supports analog signals.” So I changed all timestamps in ml20m-all to 1, and ran DSSTNE with modified data. eg) 2,1112486027:29,1112484676:32,1112484819 to 2,1:29,1:32,1 I thought results would be the same, but it was not. I am guessing that DSSTNE treats feature value as continuous value. Is this right? Then why did DSSTNE give me a different result?

Q3. Does DSSTNE support digital inputs? On Issue#11, Mr.Rejith said “DSSTNE Engine supports analog inputs but we have not exposed it in the wrapper . if the Rating comes it could be viewed as an analog signals” Analog inputs like Rating are continuous value, so I wondered if DSSTNE supports digital inputs like category id which is discrete value.

DSSTNE is wonderful. I feel like it has so much potential. But I couldn’t figure how to use it well, and I couldn’t find detailed documentations online.

Thank you, yuasa

Hi,

I have used the 'train -n gl.nc ...' to get a network file. In the future, if I have more new sample data can be used to training. How to continue training? I used the command 'train -n gl.cn ..' again, but got below error:

Error: Network file already exists: gl.nc

Thanks.

Hi,

I am trying to setup DSSTNE on AWS following this guide but I have found problems regarding my gpu architecture

[email protected]:~/amazon-dsstne/src/amazon/dsstne$ make cd engine && make ************ RELEASE mode ************ make[1]: Entering directory /home/ubuntu/amazon-dsstne/src/amazon/dsstne/engine' nvcc -use_fast_math --ptxas-options="-v" -gencode arch=compute_50,code=sm_50 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_60,code=sm_60 -DOMPI_SKIP_MPICXX -std=c++11 -I/usr/local/cuda/include -IB40C -IB40C/KernelCommon -I/usr/local/include -I/usr/local/openmpi/include -I/usr/include/jsoncpp -I../utils -I../engine -c kernels.cu nvcc fatal : Unsupported gpu architecture 'compute_60' make[1]: *** [kernels.o] Error 1 make[1]: Leaving directory/home/ubuntu/amazon-dsstne/src/amazon/dsstne/engine' make: *** [lib/libdsstne.a] Error 2

In my EC2 instance I have previously installed TensorFlow plus CUDA, CUDNN and Bazel. Versions are: tensorflow-0.11.0rc0-cp27-none-linux_x86_64 cuda-repo-ubuntu1404_7.5-18_amd64 cudnn-7.5-linux-x64-v5.1 bazel-0.3.2

Everything seems to work fine but DSSTNE.

Setup guide saids you used hdf5 1.8.9, but it was broken and cannot be compiled(with both gcc-4.9 and gcc-5).

th5s.c:733:9: error: C++ style comments are not allowed in ISO C90
         // ret = H5Pset_alloc_time(plist_id, alloc_time);
         ^
th5s.c:733:9: error: (this will be reported only once per input file)

I think you should update document to use 1.8.12 that was fixed this compile error.

Hi All,

I am following the example with movielens and got the error as below:

train -c config.json -i gl_input.nc -o gl_output.nc -n gl.nc -b 256 -e 10

NNNetwork::NNNetwork: 1 input layer NNNetwork::NNNetwork: 1 output layer NNWeight::NNWeight: Allocating 13697024 bytes (128, 26752) for weights between Input and Hidden Error: invalid device function launching kernel kScaleAndBias_kernel GpuContext::Shutdown: Shutting down cuBLAS on GPU for process 0 GpuContext::Shutdown: CuBLAS shut down on GPU for process 0 GpuContext::Shutdown: Shutting down cuRand on GPU for process 0 GpuContext::Shutdown: CuRand shut down on GPU for process 0 GpuContext::Shutdown: Process 0 out of 1 finalized.

Could you suggest some hints to figure out what did I do wrong?

Thanks a lot.

Hello,

I made an AWS GPU instance from the ami called "amazon-dsstne" (ami-7a0df81a) and made docker image of dsstne there. I followed the example in the document. Then when running "train", it fails with the following error message:

modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/3.13.0-83-generic/modules.dep.bin' cudaGetDeviceCount failed unknown error

Thanks.

Hello all,

I am following the example with movielens and got the error

generateNetCDF -d gl_input -i ml20m-all -o gl_input.nc -f features_input -s samples_input -c
generateNetCDF: error while loading shared libraries: libnetcdf_c++4.so.1: cannot open shared object file: No such file or directory

I installed all the libraries required in the homepage (I am running Ubuntu 14.04 with GPU CUDA). Could you suggest some hints to figure out what did I do wrong?

Thanks a lot,

Hi team,

The AMI ami-fe173884 provided in the setup guide is no longer available. https://github.com/amzn/amazon-dsstne/blob/master/docs/getting_started/setup.md#ami-with-nvidia-docker

Also tried to use the EC2 Launch Instance Wizard and it results in the below error when trying to launch an EC2 instance:

We cannot proceed with your requested configuration. You cannot use this AMI (ami-25c0eb32). This AMI has been deregistered, or you do not have permission to use it. Try again with another AMI for which you have permissions, or request permission to use this AMI from its owner.

Hi, I'm trying to test dsstne module using python, following this document. Here's the complete error log: I also changed the alpha to a lower amount, but still getting malloc error. Any suggestions here?

NNLayer::Allocate: Allocating 3538944 bytes (864, 1024) of delta data for layer P3
NNLayer::Deallocate: Deallocating all data for layer Hidden10
NNLayer::Allocate: Allocating 524288 bytes (128, 1024) of unit data for layer Hidden10
NNLayer::Allocate: Allocating 524288 bytes (128, 1024) of delta data for layer Hidden10
NNLayer::Allocate: Allocating 524288 bytes (128, 1024) of dropout data for layer Hidden10
NNLayer::Deallocate: Deallocating all data for layer Output
NNLayer::Allocate: Allocating 40960 bytes (10, 1024) of unit data for layer Output
NNLayer::Allocate: Allocating 40960 bytes (10, 1024) of delta data for layer Output
NNDataSet<T>::Shard: Model Sharding sparse dataset output across all GPUs.
Getting algorithm between Input and C1
Getting algorithm between C1 and C1a
Getting algorithm between P1 and C2
Getting algorithm between C2 and C2a
Getting algorithm between P2 and C3
Getting algorithm between C3 and C3a
NNNetwork::RefreshState: Setting cuDNN workspace size to 4442259456 bytes.
GpuBuffer::Allocate failed (cudaMalloc) out of memory
python: GpuTypes.h:522: void GpuBuffer<T>::Allocate() [with T = unsigned char]: Assertion `0' failed.
[e501c14cf80e:19359] *** Process received signal ***
[e501c14cf80e:19359] Signal: Aborted (6)
[e501c14cf80e:19359] Signal code:  (-6)
[e501c14cf80e:19359] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fbe10a30390]
[e501c14cf80e:19359] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fbe1068a428]
[e501c14cf80e:19359] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fbe1068c02a]
[e501c14cf80e:19359] [ 3] /lib/x86_64-linux-gnu/libc.so.6(+0x2dbd7)[0x7fbe10682bd7]
[e501c14cf80e:19359] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x2dc82)[0x7fbe10682c82]
[e501c14cf80e:19359] [ 5] /usr/local/lib/python2.7/dist-packages/dsstne.so(_ZN9NNNetwork12RefreshStateEv+0x4a0)[0x7fbdffeebe80]
[e501c14cf80e:19359] [ 6] /usr/local/lib/python2.7/dist-packages/dsstne.so(_ZN9NNNetwork5TrainEjfffff+0x69)[0x7fbdffef5289]
[e501c14cf80e:19359] [ 7] /usr/local/lib/python2.7/dist-packages/dsstne.so(_ZN18NNNetworkFunctions5TrainEP7_objectS1_+0xfb)[0x7fbdffe3cccb]
[e501c14cf80e:19359] [ 8] python(PyEval_EvalFrameEx+0x5ca)[0x4bc9ba]
[e501c14cf80e:19359] [ 9] python(PyEval_EvalCodeEx+0x306)[0x4ba036]
[e501c14cf80e:19359] [10] python[0x4eb32f]
[e501c14cf80e:19359] [11] python(PyRun_FileExFlags+0x82)[0x4e5592]
[e501c14cf80e:19359] [12] python(PyRun_SimpleFileExFlags+0x186)[0x4e3e46]
[e501c14cf80e:19359] [13] python(Py_Main+0x54e)[0x493ade]
[e501c14cf80e:19359] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fbe10675830]
[e501c14cf80e:19359] [15] python(_start+0x29)[0x4934a9]
[e501c14cf80e:19359] *** End of error message ***
Aborted (core dumped)

So I have some questions about input data and updating the trained model:

I've seen in the documentation that we should provide a csv file and then the input data format will be changed to the desired one using generateNETCDF. But I want to know if it's possible to feed streaming data to this engine, and if yes then how? Assuming a website which has many users that their data is getting changed and updated every moment, also new users join the website and they're gonna have new profiles and producing new data, So then how is the model going to be updated?

Regards,

Issue #, if available: No issue

*Description of changes: Adds NVLINK support through DGX-1V, and enables more efficient sparse kernels for smaller embedding widths.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

it threw:

nvcc -O3 -std=c++11 --compiler-options=-fPIC -use_fast_math --ptxas-options="-v" -gencode arch=compute_70,code=sm_70 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_30,code=sm_30 -DOMPI_SKIP_MPICXX --keep-dir /test/workspace/ctr/amazon-dsstne/build/tmp/engine/cuda -I/usr/local/include -isystem /usr/local/cuda/include -isystem /usr/lib/openmpi/include -isystem /usr/include/jsoncpp -IB40C -IB40C/KernelCommon -I/test/workspace/ctr/amazon-dsstne/build/include -I../utils  -c kernels.cu -o /test/workspace/ctr/amazon-dsstne/build/tmp/engine/cuda/kernels.o
ptxas /tmp/tmpxft_00000a17_00000000-8_kernels.compute_70.ptx, line 61962; error   : Instruction 'shfl' without '.sync' is not supported on .target sm_70 and higher from PTX ISA version 6.4

the old CUDA shuffle API have been deprecated in CUDA 9.0, and not available after CUDA 10 (link)

• CUDA: 10.1
• GPU: NVIDIA Tesla V100