Cloud Native Data Plane (CNDP) is a collection of user space libraries to accelerate packet processing for cloud applications.

CNDP - Cloud Native Data Plane

Overview

Cloud Native Data Plane (CNDP) is a collection of userspace libraries for accelerating packet processing for cloud applications. It aims to provide better performance than that of standard network socket interfaces by taking advantage of platform technologies such as Intel(R) AVX-512, Intel(R) DSA, CLDEMOTE, etc. The I/O layer is primarily built on AF_XDP, an interface that delivers packets straight to userspace, bypassing the kernel networking stack. CNDP provides ways to expose metrics and telemetry with examples to deploy network services on Kubernetes.

CNDP Consumers

  • Cloud Network Function (CNF) and Cloud Application developers: Those who create applications based on CNDP. CNDP hides the low-level I/O, allowing the developer to focus on their application.

  • CNF and Cloud Application consumers: Those who consume the applications developed by the CNF developer. CNDP showcases deployment models for their applications using Kubernetes.

CNDP Characteristics

CNDP follows a set of principles:

  • Functionality: Provide a framework for cloud native developers that offers full control of their application.

  • Usability: Simplify cloud native application development to enable the developer to create applications by providing APIs that abstract the complexities of the underlying system while still taking advantage of acceleration features when available.

  • Interoperability: The CNDP framework is built primarily on top of AF_XDP. Other interfaces, such as memif, are also supported, however building on AF_XDP ensures it is possible to move an application across environments wherever AF_XDP is supported.

  • Portability/stability: CNDP provides ABI stability and a common API to access network interfaces.

  • Performance: Take advantage of platform technologies to accelerate packet processing or fall-back to software when acceleration is unavailable.

  • Observability: Provide observability into the performance and operation of the application.

  • Security: Security for deployment in a cloud environment is critical.

CNDP background

CNDP was created to enable cloud native developers to use AF_XDP and other interfaces in a simple way while providing better performance as compared to standard Linux networking interfaces.

CNDP does not replace DPDK (Data Plane Development Kit), which provides the highest performance for packet processing. DPDK implements user space drivers, bypassing the kernel drivers. This approach of rewriting drivers is one reason DPDK achieves the highest performance for packet processing. DPDK also implements a framework to initialize and setup platform resources i.e. scanning PCI bus for devices, allocating memory via hugepages, setting up Primary/Secondary process support, etc.

In contrast to DPDK, CNDP does not have custom drivers. Instead it expects the kernel drivers to implement AF_XDP, preferably in zero-copy mode. Since there are no PCIe drivers, there's no PCI bus scanning, and does not require physically contiguous and pinned memory. This simplifies deployment for cloud native applications while gaining the performance benefits provided by AF_XDP.

CNDP notable directories

The following shows a subset of the directory structure.

.
├── ansible                   # Ansible playbook to install in a system(s)
├── containerization          # Container configuration and setup scripts for Docker/K8s
├── doc                       # Documentation APIs, guides, getting started, ...
├── examples                  # Example applications to understand how to use CNDP features
├── lang                      # Language bindings and examples
│   ├── go                    # Go Language bindings to CNDP and tools (WIP)
│   └── rs                    # Rust Language bindings for CNDP/Wireguard (WIP)
├── lib                       # Set of libraries for building CNDP applications
│   ├── cnet                  # Userspace network stack
│   ├── common                # Libraries used by core and applications libraries
│   ├── core                  # Core libraries for CNDP
│   ├── include               # Common headers for CNDP and applications
│   └── usr                   # User set of libraries that are optional for developer
├── test                      # Unit test framework
│   ├── common                # Common test code
│   ├── fuzz                  # Fuzzing (WIP)
│   └── testcne               # Functional unit testing application
├── tools                     # Tools for building CNDP
│   └── vscode                # Configuration files for vscode
└── usrtools                  # Tools for users
    ├── cnectl                # Remote CLI for CNDP applications
    └── txgen                 # Traffic Generator using AF_XDP and CNDP
Owner
Cloud Native Data Plane
Cloud Native Data Plane (CNDP)
Cloud Native Data Plane
Comments
  • Rust language bindings and directory structure needs cleanup and restructure

    Rust language bindings and directory structure needs cleanup and restructure

    Hi Folks I recently started looking through the Rust language bindings for CNDP and ran into a few issues. Following the readme things don't work out of the box. In addition the directory structure is completely confusing (there's no real structure to what's an example vs what's the set of libraries I can use as part of my Rust application). Also I believe the naming should be libcndp-sys for the directory that contains the language bindings (cne is only a small library set of what CNDP has to offer).

    The current dir structure is shown below:

    .
    ├── README.md
    ├── bindings
    │   ├── cne
    │   │   ├── Cargo.toml
    │   │   ├── README.md
    │   │   ├── examples
    │   │   │   └── loopback
    │   │   │       ├── Cargo.toml
    │   │   │       ├── run.sh
    │   │   │       └── src
    │   │   │           └── main.rs
    │   │   ├── fwd.jsonc
    │   │   ├── run_test.sh
    │   │   └── src
    │   │       ├── config.rs
    │   │       ├── error.rs
    │   │       ├── instance.rs
    │   │       ├── lib.rs
    │   │       ├── packet.rs
    │   │       ├── port.rs
    │   │       └── util.rs
    │   ├── cne-sys
    │   │   ├── Cargo.toml
    │   │   ├── README.md
    │   │   ├── build.rs
    │   │   └── src
    │   │       ├── bindings
    │   │       │   ├── bindings.c
    │   │       │   ├── bindings.h
    │   │       │   ├── meson.build
    │   │       │   └── wrapper.h
    │   │       └── lib.rs
    │   └── examples
    │       └── echo_server
    │           ├── Cargo.toml
    │           ├── fwd.jsonc
    │           ├── run.sh
    │           └── src
    │               └── main.rs
    ├── helloworld
    │   └── main.rs
    ├── mmap
    │   ├── Cargo.toml
    │   └── src
    │       └── lib.rs
    ├── mmap-cndp
    │   ├── Cargo.toml
    │   ├── build.rs
    │   ├── src
    │   │   └── lib.rs
    │   └── wrapper.h
    ├── pktfwd
    │   ├── Cargo.toml
    │   ├── build.rs
    │   ├── fwd.jsonc
    │   ├── jcfg_parse
    │   │   ├── fwd.h
    │   │   ├── meson.build
    │   │   ├── parse-jsonc.c
    │   │   ├── rust_helper.c
    │   │   ├── rust_helper.h
    │   │   └── stats.c
    │   ├── runcmd.sh
    │   ├── src
    │   │   ├── cndp.rs
    │   │   ├── main.rs
    │   │   ├── packet.rs
    │   │   └── util.rs
    │   └── wrapper.h
    ├── ring_test
    │   ├── Cargo.toml
    │   ├── build.rs
    │   ├── src
    │   │   ├── lib.rs
    │   │   └── main.rs
    │   └── wrapper.h
    └── wireguard
        └── patch
            ├── 0001-Integrate-CNDP-Cloud-Native-Data-Plane-with-Wireguar.patch
            ├── 0002-Rename-variable-private-to-priv_-to-fix-build-error.patch
            └── 0003-Remove-extra-argument-from-pktmbuf_dump.patch
    

    Finally - there's only a handful of APIs that have been converted to Rust and the naming is separate to the C libraries - so tracking what's where is going to be difficult.

    Is there a plan to cleanup the Rust implementation and port more libraries?

  • idle manager to enable interrupt mode polling

    idle manager to enable interrupt mode polling

    The current design in CNDP is 100% polling of the receive ring in AF_XDP or PMD which means burning core power. The patch adds two parameters to the jsonc file to control polling idle_timeout and intr_timeout in thread section.

    The idlemgr library handles managing epoll and calling epoll_wait when the idle_timeout value has been met, then it calls epoll_wait(). The idlemgr operates on threads, using file descriptors as the method to wakeup a sleeping thread. The caller needs to add file descriptors to the idlemgr instance and then call idlemgr_process() with a flag if the thread thinks it is idle or not.

    The first one gives the number of milliseconds to wait for the RX ring to be idle, then call epoll() with a timeout of intr_timeout. When no Rx traffic for a given time the idlemgr will call epoll(), which reduces the lcore load to effectively zero and only waking up when packets arrive or a timeout occurs.

    In testing of performance it appears to be very little impact when interrupt mode is enabled compared to when it is not enabled. Added some counters to help determine how the new mode is operating.

    The idle_timeout value in the jsonc file for given thread is how this feature is controlled. If not defined or set to zero interrupt mode is disabled. When set to a non zero value will enable interrupt mode. The intr_timeout value is only used if idle_timeout is non-zero and will be used in the poll() call as the timeout value. Each of these values are in milliseconds. Signed-off-by: Keith Wiles [email protected]

  • bpf: add libbpf 8.0 support

    bpf: add libbpf 8.0 support

    • [x] Baseline support for libxdp 1.2.5 and libbpf 0.8.0
    • [x] Update readme to show how to build and install libxdp 1.2.5 and libbpf 0.8.0
    • [x] Update implementation to use USE_LIBXDP and USE_LIBBPF_8 directly
  • build:enable building static executables

    build:enable building static executables

    Enable the build system to create statically linked executables. Currently we build both static and shared libraries, but the example executables were always dynamically linked executables. Change the meson.build files to build only static or shared libraries, the default is to build shared libraries.

    To simpify the meson.build dependencies and utilize the newer dependency() function it is now required to use meson >= 0.60.0 version.

    Adding 'static_build=1' to the make line 'make static_build=1 rebuild install' will rebuild all of CNDP as a set of static libraries and executables. A new cne-build.sh command line option was created called 'static' to build the system e.g.: 'cne-build.sh static clean build'

    Added a new makefile target to 'make help' to display the usage help text.

    Building a debug image can also be built statically using the static_build=1 on the make command line e.g.: 'make clean static debug'

    Building as static makes the executables larger, but they do not require using the LD_LIBRARY_PATH variable, which means 'sudo ./usr/local/bin/test-cne ring' should work as expected.

    Signed-off-by: Keith Wiles [email protected]

  • Update wireguard patches to use Rust CNE library.

    Update wireguard patches to use Rust CNE library.

    • Wireguard patches updated to use Rust CNE library.
    • Fix build issues on benchmark tests.

    Signed-off-by: Manoj Gopalakrishnan [email protected]

  • graph_test and graph_perf_test improvements

    graph_test and graph_perf_test improvements

  • go bindings test fail

    go bindings test fail

    lang/go/bindings/cne$ ./run_test === RUN TestStarting --- PASS: TestStarting (0.33s) === RUN TestGetChannel === RUN TestGetChannel/TestLoopback === RUN TestGetChannel/TestLoopback#01 --- PASS: TestGetChannel (0.00s) --- PASS: TestGetChannel/TestLoopback (0.00s) --- PASS: TestGetChannel/TestLoopback#01 (0.00s) === RUN TestRxBurst === RUN TestRxBurst/TestRxBurst cne.test: ../lib/core/cne/cne.c:28: cne_id: Assertion `per_thread__cne != NULL' failed. SIGABRT: abort PC=0x7fdce9f0000b m=3 sigcode=18446744073709551610 signal arrived during cgo execution

    The same thing happens when I run OpenWIthConfig() in my own code.

  • Fix a compiling error on RHEL/Rocky 9

    Fix a compiling error on RHEL/Rocky 9

    We got a compile error when we moved to RHEL/Rocky 9 recently, the proposed code is to disable "uintr" flag.

    [20/377] Compiling C object lib/usr/slib/ibroker/libcne_ibroker.so.p/uintr_handler.c.o FAILED: lib/usr/slib/ibroker/libcne_ibroker.so.p/uintr_handler.c.o cc -Ilib/usr/slib/ibroker/libcne_ibroker.so.p -Ilib/usr/slib/ibroker -I../lib/usr/slib/ibroker -I. -I.. -Ilib/include -I../lib/include -Ilib/core/osal -I../lib/core/osal -Ilib/core/log -I../lib/core/log -Ilib/core/ring -I../lib/core/ring -Ilib/usr/clib/timer -I../lib/usr/clib/timer -Ilib/core/cne -I../lib/core/cne -I/usr/include/libnl3 -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -Werror -O3 -g -muintr -Wno-pedantic -Wcast-qual -Wdeprecated -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned -Wno-missing-field-initializers -D_GNU_SOURCE -include cne_build_config.h -march=native -DCC_AVX2_SUPPORT -fPIC -MD -MQ lib/usr/slib/ibroker/libcne_ibroker.so.p/uintr_handler.c.o -MF lib/usr/slib/ibroker/libcne_ibroker.so.p/uintr_handler.c.o.d -o lib/usr/slib/ibroker/libcne_ibroker.so.p/uintr_handler.c.o -c ../lib/usr/slib/ibroker/uintr_handler.c /tmp/ccvboPZ5.s: Assembler messages: /tmp/ccvboPZ5.s:136: Error: no such instruction: uiret' [21/377] Compiling C object lib/usr/clib/nodes/libcne_nodes.so.p/pkt_drop.c.o …… [73/377] Compiling C object lib/usr/slib/ibroker/libcne_ibroker.so.p/ibroker.c.o FAILED: lib/usr/slib/ibroker/libcne_ibroker.so.p/ibroker.c.o cc -Ilib/usr/slib/ibroker/libcne_ibroker.so.p -Ilib/usr/slib/ibroker -I../lib/usr/slib/ibroker -I. -I.. -Ilib/include -I../lib/include -Ilib/core/osal -I../lib/core/osal -Ilib/core/log -I../lib/core/log -Ilib/core/ring -I../lib/core/ring -Ilib/usr/clib/timer -I../lib/usr/clib/timer -Ilib/core/cne -I../lib/core/cne -I/usr/include/libnl3 -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -Werror -O3 -g -muintr -Wno-pedantic -Wcast-qual -Wdeprecated -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned -Wno-missing-field-initializers -D_GNU_SOURCE -include cne_build_config.h -march=native -DCC_AVX2_SUPPORT -fPIC -MD -MQ lib/usr/slib/ibroker/libcne_ibroker.so.p/ibroker.c.o -MF lib/usr/slib/ibroker/libcne_ibroker.so.p/ibroker.c.o.d -o lib/usr/slib/ibroker/libcne_ibroker.so.p/ibroker.c.o -c ../lib/usr/slib/ibroker/ibroker.c /tmp/cccltTdo.s: Assembler messages: /tmp/cccltTdo.s:254: Error: no such instruction:stui'

    Mark Liu

  • make static_build=1 rebuild fails

    make static_build=1 rebuild fails

    Using Ubuntu 20.04 setup using ansible

    "/home/canopus/cndp/tools/cne-build.sh" clean static build Build environment variables and paths: CNE_SDK_DIR : /home/canopus/cndp CNE_TARGET_DIR : usr/local CNE_BUILD_DIR : /home/canopus/cndp/builddir CNE_DEST_DIR : /home/canopus/cndp PKG_CONFIG_PATH : /usr/lib64/pkgconfig build_path : /home/canopus/cndp/builddir target_path : /home/canopus/cndp/usr/local

    *** Removing '/home/canopus/cndp/builddir' directory

    Static build in '/home/canopus/cndp/builddir' Release build in '/home/canopus/cndp/builddir' Ninja build in '/home/canopus/cndp/builddir' buildtype='release' The Meson build system Version: 0.53.2 Source dir: /home/canopus/cndp Build dir: /home/canopus/cndp/builddir Build type: native build Program cat found: YES (/bin/cat) Project name: CNDP Project version: 22.08.0 C++ compiler for the host machine: c++ (gcc 9.4.0 "c++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0") C++ linker for the host machine: c++ ld.bfd 2.34 C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0") C linker for the host machine: cc ld.bfd 2.34 Host machine cpu family: x86_64 Host machine cpu: x86_64 Checking for size of "void *" : 8 meson.build:91: WARNING: Consider using the built-in debug option instead of using "-g". Has header "execinfo.h" : YES Found pkg-config: /usr/bin/pkg-config (0.29.1) Did not find CMake 'cmake' Found CMake: NO Run-time dependency libexecinfo found: NO (tried pkgconfig and cmake) Run-time dependency numa found: NO (tried pkgconfig and cmake) Run-time dependency libbsd found: YES 0.10.0 Run-time dependency json-c found: YES 0.13.1 Run-time dependency libpcap found: YES 1.9.1 Run-time dependency libnl-3.0 found: YES 3.4.0 Run-time dependency libnl-cli-3.0 found: YES 3.4.0 Run-time dependency libnl-route-3.0 found: YES 3.4.0 Run-time dependency libxdp found: NO (tried pkgconfig) Run-time dependency libbpf found: YES 0.3.0 Has header "linux/if_xdp.h" : YES Has header "bpf/xsk.h" : YES Has header "bpf/bpf.h" : YES Run-time dependency libbpf found: YES 0.3.0 Compiler for C supports arguments -muintr: NO Run-time dependency libdlb found: NO (tried pkgconfig) Compiler for C supports arguments -Wno-pedantic -Wpedantic: YES Compiler for C++ supports arguments -Wno-pedantic -Wpedantic: YES Compiler for C supports arguments -Wcast-qual: YES Compiler for C++ supports arguments -Wcast-qual: YES Compiler for C supports arguments -Wdeprecated: YES Compiler for C++ supports arguments -Wdeprecated: YES Compiler for C supports arguments -Wformat-nonliteral: YES Compiler for C++ supports arguments -Wformat-nonliteral: YES Compiler for C supports arguments -Wformat-security: YES Compiler for C++ supports arguments -Wformat-security: YES Compiler for C supports arguments -Wmissing-declarations: YES Compiler for C++ supports arguments -Wmissing-declarations: YES Compiler for C supports arguments -Wmissing-prototypes: YES Compiler for C++ supports arguments -Wmissing-prototypes: NO Compiler for C supports arguments -Wnested-externs: YES Compiler for C++ supports arguments -Wnested-externs: NO Compiler for C supports arguments -Wold-style-definition: YES Compiler for C++ supports arguments -Wold-style-definition: NO Compiler for C supports arguments -Wpointer-arith: YES Compiler for C++ supports arguments -Wpointer-arith: YES Compiler for C supports arguments -Wsign-compare: YES Compiler for C++ supports arguments -Wsign-compare: YES Compiler for C supports arguments -Wstrict-prototypes: YES Compiler for C++ supports arguments -Wstrict-prototypes: NO Compiler for C supports arguments -Wundef: YES Compiler for C++ supports arguments -Wundef: YES Compiler for C supports arguments -Wwrite-strings: YES Compiler for C++ supports arguments -Wwrite-strings: YES Compiler for C supports arguments -Wno-address-of-packed-member -Waddress-of-packed-member: YES Compiler for C++ supports arguments -Wno-address-of-packed-member -Waddress-of-packed-member: YES Compiler for C supports arguments -Wno-packed-not-aligned -Wpacked-not-aligned: YES Compiler for C++ supports arguments -Wno-packed-not-aligned -Wpacked-not-aligned: YES Compiler for C supports arguments -Wno-missing-field-initializers -Wmissing-field-initializers: YES Compiler for C++ supports arguments -Wno-missing-field-initializers -Wmissing-field-initializers: YES Fetching value of define "SSE4_2" : 1 Fetching value of define "AES" : 1 Fetching value of define "PCLMUL" : 1 Fetching value of define "AVX" : 1 Fetching value of define "AVX2" : 1 Fetching value of define "AVX512F" : Fetching value of define "RDRND" : 1 Fetching value of define "RDSEED" : 1 Fetching value of define "AVX512F" : (cached) Fetching value of define "AVX512DQ" : Message: *** IPv4 dump output disabled Message: *** TCP support disabled Message: *** Enable support for punting to Linux kernel stack Message: *** TCP dump output disabled Configuring cne_build_config.h using configuration Fetching value of define "AVX2" : 1 (cached) Fetching value of define "AVX512F" : (cached) Fetching value of define "AVX512VL" : Fetching value of define "AVX512CD" : Fetching value of define "AVX512BW" : Compiler for C supports arguments -mavx512f -mavx512vl -mavx512cd -mavx512bw: YES Compiler for C supports arguments -mavx512f -mavx512dq: YES Message: *** have multiple avx512f and avx512dq Compiler for C supports arguments -mavx512bw: YES Compiler for C supports arguments -mavx512bw: YES Program python3 found: YES (/usr/bin/python3) Program doxygen found: YES (/usr/bin/doxygen) Program generate_doxygen.sh found: YES (/home/canopus/cndp/doc/api/generate_doxygen.sh) Program generate_examples.sh found: YES (/home/canopus/cndp/doc/api/generate_examples.sh) Program doxy-html-custom.sh found: YES (/home/canopus/cndp/doc/api/doxy-html-custom.sh) Configuring doxy-api.conf using configuration Program sphinx-build found: YES (/usr/bin/sphinx-build) Message: Fuzz tests require clang. Fuzz tests will not be built. Message: **** Missing Linux source code, not building afxdp_user example:/usr/src/linux-source-5.13.0 Library quicly found: NO Message: **** Quicly not found at /work/projects/intel/networking.dataplane/quicly Update meson_options.txt file. The quic-echo example cannot be built. Message: DLB test build missing library Program tools/mklib.sh found: YES (/home/canopus/cndp/tools/mklib.sh) Program cp found: YES (/bin/cp) Program cp found: YES (/bin/cp) Message: >>> Create pkg-config file Message: <<< Done pkg-config file Build targets in project: 69

    Option default_library is: static [default: shared] Found ninja-1.10.0 at /usr/bin/ninja ninja: Entering directory /home/canopus/cndp/builddir' [75/309] Linking target lib/core/pmds/net/af_xdp/libpmd_af_xdp.so. FAILED: lib/core/pmds/net/af_xdp/libpmd_af_xdp.so cc -o lib/core/pmds/net/af_xdp/libpmd_af_xdp.so 'lib/core/pmds/net/af_xdp/[email protected]@[email protected]/pmd_af_xdp.c.o' -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,libpmd_af_xdp.so -Wl,--no-as-needed -pthread -lm -ldl -lbsd -ljson-c -lpcap -lnl-3 -lnl-cli-3 -lnl-route-3 lib/core/osal/libcne_osal.a lib/core/log/libcne_log.a lib/core/mmap/libcne_mmap.a lib/core/cne/libcne_cne.a lib/core/kvargs/libcne_kvargs.a lib/core/pktdev/libcne_pktdev.a lib/core/mempool/libcne_mempool.a lib/core/ring/libcne_ring.a lib/core/pktmbuf/libcne_pktmbuf.a lib/common/uds/libcne_uds.a lib/core/xskdev/libcne_xskdev.a /usr/lib/libbpf.a /usr/lib/x86_64-linux-gnu/libelf.a /usr/lib/x86_64-linux-gnu/libz.a -Wl,--end-group '-Wl,-rpath,$ORIGIN/../../../osal:$ORIGIN/../../../log:$ORIGIN/../../../mmap:$ORIGIN/../../../cne:$ORIGIN/../../../kvargs:$ORIGIN/../../../pktdev:$ORIGIN/../../../mempool:$ORIGIN/../../../ring:$ORIGIN/../../../pktmbuf:$ORIGIN/../../../../common/uds:$ORIGIN/../../../xskdev' -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/osal -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/log -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/mmap -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/cne -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/kvargs -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/pktdev -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/mempool -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/ring -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/pktmbuf -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/common/uds -Wl,-rpath-link,/home/canopus/cndp/builddir/lib/core/xskdev /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libelf.a(elf_error.o): relocation R_X86_64_TPOFF32 againstglobal_error' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/libbpf.a(libbpf.o): relocation R_X86_64_PC32 against symbol `[email protected]@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status [84/309] Compiling C object 'lib/usr/clib/acl/[email protected]@[email protected]/acl_run_avx512.c.o'. ninja: build stopped: subcommand failed. make: *** [Makefile:46: rebuild] Error 1

  • fix up performance drop for FQ

    fix up performance drop for FQ

    The FQ routine fq_add() was allocating mbufs and freeing them when the FQ was full, this caused a performance drop. The code will now determine if the FQ needs mbufs and how many before allocating the mbufs from the pktmbuf pool.

    During testing of tx only cndpfwd and the idlemgr was enabled causing the thread to sleep due to no RX traffic. Modified the txonly routine to return 1 to make it look like the port was active to idlemgr.

    Signed-off-by: Keith Wiles [email protected]

  • Interrupt mode support via idlemgr

    Interrupt mode support via idlemgr

    add idle manager library and test-cne test

    The current design in CNDP is 100% polling of the receive ring in AF_XDP or PMD which means burning core power. The patch adds two parameters to the jsonc file to control polling idle_timeout and intr_timeout in thread section.

    The idlemgr library handles managing epoll and calling epoll_wait when the idle_timeout value has been met, then it calls epoll_wait(). The idlemgr operates on threads, using file descriptors as the method to wakeup a sleeping thread. The caller needs to add file descriptors to the idlemgr instance and then call idlemgr_process() with a flag if the thread thinks it is idle or not.

    The first one gives the number of milliseconds to wait for the RX ring to be idle, then call epoll() with a timeout of intr_timeout. When no Rx traffic for a given time the idlemgr will call epoll(), which reduces the lcore load to effectively zero and only waking up when packets arrive or a timeout occurs. In testing of performance it appears to be very little impact when interrupt mode is enabled compared to when it is not enabled. Added some counters to help determine how the new mode is operating.

    The idle_timeout value in the jsonc file for given thread is how this feature is controlled. If not defined or set to zero interrupt mode is disabled. When set to a non zero value will enable interrupt mode. The intr_timeout value is only used if idle_timeout is non-zero and will be used in the poll() call as the timeout value. Each of these values are in milliseconds.

    Signed-off-by: Keith Wiles [email protected]

  • Add L3 forwarding to cndpfwd example

    Add L3 forwarding to cndpfwd example

    This commit enhances the cndpfwd example application by adding a new l3-fwd mode. In this mode, incoming packets are forwarded based on a FIB defined in fwd.jsonc, in the options:l3fwd-fib-rules section. Each FIB entry consists of an IP/prefix to match, a destination MAC for the next hop, and a TX port to use.

    Targets #231.

  • Add docker build cache to github actions cache to improve CI build times

    Add docker build cache to github actions cache to improve CI build times

    Potential enhancement idea for future work - it seems like it should be possible to cache part of the docker build layers to improve build times using the GitHub Actions cache system. The docker build is far slower than all the other build steps combined.

    Tricky part is to do so in a way that doesn't eventually fill up the actions cache. We were thinking about trying to do this but ran out of time. Here's a decent article with a possible approach to this - https://evilmartians.com/chronicles/build-images-on-github-actions-with-docker-layer-caching

    Maintainers, feel free to close or comment on this if you would be interested in this in the future.

  • Add InsertBraces: false to .clang-format

    Add InsertBraces: false to .clang-format

    Update .clang-format file to include the new option InsertBraces: false when clang 15 is available in Fedora + Ubuntu.

    https://clang.llvm.org/docs/ClangFormatStyleOptions.html

    this will add the check for {} in single line statements

  • Do CNDP will support jumbo frames and when do CNDP would support pktmbuf chaining ?

    Do CNDP will support jumbo frames and when do CNDP would support pktmbuf chaining ?

    Hello,

    I'm analyzing porting dpdk apps to cndp. We need jumbo frames (up to 9KB) and we use chaining pktmbuf functions. It seems that XDP supports jumbo frames using multi-buffer. Is it planned to support it in CNDP ? Moreover, when do pktmbuf chaining capability would be available please ?

    Thanks in advance for your answer.

    Niry

  • receiving zero length packets

    receiving zero length packets

    Just with a simple go program main.go.txt that loops receiving packets, I am getting zero length packets (even at a low rate of 100pps, with all > 124bytes). This is on an interface with just 1 channel configured, running on ubuntu (setup with ansible scripts). If I up the bufcnt to 8 I don't see these anymore.

Related tags
A collection of user-space Linux kernel specific guided fuzzers based on LKL

kBdysch kBdysch is a collection of fast Linux kernel specific fuzzing harnesses supposed to be run in userspace in a guided fuzzing manner. It was des

Nov 25, 2022
Sloth 🦥 is a coverage guided fuzzing framework for fuzzing Android Native libraries that makes use of libFuzzer and QEMU user-mode emulation
Sloth 🦥 is a coverage guided fuzzing framework for fuzzing Android Native libraries that makes use of libFuzzer and QEMU user-mode emulation

Sloth ?? Sloth is a fuzzing setup that makes use of libFuzzer and QEMU’s user-mode emulation (qemu/linux-user) on x86_64/aarch64 host to emulate aarch

Nov 29, 2022
Visual Studio native debugger extension to help debug native applications using Mono.
Visual Studio native debugger extension to help debug native applications using Mono.

Unity Mixed Callstack UnityMixedCallstack is a Visual Studio 2017/2019 extension to help debug native applications embedding Mono, like Unity. If you

Nov 28, 2022
A native plugin for Unity that provides simple packet division and restoration.

uPacketDivision This is a native plug-in that divides a given data (System.IntPtr or array) into specified sizes and restores them regardless of the i

Feb 20, 2022
Suite of C++ libraries for radio astronomy data processing

Casacore A suite of C++ libraries for radio astronomy data processing. Installation Debian / Ubuntu Casacore is now part of Debian and Ubuntu, use apt

Jan 7, 2023
A data plane framework that supports any layer-7 protocols.
A data plane framework that supports any layer-7 protocols.

中文 meta-protocol-proxy Why MetaProtocol is needed? Almost all open source and commercial Service Meshes currently support only two Layer-7 protocols -

Oct 17, 2022
A framework for implementing block device drivers in user space

BDUS is a Linux 4.0+ framework for developing block devices in user space. More specifically, it enables you to implement block device drivers as regu

Dec 13, 2022
our supper awesome kernel and user space system

osakauss our super awesome kernel and user space system memory layout The kernel is loaded at 0x00100000. kmalloc initially uses 'placement' allocatio

Aug 26, 2021
User space configuration tool for RME HDSPe MADI / AES / RayDAT / AIO and AIO Pro cards driven by the snd-hdspe driver.

hdspeconf User space configuration tool for RME HDSPe MADI / AES / RayDAT / AIO and AIO Pro cards, driven by the snd-hdspe driver. Building hdspeconf

Nov 29, 2022
Project Etnaviv is an open source user-space driver for the Vivante GCxxx series of embedded GPUs.

Introduction Project Etnaviv is an open source user-space driver for the Vivante GCxxx series of embedded GPUs. This repository contains reverse-engin

Oct 29, 2022
Selective user space swap (kubernetes swap / kubeswap)
Selective user space swap (kubernetes swap / kubeswap)

BigMaac ?? ?? ( Big Malloc Access And Calloc ) because sometimes a happy meal is not big enough BigMaac can be used in userspace (e.g. inside Kubernet

Dec 9, 2022
POCO C++ Libraries are powerful cross-platform C++ libraries for building network
POCO C++ Libraries are powerful cross-platform C++ libraries for building network

The POCO C++ Libraries are powerful cross-platform C++ libraries for building network- and internet-based applications that run on desktop, server, mobile, IoT, and embedded systems.

Jan 1, 2023
Recognize stairs with lidar. Project the laser points to X-Z plane and use least squares for linear fitting.
Recognize stairs with lidar. Project the laser points to X-Z plane and use least squares for linear fitting.

stairs_recogniton Recognize stairs with lidar. Project the laser points to X-Z plane and use least squares for linear fitting. Dependencies PCL 1.8 Ei

Nov 25, 2022
Mod - MASTERS of DATA, a course about videogames data processing and optimization
Mod - MASTERS of DATA, a course about videogames data processing and optimization

MASTERS of DATA Welcome to MASTERS of DATA. A course oriented to Technical Designers, Technical Artists and any game developer that wants to understan

Dec 28, 2022
A repository for experimenting with elf loading and in-place patching of android native libraries on non-android operating systems.

droidports: A repository for experimenting with elf loading and in-place patching of android native libraries on non-android operating systems. Discla

Dec 15, 2022
React Native polyfill for crypto.getRandomValues. Used in libraries like uuid.

react-native-random-values-jsi-helper React Native polyfill for crypto.getRandomValues. Used in libraries like uuid. Installation yarn add react-nativ

Oct 30, 2022
A collection of post-processing shaders written for ReShade.

ReShade FX shaders This repository aims to collect post-processing shaders written in the ReShade FX shader language. Installation Download this repos

Dec 30, 2022
Collection of cross-platform single-header C libraries for doing a lot of stuff! (Still WIP)

ice_libs Collection of cross-platform single-header C libraries for doing a lot of stuff! (Still WIP) Brief ice_libs is collection of Single-Header C

Dec 6, 2022
A collection of tools, libraries, and tests for Vulkan shader compilation.

Shaderc A collection of tools, libraries and tests for shader compilation. At the moment it includes: glslc, a command line compiler for GLSL/HLSL to

Dec 29, 2022