RDMA for GEV

Table of Contents

• • Introduction
  • Hardware requirements
  • Hardware configuration
  • Payload
  • Further Reading

<< GigEVision

Introduction

Remote Direct Memory Access (RDMA) is a transformative technology in high-performance networking that enables direct memory transfer between systems without involving their operating systems, reducing latency and CPU on the host system. CVB implements this in the context of GigEVision (GEV) while offering the known interfaces from CVBs acquisition and streaming. The term RoCEv2 is a standard for RDMA over Converged Ethernet in the version 2. For simplicity - in the context of GEV - those terms can be used interchangeably.

Hardware requirements

Beyond the known requirements for a GEV system for RDMA streaming with CVB, specialized hardware is needed.

• A network interface card (NIC) supporting RDMA/RoCEv2 technology. Some examples are the Mellanox/NVidia Connect X series (5 onwards) and Broadcom P series (e.g. P150P or P225P).
• A RDMA capable camera with support for GEV 3.0.
• Common Vision Blox (version higher than 14.1)

Note

For more support on the selection of the correct hardware, please contact STEMMER IMAGING support.

Hardware configuration

As the technology leverages several hardware features, the used devices need a configuration for seamless use.

Note

The naming of the configuration properties can vary over the different hardware vendors. In case of doubt contact STEMMER IMAGING support.

The following setup must be completed (The features wording is referring to Windows, Linux properties names may vary.):

• Install the latest device drivers and firmware for the NIC. The vendors provide the required resources to install and enable configurations.
• Set up the desired network topology and also take care of the IP configuration to make the camera available for the NIC server.
• Configure MTU size to 9k Bytes (OS networking settings)
• Enable the NetworkDirect Functionality (OS networking settings)
• Set the NetworkDirect MTU to 4k (OS networking settings)
• Change the NetworkDirect Technology to RoCEv2 (OS networking settings)
• Enable the RDMA functionality in CVB:

C++C#Python

C++

#include <cvb/device_factory.hpp>
#include <string>
 
auto devices = Cvb::DeviceFactory::Discover(Cvb::DiscoveryFlags::IgnoreVins);
auto desired = devices[0]; // for simplicity take the first
desired.SetParameter(CVB_LIT("UseRdma"), "1");

C#

using Stemmer.Cvb;
using Stemmer.Cvb.Driver;
 
var devices = DeviceFactory.Discover(DiscoverFlags.IgnoreVins);
devices[0].SetParameter("UseRdma", "1");

Python

import cvb
discover = cvb.DeviceFactory.discover_from_root(cvb.DiscoverFlags.IgnoreVins)
info: cvb.DiscoveryInformation = discover[0]
info.set_parameter("UseRdma", "1")

Note

In the GenICam Browser, the corresponding configured device option can be found under the same name.

• To have a larger receive capacity available it can help to increase the amount of flow sets, depending on your needs. See this page for an instruction to resize the flow set pool.

Warning

Be aware to have sufficient processing power available on the host system as data rates of the NIC exceed previously known capacities.

After configuration, no special actions need to be taken to use RDMA in CVB for acquisition and the same workflow for non-RDMA GEV cameras can be used (see this guide).

Payload

The hosting standard GEV version 3.0 requires the payload for RDMA streaming to be GenDC. If the payload contains a complex set of payload formats, the Cvb::CompositeStream can be used. The user doesn't have to extract the payload. In this section the stream type selection is described.

Further Reading

Setting device IPs

Getting started with GEV

GEV Standards Host: A3

CVB User Forum