How-to: Build VPP FD.IO with Mellanox DPDK PMD on top CentOS 7.7 with inbox drivers.

https://docs.mellanox.com/pages/releaseview.action?pageId=15053848

1. Install CentOS 7.7

2. Set up an account to use the Linux Foundation resources. To set up the account, refer to The Linux Foundation website: https://identity.linuxfoundation.org.

3. Set up a Gerrit account and apply an ssh public key. Follow the instructions on the FD.IO wiki web page: https://wiki.fd.io/view/DEV/Setting_up_Gerrit

4. Install the following prerequisites packages:

Cli command

yum install -y git cpp gcc rpm-build openssl-devel libmnl-devel numactl-devel epel-release net-tools rdma-core-devel nasm  

Installation, Compilation and Configuration

1. Download VPP main master branch.

Cli commands

git clone ssh://<USERNAME>@gerrit.fd.io:29418/vpp.git

git checkout origin/stable/<"Release_version_number">

Use your Linux Foundation user account name (see Prerequisites above) instead of <USERNAME> as shows in above example.
To find desired <“Release_version_number”> do: “git -r branch” form VPP root git directory.

2. For VPP 18.07 ONLY: Enable the mlx5 PMD driver compilation as follow:

Cli commands

cd /vpp

sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk

3. Compile and install VPP with DLopen linkage

Cli commands

cd /vpp
make install-dep
make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=y
## Copy manually newly compiled shared module: ##
cp /opt/vpp/external/x86_64/lib/librte_pmd_mlx5_glue.so* /usr/lib64/
## For VPP 18.07 ONLY, check vpp-dpdk-devel package is installed: ##
rpm -qa | grep vpp-dpdk-devel
## Compile new binary: ##
make build-release vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y
## Building RPM packages: ##
make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y
## Install VPP RPMs: ##
rpm -ivh build-root/vpp-sel*.rpm
rpm -ivh build-root/vpp-lib*.rpm
rpm -ivh build-root/vpp-19*.rpm
rpm -ivh build-root/vpp-plug*.rpm 

4. Compile and install without DLopen linkage

Cli commands

cd /vpp
make install-dep
make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n
## Compile new binary: ##
make build-release DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n
## Building RPM packages: ##
make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=n
## Install VPP RPMs: ##
rpm -ivh build-root/vpp-sel*.rpm
rpm -ivh build-root/vpp-lib*.rpm
rpm -ivh build-root/vpp-19*.rpm
rpm -ivh build-root/vpp-plug*.rpm 

5. Edit the /etc/vpp/startup.conf start file to include the following:

Important: Uncomment no-mulit-seg parameter. It disables jumbo packets handling and improves performances significantly.

File

unix {

  nodaemon

  log /tmp/vpp.log

  full-coredump
}

dpdk {

        dev 0000:07:00.0

        dev 0000:07:00.1

        no-multi-seg

        dev default {

                num-rx-queues 2

        }
}

Insert PCI slot numbers of ConnectX-4/5 interfaces with correct IDs at dpdk { dev } section, as above example.

To verify the correct IDs use:

Cli command

lspci -nn | grep -i mel
07:00.0 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]
07:00.1 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]

6. Start VPP.

Cli command

vpp -c /etc/vpp/startup.conf

7. Enter the VPP Command Line Interface.

Cli command

vppctl

8. Configure L3 interfaces as shown in the example below:

VPP Cli

_______    _        _   _____  ___
__/ __/ _ \  (_)__    | | / / _ \/ _ \

_/ _// // / / / _ \   | |/ / ___/ ___/

/_/ /____(_)_/\___/   |___/_/  /_/
 
vpp#  set int ip address HundredGigabitEthernet7/0/0 1.1.1.1/24

vpp#  set interface state HundredGigabitEthernet7/0/0 up

vpp#  set int ip address HundredGigabitEthernet7/0/1 2.2.2.2/24

vpp#  set interface state HundredGigabitEthernet7/0/1 up

vpp# show int

              Name               Idx       State          Counter          Count

HundredGigabitEthernet7/0/0              1         up

HundredGigabitEthernet7/0/1              2         up

vpp# show int address

HundredGigabitEthernet7/0/0 (up):

  1.1.1.1/24

HundredGigabitEthernet7/0/1 (up):

  2.2.2.2/24
local0 (dn):