original_kernel/tools/testing/selftests/drivers/net
Joe Damato a61a459f58 testing: net-drv: use stats64 for testing
Testing a network device that has large numbers of bytes/packets may
overflow. Using stats64 when comparing fixes this problem.

I tripped on this while iterating on a qstats patch for mlx5. See below
for confirmation without my added code that this is a bug.

Before this patch (with added debugging output):

$ NETIF=eth0 tools/testing/selftests/drivers/net/stats.py
KTAP version 1
1..4
ok 1 stats.check_pause
ok 2 stats.check_fec
rstat: 481708634 qstat: 666201639514 key: tx-bytes
not ok 3 stats.pkt_byte_sum
ok 4 stats.qstat_by_ifindex

Note the huge delta above ^^^ in the rtnl vs qstats.

After this patch:

$ NETIF=eth0 tools/testing/selftests/drivers/net/stats.py
KTAP version 1
1..4
ok 1 stats.check_pause
ok 2 stats.check_fec
ok 3 stats.pkt_byte_sum
ok 4 stats.qstat_by_ifindex

It looks like rtnl_fill_stats in net/core/rtnetlink.c will attempt to
copy the 64bit stats into a 32bit structure which is probably why this
behavior is occurring.

To show this is happening, you can get the underlying stats that the
stats.py test uses like this:

$ ./cli.py --spec ../../../Documentation/netlink/specs/rt_link.yaml \
           --do getlink --json '{"ifi-index": 7}'

And examine the output (heavily snipped to show relevant fields):

 'stats': {
           'multicast': 3739197,
           'rx-bytes': 1201525399,
           'rx-packets': 56807158,
           'tx-bytes': 492404458,
           'tx-packets': 1200285371,

 'stats64': {
             'multicast': 3739197,
             'rx-bytes': 35561263767,
             'rx-packets': 56807158,
             'tx-bytes': 666212335338,
             'tx-packets': 1200285371,

The stats.py test prior to this patch was using the 'stats' structure
above, which matches the failure output on my system.

Comparing side by side, rx-bytes and tx-bytes, and getting ethtool -S
output:

rx-bytes stats:    1201525399
rx-bytes stats64: 35561263767
rx-bytes ethtool: 36203402638

tx-bytes stats:      492404458
tx-bytes stats64: 666212335338
tx-bytes ethtool: 666215360113

Note that the above was taken from a system with an mlx5 NIC, which only
exposes ndo_get_stats64.

Based on the ethtool output and qstat output, it appears that stats.py
should be updated to use the 'stats64' structure for accurate
comparisons when packet/byte counters get very large.

To confirm that this was not related to the qstats code I was iterating
on, I booted a kernel without my driver changes and re-ran the test
which shows the qstats are skipped (as they don't exist for mlx5):

NETIF=eth0 tools/testing/selftests/drivers/net/stats.py
KTAP version 1
1..4
ok 1 stats.check_pause
ok 2 stats.check_fec
ok 3 stats.pkt_byte_sum # SKIP qstats not supported by the device
ok 4 stats.qstat_by_ifindex # SKIP No ifindex supports qstats

But, fetching the stats using the CLI

$ ./cli.py --spec ../../../Documentation/netlink/specs/rt_link.yaml \
           --do getlink --json '{"ifi-index": 7}'

Shows the same issue (heavily snipped for relevant fields only):

 'stats': {
           'multicast': 105489,
           'rx-bytes': 530879526,
           'rx-packets': 751415,
           'tx-bytes': 2510191396,
           'tx-packets': 27700323,
 'stats64': {
             'multicast': 105489,
             'rx-bytes': 530879526,
             'rx-packets': 751415,
             'tx-bytes': 15395093284,
             'tx-packets': 27700323,

Comparing side by side with ethtool -S on the unmodified mlx5 driver:

tx-bytes stats:    2510191396
tx-bytes stats64: 15395093284
tx-bytes ethtool: 17718435810

Fixes: f0e6c86e4b ("testing: net-drv: add a driver test for stats reporting")
Signed-off-by: Joe Damato <jdamato@fastly.com>
Link: https://lore.kernel.org/r/20240520235850.190041-1-jdamato@fastly.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2024-05-23 10:18:29 +02:00
..
bonding
dsa
hw selftests: drv-net: add checksum tests 2024-05-08 18:57:55 -07:00
lib/py net: selftest: add test for netdev netlink queue-get API 2024-05-08 18:59:47 -07:00
microchip selftests: microchip: add test for QoS support on KSZ9477 switch family 2024-05-08 10:35:11 +01:00
mlxsw
netdevsim
ocelot
team
virtio_net selftests: virtio_net: add initial tests 2024-04-26 13:26:53 +02:00
Makefile net: selftest: add test for netdev netlink queue-get API 2024-05-08 18:59:47 -07:00
README.rst selftests: drv-net: extend the README with more info and example 2024-04-26 16:10:26 -07:00
config selftests: drv-net: add config for netdevsim 2024-04-16 18:54:06 -07:00
ping.py selftests: drv-net: add require_XYZ() helpers for validating env 2024-04-23 10:13:56 -07:00
queues.py net: selftest: add test for netdev netlink queue-get API 2024-05-08 18:59:47 -07:00
stats.py testing: net-drv: use stats64 for testing 2024-05-23 10:18:29 +02:00

README.rst

.. SPDX-License-Identifier: GPL-2.0

Running driver tests
====================

Networking driver tests are executed within kselftest framework like any
other tests. They support testing both real device drivers and emulated /
software drivers (latter mostly to test the core parts of the stack).

SW mode
~~~~~~~

By default, when no extra parameters are set or exported, tests execute
against software drivers such as netdevsim. No extra preparation is required
the software devices are created and destroyed as part of the test.
In this mode the tests are indistinguishable from other selftests and
(for example) can be run under ``virtme-ng`` like the core networking selftests.

HW mode
~~~~~~~

Executing tests against a real device requires external preparation.
The netdevice against which tests will be run must exist, be running
(in UP state) and be configured with an IP address.

Refer to list of :ref:`Variables` later in this file to set up running
the tests against a real device.

Both modes required
~~~~~~~~~~~~~~~~~~~

All tests in drivers/net must support running both against a software device
and a real device. SW-only tests should instead be placed in net/ or
drivers/net/netdevsim, HW-only tests in drivers/net/hw.

Variables
=========

The variables can be set in the environment or by creating a net.config
file in the same directory as this README file. Example::

  $ NETIF=eth0 ./some_test.sh

or::

  $ cat tools/testing/selftests/drivers/net/net.config
  # Variable set in a file
  NETIF=eth0

Local test (which don't require endpoint for sending / receiving traffic)
need only the ``NETIF`` variable. Remaining variables define the endpoint
and communication method.

NETIF
~~~~~

Name of the netdevice against which the test should be executed.
When empty or not set software devices will be used.

LOCAL_V4, LOCAL_V6, REMOTE_V4, REMOTE_V6
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Local and remote endpoint IP addresses.

REMOTE_TYPE
~~~~~~~~~~~

Communication method used to run commands on the remote endpoint.
Test framework has built-in support for ``netns`` and ``ssh`` channels.
``netns`` assumes the "remote" interface is part of the same
host, just moved to the specified netns.
``ssh`` communicates with remote endpoint over ``ssh`` and ``scp``.
Using persistent SSH connections is strongly encouraged to avoid
the latency of SSH connection setup on every command.

Communication methods are defined by classes in ``lib/py/remote_{name}.py``.
It should be possible to add a new method without modifying any of
the framework, by simply adding an appropriately named file to ``lib/py``.

REMOTE_ARGS
~~~~~~~~~~~

Arguments used to construct the communication channel.
Communication channel dependent::

  for netns - name of the "remote" namespace
  for ssh - name/address of the remote host

Example
=======

Build the selftests::

  # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw"

"Install" the tests and copy them over to the target machine::

  # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" \
     install INSTALL_PATH=/tmp/ksft-net-drv

  # rsync -ra --delete /tmp/ksft-net-drv root@192.168.1.1:/root/

On the target machine, running the tests will use netdevsim by default::

  [/root] # ./ksft-net-drv/run_kselftest.sh -t drivers/net:ping.py
  TAP version 13
  1..1
  # timeout set to 45
  # selftests: drivers/net: ping.py
  # KTAP version 1
  # 1..3
  # ok 1 ping.test_v4
  # ok 2 ping.test_v6
  # ok 3 ping.test_tcp
  # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
  ok 1 selftests: drivers/net: ping.py

Create a config with remote info::

  [/root] # cat > ./ksft-net-drv/drivers/net/net.config <<EOF
  NETIF=eth0
  LOCAL_V4=192.168.1.1
  REMOTE_V4=192.168.1.2
  REMOTE_TYPE=ssh
  REMOTE_ARGS=root@192.168.1.2
  EOF

Run the test::

  [/root] # ./ksft-net-drv/drivers/net/ping.py
  KTAP version 1
  1..3
  ok 1 ping.test_v4
  ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity
  ok 3 ping.test_tcp
  # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:1 error:0