Linux: Network Debugging

There are several steps to go through to debug issues with the networking.

network-scripts

Each interface has a config script. These are found in /etc/sysconfig/network-scripts/

[root@head ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
HWADDR="00:25:90:00:82:E0"
ONBOOT="yes"
IPADDR=172.16.1.1
NETMASK=255.255.0.0
BOOTPROTO="static"

The script is read each time the network interface is started. The scirpts defines what settings will be used.

The HWADDDR defines the mac address of the interface to be used. This must be correct is it is in the file of the network will fail to start.

other network files

resolve.conf

• /etc/resolve.conf

Defines the DNS servers for the network. These can also be defined in the network-scripts

network

• /etc/sysconfig/

Defines the Hostname of the system, and determines if the networking is avaialable.

70 persistent rules

/etc/udev/rules.d/70-persistent-net.rules

The file defines which macaddess is felated to which network interface. This is the file that the network-script checks the mac address against. Each interface will have a line defining the drivers and macaddress of each interface. This file is auto generated an can get it wrong.

The file can either be edited directly to correct any mistakes. The file can be deleted as it will be regenerated - this may also correct the problem.

# PCI device 0x8086:0x10c9 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:25:90:00:82:e0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

kernel modules

The kernel makes use of several possible network modules. The full list of network modules can be found in a location based on the kernel version of the system.

less /lib/modules/2.6.32-220.el6.x86_64/modules.networking

##extract of file##

dummy.ko
dvb-core.ko
e100.ko
e1000.ko
e1000e.ko
enic.ko
epic100.ko
et1011c.ko
ethoc.ko
fakehard.ko
fealnx.ko
firewire-net.ko
fmvj18x_cs.ko

Ethernet Modules

• igb
• e1000
• e1000e
• mlx4_en

Infiniband Modules

• mlx4_ib

Module Commands

• modprobe <module name>

Can be used to install, remove and edit modules. It is the main command when it comes to modules - but it is easier to look at the man pages for full details of its abilities.

• module load <module name> / insmod <module name>

Alternative command to install a module

• module unload <module name> / rmmod <module name>

Alternative command to remove a module

• lsmod

Lists all of the available modules

Module                  Size  Used by

iw_nes                205304  0
libcrc32c               1246  1 iw_nes
iw_cxgb4              119289  0
cxgb4                 346764  1 iw_cxgb4
iw_cxgb3              133370  0
cxgb3                 196201  1 iw_cxgb3
ib_qib                606030  1
mlx4_ib                76784  0
mlx4_en                74570  0
mlx4_core             123595  2 mlx4_ib,mlx4_en
ib_mthca              141055  0

• modinfo <module name>

Show all of the settings for a module. Can be useful to check that you are using the correct version.

[root@blade1 ~]# modinfo mlx4_ib
filename:       /lib/modules/2.6.32-220.el6.x86_64/extra/ofa_kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko
version:        1.0-mlnx_ofed1.5.3
license:        Dual BSD/GPL
description:    Mellanox ConnectX HCA InfiniBand driver
author:         Roland Dreier
srcversion:     4EA6D043CEB8ED80480B5C8
depends:        mlx4_core,ib_core,ib_mad
vermagic:       2.6.32-220.el6.x86_64 SMP mod_unload modversions
parm:           debug_level:Enable debug tracing if > 0 (int)

lspci

List all of the pci connections in a system. Combines with grep you can check that the connections have been found.

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
04:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
06:01.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200eW WPCM450 (rev 0a)

dmesg

dmesg shows all of the boot and kernel messages from the system. Useful to check that nmodules are loaded correctly, and that network interfaces are correct. It can show what mac address is being used for the interface, and which driver.

dmesg | grep eth0

igb 0000:01:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 00:25:90:00:82:e0

ethtool

Gives full information as to the state of a network interface. Useful to check that the connection is working

[root@head ~]# ethtool eth0
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000003 (3)
        Link detected: yes