为什么我收到的打包机比原始数据包大
Why am I receiving packer bigger than with raw packet
我正在尝试使用原始数据包（仅用于播放）将数据包从一个接口传输到另一个接口。
首先我关注接收到的数据包。
在我的机器上（archlinux，有 192.168.30.3 作为 IP）我创建了这个代码：
#include <stdio.h>
#include <net/ethernet.h> /* the L2 protocols */
#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
int main()
{
int packet_socket = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP));
/* test reception */
char packet[4096];
struct sockaddr rcvaddr;
struct in_addr addr;
addr.s_addr = inet_addr("192.168.30.3"); //my ip
// use nc to send a use packet
while (1) {
int len = sizeof(rcvaddr);
int len_packet =
recvfrom(packet_socket, packet, 4096, 0, &rcvaddr, &len);
// check if the packet is for us
struct iphdr *iph =
(struct iphdr *) (packet + sizeof(struct ethhdr));
if (iph->daddr != inet_addr("192.168.30.3"))
continue;
// check if tcp
if (iph->protocol != IPPROTO_TCP)
continue;
printf("Total packet length: %d\n",
sizeof(struct ethhdr) + ntohs(iph->tot_len));
}
}
然后我运行它以root身份执行nc -lp 12345 -n > /dev/null.
在另一台机器上（debian，192.168.30.4）我运行 dd if=/dev/urandom | nc 192.168.30.3 12345
这使得我之前的程序打印接收到的数据包的长度。
从中，我看到有大于 MTU 大小的数据包（即 1500
在两台机器上）。例如，我可以从我的程序中读取“总数据包长度：16962”。 （也被 linux raw ethernet socket receive more bytes than MTU 观察到）。
我知道IP分片，所以我首先想到了IP重组。
但是我在 man 7 raw 中读到：
“请注意，与原始套接字不同，数据包套接字不会重组 IP 片段。”
因为我使用了数据包套接字（AF_PACKET）我不应该有数据包
重新组装然后保持 MTU 大小正确？
我也做了 sudo ethtool -K ens3 tx off sg off tso off 并用值 0、1、2 和 3 进行了测试
在两台机器上 /proc/sys/net/ipv4/ip_no_pmtu_disc。
你认为 192.168.30.4 发送的 MTU 多吗？还是我的机器执行了一些重新组装，尽管
说明书上写的是什么？
ethtool -k ens3 给出：
在 192.168.30.4 上：
seb@SERVER:~$ sudo ethtool -k ens3
Features for ens3:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
在 192.168.30.3 上：
[seb@archlinux ~]$ sudo ethtool -k ens3
Features for ens3:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: off [fixed]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
tx-gso-list: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]
此外，请注意，这两台机器是 GNS3 的 qemu 机器 运行，具有以下网络选项：-net none -device e1000,mac=0c:7e:08:49:13:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:20049,localaddr=127.0.0.1:20048
由于观察到的总数据包长度远大于典型的巨型帧 (MTU 9k)，很明显接收端采用了 Large Receive Offload (LRO)或 Generic Receive Offload (GRO) 从而在网络接口驱动程序级别将较小的数据包重新组装成较大的数据包。这可以解释为什么有问题的数据包套接字看到已经重新组装的（大）数据包。
在这种特定情况下，ethtool -k 输出清楚地表明 LRO 始终处于禁用状态，而 GRO 确实处于活动状态并且可以进行调整。根据评论中的讨论，禁用 GRO 确实取得了成果。
我正在尝试使用原始数据包（仅用于播放）将数据包从一个接口传输到另一个接口。 首先我关注接收到的数据包。
在我的机器上（archlinux，有 192.168.30.3 作为 IP）我创建了这个代码：
#include <stdio.h>
#include <net/ethernet.h> /* the L2 protocols */
#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
int main()
{
int packet_socket = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP));
/* test reception */
char packet[4096];
struct sockaddr rcvaddr;
struct in_addr addr;
addr.s_addr = inet_addr("192.168.30.3"); //my ip
// use nc to send a use packet
while (1) {
int len = sizeof(rcvaddr);
int len_packet =
recvfrom(packet_socket, packet, 4096, 0, &rcvaddr, &len);
// check if the packet is for us
struct iphdr *iph =
(struct iphdr *) (packet + sizeof(struct ethhdr));
if (iph->daddr != inet_addr("192.168.30.3"))
continue;
// check if tcp
if (iph->protocol != IPPROTO_TCP)
continue;
printf("Total packet length: %d\n",
sizeof(struct ethhdr) + ntohs(iph->tot_len));
}
}
然后我运行它以root身份执行nc -lp 12345 -n > /dev/null.
在另一台机器上（debian，192.168.30.4）我运行 dd if=/dev/urandom | nc 192.168.30.3 12345
这使得我之前的程序打印接收到的数据包的长度。
从中，我看到有大于 MTU 大小的数据包（即 1500 在两台机器上）。例如，我可以从我的程序中读取“总数据包长度：16962”。 （也被 linux raw ethernet socket receive more bytes than MTU 观察到）。
我知道IP分片，所以我首先想到了IP重组。
但是我在 man 7 raw 中读到：
“请注意，与原始套接字不同，数据包套接字不会重组 IP 片段。”
因为我使用了数据包套接字（AF_PACKET）我不应该有数据包
重新组装然后保持 MTU 大小正确？
我也做了 sudo ethtool -K ens3 tx off sg off tso off 并用值 0、1、2 和 3 进行了测试
在两台机器上 /proc/sys/net/ipv4/ip_no_pmtu_disc。
你认为 192.168.30.4 发送的 MTU 多吗？还是我的机器执行了一些重新组装，尽管 说明书上写的是什么？
ethtool -k ens3 给出：
在 192.168.30.4 上：
seb@SERVER:~$ sudo ethtool -k ens3
Features for ens3:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
在 192.168.30.3 上：
[seb@archlinux ~]$ sudo ethtool -k ens3
Features for ens3:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: off [fixed]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
tx-gso-list: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]
此外，请注意，这两台机器是 GNS3 的 qemu 机器 运行，具有以下网络选项：-net none -device e1000,mac=0c:7e:08:49:13:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:20049,localaddr=127.0.0.1:20048
由于观察到的总数据包长度远大于典型的巨型帧 (MTU 9k)，很明显接收端采用了 Large Receive Offload (LRO)或 Generic Receive Offload (GRO) 从而在网络接口驱动程序级别将较小的数据包重新组装成较大的数据包。这可以解释为什么有问题的数据包套接字看到已经重新组装的（大）数据包。
在这种特定情况下，ethtool -k 输出清楚地表明 LRO 始终处于禁用状态，而 GRO 确实处于活动状态并且可以进行调整。根据评论中的讨论，禁用 GRO 确实取得了成果。