本文将mark下Linux TC的ifb(Intermediate Functional Block)相关notes。

Why

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
/* drivers/net/ifb.c:

The purpose of this driver is to provide a device that allows
for sharing of resources:

1) qdiscs/policies that are per device as opposed to system wide.
ifb allows for a device which can be redirected to thus providing
an impression of sharing.

2) Allows for queueing incoming traffic for shaping instead of
dropping.

The original concept is based on what is known as the IMQ
driver initially written by Martin Devera, later rewritten
by Patrick McHardy and then maintained by Andre Correa.

You need the tc action mirror or redirect to feed this device
packets.


Authors: Jamal Hadi Salim (2005)

*/

从内核的注释中可知,ifb的motivation是为了解决如下两个问题:

  1. Qdisc的多网卡共享
  2. 对输入方向的流量做队列调度

Qdisc的多网卡共享

在多个网卡之间共享一个根Qdisc是ifb实现的一个初衷。如果你有10块网卡,想在这10块网卡上实现相同的流控策略,你需要配置10遍吗?将相同的东西抽出来,实现一个ifb虚拟网卡,然后将这10块网卡的流量全部重定向到这个ifb虚拟网卡上,此时只需要在这个虚拟网卡上配置一个Qdisc就可以了。

对输入方向的流量做队列调度

Linux中的QoS分为入口(Ingress)部分和出口(Egress)部分,入口部分主要用于进行入口流量限速(policing),出口部分主要用于队列调度(queuing scheduling)。

大多数排队规则(qdisc)都是用于输出方向的,输入方向只有一个排队规则,即ingress qdisc。ingress qdisc本身的功能很有限,但可用于重定向incoming packets。通过Ingress qdisc把输入方向的数据包重定向到虚拟设备ifb,而ifb的输出方向可以配置多种qdisc,就可以达到对输入方向的流量做队列调度的目的。

Example

IFB is an alternative to tc filters for handling ingress traffic, by redirecting it to a virtual interface and treat is as egress traffic there.You need one ifb interface per physical interface, to redirect ingress traffic from eth0 to ifb0, eth1 to ifb1 and so on.

When inserting the ifb module, tell it the number of virtual interfaces you need. The default is 2:

1
modprobe ifb numifbs=1

Now, enable all ifb interfaces:

1
ip link set dev ifb0 up # repeat for ifb1, ifb2, ...

And redirect ingress traffic from the physical interfaces to corresponding ifb interface. For eth0 -> ifb0:

1
2
tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0

Again, repeat for eth1 -> ifb1, eth2 -> ifb2 and so on, until all the interfaces you want to shape are covered.

Now, you can apply all the rules you want. Egress rules for eth0 go as usual in eth0. Let’s limit bandwidth, for example:

1
2
3
tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 1mbit

Needless to say, repeat for eth1, eth2, …

Ingress rules for eth0, now go as egress rules on ifb0 (whatever goes into ifb0 must come out, and only eth0 ingress traffic goes into ifb0). Again, a bandwidth limit example:

1
2
3
tc qdisc add dev ifb0 root handle 1: htb default 10
tc class add dev ifb0 parent 1: classid 1:1 htb rate 1mbit
tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 1mbit

The advantage of this approach is that egress rules are much more flexible than ingress filters. Ingress filters only allow you to drop packets, not introduce wait times, for example. By handling ingress traffic as egress you can setup queue disciplines, with traffic classes and, if need be, filters. You get access to the whole tc tree, not only simple filters.


参考资料:

  1. man tc-mirred
  2. Linux TC的ifb原理以及ingress流控
  3. Intermediate Functional Block
  4. 输入方向的流量控制 –ifb
  5. Tc: ingress policing and ifb mirroring
  6. Introduction to Linux interfaces for virtual networking
  7. Linux Foundation wiki on IFB