Differences between version 6 and predecessor to the previous major change of LinuxQualityOfService.
Other diffs: Previous Revision, Previous Author, or view the Annotated Edit History
| Newer page: | version 6 | Last edited on Sunday, March 21, 2004 5:32:32 am | by StuartYeates | Revert |
| Older page: | version 3 | Last edited on Saturday, March 20, 2004 6:21:59 pm | by JamesSpooner | Revert |
@@ -1,9 +1,9 @@
Before you start, if you want an authoritive guide to the following material, read [The Linux Advanced Routing and Traffic Control HOWTO | http://lartc.org/lartc.html ] instead.
-This is a brief but concerted effort to minimise the effect of TCP traffic on interactive applications, using QoS under linux
.
+This is a brief but concerted effort to minimise the effect of [
TCP]
traffic on interactive applications, using QoS under [Linux]
.
-The major motivating factor behind this work was the effect the new queuing imposed _before_ the bandwidth throttling at the RAN, on Telecom's network in early 2004.
+The major motivating factor behind this work was the effect the new queuing imposed _before_ the bandwidth throttling at the RAN, on [
Telecom]
's network in early 2004.
I live in a flat where we pay 3rds of the cost of the internet bill, so between three people, they each should have an even crack at the bandwidth.
I devised two queuing structures that should give a decent balance between users, one for egress on the internal interface and one for egress on the external interface.
@@ -79,9 +79,9 @@
| Qdisc 101: SFQ |
| -------------------------------------- |
| | High Priority Per-User Queue | |
| | For interactive applications | |
- | | and ICMP etc. | |
+ | | and [
ICMP]
etc. | |
| -------------------------------------- |
| Class 10:2(CBQ Class, 128kbit) |
| Qdisc 102: SFQ |
| -------------------------------------- |
@@ -99,9 +99,9 @@
| Class 10:5(CBQ Class, 120kbit) |
| Qdisc 105: TBQ (Rate 96kbit) |
| -------------------------------------- |
| | Low Priority Per-User Queue | |
- Routing -> | | For TCP and other non-interactive | | -> ADSL modem
+ Routing -> | | For TCP and other non-interactive | | -> [
ADSL]
modem
| | applications. | |
| -------------------------------------- |
| Class 10:6(CBQ Class, 120kbit) |
| Qdisc 106: TBQ (Rate 96kbit) |
@@ -278,4 +278,120 @@
Sent 244 bytes 4 pkts (dropped 0, overlimits 0)
borrowed 0 overactions 0 avgidle 283475 undertime 0
Alright, now we're getting some traffic in the SFQ now!
+
+Adding the rest of the queues is relatively straight forward, providing you use a sane numbering scheme for your qdisc/class handles.
+
+Here is the script to setup the QoS structure for ADSL as above, we're adding 5 user queues, for expansions sake, if they're not used, they don't affect anyone else.
+
+ # Clear the slate
+ tc qdisc del dev ppp0 root
+
+ # Add a classful qdisc (cbq) to the root class
+
+ tc qdisc add dev ppp0 root handle 10: cbq bandwidth 6Mbit rate 127kbit avpkt 1000
+
+ # High priority Queues
+ # Add the 5 High Priority classes under the CBQ Qdisc
+ tc class add dev ppp0 parent 10: classid 10:1 cbq bandwidth 100Mbit rate 127kbit weight 20kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 200 bounded
+ tc class add dev ppp0 parent 10: classid 10:2 cbq bandwidth 100Mbit rate 127kbit weight 20kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 200 bounded
+ tc class add dev ppp0 parent 10: classid 10:3 cbq bandwidth 100Mbit rate 127kbit weight 20kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 200 bounded
+ tc class add dev ppp0 parent 10: classid 10:4 cbq bandwidth 100Mbit rate 127kbit weight 20kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 200 bounded
+ tc class add dev ppp0 parent 10: classid 10:5 cbq bandwidth 100Mbit rate 127kbit weight 20kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 200 bounded
+
+ # Add leaf qdiscs to these queues
+ tc qdisc add dev ppp0 parent 10:1 handle 101: sfq
+ tc qdisc add dev ppp0 parent 10:2 handle 102: sfq
+ tc qdisc add dev ppp0 parent 10:3 handle 103: sfq
+ tc qdisc add dev ppp0 parent 10:4 handle 104: sfq
+ tc qdisc add dev ppp0 parent 10:5 handle 105: sfq
+
+ # Add filters to match FWMARK packets and sort into these queues
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 101 fw flowid 10:1
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 102 fw flowid 10:2
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 103 fw flowid 10:3
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 104 fw flowid 10:4
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 105 fw flowid 10:5
+
+
+ # Low Priority Queues
+ tc class add dev ppp0 parent 10: classid 10:6 cbq bandwidth 100Mbit rate 120kbit weight 5kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 1500 bounded
+ tc class add dev ppp0 parent 10: classid 10:7 cbq bandwidth 100Mbit rate 120kbit weight 5kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 1500 bounded
+ tc class add dev ppp0 parent 10: classid 10:8 cbq bandwidth 100Mbit rate 120kbit weight 5kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 1500 bounded
+ tc class add dev ppp0 parent 10: classid 10:9 cbq bandwidth 100Mbit rate 120kbit weight 5kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 1500 bounded
+ tc class add dev ppp0 parent 10: classid 10:10 cbq bandwidth 100Mbit rate 120kbit weight 5kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 1500 bounded
+
+ # Add leaf qdiscs to these queues
+ tc qdisc add dev ppp0 parent 10:6 handle 106: tbf rate 96kbit buffer 16384 limit 16384 mpu 200
+ tc qdisc add dev ppp0 parent 10:7 handle 107: tbf rate 96kbit buffer 16384 limit 16384 mpu 200
+ tc qdisc add dev ppp0 parent 10:8 handle 108: tbf rate 96kbit buffer 16384 limit 16384 mpu 200
+ tc qdisc add dev ppp0 parent 10:9 handle 109: tbf rate 96kbit buffer 16384 limit 16384 mpu 200
+ tc qdisc add dev ppp0 parent 10:10 handle 110: tbf rate 96kbit buffer 16384 limit 16384 mpu 200
+
+ # Add filters to match FWMARK packets and sort into these queues
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 106 fw flowid 10:6
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 107 fw flowid 10:7
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 108 fw flowid 10:8
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 109 fw flowid 10:9
+ tc filter add dev ppp0 protocol ip parent 10: prio 1 handle 110 fw flowid 10:10
+
+
+Now, all that is needed is to use iptables to MARK packets for the appropriate queues.
+Remember MARK does not immediately RETURN the packet. I get around this by marking every packet first, then having the sorting rules match on that mark, so once a packet is marked, it won't be marked again in that table.
+
+Heres what my POSTROUTING chain looks like.
+
+ Chain PREROUTING (policy ACCEPT 294 packets, 114705 bytes)
+ pkts bytes target prot opt in out source destination
+ 716 207616 MARK all -- eth0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x80
+ 18 1080 MARK icmp -- etho * 10.1.13.3 0.0.0.0/0 MARK match 0x80 MARK set 0x65
+ 0 0 MARK icmp -- eth0 * 10.1.13.7 0.0.0.0/0 MARK match 0x80 MARK set 0x66
+ 5 420 MARK icmp -- eth0 * 0.0.0.0/0 0.0.0.0/0 MARK match 0x80 MARK set 0x69
+ 5 320 MARK udp -- eth0 * 10.1.13.3 0.0.0.0/0 MARK match 0x80 MARK set 0x65
+ 0 0 MARK udp -- eth0 * 10.1.13.7 0.0.0.0/0 MARK match 0x80 MARK set 0x66
+ 52 6074 MARK udp -- eth0 * 0.0.0.0/0 0.0.0.0/0 MARK match 0x80 MARK set 0x69
+ 159 7296 MARK all -- eth0 * 10.1.13.3 0.0.0.0/0 MARK match 0x80 MARK set 0x6a
+ 435 189427 MARK all -- eth0 * 10.1.13.7 0.0.0.0/0 MARK match 0x80 MARK set 0x6b
+ 42 2999 MARK all -- eth0 * 0.0.0.0/0 0.0.0.0/0 MARK match 0x80 MARK set 0x6e
+
+Cross-Referencing with the tc filters, we see ..
+
+ root@sleepy:~# tc filter show dev ppp0
+ filter parent 10: protocol ip pref 1 fw
+ filter parent 10: protocol ip pref 1 fw handle 0x65 classid 10:1
+ filter parent 10: protocol ip pref 1 fw handle 0x66 classid 10:2
+ filter parent 10: protocol ip pref 1 fw handle 0x67 classid 10:3
+ filter parent 10: protocol ip pref 1 fw handle 0x68 classid 10:4
+ filter parent 10: protocol ip pref 1 fw handle 0x69 classid 10:5
+ filter parent 10: protocol ip pref 1 fw handle 0x6a classid 10:6
+ filter parent 10: protocol ip pref 1 fw handle 0x6b classid 10:7
+ filter parent 10: protocol ip pref 1 fw handle 0x6c classid 10:8
+ filter parent 10: protocol ip pref 1 fw handle 0x6d classid 10:9
+ filter parent 10: protocol ip pref 1 fw handle 0x6e classid 10:10
+
+For example, a udp packet arriving on eth0 with source address 10.1.13.3 will be market with 0x65, and sent to class 10:1, which has a SFQ qdisc on the child class of CBQ, while a TCP packet from 10.1.13.7 will be marked with 0x6b and sent to class 10:7 which has a token bucken qdisc on a child class of a CBQ.
+
+ root@sleepy:~# tc -s -d class show dev ppp0
+ ....
+ class cbq 10:1 parent 10: leaf 101: rate 127Kbit cell 8b (bounded) prio no-transmit/8 weight 20Kbit allot 1514b
+ level 0 ewma 5 avpkt 200b maxidle 10898us
+ Sent 23370 bytes 383 pkts (dropped 0, overlimits 0)
+ borrowed 0 overactions 0 avgidle 285710 undertime 0
+ ...
+ class cbq 10:7 parent 10: leaf 107: rate 120Kbit cell 8b (bounded) prio no-transmit/8 weight 5Kbit allot 2250b
+ level 0 ewma 5 avpkt 1500b maxidle 86516us
+ Sent 4149672 bytes 6294 pkts (dropped 341, overlimits 0)
+ backlog 11p
+ ....
+
+ root@sleepy:~# tc -s -d qdisc show dev ppp0
+ ...
+ qdisc tbf 107: rate 96Kbit burst 16Kb/8 mpu 200b lat 1us
+ Sent 4149672 bytes 6294 pkts (dropped 341, overlimits 35289)
+ ...
+ qdisc sfq 101: quantum 1500b limit 128p flows 128/1024
+ Sent 23370 bytes 383 pkts (dropped 0, overlimits 0)
+ ...
+
+
+What remains to do is setup the egress on the internal interface, eth0, and FWMARK packets coming in on the external interface, so we can remark them into the queues.
