Penguin
Recent Changes
Blame: RouteBasedTrafficShaping
EditPageHistoryDiffInfoLikePages
Annotated edit history of RouteBasedTrafficShaping version 2, including all changes. View license author blame.
Rev Author # Line
1 GreigMcGill 1 Initial braindump. Please feel free to improve or explain why I'm "doing it wrong" - there's not a lot of good documentation out there for doing this, so here's a start, it works for me.
2
3 *Scenario*: We are a "mini ISP", or more a bandwidth reseller for our clients. We onsell bandwidth from [FX Networks] to clients connecting via [Velocity]. We wish to ensure international bandwidth is fairly provisioned.
4
5 Our (anonymised) network looks like this:
6
7 <pre>
8
9 Client LAN Client Firewall Firewall/Router ITP Edge Router/Firewall
10 {10.7.5.0/24} ---|10.7.5.254 - 1.1.1.1|--velocity--|1.1.1.254 - 1.1.2.1|--|1.1.2.254 - 1.1.3.1|-- {inet}
11 eth0 eth1 vlan eth0 eth1 eth0 eth1
12 </pre>
13
14 Step 1: Get a BGP feed from your ISP containing only domestic routes (administrative step) and configure quagga running ONLY the bgpd process. Configs (as on Ubuntu Hardy Heron after apt-get install quagga) as follows:
15
16 /etc/quagga/daemons
17
18 <pre>
19 zebra=no
20 bgpd=yes
21 ospfd=no
22 ospf6d=no
23 ripd=no
24 ripngd=no
25 isisd=no
26 </pre>
27
28 /etc/quagga/debian.conf
29
30 <pre>
31 vtysh_enable=yes
32 zebra_options=" --daemon -A 127.0.0.1"
33 bgpd_options=" --daemon -A 127.0.0.1"
34 ospfd_options=" --daemon -A 127.0.0.1"
35 ospf6d_options="--daemon -A ::1"
36 ripd_options=" --daemon -A 127.0.0.1"
37 ripngd_options="--daemon -A ::1"
38 isisd_options=" --daemon -A 127.0.0.1"
39 </pre>
40
41 /etc/quagga/bgpd.conf
42
43 <pre>
44 hostname firewall
45 password secretpass
46 enable password secretenablepass
47 log syslog
48 !
49 router bgp YourASNumber
50 bgp router-id 1.1.3.1
51 network 1.1.1.0/24 # a line for each network route you wish to announce - may be none
52 redistribute kernel
53 redistribute connected
54 redistribute static
55 neighbor 131.203.118.254 remote-as 9503 # specific to FX - your ISP will advise you.
56 neighbor 131.203.118.254 description FX
57 !
58 line vty
59 !
60 </pre>
61
62 Once this is working, you should get a large list of NZ domestic routes from the output of vtysh -c 'sh ip bgp'
63
64 Step 2: Get the routes on the box doing the shaping. In our scenario, we're shaping on a different box to the one we're running our bgpd on. You may not need this setup. Either way, you need to get those routes into a "realm" which you can then shape on. We have an entry in our authorised_keys file on the firewall allowing the client to ONLY run vtysh -c 'sh ip bgp' with no password, so a simple ssh to the firewall results in the list of routes being dumped. With that in mind, here is the script (we run it hourly):
65
66 /etc/cron.hourly/getroutes
67
2 DanielLawson 68 <verbatim>
1 GreigMcGill 69 #!/bin/bash
70 #
71 # This script grabs route prefixes from bgp on the firewall and dumps them into a local route table
72 # This table can then be used to shape, as anything NOT in the table is international traffic
73
74 ipcmd=/sbin/ip
75 table=99
76 gw=1.1.2.254
77 dev=eth1
78
79 # Remove the lookup rule
80 $ipcmd rule del table $table
81
82 # Flush the existing routing table
83 $ipcmd route flush table $table
84
85 for prefix in `ssh -q firewall | grep ^\*\> | awk '{print $2}' | sed -e 's/^M//g'` ; do
86 if ! [[ $prefix == */* ]] ; then
87 addr=$prefix/`ipcalc -c $prefix`
88 $ipcmd route add table $table $addr via $gw dev $dev realm 99
89 else
90 $ipcmd route add table $table $prefix via $gw dev $dev realm 99
91 fi
92 done
93
94 # Add local routes
95 $ipcmd route add table $table 10.7.5.0/24 via 1.1.1.1 dev eth0 realm 2
96 # You might need to add other local routes - as usual, don't EVER paste scripts blindly.
97 # I've removed mine for anonymisation purposes.
98
99 # add lookup rule
100 $ipcmd rule add table $table
101
102 exit 0
2 DanielLawson 103 </verbatim>
1 GreigMcGill 104
105 Step 3: Shape it! This script creates the queues and sends traffic in realm 2 to the local queue, realm 99 to the national queue, and all other traffic to the international (default) queue.
106
107 /usr/local/sbin/traffic
108
109 <pre>
110 #!/bin/sh
111 #
112 # traffic - script that configures network traffic shaping
113
114 extif=eth1
115
116 # line speed of the network interface
117 ifrate=1000mbit
118
119 # maximum traffic rate
120 maxrate=100mbit
121
122 # shaped limits
123 localrate=100mbit
124 intrate=3mbit
125 natrate=80mbit
126
127 TC=/sbin/tc
128
129 start() {
130 # clear existing rules
131 $TC qdisc del dev $extif root 2>/dev/null
132
133 # root qdisc 1:0 (default queue - rate limit to international)
134 $TC qdisc add dev $extif root handle 1: htb default 12
135
136 # root class 1:1
137 $TC class add dev $extif parent 1:0 classid 1:1 htb rate $maxrate
138
139 # class 1:10 -- local destinations
140 $TC class add dev $extif parent 1:1 classid 1:10 htb rate $localrate
141
142 # class 1:11 -- national destinations
143 $TC class add dev $extif parent 1:1 classid 1:11 htb rate $natrate
144
145 # class 1:12 -- international destinations
146 $TC class add dev $extif parent 1:1 classid 1:12 htb rate $intrate
147
148 # qdisc defs for classes
149 $TC qdisc add dev $extif parent 1:10 handle 10: sfq quantum 1514b perturb 15
150 $TC qdisc add dev $extif parent 1:11 handle 11: sfq quantum 1514b perturb 15
151 $TC qdisc add dev $extif parent 1:12 handle 12: sfq quantum 1514b perturb 15
152
153 # filter for 1:10 -- local destinations
154 $TC filter add dev $extif parent 1:0 protocol ip pref 100 route to 2 flowid 1:10
155
156 # filter for 1:11 -- national routes
157 $TC filter add dev $extif parent 1:0 protocol ip pref 100 route to 99 flowid 1:11
158 }
159
160 stop() {
161 # clear existing rules
162 $TC qdisc del dev $extif root 2>/dev/null
163 }
164
165 status() {
166 echo "qdisc:"
167 $TC qdisc show dev $extif
168 echo "filter:"
169 $TC filter show dev $extif parent 1:
170 echo "class:"
171 $TC class show dev $extif
172 }
173
174 counts() {
175 echo "qdisc:"
176 $TC -s qdisc show dev $extif
177 echo "class:"
178 $TC -s class show dev $extif
179 }
180
181 case "$1" in
182 start)
183 start
184 ;;
185
186 stop)
187 stop
188 ;;
189
190 status)
191 status
192 ;;
193 counts)
194 counts
195 ;;
196 restart)
197 stop
198 start
199 ;;
200 *)
201 echo $"Usage: $0 {start|stop|restart|status|counts}"
202 exit 1
203 esac
204
205 exit 0
206 </pre>