Initial braindump. Please feel free to improve or explain why I'm "doing it wrong" - there's not a lot of good documentation out there for doing this, so here's a start, it works for me.
Scenario: We are a "mini ISP", or more a bandwidth reseller for our clients. We onsell bandwidth from FX Networks? to clients connecting via Velocity?. We wish to ensure international bandwidth is fairly provisioned.
Our (anonymised) network looks like this:
Client LAN Client Firewall Firewall/Router ITP Edge Router/Firewall
{10.7.5.0/24} ---|10.7.5.254 - 1.1.1.1|--velocity--|1.1.1.254 - 1.1.2.1|--|1.1.2.254 - 1.1.3.1|-- {inet}
eth0 eth1 vlan eth0 eth1 eth0 eth1
Step 1: Get a BGP feed from your ISP containing only domestic routes (administrative step) and configure quagga running ONLY the bgpd process. Configs (as on Ubuntu Hardy Heron after apt-get install quagga) as follows:
/etc/quagga/daemons
zebra=no bgpd=yes ospfd=no ospf6d=no ripd=no ripngd=no isisd=no
/etc/quagga/debian.conf
vtysh_enable=yes zebra_options=" --daemon -A 127.0.0.1" bgpd_options=" --daemon -A 127.0.0.1" ospfd_options=" --daemon -A 127.0.0.1" ospf6d_options="--daemon -A ::1" ripd_options=" --daemon -A 127.0.0.1" ripngd_options="--daemon -A ::1" isisd_options=" --daemon -A 127.0.0.1"
/etc/quagga/bgpd.conf
hostname firewall password secretpass enable password secretenablepass log syslog ! router bgp YourASNumber bgp router-id 1.1.3.1 network 1.1.1.0/24 # a line for each network route you wish to announce - may be none redistribute kernel redistribute connected redistribute static neighbor 131.203.118.254 remote-as 9503 # specific to FX - your ISP will advise you. neighbor 131.203.118.254 description FX ! line vty !
Once this is working, you should get a large list of NZ domestic routes from the output of vtysh -c 'sh ip bgp'
Step 2: Get the routes on the box doing the shaping. In our scenario, we're shaping on a different box to the one we're running our bgpd on. You may not need this setup. Either way, you need to get those routes into a "realm" which you can then shape on. We have an entry in our authorised_keys file on the firewall allowing the client to ONLY run vtysh -c 'sh ip bgp' with no password, so a simple ssh to the firewall results in the list of routes being dumped. With that in mind, here is the script (we run it hourly):
/etc/cron.hourly/getroutes
#!/bin/bash
#
# This script grabs route prefixes from bgp on the firewall and dumps them into a local route table
# This table can then be used to shape, as anything NOT in the table is international traffic
ipcmd=/sbin/ip
table=99
gw=1.1.2.254
dev=eth1
# Remove the lookup rule
$ipcmd rule del table $table
# Flush the existing routing table
$ipcmd route flush table $table
for prefix in `ssh -q firewall | grep ^\*\> | awk '{print $2}' | sed -e 's/^M//g'` ; do
if ! [[ $prefix == */* ]] ; then
addr=$prefix/`ipcalc -c $prefix`
$ipcmd route add table $table $addr via $gw dev $dev realm 99
else
$ipcmd route add table $table $prefix via $gw dev $dev realm 99
fi
done
# Add local routes
$ipcmd route add table $table 10.7.5.0/24 via 1.1.1.1 dev eth0 realm 2
# You might need to add other local routes - as usual, don't EVER paste scripts blindly.
# I've removed mine for anonymisation purposes.
# add lookup rule
$ipcmd rule add table $table
exit 0
Step 3: Shape it! This script creates the queues and sends traffic in realm 2 to the local queue, realm 99 to the national queue, and all other traffic to the international (default) queue.
/usr/local/sbin/traffic
#!/bin/sh
#
# traffic - script that configures network traffic shaping
extif=eth1
# line speed of the network interface
ifrate=1000mbit
# maximum traffic rate
maxrate=100mbit
# shaped limits
localrate=100mbit
intrate=3mbit
natrate=80mbit
TC=/sbin/tc
start() {
# clear existing rules
$TC qdisc del dev $extif root 2>/dev/null
# root qdisc 1:0 (default queue - rate limit to international)
$TC qdisc add dev $extif root handle 1: htb default 12
# root class 1:1
$TC class add dev $extif parent 1:0 classid 1:1 htb rate $maxrate
# class 1:10 -- local destinations
$TC class add dev $extif parent 1:1 classid 1:10 htb rate $localrate
# class 1:11 -- national destinations
$TC class add dev $extif parent 1:1 classid 1:11 htb rate $natrate
# class 1:12 -- international destinations
$TC class add dev $extif parent 1:1 classid 1:12 htb rate $intrate
# qdisc defs for classes
$TC qdisc add dev $extif parent 1:10 handle 10: sfq quantum 1514b perturb 15
$TC qdisc add dev $extif parent 1:11 handle 11: sfq quantum 1514b perturb 15
$TC qdisc add dev $extif parent 1:12 handle 12: sfq quantum 1514b perturb 15
# filter for 1:10 -- local destinations
$TC filter add dev $extif parent 1:0 protocol ip pref 100 route to 2 flowid 1:10
# filter for 1:11 -- national routes
$TC filter add dev $extif parent 1:0 protocol ip pref 100 route to 99 flowid 1:11
}
stop() {
# clear existing rules
$TC qdisc del dev $extif root 2>/dev/null
}
status() {
echo "qdisc:"
$TC qdisc show dev $extif
echo "filter:"
$TC filter show dev $extif parent 1:
echo "class:"
$TC class show dev $extif
}
counts() {
echo "qdisc:"
$TC -s qdisc show dev $extif
echo "class:"
$TC -s class show dev $extif
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
counts)
counts
;;
restart)
stop
start
;;
*)
echo $"Usage: $0 {start|stop|restart|status|counts}"
exit 1
esac
exit 0
One page links to RouteBasedTrafficShaping:
