View Source: RouteBasedTrafficShaping - Waikato Linux Users Group

Edit PageHistory Diff Info LikePages
Initial braindump. Please feel free to improve or explain why I'm "doing it wrong" - there's not a lot of good documentation out there for doing this, so here's a start, it works for me.

*Scenario*: We are a "mini ISP", or more a bandwidth reseller for our clients. We onsell bandwidth from [FX Networks] to clients connecting via [Velocity]. We wish to ensure international bandwidth is fairly provisioned.

Our (anonymised) network looks like this:

<pre>

 Client  LAN          Client Firewall                Firewall/Router       ITP Edge Router/Firewall
{10.7.5.0/24} ---|10.7.5.254 - 1.1.1.1|--velocity--|1.1.1.254 - 1.1.2.1|--|1.1.2.254 - 1.1.3.1|-- {inet}
                  eth0            eth1     vlan     eth0           eth1    eth0           eth1
</pre>

Step 1: Get a BGP feed from your ISP containing only domestic routes (administrative step) and configure quagga running ONLY the bgpd process. Configs (as on Ubuntu Hardy Heron after apt-get install quagga) as follows:

/etc/quagga/daemons

<pre>
zebra=no
bgpd=yes
ospfd=no
ospf6d=no
ripd=no
ripngd=no
isisd=no
</pre>

/etc/quagga/debian.conf

<pre>
vtysh_enable=yes
zebra_options=" --daemon -A 127.0.0.1"
bgpd_options="  --daemon -A 127.0.0.1"
ospfd_options=" --daemon -A 127.0.0.1"
ospf6d_options="--daemon -A ::1"
ripd_options="  --daemon -A 127.0.0.1"
ripngd_options="--daemon -A ::1"
isisd_options=" --daemon -A 127.0.0.1"
</pre>

/etc/quagga/bgpd.conf

<pre>
hostname firewall
password secretpass
enable password secretenablepass
log syslog
!
router bgp YourASNumber
 bgp router-id 1.1.3.1
 network 1.1.1.0/24 # a line for each network route you wish to announce - may be none
 redistribute kernel
 redistribute connected
 redistribute static
 neighbor 131.203.118.254 remote-as 9503 # specific to FX - your ISP will advise you.
 neighbor 131.203.118.254 description FX
!
line vty
!
</pre>

Once this is working, you should get a large list of NZ domestic routes from the output of vtysh -c 'sh ip bgp'

Step 2: Get the routes on the box doing the shaping. In our scenario, we're shaping on a different box to the one we're running our bgpd on. You may not need this setup. Either way, you need to get those routes into a "realm" which you can then shape on. We have an entry in our authorised_keys file on the firewall allowing the client to ONLY run vtysh -c 'sh ip bgp' with no password, so a simple ssh to the firewall results in the list of routes being dumped. With that in mind, here is the script (we run it hourly):

/etc/cron.hourly/getroutes

<verbatim>
#!/bin/bash
#
# This script grabs route prefixes from bgp on the firewall and dumps them into a local route table
# This table can then be used to shape, as anything NOT in the table is international traffic

ipcmd=/sbin/ip
table=99
gw=1.1.2.254
dev=eth1

# Remove the lookup rule
$ipcmd rule del table $table

# Flush the existing routing table
$ipcmd route flush table $table

for prefix in `ssh -q firewall | grep ^\*\> | awk '{print $2}' | sed -e 's/^M//g'` ; do
        if ! [[ $prefix == */* ]] ; then
                addr=$prefix/`ipcalc -c $prefix`
                $ipcmd route add table $table $addr via $gw dev $dev realm 99
        else
                $ipcmd route add table $table $prefix via $gw dev $dev realm 99
        fi
done

# Add local routes
$ipcmd route add table $table 10.7.5.0/24 via 1.1.1.1 dev eth0 realm 2
# You might need to add other local routes - as usual, don't EVER paste scripts blindly.
# I've removed mine for anonymisation purposes.

# add lookup rule
$ipcmd rule add table $table

exit 0
</verbatim>

Step 3: Shape it! This script creates the queues and sends traffic in realm 2 to the local queue, realm 99 to the national queue, and all other traffic to the international (default) queue.

/usr/local/sbin/traffic

<pre>
#!/bin/sh
#
# traffic - script that configures network traffic shaping

extif=eth1

# line speed of the network interface
ifrate=1000mbit

# maximum traffic rate
maxrate=100mbit

# shaped limits
localrate=100mbit
intrate=3mbit
natrate=80mbit

TC=/sbin/tc

start() {
        # clear existing rules
        $TC qdisc del dev $extif root 2>/dev/null

        # root qdisc 1:0 (default queue - rate limit to international)
        $TC qdisc add dev $extif root handle 1: htb default 12

        # root class 1:1
        $TC class add dev $extif parent 1:0 classid 1:1 htb rate $maxrate

        # class 1:10 -- local destinations
        $TC class add dev $extif parent 1:1 classid 1:10 htb rate $localrate

        # class 1:11 -- national destinations
        $TC class add dev $extif parent 1:1 classid 1:11 htb rate $natrate

        # class 1:12 -- international destinations
        $TC class add dev $extif parent 1:1 classid 1:12 htb rate $intrate

        # qdisc defs for classes
        $TC qdisc add dev $extif parent 1:10 handle 10: sfq quantum 1514b perturb 15
        $TC qdisc add dev $extif parent 1:11 handle 11: sfq quantum 1514b perturb 15
        $TC qdisc add dev $extif parent 1:12 handle 12: sfq quantum 1514b perturb 15

        # filter for 1:10 -- local destinations
        $TC filter add dev $extif parent 1:0 protocol ip pref 100 route to 2 flowid 1:10

        # filter for 1:11 -- national routes
        $TC filter add dev $extif parent 1:0 protocol ip pref 100 route to 99 flowid 1:11
}

stop() {
        # clear existing rules
        $TC qdisc del dev $extif root 2>/dev/null
}

status() {
        echo "qdisc:"
        $TC qdisc show dev $extif
        echo "filter:"
        $TC filter show dev $extif parent 1:
        echo "class:"
        $TC class show dev $extif
}

counts() {
        echo "qdisc:"
        $TC -s qdisc show dev $extif
        echo "class:"
        $TC -s class show dev $extif
}

case "$1" in
        start)
                start
                ;;

        stop)
                stop
                ;;

        status)
                status
                ;;
        counts)
                counts
                ;;
        restart)
                stop
                start
                ;;
        *)
                echo $"Usage: $0 {start|stop|restart|status|counts}"
                exit 1
esac

exit 0
</pre>
One page links to RouteBasedTrafficShaping:
TrafficShaping
Last edited on Thursday, November 6, 2008 8:57:05 am by DanielLawson
Edit PageHistory Diff Info LikePages