[Host A] <-> [Router B] <-> [Router C] <-> [Router D] <-> [Router E] <-> [Host F ]
Now, Router B has a route to some RFC:1918 space (perhaps Host A is even on RFC:1918 behind NAT), Router D is numbered using RFC:1918 space (differently routable than Router B). Now, if the link from Router D and router E has a smaller MTU than the rest of the network, and Router B has reverse path filtering on, then A can no longer talk to F. If A traceroutes to F then it misses seeing D as well for the same reason.
RFC:1918 is evil and should be avoided at all costs, if you must use it, use it only on the edges on networks you control.
Network interfaces can be assigned more than one IP address.
Eg, to create aliases for the range 192.168.0.1 - 192.168.0.20:
for i in `jot 20 1` ; do ifconfig fxp0 inet 192.168.0.$i alias ; done
for i in `seq 1 20` ; do ifconfig eth0:$i 192.168.0.$i ; done
for i in `seq 1 20` ; do ip addr add 192.168.0.$i dev eth0 ; done
ip addr add 192.168.0.0/24 dev eth0
This page discusses the difference in tuning networking on various OS's: http://foureleven.org:81/rosetta-stone-performance-tuning.html