Ivory, as in made from elephants.
You probably need at least 9 elephants for a decent tower. Not sure how you convince them to stay still though.
Probably feed them some Mnokeys.
2 pages link to IvoryTower: