Penguin
Recent Changes
Annotated edit history of NFSRoot version 13, including all changes. View license author blame.
Rev Author # Line
10 IainHogg 1 [NFSRoot] is the term used to describe running an operating system with its root filesystem (/) mounted via [NFS].
2 Why would anyone do this? Thin-clients, kiosks, industrial uses, and so on. If you dont need to touch a local harddisk, you can save a lot in the way of moving parts.
11 JoshEngland 3
13 MattBrown 4 [oneSIS | http://onesis.org] is a package that makes running [NFSRoot] very easy and has excellent documentation.
10 IainHogg 5
6 Other places to look for similar information include the DisklessWorkstationNotes page, and the notes on [PXE] and so forth.
7
8 This set up uses [PXE] booting network cards, via [PXELinux], to load a kernel and then mount / via NFS. Its very debian oriented, although the principles could be adapted to any situation.
9
10 Things you'll need:
11 * a [DHCP] server, preferably ISC DHCP version 3 or later. (dhcpd3-server in woody)
12 * a [TFTP] server that supports PXE. (tftpd-hpa in woody)
13 * [PXELinux] set up on your server.
14
15 This also makes use of the debian package 'diskless' to create the NFS root filesystem. If you dont run debian you'll need to find another way to make a nfsroot fs suitable for your distribution.
16
17 !Procedure:
18
19 !Kernel Compilation
20 Compile a kernel that is suitable for booting via NFS. In particular, you'll need to make sure that the kernel supports IP level autoconfiguration and mounting NFS root. Also, its probably a good idea to compile in the drivers for your network cards, as opposed to leaving them as modules.
21
22 CONFIG_NET=y
23 CONFIG_UNIX=y
24 CONFIG_INET=y
25 CONFIG_IP_PNP_BOOTP=y
26 CONFIG_IP_PNP_RARP=y
27 CONFIG_NFS_FS=y
28 # CONFIG_NFS_V3 is not set
29 CONFIG_ROOT_NFS=y
30 # CONFIG_NFSD is not set
31 CONFIG_SUNRPC=y
32 CONFIG_LOCKD=y
33
34
35 You can either compile this by hand or use make-kpkg under debian to install it, but however you do it you'll need to be able to install it into the nfs root we'll create in a second.
36
37 I also compiled my kernel with devfs support, and automount devfs on boot ticked.
38
39
40 !Set up the root fs
41 Because I'm using the 'diskless' package under debian, this bit is fairly easy.
42 diskless comes with a program called 'diskless-createbasetgz' which creates a base.tgz (potato-install style) from a basic system root. It uses debootstrap or similar to download the required packages
43
44 diskless-createbasetgz /tmp/nfsroot/ woody http://ftp.nz.debian.org/debian /tmp/base.tgz
45
46 This creates a new system under /tmp/nfsroot, from the specified mirror. It then tars this up into /tmp/base.tgz
47
48 If you are using diskless, you'll also need to download a copy of diskless-image-simple. NOTE: do NOT install this on your server. Download it manually from packages.debian.org. Copy the deb to /tmp (or to the same dir your base.tgz is in)
49
50 Now, if you run diskless-newimage it will create a new diskless group image in the directory you specify. You must run this program from a directory containing base.tgz and diskless-image-simple
51
52 # cd /tmp
53 # ls -l
54 total 56192
55 -rw-r--r-- 1 root root 55278350 May 11 12:49 base.tgz
56 -rw-r--r-- 1 root root 13742 May 11 12:32 diskless-image-simple_0.3.17_all.deb
57 drwxr-xr-x 19 root root 4096 May 11 12:48 nfsroot
58
59 Now, run diskless-newimage
60
61 # diskless-newimage
62
63 Follow the instructions - the defaults are probably good enough
64
65 NOTE: if you want to add things to your diskless image first, this is a good time to do it
66 chroot /var/lib/diskless/default/root
67
68 This puts you in a chroot jail inside the diskless image. You can use apt-get update and apt-get install to add new packages. EG, I added devfsd support here.
69
70 As this is effectivly a debian install which has only done the first reboot, you may wish to finish the install. Inside the chroot run /usr/sbin/base-config and run as usual.
71
72 Also, if you want to change the template files for /etc on your diskless machines, you can do that here. Edit /usr/lib/diskless-image/template/etc/<file> inside the chroot appropriately, before you run diskless-newhost
73 Note: the templates reside in the above path *inside* the chroot, or else look in /var/lib/diskless/default/root/usr/lib/diskless-image/template/etc/<file>
74
75 You now have a basic nfsroot filesystem. Now you must create a newhost using this image (this allows you to have per-host settings for hostname etc). Use:
76
77 # diskless-newhost /path/to/deskless/root <ip address of host>
78
79 ! Set up the kernel for your nfsroot
80
81 If you made a debian kernel-package, copy it to /var/lib/diskless/default/root/root (or wherever you put your default image), then chroot to /var/lib/diskless/default/root, and run dpkg -i root/kernel-image.....
82
83 You'll also need to copy the kernel image to /tftpboot/. Name it something useful. I called it 'nfsroot-kernel'
84
85 ! Set up TFTP
86 Make a directory for the tftp root dir. I usually use /tftpboot
87 Under debian, make sure you have the following entries in /etc/inetd.conf:
88 tftp dgram udp wait root /usr/sbin/in.tftpd -p -s /tftpboot
89
90 This tells tftp to use /tftpboot as its root - all paths are now relative to this.
91
92 ! Set up DHCP
93 Make sure your dhcpd config file contains something like (for dhcp 2.x, default in Debian Woody):
94
95 allow bootp;
96 allow booting;
97
98 authoritative;
99 ...
100 host nfsroot {
101 hardware ethernet 00:01:02:03:04:05;
102 fixed-address nfsroot.mydomain.tla;
103 filename "pxelinux.0";
104 }
105
106 This tells the client with the mac address 00:01:02:03:04:05 to use the IP address associated with the name nfsroot.mydomain.tla (this only works if you have your DNS set up correctly. You can specify an IP address here instead of a name if you dont have DNS working). It then loads the file pxelinux.0, which is its bootloader.
107
108
109
110 ! Set up PXE
111 Copy pxelinux.0 from the pxelinux distribution under /tftpboot
112
113 Make the directory /tftpboot/pxelinux.cfg - this is where you'll store all your configurations.
114
115 In /tftpboot/pxelinux.cfg create a file called 'default', and add something like the following to it:
116
117 /tftpboot/pxelinux.cfg/default:
118 default nfs
119
120 label nfs
121 kernel netboot-kernel
122 append root=/dev/nfs \
123 nfsroot=10.66.1.1:/var/lib/diskless/default/root/ \
124 ip=dhcp
125
126 ** IMPORTANT: concatenate the 'append' line, and remove the \ characters
127
128 Note, that the default file is the the default configuration file for the client. PXE actually searches first of all for its IP address printed in hexadecimal, then drops a byte off, and continues looking for this until it has to give up and use the default file. This lets you add a config file for each host you boot via PXE.
129
130 This pxe config file sets the default boot option to nfs. It tells it to boot the kernel 'netboot-kernel' which it gets via tftp. It appends the specified kernel options to the kernel. These options are all, as far as I can tell, ABSOLUTELY required.
131
13 MattBrown 132 __Note:__ you can specify NFS mount options via the nfsroot kernel parameter. Simply append them after the root dir. The syntax is nfsroot=[[<server-ip>:]<root-dir>[[,<nfs-options>]. You will need this if you want to mount the filesystem using NFS v3 which is needed if you want to work with large files. (In NFSv2, the default for Linux clients, there is a 2GB filesize limit.). To enable NFSv3 use the "v3" option.
10 IainHogg 133
134 ! Set up NFS
135 You'll need to add something like the following lines to /etc/exports
136
137 # per-host diskless exports
138 /var/lib/diskless/default/10.66.1.104 10.66.1.104(rw,no_root_squash)
139
140 # global diskless exports
141 /home 10.66.1.0/255.255.255.0(rw,no_root_squash)
142 /var/lib/diskless/default/root 10.66.1.0/255.255.255.0(rw,no_root_squash)
143
144 Note that these are different from the suggested options in the diskless package. I found mounting NFS root ro caused a lot problems.
145
146 Restart your nfs server (/etc/init.d/nfs-kernel-server restart in debian) and you are away
147
148
149 ! Booting the client
150
151 Set up your client to boot via PXE, either by using a PXE enabled boot card or using the pxe-on-disk image you can find under [PXE]. This only works with certain cards tho - intel eepro, 3com, realtek 8139, netgear (natsemi chipset), and some others.
152
153 This *should* all go!
154
155 ! Cavaets
156
157 * I found that createbasetgz didn't do a very good job of it. I had to chroot to /var/lib/diskless/default/root and run apt-get -f install to tidy it up.
158
159 * If you still have problems with logging in, and you see init complain about a read-only root fs, you might need to edit the fstab file of each client. Look in /var/lib/diskless/default/<client-ip>/etc and change the 'ro' options in /etc/fstab to 'rw'. You can set this for all new clients by editing /var/lib/diskless/default/root/usr/lib/diskless-image/template/etc/fstab and changing it there.
160
161 * Re-running diskless-newhost on an IP that you've already used will update its /etc configuration.

PHP Warning

lib/blame.php:177: Warning: Invalid argument supplied for foreach() (...repeated 5 times)