Debian on USB
From FlimzyWiki
My employer has a need for a highly-available Linux router. It will be located about 4-hours drive from our office, so if it fails, we won't be able to replace it quickly. To accomodate this, I am intending to set up redundant Linux firewall/router machines using keepalived, and various other technologies to make them as reliable as possible.
Contents |
Hard-diskless booting options
One way to improve on the physical reliability of these machines is to reduce the number of moving parts in them. One particular piece of hardware that is known to fail frequently is the hard drive. For a simple Linux router and firewall, a hard drive is hardly a necessity. There are many methods out there for booting Linux without the use of a hard drive:
Floppy Disk
This option was easily ruled out for my application, for a number of reasons. First off, I do not want to be limited by the capacity of a floppy disk. There are many floppy-based distrubutions out there, and many of them work for simple router applications. But we may need to install custom applications on our routers in the future, and being limited by these trimmed-down distributions is not ideal. Secondly, floppy disks really aren't every reliable, either. If our objective is high reliability, then trading a hard drive in for a floppy disk really isn't much of an improvement. The only possible advantage a floppy has in this type of scenario is that it is only accessed once during boot, then sits idle until the next reboot. It is also relatively easy to swap floppy disks out if one should become unreadable. In any case, I chose not to persue this course.
CD-ROM
There are many Linux distributions now that can boot from a CD-ROM, and many of them act as routers. A CD-ROM does not have the serious space constraints that a floppy disk has. CD-ROM media is typically more reliable than floppy disks, and replacing a CD-ROM should it die is also not a big deal. But CD-ROMs do not allow us to store configuration changes on the media. It could be possible to store configuration changes over the network, and retrieve them on boot via TFTP or NFS, but that adds another layer of physical complexity, and another point of failure. We do not want to depend on our router _and_ a network file server both working 100% of the time for our router to remain online. So the CD-ROM option is not viable for us.
Netboot
One option would be to boot over a network, or to at least mount a remote filesystem. But as mentioned above in the discussion about CD-ROM booting, this adds another point of failure to our setup which is not desirable for our application.
USB Flash
USB Flash does not have the space constraints of a floppy disk. It does not have the inability to write that a CD-ROM has. It does, however, have the annoying trait that flash media wears out over time--particularly after writing to it many times. According to my research, typical flash media can be written to up to 100,000 times before it stops working reliably. That means 100,000 per bit on the media. So if one area of the media is written to routinely, it could wear out relatively quickly. This makes running a live operating system of any sort--espeically one in a high-reliability environment--not very realistic. Many files are created, deleted, updated, moved, renamed, recreated, etc, during normal operation of even a relatively idle Linux system. And what's worse--while many of those writes may be spread out physically over the media, depending on the filesystem in use, many of the changes will affect writes to very specific portions of the disk over and over again.
USB Flash RAM Disk
This approach is common, and is essentially the same approach that is made by most CD-ROM based live distributions such as Knoppix and Ubuntu. Booting from USB Flash, then running the OS from RAM disk avoids the problem of excessive writes to the flash media, but also introduces the problem of an inability to write to the physical media. The USB Flash RAM Disk options out there now are basically the same as a CD-ROM boot option, but use a USB Flash drive instead of a CD-ROM as the boot media.
My Solution
For lack of a better name, I call my solution "Debian on USB". Perhaps I will think of a more clear name later. It is built using almost exclusively Debian tools. The only exceptions are some custom modifications to casper, and a couple custom scripts.
I have prepared complete step-by-step instructions to Install Debian on USB using my method. There is also the Debian on USB Quick Install guide if you want to take some short cuts, although I recommend folloing the full instructions yourself, as you may not want to do things exactly as I did.
In designing my system, I had two primary objectives:
"True" Debianness
That is, the booted system must behave, in every meaningful way, like a "true" Debian installation. I must be able to use apt-get, dpkg, aptitude, or any other tools I choose, just as I could if the system was booted from a standard hard drive.
Controlled Commits
That is, I must be able to control what is written to the USB media, and when. If I perform a dist-upgrade, I want to be able to write the changes to the USB media only once (thus only one write) rather than throughout the process (which would create hundreds or thousands of writes).
The combination of these two should give me the 'best of both worlds', with regard to having a fully-functional Debian system (and should be easily adaptable to other Linux distros if desired), while preserving the life of the USB Flash media as long as possible.
