FreeBSD and Solid State Devices

The scope of this article will be limited to solid state disk devices made from flash memory. Flash memory is a solid state memory (no moving parts) that is non-volatile (the memory maintains data even after all power sources have been disconnected). Flash memory can withstand tremendous physical shock and is reasonably fast (the flash memory solutions covered in this article are slightly slower than a EIDE hard disk for write operations, and much faster for read operations). One very important aspect of flash memory, the ramifications of which will be discussed later in this article, is that each sector has a limited rewrite capacity. You can only write, erase, and write again to a sector of flash memory a certain number of times before the sector becomes permanently unusable. Although many flash memory products automatically map bad blocks, and although some even distribute write operations evenly throughout the unit, the fact remains that there exists a limit to the amount of writing that can be done to the device. Competitive units have between 1,000,000 and 10,000,000 writes per sector in their specification. This figure varies due to the temperature of the environment.

Specifically, we will be discussing ATA compatible compact-flash units, which are quite popular as storage media for digital cameras. Of particular interest is the fact that they pin out directly to the IDE bus and are compatible with the ATA command set. Therefore, with a very simple and low-cost adaptor, these devices can be attached directly to an IDE bus in a computer. Once implemented in this manner, operating systems such as FreeBSD see the device as a normal hard disk (albeit small).

Other solid state disk solutions do exist, but their expense, obscurity, and relative unease of use places them beyond the scope of this article.

A few kernel options are of specific interest to those creating an embedded FreeBSD system.

All embedded FreeBSD systems that use flash memory as system disk will be interested in memory disks and memory filesystems. As a result of the limited number of writes that can be done to flash memory, the disk and the filesystems on the disk will most likely be mounted read-only. In this environment, filesystems such as /tmp and /var are mounted as memory filesystems to allow the system to create logs and update counters and temporary files. Memory filesystems are a critical component to a successful solid state FreeBSD implementation.

You should make sure the following lines exist in your kernel configuration file:

options         MFS             # Memory Filesystem
options         MD_ROOT         # md device usable as a potential root device
pseudo-device   md              # memory disk

The post-boot initialization of an embedded FreeBSD system is controlled by /etc/rc.initdiskless.

/etc/rc.d/var mounts /var as a memory filesystem, makes a configurable list of directories in /var with the mkdir(1) command, and changes modes on some of those directories. In the execution of /etc/rc.d/var, one other rc.conf variable comes into play – varsize. A /var partition is created by /etc/rc.d/var based on the value of this variable in rc.conf:

varsize=8192

Remember that this value is in sectors by default.

The fact that /var is a read-write filesystem is an important distinction, as the / partition (and any other partitions you may have on your flash media) should be mounted read-only. Remember that in Section 1, “Solid State Disk Devices” we detailed the limitations of flash memory - specifically the limited write capability. The importance of not mounting filesystems on flash media read-write, and the importance of not using a swap file, cannot be overstated. A swap file on a busy system can burn through a piece of flash media in less than one year. Heavy logging or temporary file creation and destruction can do the same. Therefore, in addition to removing the swap entry from your /etc/fstab, you should also change the Options field for each filesystem to ro as follows:

# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ad0s1a             /               ufs     ro              1       1

A few applications in the average system will immediately begin to fail as a result of this change. For instance, cron will not run properly as a result of missing cron tabs in the /var created by /etc/rc.d/var, and syslog and dhcp will encounter problems as well as a result of the read-only filesystem and missing items in the /var that /etc/rc.d/var has created. These are only temporary problems though, and are addressed, along with solutions to the execution of other common software packages in Section 5, “System Strategies for Small and Read Only Environments”.

An important thing to remember is that a filesystem that was mounted read-only with /etc/fstab can be made read-write at any time by issuing the command:

# /sbin/mount -uw partition

and can be toggled back to read-only with the command:

# /sbin/mount -ur partition

Since ATA compatible compact-flash cards are seen by FreeBSD as normal IDE hard drives, you could theoretically install FreeBSD from the network using the kern and mfsroot floppies or from a CD.

However, even a small installation of FreeBSD using normal installation procedures can produce a system in size of greater than 200 megabytes. Most people will be using smaller flash memory devices (128 megabytes is considered fairly large - 32 or even 16 megabytes is common), so an installation using normal mechanisms is not possible—there is simply not enough disk space for even the smallest of conventional installations.

The easiest way to overcome this space limitation is to install FreeBSD using conventional means to a normal hard disk. After the installation is complete, pare down the operating system to a size that will fit onto your flash media, then tar the entire filesystem. The following steps will guide you through the process of preparing a piece of flash memory for your tarred filesystem. Remember, because a normal installation is not being performed, operations such as partitioning, labeling, file-system creation, etc. need to be performed by hand. In addition to the kern and mfsroot floppy disks, you will also need to use the fixit floppy.

In Section 3, “The rc Subsystem and Read-Only Filesystems”, it was pointed out that the /var filesystem constructed by /etc/rc.d/var and the presence of a read-only root filesystem causes problems with many common software packages used with FreeBSD. In this article, suggestions for successfully running cron, syslog, ports installations, and the Apache web server will be provided.

# touch /var/log/security /var/log/maillog /var/log/cron /var/log/messages
# chmod 0644 /var/log/*

# mkdir /etc/pkg

# ln -s /etc/pkg /var/db/pkg

# chmod 0774 /var/log/apache
# chown nobody:nobody /var/log/apache

# rm -rf apache_log_dir
# ln -s /var/log/apache apache_log_dir