256
Logo

Soekris Net4801

Soekris on OpenBSD Running Diskless

Running Diskless | Kernel Config | Creating Filesystems | Ramdisk Script | CF Script

The goal of this project was to build a OpenBSD image that ran diskless on my NET4801 Soekris box -- it should work, however, on any hardware and has been utilized to boot standard PC servers diskless. The Soekris boxes are small 486 and Pentium based systems that are designed to be imbedded routers, wireless access points, etc.. They come with a compact flash (CF) card for booting although an IDE port is also provided. PC servers can use CF to IDE/ATA adapters. I initially looked at the flashdist and OpenSoekris scripts and although they seemed like the right tools, they both unfortunately made assumptions about configuration files and did not give me enough fine grained control over the installation process.

Also, neither of the scripts addressed the running diskless scenarios that I wanted although they were not necessarily precluded. I wanted to boot from the CF card for security (as opposed to network boot), but run diskless to not kill the CF card with tons of little read and writes. I found the excellent article written by Francisco Roque on how to get a diskless OpenBSD box using the Raidframe driver. This is a very innovative piece with the end result of an system living only in memory, however, this was a bit more of a hack than I wanted. The mount table, for instance, is very confused with /dev/wd0a still listed even though it is not in use. Also, you cannot remount the compact flash (CF) to save things back to permanent storage because the device is already in use and if you try to have the RAID system rediscover it, this supposedly panics the system.

This documentation file, my scripts, config files, etc. are wrapped up into the soekris256.tgz gzipped tar release file. Please note that the heart of these procedures are the ramdisk and compact flash scripts explained at the bottom of this page but I will talk a bit about other issues before getting to them.

Using bsd.rd to Run Diskless

After poking around a bit I found the bsd.rd ramdisk kernel which is usually used to recover a filesystem. Thanks to attila for pushing me in this direction. The ramdisk filesystem is compiled into the kernel via the kernel source file sys/dev/rd.c. You use the rdsetroot program from the kernel source directory distrib/common to copy a filesystem image that you build, into the kernel. To build a ramdisk enabled kernel you add the following settings to your kernel configuration file:

option  RAMDISK_HOOKS             # enable mini root hooks
option  MINIROOTSIZE=23000        # mini root size of 11.5mb
...
# root lives on rd0a (ramdisk) with swap maybe on rd0b or wd0b disks
config          bsd     root on rd0a swap on rd0b and wd0b
...
pseudo-device   rd         1      # ramdisk device driver

So initially I thought that I could build everything that I needed into the kernel ramdisk but unfortunately, there are kernel linking and runtime limitations on the size of this filesystem. After experimentation, I determined that a miniroot size of 23000 blocks (11.5mb) was about the maximum size which works. Sizes of 100000 (50mb) or above fail at the kernel link time, probably due to some sort of address violation (bleah). 30000 (15mb) sizes cause the kernel to immediately reboot, probably exceeding some boot or AT architecture limitation (grumble). 25000 (12.5mb) seems to hang immediately after kernel load (sigh). So 23000 (11.5mb) is the magic working size. I hear reports that under OpenBSD 3.7, numbers above 17000 fail. I figured that I can get just about all of the files from / that I want into this size. The other partitions would reside on the compact flash and would be copied into memory filesystems created at boot time by the /etc/rc boot script.

To facilitate the process of building the ramdisk filesystem image to be written into the kernel and the CF image to be loaded onto the CF card to be inserted into the system, I decided to roll my own scripts. Thanks much to Chris Cappuccio and Ron Rosson for the flashdist script. I copied from it in a couple of places, especially most of the fdisk, disklabel, and boot block code.

Håkan Lindqvist's experience so far tells him that the memory size of the kernel plus the ramdisk must be less than 16mb (32768 blocks). If you take the time to remove unnecessary drivers from the kernel, you should have more space for your ramdisk.

Mark Redding was about to go up to a size of 26325 if you have the following in your /etc/disktab file. I've not tested it.

rdroot26325|ramdiskroot26325|RAM-disk root 26325 image:\
        :ty=ramdisk:se#512:nt#2:ns#960:nc#16:\
        :pa#26325:oa#0:ta=4.2BSD:ba#4096:fa#512:\
        :pb#0:ob#0:tb=swap:\
        :pc#26325:oc#0:

Kernel for Your Soekris Box

Download NET4801 kernel config file.

This is the kernel config that I am using for my NET4801 Soekris. I got this from the latest flashdist script version. There are ones for other devices in that package. Below are a couple of important settings for the NET4801 and to get it to boot diskless.

option  RAMDISK_HOOKS             # mini root hooks
option  MINIROOTSIZE=23000        # mini root size of 11.5mb
...
config          bsd     root on rd0a swap on rd0b and wd0b
...
option PCCOMCONSOLE       # default is to use the pccom console
option CONSPEED=19200     # at 19200 baud
...
# Drivers available in OpenBSD 3.4
geodesc* at pci? dev ? function ?               # NSC Geode System Controller
elansc*  at pci? dev ? function ?               # AMD Elan SC520 System Control
...
sis*    at pci? dev ? function ?                # SiS 900/7016 ethernet
...
# Wireless network cards
wi*     at pci? dev ? function ?                # WaveLAN IEEE 802.11DS
an*     at pci? dev ? function ?                # Aironet IEEE 802.11DS
...
# crypto support
hifn*   at pci? dev ? function ?                # Hi/fn 7751 crypto card
...
pseudo-device   crypto     1                    # crypto hardware support
...
pseudo-device   rd         1                    # ramdisk device

Creating Your Filesystems

To configure what files will be build into the Ramdisk and CF images, you need create a directory of files for your ramdisk and CF images. The directories should hold all of the files that you'd like to be in the images. I found it much easier to manage the files in a directory hierarchy instead of a list of files and distributions. This allows you to edit configuration files in place, tune permissions, etc..

You can use one directory for both images or a separate one for each image. When copying files into the ramdisk filesystem, the ramdisk script will ignore the /boot and /bsd* files as well as the /usr directory. Also it will look for any files or directories with a .rd extension and will ask if you want to rename them to the same name without the extension. The CF script will do the same for files with a .cf extension. That means that you can have a /etc/fstab.rd and a /etc/fstab.cf file (for example) and the scripts will move the correct file (if you agree) to /etc/fstab when building the image. I have the above fstab files and /etc/rc.diskless.rd (see below for explanation of this file).

You can use any number of mechanisms to create your directories. I took an unused Pentium system I had around and loaded a virgin copy of OpenBSD 3.4 on it. I only loaded the 'bsd', 'base', and 'etc' packages. I them went though the directories and using du | sort -rn identified the largest files and directories that I would not need that could be removed. Here are the files and directories that I removed from the OpenBSD 3.4 distribution. You may need to tune and feel free to point out to me important files that I removed in ignorance.

After a bit of work I was down to ~40mb which with 64mb of memory would leave ~20mb for kernel and program runtime memory. If you have 128mb of memory then you don't have to pair it down as much -- see the machine memory line in the boot.conf if not all of the memory in your box is recognized by the kernel. I then tar'd the system up and copied in over to the system where I was building the images. The archive should be unpacked and the files edited as root to preserve the permissions of the standard install. To copy a directory, for example, make sure to use cp -pR as root to preserve all permission settings.

Kernel Ramdisk Partition

Download my list of files in my ramdisk partition.

I moved the /var into /usr/var and put a symlink from /var to /usr/var. I then divided the / from the /usr directory and set about to fine tune the / partition to get below the image limit if necessary. Remember that the ramdisk image does not include the kernel /bsd itself nor the second-stage /boot. Due to filesystem limitations, you don't want to be right at the limit (11.5mb with a kernel ramdisk size of 23000) so I shot for 10mb. I figured that if there were any files that needed to be in / I could create a /usr/root/ hierarchy and symlink up to / if necessary with the /etc/rc boot script.

So the boot process would be:

  1. Load the first stage bootstrap code from the CF.
  2. Read the second stage boot program from the CF in /boot.
  3. This reads and uses the boot configuration file from the CF /etc/boot.conf.
  4. Load the ramdisk kernel from the CF which is probably /bsd.
  5. Discover all of the devices, etc..
  6. Run the /etc/rc runtime configuration boot script from the ramdisk image.
  7. The rc script creates the memory file system for /usr and copies the files into it from the /usr on the CF.
  8. The rc script from the ramdisk kernel image continues to run and the system boots up into multiuser mode.

Special Files for the Ramdisk Partition

Compact Flash (CF) Partition

Download my list of files in my CF partition.

The CF partition contains all of the files from the ramdisk image as well as the second-stage boot program and the kernels. I use a ramdisk kernel as the default bsd and a kernel for booting from the CF device named bsd.wd which is just the same kernel without the MINIROOT options enabled. I made /var a symlink to /usr/var. Initially I would then tar up /usr and gzipped it into /usr.tgz. I had to recompile gzip from the sources to link it statically because otherwise it depended on shared libs in /usr. I ran it as gzip -dc usr.tgz | tar -xpf - because tar expects gzip to be in /usr/bin/gzip if you use -z and it can't be there since /usr doesn't exist yet.

More recently I've just left the /usr partition and let the rc script copy the files using a cp -rP of the /usr directory. If you have a small CF card (I use a 256mb one) this allows you to save space by not having the archive of the /usr files. If you never will be booting from the CF, you can just have the files in the archive only to save more space. The /etc/rc.diskless script in the ramdisk partition will ungzip a gzipped tar file, untar a tar file, or copy the files from /usr, whatever is available.

Special Files for the CF Partition

Compact Flash (CF) Hardware

Couple of quick comments about CF hardware with the Soekris boxes.

Ramdisk Script

Download ramdisk script.

This script is used to create the disk image file image_rd. Using the rdsetroot utility, it will then copy the image into a kernel file which has been configured with the ramdisk hooks. It should be run as root.

Requirements:

What the Ramdisk script does:

  1. Prompts for the name of the output image file. Default is ./image_rd.
  2. Verifies that rdsetroot utility can be executed from the local directory.
  3. Prompts for the name of the miniroot kernel. Default is bsd.
  4. Prompts for the number of blocks that the variable MINIROOTSIZE is set to in the kernel config file. The default is 23000.
  5. Prompts for the vnode device to use. The default is svnd0. If you don't know what this is then just press enter.
  6. Prompts for the name of the image files directory where you put the files to be copied into the image. This defaults to root_rd in the current directory.
  7. If the image file image_rd exists, the script will warn you and ask if you want to overwrite it.
  8. If overwrite is confirmed or the image file doesn't exist, it will be created by copying the appropriate number of blocks from /dev/zero.
  9. Maps the image file to the vnode device.
  10. Creates a disklabel for the image with the 'a' partition added.
  11. Builds a new filesystem on the 'a' partition.
  12. Mounts the new filesystem on the ./__mount_rd__ temporary directory.
  13. Copies the files from the image file directory into the new filesystem. It will not copy the boot or any files starting with bsd since these should only be in the CF image.
  14. Unmounts and unmaps the image.
  15. Writes the image into the kernel.
  16. Prompts if you want to copy the kernel into the default CF image file directory (root_cf). Default is yes. If you say no you will have to copy the kernel into place by hand.

CF Script

Download CF script.

This script is used to create the disk image in the file image_cf. This image should then be copied to your compact flash card. The script should be run as root.

Requirements:

What the CF script does:

  1. Prompts for the name of the output image file. Default is ./image_cf.
  2. Prompts for the size in megabytes of the CF card. It understands 32, 64, 128, and 256. If the size of your card isn't in the list or if there are problems with the default settings for these sizes then you should press return at the size prompt and the script will prompt you for the number cylinders, tracks per cylinder, and sectors per track.
  3. Calculates the total partition size, sectors per cylinder, and the size of the 'a' partition and displays it for your confirmation.
  4. Prompts for the vnode device to use. The default is svnd0. If you don't know what this is then just press enter.
  5. Prompts for the name of the image files directory where you put the files to be copied into the image. This defaults to root_cf in the current directory.
  6. If the image file image_cf exists, the script will warn you and ask if you want to overwrite it.
  7. If overwrite is confirmed or the image file doesn't exist, it will be created by copying the appropriate number of blocks from /dev/zero.
  8. Maps the image file to the vnode device.
  9. Updates the partition map with the correct device information. It uses the master-boot-record template from usr/mdec/mbr in the image files directory.
  10. Creates a disklabel for the device with the 'a' partition added. There's a whole bunch of output from this command.
  11. Builds a new filesystem on the 'a' partition.
  12. Mounts the new filesystem on the ./__mount_cf__ temporary directory.
  13. Copies the second-stage boot file named boot from the top of the image files directory into the new filesystem.
  14. Installs the boot blocks using the first-stage boot file named usr/mdec/biosboot from the image files directory.
  15. Copies the files from the image files directory into the new filesystem. First come the kernel files (bsd*), then the rest of the files (not including boot).
  16. Unmounts and unmaps the image.
  17. You now need to take the image file and write it to your compact flash device using dd. Make sure you are using the correct device otherwise you can blow away your hard-disk or other devices.
    dd if=image_cf of=/dev/??? bs=1k

Summary

So when you are done, you should be able to boot your box from the CF. After the kernel loads and the devices show up, you should see the console messages that the /usr is being created and populated. When you finally get a login: prompt and you login, df should show you the kernel ramdisk / on /dev/rd0a and a memory filesystem /usr as mfs:... and you should be free to mount and unmount the CF device as /dev/wd0a at will.

Enjoy. Comments welcomed.

Copyright 2004 Gray Watson
http://256.com/gray/docs/soekris_openbsd_diskless/

Free Spam Protection   Eggnog Recipe   Android ORM   Simple Java Magic   JMX using HTTP