Experiment 2

Sharing EROS cross-compilers from a diskless server

1 Introduction

The EROS [1] cross-compilers can be shared via NFS from laboratory server disks, this is useful when there are many persons building EROS simultaneously, and when there is the need to build on different combinations of systems and processors. This experiment was an attempt to provide integrity assurance for cross-compiler programs on the physical medium, while retaining an unmodified cross-development tree, and to provide protection of operating system files for the server that shares the cross-compilers via NFS. Also, it was desirable to use a simple and rapidly implementable method that would not complicate excessively the maintenance procedures.

In the course of the experiment two minimal Linux distributions were reviewed. While one was not viable due to the excessive amount of work necessary to obtain a working system for this project, the other was adaptable with a very small amount of work, and met the requirement of being readily useable for the purpose.

2 Implementation

2.1 The operating system: Trinux

Trinux is a minimal Linux distribution aimed at security, network monitoring, and other useful tasks. The system is ramdisk-based. More accurate details about Trinux can be found on the Trinux web site [2].

In the past we used Trinux boot floppies for various tasks of hardware diagnostics and network monitoring, acquiring expertise with the system. Subsequently to review, Trinux has been recorded as a commodity tool into the toolchains list of the laboratory. We were not satisfied with the floppy bootstrap times greater than ninety seconds (as measured from power-on to system prompt, because all daemon programs are started by the linuxrc script before the system prompt, since the shell is the last program started). We then decided to switch to the usage of the CD-based version.

2.2 Building a Trinux CD and including the EROS-XENV

A set of three shell scripts that facilitates the modification of the Trinux initial ramdisk was developed. The scripts have to be run in sequence. Using gzip and a loop-mounted filesystem, the scripts manipulate the compressed initial ramdisk image. The first script expands the initial ramdisk into a temporary directory for modification, the second script compresses the modified directory back into a ramdisk. The third script automates the creation of an ISO image using the mkisofs utility, then taking the ISO image as input, it writes the CD using the cdrecord program. Using such scripts, a Trinux cd-rom was built, containing the same cross-compilers tree that we share from our laboratory server, as compiled directly from the source tarball available on the EROS web site. Grace to the scripts, the entire process is reproducible without effort.

2.3 The boot process

On boot, after the initial configuration, the system mounts the cd-rom under the /user directory, starts the portmap server, starts the mount daemon, and starts eight threads of the User-space NFS server that exports the /user directory containing the eros-xenv tree. All commands are issued from the linuxrc file.

The relevant NFS configuration file is /etc/exports as usual, but the number of NFS daemon threads can be specified on the command line. In our case we can safely use the option no_root_squash.

The latest version of the portmap server is protected by TCP wrappers, a line must be added in the /etc/hosts.allow file to enable access to the portmap server from the network.

2.4 Troubleshooting

In our laboratory we use automount, and we mount the /user directory with the "hard" option, because we want to block build processes upon server problems. The Trinux system on cd-rom is configured with static IP address, so there are no problems with machine replacements.

3 The test machines

3.1 The server

Our laboratory server is currently an Intel Pentium II (Klamath) stepping 04 at 266MHz [BogoMIPS=532.48], with a TekRam DC-390F controller, two SCSI-II disks, a 24X IDE cd-rom, the network interface card is a 3Com PCI 3c905B Cyclone 100baseTx.

3.2 The client

The client used for the test build is an Intel Pentium III (Coppermine) stepping 01 at 450MHz [BogoMIPS=897.84] with one IDE disk, and a 3Com PCI 3c905B Cyclone 100baseTx network interface card.

3.3 The network

The network device is a 10/100 ethernet switch with full duplex enabled. All cables are three meters long, standard Cat5 copper wires of EIA/TIA 568A type.

3.4 Note on BogoMIPS

We are aware that the BogoMIPS values do not constitute an absolute scientific performance indicator, but we have included them here as they provide some hint of the performance range of our machinery, hint that we hope can be correlated with the numbers reported below in section 4.

4 Measurements

Performance of the EROS build process and bootstrap times of the server were compared using both the laboratory server disk and the diskless method. NFS is the only service running on our server, started either from hard disk or cd-rom. To ensure that the measurements were consistent, both tests were conducted using the same machines and eight NFS daemon threads. The build was executed from the text console, without the XFree/KDE graphic environment. Performance measurements were made using the Linux time command. Inside the ./eros/src/ directory, the command line issued to start the EROS build was time make pristine

4.1 Build performance with Pentium-II server

+ EROS Build with SCSI-II Hard Disk (Red Hat Linux 7.3)

+ EROS Build with 24X IDE CD-ROM (Trinux 0.890)

4.2 Bootstrap times of Pentium-II server

+ Red Hat Linux 7.3 boot from SCSI-II Hard Disk: 88 seconds

+ Trinux 0.890 boot from 24X IDE CD-ROM: 56 seconds

NOTE: In this particular case the machine's SCSI controller has to scan the SCSI bus up to 15 devices, Trinux boot time on a truly diskless machine is even shorter. The shortest time achievable is when the BIOS is set with no device autoconfiguration probing.

5 Other Systems Tested

5.1 Morecram

Morecram [3] is a minimal Linux distribution, it is a specialized version of Cramdisk that creates an instant NFS server floppy disk. On boot the system automatically mounts the cd-rom and shares its contents. A first attempt was made using Morecram-1.0, the attempt has failed. Various issues were found that could not be fixed in reasonable time. The bundled setfdprm and superformat utilities failed to run on our test workstation, it was necessary to download and recompile a fresh fdutils-5.4 package [4].

Morecram uses 1920K formatted floppies, but the superformat utility was unable to properly format our floppies, the format process resulted always in broken, unreadable floppies. At some point in the course of work, we realized that Morecram 1920k floppies could be even slower at the boot phase than Trinux 1440k floppies seen the increased quantity of data to load. Unfortunately, we were unable to verify this hypothesis. Furthermore, the Morecram system is currently unmaintained since 1998. For all the above reasons we have abandoned Morecram.

6 Related Considerations

There is the possibility to install Trinux on an hard disk unit, and some machines come with a BIOS option that allow to write-protect attached devices. That is an oppotunity that we plan to experiment, however, some considerations can be made. We believe that the CD-based method is a better solution for two aspects: integrity assurance, and system availability.

+ Because it forces the build of a new medium each time a modification is necessary, instead of allowing modifications in place, the CD-based method is safer, since modifications in place can leave us with an unusable system, and might require additional system recovery procedures. In case of human error or other types of failures that may occur during modification, the CD-based method ensures that the previous system image remains available.

+ The CD-based method benefits of a greater availability, since it does not require downtime during modifications. It does not force developers to stop working while the system is being modified.

7 Conclusion

The experiment consisted in implementing a secure method to share the EROS cross-compilers safely over an open network. We are currently using the cd-rom in our laboratory on a Pentium/166MHz class machine, and we are satisfied of our EROS build speed. It remains to be seen what is the performance when a high number of clients builds simultaneously. We are not currently in a position to experiment that in our laboratory, unless we set up a simulation environment, however we have no plans to do so.

At the beginning of the experiment it was immediately evident to us that the simplest method was to use write-protected floppy disks, or cd-roms. Beside the failure of the first attempt with Morecram, we could always use a Trinux floppy, and a number of methods to load packages from the network, but that would not guarantee integrity assurance for the operating system, would complicate the maintenance procedures, and the boot time would be longer than with cd-rom. Conversely, after building carefully the cd-rom, we are guaranteed that the integrity of the entire system is maintained in time.

Having attempted both methods led us to understand better the value of an operating system that boots from cd-rom, as the medium that ensures immutability by virtue of its read-only property also protect us from inadvertent modification. Human error is the first cause of trouble, and in a laboratory this is an especially critical issue. According to our current knowledge of the EROS cross-development environment, the cross-compilers versions are stable enought that the read-only medium poses no compelling issues related to maintenance and upgrade procedures that imply a rebuild of the CD. To date, we haven't encountered any restrictions in the daily usage of this method. Therefore, we are confident that its usage can improve the overall reliability of the build process in the lab, however it can be used only when the build environment is stable [5].

8 References

[1] The EROS Web Site, http://www.eros-os.org, The EROS Group, LLC

[2] Trinux web site, http://www.trinux.org

[3] Morecram, FTP from sunsite.unc.edu, directory /pub/Linux/system/recovery/images

[4] fdutils, http://fdutils.linux.lu/

[5] Errata corrige: 21.2.2006