What Should Be Incorporated Into BSD/OS

Background Information

I have been a longtime user of the BSD/OS operating system. I started using it before it was called BSD/OS, back when it was called BSD/386 (version 0.9.3). Over the course of several years while I worked at UUNET, I had the opportunity to fund the incorporation of certain features into the BSD/OS operating system. At UUNET, we thought these features to be important enhancements that would be valuable to UUNET down the road when deploying applications on top of this platform. Like many experimental software features, some of these were useful, some were not, some never actually got finished and/or delivered in a useful manner.

Without a doubt, the most important, forward thinking feature that UUNET was willing to fund in BSD/OS was the inclusion of a truly symmetric, multi-processor design for the kernel. BSDi started this implementation process, did a pretty good design and delivered a not-ready for prime-time implementation of this system on both the UltraSparc and x86 architectures in 1999.

The funding for this project provided the groundwork for two important projects spawned from the same design work:

• The SMPng kernel design was taken from BSD/OS and given to the FreeBSD project in early 2000. This design forms the basis of the the SMP technology that FreeBSD is currently implementing (2000, 2001). Please note -- this is not an attempt to sell short the FreeBSD's teams efforts -- they have worked pretty hard at making this work in their kernel tree. As of this writing (Oct 2001), however, it's not yet ready for prime time.
• The ``BSDng'' work that Wind River's BSD/OS group is currently working on in their ``Itasca'' project. (The name for this was finally uttered publicly at the BSDCon 2002 conference, so I figure it is no longer a secret.) I like to think of this project as the culmination of the operating system work that I helped to guide through funding the design and implementation process in 1998 and 1999. I believe this fundamental reworking of how the BSD kernel works internally might be the largest pervasive change to BSD kernel since the introduction of a pageable VM system in the 3BSD release.

All that being said, the concepts listed below are what I think ought to be incorporated into BSD/OS. Of course, it's always been easier to be the ``armchair quarterback'' of specifying this, that and the other thing, than actually being the one that have to make the code work. All of these comments apply to the BSD/OS 4.3 release, which is the currently shipping version of the operating system. I will update this document after the next version ships (or at least a Beta release is available).

Many of these ideas are not new. I sent some of these ideas to to BSDi in 1997, as part of a list of things we might be willing to fund. An updated list was sent to BSDi in 1999 during another contract negotiation period. I sent a revised list to BSDi in 2000. And I sent a newly revised list again in 2001, a week after I quit my job at UUNET. I have not sent a revised list to WRS since they bought the software assets of BSDi. But, I started this web page instead, and made various members of the WRS technical staff aware of its existence. I've been told that this document has seen limited circulation internal to WRS.

I thought sharing my ideas about BSD/OS in a public forum would be a good thing. Without cohesive thoughts about the future of the BSD/OS operating system, who knows where it might be pushed. Lack of direction is a horrible thing -- many large software projects have been killed due to lack of vision. I've seen it happen up close, and it is a terrible thing to see happen.

Application Enhancements List

Certain applications that are shipped with BSD/OS are ``contributed software.'' Updating them to more current versions would alleviate the need for customer sites to have do this again and again. These could probably be fairly easily updated by non-intrusive patches to the current released and supported versions of BSD/OS.

Ship a newer version of BIND (Added November, 2002)(KJL)

The BIND release that BSD/OS 4.3 ships with is old, and has had numerous security problems during its lifetime. A newer, better engineered, stable major version number release has been available for quite a while. BIND9. This is the version of the nameserver that gets patches released for it first, and is thought to be fundamentally more secure than any BIND8.x release. Note: I am an employee of ISC, the makers of the BIND software product. While I do not work on BIND, I work with people that work on BIND.

Ship a newer perl version (Added November, 2002)(KJL)

The perl interpreter that BSD/OS 4.3 ships with is old, old, old. Two newer stable versions have been released since the release that ships with BSD/OS. Perl 5.8.0 is the current stable release, and the well-regarded Perl 5.6.1 release has been stable for over a year. Many free software projects require some of the advanced debugging and diagnostic features of a more modern perl than the old Perl 5.00503 that is shipped with BSD/OS 4.3.

Ship a newer groff version (Added October, 2002)(KJL)

The groff installation that BSD/OS 4.3 ships with is old and out of date. Several newer versions have been released. Of note to European customers is the support for Euro symbol printing in groff 1.18 and newer releases.

Ship a newer ghostscript version (Added November, 2001)(SMS)

The ghostscript PostScript interpreter that BSD/OS 4.3 ships with is old. Several newer versions have been released under the GPL. The newest GPL'd release should be included.

Update JAVA support (Added November, 2001)(SMS)(Updated March 2002)(KJL)

Under BSD/OS 4.2, there was support for the JAVA 1.2.1 JRE and JDE. This was useful for doing small-scale Java development and debugging. Lack of a JIT compiler makes the JAVA runtime exceptionally slow, such that the JRE is almost useless for running any large-scale applications. In BSD/OS 4.3, the Java component was dropped from the product release, due to reported licensing difficulties.

The Java support should be re-introduced and updated to the current release of Java -- version 1.4. Making the hotspot JIT compiler work would be a nice improvement as well. Without this level of native platform Java support, it will be impossible to have a Java applet support for Mozilla, among other consequences.

Update PINE mailer (Added November, 2001)(SMS)

Update the contributed software of the pine mailer to the latest release. The included version is very old and many improvements have been made since that version was released.

LAP enhancements (Added November, 2001)(SMS)

While the LAP emulation environment is pretty good and a novel way of using the dynamic library linker, a small amount of kernel support would probably go a long way to making this emulation more complete.

Ship the fvwm2 window manager (Added October, 2001)

The fvwm window manager that BSD/OS ships with is old. Ancient, actually. It's the last stable patch release of the original fvwm code base (version 1.24r), and is completely unsupported and not being maintained. The developers have abandoned the code, by and large, and have moved onto newer versions of fvwm2. The current release of the stable branch is 2.4.3. More details are available at http://www.fvwm.org.

Update enlightenment to v16.5 (Added October, 2001)

The latest stable release of the enlightenment window manager. This version has been released for just shy of a year (only 5 more days to go to be a year, as of this writing) and considered highly stable. This version is one that support Xinerama. Rather than force the users that wish to use enlightenment to go and compile it themselves, update the shipped version. More details are available at http://www.enlightenment.org.

Update Apache to version 1.3.22 (Updated October, 2001)

The latest and most secure version of Apache. Given that many of the sites that are running BSD/OS provide web services, having an out of date version of the webserver is counterproductive. Tracking the Apache releases closely would be worthwhile.

Various IPFW improvements (Added September, 2001)

This was a list of improvements that I felt would make IPFW work better or fill in gaps in the system. These were sent as mail to the Request For Enhancement (rfe@bsdi.com) mailing alias, and nothing else has been heard. So, I've dusted off my list, turned it into HTML and made it available.

Integrate KerberosV Support (Added November 1999)

The KerberosIV support in BSD/OS is ancient. It has no hope of ever working on anything other than a single interface, IPv4 machine. MIT has stopped supporting this codebase and no new development is happening in the world of kerberosIV. KerberosV is superior to KerberosIV in almost every way -- except for code size -- it is huge. However, the superior administration tools, the possibility to work over more than just IPv4 networks, and the greater range of flexibility in the system are a worthwhile tradeoff.

Kernel Enhancements List

The following list of things apply more to enhancements and additions to the BSD/OS kernel. Often there would be related work in certain pieces of user software that would need to be added or modified.

Fix softrdonly mount option support (Added March 2002)(KJL)

The BSD/OS system introduced the softrdonly mount option in 4.0 BSD/OS. This option is supposed to allow for selectively upgrading and downgrading a filesystem mount from rw to ro on the fly. If there are no open writable file descriptors on the filesystem, it should have all dirty blocked flushed to the disk and have the mount downgraded to read-only and the superblock marked clean. If a write operation takes place to the filesystem, rather than failing with a EROFS (read-only file system), the mount point should be upgraded to read-write and the write should succeed.

Unfortunately, the support required to safely fully implement this technology does not exist in the BSD/OS 4.x series of kernel. Nor is likely to show up in a patch, as the changes required to support this functionality are widespread throughout UFS. Use of this option can result in lost data and damaged filesystems. I state this fact on personal experience. Until the support is fixed, you should not use this mount option.

Conveniently, Kirk McKusick has done the heavy lifting of making a kernel framework for this type of operation work, in the form of his work on UFS snapshots and running ``fsck'' in the background. This is a great paper and worth spending the time to read it.

While merely adding the snapshot facility to BSD/OS will not fix the softrdonly mount option, it does provide the necessary framework to fix the problems with the current implementation. It is my belief that fixing softrdonly after snapshot support is in the kernel is a relatively easy thing to do. It should go without saying why this is extremely valuable for an embedded operating system -- having filesystems that don't need to be recovered after a system crash or reboot speeds up reboot speeds considerably. On systems with FLASH devices acting as the filesystem storage, not constantly updating the UFS superblock is a huge gain. Not only does it reduce I/O operations, but it also prolongs the life of the FLASH device. All these things are extremely desirable in an embedded operating system. These are all the reasons that BSDi originally put this mount option into the system and they remain relevant today.

Implement kernel scheduled threads (Added November, 2001)(SMS)(KJL)

By implementing kernel scheduled threads, applications that use pthreads would be able to use multiple CPUs. Currently an application that uses pthread can use only a single processor, so the total amount of CPU available to a single application on a multiprocessor machine is one CPU.

One way this could be addressed is porting the IBM Next Generation POSIX Threads project from Linux to BSD/OS. This appears to be excellent work for doing NxM thread to kernel thread multiplexing. Given that this is licensed under the LGPL and not just the GPL it is possible to consider using this software.

Support for 16650/16750 serial chips (Added November, 2001)(SMS)(KJL)

With the discontinuance of the USENET-II ISA serial port controller board, it is impossible to get true hardware flow control serial ports under BSD/OS. With the USENET-II serial board, the 16550 serial chips had a small amount of additional hardware added to implement true hardware flow control. Without that hardware, the software driver must still be called to turn on the serial port bits that indicate that sender should stop transmitting.

By supporting the 16650 and 16750 serial controller ports natively (as opposed to in 16550 compatibility mode), true hardware flow control can be turned on in these controllers. This is relatively small change to make to the serial driver, and would be useful for people attempting to run very high speed serial ports under BSD/OS.

User writable LDTs so that WINE could run (Added November, 2001)(SMS)

WINE is a emulator that runs on many different flavors of UNIX-like operating systems, that provides some support for running of MicroSoft applications. In order to make it possible to run this emulator, the kernel would need to support user writable LDTs.

Adapt the FreeBSD new dirpref code in FFS (Added October, 2001)

A fine bit of optimization for how directories are laid out in FFS. It improves performance in many areas and apparently doesn't hurt performance anywhere. FreeBSD, NetBSD and OpenBSD all have this code integrated. Kirk McKusick has done a review of the code, and I don't think he had any great issue with it.
See this webpage for the original written details about the root of the problem and the tests to show the problem.
There is no reason to have 1/10 the filesystem performance of the other *BSD systems, given the relatively small number of changes needed to implement this change.
The NetBSD project has further commentary on the commit they did here.

Implement kernel support for SSE instructions (Added October, 2001)

The Intel SSE (and SSE2) (Streaming SIMD Enhancements) instruction sets offer the assembly programmer additional speed when writing application programs. The SSE and SSE2 extensions require both support from the CPU and from the operating system. BSD/OS should be altered to provide the support that is required to allow SSE instructions to be used.

Resurrect the Sparc/UltraSparc port (Added June, 2001)

Don't even get me started. Having a second supported architecture, especially one that is of a different endian machine keeps the software intellectually honest. Not to mention that in the SPARC world, it is relatively easy to get multi-processor machines with more than 4 processors. This is an excellent area for proving the scaling (or lack thereof) of the SMPng software architecture.

Implement IPFW filtering for IPv6 (Added January, 2001)

Hook up support for filtering of IPv6 packets in IPFW. Most or all of the low-level work to support IPv6 has already been added to the IPFW system. But, the appropriate IPFW entry points have not been wiring into the KAME IPv6 stack included in the kernel.

Port NetBSD UVM system to kernel when ready (Added December 2000)

The NetBSD UVM system is pretty nice looking. It seems to have a robust design, with many desirable features. It is obviously cross platform, and has been implemented on a wide variety of machine architectures. This would also lead directly down the path of having a unified kernel memory/disk buffer cache. The current VM system in BSD/OS is showing its age.
This strikes me as a problem similar to the that of the historical 4.3 -> 4.4 release at CSRG. The new (Mach derived) VM system was a major stumbling block for each architecture that needed support in the BSD4.4 release cycle. Enough of a stumbling block that BSD4.4 dropped support for the VAX architecture -- mostly due to lack of support in the new VM system.

Port NetBSD USB framework (Added December 2000)

Support for keyboards and mice at a bare minimum. Extension of USB support for generic printer devices would be a very useful extension. This would allow low-cost postscript printers could be attached via this bus. A follow-on project would be to implement support for USB attached scanners. Classes of so-called legacy-free devices are available now -- these devices do not have anything but USB ports for connectivity of keyboards and other input devices.

Port kqueue() kernel framework from FreeBSD (Added December 2000)

This is just plain good engineering work. It solves many of the commonly encountered issues for UNIX applications handling a very large number of file descriptors in a fairly elegant manner. Additionally, it allows for a nice, low overhead, way of waiting for event to occur in the filesystem (e.g. watching a directory).

Support PPPoE in the Kernel PPP stack (Added May 2000)

PPPoE (PPP over Ethernet) is a popular connection mechanism for many ISPs providing consumer oriented DSL connections. Sometimes also found in use at cable modem operations. Having both the server side and a client side implementation might be useful for ISPs. Having only the client side would be useful for end users.

Support IPv6 in the Kernel PPP stack (Added May 2000)

As IPv6 becomes more and more important to the world, in particular the Far East, it is important to have it well supported for all types of interfaces. Currently, IPv6 support is available for FDDI, Ethernet and Gigabit Ethernet interfaces. Inclusion of IPv6 support in the kernel PPP stack would add dialup and leased line interfaces to that list. For developers wanting to experiment with IPv6 this would be an important feature.

Implement recvmfrom() system call (Added May 2000)

A high performance syscall interface for getting multiple datagram packets out of the kernel in a single system call. In crude terms, you can think of this as being to recvfrom what readv() is to read(). This is critical for certain high performance networking daemons, such as programs that need to serve DNS queries or deal with RADIUS packets. Having a way of writing multiple packets into the kernel in a single system call would be useful as well, but probably not as important as a way to get multiple packets out of the kernel quickly. It could be argued that to make the system fully orthogonal, adding a multiple packet writing command would be the right thing to do. The addition of this system call would almost certainly be very useful for multi-media and multicast applications in addition to the aforementioned applications.

Implemented pread/preadv and pwrite/pwritev system calls (Added December 1999)

These system calls are similar to read() and write() (or readv() and writev()), except that a seek offset is in the argument list, in addition to the fd, buffer pointer and size of the data. These calls allow for easier writing of multiple datablocks into a file in a multithreaded application. They are also of interest to people writing database logging code.

Release new and improved select() system call (Added November 1999)

Faster, better, stronger, what's not to like? This was another feature that UUNET funded in BSD/OS. I don't think that BSDi ever managed to integrate this into a released kernel. Pity, they ought to do it -- everything that uses select() will get faster.

Change select() system call to update timeout values (Added February 1999)

A minor change to the select() system call so that the timeout value that is passed into the kernel is updated in the case where the syscall completes before the timeout is reached. This would be very useful for writing certain classes of applications that need to have regularly scheduled operations happen, but at the same time, must be responsive to other events that are occurring. This is a minor change and is explicitly allowed as a possible behavior for the system. A sysctl() interface to toggle this behavior on and off would be needed, at least during the phase after the above timeout code is implemented, but before all the applications that improperly rely on the existing behavior are fixed.

Implemented IEEE-1394 (FireWire) support (Added October 1997)

Add support into the system to be able to support devices on a IEEE-1394 bus. This should be of sufficient quality to be able to control at least a couple of video camera devices and capture the video stream that one of those devices sends out the FireWire bus. Also be able to read/write mass storage devices that located on the FireWire bus (particularly disk drives).

Miscellaneous Enhancements

The following is a list of things that fits into neither of the above lists. Often a meta issue, which requires more than a single definable act of engineering or development is required to address this item.

Entice VMware to port to BSD/OS (Added November, 2001)(SMS)

While BSD/OS is known to work as a VMware client operating system, it would be extremely useful to have BSD/OS also be able to act the hosting system. Being able to host other virtual x86 machine instances would be useful to just about everybody that needs to work in more than one PC based operating environment.

Open Source the IPFW kernel subsystem (Added December 2000)

The BSD/OS IPFW system is an excellent piece of software engineering. Releasing it so all the BSD systems can have a unified packet filtering system would be great. Unfortunately, it might well be too late to have any of the other BSD groups gladly adopt it as their packet filtering system. They have all invested time and effort in their own filtering systems. Additionally, I am afraid that the not invented here (NIH) meme will cause this code to be ignored, even if it were to be released. However, that isn't a forgone conclusion, so the IPFW code should be released.

Feedback

Well, there you have it. Some of my thoughts on what would make the BSD/OS operating system a nicer place in which to work and play. If you have a thought or two about this, feel free to send them to me. If you can convince me that it's a good idea, I'll be happy to add it to my list and give you credit. If you don't convince me its a good idea, well, I might not add it to my list, but there is nothing stopping you from making your own list.

Copyright

This page has been accessed times since this counter was installed (23 January 2002).

lidl@pix.net
$Date: 2007/12/07 02:57:23 $