Technology – Page 2 – Jeff Carl.com

Running Linux Programs on FreeBSD

By Jeffrey Carl

Boardwatch Magazine was the place to go for Internet Service Provider industry news, opinions and gossip for much of the 1990s. It was founded by the iconoclastic and opinionated Jack Rickard in the commercial Internet’s early days, and by the time I joined it had a niche following but an influential among ISPs, particularly for its annual ranking of Tier 1 ISPs and through the ISPcon tradeshow. Writing and speaking for Boardwatch was one of my fondest memories of the first dot-com age.

Even though many server admins prefer BSD Unix, there’s no denying that Linux is “where it’s at” for third-party software development. So, what’s a BSD admin to do?

In the bygone misty past of Unix (in the era historians call “the early ‘90s”), acceptance and market share were hampered by the diverging standards that forced application developers to write for only one version of Unix. This (as the youngsters today say) “really sucked,” and resulted in the Unix wars that left many syadmins hospitalized with “flame” wounds from heated Usenet exchanges. However, the tide has fortunately turned in favor of compatibility among the many *nixes.

To commercial software vendors, market share is everything. As such, if *BSD forced Linux vendors to develop for their OSes, it would be like your average Unix sysadmin saying, “If Cindy Crawford wants to go out, she can call me.” Fortunately, Jordan Hubbard of FreeBSD has indicated a prudent recognition that Linux’s API is becoming the default for third party developers (Indpendent Software Vendors, or ISVs) to develop for free *nixes. Therefore, it’s better for BSD users to accept the Linux ABI (Application Binary Interface) rather than to force ISVs to choose between Linux and BSD. This, in my opinion, is a very pragmatic and wise attitude.

FreeBSD, NetBSD and OpenBSD all include options to allow Linux binaries to run on their systems (at least for the Intel x86 architecture; on other architectures, your mileage may vary). For OpenBSD and NetBSD, you can check your system documentation or see their respective websites (www.openbsd.org and www.netbsd.org); for the purpose of this article, we’ll look at Linux binary compatibility on FreeBSD for x86. NOTE: Much of the information on the inner workings of Linux compatibility came from information graciously provided by Terry Lambert ([email protected]).

Under FreeBSD, Linux binary compatibility is managed by creating a “shadow” Linux file system with the requisite programs and libraries, and re-routing the Linux program’s system calls into this file system. These libraries then deal with a Linux kernel embedded into the FreeBSD kernel.

Setting Up Linux Compatibility

The development of an “executable class” loader for FreeBSD has been under development since 1994 (although when the project was started, Linux wasn’t one of the primary target OSes). There are three essential items that enable this functionality.

The first is a KLD (“Kernel LoaDable”) object called linux.ko, which is essentially a Linux kernel that can be loaded dynamically into the FreeBSD kernel. As a KLD, it can be loaded or unloaded without rebooting, using the kldload and kldunload commands. To check and see whether linux.ko is loaded properly on your system, use the kldstatcommand:

schnell# kldstat

Id Refs Address Size Name

1 4 0xc0100000 1d9a60 kernel

2 1 0xc1038000 3000 daemon_saver.ko

3 1 0xc1822000 4d000 nfs.ko

4 1 0xc1020000 10000 linux.ko

The second necessary item is the set of basic Linux libraries (pre-existing code that developers can link their programs to, so they don’t have to write every function from scratch) and binaries (like bash, cp, rm, etc.). These can be installed via the ports collection from /usr/ports/emulators/linux_base, which is based on a minimal Red Hat installation. As of this writing in late September, for the i386 architecture, the installed libraries matched a Red Hat 6.1-1 release including glibc 2.1.2-11, libc 5.3.12-31, glib 1.2.5-1, ld.so 1.9.5-11, libstdc++-2.9.0-24, gdbm-1.8.0-2 and XFree86 libs 3.3.5-3.

To be able to use a wider range of Linux apps, you may also wish to install the Linux development tools, found in the ports collection at /usr/ports/devel/linux_devtools. Note that you’ll want to do this after installing linux_base, since linux_devtools requires rpm (the Red Hat Package Manager application, also available as a FreeBSD app through the ports collection) and /compat/linux/etc/redhat-release to be installed. This currently includes (also for i386) kernel-headers-2.2.12-20, glibc-devel-2.1.2-11, make-3.77-6, cpp-1.1.2-24, egcs (plus egcs-c++ and egcs-g77)-1.1.2-24, gdb-4.18-4, and XFree86-devel-3.3.5-3.

Also, any necessary binaries/libraries can be set up manually by installing them into their original paths under the FreeBSD directory /compat/linux (e.g., install the Linux /usr/X11R6/lib/libX11.so.6.1 library onto FreeBSD as /compat/linux/usr/X11R6/lib/ libX11.so.6.1). This process is also necessary for installing libraries that aren’t part of the default ports collection sets.

The third item is “branding” Linux ELF (“Executable and Linking Format”) binaries. Although the GNU toolchain now brands Linux ELF binaries with the OS name (so newer Linux binaries should work without branding), some older binaries may not include this information (although ELF has replaced a.out as the primary binary format since Linux kernel 2.x and FreeBSD 3.0). To do this, use the brandelf command (which sounds like a move from “Dungeons and Dragons” to poke elves with hot sticks) with the “-t” (type) flag like:

schnell# brandelf -t Linux /usr/local/games/doom

How Linux Compatibility Works

Originally, Unix supported only one “binary loader.” When you ran a program, it examined the first few bytes of the file to see if it was a recognized binary type. If the file type that was attempted to being loaded and run wasn’t the singular binary type that it recognized, it would pass the file in question to the basic shell interpreter (/bin/sh) and try to run the program with that. Failing that, it would spit a nasty message back to you about how you couldn’t run whatever it was that you were trying to run.

Nowadays, FreeBSD supports a number of loaders, which include an ELF loader (by the way, Linux binaries can be stored on either a FreeBSD UFS or Linux ext2 file system, assuming you have mounted the ext2 file system the app is located on using the mount -t ext2fs command). If the “brand” is seen in the ELF file as “Linux,” it replaces a pointer in the “proc” structure of the binary to point automatically to /compat/linux/, and the binary is executed using the Linux libraries and binaries, instead of the FreeBSD ones. If it isn’t a FreeBSD or Linux binary, and the list of loaders (including those examining the first line for “#!” lines like Perl, Python or shell scripts) is exhausted without recognizing a “type,” then it is attempted to be executed as a /bin/sh script. If this attempt to run is unsuccessful, it is spit back out as an “unrecognized” program.

Linux and BSD Binaries: Both “Native?”

If a binary or library is required by a Linux program that is not included under the Linux compatibility “area,” the FreeBSD tree is searched. Thus, a Linux application that is expected to run with the Linux version of /compat/linux/usr/lib/libmenu.so – if it doesn’t find it – is checked to see whether it can run under the FreeBSD /usr/lib/libmenu.so. Only if a compatible library is not found in either the Linux-compatible “re-rooted (/compat/linux/whatever)” filesystem or the regular FreeBSD libraries is an executable rejected.

Because of this construction, it isn’t really accurate to say that Linux is “emulated” under FreeBSD, especially in the sense that, say, platforms are emulated under the MAME (www.mame.net) emulator, since it doesn’t involve a program which impersonates another platform on a binary level and translates all of its native system calls into local system calls. Linux binaries are simply re-routed to a different set of libraries, which interface natively with a (Linux kernel) module plugged into the FreeBSD kernel. Rather than “emulation,” this is largely just the implementation of a new ABI into FreeBSD.

Under the current implementation of binary compatibility, programs calling FreeBSD’s glue() functions are statically linked to FreeBSD’s libraries, and programs calling the Linux glue() function calls are statically linked to its own libraries, so their executions are dependent on completely different code libraries; however, the importance of glue()is on the wane. In the future, this might change, which would seriously open the question as to whether Linux apps under FreeBSD are any more “native” than apps written specifically for FreeBSD are.

Running Linux Binaries on BSD

Currently, the major problem with running Linux binaries on FreeBSD is similar to the major problem with using new apps on Linux: discovering which libraries they require and downloading and installing them. Unfortunately for BSD users, the only certain way to do this right now is to have access to a Linux system which already has the app in question installed, and run the ldd command on the application to list the libraries needed. If you don’t have access to a Linux system that already has the program installed, your next best bet is to check info on the program’s website, or search for it at Freshmeat (www.freshmeat.net).

Under current FreeBSD Linux binary emulation, most Linux binaries should work without problem. However, due to operating system differences, it won’t support those that heavily use the Linux /proc filesystem (which operates differently from the FreeBSD /proc), ones that make i386-specific calls like enabling virtual 8086 mode, or ones that use a significant amount of Linux-specific assembly code. While some people have reported that some Linux binaries run faster in this BSD environment than they do natively in Linux, this is a highly subjective claim, and one that should be taken with a grain of salt (at least until some real standardized testing is done with it). While Linux binary emulation on FreeBSD shouldn’t be considered as a major security risk, it is worth noting that a buffer overflow bug was discovered in August 2000 (www.securityfocus.com/frames/?content=/vdb/bottom.html%3Fvid%3D1628).

Popular Uses for Linux Compatibility

The most popular binaries for using Linux compatibility are the ISV applications that have been developed for Linux from popular commercial/semi-commercial vendors. These include programs like Sun’s StarOffice, Wolfram’s Mathematica, Oracle 8, and games like Bungie’s Myth II and id Software’s Quake III (there are already tutorials on installing Oracle 8.0.5 and Mathematica 4.x included in the FreeBSD Handbook; see below for more).

Other popular installations include the Netscape Navigator 4.6-7.x browser. Particular advantages to installing the Linux version (even though there is a FreeBSD version already) are that the Linux version under FreeBSD compatibility is reportedly more stable than the “native” FreeBSD version. Also, binaries for popular plug-ins (Flash 4, Real Player 7, etc.) are available, since “native” FreeBSD versions aren’t. Info on installing VMware for Linux on FreeBSD can be found at www.mindspring.com/~vsilyaev/vmware/.

Getting Help with Linux Compatibility

The primary reference on Linux binary emulation on FreeBSD is the “handbook” page at www.freebsd.org/handbook/linuxemu.html, including specific pages on Oracle (www.freebsd.org/handbook/linuxemu-oracle.html) and Mathematica (www.freebsd.org/handbook/linuxemu-mathematica.html). To keep up-to-date on Linux emulation under FreeBSD, subscribe to the mailing list freebsd-emulation (send e-mail to [email protected] with the text [in the body] subscribe freebsd-emulation).

Additional information on Linux compatibility (including instructions for older versions of FreeBSD), see www.defcon1.org/html/Linux_mode/linux_mode.html. For a list of Linux applications that are known to run on FreeBSD under Linux binary compatibility, see www.freebsd.org/ports/linux.html. For an example of setting up Word Perfect 8 in Linux compatibility – as well as an account of running Windows 2000 on top of VMware on top of Linux on top of FreeBSD, read the excellent article at BSD Today (www.bsdtoday.com/2000/August/Features252.html). You can find some (mildly outdated but still useful) instructions on installing StarOffice on FreeBSD at www.stat.duke.edu/~sto/StarOffice51a/install.html. And, for fun, there’s info on setting up an Unreal Tournament server on FreeBSD via Linux compatibility at www.doctorschwa.com/ut/freebsd_server.html.

Trying on Red Hat – Questions, Answers and Red Hat Linux 7

By Jeffrey Carl

Red Hat Linux is the unquestioned leader in the U.S. for Linux market share. According to research firm IDC, Red Hat shipped 48 percent of the copies of Linux that were purchased in 1999. Many smaller distributions are based on its components, and it’s often used as the “target” distro that third-party (especially commercial) software is developed for. Still, the company has found it difficult sometimes to reconcile its place in the Linux community with its place as the Linux standard-bearer on Wall Street. With the release of Red Hat Linux 7.0, is the company still on top?

Under the Hat

Red Hat has been around since the days when Linux distributions were loose, volunteer-driven projects with users numbering in the thousands. Red Hat gradually reached a position of great popularity, based on several factors. First, they put a significant amount of effort into creating ease of use for inexperienced admins, with semi-graphical installation/configuration programs and the Red Hat Package Manager (RPM) software installation system. Second, Red Hat shipped a solid distribution that tended to include the “latest and greatest” features. Lastly, Red Hat showed a genius for building and marketing a brand name, and getting a shrink-wrapped, user-friendly product into stores.

Once Linux began to appear on the radar scopes of investors, Red Hat became the high-profile “star” of the Linux world. Red Hat went public on August 11, 1999 and soared immediately – making paper fortunes overnight for of some of the Linux world’s luminaries. The stock reached a dizzying high of 151, but later became a victim of the hype as Wall Street’s love affair with Linux cooled; as of October 1, RHAT was trading near its 52-week low of 15.

The company has also faced the very difficult problem of maintaining its standing in the open source world (which is often distrustful of corporations and for-profit ventures) while pleasing its investors and shareholders. Like Robin Hood becoming the new Sheriff of Nottingham, they have found that it’s sometimes impossible to please all of your constituencies at once. Red Hat has made some occasionally unpopular moves, and faced criticism from the more evangelical Linux enthusiasts that it was trying to “co-opt” Linux standards as its own; however, the company has overall done a good job of pleasing the majority of its disparate audiences.

Introducing Red Hat Linux 7.0

As of this writing, Red Hat Linux 7.0 has been out for only a few days, but seems to be well-received. There appear to be some viable reasons to upgrade (even beyond the fact that RH 7 is supposed to be ready out of the box for a 2.4 kernel when it’s released).

Red Hat 7 is more of an evolutionary upgrade than a revolutionary one, and this will please some users and disappoint others. Still, this is generally a positive step as far as ISP/server users are concerned – when compatibility with hardware and a stable system are your prime requirements, “conservative” is never a bad approach.

As Red Hat itself is quick to point out, the value of a distribution is in meshing functionality with the expertise and testing to make sure that all of its included parts “play nicely with each other” as much as possible. In the past, Red Hat has been (depending on your viewpoint) applauded or derided for being an “early adopter” (the company brushes aside this characterization in the interview below) of new versions of libraries and applications.

An example of the tack they’ve taken with Red Hat 7 is that because of stability concerns about KDE 2’s prerelease versions by Red Hat testers, version 1.1 was included with the final release. On the other hand, some users (rightly or not) questioned the use of a non-“stable release” version of the GNU C Compiler (gcc 2.96 20000731) as the default compiler. Again, whether these are steps forward or backward is a matter of personal preference; you can’t please everybody. Still, it appears that Red Hat has worked hard to avoid a “buggy x.0 release” that some have complained about in the past.

The big differences in Red Hat 7 for desktop users are built-in USB support and the default inclusion of XFree86 4.0, with its (in my opinion, much-needed) major overhaul and modularization of the X server. Also, Sawfish is now used as the default window manager with GNOME rather than Enlightenment.

Overall, administration is roughly the same as with the 6.2 distro, with a few bugfixes and improvements here and there. The basic US package still includes two CDs (other geographical versions will include more); while this won’t please SuSE users who love the extra CDs full of applications, the included software still represents a pretty good sampler of the software you’d want (with the possible exception of “office suite” software). For more information on the included software, read the interview below.

Q & A with Red Hat

The following is from an e-mail interview with Paul McNamara, VP, Products and Platforms for Red Hat.

Q: Could you give me a brief history of Red Hat?

A: Founded in 1994, Red Hat (Nasdaq:RHAT), is the leader in development, deployment and management of Linux and open source solutions for Internet infrastructure ranging from small embedded devices to high availability clusters and secure web servers. In addition to the award-winning Red Hat Linux server operating system, Red Hat is the principal provider of GNU-based developer tools and support solutions for a wide variety of embedded processors. Red Hat provides run-time solutions, developer tools, Linux kernel expertise and offers support and engineering services to organizations in all embedded and Linux markets.

Red Hat is based in Research Triangle Park, N.C. and has offices worldwide. Please visit Red Hat on the Web at www.redhat.com.

Q: What platforms does Red Hat support? What are the minimum requirements for Red Hat?

A: Red Hat supports Intel, Alpha, and SPARC processors. Minimum requirements for the Intel product are 386 or better (Pentium recommended) 32MB RAM and 500MB free disk space.

Q: What is included with the newest release of Red Hat? (i.e., kernel version, version of Apache, Perl, Sendmail, etc. for other software packages you feel are important, plus what other third-party software do you add?)

A: A significant new feature of Red Hat Linux 7 is Red Hat Network. Red Hat Network is a breakthrough technology that gives customers access to a continuous stream of managed innovation. This facility will dramatically improve customers abilities to extract maximum value from Red Hat Linux.

We ship the following: 2.2.16 kernel, Apache 1.3.12, openssl 0.9.5, [ed: openssh 2.1.1p4 is also included] sendmail 8.11.0. Complete package description can be found at www.redhat.com/products/software/linux/pl_rhl7.html.

Third party apps can be found at www.redhat.com/products/software/linux/pl_rhl7_workstation.html and www.redhat.com/products/software/linux/pl_rhl7_server.html.

Q: What are some configurations that Red Hat would recommend, or it really excels with?

A: While Red Hat Linux is a superior general purpose OS supporting a wide range of application segments, the most popular configurations are: web servers, secure web servers, database servers, internet core services (DNS, mail, chat, etc), and technical workstations.

Q: What is Red Hat’s ‘Rawhide’ development tree? Who is it suitable for?

A: The Rawhide development tree represents our latest development code drop. It is our next release, in progress. In the traditional software development model, the developing company provides the latest engineering build to its internal developers to use to drive the development effort forward. Since Red Hat uses the collaborative development style, our ‘internal’ development release is made available to the community. This release is intended for community OS developers and is not intended to be used by customers for production environments.

Q: When Linux is united by a common kernel, what is it that keeps Red Hat as the “number one” distribution? What would you say differentiates Red Hat from other Linux distros.

A: Your question is a lot like asking since all car makers use a four cycle internal combustion engine as the primary component, what differentiates Lexus from other cars? Note that all cars are essentially compatible (can be driven on the same roads, use the same fuel, and a driver trained to drive one brand of car can easily drive another brand). What sets our brand apart is the mix of features and the quality of the finished product. We process “raw materials” (the various packages) and turn them into a finished product. Our selection of the packages, the engineering we do to create an integrated product, and our ability to deliver a quality result make the difference.

Q: Red Hat has often been cited as an “early adopter,” moving to new library versions, etc. in its releases before other distributions do. Is this a fair characterization? What advantages and/or disadvantages does this have?

A: I’ve only heard us described in this way by a competitor. I don’t know what this means. We clearly drive an agenda, and others tend to follow.

Q: Red Hat obviously has many strengths. What users, if any, should *not* choose Red Hat? For what reasons?

A: We generally discourage people interested in a legacy desktop OS from purchasing Red Hat. Red Hat is designed for servers, technical workstations, and post-PC embedded devices. It is either (1) intended for internet and IT professionals who need a high performance, internet-ready OS, or (2) is designed to be built into post-PC consumer information appliances where the device manufacturer has integrated our product into the consumer product.

Q: What advice would you give for ISP administrators about when/when not to upgrade their servers running Red Hat?

A: Red Hat Linux is a different kind of OS. Customers can choose, on a feature by feature basis, which packages to upgrade and when. Through a subscription to the Red Hat Network, customers can receive proactive notifications when new features become available and can receive a continuous stream of managed innovation to give them strategic advantage.

Q: What is the Red Hat Certification program? What benefits does it offer to Internet server administrators?

A: The Red Hat Certified Engineer (RHCE) program is the leading training and certification program on Linux. RHCE is a performance-based certification that tests actual skills and competency at installing, configuring, and maintaining Internet servers on Red Hat Linux. Complete details can be found at www.redhat.com/training/rhce/courses/ and www.redhat.com/training.

RHCE program courses and the RHCE Exam are regularly scheduled at Red Hat, Inc. facilities in Durham, NC, San Francisco CA, and Santa Clara, CA. Global Knowledge and IBM Global Services are Red Hat Certified Training Partners for the RHCE Program, offering RHCE courses and the RHCE Exam in over 40 locations in North America. Red Hat can also run Red Hat training on-site for 12 students or more.

Red Hat offers the most comprehensive Developer Training for systems programmers and application developers on Linux, as well as training on Advanced Systems and Advanced Solutions, including the only regularly scheduled training on Linux on IA-64 architecture.

Q: What plans does Red Hat have for the IA-64 platform?

A: Red Hat is a leading participant in the IA-64 consortium, and we intend to aggressively support this new platform by delivering Red Hat Linux concurrently with the availability of IA-64 hardware.

Q: What advantages would you cite for someone choosing Red Hat Linux over another server OS, like Windows 2000, Solaris or FreeBSD?

A: Red Hat offers a superior mix of reliability, performance, flexibility, total cost of ownership and application availability. We believe it is simply the best choice for deploying Internet infrastructure.

Q: Where could someone running an Internet server go for help and tips on Red Hat? Do you specifically recommend any advice?

A: There is a huge volume of information available for Red Hat Linux. Sources include a large selection of books available at leading book stores, on-line information from news groups and mailing lists, a worldwide network of Linux Users Groups (LUGs), on-line help in the form of man and info pages, and support offerings available directly from Red Hat and from www.redhat.com.

Hosting FrontPage Sites on Unix Servers – Dealing with the Great Satan of Server Extensions

By Jeffrey Carl

Ever since a version of FrontPage started getting bundled in with Microsoft Office 2000, FrontPage has quickly become the web site design tool of choice for people who enjoy being lectured by animated paper clips. If your hosting environment is Linux or *BSD, then FrontPage needs to be understood and dealt with.

FrontPage and How it Works

FrontPage is a simple GUI tool which automates the things that most basic web designers are looking for these days, including graphical page design, publishing their sites and CGI-type actions (forms, dynamic pages, etc.). Throughout the rest of this column, I don’t want to disparage what the creators of FrontPage have made – a truly easy, user-friendly program for the design of simple to very complex web sites – a real achievement.

But, for a FrontPage-created site to take use of all the easy-to-create “all-singing, all-dancing” features that make it such a hit with people who wouldn’t know an HTML tag if it crawled up and bit them, the FrontPage Server Extensions (FPSE) must be present on the hosting server. The FPSE are essentially Microsoft-built proprietary CGIs executed over HTTP – designed for Windows NT and Microsoft IIS, but ported to Unix and supported on Microsoft’s behalf by Unix porting house Ready-to-Run Software (RTR). This is where the trouble starts.

Can You Support FrontPage on a Freenix?

Supporting FrontPage on Unix is both a help and a hindrance to ISPs. It allows you to host sites for clients using FrontPage – and this faction is rapidly growing. It can provide you with a competitive advantage over ISPs that don’t support FrontPage sites.

The extensions are currently available for a number of platforms including FreeBSD and Linux (for x86 architectures only), and for a number of webservers including Apache, NCSA, Netscape and Stronghold (although only Apache will be discussed in this column). I’m sure that RTR has done a herculean task porting these server extensions to Unix, but the fact remains that this software is poorly documented and frequently difficult to troubleshoot.

The FrontPage extensions for Unix don’t allow you to support the full range of capabilities of FrontPage-designed sites. Active Server Pages (ASP) aren’t natively handled with the FPSE for Unix, and neither is native ActiveX nor ODBC/Microsoft Access database connectivity.

You can, fortunately, look for outside support. Software vendor Chili!Soft (www.chilisoft.com/allaboutasp/) offers a Unix-native commercial ASP package, Halcyon (www.halcyonsoft.com/products/iasp.asp) offers a Java-based ASP commercial package, and mod_perl (perl.apache.org) with the Apache::ASP (www.nodeworks.com/asp/) module provides a free, Perl-based solution. Commercial ODBC resources are available at www.openlinksw.com, and open-source resources can be found at www.jepstone.net/FreeODBC.

The Pitfalls of FrontPage Extensions

Unfortunately, offering FPSE/Unix commits you to supporting what is, frankly, some occasionally very frustrating software. While security problems with FP-administered sites have decreased, it’s still not suitable for the truly paranoid (a full discussion would take up this whole magazine). The author.exe program that is part of the FPSE seems, on Unix, to eat up an unseemly amount of CPU cycles while it’s running. The .htaccess files that FrontPage installs for each of the directories in a FrontPage-enabled site may interfere with the server administrator’s security policies. And, in my experience, the FPSE will occasionally just stop working, with a reinstall required.

If you can’t find the free support resources you need, you can of course contact RTR for phone support ($245 per incident) or e-mail support ($195 per incident). And, if you don’t support it, FrontPage users are advised in the FrontPage/Unix FAQ to “find another ISP” (http://www.rtr.com/fpsupport/faq2000g.htm#7newISP).

So much for “alternatives.” And the customer pressure to host FrontPage sites is quickly increasing.

Before I go on to discuss how to support FrontPage on Unix, I should mention that there is another option – set up a server running Windows NT or Windows 2000. This has its own set of problems and complications (which could take up a book, not a column), but at least the FPSE work flawlessly. After two years and a multitude of problems supporting 50 or so FrontPage users on Red Hat Linux and FreeBSD, and seeking more scalability without headaches, I shamefully admit that this is what I ultimately did. Unfortunately, I think this is what Microsoft was hoping for all along.

Getting the FrontPage Extensions

For a detailed understanding of the installation process and some help with common problems, first visit the Server Extensions Resource Kit (“SERK”) page for Unix installation (www.rtr.com/fpsupport/serk4.0/inunix.htm). For a full list of the files included in an installation and their permissions structures, see www.rtr.com/fpsupport/serk4.0/apndx01.htm.

If you’re installing Apache on the server for the first time, you can use the apache-fp package (www.westbend.net/~hetzels/apache-fp/), available as a Linux RPM or through the FreeBSD ports collection, which integrates an Apache distribution with the Unix FPSE. If you’re installing on an existing Apache setup, go directly to RTR Software’s FPSE/Unix homepage (http://www.rtr.com/fpsupport/). Download the FrontPage 2000 SR 1.2 extensions for your platform (they’re backward-compatible with FP ’97 and ’98 clients), which come in a zipped or GNU-zipped tar file, as well as the shell scripts fp_install.sh and change_server.sh.

The extensions tarball will be named like fp40.[osname].tar (version 4 is the FP 2000 extensions; version 3 is for FP ’98). The fp_install.sh script converts the results of the uname -a command to a $machine variable, and won’t install if the extensions tarball name doesn’t match the $machine name.

For a long time, it was necessary to get the extensions to install on FreeBSD by “hacking” the script. In this case, adding the line:

FreeBSD*)            machine="bsdi" ;;

as line 70 of the fp_install.sh script would get the BSDI extensions to install. This has been fixed if you download the most recent versions of the software.

Installing Unix FrontPage Extensions

The default install is into /usr/local/frontpage/version40, or in the directory of your choice, although it will create a link to the above. The installation script installs the FPSE, an HTML version of the SERK, the command-line administration tool (fpsrvadm.exe), the web-based administration tools (in the admcgi directory), the Apache-FP patch, and assorted server and site administrator support files.

If you have already installed the version30 extensions, the script will move its currentversion symbolic link to point to the version40 directory while leaving the older version intact (actually a good choice, and one that many other software packages might adopt!). The installation script will also prompt you to upgrade the “stub extensions” (located inside each FrontPage-enabled host’s document root, linking to the central extension executables) for each virtual host which has had the older stub extensions installed.

While installing, note that each site’s FrontPage username and password (used for administering their site through the FrontPage client) have no relation to their Unix username and password. For security’s sake (unless the users will be administering their sites through secure server access), it is advisable to make these username/passwords different from their Unix user/password counterparts. Note that if each of your virtual hosts are associated with different Unix users/groups, those “subwebs” should be owned by the Unix user/group of the site maintainer, or they will be unable to administer their site properly.

Next, run the change_server.sh script to automatically replace the existing httpd daemon with a FPSE-supplied one which has the FrontPage patch/module installed (with only the mod_frontpage module and a few other basic modules compiled in; it moves the old one to httpd.orig). The script further creates a suidkey file, used when the FPSE exercise their suid (“set user ID”) bit to make root-level modifications. It also prompts you to upgrade any virtual hosts if necessary.

The Apache version used with the current extensions (as of this writing) is 1.3.12. Note, however, that using an older version of the extensions may replace your Apache daemon with an older Apache daemon. If you would like to compile your own Apache daemon instead of using the change_server.sh-installed one (recommended), you may wish to download the Improved Mod_Frontpage (home.edo.uni-dortmund.de/~chripo/) and then follow the installation/compilation directions listed at www.rtr.com/fpsupport/serk4.0/inunix.htm#installingtheapachepatch. If the patch fails, check out the info at home.edo.uni-dortmund.de/~chripo/faq.asp).

Administering FrontPage Sites

For the server administrator, the command-line tool fpsrvadm.exe is your primary interface. This tool allows you to install or uninstall FPSE for virtual hosts, set authoring options for them, set directory/executable permissions, chown files, and create or merge subwebs. Running fpsrvadm.exe without any arguments brings you to a GUI-like interface; or, you can run the program directly with command-line options which are detailed at www.rtr.com/fpsupport/serk4.0/adfpsr_3.htm.

When adding FPSE to virtual hosts, choose Apache-FP for “server type” unless you have chosen not to install the FrontPage patch. During a host installation, “stub extensions” will be installed for the selected host, and its executable directories aliased appropriately.

Through fpsrvadm.exe, you can also run a “check and fix” option on existing hosts. While this option won’t tell you anything other than whether the extensions are installed or not (and which version), it can frequently be a valuable tool. When running a check or installing new virtual hosts, be sure to specify the port number (e.g., www.somehost.com:80) or it won’t be recognized.

If e-mailing FrontPage forms fails, be sure to edit your /usr/local/frontpage/we80.cnf file and add the line:

SendMailCommand:/usr/sbin/sendmail %r

…replacing the above path with the correct path to your MTA of choice. Otherwise, FrontPage forms will not be able to send e-mail. I’m not sure why this isn’t part of the default installation, but it isn’t. Also, note that frequently users may use FTP rather than use their FrontPage client to upload files to their site. A common problem is that if an entire directory is uploaded by the user via FTP, it may overwrite the FPSE located in that directory, and it may be necessary to reinstall the FPSE for that host.

Documentation and Support

Your official first stop for documentation will be the “official” version of the SERK at officeupdate.microsoft.com/frontpage/wpp/serk/. The SERK is required reading for any sysadmin to understand FrontPage, especially those with training in Unix and not Microsoft systems. Unfortunately, the FP 2000 SERK (the FP 98 SERK did not share this problem) is the first Unix documentation package I have ever encountered which does not even mention the terms “problem” or “troubleshooting.”

For real-world administrators, your un-official first stops will be the RTR FrontPage 2000 FAQ (www.rtr.com/fpsupport/faq2000.htm) and discussion board (www.rtr.com/fpsupport/discuss.htm). While the discussion boards aren’t always much help, the FAQ is absolutely essential, answering most of the common questions that can make the difference between supporting FrontPage and chewing the Ethernet cable off the server in frustration. I’m not sure why elements of the FAQ haven’t been integrated with the SERK (by the way, would a few man pages be too much to ask?). It is essential that, because of how disorganized much of the FrontPage/Unix documentation is, you should scour everyavailable resource before considering a problem unsolvable.

Part of my problem with the Unix FPSE is that they use Microsoft’s seemingly proprietary way of describing things, which makes life difficult for Unix administrators. For example, the collections of a user’s web pages are normally known as a “web site,” which has a “document root” and may be a “virtual host.” In the Front Page Mirror Universe, the evil Captain Kirk has a goatee, and this is known as a “FrontPage web,” with a “root web” and “virtual webs.” To use my favorite acronym, “WTF?”

There is a newsgroup available at microsoft.public.frontpage.extensions.unix. However, a quick perusal of this newsgroup’s messages over the past several months shows that the postings appear to be mainly problems and questions – without many answers. There was a FrontPage/Unix mailing list ([email protected]), but as of this writing [email protected] no longer seems to be responding for subscriptions. I recommend searching Google (www.google.com) for FrontPage web resources.

Conclusions

Supporting FrontPage websites on Freenixes is very possible, and for basic purposes, it works very well. However, be prepared to do your homework, and know that it may not be easy. Administering a web server always requires serious research and work; depending on your tastes, dealing with FrontPage on Unix may require too much.

Getting to Know SuSE Linux – An Interview with SuSE CTO Dirk Hohndel

By Jeffrey Carl

SuSE Linux is often referred to as “the European Red Hat,” since SuSE enjoys the kind of market domination there that Red Hat does in the U.S. For the record, SuSE is pronounced “SUE-zuh,” and the name was a catchy little acronym for the rather awkward “Gesellschaft für Software und Systementwicklung mbH.” A reader survey by a German Linux magazine found SuSE with about 75 percent market share to Red Hat’s 11 percent and Debian’s 8.5 percent. Although I wasn’t able to find figures, it’s also safe to say that SuSE enjoys a commanding market share in many other areas outside the U.S. as well.

A major factor cited for this is the fact that SuSE’s distribution includes six CDs with more than 1500 extra software packages to install. While many U.S. users with broadband connections don’t particularly care, it’s an important factor for users (especially for international users) who have slow connections, pay per-minute charges for their bandwidth or otherwise find it inconvenient to spend large amounts of time downloading new software.

Advantages frequently cited by SuSE users aside from the copious CD software collection include SaX, an excellent X Windows configuration tool; and YaST, SuSE’s LinuxConf-like administration tool. While these features are a favorite for some users, others complain that the enormous number of available applications in the install CDs makes installation cumbersome. And YaST, like LinuxConf, seems very much to be a “love it or hate it” application, with opinions varying widely by personal preferences.

SuSE distributions tend to be a little less “cutting-edge” than Red Hat’s, lagging a couple months behind with the “latest and greatest,” but as a result tending to have fewer bugs. Whether this tradeoff is acceptable is up to you; it should be noted that SuSE acquitted itself fairly well in the recent Security Portal Linux Distribution Security Report (http://www.securityportal.com/cover/coverstory20000724.html).

The current version of SuSE for x86 hardware (as of this writing) is 7.0. SuSE recently released a well-reviewed version of their 6.4 distribution for PowerPC-based Macintoshes, which includes the MOL (Mac on Linux) emulator among other goodies. To find out more, I asked SuSE Chief Technical Officer Dirk Hohndel and press representative Xenia von Wedel:

Carl: Could you give me a brief history of SuSE?

Hohndel: In 1992, the four founders ran into the (back then largely unknown) Linux operating system (or more precisely, the beginnings thereof). They quickly saw the need for Linux on “off-line media”, as Internet connections were not commonplace in Germany back then (and still are quite expensive). More importantly, a physical distribution made it easier to bundle documentation and support, and of course to make SuSE’s own developments for installation and configuration available as part of the package. Soon SuSE Linux became the standard Linux OS in Europe, and in the past few years SuSE has been quite successful outside Europe as well (less because of a strong marketing arm, but more due to its focus on sound technology and good engineering).

Von Wedel: SuSE Linux AG, headquartered in Germany, and SuSE Inc., based in Oakland, CA, are privately held companies focused entirely on supporting the Linux community, Open Source development and the GNU General Public License. With a workforce of over 450 people worldwide, SuSE has offices all over Europe, Venezuela and in the US. More than 50,000 business customers use SuSE Linux worldwide due to its stability and high quality.

SuSE received the “Show Favorites” award at LinuxWorld Expo in February 2000 and March 1999. SuSE contributes considerably to Linux development projects such as the Linux kernel, glibc, XFree86TM, KDE, ISDN4Linux, ALSA (Advanced Linux Sound Architecture) and USB (Universal Serial Bus). Additional information about SuSE can be found at http://www.suse.com.

Carl: What is included with the newest release of SuSE?

Hohndel: Linux kernel 2.2.17/PRE (2.2.16 plus the relevant patches for it) with enhanced raw device support and 4 GB main memory addressing, full USB support and ReiserFS XFree86 4.0 and SaX 2, the graphical installation tool for XFree86 4

SuSE Linux Professional includes more than 1500 apps such as StarOffice 5.2; Acrobat Reader; Samba 2.0.7; Apache 1.3.12 including PHP-4, Zope, Midgard, JServ, Tomcat, backhand and WebDAV; Lutris Enhydra 3.0; teleconferencing; Sendmail 8.10.2; Postfix 19991231pl08; Perl 5.005_03; Logical Volume Manager; PAM; KDE 1.1.2, KDE 2.0 Beta3, and GNOME 1.2.

SuSE Linux is known to be one of the richest and most complete versions of Linux, so almost all the important packages are there. One of the interesting new features is that SuSE Linux 7.0 can be installed and used using Braille; it can therefore be used and managed by the visually impaired.

Carl: What platforms does it support? What are the minimum requirements for SuSE?

Hohndel: At the moment, SuSE Linux is available as a shrink-wrapped product for Intel x86 and compatible processors (IA32), Motorola PowerPC and Compaq Alpha processors. A beta version for IBM mainframe S/390 can be taken from SuSE’s ftp server. Versions for SPARC and IA64 are under development and also available via FTP.

Von Wedel: Please see detailed information about some special hardware in our support database. Just search for the desired keyword at http://www.suse.com/support/hardware/index.html .

Carl: What are some configurations that SuSE would recommend, or it really excels with?

Hohndel: We have certified quite [a lot of] hardware with SuSE Linux, a current list can be found on our web site. I don’t think that it makes sense to point out specific configurations, as those tend to change all too frequently.

Von Wedel: If you are running a NIS or simple file server, a slower Pentium or even a 486 would work fine. For a standard all-around machine, you would probably want the average machine found on the shelf at Best Buy or CompUSA [including] big hard drives, a Pentium III and at least 128 MB of RAM. This machine will allow you to run programs like Quake III and VMware without problem. We tell our customers to avoid any hardware where the manufacturer refuses to release open source drivers for it.

Carl: What would you say differentiates SuSE from other Linux distributions?

Hohndel: SuSE maintains consistent technical quality throughout all platforms and all languages. We also have an encyclopedic set of Linux tools, for which SuSE Linux is already famous. YaST and YaST2 are well known as solid and easy-to-use installation and configuration tools that are flexible enough to support a wide range of installation options for the enterprise environment. Additionally, for optimized support for fully automated installation, SuSE’s new ALICE (Automatic Linux Installation and Configuration Environment) tool allows central configuration management for computer networks.

Carl: What is YaST? Where can I find more information about it?

Hohndel: YaST stands for Yet another Setup Tool. A white paper that covers some of the important features is available on the web at http://www.suse.de/de/linux/whitepapers/yast/Auto_Install_English_text.txt.

What is important to know about YaST is that it offers a central configuration and administration interface, but can be told to stay out of the way if the admin chooses to configure parts of the system “manually.” So, you can benefit from the flexibility and know-how that has been put into the tool without being stuck with it. Many typical administration tasks (adding printers, setting up accounts, adding software packages) can comfortably be done from within YaST.

Carl: For someone operating an ISP, what reasons could you give to choose SuSE over another Linux distribution?

Hohndel: SuSE has a strong focus on security within its distribution. We do not only have an internal security team within SuSE Labs that audits all major packages and closely follows all relevant information sources, but we also maintain an active dialogue with our customer base, through mailing lists and security alerts.

Furthermore, SuSE is extremely well connected with numerous industry leaders, from both the technology and the business perspective. We maintain strategic alliances with IBM, Compaq, Fujitsu Siemens Computers, Oracle, SGI, and numerous other independent software vendors. And [since] SuSE supports the Free Standards Group, SuSE customers will be sure to use the widely spread Linux distribution that sets up the standards. Already today SuSE is compliant with all the draft standards from LSB [Linux Standards Base].

Carl: For someone operating an Internet server – what, if any, are the drawbacks for choosing SuSE?

Hohndel: Quite frankly, I don’t see any drawbacks in choosing SuSE. Our system is well tested for exactly this use case. Interestingly enough, while globally about 30 percent of all web servers run Apache on Linux, in Germany, our “home turf,” that number is above 40 percent. So SuSE Linux is in very heavy use for exactly that – an Internet server.

Carl: What advantages would you cite for someone choosing SuSE Linux over another server OS, like Windows 2000, Solaris or FreeBSD?

Hohndel: Open Source is by many seen as the software development methodology of the future. Especially in the area of infrastructure systems, for example Internet servers, Open Source has gained the respect and the trust of the companies deploying these systems.

Linux is finally keeping the old Unix promise. It is the first truly homogeneous OS in a heterogeneous hardware environment. At the same time, it is a very flexible environment that allows for easy customization and can be adapted to the needs of very different use cases, from tightly controlled firewall system to fully-fledged server to complete desktop.

And, of course, there are lots of companies that develop on Linux and for Linux, and there is plenty of companies supporting Linux and offering professional services around Linux. SuSE is obviously one of them.

Depending on the OS that you compare with, a different combination of these arguments apply. 🙂

Carl: Where could someone running an Internet server go for help and tips on SuSE?

Hohndel: We offer both support and professional services around SuSE Linux. A good place to start is http://www.suse.com/suse/news/PressReleases/proserv.html. We already offer 24×7 support here in Germany and will roll out this service globally, soon.

And of course we offer the famous support database and component database (http://sdb.suse.de/sdb/en/html/index.html and http://cdb.suse.de/cdb_english.html).

Carl: What’s next for SuSE? What improvements are you planning for the future?

Hohndel: We continuously expand the hardware base that SuSE Linux supports. As I mentioned above, S/390, IA64 and SPARC are already in beta test (or close to production), and others will follow.

The installation and configuration tools are continuously being developed and will of course be one of the areas that we continue to focus on. Especially things like improved hardware detection, but also ergonomic aspects of the administration process.

And of course we are putting a lot of effort in many Open Source projects. [These include] “enterprise” features like high availability, improved SMP support or our work on directory services and better file systems, or desktop-oriented things like XFree86 and KDE.

An important focus of improvements is ISV [Independent Software Vendor] support. We are working closely with many ISVs to help them port their software to a standardized Linux environment to get the largest set of applications for Linux possible.

So there are a lot of things that you can expect from SuSE in the future. We are excited to be in this fast-growing industry and are looking forward to the things to come.

The Continuing Evolution of Darwin – Apple’s BSD Unix Has Lots of Promise, and Lots of Rough Edges

By Jeffrey Carl

There’s a new BSD out there, and it’s unquestionably the odd member of the family. “Darwin” is the BSD Unix core of Apple’s next generation OS, MacOS X (see page 39 for the lowdown on OS X). While MacOS X (which includes Darwin) is a proprietary commercial product, Darwin by itself is open-source and available for free.

So why should you care (or think about deploying it)? First, it’s being developed for both PowerPC and Intel x86 architectures. Second, it’s being developed not only by its user community by also by Apple – which, like ‘em or hate ‘em, has considerable manpower and money to throw at the project.

Lastly, Apple is scheduled to start shipping MacOS X on new Macintoshes in January 2001, as well as making it available for current machines – meaning that there will probably be a couple million boxes out there with Darwin at their heart within a year. While that’s no guarantee that all of your favorite software will rush to support it or optimize for it, it will certainly have the user numbers to make it worth noticing.

What is Darwin?

To get a feel for where Darwin is and where it’s headed, I asked Ernie Prabhakar, Apple’s Developer Platforms Product Line Manager.

Q: Can you give me a brief history of Darwin (including its roots in NextStep/ Rhapsody/Mac OS X Server)?

Ernie Prabhakar: Darwin is based on the Mach/BSD technology that traces its roots back through Mac OS X Server, Rhapsody, OpenStep for Mach, NextStep, and ultimately the original Mach 2.0 work at CMU. In all the NeXT/Apple products, this UNIX core provided the support for advanced GUIs and development environments. Over time, we’ve evolved from BSD 4.3 to BSD 4.4, and from Mach 2.0 to Mach 2.5 and ultimate Mach 3.0 (based largely on OSF MK 7.3).

Q: Who is Darwin’s intended audience of consumers?

EP: Darwin is targeted at Macintosh developers who want to leverage the power of open source, as well as Open Source developers who want to enjoy the performance, consistency, and advanced GUI available on the Apple platform. Certain large customers (e.g., research universities) will take advantage of the open core to customize it to their environments, as well as helping students and researchers studying open source.

Q: What is the relationship between Darwin, its BSD kernel, and the Mach microkernel?

EP: Darwin is the entire product, including a hybrid Mach/BSD kernel plus assorted utilities (mostly from FreeBSD) and developer tools (primarily from the GNU project).

Q: How does using Mach affect its characteristics or performance?

EP: Very little. We use a microkernel architecture but a monolithic implementation, so Mach and BSD normally run in the same address space.

Q: Where does Darwin come from? Which parts are built around each of the different BSDs and/or NextStep?

EP: The current 1.0 version of Darwin actually shares very little code with the original NextStep implementation. The Mach microkernel is actually based on the OSF code used in MkLinux. The BSD kernel is based on BSD 4.4Lite, primarily the FreeBSD 3.2 distribution, with a healthy dose of NetBSD.

Q: What does Darwin add to the mix? Are there any innovations new to *BSD in Darwin?

EP: For an existing BSD developer, Darwin has the advantages of:

• a powerful kernel extension mechanism and I/O Kit for writing drivers

• traditional Apple technology (HFS+, AppleTalk)

• a well-funded commercial developer with a single well-supported reference release

• a very cool optional (though proprietary) GUI called Aqua (i.e., Mac OS X)

Q: Is there any other OS which Darwin is trying to match or beat in features?

EP: Darwin’s top priority is making Mac OS X the world’s best personal computer operating system. Beyond that, we want to help our developers turn Darwin itself into a fully-functional standalone open source operating system, compatible with FreeBSD and comparable to Linux.

Q: Will Darwin include features like FreeBSD’s sysinstall or ports tree?

EP: Actually, we already ship Darwin with the Debian packaging mechanism.

Q: Is Darwin POSIX-compliant? How well should existing open-source apps (especially server apps like Apache, sendmail, qmail, PHP, etc.) compile on it or port to it?

EP: Most BSD and GNU stuff compiles out of the box. The major limitation used to be pthreads, but that was pretty much fixed in Darwin 1.0. We have a very active developer community that is ensuring third-party UNIX software is becoming “Darwin-ready” which usually means just defining the appropriate #defines and makefiles.

Q: Does Darwin bring any particular optimizations or improvements that would make it good from an ISP/Internet server perspective?

EP: Well, we have a dedicated team of networking engineers focusing on giving it best-of-breed performance integrated with Apple hardware. As well as a really cool GUI for administration (if you buy Mac OS X). Other than that, we use generally the same code as FreeBSD and NetBSD, which is already pretty thoroughly optimized.

Q: Are there any reasons you could think of that someone might choose to use Darwin instead of FreeBSD/NetBSD/etc.?

EP: Several:

1. We will be shipping Darwin (as Mac OS X) on millions of machines next year, giving developers an enormous potential user base.

2. You have the option of purchasing a very cool UI (Mac OS X).

3. We have excellent support for the latest high-performance PowerPC hardware

Q: How much of a priority is the x86 port of Darwin?

EP: A lot of Darwin developers are excited about being able to use the exact same open source OS on both PowerPC and Intel hardware, so we’re doing what we can to help. We demonstrated some major progress at WWDC [the Apple World Wide Developers’ Conference], and I think the community is gearing up to finish the work on their own.

Where is Darwin Now?

As of this writing, the current version of Darwin is 1.0.2. Darwin currently boots on Intel hardware, but there are a number of unresolved issues right now which keep it from being genuinely functional. Apple provides a binary release and installer for Darwin on the PowerPC architecture (located at http://www.publicsource.apple.com/projects/darwin/release.html), and it current supports G3/G4 Macintoshes as well as a few older desktop models. The PowerPC version sits entirely inside a single UFS or Apple HFS+ disk partition (current limitations of the Darwin booter limit the boot volume size to 8 GB).

Once you install Darwin, you can tell pretty quickly that most of Apple’s efforts are (understandably) going towards making Darwin work with MacOS X, rather than on its own. (In fact, most of the Darwin projects in its CVS repository are live and being used by the Apple MacOS X team at the same time as the Darwin community.) This is most obvious in the fact that the Darwin-specific documentation and install/setup goodies included with the OS are about zero – Apple is working on the MacOS X GUI to include these things.

When you install Darwin 1.0.2, you’ll boot and be dropped off into a tcsh shell. From there, you’re on your own – don’t expect any sysinstall or LinuxConf-style goodies to help with setup and configuration. For most experienced administrators, this wouldn’t pose too much of a problem, except that Darwin has a lot of its files stashed in MacOS X-centric locations that take a while to puzzle out. For example, don’t bother looking in /etc/httpd or /usr/local for a httpd.conf file – you’ll find it as /Local/Library/WebServer/Configuration/apache.conf.

At this point, Darwin is certainly more of a toy for developers and “early adopters” than an OS worth implementing for an Internet server. OpenSSH hasn’t been finished up yet, nntpd and some other server staples seem to be MIA, there are plenty of device drivers missing, and there’s plenty of work yet to be done on the Intel side. Documentation is still pretty sparse, but some good stuff is developing out there, including a helpful “Unofficial Darwin FAQ” at http://www.synack.net/darwin/faq.html. You can keep up on Darwin’s progress by subscribing to any of the Darwin mailing lists at Apple (http://www.publicsource.apple.com/projects/mail.html) or, if you’re lazy like me, checking out project leader Wilfredo Sanchez’s updates to his Advogato page (http://www.advogato.org/person/wsanchez/).

Darwin is very new, and time will take care of many of its current deficiencies (many of the ones I’ve described will almost certainly be fixed by the time you read this). Despite its shortcomings, Darwin 1.0.2 is a fully functional BSD Unix system, and the default install includes several of the third-party packages you’d hope to find like Apache, Sendmail, Perl and BIND. Ports for XFree86 (much of the original work on this port was done by id Software’s John Carmack), OpenSSL and other common tools are available in source form from the Darwin CVS repository.

Where is Darwin Going?

Darwin’s promise lies not in where it is now, but where it will be. Darwin can catch up on plenty of things by grabbing code from the other BSDs, and it is likely only a matter of time before Darwin develops rough feature parity with some of your favorite BSDs. The Apple Darwin team has said that future releases of Darwin will try to track FreeBSD; Jordan Hubbard was one of the speakers at the BSD session at May’s Apple World Wide Developer Conference. The IOKit driver framework that Apple has created for MacOS X has a lot of potential for (relatively) easy device driver development.

The real promise lies in the possibility that the large installed user base of MacOS X systems that will be out there will drive developers (especially commercial ones) to bring their server apps natively to a BSD-based system for the first time. Of course, porting something to MacOS X isn’t the same as porting to BSD; most commercial applications coming to MacOS X will use its API rather than the Darwin/BSD API. Nonetheless, even if some of these apps which currently exist for [insert your least-favorite commercial Unix here] are ported to the Darwin layer of MacOS X, it will be a victory for BSD.

The close relationship between Apple’s Darwin team, the Darwin community, and the FreeBSD community is an excellent sign for Darwin’s future. While it may not be much more than a curiosity right now, Darwin is a welcome addition to the free *BSD family, and will definitely be something to watch in the next year.

Trusting BSD FreeBSD for Security Ultra-Gurus

By Jeffrey Carl

While most Freenix admins are used to securing their servers, there is a “higher” world of security that has never been touched by free *nixes. The realm of “trusted” operating systems, long the province only of military and other ultra-secure environments, represents a security level beyond that of all but a few operating systems. Now, however, the TrustedBSD (http://www.trustedbsd.org/) project is working to bring that security level to FreeBSD and other *BSD OSes. If you operate a multi-user environment and you’re looking for the optimum in security, then learning about TrustedBSD is a very exciting project.

I asked Robert Watson, one of TrustedBSD’s lead developers, to describe the project for Internet providers and other BSD server users:

Q: What is the purpose of TrustedBSD?

A: TrustedBSD provides a set of security extensions to the FreeBSD operating system under a liberal license. Targeting both military and commercial models, TrustedBSD includes traditional trusted operating system features, as well as an extensible authorization architecture to simplify development of new security features. The TrustedBSD project provides an organizational framework through which to discuss, design, develop, and document these extensions.

Q: What is the Common Criteria for Information Technology Security Evaluation (CCITS)? What does it mean to “real world” uses of the OS?

A: The Common Criteria (“CC”) are a set of security description and evaluation documents developed by the United States and other governments, as well as ISO standards. Using the CC, you can use a common terminology to describe specific sets of features and degrees of evaluation. Many readers will be familiar with the Orange Book (and entire Rainbow Series) which can be considered precursors to the CC in the United States. While the Orange Book largely targeted military applications, the CC really provides a language for setting goals and determining if they have been achieved, rather than prescribing a particular set of features required for a given certification. Unlike the Orange Book, TrustedBSD specifically targets commercial and network-centric environments.

TrustedBSD is being developed with this vocabulary and evaluation system in mind, as the CC by definition provides a common criteria for understanding security products. At this time, no formal evaluation is planned, as formal evaluation requires substantial investment of resources, including financial resources. However, TrustedBSD will make an excellent candidate for evaluation. It is possible that companies choosing to resell TrustedBSD may seek formal evaluation [as a “trusted” operating system].

Q: What benefits will the TrustedBSD extensions to FreeBSD provide to admins using FreeBSD as an Internet server?

A: It is fair to say that some features of TrustedBSD will have an immediate impact on securing every-day server systems. Other features will come into play only in extremely security-centric contexts, such as military, electronic commerce, and banking environments. Regular systems may see the use of least privilege capabilities rather than volumes of less safe setuid and setgid programs. Similarly, they may take advantage of ACLs on files allowing more flexible discretionary access control. They may also take advantage of auditing features for intrusion detection. However, they are less likely to take advantage of more intrusive functionality, such as the confidentiality and integrity-oriented mandatory access policies that will be implemented.

Q: Why should an administrator of a FreeBSD web/mail/etc. server use the TrustedBSD extensions to the OS?

A: It is the intent of the TrustedBSD project to make almost all code developed for TrustedBSD available as part of the base FreeBSD operating system. Thus, all users of FreeBSD will benefit from the project by virtue of using FreeBSD. However, specific advantages, as described above, include the ability to more generally specify permissions on files and allow users to more easily manage resources, and to be able to run far less code with root privileges. As such, administrators taking advantage of this functionality will be able to run a more secure system.

A recent paper from Argus Systems, a commercial provider of trusted extensions for Solaris, described how use of mandatory integrity policies could have protected against the recent penetration of the Apache Project’s web server. Many of the same protection services will be available as part of TrustedBSD.

Q: What is the downside, if any, in installing the TrustedBSD extensions? Is administration significantly more complex? Are there any performance penalties involved?

A: Improved security is almost always a tradeoff against improved usability. That said, one important goal of TrustedBSD is to make the feature set of a trusted operating system easily accessible to users of a widely distributed free operating system. Any changes in system behavior will require administrators to understand the differences, but in most cases the differences will not be as substantial. Administrators will need to learn how to read, manipulate, and back up Access Control Lists (ACLs) on files, and understand how capabilities on files behave.

Performance implications depend on the feature in question: support for most new security checks introduces little or no overhead. However, features such as fine-granularity auditing require creation and management of large quantities of data. Performance-sensitive sites may wish to avoid using some features, such as auditing, as a result. We anticipate producing comprehensive evaluations of the performance impact as code becomes available.

Q: Are the TrustedBSD extensions planned to be architecture-specific? e.g., are the efforts of TrustedBSD only for i386 machines, or are they portable to other architectures that FreeBSD may be ported to?

The code base is intended to be architecture-neutral, and is written entirely in portable C. i386 platforms are the primary ones in use for development, but we also have access to Alpha-based systems for testing. As TrustedBSD’s supporting infrastructure allows for third-party extension, it is possible that third party security providers might distribute extension in binary-only form, but all base code will be freely available in source form.

Formal evaluation for certification requires selection of specific hardware, as whole systems are evaluated rather than purely software products.

Q: Does the TrustedBSD project intend on making its extensions portable to OpenBSD/NetBSD/Apple Darwin? What are the barriers to porting them?

A: TrustedBSD is made up of a number of components, some of which do require extensive modifications to the kernel structure. However, where such changes are made, they will improve kernel modularity and extensibility, supporting modular insertion of TrustedBSD components. If these modularity changes are introduced in other *BSD kernels, they can be used to support the higher level functionality. As the *BSD kernels are relatively close in structure, this task is well within the realm of possibility. The KAME IPv6/IPsec implementation is an example of a project that has successfully developed software for multiple BSD platforms. The BSD-style license used in TrustedBSD should pose no problem for integration with almost any other project, open source or commercial, and the TrustedBSD project would be glad to see and assist in integration into other operating systems.

Q: Do the TrustedBSD extensions to *BSD open the doors to any new uses for the OS? e.g., Are there any tasks to which the existing BSD flavors were unsuited to before for which they are now suitable?

A: Yes, it does, as it opens to doors to uses that have more demanding security requirements. Examples include banking, military, and electronic commerce. That is not to say that FreeBSD wasn’t used there before, as it was already a very qualified OS for these environments, but his provides a higher degree of assurance. An extensible security infrastructure will make FreeBSD/TrustedBSD a more appealing platform for security research and development, also.

If formally evaluated, then it would be open for use on classified networks in ways in which it currently isn’t usable.

Q: How does TrustedBSD compare to OpenBSD’s security goals? Are they competitive projects or complementary projects?

A: Complementary. To our knowledge, OpenBSD has really taken quite a different approach to security: fine-grained source code auditing and integration of cryptography. TrustedBSD introduces new authorization models, as well as auditing. While we won’t be porting the changes over to OpenBSD at this time, this doesn’t mean others can’t, or that we won’t port them in the future.

Q: How many active contributors are there to the TrustedBSD project? What areas are you looking for contributors to?

A: TrustedBSD has a relatively small developer community, although we hope it will grow as interest grows. At this point, the number of active developers is around 6 to 10, but there is substantial design discussion on our mailing list from a far larger group bringing experience from a variety of other platforms. There is also substantial interest in cross-platform portability for applications.

At this point, contributors should have strong kernel and application security experience; in particular, we’re interested in developers who have past experience with trusted operating systems. As the project progresses and the kernel component reaches greater maturity, we’ll be interested in application programmers to help adapt existing applications for any changes in the trusted OS environment. As there is interest in portable APIs, we hope to be able to leverage other work in this area.

Q: What are TrustedBSD’s goals for the next six months/year/unspecified dates in the future?

A: As with all operating system scale projects, TrustedBSD will involve an iterative development process, with features becoming available over time. We feel that the current goals can be broken down conceptually into a number of implementation phases:

• In the first phase, the goal is to introduce TrustedBSD security changes directly and within the context of the existing FreeBSD security implementation. This phase is well underway, and will result in a usable, secure system. However, the goal of this phase is to gain both development and operational experience: while some of this code will be integrated into the base system (some already has – support for extended file attributes, and ACL interfaces), the majority will be made available via seperate distribution mechanisms.

• In the second phase, the goal is to generate stronger security infrastructure in the kernel, aiming for greater generalization and modularity. In this phase, we build on the practical development experience in the first phase, having a strong understanding of the requirements base on having implemented the same features. One of the main targets of this phase is a generalized kernel authorization framework, allowing modular and pluggable security extensions to be introduced without substantial source modification (as required in the first phase). This will allow both the TrustedBSD project, and third party developers and vendors, to distributed FreeBSD security enhancements with substantially lower development and maintenance costs.

The first phase is well underway — some interfaces (in particular, for ACLs) were included in 4.0-RELEASE, and supporting infrastructure such as extended file attributes have been committed to the FreeBSD 5.0-CURRENT development branch. Support for fine-grained least privilege (a variant of POSIX.1e capabilities), native ACL support in the FFS file system, fine-grained event auditing, and MLS MAC policy support is in the works. We see a time frame for the completion of the implementation of this component within 12 months; however, one aspect to trusted operating systems is strong supporting documentation, including design, implementation, and operation. Completing this documentation may take additional time.

The second phase is currently under design–we’ve gained substantial experience in that which we have completed of the first phase, and are ready to begin discussing the stronger and more general abstractions in the second phase. By the time this interview is published, drafts for at least the modular authorization system will be under discussion on the TrustedBSD mailing lists. So far, the results look promising. We hope to have a first prototype of the generalized authorization system completed within six months, and to being migrating the security models implemented in the first phase over to this mechanism shortly thereafter.

We hope to present a number of papers detailing aspects of the TrustedBSD project at the BSD Con 2000 conference in October.

What’s New with Linux-Mandrake 7.0? An interview with Steve Schafer of Macmillan

By Jeffrey Carl

This column was originally supposed to be about supporting FrontPage users on Unix, but Boardwatch’s editors edited all the swear words out of my 2,000 word column and were left with a total of 12. So instead, I turned to Macmillan’s Linux-Mandrake 7.0.

The Basics

Linux-Mandrake 7 is earning rave reviews all over the place right now, and with pretty good reason. Linux-Mandrake is heavily optimized for Intel Pentium-family chips, but should run fairly well on AMD or Cyrix-based systems. System requirements are listed as any 586 (Pentium-equivalent) or higher CPU, 16 MB RAM (64 MB recommended), 500 MB disk space minimum (1 GB+ recommended), and a 3.5″ floppy drive or CD-ROM drive for installation.

Linux-Mandrake 7 is available free via FTP from http://www.linux-mandrake.com/en/ftp.php3. It comes in standard FTP install format, as well as a downloadable ISO 9660 CD image. MandrakeSoft itself sells distro CDs, and Macmillan also distributes and sells three shrink-wrapped versions of Linux-Mandrake 7.0: Complete, Deluxe and Secure Server (see below in the interview for more information on what the different packages contain). The Complete package retails for $29.95 (although I found mine at Wal-Mart for $25); Deluxe sells for $55-$60, and the Secure Server package sells for about $100.

The Distribution

The distribution is pretty up-to-date (at least as of this writing). It includes kernel 2.2.14, glibc 2.1.2, XFree86 3.3.6, and more. All of the defaults here are stable – you won’t find anything too bleeding edge in the included components, so if you want to wreck your system with the development kernel or XFree 4.0 beta du jour, you’ll need to go out and download it for yourself.

Most of the clamor for Linux-Mandrake 7 has been over the reports that it’s the easiest Linux distribution ever for a new user to install and set up. Mandrake’s efforts in this direction are focused on three new (or at least fairly new) tools.

The first and second are DrakX, first introduced with Mandrake 6.1, and DiskDrake. DrakX is the new all-singing, all-dancing, all-graphical installation tool – and it’s certainly the most user-friendly install tool that I’ve seen for a Freenix (an install tool with themes?). It handles all of the basics you’d expect, as well as taking a good stab at autoconfiguring the LILO boot loader and X Windows (plenty of drivers for popular video cards are included). DiskDrake is the graphical disk-partitioning tool for use with DrakX, although you can also use good old fdisk or cfdisk.

The third new tool is DrakConf, which bundles in LinuxConf as well as its own versions of hardware, software, networking and security tweaking tools. For a user or administrator new to the platform, DrakConf keeps a lot of common tools very handy. Also available are RPMdrake (for RedHat Package Manager installations) and Lothar, a nifty program for autoconfiguring sound cards and other audio-related devices.

So, what’s not to like? With all of the goodies in this distribution, Mandrake has its sights squarely focused on its intended consumer – and that consumer probably isn’t you. Linux newbies and desktop/workstation users are the target market here, and it isn’t evident that much has been done to optimize or enhance Linux-Mandrake’s performance specifically as an Internet server.

However, that last assessment may be a little too harsh (or too much to expect). Regardless of what type of user you are, good administration tools help everybody; and the Pentium optimization certainly helps if you’re running an Intel-based server. And to some extent, every Linux distribution uses some version of the same kernel, so there isn’t much you can expect Mandrake to do to increase the performance of the Linux TCP stack.

The Questions

To find out more about the appeal of Linux-Mandrake 7 for ISP/webhosting users, I asked Steve Schafer, the Senior Title Manager for Linux at Macmillan.

Jeff: Can you give a brief history of Linux-Mandrake?

Steve Schafer: Linux-Mandrake started as a volunteer project to help make the standard Red Hat Linux distribution more robust and yet more user friendly. Approximately two years ago the principals of the project created a company, MandrakeSoft, to oversee and lead the project into a retail venture. Version 5.3 of Mandrake was built from the Red Hat 5.2 distribution and was distributed through outlets like LinuxMall. The distribution continued to gain a following for power and ease of use. Versions 6.0 and 6.1 went on to win the “Editor’s Choice: Product of the Year” award at LinuxWorld in 1999, and various kudos for being a “better Red Hat than Red Hat.” Mandrake 7.0 raises the bar even further as you will see below.

J: Why should I use Linux instead of another Freenix like FreeBSD or OpenBSD?

SS: One word: support. Linux remains on the cutting edge of technology, with solid support both online and paid, the latter provided through retail purchase or a support contract through a dedicated Linux support organization like Linuxcare. The interface and standards continue to evolve, presenting a clean, more user-friendly environment for a technical OS.

J: What’s new with Linux-Mandrake 7?

SS: The main differences in 7.0 come in the way of installation and configuration. The new graphical install (DrakX) can be tailored for each user’s tastes and technical level – the “Recommended” install provides for less decision making on the part of the user and makes various assumptions about the target machine to make the install fairly seamless, while the “Expert” install allows the user full-control over how the OS will be installed. Several new customization utilities allow the user to quickly and effectively change the configuration once the OS is installed, changing the interface, security options, adding and configuring hardware, and more. Mandrake continues their tradition of offering more base utilities and pre-configured desktops as well.

J: How does Linux-Mandrake perform for Internet serving tasks in relation to other Linux distros? Is the differentiation something other than performance (e.g., ease of use, pre-installed apps, etc.)?

SS: By and large, most Linux distributions perform the same types of serving tasks since they are all cut from basically the same mold. The difference Mandrake makes comes in two areas: customization and Pentium optimization. See the other section(s) for info on the customization (both what Mandrake does automatically, and what tools exist for the user). As for Pentium optimization: Mandrake recompiles ALL packages with Pentium optimization. Although the performance increase for workstations is slight, it is much more pronounced in the server environment. When a server is loaded with several users all utilizing resources, the faster the server can complete tasks, the better. (Of course, this is only true if the server’s processor is Pentium [or derivative] based.)

J: What are Linux-Mandrake’s different products, and whom are they intended for?

SS: Macmillan offers three distinct Linux-Mandrake products, geared toward specific Linux customers:

Complete: This value-packed product is geared toward the beginning user, or the user who is taking Linux for a “test drive.” Macmillan adds the following components to the base Linux-Mandrake OS:

• PartitionMagic and BootMagic – for ease of installation on a Windows machine for dual-booting between OSes. (Recent research showed that 70% of retail Linux purchasers install Linux in a dual-boot configuration.)

• StarOffice 5.2 – This powerful office suite provides word processing, spreadsheet, presentation, and graphics functionality. Compatibility with Microsoft products ensures maximum data transportability.

• Linux Library – 3500 pages of additional Linux documentation (in electronic form) from the Macmillan imprints Que and Sams.

Deluxe: This six-CD set provides the most Linux content for the money. Geared to address the professional user, this product provides more of everything Linux. In addition to the base Mandrake OS and its sources, these additional 4 CDs are provided:

• Contributors CD: Close to 900 additional utilities, applications, documentation, etc from the Linux community.

• Contributors Source CD: Source code for the Contributors CD.

• Applications CDs (1 & 2): Over 30 commercial and demo applications, including StarOffice, WordPerfect for Linux, etc.

[Jeff’s Note: The Commercial Applications CDs (available with the Deluxe version) includes full and limited versions of some very tasty goodies including: Acrobat Reader 4; Executor (a MacOS emulator); IBM’s JDK, Lotus Notes and ViaVoice (voice recognition word processor). It also has demos of the games Civilization: Call to Power, Railroad Tycoon II and Myth II; an evaluation version of VMware; and WordPerfect. True, almost all of this is available via download – but if you’d like it all in one easy-to-install package, this is a mother lode.]

Secure Server: For the professional “with a purpose,” this product offers the base Mandrake OS bundled with a secure Web server for starting an e-commerce operation. Additional utilities and a custom Linux Library (over 4200 pages of electronic docs from Que and Sams) round out this product.

J: What are Linux-Mandrake’s particular strengths? What are its weaknesses?

SS: Mandrake’s strengths include:

• Red Hat compatibility

• Pentium optimization

• Cutting edge components (latest kernel, XFree, etc)

• Scalable graphical installation

• Pre-configured desktops and user interfaces

• Comprehensive graphical configuration tools

• Additional packages, drivers, utilities, apps and more

Honestly, it’s hard for me to think of weaknesses regarding Mandrake. This distribution has the depth to satisfy the hardcore user, but the simplicity (install and customization) to engage the beginner as well. Linux overall does have weaknesses, mostly surrounding the implications of open source and staying ahead of the technology curve. For example, there isn’t one multi-million-dollar firm behind the Linux distros (Microsoft) and the kernel has just begun to support technologies like USB. Keep in mind that these disadvantages exist across the spectrum of Linux, not just with Mandrake.

J: Why should I use Linux-Mandrake instead of another distro for my web, mail or dialup authentication server?

SS: All the reasons stated above, particularly the ability to support the latest hardware, ease of customization, bundled components, etc. This is true across the board of applications (web, mail, dialup authentication, etc).

J: What’s the recommended hardware for a Linux-Mandrake Internet server? Is there anything that works particularly well?

SS: The answer really depends on the workload and user load the particular box will experience. One prominent ISP (CiHost) uses Red Hat for their Web hosting servers which are dual-Pentium boxes. Recently LinuxWorld (February in New York) used Mandrake for their registration system, probably high-end Pentiums using dumb terminals (low-end boxes) for input.

When considering Linux-Mandrake, it’s important to consider running on a Pentium or derivative due to the recompilation using Pentium optimization.

J: How easy/difficult is it to migrate from another Linux distribution to Linux-Mandrake? Is there anything I should know?

SS: It depends on what you are migrating from. Although most Linux distros are pretty much the same, some vary considerably in customization, installed tool sets, libraries, etc. Like migrating between Windows versions, you want to ensure that your added tools (applications, utilities, etc) are compatible with the new system. Red Hat and derivative Linux distros use some different libraries than other distros, requiring different base level support for some programs.

J: On the client side, can ISPs support Linux-Mandrake dialup users easily? Why?

SS: You should be able to with no problem. Linux speaks the Internet natively (TCP/IP) so utilizing a PPP dialup is almost second nature to the OS. It’s a bit tougher [for the user] to set up than [for the ISP to] support really … the ISP shouldn’t have to change a thing.

The Conclusions

As Linux distributions go, Linux-Mandrake 7.0 is very good. Aside from some strange gaps in documentation (for example, my Macmillan Linux-Mandrake Complete “User Guide and Reference Manual” didn’t even mention DrakConf), it seems pretty solid.

For the ISP/web user, there are two groups that will find it particularly compelling. First are new users/admins who want the easiest, most pain-free installation and setup. Second are users with Pentium-based machines, who may see impressive speed gains over other distributions. Note: I wasn’t able to throw enough machine load at my test server to test this out reasonably; if someone out there does, please let me know the results. For anyone, though, Linux-Mandrake 7 is certainly worth a look.

Jabber and Zope – Can Open-Source Beat the Most Popular Commercial Products?

By Jeffrey Carl

“Jabber” and “Zope” sound like two characters from “Alice in Wonderland.” Or maybe characters in Quake II (“jabber saw the light from zope’s BFG-10K”). Either way, what they really represent are attempts by the open-source community to beat commercial offerings in emerging critical Internet areas: instant messaging (IM) and web application development.

Can they do it? On one hand, projects like Apache and Linux or *BSD have shown that open source software can lead the industry in performance. On the other hand, both projects face the perennial Achilles’ heel of open-source projects – lack of commercial distribution channels and a corresponding difficulty in gaining enough market share to become a standard. Let’s take a look at both projects, how ISPs can use them, and whether they can beat the big commercial packages.

Jabber

Once upon a time, there was Mirabilis ICQ (“I Seek You”). And lo, it was good. Then it was bought by AOL and became the basis for AOL Instant Messenger (AIM). Then Microsoft’s MSN and Yahoo! got into the act with similar programs that could communicate with the zillions of ICQ/AIM users. Then AOL said, “Oh no you don’t.” Then Hell froze over and Microsoft started complaining to AOL about “open standards” and MSN et. al. whined to Congress about it…

Right now, IM is a mess. AIM, with an astonishing 45 million users, holds tight to its grip on ICQ, keeping its service proprietary. Few people are inclined to shed tears for Microsoft because it’s being disadvantaged by proprietary systems. Still, many users are seeking an IM program that, like J.R.R. Tolkien’s “One Ring,” would “bind them all together.”

What is it?

There are plenty of open-source ICQ clones (like LICQ [www.licq.org], popular on Linux). Jabber is quick to point out that it isn’t just another ICQ clone. As the project’s FAQ page says:

Put simply, Jabber is an open source, cross-platform, completely extensible, end-all to Instant Messaging as we know it. Never again will you have to worry about finding the right client to talk to your friends, nor will you have to concern yourself with having three or four different clients open so that you can chat with all of your associates. And all you have to do is pick the client that you like for the platform you want!

Jabber is genuinely different from other instant-messaging products, although the differences are “under the hood.” A Jabber user gets an account with a Jabber server, and the server then takes care of all the “heavy lifting” – communicating with other servers and translating the protocols of other IM systems into Jabber’s own XML (eXtensible Markup Language)-based messaging protocol. In theory, server modules can be written to parse any other IM protocol, and connectivity to those systems is then automatically provided to the client.

How does it work?

Joe User gets an account with a Jabber server. He also gets accounts with AIM and Yahoo! Pager. His Jabber server stores his account info and “buddy lists” (roster) for each IM service, allowing him to access all of them through his one server and one client. Assuming that the server has the requisite protocol “transports,” he can then communicate with users of any of these services from his Jabber client.

The native Jabber protocol is an XML stream over a TCP socket, allowing for a number of possibilities for how and over which port it can be sent (a bonus for its potential ability to evade firewalls, critical to the success of any IM protocol). The Jabber client communicates only with its server, which then handles all of the protocol-translation and communication issues.

What’s so great about it?

Obviously, the ability to communicate with any IM protocol is a huge plus, potentially making Jabber the “holy grail” of IM platforms. Also, its reliance on the server model simplifies matters for users significantly.

While it promotes unity among IM protocols, Jabber also promotes competition within its own ranks, and Jabber users can choose from a number of different Jabber clients, each with their own foci, strengths and weaknesses. Its open standard and protocol means that anyone who would like to create a new client or offer new features can do so with (relative) ease.

Also, Jabber is intended to be cross-platform. Work on clients is reportedly underway for *nix on command line and X-Windows, Win32, MacOS, browser-based, and even Newton.

What still needs work?

As of this writing in mid-April, just about everything. Jabber is still very much a work in progress (its server has yet to reach 1.0 status or implement working transports for other IM systems), and it remains to be seen a.) how well its compatibility with other IM protocols will work, and b.) how well it will work by itself. Most of the clients are still missing some user-interface niceties, and a working standard for security/encryption of Jabber messages has yet to be implemented.

Jabber is like Rambus – one of those long-shot bets which will either take over the world or be a perennial also-ran. It has all of the right ingredients, but only time will tell whether it fulfills its promise.

A critical hurdle is the same interoperability issue which has plagued other IM clients: will AOL try to block it? AOL’s ICQ users are such a tremendous asset to AOL that it’s hard to imagine that AOL won’t try to block Jabber out if it gains much share of the market. The jury is still very much out on Jabber’s eventual success … and whether, if it can’t communicate with others, it can succeed on its own.

Zope

Zope (the “Z Object Publishing Environment”) is … well, it’s hard to describe in a hurry. But it’s very cool.

What is it?

It describes itself as an “application server,” and its goal is to provide a framework for creating and maintaining large websites with advanced functionality like site users, feedback and content databases. Elements of Zope (currently at version 2.1.6) compete with both higher-end commercial software like WebObjects or Cold Fusion, as well as open-source site spinoffs like slash (the source code for the backend to slashdot.org).

Zope is portable, since it’s written in Python (with some performance-critical parts in C); currently, it runs on Windows NT and nearly any Unix. Installing and running Zope requires that you have Python (http://www.python.org) version 1.5.2 or higher installed on your server. It requires Python to be built with threads support; if you’re installing Python to use Zope, be sure to run the pre-installation configure script with the –with-thread option.

Zope began life in the Bobo Web Object System, developed by a company called Digital Creations (http://www.digicool.com). The Bobo framework was open-sourced, while the company sold a commercial application server based on it, which was named Principia.

Seeing that the application server market was dominated by big names whose marketing muscle Digital Creations could never compete with, the company decided in December 1998 to combine Principia and Bobo, rename it Zope, and open-source the whole shebang. Zope began to gain recognition and users, while Digital Creations shifted its business model to providing commercial support and consulting for Zope.

How does it work?

The heart of the system combines an ORB (Object Request Broker) with an object database, allowing pages to be simply and dynamically created. The main components of Zope are:

• the ORB, which extracts information from the database

• the object database itself

• the publisher, which interfaces with a web server

• the template system, for dressing up content in your customized page templates

• the management structure, which handles authoring/access permissions

Here’s the short, short version: you download Zope to the server where its content will be server from. Zope is available as a Windows executable, in RPM format for Linux, or tar-gzipped source archives for Solaris, Linux or any other *nix. For a quick start, just run the Zope setup script, then run the start script it builds, and away you go. Zope launches its own web server on the port of your choice (default 8080), although you can easily configure it to use Apache, IIS or another server.

You can then use a web browser to view pages or go to its “manage” interface. Using these controls, you can create new folders and add content to them. You also use the manage interface to assign basic functionality (who can access or alter the contents, HTML templates and error messages, etc.). This interface also provides “approval” and “undo” functionality as well.

The content comes in the form of … well, lots of things – users, images, or other items usable by the database. Primarily, though, it will be DTML (Document Template Markup Language) objects, which can contain XML, DTML or good plain old HTML. DTML works like server-parsed HTML on steroids, offering very advanced functionality. An example might look like this:

<!–#var my_header–>

<!–#if “AUTHENTICATED_USER==’Jeff'”–>

<h3>Here is the customer list:</h3>

<table>

<!–#in showCustomers–>

<tr>

</tr>

<!–#/in–>

</table>

<!–#else–>

<em>Sorry, you can’t read this!</em>

<!–#/if–>

<!–#var my_html_footer–>

And there, you have a complete, well-formatted HTML page. The DTML page above calls in an external HTML page header that you uploaded named my_header, checks to see if the user has permission to view a table, and if so calls an object which is a SQL query that you defined earlier called “showCustomers” and displays the results in a table. If the user viewing the page isn’t the one it expected, it prints an innocuous HTML message. For everyone, it then prints the external HTML footer you named my_footer.

All of this can be easily done with CGI, but learning the DTML syntax is significantly easier for most people than learning Perl or C. Furthermore, the ability to integrate this into a sitewide management structure is a big plus.

So, essentially, with Zope you don’t so much build pages of a site as you build objects of content – FAQs, stories, other information – which get stuffed into a database. You then create folders and pages that are DTML frameworks which can include one simple content object (for a simple, non-dynamic page) or make complicated queries (the last five news objects submitted, all objects submitted by a certain author, the most recent graphic of a certain type, SQL queries, etc.). If you’re unfamiliar with the process, it takes a while to “wrap your head around it,” but it provides a tremendous amount of possibilities.

A very readable introduction that lays out the functional aspects of using Zope (as well as giving a much better explanation of DTML examples like the one above) can be found at http://www.devshed.com/Server_Side/Zope/Intro/. An in-depth look focusing on Zope’s database model is at http://webreview.com/pub/1999/03/05/feature/index2.html. There is an excellent library of pre-built and member-contributed projects that provide example code to learn from at http://www.zope.org/Products, as well.

What’s so great about it?

First, Zope provides an interface to adding/administering the site’s content which is accessible through a web browser. It allows for relatively complex security/permissions setups, which allow the administrator to set access privileges for various groups of contributors (administrators, writers, designers, etc.). Zope makes use of the emerging WebDAV (“Web-based Distributed Authoring and Versioning”[www.webdav.org]) standard to collaboratively manage and handle versions of pages or objects on a site. This also allows for good integration with existing software which supports WebDAV, like Microsoft Office 2000 or Internet Explorer 5 on the client side or Apache with mod_dav on the server side.

Zope provides its own transactional object database, which means that you won’t need to add another database (e.g., MySQL, Oracle, MS SQL, etc.) to provide a back-end. One of the primary benefits to using a transactional (keeping track of additions/changes as “transactions”) database like the one Zope provides is that it allows you to “undo” changes if you need to. The database is optimized to match the needs of most web databases, which are frequently being queried but only occasionally have new data written to them. The Zope database is also easy to back up, and is designed to work with “simple URLs” (unlike those created by WebObjects).

One of Zope’s strong features is interoperability with existing SQL databases. You can set up a Zope database connection to an existing database and apply SQL queries from a Zope DTML page to the older database. Zope also makes good use of XML, allowing interoperation with XML data sources. The Zope database stores a great deal of information besides just your primary content objects; it also keeps track of user postings and threaded discussions, user logins and permission settings, and allows users to customize the way they view the site.

Like any good open-source product, Zope makes very good use of open standards. Aside from WebDAV, SQL and XML, Zope supports HTML 4 and CSS, HTTP 1.1, FTP, FastCGI, DOM, LDAP and other goodies.

Lastly, it was announced in late March that Digital Creations would be open-sourcing its ZEO (Zope Enterprise Option), which can turn Zope into a distributed system. Allowing Zope to scale to multiple machines should make it a feasible choice for high-demand sites which need to be served from multiple machines. Zope has also announced interoperability with Microsoft’s SOAP (Simple Object Access Protocol) sort-of-standard. Whether this will become a “big deal” remains to be seen.

What still needs work?

At first glance, Zope is just plain difficult to understand for most people. The Zope feature tour (http://www.zope.org/Tour/ar01.html) is a good start, and the Zope Guides (http://www.zope.org/Documentation/Guides) are helpful (if understandably incomplete) but some more “hand-holding” higher up in the online documentation might convince more users to “get their feet wet” with Zope.

Zope is certainly on the right track with the emerging standards it uses, but some of these standards will take time to build the user base to allow Zope to really shine (for example, when WebDAV publishing will work from more applications than just IE 5 and Office 2000). The WebDAV standard itself isn’t fully fleshed out yet, and I have to admit that any standard which Microsoft has taken a strong interest in makes me worry a bit.

Lastly, Zope simply isn’t for everyone. If you have a relatively small or simple site, the overhead of learning and serving Zope will probably be a waste of time. If you’re willing to put in the time, using Zope from the get-go will help you scale your site if you need to – but in many cases, using Zope is swatting a fly with a sledgehammer.

The Verdict: Jabber, Zope and You

What can Jabber and Zope do for you? Well, in the short term, you can offer support for them and promote them to your customers. With Zope, ISPs can offer Zope support and make a presence in the small (but growing) market for Zope site-hosting.

With Jabber, ISPs can run their own local Jabber servers. In the short term, it can be used by supplying your NOC with a Jabber nick through which clients can contact them; in the long term, if Jabber meets its promise, your local Jabber server can be a great way to tie users to you (to be Machiavellian, your Jabber server will hold all of their IM contacts and make it a pain to migrate elsewhere).

Will either of these projects ever gain the ubiquity of Apache, sendmail or Linux/BSD? Probably not. But Jabber holds an extraordinary amount of promise, and can be an excellent value-added tool for ISPs – and it’s free – so there’s little reason not to be an early adopter. Zope will almost certainly remain a limited-interest project, but it can be of use to you for your own web sites as well as providing a differentiating service for site hosting.

Can Jabber and Zope beat out the commercial competition? Only time will tell. But, considering the cost (nothing but your time), it’s almost a no-brainer for you to “do the right thing” and support open standards, and to investigate providing support for both with your ISP.

Who knows? If you bet on enough long shots, one of them might turn out to be a winner.

BSDI, Walnut Creek CD-ROM Merge – Move Brings Two Biggest BSD Unix Variants, FreeBSD and BSD/OS, Under One Tent

By Jeffrey Carl

Confirming a deal that had been whispered about among the BSD Unix community for months, Berkeley Software Design, Inc. (BSDI) and Walnut Creek CD-ROM announced on March 9 that they would merge to form a new company under the BSDI name. The significance of the announcement lies in that it brings the vendor of commercial BSD-based systems (BSDI’s BSD/OS operating system) together with the primary distributor of the largest free BSD variant (FreeBSD), signaling a major push toward unity among the BSD Unix flavors.

The move also opens the door to more intermingling of code between the two OSes, since some of FreeBSD’s key developers were employed by Walnut Creek and will now be employed by BSDI, and members of BSDI’s BSD/OS engineering team may be contributing code to FreeBSD. In a separate move, Walnut Creek’s Slackware Linux distribution will be spun off into a separate company, run by longtime Slackware chief Patrick Volkerding.

In the short term, users of FreeBSD and BSD/OS won’t see much change to the current products (FreeBSD 4.0 and BSD/OS 4.1). BSDI will continue to offer their commercial BSD/OS and provide paid support and consulting, while FreeBSD will remain free. However, BSDI will soon offer commercial support options for FreeBSD – potentially paving the way for FreeBSD’s entry into the enterprise market in the same way that commercial support for Linux from RedHat, LinuxCare and others did for Linux. Just as important to the enterprise market, BSDI CEO Gary Johnson said that there are plans to implement a “BSD engineer” certification program by the end of Q2 2000.

In the longer term, the move offers a wealth of potential possibilities by placing the resources of a well-funded corporate entity behind promotion of BSD Unix and encouraging commonality between the OSes. BSDI has said that it will work to develop a common Application Binary Interface (ABI) between FreeBSD and BSD/OS, enabling an application written for one OS to run on the other without any modification. While specifics are still very much up in the air, it is likely that some elements from each OS will find their way into the other, over the course of the next few OS releases.

Unconfirmed possibilities include extending FreeBSD’s hardware support with the addition of some BSD/OS code, and making user-friendly elements like the FreeBSD “ports collection” software installation scheme available as an optional add-on to BSD/OS. Still, BSDI is adamant that the merger won’t make users of FreeBSD or BSD/OS sacrifice any of the qualities they treasure in their OS of choice.

Most exciting for many *BSD users is the likelihood that many of the “old school” computer scientists who were part of the UC Berkeley Computer Science Research Group (CSRG) that developed BSD Unix and are currently affiliated with BSDI will be brought into the BSD community. While BSDI hasn’t firmed up its message about which OS to push to what constituency, it is likely that FreeBSD will continue to appeal to the open-source community while BSD/OS will be aimed at commercial and enterprise users.

Why was the merger done? To paraphrase Benjamin Franklin, the BSDs must all hang together, or surely they will hang separately. BSDI says that, between the two operating systems, there are at least two million BSD servers running out there. BSDI cites their collective market share as 15 percent of all Internet sites, as well as being used by nine out of ten ISPs or NSPs. Between the two OSes, customers include Microsoft Hotmail (which reportedly tried to convert to Windows NT but reverted to FreeBSD after numerous problems), UUnet and Yahoo! (which also made an equity investment into the new BSDI).

Still, the public presence of BSD Unix outside these groups is nearly nil, while Linux has garnered massive mindshare. “BSD is pervasive throughout the Internet … but the world at large doesn’t know it,” said BSDI Marketing Manager Kevin Rose. While this can be attributed to large number of factors, it lies largely in the fact that commercial BSD/OS has never had mass mindshare, while the free BSD variants with more users have never had the money to undertake real promotional campaigns.

While BSDI won’t discuss what the relative portions of those two million servers are FreeBSD or BSD/OS, it is relatively certain that the majority are running FreeBSD – which has never had the funding to promote its offerings on a level equivalent to what Linux distribution makers have done. The merger will likely put significant cash resources behind the promotion of BSD Unix for the first time. BSDI CEO Johnson said, “everything we’re doing is for the betterment of BSD … the idea is to promote not BSD/OS, not FreeBSD, not NetBSD, not OpenBSD, but BSD.”

FreeBSD Chief Evangelist Jordan Hubbard said that the merger also leaves the door open to cooperation with the other free BSD variants, NetBSD and OpenBSD. BSDI will be working to bring more third-party applications to BSD, as well as promoting publication of BSD books and developing user groups.

The fact that many of the technical details of the merger are still undecided leaves the door open to speculation that philosophical differences (or developer egos) may complicate the implementation of the full possibilities of the merger. Still, the merger announcement is a true turning point in the history of BSD Unix, and a much-needed encouraging sign for the platform.

Webmin Easy Freenix Administration through a Web Interface

By Jeffrey Carl

Are you looking for an easy web-based GUI to administer Unix servers? Want to provide your Unix clue-challenged hosting customers with an easy way to administer their machines?

If so, try Webmin (http://www.webmin.com/webmin/), a free application which allows you or your users to easily administer their Freenix system through a web interface. You can use it yourself, or you can offer it to clients with dedicated web or mail servers to do some of their own administration and take the burden off of you.

Installation and Setup

Webmin is a Perl 5 program that allows you to configure everything from user accounts to DNS and Samba filesharing and more. Webmin is free, and runs on a wide variety of Linuxes (including Caldera OpenLinux, RedHat, S.u.S.E., Slackware, Debian, TurboLinux, Mandrake, Delix DLD, Apple MkLinux) as well as FreeBSD and OpenBSD. It has been most thoroughly tested on Caldera OpenLinux, RedHat and FreeBSD, but it should run fine on other systems with potentially a bit of tweaking.

To install, go to ftp://ftp.webmin.com, and select the most recent version of Webmin (webmin-0.78.tar.gz at the time of this writing). Unzip and untar the file, then run the included setup.sh to install Webmin. Answer a few questions about your system setup, create a username and password for yourself, select a port for Webmin to run on, and you’re ready to go. To upgrade, download the source for a new version and specify the source file’s location in the Webmin interface’s Webmin Configuration -> Webmin Upgrade option.

Webmin is modular in nature, and comprises a “core” Webmin server with a number of default modules. Each module (like Cron, BIND, Syslog, etc.) provides administration functionality for its specified service. At installation time, all default modules are installed; you can remove modules through the Webmin interface, or download new third-party modules from a link on the Webmin home page. Webmin stores configuration files for all of its modules inside directories located (usually) in /etc/webmin/modulename/. The start and stop scripts for Webmin are also stored (somewhat confusingly) in /etc/webmin, rather than in /usr/local/sbin or in the Webmin home directory. Its logs are by default stored in /var/webmin/, rather than in /var/log/webmin/.

Webmin includes its own “miniature” webserver, so you don’t need to alter your Apache (or other web server) configuration to use it. The mini server is also a Perl script (/usr/local/webmin-0.78/miniserv.pl or something similar), which runs (owned by the root user) until the process is killed. This isn’t a terribly elegant solution, and it eats up about 3 MB of RAM as long as Webmin is running, but we’re assuming here that convenience is more of an issue here than absolute maximum performance.

If the idea of running a root-owned process over unencrypted HTTP scares you, you’re right. Webmin includes functionality to use Perl’s Net::SSLeay module to run its miniserver through HTTPS. If you don’t have this Perl module installed (and you’ll need to have the C libraries included with OpenSSL to get SSLeay to work), you’ll find download links and (relatively) helpful instructions for OpenSSL and SSLeay on the Webmin home page. Keep in mind, however, that setting SSLeay up can sometimes be, to use the technical term, a “major pain in the butt.”

Even better for security, you can also use Webmin’s interface to specify specific IP addresses from which Webmin can be accessed. This isn’t a foolproof setup, but it should be good enough for many system administrators.

Fun with Modules

Webmin’s interface is no-frills. It has very plain and simple graphics, loads quickly and gets the job done – a very wise choice, in my opinion. All functionality is provided in HTML tables instead of through a contrived graphical user interface.

As mentioned before, Webmin’s functionality is based on its included modules, each of which provides an interface to a specific service, application or daemon. The default installation includes all of the Webmin modules, which include such helpful items as MySQL, WU-FTPD, NFS, Apache, Users, Groups and Passwords, and a large number of other actions (for a complete list, see www.webmin.com/webmin/standard.html). Some default modules are OS-specific, like LinuxRAID, Linux Boot Loader and Network Configuration (for Linux and Solaris). Third-party modules which are available for Webmin (including ones for QMail, VNC and one which allows SSH logins in place of Webmin’s telnet tool) are available at www.webmin.com/webmin/third.html. There’s also a “wish list” of modules currently planned or under development at www.coastnet.com/~ken/webmin/wish.html.

Of course, having all of these modules available doesn’t mean that all of these services are available to you. Despite the fact that there’s a Samba Windows File Sharing module, for example, you’ll still need to manually download and install Samba on your machine before you can use Webmin to configure it.

Each of the included modules is well written, and provides a wide range of functionality. For example, the Apache module allows you to set up or alter virtual hosts, set permissions, add or modify MIME types, change Apache’s process limits and more. Even better, the module-writing spec is open, allowing you to write your own modules if you have a good knowledge of Perl and the application or service that you’re writing your module for.

One exception to this is the included Telnet Login module, which offers up a Java applet allowing you a telnet login through the web browser. This module is (surprise!) unfortunately dependent upon the Java Virtual Machine (JVM)/ Just-In-Time compiler (JIT) your browser is using, and can be unreliable in some cases. For example, it runs fine with the Apple JVM/JIT used by Netscape/MacOS, but is unusable with the Symantec JVM/JIT used by Microsoft Internet Explorer/MacOS.

Overall, however, Webmin’s functions are well defined and easily accessible. If you are at all familiar with the service that you’re configuring, Webmin provides a simple point-and-click interface that absolves you from needing to remember file locations and command-line switches.

Fun with Configurability

Through the Webmin Configuration option on its index page, you can set up a variety of options, including Webmin logging, port/address for Webmin, and interface look and feel. Perhaps unsurprisingly, this is a significant improvement over command-line based programs which often leave no clues as to where their configuration files are. Also, most of these configuration options can be set manually via the command line in /etc/webmin/miniserv.conf.

Another handy feature if you’re using Webmin to administer a number of machines is its Webmin Servers Index function (available from your Webmin index page). Choose a central machine where you do most of your administration, and then fill out the forms to “register” the other servers you’re running Webmin on. Alternatively, you can set up a list of servers on one machine, then copy the files in /etc/webmin/servers/ from that server to all of your other servers and have those links automatically established.

Every time thereafter that you click on the Servers Index button, you’ll be presented with a quick link to all of your other Webmin-enabled servers. You can specify a username and password to quickly log in to the other servers for convenience, or you can create a normal connection that will prompt you for a username and password for extra security.

An especially useful configuration option on the index page is Webmin Users. Through this, you can set up a variety of username/password logins for Webmin, and the modules that they’re allowed to access. This is particularly worthwhile if you want to set up one user for you (allowing access to all modules) and another user for your customer (only allowing access to modules for adding/removing users, Sendmail, Apache, etc.). With this setup, you can allow customers access to commonly used features but keep them from doing anything which might seriously “hose” their system.

This isn’t a completely secure setup, however, since information about the modules that users can access is store in a plain text file as /etc/webmin/webmin.acl (usernames and passwords are stored in /etc/webmin/webmin.users), and a user with root access could easily change this.

You Can Lead a User to Man Pages, But You Can’t Make Them Think

Webmin provides a great deal of functionality in the modules it provides; but what it doesn’t provide is help in understanding them. This is almost certainly too much to ask from a free admin program, but it does limit Webmin’s usefulness in some ways (at least for users who are not already very familiar with Unix). For example, it can allow a user to enable or disable the chargen service or edit /etc/fstab, but it provides no information about what those things are, or why you might want (or might not!) want to change them.

While a truly novice-friendly administration interface is too much to ask for, clickable keywords with glossary listings probably aren’t too much to ask for. The lack of documentation and help defeats one of Webmin’s primary benefits: the ability for a Unix novice to easily administer their system. While Webmin certainly aids new users by removing the burden of needing to know command-line options, it certainly won’t help them to configure Apache options if they don’t know what “Maximum requests per server process” means. Novice users are one of Webmin’s potentially largest markets, and it would be a shame if they didn’t provide explanatory text for their options in a future release version.

Still, this is a very forgivable gripe for a program which still isn’t even at release 1.0. What isn’t forgivable, however, is Webmin’s severe lack of documentation about itself, what it does, and how it does what it does. While this won’t deter the experienced system administrator, it limits how useful Webmin can be to administrators who would like an explanation of what they’re doing to their system, but don’t have the skills or knowledge to examine Webmin and its modules closely.

On the positive side, the Webmin site answers many frequently asked questions, and each built-in module also contains its own help information. A Webmin mailing list (frequently posted to by the authors) is also available; subscription information and a searchable archive are available from the Webmin home page. Even better, a Webmin Helpoption is always available from the index screen. Unfortunately, the help that is available appears to have been written as an afterthought, and the regexp searches that Webmin appears to do when you’re looking for help aren’t always very useful.

The installation doesn’t even include a documentation directory or man page, and users are left to figure out for themselves how the system works and what goes on. Most of the information I managed to gather about Webmin’s internal workings was from reading the source for the installer shell script and the Perl code of the individual modules. If you’re like me (and I feel bad for you if you are), you will want lots of documentation about any third-party tool that runs as root.

Conclusions: Good, But Not Perfect

Is Webmin worth downloading, installing and trying? Absolutely – it offers excellent features, and the price (none) can’t be beat. Is it worth deploying for your technical support staff or customers? It depends on whether you’re willing to accept its limitations (and potential security or system integrity risks).

Nonetheless, Webmin has tremendous potential to provide a great web interface for Unix control. If your needs match its strengths and you aren’t too concerned about its weaknesses, then it’s something you should add to your administration arsenal right away. Even if it doesn’t meet your needs now, it certainly is a tool worth watching for the future.