2008/07/25

The Labs.Com OS Lab InterOS InterOS Example
Last update 2001/03/03
The Labs - Design & Functionality For The Net

InterOS Example - A Real Life Application

This is one of our setup of an InterOS (or multi-OS) environment. Many people do it, but it's only a few times really documented for beginners and newbies.
  1. Overview
  2. PPro 200
  3. 'Deity' (server)
  4. K6-2/500 'ProfX'
  5. K6-2/500
  6. 'Cerise'
  7. K6-2/500
  8. 'Magneto'
  9. K6-2/500
  10. 'Storm'
  11. Rise mP6-266
  12. 'Psy' & 'Siena'
  13. Celeron 700
  14. 'Husk'
  15. Athlon 1000
  16. 'Colossus'
  17. Athlon 1000
  18. 'Elektra'
  19. Celeron 500
  20. 'Polaris'
  21. X11
InterOS Example
1. Overview
While testing several OS interdependencies we setup following small network with a few machines in which Linux, FreeBSD, NetBSD, OpenBSD, WinXX, BeOS, MacOS and QNX work together. It is part of our HomeNetwork-project.

On the workstation I don't test new software at all, just keep track of the FreeBSD-stable (kernel + world). The server isn't upgraded very frequently either, right now Linux-SuSE runs there but I will switch at the next upgrade to FreeBSD. On the lab-machines as I call them I do the heavy testing, new operating systems, multi-os boot, VMware-machines, and testing experimental kernel upgrades.

  • deity: main-server: nfs-server, dns-server, samba, network-address-translation (firewall), internet connection
  • profx: win98 & linux with video-capture card
  • cerise: freebsd cvsup machine
  • storm: linux.debian machine, if required 2nd win98 or freebsd, testing linux-kernels/modules along with vmware-virtual-machines, and uml-virtual-machines
  • magneto: linux.debian machine mainly (netbsd & freebsd also) for vmware-virtual-machines and uml-virtual-machines
  • husk: netbsd, openbsd, freebsd and linux.debian, but also used diskless (remote boot)
  • colossus: freebsd primary workstation (gimp, picart, etc)
  • elektra: freebsd secondary workstation, same as colossus but as backup, if required running linux.debian or win98

InterOS Example
2. PPro 200: 'Deity' (server)

Hardware
 PPro 200, 256MB EDO RAM, 90GB UWSCSI (various disks)

NAT
 NAT (Network Address Translation) or also known as IP-masquarading brings you the ability to have a local-network in which each machine can access the Internet too (via the server used as gateway). Depending on your UNIX OS (Linux, or *BSD) there are several approaches available, the setup with Linux-SuSE was fairly simple with SuSEfirewall.conf files (well documented).

DNS
 I like to give each machine a name, and therefore prefere static IP-assignment. If you have 50+ clients, then DHCP certainly is a better choice, but I personaly still would like to know their IP and give them alias or names to each machine (that's one of my manners).

We use our DNSTool to maintain our small network. We also run several virtual web-sites (VirtualHost in apache) while developing web-sites for customers of us, so we assign unique IPs (192.168.0.x) and give them names too (most times the same as their domain without .org/.com/.net appendix).

NFS
 We enabled on the server nfs and export the / (root) to all clients. One thing to consider are consistant User-ID aka UID (defined in /etc/passwd) so each client and server can remote login into each other (best create .rhosts so it can be done without password) can work there and all UID are consistant. This enables you have one location where your data are (server-client principe this is) and several machines (clients) you can do the computation.

For consistancy each client has also an non-nfs exported user/home-dir which is called the same as the client itself, this is one of the habits we have.

/etc/exports/

 # See exports(5) for a description. 
 # This file contains a list of all directories exported to other computers. 
 # It is used by rpc.nfsd and rpc.mountd. 
 / 192.168.0.32(rw) 
 / 192.168.0.33(rw) 

or something like this:

 / 192.168.0.255/255.255.255.0(rw) 

and then on each client has this line in /etc/fstab

 deity:/         /mnt/nfs     nfs     rw      0 0 

and then

 cd /home/ 
 ln -s /mnt/nfs/home/myuser 

then you have /home/myuser on the server nfs-mounted (valid /etc/passwd and /etc/groups assumed)

Samba
 For the WinXX world we run samba and works well. One recommendation, do not give WinXX write access to any significant part of your server. On WinXX virus and trojan horses are part of life (unlike in the UNIX world) and you might have a virus deleting all files on your WinXX machine, and if you have a samba-partition on your server too, then in the worst case you might loose that part too, so never export root (/) as samba, or if you do, then read-only.

Here an /etc/smb.conf excerpt, whereas the workgroup is defined there like

 workgroup = your-workgroup-id 

which you use under Win98: Control-Panel -> Networks -> Identification and there under "Workgroup:" set the same.

InterOS Example
3. K6-2/500 'ProfX'

Hardware
 K6-2/500, 128MB PC-133 RAM, 20GB IDE, Shuttle-599 (on-board sound & video), connected to KVM switch (no dedicated KB, Mouse, Monitor), it additionally has a TV-card which allows to stream MPEG-I over the network (and store, replay etc).

Win98
 Since Micros*ft has such a cooperative OS you must install it first on your machine, because otherwise it will trash your other OS's which you might have on your harddisk already without even telling you.

  • BeOS:
    • Install: Just download the BeOS PE 5 (Personal Edition) from free.beos.com.
    • Usage: Within WinXX BeOS is easy to install and also to start with one click. For system-setup check our BeOS-section. We use the server as gateway (to the Internet), DNS-server, NFS-server (for other UNIXes) and SAMBA-server (for WinXX world).

  • MacOS:
    • Install: There is an emulator called Basilisk II which runs on Linux, WinXX and also BeOS. The installation on a WinXX seems easiest, check our MacOS-section.
    • Usage: With MacTCP/IP you can assign a destinct IP (as WinXX has host is still active) and access your local-network, and via your gateway also the Internet.

  • QNX: We just downloaded the demo-disk from QNX.Com which provides on a 1.44MB floppy-disk image an GUI with browser and few graphic demos, very impressive (check also our Embedded-section).

Linux (SuSE)
 Currently runs SuSE-distribution of Linux but will install Linux.Debian soon. With VMWare you can run various virtual machines (with their own BIOS, Memory, Screen and Disks) several other OS's, yet there are limited support of sharable devices of your host (in our case Linux). But it certainly it a great program to test installations and working within an InterOS environment additionally.

We use LILO to boot WinXX, Linux and FreeBSD, here our /etc/lilo.conf.

Three Virtual Machines run on ProfX using VMWare with Linux as host-OS. Each machine runs as a vmware-virtual-machine and all three (or even more) appear as independent machines to the network; each imports / from the server (rw), with two logins (root & kiwi).

We run on all clients (profx, cerise, ..) a localhost web-site, which is developed using our ProgrammerHTML-handler which allows us to write and develop one site (exported via nfs) but accessed via several machines (which all run their own apache+mod_perl) to appear with unique 'uname -a' output: (screenshots follow).

This allows us additionally to use our mod_perl handlers under different OSes and CPU power.

InterOS Example
4. K6-2/500: 'Cerise'

Hardware
 K6-2/500, 256MB PC-133 RAM, 20GB IDE, Shuttle-599 (on-board sound & video), with dedicated KB, mouse and 21" Sony G-500 monitor: my workstation.

FreeBSD
 This is our mirror system of a production server (which runs an Athlon with 256MB and 19GB 10K RAID0 disk) in which we test upgrades of our FreeBSD-STABLE parallel. More info on FreeBSD.

Additionally we have two logins, one my usual kiwi which is nfs-exported from the server, and local which is the local-user with non-nfs home-directory.

InterOS Example
5. K6-2/500: 'Magneto'

Hardware
 K6-2/500, 128MB PC-133 RAM, 20GB IDE, Shuttle-599 (on-board sound & video), connected to KVM switch (no dedicated KB, Mouse, Monitor)

Multi-OS Boot
 Right now the machine has NetBSD, FreeBSD and Linux/Debian, this is what I did:
  1. partition the disk with FreeBSD or Linux boot (don't install neither one of them): three partitions 1) netbsd (type=a9) 2) bsd/386 (type=a5) 3) linux (type=83) (no swap)
  2. boot NetBSD-CD and put it on the first partition, install all (don't install a MBR (Master Boot Record) boot-loader)
  3. boot Linux/Debian-CD and install it, install LILO on root-filesystem (e.g. /dev/hda3) - (not on the MBR)
  4. boot FreeBSD-CD and install it, fdisk will show 2nd partition (slice), nothing to change, in disklabel you create swap and root-filesystem (/); let FreeBSD install the MBR boot-loader (selector to boot NetBSD and Linux then too)
  5. reboot and finish each OS (e.g. network, root-passwd etc)

     F1   NetBSD 
     F2   FreeBSD 
     F3   Linux 
      
     Default: F1 

  6. Finally added a file-based swap-space for the Linux (not wasting a primary or logical partition), check Linux-section) for the How-To.
I first installed NetBSD above '1024-cylinder limit' and screwed my other two OS installations, so, put NetBSD as first OS, then Linux or FreeBSD, let FreeBSD install its boot-loader, it works quite well (better than LILO).

KVM-Problems: the IOGear 8-port KVM I use causes some nasty PS/2 problems; under FreeBSD which probes the mouse-type it reports the wrong mouse-type (Linux doesn't probe the PS/2 mouse-type), and causes the psm-driver under FreeBSD not to work (works if the mouse is attached direct). To solve the problem edit the kernel-config file as follows:

 device          psm0    at atkbdc? irq 12 flags 0x200 

The flags 0x200 disables the probing of the mouse and falls back to oridinary PS/2 mouse (check man psm).

InterOS Example
6. K6-2/500: 'Storm'

Hardware
 K6-2/500, 128MB PC-133 RAM, 20GB IDE, Shuttle-599 (on-board sound & video), connected to KVM switch (no dedicated KB, Mouse, Monitor)

Multi-OS Boot
 I installed Linux, Win98 and FreeBSD on it this way:
  1. boot Linux-Debian CD, make four partitions: 1) vfat (type=0c), 2) bsd/386 (type=a5), 3) linux (type=83), 4) linux-swap (type=82)
  2. install Linux-Debian on 3rd partition (it will recognize it automatically), install LILO on root-filesystem (not on the MBR!)
  3. install Win98 from CD, it will use the first partition and format it, go through all til done (Win98 may host QNX, MacOS Emulator, BeOS 5)
  4. install FreeBSD from CD:
    • fdisk will show freebsd-partition (slice), don't alter anything
    • disklabel will show DOS partition as well, leave it untouched; add swap-space (twice your physical memory) plus root-filesystem (to keep it simple), it will put it within the 2nd partition (Note: FreeBSD partition(s) live within a slice; a slice is a partition in Linux/Windows terminology).
    • install boot-loader on MBR, it will recognize the DOS & Linux automatically:

       F1   Windows 
       F2   FreeBSD 
       F3   Linux 
        
       Default: F1 

It will take you aprx. 3hrs to install all three OSes, assuming you are experienced with all three OS installs.

InterOS Example
7. Rise mP6-266: 'Psy' & 'Siena'

Hardware
 Rise mP6-266, 32MB RAM, 12GB Notebook IDE aka IOpener, having their own KB, Mouse (replaced by an wireless kb/mouse) & 10" LCD Screen

Tripple OS Boot
 'Psy' has tripple-boot with WinXX, Linux and FreeBSD (installation procedure mentioned above with 'ProfX'-machine), whereas WinXX has then BeOS also. More info at IOpener. 'Siena' will have a linux-dist called Jailbait on it, and remains diskless therefore.

InterOS Example
8. Celeron 700: 'Husk'

Hardware
 Celeron 700, Asus-CUSI-FX (FlexATX), 128MB, 20GB IDE, on-board VGA+Sound+LAN connected to the KVM (build Jan 2001, cost ~$360).

Quadruple OS Boot
 This machine is supposed to boot diskless, yet, for test-purposes I setup a four OS boot:
  1. boot Linux.Debian CD, go direct to partition harddisk:
    1. primary partition 1: type=a9 (NetBSD)
    2. primary partition 2: type=a6 (OpenBSD)
    3. primary partition 3: type=a5 (FreeBSD)
    4. primary partition 4: type=83 (Linux)
    on my small 20GB I dedicated for each OS 5GB, also the Linux is setup without swap-partition as we create a file-based swap-file later (so we don't waste a partition for it). After partitioning do not proceed to install Linux but abort it (reboot) and ..

  2. boot NetBSD CD, leave the disk-layout as it is and when asked about 'using part of the harddisk' select this one, do not install the NetBSD boot-selector when asked (NetBSD install is a bit verbose and you need to read things carefully). Once in fdisk leave the four partition untouched, once in disklabel it provides already a suggestion of bsd-partition living within the NetBSD-partition, just accept the defaults:

     /dev/wd0a   4.2BSD     /      230MB   (root-fs, just 230MB) 
     /dev/wd0b   swap              256MB   (swap-space) 
     /dev/wd0c   unused           4800MB   (partition or slice) 
     /dev/wd0d   unused          20000MB   (entire disk) 
     /dev/wd0e   4.2BSD     /usr  4500MB   (/usr/ fs, the real data) 

    Partition c and d are actually just containers, not direct usuable, so partition b is reserved for swap, this leaves a and e, f, g etc for data-partitions, this is one of the BSD-conformity. The rest of the installation is easy and done fast.

  3. boot OpenBSD CD, leave the disk-layout of wd0 (partitions) as it is, once in disklabel it will give four partitions as default (type 'p' for print):

     /dev/wd0a   4.2BSD     /      230MB   (root-fs, just 230MB) 
     /dev/wd0b   swap              256MB   (swap-space) 
     /dev/wd0c   unused           4800MB   (partition or slice) 
     /dev/wd0d   unused          20000MB   (entire disk) 
     /dev/wd0e   4.2BSD     /usr  4500MB   (/usr/ fs, the real data) 

    Same BSD conformity as mentioned as above for NetBSD. As of OpenBSD 2.7 this partition procedure is cumbersome (definitly read OpenBSD: Installation Guide), if you make one little mistake you have to start-over to disklabel (before it goes again into fdisk even this is not what you want); so pay close attention what you enter, the partitioning is anything than intuitive. After the partitioning the rest is easy, configuring network and installing the packages and you are quickly done.

  4. boot Linux.Debian CD, proceed as you know it just make sure you do write the LILO into /dev/hdax (x = 1,2,3 or 4) (but not on MBR).

  5. boot FreeBSD CD, I usually choose the express installation, once in fdisk leave all untouched, the moment you are in disklabel create your swap & root-fs (/), and then force FreeBSD to install the boot-selector on MBR (Master-Boot-Record). The FreeBSD layout:

     /dev/ad0s3a   4.2BSD      /     4800MB   (root-fs) 
     /dev/ad0s3b   swap               256MB   (swap-space) 
     /dev/ad0s3c   unused            5000MB   (entire slice) 

    The same disk-layout conformity as mentioned above for NetBSD and OpenBSD, the ad0 is the first IDE disk, s3 the third slice (third partition of four).

  6. last step is to create swap-file for Linux, check Linux-section how to do. Once all four OS'es boot fully, I configured the network and nfs imports and added users and so forth.

    I first installed (against my earlier insights about the 1024-cylinder limit) OpenBSD first, then NetBSD and it didn't work (FreeBSD and Linux already installed). I switched then and put NetBSD on the first partition and then OpenBSD, yet, the OpenBSD refused to boot then (via F2 from the FreeBSD boot-selector). The solution was to boot again OpenBSD CD and install all, but before rebooting running:

     cd /usr/mdec/ 
     ./installboot boot biosboot /dev/rwd0c 

    and then started FreeBSD install again writing back partition-table (even nothing was changed) with fdisk and writing boot-selector and then aborted the install, which resolved it.

    My FreeBSD boot-selector looks like this now:

     F1  NetBSD 
     F2  OpenBSD 
     F3  FreeBSD 
     F4  Linux 
      
     Default: F1 

    The disk-layout of this installation:

Note: As of Feb 2001 the SIS-630E chipset is best supported by Linux-2.4.* (2.2.* doesn't recognize SIS-900 on-board ethernet), FreeBSD-4.2-RELEASE does well but is slower at booting (IDE probing disk & fd takes too long) and doesn't query the MAC address of the ethernet correctly (gives 00:00:00:00:00:00 instead), NetBSD and OpenBSD seem behind and on-board ethernet isn't recognized at all.

InterOS Example
9. Athlon 1000: 'Colossus'

Hardware
 Athlon 1000, Asus-A7Pro, 768 MB, Matrox G450-DH AGP, 2x 45GB IDE, the work-station (build Dec 2000, cost ~$1400). No kb, mouse or screen required as is connected to KVM.

OS
 Latest FreeBSD-STABLE, UFS (FFS) supports large files (>2GB) by default and suits well for backuping the rest of the system on the 90GB. The UDMA-66 also is quite fast compared to the older U2W SCSI to my great surprise. Currently (Jan 2001) this machine is the main workstation replacing the 'Cerise' (K6/2-500) as workstation, yet, 'Cerise' remains the main FreeBSD machine which cvsup's the kernel and world-sources.

X11
 Setting up XFree86 is still a hassle (Jan 2001) as most tools simply don't create valid or working /etc/{X11/}XF86Config files, setting up any machine with Linux or FreeBSD the XFree86 didn't work without editing the XF86Config by hand.

In order to be able to move XF86Config between Linux and *BSD machines, I use /dev/mouse as mouse-device, and if required create sym-links (e.g. Linux: /dev/mouse -> /dev/psaux, FreeBSD: /dev/mouse -> /dev/psm0); and since 'Colossus' works as main workstation I attached the wacom-tablet to it as well (FreeBSD: /dev/wacom -> /dev/cuaa0), check GIMP-section for details for the gtk/gimp compile setup.

I downloaded the mga_drv.o from Matrox.Com and copied into /usr/X11R6/lib/modules/drivers/mga_drv.o and it worked just fine then. I first run xf86cfg and then hand-edited the config-file, here my XF86Config for XFree86-4.0.2 with Wacom-Tablet attached for Sony G-500 21" with 2048x1536 (110kHz @ 70Hz). Once I run GIMP I went under 'File' -> 'Dialogs' -> 'Input Devices' and then enabled 'Stylus0' and 'Eraser0' to 'Screen' and saved it; after this the tablet works with GIMP.

The A7Pro is a down-scaled A7V Asus-board, instead UDMA-100 just UDMA-66, the BIOS setup is really handy compared to the older BIOS on the Shuttle K6/2 board.

InterOS Example
10. Athlon 1000: 'Elektra'

Hardware
 Athlon 1000, Asus-A7Pro, 384 MB, Voodoo3/3000 PCI, 45GB IDE (build Jan 2001, cost ~$800) No kb, mouse or screen required as is connected to KVM.

Tripple OS Boot
 It's a Win98, FreeBSD and Linux/Debian install (described above already), it mainly runs Linux/Debian:
  • VMware-machines: *BSD, Linux-* and Win-XX virtual machines
  • UserModeLinux: virtual linux-boxes running on Linux
  • Win4Lin: Win-XX emulator for Linux (faster than VMware + Win98)
Later when Plex86 is faster will host 'plexed' virtual-machines as well. The 'Elektra' is mainly used to provide Linux-based virtual machines a solid host-basis.

X11
 Here 1920x1440 mode for Voodoo3/3000 for 21" Sony-G500. I used kvideogen to create Modeline for the XF86Config, highly recommended:

 Modeline "1920x1440" 259.83 1920 1936 2384 2496 1440 1440 1456 1504 

I started to acquire from each machine-type at least two which simplifies tracing possible problems with single components by switching them arround (in case of 'Elektra' occured and problem could be located).

InterOS Example
11. Celeron 500: 'Polaris'

Hardware
 Celeron 500 (will be upgraded to 700), 64MB, 15" LCD, 10GB IDE, 10/100 NIC, (Acer Veriton C500A, purchased Feb 2001, cost $950), nice all-in-one machine


Tripple OS Boot
 I run SuSE-7.1 install until the partitioning, it allowed me to shrink the Win98 partition, then I created three new primary partitions: BSD/386 (type=a5) and Linux (type=83) and Linux-Swap (type=82). Then I aborted SuSE-7.1 install and inserted the Debian-2.2 CD and went ahead and installed it. Finally installed FreeBSD-4.1-RELEASE and then to updated to -STABLE via cvsup. Unfortunatly the FreeBSD bootmanager doesn't work here, before I spent too much time I reinstalled LILO via Debian-2.2 CD install to boot Win98, FreeBSD and Debian/Linux. On Win98 I installed BasiliskII (MacOS Emulator) & BeOS-5.

InterOS Example
12. X11

Most machines run X11 (or WinaXe under WinXX); for the XFree86-based X11 machines I use x2x: it's a small program which allows to connect two X-servers (-form source-X11 with -to dest-X11) together:

 colossus% x2x -from colossus:0.0 -to polaris:0.0 -west -resurface 
 colossus% x2x -from colossus:0.0 -to storm:0.0 -east -resurface 

This way you can control from the keyboard & mouse of colossus (in my setup) two other screens on the left (west) and one on the right (east); under XFree86-3.3.x running two x2x caused problems, under XFree86-4.x this seems resolved.

Since I run AfterStep-Classic and its Pager (superior to anything I have seen to far), the other two screens aside need EdgeResistance 0 0 (turned off) otherwise x2x causes fast page-changes; the main-screen still has it enabled (100 100).

X2X (digital.com)
Source-code
X2VNC
X2X for VNC, homepage & sources

                                                                                                                                   

InterOS

Last update 2001/03/03

All Rights Reserved - (C) 1997 - 2008 by The Labs.Com

Top of Page

The Labs.Com