The Labs.Com OS Lab UNIX Plus

The Labs - Design & Functionality For The Net

UNIX Plus: Next Generation UNIX

UNIX Plus is a brainstorming about improving command-line environment of UNIX.

1. Introduction
2. Undo
3. Virtual Filesystems
4. Breaking Through Hard-Limits
5. The Filesystem
6. System Level
7. Package System
8. Cosmetic Considerations
9. And Beyond ...

I'm a firm believer in command-line and batch-mode/scripting environment. Point'n'Click I like when I draw a graphic or when I surf the web - but within the daily work of programming, maintainance and system-admin I prefere the command-line interface.

Now, this doesn't mean the current implementation of the shells (sh, tcsh, bash, etc) are the last answer for the command-line world. I think it's getting time to make some significant improvements.

The primary intention is to build another layer on an existing UNIX environment to create UNIX Plus, without touching the kernel or deep details of the OS. The basis of this undertake to build UNIX Plus is FreeBSD (but it may also work then on Linux and *BSD).

Since 2003 when I started this "brain storming" Apple seems to have alike ideas, and their MacOS-X has meanwhile implemented many of the ideas as listed below - which is really nice to see. It seems I'm not alone pushing the features of UNIX environment further.

UNIX Plus 2. Undo

One of the main features I want in UNIX is an undo command, which applies to all actions I might have done before, unlimited undos on:

• file-manipulation commands such as RM, RMDIR, CP, CHMOD, LN, MKDIR, MV, TAR, DD, etc - bring the filesystem back to the previous state before the command was launched.
• process related commands: KILL (bring program back to state before it received signal), undo started program (alike kill),
• any other command which can be undone, like DATE-set, MOUNT, CD, etc.

Apple MacOS-X: Time Machine Everything backuped, any file-change is monitored	EXT3COW.com ext3 extension for LINUX: have any file available in all past versions (ala Apple's "Time Machine")	UndoRM Package we host undo rm
PPT (Perl Power Tools) UNIX standard utilities written in perl - might be a good base to replace C-based tools

UNIX Plus 3. Virtual Filesystems

A few projects have been developed to support virtual filesystem, the idea would be to have:

• FTP (ftp://)
• HTTP/WWW (http://)
• SSH (ssh://)
• NNTP (news://)
• CVS (cvs://)
• SQL (sql://)
• ZIP, TAR, TGZ (zip://, tar://)
• FREENET (freenet://)
• GNUTELLA (gnutella://)
• BITTORRENT (bt://)

interfaced as file-system, like

% cd /http/www.cnn.com/

% ls

which downloads index.html and associated images.

Mounts URL (mount_url)

% mount -t url ftp://anonymous:something@ftp.mozilla.org /ftp/ftp.mozilla.org   % cd /ftp/ftp.mozilla.org   % ls  or even more simple:  % cd ftp://anonymous:something@ftp.mozilla.org   % pwd   ftp://ftp.mozilla.org/   % ls -l  Technical implementation: Easiest way I thought would be to write a perl NFS-server, which pretends to be a remote machine but actually is a perl-program, in which several protocol can be added (even on the fly); downside: using socket, might be slow for local-fs, but not necessarly for network based fs (e.g. ftp or so); upside: extremely flexible, doesn't need any special kernel module (would work actually on all system where perl runs). But since there is a port of FUSE for FreeBSD I'm planning to make a perl interface for it in order to be more flexible than with C or C++.

Files via URL

Additional, supporting all URL whenever (!!) a file is opened, no distinction if it's locally, remotely, or whatever; a file is a file, whatever protocol we need to retrieve it:  % grep -i "alien invasion" http://www.cnn.com/   % cat http://slashdot.org | grep -i "microsoft adopts linux kernel"  Technical implementation: an easy way would be to use LD_PRELOAD or alike to override open(), read(), write(), and close().

To bring both approaches together, amd (auto mounter) might be used.

Links

FUSE4BSD Userland Filesystem for FreeBSD LUFS (Linux Userland File-System) Linux module for virtual fs AVF (A Virtual File-System) Linux layer for userland fs PerlFS (dead-link) Provides HTTPFS, FTPFS and custom-build FS via perl-code (Linux only) FTPFS Linux module to access FTP sites as mount NFS (Network File-System) RFC's on NFS C.B. Brown's FS page Great overview with links

UNIX Plus 4. Breaking Through Hard-Limits

There are several hard-limits which are not known very much, which require to be opened:

• Unlimited directories in a directory (e.g. UFS has max 65,536 dirs in a dir)
• Unlimited disk-space; the idea would be to make one name-space among several machines (unify NFS mounted disks among all machines as one filesystem), refering to Plan9 approach; attaching machines/disks on the fly (hotswap) transparently; and using P2P Filesharing protocols to extend filespace further.

UNIX Plus 5. The Filesystem

Right now the Unix Filesystem (fs) has mainly data-files aka "files", these can be binary data, ASCII text, pixels, etc, the program interprets the data. One drawback of the fs as it is now it has almost no metadata but only permissions, date of creation, modification etc are stored, but nothing else.

The file-system should support:

• history of who (user), when (time), what (program-name with args) manipulated the file; in conjunction of "Undo" feature mentioned above changes can be undone user-, time- or program-wise.
• unlimited meta-data fields should be appendable aside of the data itself. The meta-data should be always copied with the data, but when ordinary programs handle the file only the raw data is accessed. new programs permit to read, write or manipulate the meta-data.
• file resizing: data anywhere removed, data inserted at any point without special tweaks (like rewriting the whole file).
• files with meta-data (based on a certain form which is freely defineable) is queryable like a database, e.g. a file with meta-data is automatically indexed, its content, and all meta-data.
• ability to search entire disk for data-strings (binary, ascii, image-frament ^*), the entire data is indexed according data-types.

*) A special feature: you provide an image (can be resized, rotated, flipped) and search with it through all images, and where found lists them. In other words, content-wise search procedures.

Links

MySQLFS Using FUSE to make MySQL as backend of a filesystem Wikipedia: Be FS BeOS's FS, supported extended file attributes (metadata)

UNIX Plus 6. System Level

Here a few features which deal with the system usage as such:

• instant suspend/shutdown (within one second)
• instant resume (within one second)
• ability to bring system back as it was x- seconds/minutes/hours/days ago (snapshots), including all devices (very challenging)
• very fast boot (within 2 seconds)

UNIX Plus 7. Package System

A lot of different package systems are available, I personally prefere the "from source" based package systems like Gentoo or the FreeBSD/NetBSD port system - yet, compilation may fail, rarely, but when it happens it's unsatifying.

Requirements of a "perfect" package system:

1. ability installing & running different versions of the same package, e.g. dedicate a directory for each application/program
2. 100% reliable (either source(-compile) or binary)
3. 100% secure, each package signed by creator/builder
4. multiplatform support (i386, sparc etc)

One of the approaches is to strictly tie every library to the application, and dedicate a directory for each application to support various version of the same application not interefering with each other; and also ensure 100% reliabity that all dependencies are satisfied since the application carries them with it.

OpenPKG.org Multi-platform RPM-based package system

Zero Install cached FS to run apps

Konvalo caching applications via CODA

UNIX Plus 8. Cosmetic Considerations

Filesystem Layout

GoboLinux has interesting ideas about getting rid of long outdated UNIX layout, e.g. removing distinction of /bin /sbin/ /usr/bin /usr/sbin etc, and bring them more consistent, but still remembering of special applications like diskless operations (/ read-only mounted). Such "cosmetic" layout change would go hand-in-hand of the "package system" where each program would be in their own dedicated directory, and libraries either sym-linked or copied.

Anti-Alias Font

Once in the window-system (with shells) all fonts must be anti-aliased, this isn't a feature but a must. X11/XFree86 still lacks of proper AA-Font support (yes, there is an extension, but why not native??? as of year 2004) Of course this means tranparency is part of the color-model (e.g. windows are created with RGB and A)

Graphical Shell

A graphical shell which permits to list files as icons, e.g. of images or postscript files etc in thumbnail size. Some attempts have been done (like ...). XMLTerm.org Mozilla as terminal PerlShell Perl based shell GoboLinux.org Alternative directory layout

UNIX Plus 9. And Beyond ...

Perl/Linux Userland programs as perl-scripts

Perlix OS written entirely in Perl

/. What's Wrong With UNIX Discussion thread, only a few posts are really addressing the problems

                                                                                                                                   

VMware

OS Lab