This document explains the low-level details of building and maintaining an application in the /usr/contributed collection. Thereis nothing too complicated or unusual in these instructions, but there are a few details that you should be aware of before you are ready to release your program.
Some of the information presented here also appears in /usr/contributed/html/contributed.html. I recommend reading the aforementioned document carefully, as it describes exactly what /usr/contributed is for.
To users, volumes appear as any other directory. However, volumes have two special characteristics that distinguish them from a typical subdirectory. The first is privileges. Each volume has its own modifiable criteria for who can look at its contents. The other is quota. Space used within the volume is limited to the quota assigned to it.
The best example of what a volume is would be your Andrew account. Your account is a volume mounted on the system as your userid. The command:
fs lq ~reports the name of your volume, how much quota you have, etc. The directory ~/OldFiles is another volume. Try performing an
fs lq ~/OldFilesand see what it says. Because OldFiles is an entirely separate volume from your account volume, its contents don't affect the quota figure in your regular account.
As a contributor, you are granted a volume in which to place the sources for your program. This volume is known as your source volume, and is mounted in an area on the system reserved for other contributed source volumes. This area is /afs/andrew/system/src/contributed.
cd to this directory and look at its contents. Each software package is stored beneath a subdirectory bearing its name. This same name is used for everything associated with your application package. You may organize its contents in any way you choose, as long as they are all related to your application.
Source volumes are mounted beneath the application-name subdirectory with a three-digit number. The source volume mounted there has the same three-digits in its volume-name. This number reflects the release level of the software. The number begins at "001" and advances by one with each new revision you make of your software. You never need to worry about changing the release level number of your software volumes. This is performed automatically by the User-Contributed Administrator.
In addition to the source volume, you are also given release volumes, one corresponding to each system-type for which you intend to compile your program. These volumes are mounted in a special area called dest. This is located in /afs/andrew/system/dest/$(SYS)/contributed, where $(SYS) represents the systype code (described below). When you compile your sources, your Makefile should install the final products into these volumes. The dest area is organized according to application names, just like the source area, with each application-subdirectory containing mountpoints to your release volume as a three-digit number. Some applications will have several of these mountpoints under their subdirectories. The mountpoint with the highest three-digit number will be the release volume containing the most recent release of the application. Commonly, old release volumes are preserved in this area in case the current release needs to be backed out.
Finally, if you request it, you may be given a common volume. This is a single volume that would be mounted with the name "common" inside every release volume. It is useful if your application uses many files that are not system-dependent but can be shared by all machine-types. Generally, however, common volumes complicate the build and release process without any significant gain (disk is cheap).
Access to these directories is controlled through protection groups. Each application comes with a protection group with a name like sysmaint:APPNAME.commanders. A protection group allows access privileges to be assigned to several users at once. Within this group are the userid's of all the users who will be developing your application. To see who has access to a directory or volume, type
fs la directory-nameat the system prompt. To display what users are within a protection group, you would use a command like
pts members sysmaint:APPNAME.commanders.
For more information on file protections and protection groups, see the online help for fs and pts.
Computing Services reserves the right to review software documentation before the package is released to campus. If it is found to be unacceptable, the release will be delayed until the documentation is made to be satisfactory. Therefore, what follows are are guidelines for how your files should be written. Further, there are a few files in /usr/contributed/lib/maintainers that should serve as examples of well written documentation:
If your software was developed at another site, it is likely that the package has man pages included with its distribution. If so, you may probably install them as your documentation with little effort on your part (you will only need to add the /usr/contributed caveat to the man pages). However, if you do you must be sure that the application behaves on Andrew precisely as described in the provided documentation. Unfortunately, this is sometimes not the case due to the unusual characteristics of our system. Another problem is man pages are commonly too technical and tersely worded, providing just a word or two describing the operation of the program. Therefore, you may need to supplement the man pages with a help file that you write yourself.
As a rule, documentation should first and foremost describe what your software package does. It should also tell what the software is for and how it is useful to the user. This information should "sell" the package to the user and encourage the use of the program. The third thing to include is a description (an example if possible) of what the software will do with some typical input. If the software uses a command interface, then some typical command sequences would be helpful. This way, users know what to expect from the program before actually running it and how to get started as well.
Of equal importance is the language of the document. The best rule to follow is to assume that the reader knows nearly nothing about Andrew. In this light, some things to look out for are buzzwords and abbreviations, such as "~cl0x" and "@sys" and "WEH5202". They might be familiar to you and me, but the new or occasional user will hesitate when they see this, and may decide right there that your application is beyond their comprehension. The reader should not have to make an effort to comprehend your help document; all the necessary information should be provided for a basic understanding of the application and its use.
Once this understanding has been established, you may go ahead and describe the details of your program's operation with more technical language as necessary. The important thing is that you sell your product to whomever may be reading your help file, be they occasional users or even administrators!
Below is a list of software packages that are designed to help with software compilation on Andrew.
When installing binaries and executable libraries, you may want to use the -s switch to install. This causes the file to be stripped as it is copied to the destination directory, which saves space on the system. Note that modern systems have more than enough disk space to handle the extra symbols, which can be very useful if you ever need to run a debugger on the program. Thus, stripping binaries is not recommended.
When installing non-executable files such as text, help, or man files, the -m 644 flag causes install to change the protection bits appropriately during the copy. For more detailed information about this useful program, see the online help pages on install.
NOTE: To get your application out for all machine-types on campus, you need to login and build the software on one of each machine-type. If you have difficulty finding the right kind of workstation or are not sure which ones are appropriate, ask the User-Contributed Administrator, there are "washing machines" available to collection maintainers for all system types. Once this has been accomplished successfully, you should try to test your program as best you can. When you are satisfied that it runs correctly, you would run carpe, requesting a software release.
To begin, we must introduce the idea of links. On workstations, /usr/contributed program binaries are found under the directory /usr/contributed/bin. Similarly, libraries are in /usr/contributed/lib, man pages in /usr/contributed/man, and so on. However, these areas are not where the actual contents are stored. Instead, they are symbolic links to an area /usr/contributed/depot where volumes are mounted according to the name of the collection.
The directory /usr/contributed is also a link. On cluster workstations, it points to the directory:
/afs/andrew.cmu.edu/$(SYS)/contributedwhose contents are exactly the same as:
/afs/andrew.cmu.edu/system/$ENV/$SYS/contributed
"$ENV" is one of two designations: beta, and gamma. These directories are independent software trees or environments. Each one serves a different purpose.
Tree Remarks ================================================== beta software in beta test, pre-release testing gamma software released to campusThus, under the directory /afs/andrew.cmu.edu/system/, there are two subdirectories (beta, gamma), representing older and newer software. And beneath each of these subdirectories are subdirectories for each system model. To developers, these two directories may be known as release areas.
On campus, the release area that /usr/contributed programs actually come from is:
/afs/andrew.cmu.edu/system/gamma/systype/contributedUnderneath this directory are the typical Unix system directories, such as lib, bin, and etc, but also one special directory: depot. Under the Depot system, all the files associated with applications are under the depot subdirectory. No real files exist in any of the other directories, bin, lib, etc. Instead, these directories contain symbolic links to the appropriate files underneath depot. This special subdirectory contains directories beneath it containing all the libraries, binaries, help files, man pages, etc. organized according to application.
Therefore, the program myprog will actually sit in the directory /afs/andrew.cmu.edu/system/gamma/$(SYS)/contributed/depot/myprog/bin/myprog, but users will find it within /afs/andrew.cmu.edu/system/gamma/$(SYS)/contributed/bin/myprog, which appears on workstations to be /usr/contributed/bin/myprog because of yet another symbolic link that is set up on individual Andrew workstations.
The public unix servers are gamma machines. The "washing machines" available for maintainers are beta machines, though they often have pre-released software depoted into them that is being tested before release to beta.
/afs/andrew.cmu.edu/system/dest/$(SYS)/contributed/APPNAME/release-numberwhere APPNAME is the name of your application and "release-number" is the special number reflecting the number of releases your application has experienced. If this is the first time your application has been part of the User-Contributed environment, the number will be 001.
Maintainers do not place their software packages in the release areas. This task is performed by the User-Contributed Administrator. When your software is ready in the dest area, the Administrator will mount the appropriate volumes into the correct release area under the depot directory.
Your files and directories will be linked to the /usr/contributed area as they appear at the top level of your release volume. This linking is done by an automatic routine called depot. and is performed by the User-Contributed Administrator. However, it is up to the maintainer (you) to make sure that your volume is correctly structured to be linked. By default, depot links files as they appear relative to the top-most directory of your release volume to the top-most directory of /usr/contributed. For example, if the top level of your release volume contained:
%ls -F bin/ lib/ man/depot would default to linking all the files beneath bin/ into /usr/contributed/bin and all the files beneath lib/ into usr/contributed/lib. man/ is special, because by convention it contains subdirectories man1/, man2/, etc. conforming to Unix documentation standards. You would create under /man whatever subdirectories you needed, such as man1/ for docs on executables, manx/ for docs on X-window applications, and so forth. Depot will link the contents of these subdirectories into the correct place under /usr/contributed/man.
Thus, if lib/ contained the file librle.a, and the directory mylibs/, then the links would appear, by default, as usr/contributed/librle.a and /usr/contributed/mylibs/. The choice of how it should appear, as a directory or a file, is up to you.
For customizations such as telling depot to move link the information in the common volume as if it was in the dest volume, you may create a depot.conf file. For more information on what a depot.conf file may do, consult the documentation on depot.conf.
If you have libraries or include files that are unique to your software, then you may consider collecting them. This occurs when you place lib files not in myprog/011/lib but rather myprog/011/lib/myprog by modifying your makefile a little. The depot utility by default will link the entire directory .../lib/myprog as ...contributed/lib/myprog and all your libraries would appear underneath. This makes the /usr/contributed/lib/ directory less cluttered and organizes your application more efficiently. If you are not sure whether this is useful to you, contact the User-Contributed Administrator.
The beta environment is reserved for more in-depth and long-term testing of software. It is the environment run by workstations with special links to the area (such as the washing machines). Many staff members in Computing Services depend on this area. The gamma environment, again, is what the campus at-large relies on.
When software is released, the completed release volumes are mounted in the gamma environment. Thus, the release volume migrates from the beta environment to the user environment through the release procedure.
One difference between the beta and gamma enviornements is that the released volumes in the beta enviornment are not replicated. That is, if the AFS server containing the files for a beta-enviornment collection crashes, that collection will be unavailable until the AFS server is restored. The gamma enviornment collections, however, are replicated across several AFS servers (and thus are not vulnuerable to this particular failure mode).
All movement between environments is the responsibility of the User-Contributed Administrator. If bugs are found in the program after a release, the administrator may back-out the software, removing it from the campus environment. A back-out may also be requested by the software maintainer if major problems are discovered. Often it is just as good to immediately release a new version of the collection with whatever necessary fixes applied.
When your software is released (even to beta), the volumes containing it are frozen. All access privileges to write and modify the volumes released to campus are taken away. This is to insure that something does not happen to the software after it has been released. In general, software cannot be changed after it has been released. In the event that the maintainer wishes to make a change, new volumes are created, as described below.
One of the advantages the depot system provides is the ease at which administrators can release and back-out software. Of course, it is intended that the back-out is used only as a last resort, and should not be relied on as a license to put out unstable software. Therefore, I cannot emphasize enough the importance of checking and double-checking your work. The performance of your software reflects on both you and the user-contributed software as a whole.
You request an update release just as you would a first-time release, except that when your new release volumes are mounted in gamma and beta, the volumes that are being replaced are no longer used.
The three digits at the end of the volume-name signify this is the first release of the package myprog. If a new version were released with a minor change or new feature, then its new volume would have the suffix ".012". The suffix reflects the progression of new releases the software has undergone in ascending order.
Volumes are always mounted according to their release numbers. In the dest area, there may be many release volumes.mounted under an application name to reflect its revision history. Typically, however, there will be only one source volume. Its release number will change automatically with each revision to reflect the most recent software version.
Another thing to be prepared for is questions from users. It is up to you to document the software, and respond to problems and bug-reports. Generally, the feedback you get will be positive and often offers ways to make your application better.