Directory Conventions: Complete Version
From DLXS Documentation
(Difference between revisions)
m (→Conventions for Content and Middleware) |
m (Replaced "%lt;" with "<" in <prefix>/img/c/collection section) |
||
| (4 intermediate revisions not shown.) | |||
| Line 24: | Line 24: | ||
The basic scheme is as follows: | The basic scheme is as follows: | ||
| - | {| | + | {|cellpadding="20" cellspacing="0" border="1" |
|<code><prefix>/bin/c/class</code> | |<code><prefix>/bin/c/class</code> | ||
| Line 46: | Line 46: | ||
|continuous tone image data | |continuous tone image data | ||
* non-web-ready (archival) images, to be delivered through middleware; i.e., like image services | * non-web-ready (archival) images, to be delivered through middleware; i.e., like image services | ||
| - | * web-ready (GIF, JPEG, PNG) images which are licensed content but to be delivered directly; to make this work, there is typically a symbolic link from | + | * web-ready (GIF, JPEG, PNG) images which are licensed content but to be delivered directly; to make this work, there is typically a symbolic link from <prefix>/web; see below |
|- | |- | ||
|<code><prefix>/idx/c/collection</code> | |<code><prefix>/idx/c/collection</code> | ||
| Line 63: | Line 63: | ||
or | or | ||
| - | <prefix>/misc/[c/collection/] | + | <code><prefix>/misc/[c/collection/]</code> |
| - | + | |additional files for DLXS distribution that don't belong elsewhere: e.g., DTDs, character entity sets, catalogs, etc. | |
| - | additional files for DLXS distribution that don't belong elsewhere: e.g., DTDs, character entity sets, catalogs, etc. | + | * optionally by class or collection, if appropriate |
| - | optionally by class or collection, if appropriate | + | |- |
| - | <prefix>/misc/db | + | |<code><prefix>/misc/db</code> |
| - | + | |text-based databases: e.g., DLXS sessions database in CSV format | |
| - | text-based databases: e.g., DLXS sessions database in CSV format | + | |- |
| - | <prefix>/misc/[c/class/]maps | + | |<code><prefix>/misc/[c/class/]maps</code> |
or | or | ||
<prefix>/misc/[c/collection/]maps | <prefix>/misc/[c/collection/]maps | ||
| - | + | |region map files | |
| - | region map files | + | * optionally by class or collection, if appropriate |
| - | optionally by class or collection, if appropriate | + | |- |
| - | <prefix>/obj/c/collection production content organized by collection identifier | + | |<code><prefix>/obj/c/collection</code> |
| - | .raw for "raw" (i.e., unprocessed OCR) text | + | | production content organized by collection identifier |
| - | .sgm for "cooked" (i.e., proofed and encoded) text | + | * .raw for "raw" (i.e., unprocessed OCR) text |
| - | .tif for page images, etc. | + | * .sgm for "cooked" (i.e., proofed and encoded) text |
| - | <prefix>/obj/d/l/p/dlpsid.vvvv.iii production content organized by DLPS identifier | + | * .tif for page images, etc. |
| - | same file naming conventions as above | + | |- |
| - | <prefix>/prep/c/collection pre-production (munge-in-process) content organized by collection identifier | + | |<code><prefix>/obj/d/l/p/dlpsid.vvvv.iii</code> |
| - | same file naming conventions as above | + | |production content organized by DLPS identifier |
| - | typically on the staging server, not production servers | + | * same file naming conventions as above |
| - | <prefix>/prep/d/l/p/dlpsid.vvvv.iii pre-production (munge-in-process) content organized by DLPS identifier | + | |- |
| - | same file naming conventions as above | + | |<code><prefix>/prep/c/collection</code> |
| - | typically on the staging server, not production servers | + | |pre-production (munge-in-process) content organized by collection identifier |
| - | <prefix>/web/c/collection static HTML pages | + | * same file naming conventions as above |
| - | <prefix>/web/c/class/graphics | + | * typically on the staging server, not production servers |
| + | |- | ||
| + | |<code><prefix>/prep/d/l/p/dlpsid.vvvv.iii</code> | ||
| + | |pre-production (munge-in-process) content organized by DLPS identifier | ||
| + | * same file naming conventions as above | ||
| + | * typically on the staging server, not production servers | ||
| + | |- | ||
| + | |<code><prefix>/web/c/collection</code> | ||
| + | |static HTML pages | ||
| + | |- | ||
| + | |<code><prefix>/web/c/class/graphics</code> | ||
or | or | ||
<prefix>/web/c/collection/graphics | <prefix>/web/c/collection/graphics | ||
| - | + | |user interface glitter: banners, icons, logos, etc. | |
| - | user interface glitter: banners, icons, logos, etc. | + | * optionally by class or collection, if appropriate |
| - | optionally by class or collection, if appropriate | + | |- |
| - | <prefix>/web/c/class/images | + | |<code><prefix>/web/c/class/images</code> |
or | or | ||
| - | <prefix>/web/c/collection/images | + | <code><prefix>/web/c/collection/images</code> |
| - | + | |web-ready image content served directly, usually figures that appear in text collections | |
| - | web-ready image content served directly, usually figures that appear in text collections | + | |
if image content is licensed, files are stored in <prefix>/img (see above) and this is just a symbolic link | if image content is licensed, files are stored in <prefix>/img (see above) and this is just a symbolic link | ||
| + | * optionally by class or collection, if appropriate | ||
| + | |} | ||
| - | + | <code><prefix></code> varies from installation to installation. All DLPS servers use at least the prefix <code>/l1</code>; most have additional file systems named /l2, /l3, etc. | |
| - | <prefix> varies from installation to installation. All DLPS servers use at least the prefix /l1; most have additional file systems named /l2, /l3, etc. | + | |
| - | Note that although classes and parts of the organization can also exist at this level (for example, h/hti for the Humanities Text Initiative or i/is for Image Services), the individual collections would exist within their own spaces, i.e., <prefix>/.../m/musart. | + | Note that although classes and parts of the organization can also exist at this level (for example, <code>h/hti</code> for the Humanities Text Initiative or <code>i/is</code> for Image Services), the individual collections would exist within their own spaces, i.e., <code><prefix>/.../m/musart</code>. |
| - | Personal directories can also exist at this level, i.e., <prefix>/cgi/c/csnavely, for areas to do testing in. | + | Personal directories can also exist at this level, i.e., <code><prefix>/cgi/c/csnavely</code>, for areas to do testing in. |
| - | The organization of files below the levels described here is outside the scope of this convention. We don't feel a need to prescribe a structure for inside /l1/web/b/bas, for example. Of course, consistency between like things is good practice. | + | The organization of files below the levels described here is outside the scope of this convention. We don't feel a need to prescribe a structure for inside <code>/l1/web/b/bas</code>, for example. Of course, consistency between like things is good practice. |
| - | Conventions for Software Packages | + | ==Conventions for Software Packages== |
| - | Basic Convention | + | ===Basic Convention=== |
The DLPS institutional convention is that software should be installed as | The DLPS institutional convention is that software should be installed as | ||
| - | /l/local/package-version | + | <code>/l/local/package-version</code> |
where | where | ||
| Line 133: | Line 143: | ||
Likewise, source code can be found in | Likewise, source code can be found in | ||
| - | /l/local/src/package-version | + | <code>/l/local/src/package-version</code> |
or compressed up in a tarball named | or compressed up in a tarball named | ||
| - | /l/local/src/package-version.tar.gz | + | <code>/l/local/src/package-version.tar.gz</code> |
| - | Special Cases | + | ===Special Cases=== |
For cases where a version-independent path is needed or wanted, a symbolic link should be created: | For cases where a version-independent path is needed or wanted, a symbolic link should be created: | ||
| - | /l/local/package -> /l/local/package-version | + | <code>/l/local/package -> /l/local/package-version</code> |
For commonly-used utilities that are conventionally installed in places like /usr/local/bin, a symbolic link should be created in /l/local/bin: | For commonly-used utilities that are conventionally installed in places like /usr/local/bin, a symbolic link should be created in /l/local/bin: | ||
| - | /l/local/bin/command -> /l/local/package/[subpath/]command | + | <code>/l/local/bin/command -> /l/local/package/[subpath/]command</code> |
if present, or | if present, or | ||
| - | /l/local/bin/command -> /l/local/package-version/[subpath/]command | + | <code>/l/local/bin/command -> /l/local/package-version/[subpath/]command</code> |
| - | Data Flow Describing the Preparation and Release of a Collection | + | ==Data Flow Describing the Preparation and Release of a Collection== |
Looking at the conventions from a "process" point of view, the following diagram illustrates how we use the directory conventions in the workflow of preparing and releasing a collection. | Looking at the conventions from a "process" point of view, the following diagram illustrates how we use the directory conventions in the workflow of preparing and releasing a collection. | ||
| + | |||
| + | [[Image:dirstructuredfd_suz2.png]] | ||
Current revision
Contents |
[edit] Rationale
Multiply the amount of data you have by the number of people dealing with it and you get a metric of complexity indicating how badly you need conventions. Having some basic directory conventions solves some general problems:
- things are easier to find (e.g., for DLXS distributions)
- things are easier to move
- it's easier to tell when you have found the thing you're looking for (i.e., if it's in the right place, it generally must be the authoritative version)
With respect to content and middleware, a good convention:
- protects data (e.g. XML, images), indexes, and CGI middleware from being accidentally served to a client by the web server
- protects static HTML and XML from CGI-related security holes
- insulates the web server software from the content and middleware, which makes web server upgrades easier
From a system administration point of view, a good convention:
- provides a structure that's pretty easy to traverse and can scale up without getting leggy
- allows for the coexistence of different classes and collections on a single server
[edit] Conventions for Content and Middleware
Content should be organized by DLPS identifier or, if that's unavailable, collection identifier. Middleware and everything else should be organized by collection identifier.
The basic scheme is as follows:
<prefix>/bin/c/class
or
| loading/munging/processing scripts
|
<prefix>/cgi/c/class
or
| CGI middleware
|
<prefix>/img/c/collection
| continuous tone image data
|
<prefix>/idx/c/collection
| search indexes and index-related files |
<prefix>/lib/[c/class]
or
| CGI middleware library modules
|
<prefix>/misc/[c/class/]
or
| additional files for DLXS distribution that don't belong elsewhere: e.g., DTDs, character entity sets, catalogs, etc.
|
<prefix>/misc/db
| text-based databases: e.g., DLXS sessions database in CSV format |
<prefix>/misc/[c/class/]maps
or <prefix>/misc/[c/collection/]maps | region map files
|
<prefix>/obj/c/collection
| production content organized by collection identifier
|
<prefix>/obj/d/l/p/dlpsid.vvvv.iii
| production content organized by DLPS identifier
|
<prefix>/prep/c/collection
| pre-production (munge-in-process) content organized by collection identifier
|
<prefix>/prep/d/l/p/dlpsid.vvvv.iii
| pre-production (munge-in-process) content organized by DLPS identifier
|
<prefix>/web/c/collection
| static HTML pages |
<prefix>/web/c/class/graphics
or <prefix>/web/c/collection/graphics | user interface glitter: banners, icons, logos, etc.
|
<prefix>/web/c/class/images
or
| web-ready image content served directly, usually figures that appear in text collections
if image content is licensed, files are stored in <prefix>/img (see above) and this is just a symbolic link
|
<prefix> varies from installation to installation. All DLPS servers use at least the prefix /l1; most have additional file systems named /l2, /l3, etc.
Note that although classes and parts of the organization can also exist at this level (for example, h/hti for the Humanities Text Initiative or i/is for Image Services), the individual collections would exist within their own spaces, i.e., <prefix>/.../m/musart.
Personal directories can also exist at this level, i.e., <prefix>/cgi/c/csnavely, for areas to do testing in.
The organization of files below the levels described here is outside the scope of this convention. We don't feel a need to prescribe a structure for inside /l1/web/b/bas, for example. Of course, consistency between like things is good practice.
[edit] Conventions for Software Packages
[edit] Basic Convention
The DLPS institutional convention is that software should be installed as
/l/local/package-version
where
package is the proper name of the software package, preferably lowercase (e.g. "apache"), and version is the full version number (e.g. "1.3.9"), if available.
Likewise, source code can be found in
/l/local/src/package-version
or compressed up in a tarball named
/l/local/src/package-version.tar.gz
[edit] Special Cases
For cases where a version-independent path is needed or wanted, a symbolic link should be created:
/l/local/package -> /l/local/package-version
For commonly-used utilities that are conventionally installed in places like /usr/local/bin, a symbolic link should be created in /l/local/bin:
/l/local/bin/command -> /l/local/package/[subpath/]command
if present, or
/l/local/bin/command -> /l/local/package-version/[subpath/]command
[edit] Data Flow Describing the Preparation and Release of a Collection
Looking at the conventions from a "process" point of view, the following diagram illustrates how we use the directory conventions in the workflow of preparing and releasing a collection.
