Mounting a Finding Aids Collection

From DLXS Documentation

(Difference between revisions)

Jump to: navigation, search

Revision as of 13:13, 14 September 2007

Main Page > Mounting Collections: Class-specific Steps > Mounting a Finding Aids Collection

This topic describes how to mount a Findaid Class collection.

Workshop materials are located at http://www.dlxs.org/training/workshop200707/findaidclass/fcoutline.html

1 Overview
- 1.1 Examples of Findaid Class implementations and practices
- 1.2 Overview of Data Preparation and Indexing Steps
2 Working with the EAD
- 2.1 EAD 2002 DTD Overview
- 2.2 Practical EAD Encoding Issues
  - 2.2.1 Types of changes to accomodate differing encoding practices and/or interface changes
  - 2.2.2 Specific Encoding Issues
3 Findaid Class Behaviors Overview
4 Preparing Data and Directories
- 4.1 Set Up Directories and Files for Data Preparation
  - 4.1.1 Fixing paths
  - 4.1.2 Step by step instructions for setting up Directories for Data Preparation
- 4.2 Set Up Directories and Files for XPAT Indexing
5 Finding Aids Data Preparation
6 Building the Index
7 Working with Fabricated Regions in Findaid Class
8 Customizing Findaid Class
- 8.1 Working with the table of contents
9 Mounting the Collection Online
10 Troubleshooting
- 10.1 General Techniques
- 10.2 Common Problems and Solutions
  - 10.2.1 Title of Finding Aid does not show up
11 Linking from Finding Aids Using ID Resolver
12 Workshop Materials
13 Working with the User Interface
- 13.1 Findaid Class Graphics Files
- 13.2 Findaid Class Processing Instructions

Overview

To mount a Finding Aids Collection, you will need to complete the following steps:

Prepare your data and set up a directory structure
Validate and normalize your data
Build the Index
Mount the collection online

Examples of Findaid Class implementations and practices

This section contains links to public implementations of DLXS Findaid Class as well as documentation on workflow and implementation issues. If you are a member of DLXS and have a collection or resource you would like to add, or wish to add more information about your collection, please edit this page.

University of Michigan, Bentley Historical Library Finding Aids: Out-of-the-box DLXS 13 implementation.
Overview of Bentley's workflow process for Finding Aids: See also the links in Practical EAD Encoding Issues for background on the Bentley EAD workflow and encoding practices

Unversity of Tennesee Special Collections Libraries: DLXS Findaid Class version ?

University of Pittsburgh, Historic Pittsburgh Finding Aids: DLXS Findaid Class version ?
Background on Pittsburgh Finding Aids workflow

University of Wisconsin, Archival Resources in Wisconsin: Descriptive Finding Aids: DLXS Findaid Class version ?

University of Minnesota Libraries, Online Finding Aids: DLXS Findaid Class version ?

EAD Implementation at the University of Minnesota

Getty Research Institute Special Collections Finding Aids: Heavily customized DLXS11a. Background on Getty customization and user interface changes to DLXS

J. Paul Getty Trust Institutional Archives Finding Aids: Heavily customized DLXS11a.

Overview of Data Preparation and Indexing Steps

Data Preparation

Validate the files individually against the EAD 2002 DTD
make validateeach
Concatenate the files into one larger XML file
make prepdocs
Validate the concatenated file against the dlxsead2002 DTD:
make validate
Normalize the concatenated file.
make norm
Validate the normalized concatenated file against the dlxsead2002 DTD
make validate

The end result of these steps is a file containing the concatenated EADs wrapped in a <COLL> element which validates against the dlxsead2002 and is ready for indexing:

<COLL>
<ead><eadheader><eadid>1</eadid>...</eadheader>... content</ead>
<ead><eadheader><eadid>2</eadid>...</eadheader>... content</ead>
<ead><eadheader><eadid>3</eadid>...</eadheader>... content</ead>
</COLL>

WARNING! If there are extra characters or some other problem with the part of the program that strips out the xml declaration and the doctype declaration the file will end up like:

<COLL>
baddata<ead><eadheader><eadid>1</eadid>...</eadheader>... content</ead>
baddata<ead><eadheader><eadid>2</eadid>...</eadheader>... content</ead>
baddata<ead><eadheader><eadid>3</eadid>...</eadheader>... content</ead>
</COLL>

In this case you will get "character data not allowed" or similar errors during the make validate step. You can troubleshoot by looking at the concatenated file and/or checking your original EADs.

Indexing

make singledd indexes words for texts that have been concatenated into on large file for a collection.
make xml indexes the XML structure by reading the DTD. Validates as it indexes.
make post builds and indexes fabricated regions based on the XPAT queries stored in the workshopfa.extra.srch file.

Working with the EAD

EAD 2002 DTD Overview

These instructions assume that you have already encoded your finding aids files in the XML-based EAD 2002 DTD. If you have finding aids encoded using the older EAD 1.0 standard or are using the SGML version of EAD2002, you will need to convert your files to the XML version of EAD2002. When converting from SGML to XML a number of character set issues may arise. See Data Conversion and Preparation: Unicode,XML, and Normalization

Resources for converting from EAD 1.0 to EAD2002 and/or from SGML EAD to XML EAD are available from:

The Society of American Archivists EAD Tools page:http://www.archivists.org/saagroups/ead/tools.html
Library of Congress EAD conversion toolshttp://lcweb2.loc.gov/music/eadmusic/eadconv12/ead2002_r.html

Other good sources of information about EAD encoding practices and practical issues involved with EADs are:

Library of Congress EAD page http://www.loc.gov/ead/ (This is the home of the EAD standard
EAD2002 tag library http://www.loc.gov/ead/tglib/index.html
The Society of American Archivists EAD Help page: http://www.archivists.org/saagroups/ead/
Various EAD Best Practice Guidelines listed on the Society of American Archivists EAD essentials page: http://www.archivists.org/saagroups/ead/essentials.html (the links to BPGs are at the bottom of the page)
The EAD listserv http://listserv.loc.gov/listarch/ead.html

Practical EAD Encoding Issues

The EAD standard was designed as a loose standard in order to accommodate the large variety in local practices for paper finding aids and make it easy for archives to convert from paper to electronic form. As a result, conformance with the EAD standard still allows a great deal of variety in encoding practices.

The DLXS software is primarily designed as a system for mounting University of Michigan collections. In the case of finding aids, the software has been designed to accommodate the encoding practices of the Bentley Historical Library. The more similar your data and setup is to the Bentley’s, the easier is will be to integrate your finding aids collection with DLXS. If your practices differ significantly from the Bentley’s, you will probably need to do some preprocessing of your files and/or make changes to DLXS.

More information on the Bentley's encoding practices and workflow:

Overview of Bentley's workflow process for Finding Aids http://bentley.umich.edu/EAD/eadproj.htm
Description of Bentley Finding Aids and their presentation on the web http://bentley.umich.edu/EAD/findaids.htm
Bentley MS Word EAD templates and macros http://bentley.umich.edu/EAD/bhlfiles.htm
Description of EAD tags used in Bentley EADs http://bentley.umich.edu/EAD/bhltags.htm

Types of changes to accomodate differing encoding practices and/or interface changes

Custom preprocessing
Add dummy EAD to data
Modify prep scripts (Makefile, preparedocs.pl, validateeach.csh)
Modify *inp files (DOCTYPE declarations and entities)
Modify fabricated regions (*.extra.srch)
Modify CollMgr entries
Modify findaidclass.cfg (change table of contents sections)
Subclass FindaidClass.pm
Modify XSL
Modify XML templates
Modify CSS

Specific Encoding Issues

There are a number of encoding issues that may affect the data preparation, indexing, searching, and rendering of your finding aids. Some of them are:

Preprocessing and Data Prep issues
- <eadid> should be less than about 20 characters in length
- Attribute ids must be unique within the entire collection
- If you use attribute ids and corresponding targets within your EADs preparedocs.pl may need to be modified.
- Character Encoding issues
- UTF-8 Byte Order Marks (BOM) should be removed from EADs prior to concatenation
- XML processing instructions should be removed from EADs prior to concatenation
- Multiline DOCTYPE declarations are currently not properly handled by the data prep scripts (There is a patch available to fix this problem: http://www.dlxs.org/products/archive-by-CDROM/13/Patches/24August2007/ )
- If your DOCTYPE declaration contains entities, you need to modify the appropriate *inp files accordingly (An example is included in the patch samplefa.text.entity.example.inp )
- Out-of-the-box <dao> handling may need to be modified for your needs

Fabricated region issues (some of these involve XSL as well)
- If your <unititle> element precedes your <origination> element in the top level <did>, you will have to modify the maintitle fabricated region query in *.extra.srch See Troubleshooting:Title of Finding Aid does not show up
- If you do not use a <frontmatter> element, you will either have to either a) create and populate frontmatter elements in your EADs manually, or b) run your EADs through some preprocessing XSL to create and populate frontmatter elements, or c) you will have to create a fabricated region to provide an appropriate "Title Page" region based on the <eadheader> and you may also need to change the XSL and/or subclass FindaidClass to change the code that handles the Title Page region.

Table of Contents and Focus Region issues
- If you do not use a <frontmatter> element you may have to make the changes mentioned above to get the title page to show in the table of contents and when the user clicks on the "Title Page" link in the table of contents
- If your encoding practices for <biohist> differ from the Bentley's, you may need to make changes in findaidclass.cfg or create a subclass of FindaidClass and override FindaidClass:: GetBioghistTocHead, and/or change the appropriate XSL files.
- If you want <relatedmaterial> and/or <separatedmaterial> to show up in the table of contents (TOC) on the left hand side of the Finding Aids, you may have to modify findaidclass.cfg and make other modifications to the code. This also applies if there are other sections of the finding aid not listed in the out-of-the-box findaidclass.cfg %gSectHeadsHash.
- See also Customizing Findaid Class: Working with the table of contents

XSL issues
- If you have encoded <unitdate>s as siblings of <unittitle>s, you may have to modify the appropriate XSL templates.
- If you want the middleware to use the <head> element for labeling sections instead of the default hard-coded values in findaidclass.cfg, you may need to change fabricated regions and/or make changes to the XSL and/or possibly modify findaidclass.cfg or subclass FindaidClass.

Findaid Class Behaviors Overview

Preparing Data and Directories

Set Up Directories and Files for Data Preparation

You will need to set up a directory structure where you plan to store your EAD2002 XML source files, your object files (used by xpat for indexing), index files (including region index files)and other information such as data dictionaries, and files you use to prepare your data.

The convention used by DLXS is to use subdirectories named with the first letter of the collection id and the collection name:$DLXSROOT/xxx/{c}/{coll}/ where $DLXSROOT is the "tree" where you install all DLXS components, {c} is the first letter of the name of the collection you are indexing, and {coll} is the collection ID of the collection you are indexing. For example, if your collection ID is "bhlead" and your DLXSROOT is "/l1", you will place the Makefile in /l1/bin/b/bhlead/ , e.g., /l1/bin/b/bhlead/Makefile. See the DLPS Directory Conventions section and Workshop discussion of Directory Conventionsfor more information.

When deciding on your collection id consider that it needs to be unique across all classes to enable cross-collection searching. So you don't want both a text class collection with a collid of "my_coll" and a finding aid class collection with a collection id of "my_coll". You will also probably want to make your collection ids rather short and make sure they don't contain any special characters, since they will also be used for sub-directory names.

Note that the Makefile we provide along with most of the data preparation scripts supplied with DLXS assume this directory structure.

We recommend you use the following directory structure:

Store specialized scripts for preparing and/or preprocessing your collection and its Makefile in $DLXSROOT/bin/{c}/{coll}/ where $DLXSROOT is the "tree" where you install all DLXS components, {c} is the first letter of the name of the collection you are indexing, and {coll} is the collection ID of the collection you are indexing. For example, if your collection ID is "bhlead" and your DLXSROOT is "/l1", you will place the Makefile in /l1/bin/b/bhlead/ , e.g., /l1/bin/b/bhlead/Makefile. See the DLPS Directory Conventions section for more information.
Store your source finding aids in $DLXSROOT/prep/{c}/{coll}/data/.

Store any DTDs, doctype, and files for preparing your data in $DLXSROOT/prep/{c}/{coll}/. Unlike the contents of other directories, everything in prep should be expendable when actually running the indexes.
After running all the targets in the Makefile, the finalized, concatenated XML file for your finding aids collection will be created in $DLXSROOT/obj/{c}/{coll}/ , e.g., /l1/obj/b/bhlead/bhlead.xml.
Store index, region, data dictionary, and init files in $DLXSROOT/idx/{c}/{coll}/ , e.g., /l1/idx/b/bhlead/bhlead.idx. These will be updated as the index related targets in the Makefile are run. See the XPAT documentation for more on these types of files.

Fixing paths

The installation script should have changed all instances of /l1/ to your $DLXSROOT and all bang prompts "#!/l/local/bin/perl" to your location of perl. However, you may wish to check the following scripts:

$DLXSROOT/bin/f/findaid/output.dd.frag.pl
$DLXSROOT/bin/f/findaid/inc.extra.dd.pl
$DLXSROOT/bin/s/samplefa/preparedocs.pl

If you use the Makefile in $DLXSROOT/bin/s/samplefa you should check that the paths in the Makefile are correct for the locations of xpat, oxs, and osgmlnorm as installed on your system. These are the Make varibles that should be checked:

XPATBINDIR
OSX
OSGMLNORM

Step by step instructions for setting up Directories for Data Preparation

You can use the scripts and files from the sample finding aids collection "samplefa" as a basis for creating a new collection. In the instructions that follow you would use /{c}/{coll} instead of /w/workshopfa where {c} is the first letter of your collection id and {coll} is your collection id. So for example if your collection id was mycoll instead of

cp $DLXSROOT/prep/s/samplefa/samplefa.extra.srch $DLXSROOT/prep/w/workshopfa/workshopfa.extra.srch

you would do

cp $DLXSROOT/prep/s/samplefa/samplefa.extra.srch $DLXSROOT/prep/m/mycoll/mycoll.extra.srch

This documentation will make use of the concept of the $DLXSROOT, which is the place at which your DLXS directory structure starts. We generally use /l1/.

To check your $DLXSROOT, type the following command at the command prompt:

echo $DLXSROOT

The prep directory under $DLXSROOT is the space for you to take your encoded finding aids and "package them up" for use with the DLXS middleware. Create your basic directory $DLXSROOT/prep/w/workshopfa and its data subdirectory with the following command:

mkdir -p $DLXSROOT/prep/w/workshopfa/data

Move into the prep directory with the following command:

cd $DLXSROOT/prep/w/workshopfa

This will be your staging area for all the things you will be doing to your EADs, and ultimately to your collection. At present, all it contains is the data subdirectory you created a moment ago. Unlike the contents of other collection-specific directories, everything in prep should be ultimately expendable in the production environment.

Copy the necessary files into your data directory with the following commands:

cp $DLXSROOT/prep/s/samplefa/data/*.xml $DLXSROOT/prep/w/workshopfa/data/.

We'll also need a few files to get us started working. They will need to be copied over as well, and also have paths adapted and collection identifiers changed. Follow these commands:

cp $DLXSROOT/prep/s/samplefa/validateeach.csh $DLXSROOT/prep/w/workshopfa/.
cp $DLXSROOT/prep/s/samplefa/samplefa.xml.inp $DLXSROOT/prep/w/workshopfa/workshopfa.xml.inp
cp $DLXSROOT/prep/s/samplefa/samplefa.text.inp $DLXSROOT/prep/w/workshopfa/workshopfa.text.inp
mkdir -p $DLXSROOT/obj/w/workshopfa
mkdir -p $DLXSROOT/bin/w/workshopfa
cp $DLXSROOT/bin/s/samplefa/preparedocs.pl $DLXSROOT/bin/w/workshopfa/.
cp $DLXSROOT/bin/s/samplefa/Makefile $DLXSROOT/bin/w/workshopfa/Makefile

(If you have installed the Release 13 August 24th patch substitute these instructions)

Now you'll need to edit these files to ensure that the paths match your $DLXSROOT and that the collection name is workshopfa instead of samplefa.

STOP!! Make sure you edit the files before going to the next steps!!

Make sure you change these files:

$DLXSROOT/prep/w/workshopfa/validateeach.csh
$DLXSROOT/bin/w/workshopfa/Makefile (see below for details)

You can run this command to check to see if you forgot to change samplefa to workshopfa:

grep "samplefa" $DLXSROOT/bin/w/workshopfa/* $DLXSROOT/prep/w/workshopfa/* |grep -v "#"

With the ready-to-go ead2002 encoded finding aids files in the data directory, we are ready to begin the preparation process. This will include:

Validating the files individually against the EAD 2002 DTD
concatenating the files into one larger XML file
validating the concatenated file against the dlxsead2002 DTD
"normalizing" the concatenated file.
validating the normalized concatenated file against the dlxsead2002 DTD

These steps are generally handled via the Makefile in $DLXSROOT/bin/s/samplefa which we have copied to $DLXSROOT/bin/w/workshopfa. Example Makefile.

Make sure you changed your copy of the Makefile to reflect

/w/workshopfa instead of /s/samplefa. You will want to change lines 2 and 3 accordingly

   1  
   2  NAMEPREFIX = samplefa
   3  FIRSTLETTERSUBDIR = s

Tip: Be sure not to add any space after the workshopfa or w. The Makefile ignores space immediately before and after the equals sign but treats all other space as part of the string. If you accidentally put a space after the FIRSTLETTERSUBDIR = s , you will get an error like "[validateeach] Error 127" If you look closely at the first line of what the Makefile reported to standard output (see below) you will see that instead of running the command:

/l1/workshop/tburtonw/dlxs/prep/w/workshopfa/validateeach.csh

which just calls the validateeach c-shell script

it tried to run a directory name: "/l1/workshop/tburtonw/dlxs/prep/w" with the argument "/workshopfa/validateeach.csh" which does not make sense

 % make validateeach
/l1/workshop/tburtonw/dlxs/prep/w /workshopfa/validateeach.csh
make: execvp: /l1/workshop/tburtonw/dlxs/prep/w: Permission denied
make: [validateeach] Error 127 (ignored)

Further note on editing the Makefile: If you modify or write your own Make targets, you need to make sure that a real "tab" starts each command line rather than spaces. The easiest way to check for these kinds of errors is to use "cat -vet Makefile" to show all spaces, tabs and newlines

You should make sure you thate $DLXSROOT, and the locations of the various binaries to have been changed to match your installation.

Change $DLXSROOT /l1/ to your $DLXSROOT on every line that uses it
Change XPATBINDIR = /l/local/bin/ to the location of the xpat binary in your installation
Change the location of the osx binary from

OSX = /l/local/bin/osx
to the location in your installation

Change the location of the osgmlnorm binary from

OSGMLNORM = /l/local/bin/osgmlnorm
to the location in your installation

Tip: oxs and osgmlnorm are installed as part of the OpenSP package. If you are using linux, make sure that the OpenSP package for your version of linux is installed and make sure the paths above are changed to match your installation. If you are using Solaris you will have to install (and possibly compile) OpenSP. You may also need to make sure the $LD_LIBRARY_PATH environment variable is set so that the OpenSP programs can find the required libraries. For troubleshooting such problems the unix ldd utility is invaluble. See also links to OpenSP package on the tools page: Useful Tools

Set Up Directories and Files for XPAT Indexing

Please substitute /{c}/{coll} where {c} is the first letter of your collection id 
and {coll}is your collection id  for any instance of /w/workshopfa 
and substitute {coll} wherever you see "workshopfa" in the following instructions.

First, we need to create the rest of the directories in the workshopfa environment with the following commands:

mkdir -p $DLXSROOT/idx/w/workshopfa

The bin directory we created when we prepared directories for data preparation holds any scripts or tools used for the collection specifically; obj ( created earlier) holds the "object" or XML file for the collection, and idx holds the XPAT indexes. Now we need to finish populating the directories.

 cp $DLXSROOT/prep/s/samplefa/samplefa.blank.dd  DLXSROOT/prep/w/workshopfa/workshopfa.blank.dd
 cp $DLXSROOT/prep/s/samplefa/samplefa.extra.srch $DLXSROOT/prep/w/workshopfa/workshopfa.extra.srch

Each of these files need to be edited to reflect the new collection name and the paths to your particular directories. Failure to change even one file can result in puzzling errors, because the scripts are working, just not necessarily in the directories you are looking at.

grep -l "samplefa" $DLXSROOT/prep/w/workshopfa/*

will check for any leftover unchanged instances of the string /samplefa. You also need to check that "/l1/" has been replacedby whatever $DLXSROOT is on your server. If you don't have an /l1 directory on your server (which is very likely if you are not here using a DLPS machine) you can check with:

grep -l "l1" $DLXSROOT/prep/w/workshopfa/*

Finding Aids Data Preparation

Building the Index

Working with Fabricated Regions in Findaid Class

Customizing Findaid Class

Working with the table of contents

The table of contents on the left-hand side of the finding aid display is based on fabricated regions set up in *.extra.srch and configured either in a configuration file or in a subclass of FindaidClass.pm

If a subclass is not being used to override the FindaidClass::_initialize method, the configuration file will be used. It is:

$DLXSROOT/cgi/f/findaidclass/findaidclass.cfg

The configuration file sets up a hash called %gSectHeadsHash. The relevant section of the findaidclass.cfg file is:

# **********************************************************************
# Hash of section heads that XPAT should search for.  A reference to
# this hash is added as member data keyed by 'tocheads' to the
# FindaidClass object at initialization time. Comment out those that
# are missing in your finding aids.
# **********************************************************************
%gSectHeadsHash = (
                  'bioghist-t'      =>  {
                                         'collection' => qq{Biography},
                                         'recordgrp' => qq{History},
                                        },
                  'controlaccess-t' => qq{Subject Terms},
                  'frontmatter-t'   => qq{Title Page},
                  'arrangement-t'   => qq{Arrangement},
                  'scopecontent-t'  => qq{Collection Scope and Content Note},
                  'summaryinfo-t'   => qq{Summary Information},
                  'contentslist-t'  => qq{Contents List},
                  'admininfo-t'     => qq{Access and Use},
                  'add-t'           => qq{Additional Descriptive Data},
                 );

The %gSectHeadsHash is normally loaded read from the configuration file and loaded into a hash called tocheads in the FindaidClass::_initialize method when the FindaidClass object is created. If you wish to change the table of contents on a collection-specific basis, you can override the FindaidClass::_initialize method in a collection-specific subclass.

For an example of using a subclass to override the default table of contents see: $DLXSROOT/cgi/f/findaid/FindaidClass/SamplefaFC.pm

Note that the default setting in the Collection Manager for the samplefa collection is to use this subclass:

The diagram below shows the fabricated region and the corresponding EAD element tags for the out-of-the-box table of contents

Changing the labels in the table of contents

If you want to change the labels for all of your Findaid Class collections, you can change the strings in the %gSectHeadsHash hash in $DLXSROOT/cgi/f/findaid/findaidclass.cfg. If you want to change the labels on a collection by collection basis, you will probably want to subclass and override the FindaidClass::_initialize method as is done in the sample file: $DLXSROOT/cgi/f/findaid/FindaidClass/SamplefaFC.pm

Adding sections to the table of contents

Changing the Bioghist labels to use the appropriate <head> elemements

Mounting the Collection Online

These are the final steps in deploying an Findaid Class collection online. Here the Collection Manager will be used to create and edit a Collection Database entry for workshopfa . The Collection Manager will also be used to check the Group Database. Finally, we need to work with the collection map and the set up the collection's web directory.

Create and edit an entry in the Collection Database for your collection with CollMgr

Each collection has a record in the collection database that holds collection specific configurations for the middleware. CollMgr (Collection Manager) is a web based interface to the collection database that provides functionality for editing each collection's record. Collections can be checked-out for editing, checked-in for testing, and released to production. In general, a new collection needs to have a CollMgr record created from scratch before the middleware can be used.

Step 1. Create a workshopfa Collmgr entry by copying from samplefa.

A. Login to Collmgr. The URL should be:

http://path_to_cgi/cgi/c/collmgr

The collmgr page is usually set up to use apache basic authorization. The username and password should have been set up when you set up your virtual host in apache. (sample apache virtual host )

B. Select Manage Collections:Findaid Class:

C. Select samplefa and click on "copy a collection" (Note: In the image below workshopfa already exists, but in your clean install it will not exist)

D. Enter your collection id (workshopfa)

E. Change all occurances of "samplefa" to "workshopfa" For example in the section below the webdir should be changed from "s/samplefa" to "w/workshopfa" (And you need to copy and rename the appropriate files from $DLXSROOT/web/s/samplefa to $DLXSROOT/web/w/workshopfa)

WARNING! If you forget to change one of the entries it can lead to very confusing results. For example if you forget to change the "dd" file entry from "/idx/s/samplefa/samplefa.dd" to /idx/w/workshopfa/workshopfa.dd", the middleware will try to search the samplefa collection but all the rest of the configuration information will point to workshopfa, which will result in erratic behavior and potentially confusing error messages.

F. Change the entry for the subclassmodule from "/FindaidClass/SamplefaFC" to "FindaidClass". This means that this collection will use the default FindaidClass.pm instead of the SampleFC subclass. (Unless you want to subclass Findaid Class in which case you would replace "SamplefaFC with the name of your collection-specific subclass)

G. Set the containerdepth field to the depth of containers in your collection

For example if you have levels c01 to c05 set the containerdepth to 5. You can use the xpat command {ddinfo regionnames} to look at your data and look for the highest c level to determine what number to put here.

xpatu $DLXSROOT/idx/s/samplefa/samplefa.dd
?>> {ddinfo regionnames}

If you have containerdepth set to a number that is higher than what is in your data, xpat will try to search for the missing c0x level elements and will produce errors. This can occur whenever xpat tries to query the 'c0xheads" fabricated region. For example we set the continer depth to 7 for the samplefa collection (the samplefa collection only has c01-c06) and then got the following error message when we tried to view a kwic (search terms in context) view for the Post Family Papers in our web browser:

Message: Query error in samplefa, samplefa.dd, query=pr.region.c0xhead 
(region "c0xhead" ^ ( region "c07" incl *detailslicesearch ));, 
Error=No information for region c07 in the data dictionary. syntax error before: ))

You will also probably want to edit:

fields related to the dynamic browse page (See Create a browse page)
fields related to searching and sorting in the user interface: regionsearch, termsearch, sortfields (Note that these need to match the entries in your map file

More Documentation

Collection Manager Field Descriptions

Review the Groups Database Entry with CollMgr

Another function of CollMgr allows the grouping of collections for cross-collection searching. Any number of collection groups may be created for Findaid Class. Findaid Class supports a group with the groupid "all". It is not a requirement that all collections be in this group, though that's the basic idea. Groups are created and modified using CollMgr.

Make Collection Map

Collection mapper files exist to identify the regions and operators used by the middleware when interacting with the search forms. Each collection will need one, but most collections can use a fairly standard map file, such as the one in the samplefa collection. The map files for all Findaid Class collections are stored in $DLXSROOT/misc/f/findaid/maps

You can find an example map file for the sample finding aids collection at DLXSROOT/misc/f/findaid/maps/samplefa.map. Rather than modifying this file, you should copy it so that you always have a blank copy to which to refer.

You can use the following commands to copy the samplefa.map file to use as a basis for your collection:

 cd $DLXSROOT/misc/f/findaid/maps
 cp samplefa.map workshopfa.map

Map files contain mapped items where one term or name for the item is mapped to another term or name. For example, a term used by an HTML form to refer to a searchable region (e.g., "entire finding aid") can be mapped to an XPAT searchable region (e.g., EAD). For more general background on map files, see [DLXS Map Files.]

Currently, the format of the map files is XML and each collection map file conforms to a simple DTD (we have considered implementation of other possible ways of mapping terms, such as a database where one could map from one column's data to another). The middleware reads the map file into a TerminologyMapper object after which the CGI program can at any time request of the object the mappings for terms. Each mapped item and its various terms are contained within a <MAPPING> element.

Each mapping element in a map file consists of the following:

label: This element determines what will display in the user's browser when constructing searches. It must match the value used in the collmgr. (See step 2.)

synthetic: This element contains the variable name as it is used in the cgi.

native: The "native" element provides an appropriate XPAT search that the system will use to discover the appropriate content. The search may be simple (e.g., region EADID) or complex (e.g., ((region DID within region ARCHDESC) not within region DSC))

nativeregionname: The element name itself, as it is indexed, without terms used in the XPAT search.

Map files take language that is used in the forms and translates it into language for the cgi and for XPAT. For example, if you want your users to be able to search within names, you need to add a mapping for how you want headings and categories to appear in the search interface (case is important, as is pluralization!), how the cgi variable is set (usually in all caps, and not stepping on an existing variable), and how XPAT will identify and retrieve this natively (in XPAT search language). The first part of the map file is operator mapping, for the form, the cgi, and XPAT, and the second part is for region mapping. You might note that some of the fields that are defined in the map file correspond to some of the fabricated regions. Note: The larger the map file, the slower your site will run, so you don’t necessarily want to map everything, such as variations of singular and plural fields.

Set Up the Collection's Web Directory

Each collection may have a web directory with custom Cascading Style Sheets, interface templates, graphics, and javascript. The default is for a collection to use the web templates at $DLXSROOT/web/f/findaid. Of course, collection specific templates and other files can be placed in a collection specific web directory, and it is necessary if you have any customization at all. DLXS Middleware uses [../ui/index.html#fallback fallback] to find HTML related templates, chunks, graphics, js and css files.

For a minimal collection, you will want two files: index.html and FindaidClass-specific.css.

mkdir -p $DLXSROOT/web/w/workshopfa cp $DLXSROOT/web/s/samplefa/index.html $DLXSROOT/web/w/workshopfa/index.html cp $DLXSROOT/web/s/samplefa/findaidclass-specific.css $DLXSROOT/web/w/workshopfa/findaidclass-specific.css

As always, we'll need to change the collection name and paths. You might want to change the look radically, if your HTML skills are up to it.

Note that the browse link on the index.html page is hard-coded to go to the samplefa hard-coded browse.html page. You may want to change this to point to a dynamic browse page (see below). The url for the dynamic browse page is ".../cgi/f/findaid/findaid-idx?c=workshopfa;page=browse".

If you would prefer a dynamic home page, you can copy and modify the home.xml and home.xsl files from $DLXSROOT/web/f/findaid/. Note that they are currently set up to be the home page for all finding aids collections, so you will have to do some considerable editing. However they contain a number of PIs that you may find useful. In order to have these pages actually be used by DLXS, they have to be present in your $DLXSROOT/web/w/workshopfa/ directory and there can't be an index.html page in that directory. The easiest thing to do, if you have an existing index.html page is to rename it to "index.html.foobar" or something.

Create a browse page

See the documentation: http://www.dlxs.org/docs/13/collmeta/browse.html

Try It Out

http://$DLXSROOT/cgi/f/findaid/findaid-idx?c=workshopfa

Troubleshooting

General Techniques

Common Problems and Solutions

Title of Finding Aid does not show up

This is usually caused by the <origination> preceding the <unittitle> in the top level <did> element of your EAD

In the *.extra.srch file

comment out the following line:

(note that the region definitions are all on one line, but have been wrapped so they will be readable in the wiki)

##
((region "<origination".."</unittitle>") 
within ((region did within region archdesc)
not within region dsc));
{exportfile "/l1/release/13/idx/s/samplefa/maintitle.rgn"}; 
 export; ~sync "maintitle";
##

copy the line but reverse the order of unittitle and origination

##
((region "<unittitle".."</origination>") 
within ((region did within region archdesc)
not within region dsc));
{exportfile "/l1/release/13/idx/s/samplefa/maintitle.rgn"}; 
export; ~sync "maintitle";
##

make post errors
- x
- y

Linking from Finding Aids Using ID Resolver

How do you do this?

Findaid Class is coded so that if there is an href attribute to the <dao> element, it will check to see if it contains the string "http". If it does, FindaidClass will not us ID Resolver, but will create a link based on the content of the href attribute of the <dao>. If there is no "http" string in the href attribute, FindaidClass assumes that the href attribute is actully an id and will look up that id in in the idresolver and build a link if it finds the ID in the IDRESOLVER table. The method FilterAllDaos_XML in $DLXSROOT/cgi/f/findaid/FindaidClass.pm can be overridden per collection if different behavior is needed.

If you decide to use this feature, you will want to modify the preprocessing script preparedocs.pl which out-of-the-box inserts the string 'dao-bhl-' after the href. Below is an example of a Bentley <dao> where the id number is 91153-1.

<dao linktype="simple" href="91153-1" show="new" actuate="onrequest">
         <daodesc>
          <p>[view selected images]</p>
         </daodesc>
        </dao>

The preparedocs.pl program would change this to:

<dao linktype="simple" href="dao-bhl-91153-1" show="new" actuate="onrequest">
         <daodesc>
          <p>[view selected images]</p>
         </daodesc>
        </dao>

The ID resolver would look up the id "dao-bhl-91153-1" and replace it with the appropriate URL.

ID Resolver Data Transformation and Deployment

The ID Resolver is a CGI that takes as input a unique identifier and returns a URI. It is used, for example, by Harper's Weekly to link the text pages in Text Class middleware to the image pages in the Image Class middleware, and vice versa.

Plug something like the following in to your web browser and you should get something back. If you choose to test middleware on a development machine that uses the id resolver, make sure that the middleware on that machine is calling the resolver on the machine with the data, and not the resolver on the production server.

http://clamato.hti.umich.edu/cgi/i/idresolver/idresolver?id=dao-bhl-bl000684
which should yield...
<ITEM MTIME="20030728142225"><ID>dao-bhl-bl000684 </ID><URI>http://images.umdl.umich.edu/cgi/i/image/image-idx?&q1=bl000684&rgn1=bhl_href&type=boolean&med=1&view=thumbnail&c=bhl </URI></ITEM>

Information on how to set up the ID resolver

</blockquote>

Workshop Materials

Working with the User Interface

General user interface customizations, such as changing rendering style (CSS) or making changes to the XSL are covered in Customizing the User Interface. Specific user-interface issues related to Findaid Class are discussed in the following sections:

Customizing Findaid Class
1. Working with the table of contents
Working with Fabricated Regions in Findaid Class
Common Problems and Solutions
1. Title of Finding Aid does not show up

Findaid Class Graphics Files

Are there findaid class specific graphics files? The existing html docs actually point to a ../t/text/ directory and it appears that the graphics are generic and not at all specific to findaid class.

Findaid Class Processing Instructions

We decided that we could not maintain a list of class specific processing instructions so this section should probably be cut.

Top

@@ Line 331: / Line 331: @@
-==Working with Fabricated Regions in Findaid Class==
+==[[Working with Fabricated Regions in Findaid Class]]==
-When you use XPAT in combination with xmlrgn and a DTD, you are identifying the elements and attributes in the DTD or tags file as "regions," containers of content rather like fields in a database. These separate regions are built into the regions file (collid.rgn) and are identified in the data dictionary (collid.dd). This is what is happening when you are running xmlrgn.
-However, sometimes the things you want to identify collectively aren't so handily identified as elements in the DTD. For example, the Findaid Class search interface can allow the user to search in Names regions. Perhaps for your collection you want Names to include persname, corpname, geoname. By creating an XPAT query that ORs these regions, you can have XPAT index all the regions that satisfy the OR-ed query. For example:
- (region "name" + region "persname" + region "corpname" + region "geoname" +
- region "famname")
-Once you have a query that produces the results you want, you can add an entry to the *.extra.srch file which (when you run the "make post" command) will run the query, create a file for export, export it, and sync it:
- {exportfile "$DLXSROOT/idx/c/collid/names.rgn"} export ~sync "names"
-Why fabricate regions? Why not just put these queries in the map file and call them names? While you could, it's probably worth your time to build these succinctly-named and precompiled regions; query errors are more easily identified in the index building than in the CGI, and XPAT searches can be simpler and quicker for terms within the prebuilt regions.
-Fabricated regions within the Findaid Class can be found in the extra.srch file for the sample collection at <tt>$DLXSROOT/prep/s/samplefa/samplefa.extra.srch</tt>. As with any other elements used in the interface for a given collection, '''fabricated regions used in the user interface, such as the names of searches available in the dropdown menu of the search box, must also be represented in the collmgr entry and the map file for that collection.'''
-The middleware for Findaid Class uses a number of fabricated regions in order to speed up xpat queries and simplify coding and configuration.
-Findaid Class uses fabricated regions for several purposes
-# To share code with Text Class (e.g. region main)
-# Fabricated regions for searching  (e.g. region names)
-# Fabricated regions to produce the Table of Contents and to implement display of EAD sections as focused regions such as the "Title Page" or "Arrangement" ( See [[#Working_with_the_table_of_contents |Working with the table of contents]] for more information on the use of fabricated regions for the table of contents.)
-# Other regions specifically used in a PI (region maintitle is used by the PI <?ITEM_TITLE_XML?> used to display the title of a finding aid at the top of each page)
-The fabricated region "main" is set to refer to <span class="unixcommand">&lt;ead&gt;</span> in FindaidClass with:
- (region ead); {exportfile "/l1/idx/b/bhlead/main.rgn"}; export; ~sync "main";
-whereas in TextClass "main" can refer to <span class="unixcommand">&lt;TEXT&gt;</span>. Therfore, both FindaidClass and TextClass can share the Perl code, in a higher level subclass, that creates searches for "main".
-Other fabricated regions are used for searching such as the maintitle and mainauthor regions.
-All of the search links in the dropdown menu for the basic search (see below) are based on indexes for fabricated regions.
-[[Image:Basic_search.png]]
-These are the default regions used for searching and the names used in the menu:
-;archdesc:Entire Finding Aid
-;names:Names
-;places:Places
-;subjects:Subjects
-;callnum:Call Number
-;maintitle:Collection Title
-;repository:Repository
-(The relationship between the region and the name in the menu is set in the map file.  See [[#Make_Collection_Map | Make Collection Map]] )
-The majority of the fabricated regions for Findaid Class are used for the creation and display of the left hand table of contents in the "outline" view. The findaidclass.cfg file contains a hash called %gSectHeadsHash which is normally loaded into FindaidClass.pm's tocheads hash in the FindaidClass::_initialize method. The elements of the hash and the corresponding fabiricated regions are used to create the table of contents and to output the XML for the corresponding section of the EAD when one of the TOC links is clicked on by a user. The fabricated regions are used so XPAT can have binary indexes ready to use for fast retrieval of these EAD sections.  See [[#Working_with_the_table_of_contents |Customizing Findaid Class:Working with the table of contents]] for more information on the use of fabricated regions for the table of contents.
-Some of the more interesting regions extracted from the samplefa.extra.srch file are listed below.
-One of these regions is the add. This used to be <span class="unixcommand">&lt;ADD&gt;</span> in the EAD 1.0 DTD, but now, is created based on the ead2002 DTD's <span class="unixcommand">&lt;descgrp&gt;</span> tag which contains a <span class="unixcommand">type</span> attribute of <span class="unixcommand">add</span>.
-A number of issues related to varying encoding practices can be resolved by the appropriate edits to the *.extra.srch file. (Although some of them may require changes to other files as well)
-* If your &lt;unititle&gt; element precedes your &lt;origination&gt; element in <span class="unixcommand">the top level &lt;did&gt;, you will have to modify the "maintitle" fabricated region query in *.extra.srch </span>
-* If you do not use a &lt;frontmatter&gt; element, you will have to make modifications to various files including modifying *.extra.srch to provide an appropriate "Title Page" region based on the &lt;eadheader&gt;
-* If your encoding practices for &lt;biohist&gt; differ from the Bentley's, you may need to make changes in the &lt;bioghist&gt; fabricated region although changes to other files may be suffient. The changes might include: modifying findaidclass.cfg or creating a subclass of FindaidClass and override FindaidClass:: GetBioghistTocHead, and/or changing the appropriate XSL files.
-* If you want sections of the finding aid that are not completely within a well-defined element such as &lt;relatedmaterial&gt;or &lt;separatedmaterial&gt; to show up in the table of contents, you may have to create a fabricated region using the appropriate xpat query and then modify findaidclass.cfg and make other modifications to the code.
-----
-<blockquote>
-<pre>
-   (region ead); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/main.rgn"}; export; ~sync "main";
-     ##
-     (((region "&lt;c01".."&lt;/did&gt;" + region "&lt;c02".."&lt;/did&gt;" + region "&lt;c03".."&lt;/did&gt;" + region "&lt;c04".."&lt;/did&gt;" + region "&lt;c05".."&lt;/did&gt;" + region "&lt;c06".."&lt;/did&gt;" + region "&lt;c07".."&lt;/did&gt;" + region "&lt;c08".."&lt;/did&gt;" + region "&lt;c09".."&lt;/did&gt;") not incl ("level=file" + "level=item")) incl "level="); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/c0xhead.rgn"}; export; ~sync "c0xhead";
-        ##
-     ((region "&lt;origination".."&lt;/unittitle&gt;") within ((region did within region archdesc) not within region dsc)); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/maintitle.rgn"}; export; ~sync "maintitle";
-     ##
-     ((region "persname" + region "corpname" + region "famname" + region "name") within (region "origination" within ( region "did" within (region "archdesc")))); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/mainauthor.rgn"}; export; ~sync "mainauthor";
-     ##
-     (region "abstract" within ((region did within region archdesc) not within region "c01")); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/mainabstract.rgn"}; export; ~sync "mainabstract";
-        ##
-        ((region unitdate incl "encodinganalog=245$f") within ((region did within region archdesc) not within region dsc)); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/colldate.rgn"}; export; ~sync "colldate";
-     ##
-     (region dsc); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/contentslist.rgn"}; export; ~sync "contentslist";
-     ##
-      ########## admininfo ########
-     admininfot = (region "descgrp-T" incl (region "A-type" incl "admin")); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/admininfo-t.rgn"}; export; ~sync "admininfo-t";
-     ##
-     ## ########## add ######
-     addt = (region "descgrp-T" incl (region "A-type" incl "add")); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/add-t.rgn"}; export; ~sync "add-t";
-   ## ########## frontmatter/titlepage ########
-   frontmattert = region "frontmatter-T"; {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/frontmatter-t.rgn"}; export; ~sync "frontmatter-t";
-     ##
-     # frontmatter itself not needed as fabricated region since it exists
-     # as a regular xml region
-     ##
-   ## ########## bioghist ########
-     bioghist = ((region "bioghist" within region "archdesc") not within region "dsc"); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/bioghist.rgn"}; export; ~sync "bioghist";
-   ##bioghisthead = ((region "&lt;bioghist" .. "&lt;/head&gt;" within region "archdesc") not within region "dsc"); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/bioghisthead.rgn"}; export; ~sync "bioghisthead";
-     ##
-   ((region did within region archdesc) not within region dsc); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/summaryinfo.rgn"}; export; ~sync "summaryinfo";;
-     ##
-   ##
-   #############################
-   (region "subject" + region "corpname" + region "famname" + region "name" + region "persname" + region "geogname"); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/subjects.rgn"}; export; ~sync "subjects";
-   (region "corpname" + region "famname" + region "name" + region "persname"); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/names.rgn"}; export; ~sync "names";
-   #(region "odd-T" ^ (region odd not within region dsc)); {exportfile "/l1/workshop/user11/dlxs/idx/s/samplefa/odd-t.rgn"}; export; ~sync "odd-t";
- </pre>
-</blockquote>
-See [[samplefa.extra.srch.txt|samplefa.extra.srch]] for all of the fabricated regions used with the samplefa collection.
-==== Fabricated regions required in Findaid Class ====
-* main
-* maintitle
-* mainauthor
-* mainabstract
-* colltitle
-* colldate
-* callnum
-* contentslist
-* contentslist-t
-* admininfo
-* admininfo
-* admininfo-t
-* frontmatter-t
-* bioghist-t
-* arrangement-t
-* controlaccess-t
-* controlaccess
-* scopecontent-t
-* summaryinfo-t
-* summaryinfo
-==== Fabricated regions commonly found in Findaid Class ====
-* subjects
-* names
 ==Customizing Findaid Class ==