Release Notes for Current DLXS Release

From DLXS Documentation

(Difference between revisions)
Jump to: navigation, search
m (→METS Pageturner)
Current revision (16:06, 20 August 2008) (edit) (undo)
(→Oai)
 
(107 intermediate revisions not shown.)
Line 10: Line 10:
   <ul>
   <ul>
         <li>[[#BibClass|BibClass version 3.8.1]]</li>
         <li>[[#BibClass|BibClass version 3.8.1]]</li>
-
         <li>[[#Oai version 1.1.1]]</li>
+
         <li>[[#Oai|Oai version 1.1.2]]</li>
 +
        <li>[[#IdResolver|IdResolver 1.1.1]]</li>
         <li>[[#broker20|broker20 version 1.5.1]]</li>
         <li>[[#broker20|broker20 version 1.5.1]]</li>
         <li>[[#Collmgr|Collmgr version 3.3.1]]</li>
         <li>[[#Collmgr|Collmgr version 3.3.1]]</li>
-
         <li>[[#FindaidClass|FindaidClass version 6.5.1]]</li>
+
         <li>[[#FindaidClass|FindaidClass version 6.5.2]]</li>
         <li>[[#ImageClass|ImageClass version 4.11.1]]</li>
         <li>[[#ImageClass|ImageClass version 4.11.1]]</li>
         <li>[[#Lib|Lib version 4.12.1]]</li>
         <li>[[#Lib|Lib version 4.12.1]]</li>
         <li>[[#TextClass|TextClass version 4.7.1]]</li>
         <li>[[#TextClass|TextClass version 4.7.1]]</li>
         <li>[[#XClass|XClass version 2.2.5]]</li>
         <li>[[#XClass|XClass version 2.2.5]]</li>
-
         <li>[[#METS Pageturner|METS Pageturner/Collection Builder version 1.26.1]]</li>
+
         <li>[[#METS Pageturner and Collection Builder|METS Pageturner and Collection Builder version 1.26.1]]</li>
         <li>[[#XPAT|XPAT version 5.3.2]]</li>
         <li>[[#XPAT|XPAT version 5.3.2]]</li>
-
         <li>SRU version 1.2.1</li>
+
         <li>[[#SRU|SRU version 1.2.1]]</li>
         <li>dlxsd version 1.0.3</li>
         <li>dlxsd version 1.0.3</li>
Line 37: Line 38:
   </ul>
   </ul>
-
      <hr />
+
 
     </p>
     </p>
Line 48: Line 49:
==Database Installation Notes==
==Database Installation Notes==
-
<p>MySQL is now the only supported database type.  CSV file-based database support has been removed. In order to run DLXS you will need to have a MySQL server installed.  Sample data is delivered in the form of a MySQL dump file which can be directly imported into a MySQL database.  The database upgrade script (<b>upgrade_5_6</b>) operates only on a MySQL database.  If you have a version 5 CSV database you should run <b>dbmove</b> to move your CSV data into a MySQL database prior to running the upgrade.  These issues are documented in detail in the [[Installing DLXS|installation instructions]] and the [[DLXS Database Upgrade Utility|upgrade instructions]].</p>
+
<p>MySQL is the supported database type.  In order to run DLXS you will need to have a MySQL server installed.  Sample data is delivered in the form of a MySQL dump file which can be directly imported into a MySQL database.  The database upgrade script (<b>upgrade_6_7</b>) operates only on a MySQL database.  These issues are documented in detail in the [[Installing DLXS|installation instructions]] and the [[DLXS Database Upgrade Utility|upgrade instructions]].</p>
==New and Changed Functionality==
==New and Changed Functionality==
Line 59: Line 60:
===Lib===
===Lib===
-
     <ul>
+
     <UL>
-
      <li>If user fails to supply email address it is now treated
+
    <LI> AuthNZ.pm
-
       as a user error not a system error. <strong>[BookBag.pm]</strong> </li>
+
       <UL>
-
      <li>Added&quot;value&quot; to the list of cgi parameters
+
        <LI>IE needs escaped return login url parameter
-
        excluded from cleaning. &quot;value&quot; needs to be excluded because
+
          to be the last cgi param. So now it is.</LI>
-
        it is used in a MySQL query for browse that does a partial phrase match
+
        <LI>Implementation of session stored return
-
        (using &quot;like&quot;). If characters such as parenthesis are stripped
+
          login url. Ran into a problem with long URLs in IE,
-
        out, the query fails. <strong>[CioFactory.pm]</strong></li>
+
          so had to do this.</LI></UL></LI>
 +
  </UL>
-
      <li>A modest performance improvement was achieved through
+
  <UL>
-
        a simple code change that drastically reduced the number of calls to
+
    <LI> CollsInfo.pm
-
        the SetUTF8Flag routine when reading the Collection Database. <strong>[CollsInfo.pm]</strong></li>
+
      <UL>
-
      <li>Added slices and tag lists to browsing. Added support for
+
        <LI>removed setting of utf8flag for data
-
        field specific browse level configuration. Added Browse navigation and
+
          out of utf8 mysql table</LI>
-
        list building was changed to optimize mysql querying and to fix anomalies
+
        <LI>Added StoreFieldValue routine. It allows
-
        in listing certain letter pairs. <strong>[DLXSApp.pm]</strong></li>
+
          middleware to update a Collmgr field value for a single
-
       <li>Several minor additions and enhancements were made, most
+
          collection and for the current user (production, release,
-
         notably automatic reconnection to database which is especially helpful
+
          dev/user). The benefit of doing it in Collsinfo is
-
         when loading large datasets. <strong>[DbUtils.pm]</strong></li>
+
          that the cio knows about the current user, making
-
       <li>Added silent assertion, which sends an asssertion email
+
          it easy to update only the current user's row.</LI></UL></LI>
-
         to developers but does not disrupt the CGI run for the user. [DlpsUtils.pm]</li>
+
  </UL>
-
      <li>Added tpl parameter for specifying an arbitrary xml template
+
  <UL>
-
        file without needing to add it to the perl hash and without affecting
+
    <LI> DLXSApp.pm
-
        program flow (using page param instead of tpl can affect program flow
+
       <UL>
-
        in an undesireable way). <strong>[DlpsUtils.pm]</strong></li>
+
         <LI>emit full path to main XML template
 +
          for &lt;TemplateName> element</LI>
 +
         <LI>changed TemplateName to TemplatePath</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> DbUtils.pm
 +
       <UL>
 +
         <LI>added random id generation for bookbagdb
 +
          table. not using auto increment anymore. avoiding
 +
          potential replication conflicts.</LI>
 +
        <LI>changed related to the upgrade of
 +
          dbd::mysql. basically, stop throwing the utf8 switch
 +
          on data coming out of mysql.</LI>
-
      <li>Code changes were made in support of collid column addition to ItemBrowse
+
        <LI>adding quoting of table names, though
-
        table. <strong>[Browse Related Scripts] </strong></li>
+
          not exhaustively.</LI>
-
      <li>Fixed a bug and improved the general situation regarding removal of
+
        <LI>fixed a bug. optimizeTable would in
-
        articles and punctuation from the beginning of browse strings. The bug
+
          some cases alter column to be varchar with not enough
-
        had to do with<br>
+
          chars.</LI></UL></LI>
-
the hanlding of multiple field values. <strong>[Browse Related Scripts]</strong></li>
+
  </UL>
-
      <li>Implemented &quot;purge&quot; to be able to remove a collection's rows
+
-
      from the Browse tables without repopulating them. <strong>[Browse Related
+
-
      Scripts]</strong></li>
+
-
       <li><strong>OAITransform</strong> has been enhanced to handle character encoding problems
+
  <UL>
-
         that may exist in incoming data. </li>
+
    <LI> DlpsSession.pm
-
       <li>The newly added DLXS statistics system consists of two parts: (1)
+
       <UL>
-
         a tool to run on each web server to parse web log files, calculate hits,
+
        <LI>Changed GetCookieDomain to use $cgi->virtual_host()
-
         and insert those hits into the database, and (2) a web interface for
+
          instead of $ENV{HTTP_HOST} because the env variable
-
       retrieving reports such as HTML or MS Excel files.</li>
+
          includes the port but $cgi->virtual_host does not.
-
     </ul>
+
          Port number should not be specified in cookie domains.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> DlpsUtils.pm
 +
     
 +
      <UL>
 +
        <LI>Changed SoftASSERT so that it forwards
 +
          the return value from ASSERT.</LI>
 +
        <LI>new feedback cgi code. generalized
 +
          for use outside of just mdp.</LI>
 +
        <LI>remove duplicate code</LI>
 +
        <LI>Add RemoveXMLPi for Kwic processing</LI>
 +
        <LI>Added a simple routine called DebugOut
 +
          that prints a string within a div (class=debug). Optionally
 +
          include the name of the sub/method being debugged
 +
          just by throwing a switch.</LI>
 +
 
 +
        <LI>Moved binmode statement out of dlpsutils
 +
          (where it messed up image class, probably due to double
 +
          encoding) and put it in collmgr where needed most.</LI>
 +
         <LI>Implementation of session stored return
 +
          login url. Ran into a problem with long URLs in IE,
 +
          so had to do this.</LI>
 +
 
 +
        <LI>Added sub CarryOverNeededAuthRetParams.
 +
          (All other params were being dropped when processing
 +
          a key_authret param, making Athens auth fail.)</LI>
 +
        <LI>no longer deleting key_authret from
 +
          session once used, because in rare cases it is used
 +
          again.</LI>       
 +
        <LI>make no referring URL "Assertion Fail Report" not send
 +
          email</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> ItemView.cfg
 +
     
 +
      <UL>
 +
        <LI>add note about pdfopt</LI>
 +
        <LI>add defaults for jpg as a image format
 +
          source type</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> ItemView.pm
 +
      <UL>
 +
        <LI>Add code to set and get the $targetImageFormat
 +
          so that the DeliverFileHandler knows what format to
 +
          deliver. It has a different signature that asks for
 +
          the format but the handlers are called generaiclly
 +
          and do not receive the format as a parameter form
 +
          the caller so ...</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
   
 +
    <LI> ItemViewApp.pm
 +
      <UL>
 +
        <LI>Set defaultpageview if no view is set
 +
          in ValidityChecks</LI>
 +
        <LI>fix path to cached image to include
 +
          host when mode is remote</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> LibGlobals.cfg
 +
      <UL>
 +
        <LI>new feedback cgi code. generalized
 +
          for use outside of just mdp.</LI>
 +
      </UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> LibVersion.pm
 +
      <UL>
 +
        <LI>$LibVersion::VERSION = 4.1201; $LibVersion::DLXS_CDROM
 +
          = '14';</LI>
 +
        <LI>14</LI></UL></LI>
 +
   
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> ProcIns.pm
 +
      <UL>
 +
        <LI>Allow string refs for handlers that
 +
          are subroutines as has well as those that are object
 +
          emthods</LI>
 +
        <LI>Removed StripPIPairContents etc. obsolete</LI>
 +
        <LI>add GetPIs</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> XPat.pm
 +
      <UL>
 +
        <LI>m,,gs to get all the error messages
 +
          back from xpat</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> XsltPIFiller.pm
 +
      <UL>
 +
       
 +
        <LI>url for Atom/Rss</LI>
 +
        <LI>changed TemplateName to TemplatePath</LI>
 +
        <LI>changed TemplateName to TemplatePath</LI>
 +
        <LI>removed setting of utf8flag for browse
 +
          list data.</LI></UL></LI>
 +
   
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> stripdoctype.pl
 +
       <UL>
 +
        <LI>regex will strip complicated DOCTYPE
 +
          declarations</LI>
 +
       
 +
        <LI>moved to $DLXSROOT/bin/f/findaid instead
 +
          of ../lib since this is findaid specific</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> BookBag/BookBagIC.pm
 +
      <UL>
 +
        <LI>added new method for checking ownership
 +
          of bookbag</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> BookBagItem/BBItemTC.pm
 +
      <UL>
 +
        <LI>Use maintitle, mainauthor, mainheader
 +
          to add to bookbag</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> DLXSApp/FullTextApp.pm
 +
      <UL>
 +
        <LI>added podBindings sub</LI>
 +
 
 +
        <LI>fix for debug=xsltwrite without remote
 +
          user. Now using REMOTE_USER if defined or $ENV{'DLPS_DEV'}
 +
          if its defined and not 1 otherwise using "anonymous"</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> DLXSClass/FullTextClass.pm
 +
     
 +
      <UL>
 +
        <LI>Removed line that was stripping out
 +
          spaces in idno in RestrictBaseQuery. This was causing
 +
          kwic view queries where the &lt;eadid> contained a space
 +
          to fail.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Config.pm
 +
      <UL>
 +
        <LI>Explicitly specify the format of the
 +
          config file.</LI>
 +
        <LI>Add method to return config file name.</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> Stats/DB.pm
 +
      <UL>
 +
        <LI>Add method delete_expired_counter_requests()</LI>
 +
         <LI>Modify query to add a COUNTER job to
 +
          use 99999 if institution is undefined.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/MainPage.pm
 +
      <UL>
 +
        <LI>State that COUNTER reports are available
 +
          starting in May 2007.</LI>
 +
       
 +
        <LI>Big rewrite to display aggregate stats
 +
          rows. Also combined some code and got rid of the extra.</LI>
 +
        <LI>New code to do COUNTER Database Report
 +
          1; also, auth algorithm changed again to show aggregate
 +
          stats *and* stats for a user's institution if their
 +
          institution can be determined from their IP address.</LI>
 +
         <LI>Add an additional message to COUNTER
 +
          request feedback.</LI>
 +
        <LI>Correct name of DR1.</LI>
 +
        <LI>Don't provide DR1 yet - its not ready.</LI>
 +
        <LI>Show COUNTER section in form if user
 +
          is logged in but IP address is unrecognized.</LI>
 +
       
 +
        <LI>Slight modification: If we know the
 +
          user's institution, but they do not have stats for
 +
          a given collection and time period, default to 0 for
 +
          all counts so a row is still displayed for them in
 +
          the results page with zeros in all fields.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Resource.pm
 +
      <UL>
 +
        <LI>Get 'public' attribute from config
 +
          files to determine if a collection is public or not.</LI>
 +
        <LI>Allow upper- or lower-case for report
 +
          types in config file.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Map/Auth.pm
 +
     
 +
      <UL>
 +
        <LI>Constructor now takes Stats::Config
 +
          object.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Map/Titles/colldb.pm
 +
      <UL>
 +
        <LI>Temporarily set $ENV{'HTTP_HOST'} to
 +
          $$self{'host'}, which is set from the query "SELECT
 +
          DISTINCT host FROM Collection WHERE userid = 'production'
 +
          and collid = ..." in case $ENV{'HTTP_HOST'} is not
 +
          set.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER.pm
 +
      <UL>
 +
       
 +
        <LI>Include support for COUNTER Database
 +
          Report 1.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER/BR1.pm
 +
      <UL>
 +
        <LI>Clean up Database Report 1 a bit.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER/BR2.pm
 +
      <UL>
 +
        <LI>Clean up Database Report 1 a bit.</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER/BR5.pm
 +
      <UL>
 +
        <LI>Clean up Database Report 1 a bit.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER/DR1.pm
 +
      <UL>
 +
        <LI>Add module for COUNTER Database Report
 +
          1.</LI>
 +
        <LI>Clean up Database Report 1 a bit.</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER/Report1.pm through Stats/Reports/COUNTER/Report5.pm
 +
      <UL>
 +
        <LI>Pass Stats::Config object to COUNTER
 +
          creation methods rather than creating other Stats::Config
 +
          files within these methods.</LI>
 +
        <LI>Now must pass Stats::Config object
 +
          to Stats::Map::Auth constructor.</LI>
 +
        <LI>Return undef if Resource object is
 +
          not created in the 'match_collid_to_resource()' method
 +
          - which, in this case, happened because I had removed
 +
          the collid from the stats config.</LI>
 +
        <LI>Clean up Database Report 1 a bit.</LI></UL></LI>
 +
   
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> Stats/Reports/COUNTER/ReportTools.pm
 +
      <UL>
 +
        <LI>Die if location of COUNTER reports
 +
          cannot be obtained from the config file.</LI>
 +
        <LI>Pass Stats::Config object to COUNTER
 +
          creation methods rather than creating other Stats::Config
 +
          files within these methods.</LI>
 +
        <LI>Include support for COUNTER Database
 +
          Report 1.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/Custom/Excel.pm
 +
        
 +
      <UL>
 +
        <LI>Big rewrite to include aggregate stats
 +
          rows and use the same struct that is created for HTML
 +
          reports.</LI>
 +
        <LI>Get rid of 'aggregate_only' concept.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Reports/Custom/Tools.pm
 +
      <UL>
 +
        <LI>Remove file - no longer needed</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
   
 +
    <LI> Stats/Tools/AuthTools.pm
 +
      <UL>
 +
        <LI>Just added comment.</LI>
 +
        <LI>Now all users are allowed, but if they
 +
          are not recognized then they can only see aggregate
 +
          stats for public collections.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Tools/DateTools.pm
 +
      <UL>
 +
        <LI>Fix bug in day() method that returns
 +
          the current day of the month (it was erroneously adding
 +
          1).</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/Tools/ResourceTools.pm
 +
      <UL>
 +
        <LI>Big modification to allow unrecognized
 +
          users to see public collections (aggregate stats only).</LI>
 +
        <LI>Bug fix: empty the list of coll ids
 +
          previously built before grabbing all coll ids in the
 +
          cache, because we don't want to include any that aren't
 +
          in the cache (but were, say, in the AUTHZD_COLL variable).</LI>
 +
        <LI>When doing regular expression with
 +
          coll IDs, use /^..$/ to get the exact coll ID.</LI>
 +
        <LI>Get rid of 'aggregate_only' concept.
 +
          Return an array of all coll ids for which the user
 +
          is an admin.</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> Stats/XML/BuildXML.pm
 +
      <UL>
 +
        <LI>This is now used for the Excel custom
 +
          reports as well as the HTML reports. Now includes
 +
          aggregate stats rows.</LI>
 +
        <LI>Get rid of 'aggregate_only' concept.</LI>
 +
        <LI>utf-8 encode collection name before
 +
          passing to LibXML method.</LI>
 +
        <LI>Fix use of utf8::upgrade for collection
 +
          name</LI>
 +
       
 +
        <LI>Slight modification: If we know the
 +
          user's institution, but they do not have stats for
 +
          a given collection and time period, default to 0 for
 +
          all counts so a row is still displayed for them in
 +
          the results page with zeros in all fields.</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> upgrade_6_7
 +
      <UL>
 +
        <LI>Initial</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> upgrade_6_7.cfg
 +
      <UL>
 +
        <LI>Initial</LI>
 +
 
 +
        <LI>add nameresolver, oai, oaisets table
 +
          creation</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> BrowseUpdate.pm
 +
      <UL>
 +
 
 +
        <LI>Insertion is now skipped if browse
 +
          field has no value. Also, field value counts are tallied
 +
          and reported.</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> browseutils.cfg
 +
      <UL>
 +
        <LI>added "an"</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> ub
 +
      <UL>
 +
        <LI>more input checking</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> updatebrowsedb.pl
 +
      <UL>
 +
 
 +
        <LI>Insertion is now skipped if browse
 +
          field has no value. Also, field value counts are tallied
 +
          and reported.</LI>
 +
        <LI>more input checking</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> BrowseUpdate/ImageMysqlBU.pm
 +
     
 +
      <UL>
 +
        <LI>Added a comment about browse_split.</LI>
 +
        <LI>commented out a debug statement.</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> main.cfg
 +
      <UL>
 +
        <LI>remove misc/db CSV related opertations</LI>
 +
 
 +
        <LI>remove replace( "$DLXSROOT/bin/i/image/html2tab.pl",
 +
          qq{\#!/l/local/bin/perl}, $perlhashbang );</LI>
 +
        <LI>remove replace( "$DLXSROOT/bin/i/image/tab2html.pl",
 +
          qq{\#!/l/local/bin/perl}, $perlhashbang );</LI>
 +
        <LI>restore Text::CSV_XS</LI>
 +
        <LI>remove ["Net::hostent", 1.01] it is
 +
          part of Perl</LI>
 +
        <LI>update Text::CSV_XS should be 0.23</LI>
 +
 
 +
        <LI>Removed all references to outdated
 +
          validate.pl script Changed references for samplefa.*inp
 +
          files to new file names samplefa.ead2002.dcl and samplef.concat.ead.dcl</LI>
 +
        <LI>fixed move of fixdoctype.pl from s/samplefa
 +
          to f/findaid</LI>
 +
        <LI>remove replace( "$DLXSROOT/bin/f/findaid/isolat128bit.pl",
 +
          qq{\#!/l/local/bin/perl}, $perlhashbang );</LI>
 +
        <LI>removed sampletc obsolete (now just
 +
          sampletc_utf8)</LI>
 +
        <LI>add setup_newcoll for findaidclass</LI>
 +
        <LI>Added section for findaid demofa collection</LI>
 +
        <LI>["XML::LibXSLT", 1.63], ["XML::LibXML",
 +
          1.65], from ["XML::LibXSLT", 1.58], ["XML::LibXML",
 +
          1.58],</LI>
 +
       
 +
        <LI>sru, broker20 optional, broker20 deprecated</LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
     <LI> dlxsglobals.xml
 +
      <UL>
 +
        <LI>url for Atom/Rss</LI>
 +
        <LI>changed TemplateName to TemplatePath</LI>
 +
       
 +
        <LI>changed TemplateName to TemplatePath</LI>
 +
        <LI>mtag changes</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> feedback.xsl
 +
      <UL>
 +
        <LI>new feedback cgi code. generalized
 +
          for use outside of just mdp.</LI>
 +
        <LI>these files were moved to web/i/image.</LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> transform.xml
 +
      <UL>
 +
        <LI>add &lt;xsl:strip-space/></LI>
 +
        <LI>&lt;xsl:strip-space elements="*"/></LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> transform_xhtml.xml
 +
      <UL>
 +
       
 +
        <LI>add &lt;xsl:strip-space/></LI>
 +
        <LI>&lt;xsl:strip-space elements="*"/></LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> transform_xml.xml
 +
      <UL>
 +
        <LI>add &lt;xsl:strip-space/></LI>
 +
        <LI>&lt;xsl:strip-space elements="*"/></LI></UL></LI>
 +
   
 +
  </UL>
 +
  <UL>
 +
    <LI> xsl2htmlutils.xsl
 +
      <UL>
 +
        <LI>bboru changes via email from Wisconsin:
 +
          added test for value of key('get-lookup',Label)
 +
          else it returns an empty value if there is a dot in
 +
          the Label value, as in 'etc.' &lt;xsl:when test="contains(Label,'.')
 +
            and key('get-lookup',Label)">  &lt;xsl:when test="contains(Label,'.')">
 +
            use 'selected' for valid xhtml
 +
            &lt;xsl:attribute name="selected">selected&lt;/xsl:attribute>
 +
            &lt;xsl:attribute name="selected">true&lt;/xsl:attribute>
 +
        </LI></UL></LI>
 +
  </UL>
 +
 
 +
  <UL>
 +
    <LI> css/feedback.css
 +
     
 +
      <UL>
 +
        <LI>new feedback cgi code. generalized
 +
          for use outside of just mdp.</LI>
 +
        <LI>these files were moved to web/i/image.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> js/feedback.js
 +
      <UL>
 +
        <LI>new feedback cgi code. generalized
 +
          for use outside of just mdp.</LI>
 +
        <LI>these files were moved to web/i/image.</LI></UL></LI>
 +
   
 +
  </UL>
===TextClass===
===TextClass===
-
    <h4>XML template and XSL stylesheet changes</h4>
+
'''web/t/text'''
-
    <ul>
+
  <UL>
-
      <li>browse.xml - Added &lt;BrowseLevels&gt;&lt;?BROWSE_LEVELS_XML?&gt;&lt;/BrowseLevels&gt;</li>
+
-
      <li>browse.xsl - Changes for subject browse and special 'browsefields' syntax to support browse levels per
+
-
                field, e.g. subject=0, author=1</li>
+
-
       <li>navheader.xsl - Avoid adding empty "tips=" parameter to url for navbar links to fix problem where additional url params tacked onto end create invalid syntax e.g. ...tips=target=_top</li>
+
  </UL>
 +
  <UL>
 +
    <LI> browse.xml
 +
       <UL>
 +
        <LI>added browseextra.xsl inc</LI>
 +
</UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> clipviewer.xml
-
       <li>results.xsl - Stub template for PDF link in reslist, desired. Handle common XML tags being passed through in KWICS. (Formerly handled in TextClass::CleanResidualTags.)</li>
+
       <UL>
 +
        <LI>added some xsl incl</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> header.xsl
 +
      <UL>
 +
        <LI>added code to allow for print on demand
 +
          links to amazon.com from podPermittedItems</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> htmlhead.xsl
 +
      <UL>
-
      <li>resultsheader.xsl -  Check for BIBLSCOPE[@TYPE='pageno'] in definePubInfoForSerialIssue.  Don't display BIBLSCOPE TYPE="datesort". </li>
+
        <LI>emit full path to main XML template
-
      <li>scopedivs.xsl - Support DATE in HEADs with new DATE template. In &lt;template match="Divhead"&gt;: removed extraneous table row. Cleaned up redundant code that was also in &lt;template name="BuildDivHeadLinkLabel"&gt;  Use BIBLSCOPE[(@TYPE='pg' or @TYPE='pageno')] instead of just pg. Separate multiple AUTHORINDs with semicolons.</li>
+
          for &lt;TemplateName> element</LI>
-
      <li>search.xsl - Move tips to below the search form so the iframe is not squeezed into a tiny box</li>
+
        <LI>changed TemplateName to TemplatePath</LI>
-
      <li> text.components.xsl -  Filter for LIST in filterNumberedNoteWithParas. Pass through value of COLSPAN in table cells.</li>
+
        <LI>If we're viewing a single item, put
 +
          its title in the HTML head.</LI>
 +
        <LI>Same as above, but also for view=trgt
 +
          and page=root.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> langmap.en.xml
-
       <li>tocheader.xsl - Correct formatting for multiple authors.  Fixed formatting for multiple authors in printsourcestmt in OutputHeader.  Made inclusion of BookmarkableUrl conditional on string not being empty. Don't display label for BIBLSCOPE TYPE="pg" if the element is empty. Multiple authors, editors formatting handled in textheader.xsl Copied AUTHOR and EDITOR templates from textheader.xsl.</li>
+
       <UL>
-
    </ul>
+
        <LI>add feature PRF (Preface) to langmaps</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> langmap.fr.xml
 +
      <UL>
 +
        <LI>add feature PRF (Preface) to langmaps</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> pageviewerheader.xsl
 +
      <UL>
-
====CGI/Middleware====
+
        <LI>Stub template for ProcessSerialarticle.</LI></UL></LI>
-
     <ul>
+
  </UL>
-
       <li> TextApp.pm - Backward compatibility for pagenname=browseentries/Na.xml. Removed ValidityChecks setting of default value for browse. It is set when building the browse page when we know more about what is available instead of just selecting 'a'</li>
+
  <UL>
-
      <li> TextClass.pm -  Allow parameter $idroot to sub FigureIDResolver to indicate extra dir level in image filepath, e.g. web/c/coll/images/idroot/idno.jpg. Extensive re-write of  entity resolution in &lt;FIGURE ENTITY=...&gt; to support more types of resolution.  Better error message when a query failure occurs fetching the text of a pageviewer page. TextClass::Filter_REFsForText now handles more kinds of REF targets. BEG Change regexp in FilterPBs_XML to accept &lt;PB .../&gt; or &lt;PB ...&gt;&lt;/PB&gt; since both are valid XML. Filter_REFsForText: check for TYPE="ptr" instead of assuming every REF with a TARGET attr is of the same type. Optimize Filter_REFsForText for case of no &lt;REF ..&gt; elements. Fix bug: Empty defaultpageview field is ok if pageimages field is empty or 0 in collmgr. Better error message in GetItemEncodingLevel()</li>
+
     <LI> results.xsl
 +
       <UL>
 +
        <LI>No more STATUS="hidden".</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> resultsheader.xsl
 +
      <UL>
 +
        <LI>BIBLSCOPE filtering: made like tocheader.xsl
 +
          (no comma before issuetitle).</LI>
-
    </ul>
+
        <LI>&lt;xsl:value-of select="$pubinfo"/> -->
 +
          &lt;xsl:copy-of select="$pubinfo"/> to write child nodes
 +
          (e.g. &lt;div>) to the HTML.</LI>
 +
        <LI>No comma after &lt;div>[issuetitle]</div>
 +
          for serialissue.</LI>
 +
        <LI>minor change to fix missing name search
 +
          bug</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> scopedivs.xsl
 +
      <UL>
 +
        <LI>TYPE="hidden" allows selective non-display
 +
          of DIVn (and all descendents) in TOC. (Note distinction
 +
          from mis-named STATUS="hidden".)</LI>
 +
        <LI>STATUS="hidden" now actually hides
 +
          the DIV. No more TYPE="hidden".</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> text.components.xsl
 +
      <UL>
 +
        <LI>FilterNumberedNotesWithParams: wrap
 +
          paragraphs after the first in &lt;p>.</LI>
 +
 +
        <LI>LG template calls template name="addRend".</LI>
 +
        <LI>In &lt;xsl:template match="P">, don't
 +
          normalize value of @ID.</LI>
 +
        <LI>More robust filtering for filterNoteWithParas.</LI>
 +
        <LI>Handle CELL/@ROWSPAN.</LI>
 +
        <LI>In &lt;xsl:template match="REF">, removed
 +
          special case for PARENT::ITEM which appeared to be
 +
          abandoned code.</LI>
 +
 +
        <LI>For PBs, Wrap the DIV in an OBJECT
 +
          so that we still have valid XHTML in the event that
 +
          we're currently inside a P.</LI>
 +
        <LI>OBJECT wrapper around PB text breaks
 +
          in Safari; better to use span with display: block.</LI>
 +
        <LI>Add anchors to ITEMs which have IDs.</LI>
 +
        <LI>Create anchors for ID'd Ps in filterNumberedNoteWithParas.</LI>
 +
        <LI>In filterNumberedNoteWithParas, check
 +
          whether we're in NOTE2 (in addition to NOTE1).</LI>
 +
        <LI>XML table elements (TABLE, ROW, CELL)
 +
          get a class applied to them, obscuring any REND styles
 +
          in the markup. I've changed code to apply the value
 +
          of @REND to the class value, e.g. &lt;td class="xmltd-rend-center">.</LI>
 +
        <LI>L: pass forward all @RENDs, nit just
 +
          ones that start with "line".</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> textclass.css
 +
      <UL>
 +
        <LI>Added rend-plain.</LI>
 +
        <LI>Added rend-isub.</LI>
 +
        <LI>Added .rend-rightjustify.</LI>
 +
 +
        <LI>Changed .pbtext.</LI>
 +
        <LI>List style .nomarker.</LI>
 +
        <LI>Add margin-bottom to div.lg and the
 +
          like.</LI>
 +
        <LI>rend-aligntop.</LI></UL></LI>
 +
 +
  </UL>
 +
  <UL>
 +
    <LI> tocheader.xsl
 +
      <UL>
 +
        <LI>BIBLSCOPE filtering.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> viewer.utils.xsl
 +
      <UL>
 +
        <LI>minor change to fix missing name search
 +
          bug</LI></UL></LI>
 +
  </UL>
 +
 +
  <UL>
 +
    <LI> viewtextnote.xsl
 +
      <UL>
 +
        <LI>remove crash your browser</LI>
 +
        <LI>OCR quality</LI></UL></LI>
 +
  </UL>
 +
 +
'''bin/t/text'''
 +
 +
  <UL>
 +
    <LI> CER.pm
 +
 +
      <UL>
 +
        <LI>added dagger2 &amp;#x2021; labr &#x3008;
 +
          long &amp;#xAF; rabr &#x3009; short &#x2D8;</LI></UL></LI>
 +
  </UL>
 +
 +
  <UL>
 +
    <LI> dtdalyzer.pl
 +
      <UL>
 +
 +
        <LI>Added ROWSPAN attribute.</LI>
 +
        <LI>Added DOC attr.</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> utf8chars
 +
 +
      <UL>
 +
        <LI>fix error checking and display data</LI></UL></LI>
 +
  </UL>
 +
 +
'''cgi/t/text'''
 +
 +
  <UL>
 +
    <LI> CVApp.pm
 +
      <UL>
 +
        <LI>added pgseq to cache filename and change
 +
          default view to pdf</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> DlpsLocalUtils.pm
 +
      <UL>
 +
        <LI>Refactored LocalIdResolver(), moving
 +
          substantial code into CreatePicklist().</LI>
 +
 +
        <LI>CreatePicklist: delete unwanted params
 +
          from tempCgi that may be hanging around (possibly
 +
          due to URL hacking) and that we don't want passed
 +
          forward in picklist links.</LI></UL></LI>
 +
  </UL>
 +
 +
  <UL>
 +
    <LI> TextAppXsltPIFiller.pm
 +
 +
      <UL>
 +
        <LI>print on demand links to amazon.com
 +
          added</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> TextClass.pm
 +
      <UL>
 +
        <LI>Add RemoveXMLPi for Kwic processing</LI>
 +
        <LI>Filter_REFsForText: treat &lt;REF TYPE="txt">
 +
          same as &lt;REF TYPE="ptr">.</LI>
 +
 +
        <LI>GetDateParsePattern: optional hyphens
 +
          between year-month-day parts of a sortdate.</LI>
 +
        <LI>Output FirstPageHref for layer2 serialissue
 +
          results.</LI>
 +
        <LI>minor change to fix missing name search
 +
          bug - close div at end of div1headbib</LI>
 +
        <LI>another change to fix missing NODE
 +
          param in search error</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> textclass.cfg
 +
      <UL>
 +
        <LI>added config to allow POD db lookups</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> ClipView/mdailyCV.pm
 +
      <UL>
 +
        <LI>new</LI>
 +
</UL></LI>
 +
  </UL>
===ImageClass===
===ImageClass===
Line 139: Line 831:
<p>Known Problems</p>
<p>Known Problems</p>
<ul>
<ul>
 +
<li>None known so far.</li>
 +
</ul>
-
<li>Metadata fields containing xml must be mapped to ic_xml in the field_admin_maps Collection Manager field. AND, there is a bug in the released version of the data2db.pm perl module that causes fields containing XML to be skipped completely. <a href="/products/archive-by-CDROM/13/Patches">Download</a> the data2db.pm patch file.</li>
+
<p>Enhancements </p>
-
<li>July 11, 2007. data2db.pm parser for FMP DSO XML failed to handle fields where the name of the field contains special regular expression characters that needed to be quoted. <a href="/products/archive-by-CDROM/13/Patches">Download</a> the data2db.pm patch file.</li>
+
<ul>
-
<li>TEXT::CSV_XS Perl module must be installed (available in CPAN) in order for the image class data loading script to perform.</li>
+
<li>Image Viewing
-
<li>The owner of a session based/temporary portfolio is not allowed to open it. For example, a non-authenticated user adds an image to a new portfolio and the user is not allowed to see the portfolio when it tries to open to display the new addition. To fix the problem, change the following lines of the <strong>GetBookBag</strong> method in <strong>$DLXSROOT/cgi/i/image/ImageApp.pm</strong>.
+
<ul>
 +
<li>Ajax based zooming and panning of imagery. Works with JPEG2000 and MrSID.</li>
 +
<li>getimage-idx cgi has been completely rewritten and is backward compatible with previous version.</li>
 +
<li>XML, XSL, CSS, and Javascript for entry/image view have changed significantly.</li>
 +
<li>Running mediaprep with purge=1 is recommended, but not required. It now stores thumbnail dimentions in the ImageClassMediaFiles table and uses them for drawing zooming reference visuals on the thumbnail adjacent to zoomable images. Image Class now requires ImageMagick and PerlMagick to be installed.</li>
 +
</ul>
 +
</li>
 +
<li>Portfolios
 +
<ul>
 +
<li>Longer descriptions are allowed in custom sorting display.</li>
 +
<li>Multiple owners/editors.</li>
 +
<li>Renaming.</li>
 +
<li>Added documentation for end-users.</li>
 +
<li>Fixed bug that kept anonymous user from opening their own session based portfolio.</li>
 +
<li>Portfolio (BookBag) IDs are now generated randomly to avoid potential database replication problems.</li>
 +
</ul>
 +
</li>
 +
<li>Searching/Browsing
 +
<ul>
 +
<li>Relevance ranking is now an option for Image Class search results. To activate simply add "relevance" to the Collmgr field "sortflds" (first list value should be "none" and second "relevance"). Sorting of results by relevance will be the default for the collection once configured. Relevance ranking is not used when searching multiple collections.</li>
 +
<li>Searching has been packaged and can be subclassed. A subclass for integrating search results from the ARTstor XML web service is included  and can be used by ARTstor members. Support for other services, databases or database schemas can be added by subclassing the ImageSearch.pm Perl module.</li>
 +
<li>Browsing of newly added or update media items is now possible. Requires reloading metadata for the collection. In Collmgr set brwsadds to "on", add "m_flm:::Recently Added/Updated" to field_labels, and add "m_flm" to sortflds, dfltentryflds, and dfltresentryflds.</li>  
 +
<li>All collections are selected by default in cross collection search. This is now a configurable option at the class level in imageclass.cfg.</li>
 +
</ul>
 +
</li>
 +
<li>Collection Size Calculation
 +
<ul>
 +
<li>Collection size counts are now stored, by image-idx, in Collmgr. See help text for new Collmgr fields "recordcount" and "mediacount".</li>
 +
</ul>
 +
</li>
 +
</ul>
-
<br><strong>Old</strong>
+
====Data Preparation====
-
<pre>
+
<p>Enhancements </p>
-
  if ( ( lc($ENV{'REMOTE_USER'}) ne lc($portfolioOwner) ) && ( ! $portfolioPublic ) )
+
<ul>
-
  {
+
<li>It is possible to configure a development mysql server for loading metadata. load.pl script will use production mysql server to get information from Collmgr, ImageClassMediaFiles, etc. but will populate new tables on development server. This is not generally necessary, but can help to reduce production server load with very large collections (75,000+ records).</li>  
-
  &DlpsUtils::errorBail( qq{Requested portfolio is not public. The owner may choose to make it so.} );
+
<li>mediaprep script now stores pixel dimensions of thumbnail images in ImageClassMediaFiles table. Pixel dimensions for thumbnails now appear in the relevant XML output of the middleware cgi (image-idx).  
-
  }
+
</ul>
-
</pre>
+
-
<strong>New</strong>
+
-
<pre>
+
-
  my $sessionid = &DlpsUtils::GetReadOnlySessionId();
+
-
  my $username = $ENV{'REMOTE_USER'} || qq{sid-} . $sessionid;
+
-
+
-
  if ( ( $username !~ m/^$portfolioOwner$/i ) && ( ! $portfolioPublic ) )
+
-
  {
+
-
  &DlpsUtils::errorBail( qq{Requested portfolio is not public. The owner may choose to make it so.} );
+
-
  }
+
-
</pre>
+
-
      </li>
+
===BibClass===
-
    </ul>
+
'''cgi/b/bib'''
 +
  <UL>
 +
    <LI> BibApp.pm
 +
      <UL>
 +
        <LI>added size as a common param</LI>
 +
        <LI>add size to SID if we have one</LI></UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> BibClassPerlFilters.pm
 +
      <UL>
 +
        <LI>BibClassPerlFilters::CollsFilter had
 +
          a minor bug in the sql query used to select the collection
 +
          name from the Collection table causing it to not always
 +
          select the right row because it wasn't consdering
 +
          the value of the DLPS_DEV environment variable. It
 +
          now correctly selects the production, release, or
 +
          user row.</LI>
 +
</UL></LI>
 +
  </UL>
 +
  <UL>
 +
    <LI> BibClassUtils.pm
 +
      <UL>
 +
        <LI>change openurl sub to be 1.0</LI>
 +
        <LI>change wording on openurl link</LI>
-
<b><p>Speed/Performance Enhancements </p></b>
+
        <LI>added support for icon in openurl links</LI></UL></LI>
-
  <ul>
+
   </UL>
-
  <li>Sliced viewing of portfolios was added to improve performance of large portfolios
+
-
  and to avoid getting throttled by the web server due to many thumbnail requests
+
-
  at once. This changed required some rearrangement of interface functions.</li>
+
-
   <li>Browse results are now sliced for better performance and a list of field
+
-
    values with occurrence counts is presented in the sidebar for further navigation
+
-
    within the browse results. </li>
+
-
   <li>Changed Image Class searching to use UNION of sql select statements rather
+
   <UL>
-
     than a temp table. The temp table was contributing to MySQL database replication
+
     <LI> bibclass.cfg
-
    errors.</li>
+
      <UL>
-
  <li>Thumbnails are now embedded in XHTML as a data:URI, reducing the number
+
        <LI>change openurl server</LI></UL></LI>
-
    of HTTP requests and improving performance. This does not work with IE, so,
+
  </UL>
-
    the common thumbnail linking method is used for IE. </li>
+
-
  <li>A limited number of search results are now cached on the session in raw
+
-
    unformatted form and as XML. Look-ahead cacheing is done as the end of the
+
-
    CGI run for the next slice as well. The cacheing improves overall performance.</li>
+
-
  </ul>
+
-
<p>Other</p>
+
-
<ul>
+
-
  <li>Made changes to <em>related views</em> to support
+
-
    direct media links.</li>
+
-
  <li>Minor xsl change related to entry display
+
===Oai===
-
    of real media and pdf links.</li>
+
-
  <li>Separated thumbnail XSL code out into new template
+
-
    for easy override.</li>
+
-
  <li>Made changes to avoid searching all
+
-
    collections when no collections are selected in group search mode. It is now
+
-
      necessary for the user to select at least one collection.</li>
+
-
  <li>Improved handling
+
-
        of missing portfolio items.</li>
+
-
  <li>CSS is used to change the background color of items just added to a
+
-
          portfolio.</li>
+
-
  <li>Enlarged the display size of the DIV that encloses each thumbnail
+
-
            in the results view. This reduces the clipping of metadata in some
+
-
          cases.</li>
+
-
  <li>Image Class was modified to use a database table (ImageClassMediaFiles)
+
'''cgi/o/oai/'''
-
          to store and retrieve technical metadata about images instead of storing
+
* oai, oai.cfg, oai_conf.xml, sample_config.xml, UMProvider.pm
-
        this information in text files on disk.</li>
+
** new -- new OAI data provider and accompanying scripts
-
  <li>Previously a record with multiple media items, each with a distinct
+
-
          caption, resulted in a single combined caption. Now captions are independently
+
-
          associated with images. Please see documentation for details of how to
+
-
          take advantage of this new functionality relating to <em>ic_vi </em>field
+
-
        mappings. </li>
+
-
  <li>Added ability to output xml (rather than xhtml or html, and without
+
-
          using debug=XML) after applying xslt to original xml.</li>
+
-
  <li>Improved retrieval of items when an exact match of the ID is not possible.
+
-
          For example, when an id stored in a portfolio fails to work as is, there
+
-
          are some simple things that can be done to find the right record in most
+
-
          cases.</li>
+
-
  <li>Fixed a bug that allowed private portfolios to be retrieved by ID,
+
-
          even if not the owner. This situation was not likely to occur in real
+
-
          use, and it did not make the content accessible, just the structure of
+
-
          the portfolio.</li>
+
-
  <li>Browse level configuration can now be done per field using Collmgr. </li>
+
'''bin/o/oai/'''
-
  <li>Image Class items can now be retrieved by filename alone if there is
+
-
          an exact match with an entry. </li>
+
-
  <li>Changed handling of captions in related views for situation where there
+
-
          is no image for the view to display the caption. This is helpful with
+
-
          a database that has a single record with &quot;views&quot; for digital
+
-
          and slide so that the slide has reasonable representation in the related
+
-
          views table.</li>
+
-
  <li>Added a status check on prep tables to make sure there are indexes before
+
-
          allowing the middleware to hit on them. This adds stability in certain
+
-
          environments, like Michigan's where MySQL replication is in use.</li>
+
-
  <li>Improved handling of AUTHZD_COLL list for comparision to entry auth
+
-
          value for allowing/denying access to full image.</li>
+
-
  <li>Enhanced getimage-idx to do a better job of finding an image where
+
* ConfirmPublicDomain.pl, loadOai.pl, mbooks_harvest_cron.pl, mbooks_update.pl, oai_conf.xml, OaiList.pl, OaiToDb.pl, RepositoryConfig.xml, revertOaiTbl.pl, updateMbooksOai.pl
-
          the viewid has changed for some reason, and a stale external link is
+
** new -- new OAI data provider and accompanying scripts
-
          being used.</li>
+
-
  </ul>
+
-
====Data Preparation====
+
-
<p>Speed/Performance Enhancements </p>
+
-
<ul><li>Image Class was modified to use a database
+
-
      table (ImageClassMediaFiles) to store and retrieve technical metadata
+
-
        about images instead of storing this information in text files on disk.
+
-
        Unofficially, metadata loading is 2-4 times faster than it was before
+
-
        because image filenaming checking is faster with the table.</li>
+
-
  <li>Tables are now locked during metadata loading. This speeds up the data
+
-
      loading.</li>
+
-
  <li>Consolidated table optimizations into single MySQL statement rather than
+
'''bin/o/oai/provider'''
-
    several. The same was done for index building. This is approach is much
+
-
      faster. </li>
+
-
  <li>Combined ic_all (all fields of the record) field is now made into a unique
+
-
    list of words to reduce the size of ic_all a little, and in turn improve
+
-
      search performance.</li>
+
-
</ul>
+
-
    <b><p>Other</p></b>
+
* AddCollToSource.pl, articlesToDc.xsl, bibClassToDc.xsl, CheckModifiedColls.pl, collmgrFullList.txt, ConvertToDc.pl, ConvertToUTF8.pl, dlxs_to_oaidc_cron.pl, exampleColls.xml, ExtractHeaders.pl, GenerateReport.pl, GetNewCollections.pl, LoadDB.pl, NewOAICollection.jpg, oai_new_coll_history.txt, oai_update_history_text.txt, oai_update.pl, README.txt, RunDlpsOaiConversion.pl, RunOaiConversion.jpg, RunOaiConversion.pl, textClassToDc.xsl, umprovider_flow_diagrams_and_examples.ppt, UMProviderFlow.jpg, updateCollConfig.pl
-
    <ul>
+
** new -- scripts to transform text, bib, and image class collections into oai_dc for use by the OAI data provider. See the README.txt file in this directory for a brief description of each file
-
      <li> Made changes to the key Image Class data loading scripts to add an
+
-
        option that allows data records to be loaded to an alternative (e.g.,
+
-
        development or prep) MySQL server. The option is configured globally
+
-
        in the icdbprep.cfg file, and can also be controlled with a command line
+
-
        parameter for load.pl, icdbprep.pl, fmpxml2mysql.pl (deprecated in favor
+
-
      of data2db.pl), and data2db.pl.</li>
+
-
      <li>The mediaprep script replaced imageprep as the utility for gathering
+
'''bin/o/oai/provider/logs'''
-
          information about media files for use with Image Class. mediaprep works
+
-
        with the new ImageClassMediaFiles table. </li>
+
-
      <li>Previously a record with multiple media items, each with a distinct
+
-
          caption, resulted in a single combined caption. Now captions are independently
+
-
          associated with images. Please see documentation for details of how to
+
-
          take advantage of this new functionality relating to <em>ic_vi </em>field
+
-
        mappings. </li>
+
-
      <li>Changed the way the ic_all field is assembled so that it includes &quot;istruct_caption_&quot; fields
+
-
      that are mapped to ic_all. </li>
+
-
      <li>Session timeouts during data loading are now avoided by making the
+
-
          sessions empty so that there is no attempt to write the session back
+
-
        to the session database.</li>
+
-
      <li>If a
+
* This directory houses the logs generated by the scripts in bin/o/oai/provider
-
    ic_id was &gt; 255, index would not get added. Now, it
+
-
    will, for those odd situations where an ic_id is so
+
-
    long (probably a data error).</li>
+
-
      <li>Case of filename field
+
-
    names was not handled properly. Uppercase fieldnames
+
-
    were ignored. This had been fixed. </li>
+
-
      <li>Changed &quot;splitrepeatingfield&quot; routine
+
-
    so that it no longer splits on a comma in an ic_vi (caption) or ic_fn (filename)
+
-
        field. ic_vi fields often include commas not meant as delimiters. </li>
+
-
      <li>Added automatic database reconnection. This is most likely to aid data
+
-
          loading processes in certain environments. This is actually a change
+
-
        to DbUtils.pm, but is most beneficial to Image Class. </li>
+
-
      <li>Removed a filter for binary files that
+
'''bin/o/oai/provider/reports-new-colls'''
-
    was getting confused about text PDFs and RAM files that simply enclose a URL
+
-
        for redirection.</li>
+
-
    </ul>
+
-
===BibClass===
+
* This directory houses the reports generated by bin/o/oai/provider/GetNewCollections.pl
-
    <ul>
+
-
      <li>Exploratory work in BibClass IMLS subclasses for clustering/categorization support.</li>
+
-
    </ul>
+
-
===Oai===
+
'''bin/o/oai/provider/reports-oai-update'''
-
    <ul>
+
* This directory houses the reports generated by bin/o/oai/provider/GenerateReport.pl
-
      <li></li>
+
 
-
    </ul>
+
'''bin/i/image/'''
 +
 
 +
* image2oai_dc.pl
 +
** new -- script for loading OAI tables with Image Class data in MySQL
 +
 
 +
'''bin/o/oaister/'''
 +
 
 +
* RepositoryConfig.cfg
 +
** new -- contains baseURLs and OAITransFixer.pm subscript calls
 +
 
 +
* oaitransform/CollsObj.cfg
 +
** new -- config for CollsObj.pm
 +
 
 +
* oaitransform/CollsObj.pm
 +
** new -- used for FMPro connection for OAIster
 +
 
 +
* oaitransform/DataConditioning.pm
 +
** Get rid of any remaining stray sp0t tags
 +
 
 +
* oaitransform/MODSTransform
 +
** Fix bug where MODS record was not ignored for re-exposure if no URL in record
 +
** Changed the tree removing metadata from the re-exposed XML, changed the language code mapping, added setSpec to about container
 +
** Swap dates in about container and change MODS loc.gov URL
 +
** Remove NS prefix and NS from metadata element
 +
** Remove all xmlns attributes as well
 +
** Add all orig setSpec to provenance element for MODS re-exposure
 +
** Hack to fix xsi: problem for Northwestern
 +
** Change to pick up URLs in identifier type=uri
 +
 
 +
* oaitransform/OAITransFixer.pm
 +
** new -- perl subscripts for fixing harvested data
 +
 
 +
* oaitransform/OAITransform
 +
** Added flag to skip DTD validation change, the PostXslt process to be after each small file, not the final large one
 +
** Added code to regenerate the perl config files from the XML config files each time this is run
 +
** Comment out LoadRepositoryLookTable() and changed SetCurrentArchive() to use XML config value
 +
** Move removing attribute to after data conditioning
 +
** Patch to fix latin1 regex problem
 +
** Unescape hex and dec entity refs
 +
** Added fixer call and other minor changes
 +
** Skip deleted records
 +
** Add skipped and deleted to repo count
 +
** Get the return from PreprocessXML bug fix
 +
** Semi opt for hex and dec ent refs
 +
** Added CollsDB
 +
 
 +
* oaitransform/browse.tpl
 +
** new -- template for building OAIster browse pages
 +
 
 +
* oaitransform/manageCollsDb.pl
 +
** new -- CGI script for managing FMPro DB for oaister
 +
 
 +
* oaitransform/mods-bibclass.xsl
 +
** Added copyrightDate YR
 +
** urls in own URL tags
 +
** Change to pick up URLs in identifier type=uri
 +
 
 +
* oaitransform/mods_repositoryNames.pl
 +
** Fixed amp;
 +
 
 +
* oaitransform/oai-bibclass3.xsl
 +
** Added tests for empty elements
 +
 
 +
* oaitransform/repositoryNames.pl
 +
** Add &amp;amp; fix
 +
 
 +
* scripts/Batch_UMHarvest
 +
** Add skip HTML flag for transform
 +
** Allow -x for transform
 +
** Added -a for incremental batch harvesting
 +
 
 +
* scripts/ListRecords
 +
** Error code checking
 +
** Do not exit on OAI error
 +
** Fixed log reporting
 +
** Make list of dirs, not files, to check for replacement records -- speed up incremental
 +
 
 +
* scripts/UMHarvester
 +
** Changed to use the XML config file; requires XML::LibXML now
 +
** Added wait flag
 +
** Check for sets cfg before trying to parse it, only loop on sets for list records
 +
** ID bug fix
 +
 
 +
* scripts/browse_check.pl
 +
** new -- checking FMPro repositories against browse pages
 +
 
 +
* scripts/id_check.pl
 +
** new -- checking FMPro repositories against incremental batch harvest
 +
 
 +
* scripts/rc_check.pl
 +
** new -- checking FMPro repositories against RepositoryConfig.cfg
 +
 
 +
* scripts/restoreHarvestedData.pl
 +
** new -- restores backup copy of repository dir
 +
 
 +
* scripts/startIndex.pl
 +
** new -- runs xpat and multirgn on xml obj files
 +
 
 +
===IdResolver===
 +
 
 +
'''cgi/i/idresolver/'''
 +
 
 +
<UL>
 +
<LI>idresolver</li>
 +
<UL><li>CGI script that returns a URL (marked up in XML) for an ID</li></UL>
 +
<LI>idresolver-nr</li>
 +
<UL><li>new -- CGI script that redirects a user from a persistent URL to one defined in the nameresolver database for an ID, can be used instead of cgi/b/bib/bibperm</li></UL>
 +
<LI>idresolver-srch</li>
 +
<UL><li>new -- CGI script that takes a text file of ids (one per line) and checks to see if they exist in the nameresolver database</li></UL>
 +
<LI>IDResolver.pm</li>
 +
<UL><li>Perl module that is used by each of the cgi scripts</li></UL>
 +
</UL>
 +
 
 +
'''bin/i/idresolver/'''
 +
 
 +
<UL><LI>cvstag.idresolver, rdist.class</li></UL>
 +
 
 +
'''bin/n/nameresolver/'''
 +
 
 +
<UL><LI>IdParser.pl</LI>
 +
<UL><li>new -- script used to populate the nameresolver database</li></UL>
 +
<LI>LoadLLMCIds.pl</LI>
 +
<UL><li>new -- this is a sample script of how you can use a .csv file to populate the nameresolver database</li></UL>
 +
<LI>CreateTable.pl, NRTable.sql</LI>
 +
<UL><li>new -- scripts for creating nameresolver table in database (which is already included in the release)</li></UL>
 +
<LI>new -- other sample files: DeepBlueShortIds.txt, getUpdatedNRfiles, InsertCrosscIssues.pl, N2TestExecution.txt, NRDevVsProdComparison.xls, README.txt, RemoveDeepBlueConflicts.pl,
 +
statusLoadingNRinProd.txt, TestIdParser.pl
 +
</LI>
 +
<UL><li>These files were used for DLPS collections and are provided only as additional resources. They are not necessary to the Idresolver/nameresolver configuration</li></UL>
 +
</LI>
 +
</UL>
===broker20===
===broker20===
Line 319: Line 1,077:
       <li>No changes.</li>
       <li>No changes.</li>
     </ul>
     </ul>
-
 
===Collmgr===
===Collmgr===
     <ul>
     <ul>
-
       <li>Labels on radio buttons now clickable.</li>
+
       <li>Supports version 7.0 database for DLXS release14.</li>
 +
      <li>default browse page for Text and Findaid Class now set from first item in Browsefields list</li>
 +
 
     </ul>
     </ul>
-
 
===FindaidClass===
===FindaidClass===
-
    <h4>XML template and XSL stylesheet changes</h4>
 
-
    <ul>
 
-
      <li>browse.xml - Added &lt;BrowseLevels&gt;&lt;?BROWSE_LEVELS_XML?&gt;&lt;/BrowseLevels&gt;</li>
 
-
      <li>browse.xsl - Changes for subject browse and special 'browsefields' syntax to support browse levels per field, e.g. subject=0, author=1</li>
 
-
      <li>navheader.xsl - Avoid adding empty "tips=" parameter to url for navbar links to fix problem where additional url params tacked onto end create invalid syntax e.g. ...tips=target=_top</li>
+
====Findaid Class Summary====
-
      <li>search.xsl - Move tips to below the search form so the iframe is not squeezed into a tiny box</li>
+
<ul>
-
      <li>results.xsl - BEG Fixed bug in hit summary creation for boolean search results.</li>
+
<li>Browse by subject added</li>
-
      <li>text.xml -  Remove TextClass PI doesn't belong &lt;DocEncodingType>&lt;?DOC_ENCODING_TYPE_XML?>&lt;/DocEncodingType></li>
+
<li>Default browse page now set in collmgr</li>
-
      <li>text.xsl -  Refer to results.str.returntoresults instead of results.str.22 so "return to results" link will show up.</li>
+
<li>Sample code and documentation included for accomodating non-Bentley encoding practices</li>
 +
<ul>
 +
<li>Code to use &lt;bioghist&gt; &lt;head&gt; tag for labels in TOC and in Finding Aid: 
 +
[[Mounting_a_Finding_Aids_Collection#Changing_the_Bioghist_labels_to_use_the_appropriate_.3Chead.3E_elements|Changing the Bioghist labels to use the appropriate head elements]]
 +
</li>
 +
<li>Code to add &lt;relatedmaterial&gt; &lt;separatedmaterial&gt; to TOC:  [[Mounting_a_Finding_Aids_Collection#Adding_sections_to_the_table_of_contents|Adding sections to the table of contents]]</li>
 +
</ul>
 +
<li>Data prep scripts reorganized and renamed</li>
 +
<li>New functionality for Makefile and preparedocs.pl</li>
 +
<li>New script to set up new collections [[setup_newcoll]]</li>
 +
<li>Bug fixes</li>
 +
</ul>
-
    </ul>
+
<h4>Prep scripts</h4>
 +
Data prep scripts have been reorganized and renamed.
 +
*New functionality for Makefile and preparedocs.pl
 +
*New script to setup new collections
-
    <h4>CGI/Middleware</h4>
+
'''New prep script: setup_newcoll'''
-
    <ul>
+
-
      <li> FindaidAppp.pm - Removed ValidityChecks setting of default value for browse. It is set when building the browse page when we know more about what is available instead of just selecting 'a'</li>
+
-
      <li>FindaidClass.pm - Added 'silent' optional parameter to ASSERT in dao resolution so an unresolvable dao does not prevent the entire page from rendering. Improved visibility of dao links and emit "[image not available]" for unresolved daos. Add 'silent' optional parameter to ASSERT so processing can continue but still give us an email warning. Move list of section heads that was hardcoded in FindaidClass::_initialize to findaidclass.cfg to make editing out those heads that are missing under sume EAD encoding practices.  Catch connect exception when doing idresolver calls. Fix FilterAllDaos_XML case where there is no href attribute to prevent infinite loop. </li>
+
$DLXSROOT/bin/f/findaid/setup_newcoll can be used to set up directories for new collections. For example, to set up the workshopfa collection based on samplefa (Assuming your $DLXSROOT variable is set)run this command:
-
    </ul>
+
  $DLXSROOT/bin/f/findaid/setup_newcoll -c workshopfa  -s $DLXSROOT/prep/s/samplefa/data
 +
 
 +
More information on the setup_newcoll script can be found by clicking [[setup_newcoll_manpage|here]] or invoking the man page:
 +
 
 +
$DLXSROOT/bin/f/findaid/setup_newcoll --man
 +
 
 +
'''New options for preparedocs.pl'''
 +
<p>The $DLXSROOT/bin/s/samplefa/preparedocs.pl script now takes several new arguments.</p>
 +
  ./preparedocs.pl --man  will give details
 +
of particular interest is the  preparedocs.pl  -i inputfilelist option which allows you to specify a file containing the full paths to the finding aids you wish to index.  This option can be used instead of the previous default behavior which was to recursively search the data directory and index all files in that directory.
 +
 
 +
'''Changes to Makefile'''
 +
<p>Addition of these new targets:</p>
 +
 
 +
;prepdocslist
 +
:Same as prepdocs but uses the -i inputfilelist to preparedocs.pl and by default reads a file $DLXSROOT/prep/c/collection/list_of_eads.
 +
;allbutprep
 +
:convenience target for use in conjunction with prepdocslist that does all the make steps except for the make prepdocs
 +
;index
 +
:convenience method that runs all 3 indexing steps
 +
 
 +
'''Changes to prep scripts'''
 +
 
 +
;$DLXSROOT/bin/s/samplefa
 +
:Moved generic bin files to f/findaid
 +
:Makefile modified
 +
:preparedocs.pl has new options
 +
 
 +
;$DLXSROOT/prep/s/samplefa
 +
:Renamed files
 +
<pre> samplefa.text.inp  to samplefa.ead2002.dcl
 +
samplefa.xml.inp to samplefa.concat.ead.dcl
 +
</pre>
 +
 
 +
:Removed:
 +
<pre> samplefa.inp
 +
</pre>
 +
:Added:
 +
<pre>  list_of_eads
 +
</pre>
 +
 
 +
;$DLXSROOT/bin/f/findaid
 +
:Removed:
 +
<pre> catsourcefiles.pl
 +
isolat128bit.pl
 +
validate.pl
 +
</pre>
 +
:Added/moved from bin/s/samplefa
 +
<pre> fixdoctype.pl
 +
stripdoctype.pl
 +
validateeach.sh
 +
        setup_newcoll
 +
</pre>
 +
 
 +
<H4>web/f/findaid</H4>
 +
 
 +
<UL>
 +
<LI> bookbagitemsstring_debug.xsl
 +
<UL>
 +
  <LI>For debugging in oxygen since the Oxygen
 +
                debugger uses Saxon and Saxon doesn't understand the
 +
                extensions</LI>
 +
</UL>
 +
</LI>
 +
</UL>
 +
 
 +
<ul>
 +
<li>browse.xsl</li>
 +
<ul>
 +
<li>Added code for subject browse</li>
 +
</ul>
 +
 
 +
</ul>
 +
 
 +
<UL>
 +
<LI> browseheader.xsl
 +
<UL>
 +
 
 +
<LI>bulk dates labelled</LI></UL></LI>
 +
</UL>
 +
<UL>
 +
<LI> htmlhead.xsl
 +
<UL>
 +
<LI>emit full path to main XML template
 +
                for &lt;TemplateName> element</LI>
 +
<LI>changed TemplateName to TemplatePath</LI></UL></LI>
 +
</UL>
 +
<UL>
 +
 
 +
<LI> text.components.xsl
 +
<UL>
 +
<LI>Modified template for processing c0x's
 +
                so that if there are two containers within a //c0x/did/
 +
                such as box/folder they will both show up in the proper
 +
                column.</LI>
 +
<LI>Highlighting fix. Replaced about 36
 +
                instances of &lt;xsl:value-of select="."/> with <xsl:apply-templates
 +
                select="*|text()"/> If value-of select gets highlighted
 +
                text in the context node: "text&lt;HIGHLIGHT>text</HIGHLIGHT>text
 +
                text" it will ignore the &lt;HIGHLIGHT> elemnts and just
 +
                render all the text. On the other hand the apply-templates
 +
                "*" will match the &lt;HIGHLIGHT> element and trigger
 +
                the appropriate highlighting template and the text()
 +
                nodes will get passed to the template that just outputs
 +
                the text</LI>
 +
<LI>Fixed code in template match=unittitle
 +
                mode=SimpleUnittitle because foreach was messing with
 +
                context Also change code for handling notes/scopeconent
 +
                in match=C01|c02... because it was an xsl:choose,
 +
                but notes and scopecontent are not mutually exclusive</LI>
 +
<LI>Changed code for handling notes/scopeconent
 +
                in match=C01|c02... now both did/note and note and
 +
                did/scopecontent and scopecontent will be rendered</LI>
 +
<LI>fixed bug in template match="list"
 +
                where there was a foreach and then value-of select
 +
                *|text() that needed to be value-of select="."</LI>
 +
<LI>space before unitdate value</LI>
 +
<LI>title styling</LI>
 +
<LI>optional labels for additional descriptive
 +
                material</LI>
 +
 
 +
<LI>hide sorting title</LI>
 +
<LI>sponsor</LI>
 +
<LI>Restored ADD to full-text view</LI>
 +
<LI>abstracts in dids</LI>
 +
<LI>Change template for index mode= add
 +
                so it doesn't produce 2 copies of any &lt;head> text.
 +
                Key is to limit the apply templates after we already
 +
                processed the head not to process the head again:
 +
                &lt;xsl:apply-templates select="*[not(self::head)]"/></LI></UL></LI>
 +
 
 +
</UL>
 +
 
 +
<H4>bin/f/findaid</H4>
 +
 
 +
*catsourcefiles.pl
 +
**No longer used for samplefa (replaced by preparedocs.pl) Also it contains bhl specific code. For bhl use the copy in $DLXSROOT/bin/b/bhlead which is in cvs.
 +
*fixdoctype.pl
 +
**Moved here from bin/s/samplefa No collection-specific customization These should work on any EAD that conforms to the ead2002.dtd
 +
*isolat128bit.pl
 +
**No longer used for findaids, since all findaids should be utf8 encoded
 +
*setup_newcoll
 +
**New script to set up directories  ./setup_newcoll --man for details
 +
*stripdoctype.pl
 +
**This library file is now used by preparedocs.pl and validateeach.sh (through $DLXSROOT/bin/s/samplefa/fixdoctype.pl)to correctly remove multiple line DOCTYPE declarations                and any entity references contained within them. It is a replacement for the one-line perl program previously used by those two programs.
 +
*validate.pl
 +
**Removed outdated file that worked on sgml files. FindaidClass now exclusively xml
 +
*validateeach.sh
 +
**Moved here from bin/s/samplefa No collection-specific customization Thise should work on any EAD that conforms to the ead2002.dtd
 +
 
 +
<H4>cgi/f/findaid</H4>
 +
 
 +
<UL>
 +
<LI> FindaidApp.pm
 +
<UL>
 +
<LI>remove FormatGuideFrame() obsolete
 +
                since the change to XML/XSL</LI></UL></LI>
 +
</UL>
 +
<UL>
 +
<LI> FindaidAppXsltPIFiller.pm
 +
<UL>
 +
<LI>Removed FormatOutlineResult_XML and
 +
                FormatOutlineFrame_XML as they are not called by any
 +
                code (or bound to any PIs).</LI></UL></LI>
 +
 
 +
</UL>
 +
<UL>
 +
<LI> FindaidClass.pm
 +
<UL>
 +
<LI>highlight hits in layer 1 result items</LI>
 +
<LI>Added highlighting to BuildItemTitle_XML
 +
                so highlighting will show up in title</LI>
 +
<LI>Fixed bug in FilterAllDaos_XML that
 +
                would not properly process daos with real hrefs and
 +
                would result in illegal xml being output when id resolver
 +
                is turned on.</LI></UL></LI>
 +
</UL>
 +
<UL>
 +
<LI> FindaidClass/ClementsmssFC.pm
 +
 
 +
<UL>
 +
<LI>removes "viewtextnote" speedbump</LI></UL></LI>
 +
</UL>
 +
 
 +
*FindaidClass/DemofaFC.pm 
 +
**Demo of subclassing
 +
**add relatedmaterial and separated material TOC heads
 +
**use &lt;head> tags for bioghist instead of bentley logic for TOC heads
 +
**change labels for several TOC heads
 +
 
 +
*FindaidClass/BioghistfaFC.pm
 +
**Demo of subclassing to use &lt;head> tags for bioghist instead of bentley logic
===XClass===
===XClass===
Line 354: Line 1,311:
     </ul>
     </ul>
-
===XClass===
+
===METS Pageturner and Collection Builder===
     <ul>
     <ul>
-
       <li>No changes.</li>
+
       <li>Continuing development work in Pageturner.  New application: Collection Builder allows users to add items to a personal collection via widgets in Pageturner.  These applications are not part of DLXS.  They use a different code base mainly under <code>DLXSROOT/{web,bin,cgi}/m/mdp</code> and <code>DLXSROOT/lib/App</code>.  Stub routines are required in Pageturner to abstract the database connections and an installation of Solr/Lucene is required to support the collection search in Collection Builder.  The user interface makes extensive use of Yahoo User Interfacen toolkit (YUI) functionality.</li>
     </ul>
     </ul>
-
===METS Pageturner/Collection Builder===
+
===SRU===
     <ul>
     <ul>
-
       <li>Ongoing development work.</li>
+
       <li>Added fielded searching to query -- not yet Level 1 or 2, though</li>
     </ul>
     </ul>
[[#top|Top]]
[[#top|Top]]

Current revision

Main Page > Release Notes for Current DLXS Release

Contents

[edit] General Information

TextClass is substantially identical to release 13 except for bug fixes as noted below. ImageClass provides new image viewing functionality. FindaidClass improves handling of the EAD DTD and includes subject browsing. BibClass is unchanged and is being de-emphasized.

Release 14 is comprised of:

[edit] Known Problems

  • None

[edit] Database Installation Notes

MySQL is the supported database type. In order to run DLXS you will need to have a MySQL server installed. Sample data is delivered in the form of a MySQL dump file which can be directly imported into a MySQL database. The database upgrade script (upgrade_6_7) operates only on a MySQL database. These issues are documented in detail in the installation instructions and the upgrade instructions.

[edit] New and Changed Functionality

[edit] XPAT

  • No changes.

[edit] Lib

  • AuthNZ.pm
    • IE needs escaped return login url parameter to be the last cgi param. So now it is.
    • Implementation of session stored return login url. Ran into a problem with long URLs in IE, so had to do this.
  • CollsInfo.pm
    • removed setting of utf8flag for data out of utf8 mysql table
    • Added StoreFieldValue routine. It allows middleware to update a Collmgr field value for a single collection and for the current user (production, release, dev/user). The benefit of doing it in Collsinfo is that the cio knows about the current user, making it easy to update only the current user's row.
  • DLXSApp.pm
    • emit full path to main XML template for <TemplateName> element
    • changed TemplateName to TemplatePath
  • DbUtils.pm
    • added random id generation for bookbagdb table. not using auto increment anymore. avoiding potential replication conflicts.
    • changed related to the upgrade of dbd::mysql. basically, stop throwing the utf8 switch on data coming out of mysql.
    • adding quoting of table names, though not exhaustively.
    • fixed a bug. optimizeTable would in some cases alter column to be varchar with not enough chars.
  • DlpsSession.pm
    • Changed GetCookieDomain to use $cgi->virtual_host() instead of $ENV{HTTP_HOST} because the env variable includes the port but $cgi->virtual_host does not. Port number should not be specified in cookie domains.
  • DlpsUtils.pm
    • Changed SoftASSERT so that it forwards the return value from ASSERT.
    • new feedback cgi code. generalized for use outside of just mdp.
    • remove duplicate code
    • Add RemoveXMLPi for Kwic processing
    • Added a simple routine called DebugOut that prints a string within a div (class=debug). Optionally include the name of the sub/method being debugged just by throwing a switch.
    • Moved binmode statement out of dlpsutils (where it messed up image class, probably due to double encoding) and put it in collmgr where needed most.
    • Implementation of session stored return login url. Ran into a problem with long URLs in IE, so had to do this.
    • Added sub CarryOverNeededAuthRetParams. (All other params were being dropped when processing a key_authret param, making Athens auth fail.)
    • no longer deleting key_authret from session once used, because in rare cases it is used again.
    • make no referring URL "Assertion Fail Report" not send email
  • ItemView.cfg
    • add note about pdfopt
    • add defaults for jpg as a image format source type
  • ItemView.pm
    • Add code to set and get the $targetImageFormat so that the DeliverFileHandler knows what format to deliver. It has a different signature that asks for the format but the handlers are called generaiclly and do not receive the format as a parameter form the caller so ...
  • ItemViewApp.pm
    • Set defaultpageview if no view is set in ValidityChecks
    • fix path to cached image to include host when mode is remote
  • LibGlobals.cfg
    • new feedback cgi code. generalized for use outside of just mdp.
  • LibVersion.pm
    • $LibVersion::VERSION = 4.1201; $LibVersion::DLXS_CDROM = '14';
    • 14
  • ProcIns.pm
    • Allow string refs for handlers that are subroutines as has well as those that are object emthods
    • Removed StripPIPairContents etc. obsolete
    • add GetPIs
  • XPat.pm
    • m,,gs to get all the error messages back from xpat
  • XsltPIFiller.pm
    • url for Atom/Rss
    • changed TemplateName to TemplatePath
    • changed TemplateName to TemplatePath
    • removed setting of utf8flag for browse list data.
  • stripdoctype.pl
    • regex will strip complicated DOCTYPE declarations
    • moved to $DLXSROOT/bin/f/findaid instead of ../lib since this is findaid specific
  • BookBag/BookBagIC.pm
    • added new method for checking ownership of bookbag
  • BookBagItem/BBItemTC.pm
    • Use maintitle, mainauthor, mainheader to add to bookbag
  • DLXSApp/FullTextApp.pm
    • added podBindings sub
    • fix for debug=xsltwrite without remote user. Now using REMOTE_USER if defined or $ENV{'DLPS_DEV'} if its defined and not 1 otherwise using "anonymous"
  • DLXSClass/FullTextClass.pm
    • Removed line that was stripping out spaces in idno in RestrictBaseQuery. This was causing kwic view queries where the <eadid> contained a space to fail.
  • Stats/Config.pm
    • Explicitly specify the format of the config file.
    • Add method to return config file name.
  • Stats/DB.pm
    • Add method delete_expired_counter_requests()
    • Modify query to add a COUNTER job to use 99999 if institution is undefined.
  • Stats/MainPage.pm
    • State that COUNTER reports are available starting in May 2007.
    • Big rewrite to display aggregate stats rows. Also combined some code and got rid of the extra.
    • New code to do COUNTER Database Report 1; also, auth algorithm changed again to show aggregate stats *and* stats for a user's institution if their institution can be determined from their IP address.
    • Add an additional message to COUNTER request feedback.
    • Correct name of DR1.
    • Don't provide DR1 yet - its not ready.
    • Show COUNTER section in form if user is logged in but IP address is unrecognized.
    • Slight modification: If we know the user's institution, but they do not have stats for a given collection and time period, default to 0 for all counts so a row is still displayed for them in the results page with zeros in all fields.
  • Stats/Resource.pm
    • Get 'public' attribute from config files to determine if a collection is public or not.
    • Allow upper- or lower-case for report types in config file.
  • Stats/Map/Auth.pm
    • Constructor now takes Stats::Config object.
  • Stats/Map/Titles/colldb.pm
    • Temporarily set $ENV{'HTTP_HOST'} to $$self{'host'}, which is set from the query "SELECT DISTINCT host FROM Collection WHERE userid = 'production' and collid = ..." in case $ENV{'HTTP_HOST'} is not set.
  • Stats/Reports/COUNTER.pm
    • Include support for COUNTER Database Report 1.
  • Stats/Reports/COUNTER/BR1.pm
    • Clean up Database Report 1 a bit.
  • Stats/Reports/COUNTER/BR2.pm
    • Clean up Database Report 1 a bit.
  • Stats/Reports/COUNTER/BR5.pm
    • Clean up Database Report 1 a bit.
  • Stats/Reports/COUNTER/DR1.pm
    • Add module for COUNTER Database Report 1.
    • Clean up Database Report 1 a bit.
  • Stats/Reports/COUNTER/Report1.pm through Stats/Reports/COUNTER/Report5.pm
    • Pass Stats::Config object to COUNTER creation methods rather than creating other Stats::Config files within these methods.
    • Now must pass Stats::Config object to Stats::Map::Auth constructor.
    • Return undef if Resource object is not created in the 'match_collid_to_resource()' method - which, in this case, happened because I had removed the collid from the stats config.
    • Clean up Database Report 1 a bit.
  • Stats/Reports/COUNTER/ReportTools.pm
    • Die if location of COUNTER reports cannot be obtained from the config file.
    • Pass Stats::Config object to COUNTER creation methods rather than creating other Stats::Config files within these methods.
    • Include support for COUNTER Database Report 1.
  • Stats/Reports/Custom/Excel.pm
    • Big rewrite to include aggregate stats rows and use the same struct that is created for HTML reports.
    • Get rid of 'aggregate_only' concept.
  • Stats/Reports/Custom/Tools.pm
    • Remove file - no longer needed
  • Stats/Tools/AuthTools.pm
    • Just added comment.
    • Now all users are allowed, but if they are not recognized then they can only see aggregate stats for public collections.
  • Stats/Tools/DateTools.pm
    • Fix bug in day() method that returns the current day of the month (it was erroneously adding 1).
  • Stats/Tools/ResourceTools.pm
    • Big modification to allow unrecognized users to see public collections (aggregate stats only).
    • Bug fix: empty the list of coll ids previously built before grabbing all coll ids in the cache, because we don't want to include any that aren't in the cache (but were, say, in the AUTHZD_COLL variable).
    • When doing regular expression with coll IDs, use /^..$/ to get the exact coll ID.
    • Get rid of 'aggregate_only' concept. Return an array of all coll ids for which the user is an admin.
  • Stats/XML/BuildXML.pm
    • This is now used for the Excel custom reports as well as the HTML reports. Now includes aggregate stats rows.
    • Get rid of 'aggregate_only' concept.
    • utf-8 encode collection name before passing to LibXML method.
    • Fix use of utf8::upgrade for collection name
    • Slight modification: If we know the user's institution, but they do not have stats for a given collection and time period, default to 0 for all counts so a row is still displayed for them in the results page with zeros in all fields.
  • upgrade_6_7
    • Initial
  • upgrade_6_7.cfg
    • Initial
    • add nameresolver, oai, oaisets table creation
  • BrowseUpdate.pm
    • Insertion is now skipped if browse field has no value. Also, field value counts are tallied and reported.
  • browseutils.cfg
    • added "an"
  • ub
    • more input checking
  • updatebrowsedb.pl
    • Insertion is now skipped if browse field has no value. Also, field value counts are tallied and reported.
    • more input checking
  • BrowseUpdate/ImageMysqlBU.pm
    • Added a comment about browse_split.
    • commented out a debug statement.
  • main.cfg
    • remove misc/db CSV related opertations
    • remove replace( "$DLXSROOT/bin/i/image/html2tab.pl", qq{\#!/l/local/bin/perl}, $perlhashbang );
    • remove replace( "$DLXSROOT/bin/i/image/tab2html.pl", qq{\#!/l/local/bin/perl}, $perlhashbang );
    • restore Text::CSV_XS
    • remove ["Net::hostent", 1.01] it is part of Perl
    • update Text::CSV_XS should be 0.23
    • Removed all references to outdated validate.pl script Changed references for samplefa.*inp files to new file names samplefa.ead2002.dcl and samplef.concat.ead.dcl
    • fixed move of fixdoctype.pl from s/samplefa to f/findaid
    • remove replace( "$DLXSROOT/bin/f/findaid/isolat128bit.pl", qq{\#!/l/local/bin/perl}, $perlhashbang );
    • removed sampletc obsolete (now just sampletc_utf8)
    • add setup_newcoll for findaidclass
    • Added section for findaid demofa collection
    • ["XML::LibXSLT", 1.63], ["XML::LibXML", 1.65], from ["XML::LibXSLT", 1.58], ["XML::LibXML", 1.58],
    • sru, broker20 optional, broker20 deprecated
  • dlxsglobals.xml
    • url for Atom/Rss
    • changed TemplateName to TemplatePath
    • changed TemplateName to TemplatePath
    • mtag changes
  • feedback.xsl
    • new feedback cgi code. generalized for use outside of just mdp.
    • these files were moved to web/i/image.
  • transform.xml
    • add <xsl:strip-space/>
    • <xsl:strip-space elements="*"/>
  • transform_xhtml.xml
    • add <xsl:strip-space/>
    • <xsl:strip-space elements="*"/>
  • transform_xml.xml
    • add <xsl:strip-space/>
    • <xsl:strip-space elements="*"/>
  • xsl2htmlutils.xsl
    • bboru changes via email from Wisconsin: added test for value of key('get-lookup',Label) else it returns an empty value if there is a dot in the Label value, as in 'etc.' <xsl:when test="contains(Label,'.') and key('get-lookup',Label)"> <xsl:when test="contains(Label,'.')"> use 'selected' for valid xhtml <xsl:attribute name="selected">selected</xsl:attribute> <xsl:attribute name="selected">true</xsl:attribute>
  • css/feedback.css
    • new feedback cgi code. generalized for use outside of just mdp.
    • these files were moved to web/i/image.
  • js/feedback.js
    • new feedback cgi code. generalized for use outside of just mdp.
    • these files were moved to web/i/image.

[edit] TextClass

web/t/text

  • browse.xml
    • added browseextra.xsl inc
  • clipviewer.xml
    • added some xsl incl
  • header.xsl
    • added code to allow for print on demand links to amazon.com from podPermittedItems
  • htmlhead.xsl
    • emit full path to main XML template for <TemplateName> element
    • changed TemplateName to TemplatePath
    • If we're viewing a single item, put its title in the HTML head.
    • Same as above, but also for view=trgt and page=root.
  • langmap.en.xml
    • add feature PRF (Preface) to langmaps
  • langmap.fr.xml
    • add feature PRF (Preface) to langmaps
  • pageviewerheader.xsl
    • Stub template for ProcessSerialarticle.
  • results.xsl
    • No more STATUS="hidden".
  • resultsheader.xsl
    • BIBLSCOPE filtering: made like tocheader.xsl (no comma before issuetitle).
    • <xsl:value-of select="$pubinfo"/> --> <xsl:copy-of select="$pubinfo"/> to write child nodes (e.g. <div>) to the HTML.
    • No comma after <div>[issuetitle]</div> for serialissue.
    • minor change to fix missing name search bug
  • scopedivs.xsl
    • TYPE="hidden" allows selective non-display of DIVn (and all descendents) in TOC. (Note distinction from mis-named STATUS="hidden".)
    • STATUS="hidden" now actually hides the DIV. No more TYPE="hidden".
  • text.components.xsl
    • FilterNumberedNotesWithParams: wrap paragraphs after the first in <p>.
    • LG template calls template name="addRend".
    • In <xsl:template match="P">, don't normalize value of @ID.
    • More robust filtering for filterNoteWithParas.
    • Handle CELL/@ROWSPAN.
    • In <xsl:template match="REF">, removed special case for PARENT::ITEM which appeared to be abandoned code.
    • For PBs, Wrap the DIV in an OBJECT so that we still have valid XHTML in the event that we're currently inside a P.
    • OBJECT wrapper around PB text breaks in Safari; better to use span with display: block.
    • Add anchors to ITEMs which have IDs.
    • Create anchors for ID'd Ps in filterNumberedNoteWithParas.
    • In filterNumberedNoteWithParas, check whether we're in NOTE2 (in addition to NOTE1).
    • XML table elements (TABLE, ROW, CELL) get a class applied to them, obscuring any REND styles in the markup. I've changed code to apply the value of @REND to the class value, e.g. <td class="xmltd-rend-center">.
    • L: pass forward all @RENDs, nit just ones that start with "line".
  • textclass.css
    • Added rend-plain.
    • Added rend-isub.
    • Added .rend-rightjustify.
    • Changed .pbtext.
    • List style .nomarker.
    • Add margin-bottom to div.lg and the like.
    • rend-aligntop.
  • tocheader.xsl
    • BIBLSCOPE filtering.
  • viewer.utils.xsl
    • minor change to fix missing name search bug
  • viewtextnote.xsl
    • remove crash your browser
    • OCR quality

bin/t/text

  • CER.pm
    • added dagger2 &#x2021; labr 〈 long &#xAF; rabr 〉 short ˘
  • dtdalyzer.pl
    • Added ROWSPAN attribute.
    • Added DOC attr.
  • utf8chars
    • fix error checking and display data

cgi/t/text

  • CVApp.pm
    • added pgseq to cache filename and change default view to pdf
  • DlpsLocalUtils.pm
    • Refactored LocalIdResolver(), moving substantial code into CreatePicklist().
    • CreatePicklist: delete unwanted params from tempCgi that may be hanging around (possibly due to URL hacking) and that we don't want passed forward in picklist links.
  • TextAppXsltPIFiller.pm
    • print on demand links to amazon.com added
  • TextClass.pm
    • Add RemoveXMLPi for Kwic processing
    • Filter_REFsForText: treat <REF TYPE="txt"> same as <REF TYPE="ptr">.
    • GetDateParsePattern: optional hyphens between year-month-day parts of a sortdate.
    • Output FirstPageHref for layer2 serialissue results.
    • minor change to fix missing name search bug - close div at end of div1headbib
    • another change to fix missing NODE param in search error
  • textclass.cfg
    • added config to allow POD db lookups
  • ClipView/mdailyCV.pm
    • new

[edit] ImageClass

[edit] CGI/Middleware

Known Problems

  • None known so far.

Enhancements

  • Image Viewing
    • Ajax based zooming and panning of imagery. Works with JPEG2000 and MrSID.
    • getimage-idx cgi has been completely rewritten and is backward compatible with previous version.
    • XML, XSL, CSS, and Javascript for entry/image view have changed significantly.
    • Running mediaprep with purge=1 is recommended, but not required. It now stores thumbnail dimentions in the ImageClassMediaFiles table and uses them for drawing zooming reference visuals on the thumbnail adjacent to zoomable images. Image Class now requires ImageMagick and PerlMagick to be installed.
  • Portfolios
    • Longer descriptions are allowed in custom sorting display.
    • Multiple owners/editors.
    • Renaming.
    • Added documentation for end-users.
    • Fixed bug that kept anonymous user from opening their own session based portfolio.
    • Portfolio (BookBag) IDs are now generated randomly to avoid potential database replication problems.
  • Searching/Browsing
    • Relevance ranking is now an option for Image Class search results. To activate simply add "relevance" to the Collmgr field "sortflds" (first list value should be "none" and second "relevance"). Sorting of results by relevance will be the default for the collection once configured. Relevance ranking is not used when searching multiple collections.
    • Searching has been packaged and can be subclassed. A subclass for integrating search results from the ARTstor XML web service is included and can be used by ARTstor members. Support for other services, databases or database schemas can be added by subclassing the ImageSearch.pm Perl module.
    • Browsing of newly added or update media items is now possible. Requires reloading metadata for the collection. In Collmgr set brwsadds to "on", add "m_flm:::Recently Added/Updated" to field_labels, and add "m_flm" to sortflds, dfltentryflds, and dfltresentryflds.
    • All collections are selected by default in cross collection search. This is now a configurable option at the class level in imageclass.cfg.
  • Collection Size Calculation
    • Collection size counts are now stored, by image-idx, in Collmgr. See help text for new Collmgr fields "recordcount" and "mediacount".

[edit] Data Preparation

Enhancements

  • It is possible to configure a development mysql server for loading metadata. load.pl script will use production mysql server to get information from Collmgr, ImageClassMediaFiles, etc. but will populate new tables on development server. This is not generally necessary, but can help to reduce production server load with very large collections (75,000+ records).
  • mediaprep script now stores pixel dimensions of thumbnail images in ImageClassMediaFiles table. Pixel dimensions for thumbnails now appear in the relevant XML output of the middleware cgi (image-idx).

[edit] BibClass

cgi/b/bib

  • BibApp.pm
    • added size as a common param
    • add size to SID if we have one
  • BibClassPerlFilters.pm
    • BibClassPerlFilters::CollsFilter had a minor bug in the sql query used to select the collection name from the Collection table causing it to not always select the right row because it wasn't consdering the value of the DLPS_DEV environment variable. It now correctly selects the production, release, or user row.
  • BibClassUtils.pm
    • change openurl sub to be 1.0
    • change wording on openurl link
    • added support for icon in openurl links
  • bibclass.cfg
    • change openurl server

[edit] Oai

cgi/o/oai/

  • oai, oai.cfg, oai_conf.xml, sample_config.xml, UMProvider.pm
    • new -- new OAI data provider and accompanying scripts

bin/o/oai/

  • ConfirmPublicDomain.pl, loadOai.pl, mbooks_harvest_cron.pl, mbooks_update.pl, oai_conf.xml, OaiList.pl, OaiToDb.pl, RepositoryConfig.xml, revertOaiTbl.pl, updateMbooksOai.pl
    • new -- new OAI data provider and accompanying scripts

bin/o/oai/provider

  • AddCollToSource.pl, articlesToDc.xsl, bibClassToDc.xsl, CheckModifiedColls.pl, collmgrFullList.txt, ConvertToDc.pl, ConvertToUTF8.pl, dlxs_to_oaidc_cron.pl, exampleColls.xml, ExtractHeaders.pl, GenerateReport.pl, GetNewCollections.pl, LoadDB.pl, NewOAICollection.jpg, oai_new_coll_history.txt, oai_update_history_text.txt, oai_update.pl, README.txt, RunDlpsOaiConversion.pl, RunOaiConversion.jpg, RunOaiConversion.pl, textClassToDc.xsl, umprovider_flow_diagrams_and_examples.ppt, UMProviderFlow.jpg, updateCollConfig.pl
    • new -- scripts to transform text, bib, and image class collections into oai_dc for use by the OAI data provider. See the README.txt file in this directory for a brief description of each file

bin/o/oai/provider/logs

  • This directory houses the logs generated by the scripts in bin/o/oai/provider

bin/o/oai/provider/reports-new-colls

  • This directory houses the reports generated by bin/o/oai/provider/GetNewCollections.pl

bin/o/oai/provider/reports-oai-update

  • This directory houses the reports generated by bin/o/oai/provider/GenerateReport.pl

bin/i/image/

  • image2oai_dc.pl
    • new -- script for loading OAI tables with Image Class data in MySQL

bin/o/oaister/

  • RepositoryConfig.cfg
    • new -- contains baseURLs and OAITransFixer.pm subscript calls
  • oaitransform/CollsObj.cfg
    • new -- config for CollsObj.pm
  • oaitransform/CollsObj.pm
    • new -- used for FMPro connection for OAIster
  • oaitransform/DataConditioning.pm
    • Get rid of any remaining stray sp0t tags
  • oaitransform/MODSTransform
    • Fix bug where MODS record was not ignored for re-exposure if no URL in record
    • Changed the tree removing metadata from the re-exposed XML, changed the language code mapping, added setSpec to about container
    • Swap dates in about container and change MODS loc.gov URL
    • Remove NS prefix and NS from metadata element
    • Remove all xmlns attributes as well
    • Add all orig setSpec to provenance element for MODS re-exposure
    • Hack to fix xsi: problem for Northwestern
    • Change to pick up URLs in identifier type=uri
  • oaitransform/OAITransFixer.pm
    • new -- perl subscripts for fixing harvested data
  • oaitransform/OAITransform
    • Added flag to skip DTD validation change, the PostXslt process to be after each small file, not the final large one
    • Added code to regenerate the perl config files from the XML config files each time this is run
    • Comment out LoadRepositoryLookTable() and changed SetCurrentArchive() to use XML config value
    • Move removing attribute to after data conditioning
    • Patch to fix latin1 regex problem
    • Unescape hex and dec entity refs
    • Added fixer call and other minor changes
    • Skip deleted records
    • Add skipped and deleted to repo count
    • Get the return from PreprocessXML bug fix
    • Semi opt for hex and dec ent refs
    • Added CollsDB
  • oaitransform/browse.tpl
    • new -- template for building OAIster browse pages
  • oaitransform/manageCollsDb.pl
    • new -- CGI script for managing FMPro DB for oaister
  • oaitransform/mods-bibclass.xsl
    • Added copyrightDate YR
    • urls in own URL tags
    • Change to pick up URLs in identifier type=uri
  • oaitransform/mods_repositoryNames.pl
    • Fixed amp;
  • oaitransform/oai-bibclass3.xsl
    • Added tests for empty elements
  • oaitransform/repositoryNames.pl
    • Add &amp; fix
  • scripts/Batch_UMHarvest
    • Add skip HTML flag for transform
    • Allow -x for transform
    • Added -a for incremental batch harvesting
  • scripts/ListRecords
    • Error code checking
    • Do not exit on OAI error
    • Fixed log reporting
    • Make list of dirs, not files, to check for replacement records -- speed up incremental
  • scripts/UMHarvester
    • Changed to use the XML config file; requires XML::LibXML now
    • Added wait flag
    • Check for sets cfg before trying to parse it, only loop on sets for list records
    • ID bug fix
  • scripts/browse_check.pl
    • new -- checking FMPro repositories against browse pages
  • scripts/id_check.pl
    • new -- checking FMPro repositories against incremental batch harvest
  • scripts/rc_check.pl
    • new -- checking FMPro repositories against RepositoryConfig.cfg
  • scripts/restoreHarvestedData.pl
    • new -- restores backup copy of repository dir
  • scripts/startIndex.pl
    • new -- runs xpat and multirgn on xml obj files

[edit] IdResolver

cgi/i/idresolver/

  • idresolver
    • CGI script that returns a URL (marked up in XML) for an ID
  • idresolver-nr
    • new -- CGI script that redirects a user from a persistent URL to one defined in the nameresolver database for an ID, can be used instead of cgi/b/bib/bibperm
  • idresolver-srch
    • new -- CGI script that takes a text file of ids (one per line) and checks to see if they exist in the nameresolver database
  • IDResolver.pm
    • Perl module that is used by each of the cgi scripts

bin/i/idresolver/

  • cvstag.idresolver, rdist.class

bin/n/nameresolver/

  • IdParser.pl
    • new -- script used to populate the nameresolver database
  • LoadLLMCIds.pl
    • new -- this is a sample script of how you can use a .csv file to populate the nameresolver database
  • CreateTable.pl, NRTable.sql
    • new -- scripts for creating nameresolver table in database (which is already included in the release)
  • new -- other sample files: DeepBlueShortIds.txt, getUpdatedNRfiles, InsertCrosscIssues.pl, N2TestExecution.txt, NRDevVsProdComparison.xls, README.txt, RemoveDeepBlueConflicts.pl, statusLoadingNRinProd.txt, TestIdParser.pl
    • These files were used for DLPS collections and are provided only as additional resources. They are not necessary to the Idresolver/nameresolver configuration

[edit] broker20

  • No changes.

[edit] Collmgr

  • Supports version 7.0 database for DLXS release14.
  • default browse page for Text and Findaid Class now set from first item in Browsefields list

[edit] FindaidClass

[edit] Findaid Class Summary

Prep scripts

Data prep scripts have been reorganized and renamed.

  • New functionality for Makefile and preparedocs.pl
  • New script to setup new collections

New prep script: setup_newcoll

$DLXSROOT/bin/f/findaid/setup_newcoll can be used to set up directories for new collections. For example, to set up the workshopfa collection based on samplefa (Assuming your $DLXSROOT variable is set)run this command: 

 $DLXSROOT/bin/f/findaid/setup_newcoll -c workshopfa  -s $DLXSROOT/prep/s/samplefa/data

More information on the setup_newcoll script can be found by clicking here or invoking the man page: 

$DLXSROOT/bin/f/findaid/setup_newcoll --man

New options for preparedocs.pl

The $DLXSROOT/bin/s/samplefa/preparedocs.pl script now takes several new arguments.

  ./preparedocs.pl --man   will give details

of particular interest is the preparedocs.pl -i inputfilelist option which allows you to specify a file containing the full paths to the finding aids you wish to index. This option can be used instead of the previous default behavior which was to recursively search the data directory and index all files in that directory.

Changes to Makefile

Addition of these new targets:

prepdocslist
Same as prepdocs but uses the -i inputfilelist to preparedocs.pl and by default reads a file $DLXSROOT/prep/c/collection/list_of_eads.
allbutprep
convenience target for use in conjunction with prepdocslist that does all the make steps except for the make prepdocs
index
convenience method that runs all 3 indexing steps

Changes to prep scripts

$DLXSROOT/bin/s/samplefa
Moved generic bin files to f/findaid
Makefile modified
preparedocs.pl has new options
$DLXSROOT/prep/s/samplefa
Renamed files 
	samplefa.text.inp  to samplefa.ead2002.dcl
	samplefa.xml.inp to samplefa.concat.ead.dcl
Removed: 
	samplefa.inp
Added: 
   list_of_eads
$DLXSROOT/bin/f/findaid
Removed: 
	catsourcefiles.pl	
	isolat128bit.pl
	validate.pl
Added/moved from bin/s/samplefa 
	fixdoctype.pl		
	stripdoctype.pl
	validateeach.sh
        setup_newcoll

web/f/findaid

  • bookbagitemsstring_debug.xsl
    • For debugging in oxygen since the Oxygen debugger uses Saxon and Saxon doesn't understand the extensions
  • browse.xsl
    • Added code for subject browse
  • browseheader.xsl
    • bulk dates labelled
  • htmlhead.xsl
    • emit full path to main XML template for <TemplateName> element
    • changed TemplateName to TemplatePath
  • text.components.xsl
    • Modified template for processing c0x's so that if there are two containers within a //c0x/did/ such as box/folder they will both show up in the proper column.
    • Highlighting fix. Replaced about 36 instances of <xsl:value-of select="."/> with <xsl:apply-templates select="*|text()"/> If value-of select gets highlighted text in the context node: "text<HIGHLIGHT>text</HIGHLIGHT>text text" it will ignore the <HIGHLIGHT> elemnts and just render all the text. On the other hand the apply-templates "*" will match the <HIGHLIGHT> element and trigger the appropriate highlighting template and the text() nodes will get passed to the template that just outputs the text
    • Fixed code in template match=unittitle mode=SimpleUnittitle because foreach was messing with context Also change code for handling notes/scopeconent in match=C01|c02... because it was an xsl:choose, but notes and scopecontent are not mutually exclusive
    • Changed code for handling notes/scopeconent in match=C01|c02... now both did/note and note and did/scopecontent and scopecontent will be rendered
    • fixed bug in template match="list" where there was a foreach and then value-of select *|text() that needed to be value-of select="."
    • space before unitdate value
    • title styling
    • optional labels for additional descriptive material
    • hide sorting title
    • sponsor
    • Restored ADD to full-text view
    • abstracts in dids
    • Change template for index mode= add so it doesn't produce 2 copies of any <head> text. Key is to limit the apply templates after we already processed the head not to process the head again: <xsl:apply-templates select="*[not(self::head)]"/>

bin/f/findaid

  • catsourcefiles.pl
    • No longer used for samplefa (replaced by preparedocs.pl) Also it contains bhl specific code. For bhl use the copy in $DLXSROOT/bin/b/bhlead which is in cvs.
  • fixdoctype.pl
    • Moved here from bin/s/samplefa No collection-specific customization These should work on any EAD that conforms to the ead2002.dtd
  • isolat128bit.pl
    • No longer used for findaids, since all findaids should be utf8 encoded
  • setup_newcoll
    • New script to set up directories ./setup_newcoll --man for details
  • stripdoctype.pl
    • This library file is now used by preparedocs.pl and validateeach.sh (through $DLXSROOT/bin/s/samplefa/fixdoctype.pl)to correctly remove multiple line DOCTYPE declarations and any entity references contained within them. It is a replacement for the one-line perl program previously used by those two programs.
  • validate.pl
    • Removed outdated file that worked on sgml files. FindaidClass now exclusively xml
  • validateeach.sh
    • Moved here from bin/s/samplefa No collection-specific customization Thise should work on any EAD that conforms to the ead2002.dtd

cgi/f/findaid

  • FindaidApp.pm
    • remove FormatGuideFrame() obsolete since the change to XML/XSL
  • FindaidAppXsltPIFiller.pm
    • Removed FormatOutlineResult_XML and FormatOutlineFrame_XML as they are not called by any code (or bound to any PIs).
  • FindaidClass.pm
    • highlight hits in layer 1 result items
    • Added highlighting to BuildItemTitle_XML so highlighting will show up in title
    • Fixed bug in FilterAllDaos_XML that would not properly process daos with real hrefs and would result in illegal xml being output when id resolver is turned on.
  • FindaidClass/ClementsmssFC.pm
    • removes "viewtextnote" speedbump
  • FindaidClass/DemofaFC.pm
    • Demo of subclassing
    • add relatedmaterial and separated material TOC heads
    • use <head> tags for bioghist instead of bentley logic for TOC heads
    • change labels for several TOC heads
  • FindaidClass/BioghistfaFC.pm
    • Demo of subclassing to use <head> tags for bioghist instead of bentley logic

[edit] XClass

  • No changes.

[edit] METS Pageturner and Collection Builder

  • Continuing development work in Pageturner. New application: Collection Builder allows users to add items to a personal collection via widgets in Pageturner. These applications are not part of DLXS. They use a different code base mainly under DLXSROOT/{web,bin,cgi}/m/mdp and DLXSROOT/lib/App. Stub routines are required in Pageturner to abstract the database connections and an installation of Solr/Lucene is required to support the collection search in Collection Builder. The user interface makes extensive use of Yahoo User Interfacen toolkit (YUI) functionality.

[edit] SRU

  • Added fielded searching to query -- not yet Level 1 or 2, though

Top

Retrieved from "http://webservices.itcs.umich.edu/mediawiki/dlxs14/index.php/Release_Notes_for_Current_DLXS_Release"
Personal tools