10 Percent Wrong for 90% Done: A Practical Approach to Collection Deduping

Fuente: ERIC Institute of Education Sciences

Enregistré dans:

Détails bibliographiques
Auteur principal:	Hamby, Rogan
Format:	Recurso educativo Open Access
Langue:	en
Publié:	2012
Sujets:	Delivery Systems Pilot Projects Libraries Information Systems Information Technology Open Source Technology Library Automation Library Networks Consortia Public Libraries Bibliographic Databases Cataloging Documentation Standards Catalogs Metadata Coding Duplication
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

_version_	1867180932199350273
author	Hamby, Rogan
author_facet	Hamby, Rogan Hamby, Rogan
collection	Education Resources Information Center
contents	10 Percent Wrong for 90% Done: A Practical Approach to Collection Deduping Hamby, Rogan Delivery Systems Pilot Projects Libraries Information Systems Information Technology Open Source Technology Library Automation Library Networks Consortia Public Libraries Bibliographic Databases Cataloging Documentation Standards Catalogs Metadata Coding Duplication The technology of the 21st century has had no shortage of challenges for libraries, but it has also brought opportunities. Operating systems, databases, and application environments that can use cheap off-the-shelf hardware to scale and handle hundreds of millions of transactions were not developed for libraries. However, libraries have taken advantage of these technologies with multiple integrated library systems (ILSs) now featuring this scalability. One of the best-known among these younger systems has been the Evergreen open source ILS, which was adopted in 2009 by the South Carolina Library Evergreen Network Delivery System (SC LENDS). Widespread resource-sharing library consortia, especially those powered by Evergreen, are quickly growing. With Evergreen, SC LENDS was able to begin its pilot program of 10 library systems and then scale to accommodate adding libraries as they elected to join. Investment in open source paid off for libraries beyond South Carolina, creating a single, more coherent collection out of many. This article discusses a practical approach to collection deduping.
format	Recurso educativo Open Access
id	eric_EJ970524
institution	ERIC Institute of Education Sciences
language	en
publishDate	2012
record_format	eric
spellingShingle	10 Percent Wrong for 90% Done: A Practical Approach to Collection Deduping Hamby, Rogan Delivery Systems Pilot Projects Libraries Information Systems Information Technology Open Source Technology Library Automation Library Networks Consortia Public Libraries Bibliographic Databases Cataloging Documentation Standards Catalogs Metadata Coding Duplication 10 Percent Wrong for 90% Done: A Practical Approach to Collection Deduping Hamby, Rogan Delivery Systems Pilot Projects Libraries Information Systems Information Technology Open Source Technology Library Automation Library Networks Consortia Public Libraries Bibliographic Databases Cataloging Documentation Standards Catalogs Metadata Coding Duplication The technology of the 21st century has had no shortage of challenges for libraries, but it has also brought opportunities. Operating systems, databases, and application environments that can use cheap off-the-shelf hardware to scale and handle hundreds of millions of transactions were not developed for libraries. However, libraries have taken advantage of these technologies with multiple integrated library systems (ILSs) now featuring this scalability. One of the best-known among these younger systems has been the Evergreen open source ILS, which was adopted in 2009 by the South Carolina Library Evergreen Network Delivery System (SC LENDS). Widespread resource-sharing library consortia, especially those powered by Evergreen, are quickly growing. With Evergreen, SC LENDS was able to begin its pilot program of 10 library systems and then scale to accommodate adding libraries as they elected to join. Investment in open source paid off for libraries beyond South Carolina, creating a single, more coherent collection out of many. This article discusses a practical approach to collection deduping.
title	10 Percent Wrong for 90% Done: A Practical Approach to Collection Deduping
topic	Delivery Systems Pilot Projects Libraries Information Systems Information Technology Open Source Technology Library Automation Library Networks Consortia Public Libraries Bibliographic Databases Cataloging Documentation Standards Catalogs Metadata Coding Duplication
url	https://eric.ed.gov/?id=EJ970524

if (filemtime($filename) + $maxlifetime < time()) { unlink($filename); $count++; } } return $count; } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ protected function saveSession($sessId, $data): bool { $sessFile = $this->path . '/sess_' . $sessId; if ($handle = fopen($sessFile, 'w')) { $return = false; // Lock the file for exclusive access to avoid issues with multiple // processes writing session simultaneously: if (flock($handle, LOCK_EX)) { $return = fwrite($handle, $data); // Make sure that there's no trailing data by truncating the file to // the correct length: ftruncate($handle, strlen($data)); fflush($handle); flock($handle, LOCK_UN); } fclose($handle); if ($return !== false) { return true; } } // If we got this far, something went wrong with the file output... // It is tempting to throw an exception here, but this code is called // outside of the context of exception handling, so all we can do is // echo a message.

// Anecdotal testing Today and Yesterday seems to indicate destroy() // is called by the garbage collector and everything is good. // Something to keep in mind though. return 0; } /** * Write function that is called when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ public function write($sessId, $data): bool { if ($this->writesDisabled) { return true; } return $this->saveSession($sessId, $data); } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ abstract protected function saveSession($sessId, $data): bool; }

* * @return void */ public function writeClose() { // The assumption is that we're using PHP's ext/session. // session_write_close() will actually overwrite $_SESSION with an // empty array on completion -- which leads to a mismatch between what // is in the storage object and $_SESSION. To get around this, we // temporarily reset $_SESSION to an array, and then re-link it to // the storage object. // // Additionally, while you _can_ write to $_SESSION following a // session_write_close() operation, no changes made to it will be // flushed to the session handler. As such, we now mark the storage // object isImmutable. $storage = $this->getStorage(); if (! $storage->isImmutable()) { $_SESSION = $storage->toArray(true); session_write_close(); $storage->fromArray($_SESSION); $storage->markImmutable(); } } /** * Attempt to set the session name * * If the session has already been started, or if the name provided fails * validation, an exception will be raised. * * @param string $name * @return SessionManager * @throws Exception\InvalidArgumentException */ public function setName($name) { if ($this->sessionExists()) { throw new Exception\InvalidArgumentException( 'Cannot set session name after a session has already started'

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
VUFIND_SESSION	"58bb5572625a40650e7fed445bfffd6c"
language	"fr"
ui	"standard"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1789135551.7956 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "58bb5572625a40650e7fed445bfffd6c" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1455}
SessionHelper	Laminas\Stdlib\ArrayObject {#1785}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"VUFIND_SESSION=58bb5572625a40650e7fed445bfffd6c; language=fr; ui=standard"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.216.170"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"57799"
REDIRECT_URL	"/Record/eric_EJ970524/Details"
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/eric_EJ970524/Details"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1789135551.7902
REQUEST_TIME	1789135551

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
VUFIND_SESSION	"58bb5572625a40650e7fed445bfffd6c"
language	"fr"
ui	"standard"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1789135551.7956 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "58bb5572625a40650e7fed445bfffd6c" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1455}
SessionHelper	Laminas\Stdlib\ArrayObject {#1785}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"VUFIND_SESSION=58bb5572625a40650e7fed445bfffd6c; language=fr; ui=standard"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.216.170"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"57799"
REDIRECT_URL	"/Record/eric_EJ970524/Details"
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/eric_EJ970524/Details"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1789135551.7902
REQUEST_TIME	1789135551

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler

Documents similaires

Environment & details:

Environment & details: