What Perl can do for you
Phil Shirley
Cuyahoga Falls Library
Goals
1. Learn about the process of running a Perl script
–So you can speak intelligently with an IT person who you want to do some Perl
–So you can try it yourself if you want
Goals
2. Learn how libraries use Perl
–Understand how some scripts basically work
–Appreciate the more complex projects
–Grasp the scope of what Perl can do
This presentation will not teach you anything about the Perl programming language or how to write, edit, or troubleshoot Perl scripts
Overview
• What is Perl
• What you need
• Beyond the basics
• What libraries are doing with Perl
• Resources
Dear computer,
Please take that file I just saved and give me a summary report.
kthxbye
“Great, somebody posted a Perl script to do exactly what I need!
I wish I knew how to use it.”
“We don’t have Perl.”
What Perl Is
Perl is a general purpose programming language that’s especially good at manipulating text.
Script? Program?
• Scripts and programs are changed (“compiled”) into commands the computer can understand.
• Scripts are compiled each time you run them.
• Programs are compiled once, when you finish writing them.
Still, we use the word “programming” to refer to creating scripts.
Some write it “PERL”
• “Practical Extraction and Report Language”
• Actually this is a backronym, just like “Gentlemen Only, Ladies Forbidden” (Golf).
What do I need to get started?
– A computer
– A text editor
– A Perl script
– A Perl interpreter
– Perl modules (if needed by the script)
– Input
Computer
– Your workstation (Windows, Mac, etc.)
– Just about any server (web, e-mail, Windows, etc.)
– Almost never your Innovative server
Which computer you use depends on the particular task and/or what computers you have access to.
Text editor
• For creating & editing Perl scripts
• A program for editing plain text files
– letters, numerals, symbols, tab, newline
– no fileformat-specific control characters for things like bold, italic, fonts, columns, tables
Text Editors
–Not Microsoft Word, Wordpad, etc.
–Built-in:
• Notepad (Windows)
• SimpleText or TextEdit (Mac)
–Other specialized editors for programming exist
Perl script
• A plain text file that you write (or copy from somebody else)
• Written in the Perl language
• Looks like this:
#!c:\perl\bin\perl.exe
print "Hello World!";
Perl interpreter
• A program you install on the computer where you’ll run scripts
• Changes the script into instructions the computer can understand
• Free!
Perl Interpreter: Active Perl
http://www.activestate.com/Products/activeperl/index.mhtml
Perl modules
• Extend the language
• Make it easier to do some kind of task
• Examples
– Net::FTP
– XML::Parser
– MARC::Record
Input
• A text file that you want the script to do something with
• A script can also get some info from the computer (like date & time)
Output
• A text file created by the Perl script
• Could be HTML, XML, MARC, RSS feed, etc.
• Or, could be just something normal for you to look at, print, paste into a report, etc.
In summary, the components
– A computer
– A text editor
– A Perl script
– A Perl interpreter
– Perl modules if needed by the script
– Input
– [Output – created by the Perl script]
• Questions so far?
Beyond the Basics
• If you want, and you have everything you need for this, you can automate the running of scripts.
• Some libraries automate the running of Create Lists, process the output with a Perl script, and output an HTML file directly on their web servers.
• Of course, you can create and post a web page without doing it this way.
Beyond the basics:Expect
An Expect script logs into the character-based system, runs Create Lists, and produces the input for a Perl script.
Actually, Expect scripts can do just about anything in the character-based system.
Beyond the basics:Scheduling an Expect script
• UNIX: cron
• Windows: Scheduled Tasks
Beyond the basics:E-mail pipe method
– UNIX mail servers only (probably)
– Receives e-mail and uses /etc/aliases file to “pipe” the e-mail to a Perl script on that server
Beyond the basics: Output
– Save the output file to the live directory on a web server or intranet server.
– It’s easier if your mail and web are on the same server.
Beyond the basics,putting it all together:
Scheduling + Expect + email pipe method + web server
Other advanced uses
• Using a Perl script to:
– Construct a database
– Query a database
– Interact with another script
– Communicate with another computer
What Libraries Are Doing with Perl
• Create Lists
• Prepare Bib & Patron Data for Loading
• OPAC functionality• Circulation
• System administration
Create Lists: How Perl Can Help
• Reformat the output of Create Lists to:
– Create a web page.
– Create an RSS feed.
– Etc.
Create Lists: How Perl Can Help
• Bring out data that Create Lists can't display.
• Further refine your Create Lists query.
• Save the data in a different database (like MySQL).
Simple Examples of Using Perl with Create Lists for the Public
Creates a web page that’s a list of circulating videos from Create Lists output
http://library.mills.edu/screens/videocirc.html
Mills College
Script available from Alma Garcia ([email protected])
Creates a web page that’s a list of journals based on output from Create Lists
http://www.scripps.edu/library/open/all.html
The Scripps Research Institute
Creates an A-Z list of periodicals
http://library.truman.edu/search_articles/
electronic-journals.htm
Stephen Wynn
Truman State University
Script available from [email protected]
Generates new items RSS feeds from create lists output.
http://library.bury.gov.uk/screens/new_items_feeds.html
Alan Brown
Bury Libraries
Script available at http://www.innovativeusers.org/
cgi-bin/clearinghouse/view.pl?id=166
Creates web pages and RSS feeds of recent acquisitions grouped by broad subject area, department/program, and call number
http://library.lafayette.edu/collections/newacq
Bob Duncan
Lafayette College
Script available from [email protected]
More Complex Examples of Using Perl with Create Lists for the Public
Creates a web page and RSS feeds of new items which can be narrowed by subject area, media, library, time (today, last month, last 6 months), and sorted by title, call number, or author
Uses Perl, MySQL, and Perl/Expect
http://trilogy.brynmawr.edu/cgi-bin/
newbooks/newbook.pl
Tri-Colleges (Bryn Mawr, Haverford, Swarthmore)
Creates a title browse for videos and DVD's which can be limited by format and limited to new titles Uses Perl to populate a MySQL database, with a PHP front end
http://library.truman.edu/videos/TrumanVideos/
videosPHP.php
Stephen Wynn
Truman State University
Script available from [email protected]
Simple Examples of Using Perl with Create Lists for Staff
Creates a shelf list, delivered automatically to a printer, of items for which overdue notices are about to be sent so that Circulation staff can check the shelves before sending notices
Stephen Wynn
Truman State University
Script available from [email protected]
Creates and sends e-mail notices to requestors, alerting them that materials they have ordered are available in the library
Stephen Wynn
Truman State University
Script available from [email protected]
Finds items that are awaiting processing which have bib-level holds and notifies cataloging staff via email
Stephen Wynn
Truman State University
Script available from [email protected]
Reads Create Lists output and creates files of OCLC numbers for batch update, which are then updated using an AutoIt script
Phil Youngholm
MARINet
More Complex Examples of Using Perl with Create Lists for Staff
Notifies selectors that their materials have arrived based on the selector field in order records
Uses Expect module for Perl
Tri-Colleges (Bryn Mawr, Haverford, Swarthmore)
Creates a list of items on reserve, sorted by course title
Uses Create Lists and Patron API
John D. Boggs
Peninsula Library System
Uses a monthly full MARC extract (using Data Exchange) and populates a rudimentary stock management tool; generates reports from this database such as top issuing authors, missing items, and worn stock
Uses Perl and MySQL
Alan Brown
Bury Libraries
You Can’t Do that with Create Lists… but you can if you add Perl
Finds and reports duplicate patrons
Stephen Wynn
Truman State University
Script available from [email protected]
Identifies duplicate copies for weeding
John Wenzler
San Francisco State University Library
Creates a web page of purchase alerts based on the holds per title ratio, based on output from Create Lists
http://odie.aadl.org/iug_clearinghouse/
pa_example.html
Glen Modell
Ann Arbor District Library
Script available at http://innovativeusers.org/
cgi-bin/clearinghouse/view.pl?id=189
Prepare Bib & Patron Data for Loading
Creates patron barcodes for incoming student data loads given a beginning barcode (without check digit)
Christopher King
Appalachian School Of Law
Script available at
http://www.asl.edu/library/hal/pbargen.pl
Adds additional patron barcode field entries before loading patron records (the barcode without leading digits and the campus ID number)
Steve Sowder
Andrews University
Script available from [email protected]
Creates MARC patron records from a delimited text file created from an Excel file received by the Admissions Office
Amy Moberly
California Western School of Law
Script available from [email protected]
Extracts last four digits of phone number from Create lists output, inserts this as a PIN field, and creates a delimited file that can be turned into MARC format using MarcEdit
Phil Shirley
Cuyahoga Falls Library
Script available from
Reorders the ISBN's in bib records, putting the 10 digit ISBN first in order to use Amazon book jackets
Alan Brown
Bury Libraries
Rearranges data in MARC records for electronic resources to make them work with existing load tables – easier than creating new load tables (uses MARC module for Perl)
Michael Kreyche
Kent State University
Creates a web-based system for retrieving, validating, reviewing, and manipulating MARC records, and documenting loads (uses Perl, the MARC module for Perl, and PHP)
Michael Kreyche
Kent State University
Coverts a special collection from excel files to MARC
Ann Anderson
The Boeing Company
Converts data into a format to feed in an enterprise search
Ann Anderson
The Boeing Company
Cleans and separates data in MARC records for uploading to union catalogs and other services
Alan Brown
Bury Libraries
OPAC Functionality
Creates a catalog feature allowing patrons to navigate through Library of Congress call number outlines to class number level, where patrons can search by that class number, search the associated subject heading, find most popular titles in a class, and more.
http://bullpup.lib.unca.edu/scripts/lcclass/outline.htm
Mark Stoffan
Western North Carolina Library Network
Creates a feature in the web OPAC which allows patrons to send SMS messages to their phones with title, shelving location and call number of any item (uses Javascript, HTML, and Perl).
http://tripod.brynmawr.edu/
Tri-Colleges (Bryn Mawr, Haverford, Swarthmore)
Script available at http://www.innovativeusers.org/cgi-bin/
clearinghouse/view.pl?id=187
Includes Syndetics and Google Books data in catalog records (using Perl and Javascript).
http://tripod.brynmawr.edu/
Tri-Colleges (Bryn Mawr, Haverford, Swarthmore)
Circulation
Updates ITYPEs for ILL materials that have not been checked out (as loan periods for ILL's are controlled through ITYPE at this library), and notifies ILL staff when such materials should be returned to the lending library
Stephen Wynn
Truman State University
Script available from [email protected]
Posts Ariel document delivery to a web server
John Dillon
Saint Anselm College
Takes the web server log, a list of items for electronic resources, and a list of courses, and produces usage stats for each resource by course
Mark Huppert
The Australian National University
Script available at
http://preview.tinyurl.com/68s6qm
•Reformats paging slips and sends them to the appropriate groups for disposition.
•Reformats and sends notices via e-mail
Ann Anderson
The Boeing Company
•Creates and sends out courtesy notices
•Sends e-mails to indicate that an item is in the mail for a patron (takes advantage of the Oracle back end)
Ann Anderson
The Boeing Company
System Administration
Logs into the character-based system to run reports and perform maintenance tasks (uses Expect module for Perl)
Tri-Colleges (Bryn Mawr, Haverford, Swarthmore)
Works with EZproxy and Patron API to allow users to verify on either PIN or last name
John D. Boggs
Peninsula Library System
Works with a proxy server for access to electronic resources
Steve Sowder
Andrews University
Synchronizes EZProzy and URL's in the catalog by creating a list of new or modified 856 fields and transforming them to EZProxy format (uses Perl Expect module, PHP, and MySQL)
Tri-Colleges (Bryn Mawr, Haverford, Swarthmore)
Examines CybraryN log files and calculates the number of users
Cuyahoga Falls Library
Processes a web form and e-mails the results
Cuyahoga Falls Library
Where to find more of other libraries' scripts
IUG Clearinghouse and listservhttp://www.innovativeusers.org/
How to learn the Perl language
– Books such as:
• Learning Perl by Randal Schwartz, Tom Christiansen, Larry WallA good book for beginners
• Programming Perl Larry Wall, Tom Christiansen, Jon OrwantA great reference
• Win32 Perl Programming by Dave RothGood for automating Windows functions
Another Resource
Perl4lib: Perl for Libraries listserv and web site
http://Perl4lib.Perl.org/
Other tools
• MarcEdit
• Other scripting languages
Questions?
pshirley@CuyahogaFalls Library.org