Anzreg Presentation about the spellchecker (powerpoint)

April 13, 2023

Webvoyage SpellcheckerWebvoyage SpellcheckerFor the LCONZ library consortiumFor the LCONZ library consortium

By James Brunskill,By James Brunskill,

Computer Systems Consultant, Computer Systems Consultant, University of Waikato LibraryUniversity of Waikato Library

April 13, 2023© THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 2

About meAbout me

• Not a librarian

• Employed as “Computer Systems Consultant” at the University of Waikato Library

• Contact Details• Email: [email protected]

• Phone: (+64) 7 838 4323


Why do we need a spellchecker?Why do we need a spellchecker?

• English as a second language students

• Typos

• Quick reminder that we might not have typed in a query we thought we had...

• Meny poeple rn’t good spelers


The Spellchecker in ActionThe Spellchecker in Action


Hat TipHat Tip

• This wasn’t really my idea, I borrowed it from Adam Epp, who posted this on voyager-l.

I recently discovered a method by which spell check and other features can be added to WebVoyage. This technique uses the XMLHttpRequest object to redirect the WebVoyage query string to an external program (written in PHP/Perl/etc.) and return the results to a div or span tag in the WebVoyage session.

<SNIP>

---Adam EppSystems LibrarianNorthwest [email protected]://library.northwestu.edu/


ASpellASpell

• Aspell is an open source spellchecking tool

• Available from http://aspell.net/

• Perl Module available from CPAN. Text::Aspell or http://search.cpan.org/~hank/Text-Aspell/Aspell.pm


AJAXAJAX

• AJAX is officially “Asynchronous Javascript And Xml”

• Unofficially it is simply a way to use javascript to add content to a webpage.

• Because it uses Javascript it means we don’t have to modify any of the webvoyage code.


Perl CodePerl Code


Perl CodePerl Code

• Based on code developed by Dean Stringer in our ITS department.

• Dean’s code was focused on providing a list of spelling suggestions for a webpage, we needed single suggestion for a short list of words.

• I decided to get the code to either produce a list of correctly spelt words or a special message to say everything is spelt correctly.

• The Javascript then added the code to make it a linkable search, and determines if the spelling suggestion needs to be displayed at all


Config ChangesConfig Changes

[Misc_Strings] Database_Name=<div class="spellcheck" id="spellcheck"> </div>Library:

Update your opac.ini to add a line something like this:

 <script src="/spell.js" type="text/javascript"></script> <script type="text/javascript"> function updatespelling(){ if(document.forms[0] != null && document.forms[0].Search_Arg != null && document.forms[0].Search_Arg.value != "") {var SearchString = document.forms[0].Search_Arg.value; loadFragmentInToElement('/cgi-bin/spell.cgi?words=' + SearchString + '&mode=correct' , 'spellcheck');}} </script>

Add the Javascript to header.htm:


MaoriMaori

• Aspell doesn’t seem to be able to handle two languages at once. So in order for our spell checker to handle Maori we had to create our ‘dictionary’ and pretend that it was English...

• Luckily I found a website that offered a list of Maori words for spellchecking.

• http://www.maorispellchecker.net.nz


Maori SpellcheckerMaori Spellchecker


False Alarms and Strange SuggestionsFalse Alarms and Strange Suggestions

• At first there were a lot valid queries that were coming up with spelling suggestions

• Eg• ISBN

• Osama bin Laden

• Lexis Nexis

• We were also getting strange suggestions • Think the abortion, adoption debacle from Amazon.com

• Or just an example where another suggestion would be much more obvious. Eg. If someone types ‘alow’ as a search they probably mean ‘allow’ not ‘a low’


Dictionary EditorDictionary Editor


Dictionary EditorDictionary Editor

• The dictionary editor allows us to add new ‘words’ and also to specify ‘replacements’ or preferred suggestions for misspelled words.

• This is good but does require a lot of work, and there is no way we could catch every possibility

• Unfortunately, I don’t see any short term solution to this but I’m open to suggestions...

• One current limitation is that you can’t provide a ‘correction’ for a correctly spelt word.

• A big thanks to Craig and Skalk for doing the work populating these lists!


Future PossibilitiesFuture Possibilities

• Improved suggestions based on other metrics like word frequency, eliminating words that will produce no hits, etc.

• Suggesting other search tips, such as using truncation or suggesting synonyms to try.

• Warning users about alternative spellings. Eg. Organisation vs Organization

• Use this AJAX based method to add other data into webvoyage. Amazon book reviews? Book popularity ratings based on ISBN’s?


Where can I get one?

• I’ll be posting this presentation along with the associated code on our newly setup blog.

•http://librarycogs.blogspot.com


The University of Waikato

Private Bag 3105

Hamilton, New Zealand

0800 WAIKATO

www.waikato.ac.nz

Questions?Questions?

Documents

Anzreg Presentation about the spellchecker (powerpoint)