Embed Size (px)
Citation preview
What Did They Learn?: Programmatic Information Literacy Planning &
AssessmentMelissa Bowles-Terry
What learning outcomes are appropriate for different groups?
• What do we teach?• To which groups?• Why?
The Big Sort
Negotiate with your group to decide which outcomes go in which column.
Then we’ll present and discuss!
At University of WyomingFirst-year seminar Communications 1 Communications 2 Communications 3 Graduate students
services offered by the library
find known items by title or author
find known items lifelong professional resources (non-UW subscriptions)
use research handbooks
resources offered by the library
come up with a researchable topic and articulate a research question
find sources to meet information needs
citation mapping and advanced strategies for literature searches
citation mapping and advanced strategies for literature searches
how to use and/or borrow library materials
identify useful keywords use controlled vocabulary use controlled vocabulary use controlled vocabulary
library organization find background information, context, and definitions
use subject specific databases and/or encyclopedias
use subject specific databases
use subject specific databases
evaluate sources for relevance & authority
evaluate sources for relevance & authority
evaluate sources for relevance & authority
citation management
evaluate arguments and research methods
evaluate arguments and research methods
writing a literature review
explain why sources are cited
explain why sources are cited
introduce major journals in area
current awareness
practice citing sources cite sources appropriately cite sources appropriately
Curriculum MapInformation Literacy Curriculum MapI=Introduce, R=Reinforce, M=Master
C1 C2 C3
Gain access to information
Students will be able to identify sources that will meet their information needs
I R M
Students will be able to find known items via title or author on the web or in a database.
I R M
Students will be able to identify and use article databases relevant to their major field of study.
I R
Students will be able to find and use resources to meet professional information needs after leaving UW.
I
Understand & evaluate
Students will be able to evaluate information sources for relevance and authority.
I R M
Students will be able to evaluate a resource’s arguments and research methods.
I R
Document sources
Students will be able to explain why sources are cited in academic writing.
I R M
Students will be able to cite sources appropriately and consistently. I R M
Example: shared learning outcome & assessment tool
Learning outcome for all first-year information literacy classes:• Students will be able to identify keywords
that represent a research topic and use keywords to find useful information sources.
Criteria No results – 0 Beginning – 1 Developing – 2 Exemplary –3
Relevance Lists no keywords or impossible to tell whether keywords are relevant. Students = 7
Lists keywords that are not relevant for the research question. Students = 28
Lists keywords that are mostly relevant OR lists not enough keywords to express all aspects of the research question. Students = 139
Lists several keywords that express all aspects of the research question.
Students =304
Quality Lists no keywords. Students = 5
Lists keywords that are meaningless (effects, impacts, etc.) and/or keywords that will retrieve biased results (negative effects, positive impacts, etc.)OR uses all natural languageStudents=46
Most keywords are meaningful and will retrieve results on most sides of the issue. Does not use natural language. Students=158
Keywords listed will lead student to subject/thesaurus terms. Students=269
Results Lists no results or indecipherable results. Students=54
Lists citation information for an article that is not relevant to the research topic OR citation is incomplete.Students=37
Lists citation information for an article that is somewhat relevant, but is too general or broad to answer research question.
Students=73
Lists citation information for an article that is relevant and appropriately focused for the research topic.
Students=314
Students are quite adept at coming up with keywords.
They are better at coming up with relevant keywords than coming up with quality keywords (for a few reasons I’ll discuss).
Analyzing Results: Keywords
• 54 out of 478 students (over 10%) left the library session without listing one potentially useful article. There may be a few reasons for this:– Students didn’t have enough time to complete the
exercise– Students didn’t know enough about the parts of a
citation to list a complete citation for an article– Note: In many cases where students didn’t have an
article listed, they did have a good search process. Therefore, I do not consider this the sole (or even the most important) measure of success.
Analyzing Results: Citations
Broad Topic
Keywords
Non-useful article
Final article that is too technical or advancedTopic: Nuclear Energy TechnologiesResult: “Transfer of Elements Relevant to Radioactive Waste From […] Boreal Plant Species,” from the journal “Chemosphere,” April 2011. No author or page #’s cited.
Hit or miss approach (taking first article that comes up)Topic: Effects of Nutrition/Food on the BodyResult: “Characterizing Whole Diets of Young Children From Developed Countries and the Association Between Diet and Health: a Systematic Review”
Less Successful Processes
Focused Topic
Keywords
Focused article
Question with several parts; statements to prove or disproveTopic: The internet has opened the door to vast amounts of information. What has been the impact of this technology on education? How has it changed the way we learn? Does it tend to enhance or impede critical thinking?Result: “The Effects of Internet Based Destruction on Student Learning.”
More Successful Processes
• time spent developing synonyms• more than 2-3 main concepts
Too many keywords
• impacts• benefits• advantages/disadvantages
Filler words
• “checking software for errors”• “autonomous, submersible ROV’s”• “proofreading software”
Phrases
Common Problems: Keywords
• Keywords need to have contentUnclear results
• Valuable keywords vs. filler
Lacks specificity
• Make a list of keywords to avoid
Disadvantages are hard to
find
Common Problems: Results
Focus topic or question
2-3 concepts or keywords
SearchRevise
keywords – more specific
Search
Towards a Process Approach
• Students perform better with a research prompt or at least plenty of direction in research question choice. The most difficult AND the most important part of the research process is selecting a topic and narrowing/refining the research question. – Questions or topics are most successful when specific– It is often useful to have students work on solving a problem
or to have them prove/disprove a statement– Classes/research projects with a theme (robotics, foodways,
American culture) give students more direction for a research project.
How can librarians support the process of coming up with a research question?
Lessons Learned
• We should focus more on the search process and not just initial keyword selection. Prompting students to search and then search again is really important.– More guidance on worksheet: limit initial keywords to 2-3,
outline a process for revision– Steer students away from quantity and instead work on
quality of keywords. (Spending too much time on developing a long list of keywords may in fact be counterproductive.)
– Start with broad keywords, and use phrase searching judiciously (if at all)
How should we prioritize our time with students?
Lessons Learned
• The “hit-or-miss” approach to choosing an article for use is a common issue, and one that is difficult (but important) to address in a 50- or 75-minute session. Students tend to take the first somewhat relevant article that they find.
How can we encourage students to persist?
Lessons Learned
– How can librarians support the process of coming up with a research question?
– How should we prioritize our time with students?
– How can we encourage students to persist?
Melissa Bowles-Terry [email protected]
Questions? Comments?
