Solving The OSP Reporting Conundrum

Solving the OSP Reporting Conundrum

Lynn Ward, Principal Systems AnalystIndiana University

Indiana University

• 7 Campuses• 2 large research

campuses (Bloomington and IUPUI)

• 5 smaller regional campuses (East, Fort Wayne, Kokomo, Northwest, South Bend, Southeast)

• Stats• > 1K degree programs• > 5K FT, 2K PT Faculty• > 11K staff• > 100K students

• Sakai is our only CMS/LMS

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 2

ePortfolio Timeline @ IU

• 1998: IUPUI campus adopts a competency or ability-based approach to general education

• 2000: IUPUI campus initiates Eport project, partnering with Ali Jafari (Epsilen)

• 2003: IU and rSmart partnership on OSP 2.0 Mellon Grant; IUPUI abandons first homegrown tool

• 2004-05: early pilots with OSP• 2006: moratorium on new portfolio projects• 2007-present: growing interest and use on all

campuses


OSP Use @IU

• > 20 active portfolio projects underway, many in professional schools

• Majority focus on outcomes assessment at the program level

• Highly individualized approach to assessment

• Individual differences accommodated via customized scaffoldings and forms


OSP Support @ IU

• .5 – 2 FTE developers• .10-20 FTE designer• .25 FTE combined consultant, analyst, tech

writer, trainer, QA specialist• Outsource some QA and development

work• Growing knowledge among consultants in

campus teaching centers


The Conundrum

• Project coordinators need to aggregate, analyze, and summarize evaluation (and other types of) data

• Evaluation forms differ widely in terms of • # of rating fields• # rating scale • # labels assigned to each point on scale

• XML Programmer required to develop custom reports for each project – DOES NOT SCALE


Our Solution

• Define local form standards • Flexible• Predictable

• Publish and promote standards• Define report definitions based on those

standards• Reports “discover” the XSD structure on

the fly and render the results


Evaluation Form Standard• Field name conventions:• Rating fields (rating, rating 1, rating2, rating3 …

rating20)• Comments (comments)

• Rating fields must be represented as a selection field (drop-down menu)

• All rating fields in the form (except rating) must use the same rating scale.

• Enumeration values must be a number, “n/a”, or null

• Rating fields may not be repeatable


Customization Options

• Rating fields and comments fields may use any display name

• Rating scales may have any number of points• The display label for each point on the scale

may be any valid string• Evaluation forms may have any valid name• Each matrix cell may use a different evaluation

form.


SAMPLE REPORTS


10th Sakai Conference - Boston, MA, U.S.A. 11

Rating Summary

Rating Details


Interactive Report – Main (Matrix) View


Click to see summary report

for cell

Interactive Report – Summary View


Click to see detailed report for

cell

Click to return to main (matrix)

view.

Interactive Report – Details View


Click to return to summary view

Click to return to main (matrix) view.

Click to open cell in new window.

Matrix Cell Status Summary


Matrix Cell Status Details


Matrix Link Summary


Next Steps

• Publish form templates and documentation on how to customize

• Refine and augment existing reports based on user feedback.

• Merge Reports tool enhancements to trunk • Post report definitions to OSP library so other

institutions can use them.


For More Information

• Contact…

– Lynn Ward ([email protected])– Chris Maurer ([email protected])– Bryan Holladay ([email protected])


Education

Solving The OSP Reporting Conundrum