Caliper: tapping into the Learn Activity Stream

Caliper: tapping into the Learn Activity StreamDaniel Knight Product Development @ BlackboardDan Rinzel Product Manager for Analytics @ Blackboard

Agenda – Q’s we will hope to A today

• What is IMS Global, Caliper Learning Analytics, why does it matter and how is Blackboard involved?

• What is the Learn activity stream?

• What do stream events look like?

• How can schools use the stream?

• How can a tool provider participate in the stream?

Statements regarding our product development initiatives, including new products and future product upgrades, updates or enhancements represent our current intentions, but may be modified, delayed or abandoned without prior notice and there is no assurance that such offering, upgrades, updates or functionality will become available unless and until they have been made generally available to our customers.

Blackboard & CaliperWork in Progress

Caliper Learning Analytics

• IMS Global as an organization

• Interoperability Standards – LTI, LIS, Common Cartridge, Extended Transcript, Caliper

• Create a viable tool ecosystem for .edu that avoids vendor lock-in

• LTI focuses on launch semantics and SSO, includes grade post-backs

• Caliper focuses on learning event interoperability, to get away from the data silo problem

Activity Stream: Service Architecture

Old School Enterprise approach

Record activity in the transactional database and run reports directly, or execute an expensive nightly ETL

Activity Stream: Service Architecture

Cloud approach

Send atomic events to a separately elastic telemetry service dedicated to collection and rollup

Activity Stream: Event Instrumentation

• More events, with better context, reducing ”noise”

• Compliant with & extends initial IMS Caliper standard

• Available to Learn SaaS deployments and planned for Enterprise 9.1 if hosted with Blackboard

• Exploring ways around technical constraints for self-hosted customers

The Goal

Integrated insights for teachers into what students are doing across tools and experiences

Usage data first, progress data to come

So what does that really mean?A look behind the curtain

Caliper Metric Profiles

• Learn SaaS was the first LMS certified against the Caliper 1.0 specification in October 2015, for 5 metric profiles: Session, Assessment, AssessmentItem, Assignable, Outcome

• Other metric profiles exist in the spec, specific to types of tools (e.g. MediaEvent, ReaderEvent)

The goal is to encapsulate events & context in a shared semantic model so that there is consistency in what can be expected in each event

Caliper Event Syntax

• JSON-LD (JSON for Linked Data)

• Defines a context and the relationships between involved entities (e.g between a user and an assessment)

• Transport-compliant custom events (e.g ForumEvent)

• Extensions allow BB to include extra specific data in related entities

Example Stream Event Syntax

{"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/sessions/21622015C7998FEA7A7824F053D514C8"}



{ "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type":"http://purl.imsglobal.org/caliper/v1/lis/Person", "extensions":{ "bb:user.id":"_50_1", "bb:user.externalId":"demo-39" },}



{ "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/gradableItems/_7219_1", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type”: "http://purl.imsglobal.org/caliper/v1/AssignableDigitalResource", "name": ”Chapter 1 Test", "dateToSubmit":"2016-07-28T13:04:50.963Z", "maxAttempts":0, "maxSubmits":0, "maxScore":25}



{ "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/gradableItems/_7219_1/attempts/_13000_1", "@type":"http://purl.imsglobal.org/caliper/v1/Attempt", "assignable": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/gradableItems/_7219_1", "actor":"https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "count":1, "startedAtTime":"2016-06-02T15:25:47.815Z",}



{ "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/applications/learn", "@type": "http://purl.imsglobal.org/caliper/v1/SoftwareApplication”}



{ "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/lis/CourseOffering", "extensions":{ "bb:course.id":"_2972_1" }, "courseNumber":"GAT-1",}



{ "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817/members/fd6adc5678514c25bd2569c266c9bf65", "@type":"http://purl.imsglobal.org/caliper/v1/lis/Membership", "extensions":{ "bb:user.externalId":"demo-39", "bb:user.id":"_50_1", "bb:course.id":"_2972_1", "bb:course.externalId":"GAT-1" }, "member": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "organization": "https://caliper-mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1-a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817", "roles": [http://purl.imsglobal.org/vocab/lis/v2/membership#Learner], "status":"http://purl.imsglobal.org/vocab/lis/v2/status#Active"}

How do we join the ecosystem?

Configuring Learn SaaS to relay events to an external Caliper record store

Enabling the Caliper “handshake” between Learn and an LTI Tool

New bb-manifest.xml permission block<permission type="blackboard.platform.blti.CaliperProfileCallbackPermission" name="*" actions="*"/></permissions>

Event Instrumentation

If your system already has event instrumentation:

• Map existing event instrumentation to Caliper metric profiles• Transform events to be compliant with the Caliper standard event syntax

If you do not currently record events:

• Determine what events are meaningful• Investigate the available Caliper Metric Profiles

Caliper Sensor Implementations

Standard implementations of the Caliper Sensor are available on GitHub:

• Java: https://github.com/IMSGlobal/caliper-java-public• Node.js: https://github.com/IMSGlobal/caliper-js-public• PHP: https://github.com/IMSGlobal/caliper-php-public• Python: https://github.com/IMSGlobal/caliper-python-public• Ruby: https://github.com/IMSGlobal/caliper-ruby-public• .NET: https://github.com/IMSGlobal/caliper-net-public

https://github.com/IMSGlobal/caliper-java-public



https://github.com/IMSGlobal/caliper-js-public

https://github.com/IMSGlobal/caliper-js-public

https://github.com/IMSGlobal/caliper-php-public



https://github.com/IMSGlobal/caliper-python-public



https://github.com/IMSGlobal/caliper-ruby-public



https://github.com/IMSGlobal/caliper-net-public



xAPI & the future

There is a similar specification in the world of SCORM – Blackboard is working with both IMS & ADL to drive towards a convergence that we hope will make it easier on everyone in the ecosystem.

Other goals for Blackboard & Caliper

• Support for Caliper 1.1 Discussion profiles

• Certification as a record store when that is available

• Convergence or “ambidexterity”• Support for 9.1 Enterprise releases

Questions?

[email protected] Product Development @ Blackboard

[email protected] Product Manager for Analytics @ Blackboard

mailto:[email protected]

mailto:[email protected]

Education

Caliper: tapping into the Learn Activity Stream