IAd-learning: A new e-learning platform by José Antonio Omedes at Big Data Spain 2015

iAdLearningNext Generation E-Learning

Oct 2015. Version [email protected]

Copyright iTopTraining. All rights reserved.

Topics

Page 2

1. Introduction and iTopTraining

2. Where are we coming from? What are we trying to achieve?

3. iAdLearning architecture

Copyright iTopTraining. All rights reserved.Page 3

1. Introduction and iTopTraining


iTopTraining

Privately held company

Based in Madrid (we have presence in Guadalajara – México – )

Founded in July 2013 Continuation from a previous company

Two main business lines: eLearning platforms on the cloud (SaaS)

eLearning innovation Research and Development of innovative e-Learning software solutions

~50% people works on Research and Development

Page 4


iTopTraining Technology Landscape

Page 5

Day to Day Operations(SaaS eLearning Platforms)

Cloud Technologies (AWS, Azure)

PHPJavascript

ApacheMySQL (Aurora)

MemCache

Moodle

Innovative eLearning Products(includes iAdLearning)

Cloud Technologies (AWS, Azure)

AngularJSNodeJS, D3jsJava, Scala

NGINXCassandra, MongoDB

Neo4j, Redis, ElasticSearch

Apache Spark

Machine Learning, NLP


2. Where are we coming from and what are we trying to achieve?


Change of Learning Paradigms (*)

Page 7

Linear Learning

Static Training Contents

Free Learning

Content Adapts to Users

(*) We are focusing on those factors relevant for our discussion but there are other changes such us the important irruption of mobile devices on eLearning

“Mimics” Internet Navigation

Learn “by interest”

Every user is different and has different learning needs even

in the context of the same course

Instructor Lead Learning eLearning

Users access to training courses whenever they want

to and wherever they are


Change of paradigm brings new challenges …

Page 8

e-Learning Volume

Content Transformation for eLearning

1

2

• eLearning market CAGR ~20% (depends on studies)

• +38% of company training is delivered online

• MooCs are consolidating

DELIVERY

ANALYSIS of INFORMATION

(is this a challenge?)

• Very man intensive task

• Very time consuming taskCOST

End User Learning Experience3

• Users are “left alone” …QUALITY

INDIVIDUAL NEEDS


Change of paradigm brings new opportunities (1) …

Page 9

Improve eLearning Experience

Using all information gathered by eLearning platforms

MACHINE LEARNING

BIG DATA

• Improving the way contents are visualized and navigated

• Guiding users to discover the content which is relevant for them

• Providing automatic feedback to trainers about the quality of the content materials and about the user experience when using those contents


Change of paradigm brings new opportunities (2) …

Page 10

Facilitate migration of contents to eLearning

• Generating eLearning contents in a format that facilitates free navigation and “learning by interest” from non-native eLearning formats (Word, PDF, Powerpoint)

• Providing a cost effective solution for content transformation

NATURAL LANGUAGE PROCESSING

MACHINE LEARNING

Processing non native eLearning contents


Wasn’t this presentation supposed to be technical?


Key Take-aways

The way people learn is changing New formats (eLearning growth)

New ways of using existing formats such as e-Learning (free navigation, learn by interest, etc.)

Growth of e-Learning provides us with an important amount of information that properly analyzed may dramatically improve the user’s eLearning Experience

Requires technology

Requires proper algorithms

Content Transformation is key to help trainers and training companies move into e-Learning

Benefit from all the information analysis based on e-Learning data

Follow the market trends (continue in the market)

Page 12


What is iAdLearning?

Page 13

iAdLearning is a software that makes the e-Learning platforms adapt to the individual needs of the students -adaptive e-Learning- providing a unique personal e-learning experience throughout the course materials while maximizing the students achievements by making intelligent study suggestions based on previous learning

experiences


iAdLearning Principles

Page 14

ADAPTIVEEach student has a complete different experience throughout the course content based on his/her background

Students can navigate through the entire training materials in a non lineal fashion based on their needs or interests

NON LINEARLEARNING

MAKE USE OF EVERYBODY’s EXPERIENCE

The system uses previous learning experiences to recommend customized learning paths throughout the content

EXISTING CONTENT

IS EASY TO IMPORT

iAdLearning allows to easily import existing non e-learning native training materials (Word, PDF, PPT)


How does iAdLearning work?

Page 15

1Importing documents, analyzing them and establishing semantic relationships among the document individual content components creating, as a result, a user navigable graph (knowledge network)

2Allowing free navigation through the knowledge network not only based on the original content structure but also on the user preferences and interests

3Discovering successful navigation patterns that maximize user performance through the course materials in a way those can be suggested to students during their learning experience

IMPORT

VISUALIZE

RECOMMEND



Page 16

Set of Documents

(Word, PPT, PDF)

Semantic AnalysisSemantic Analysis

ACE

1

1.2.2

2

Course GraphKnowledge Net

1.1 1.2

1.2.1

• A set of documents belonging to an e-Learning course is imported, analyzed and broken down into semantically relevant fragments called ACEs (Atomic Content Elements). ACEs represent course fragments that can be individually studied and understood

• iAdLearning creates a graph representing the relationships between the different ACEs:

• Structural Relationships (blue lines): represent connections related to the course structure as initially established by documents authors

• Semantic Relationships (red lines): represent connections created due to the similarity of the contents being described by the connected ACEs

IMPORT



Page 17

IMPORT

Section 1Section 1

Section 1.1Section 1.1


Section 2Section 2



Section 2.2.1Section 2.2.1


Section 7Section 7

…




ACE 1ACE 1

ACE 2ACE 2

ACE 3ACE 3

ACE 4ACE 4

ACE 5ACE 5

ACE 6ACE 6

ACE 7ACE 7

0.95

0.72

ACE 1 ACE 1

ACE 2 ACE 2

ACE 3 ACE 3

ACE 4 ACE 4

ACE 5 ACE 5

ACE 6 ACE 6

ACE 7 ACE 7

FragmentationAnalysis of

Relationships



Page 18

IMPORT

STRUCTURE OF THE DOCUMENT AS DISCOVERED BY iAdLearning



Page 19

ACE

1

1.2.2

2

Course GraphKnowledge Net

1.1 1.2

1.2.1

VISUALIZE

EXAM { 1, 1.1, 1.2.1, 2, 1.2.2, 1.2, EXAM}

{ 1, 1.2, 1.2.1, 1.1, 1.2.2, 2, EXAM}

USER 1

USER 2

User Navigation Patterns

• When accessing the e-Learning course contents, users navigate through the graph (*)

either following the initial course structure (blue connections) or jumping into other related content elements according to their preferences/interests (red connections)

• User navigation patterns (visited ACEs and their sequence) are recorded together with the user evaluation results (exam results), in a way they can be further analyzed

(*) On top of the graph navigation, iAdLearning offers other navigation views such a tree-view

(**) Contents created with iAdLearning can be visualized in Moodle through a plugin



Page 20

VISUALIZE

STRUCTURALRELATIONSHIPS

SEMANTICRELATIONSHIPS



Page 21

{ 1, 1.1, 1.2.1, 2, 1.2.2, 1.2}

{ 1, 1.2, 1.2.1, 1.1, 1.2.2, 2}

USER 1

USER n

User Navigation Patterns

… Behavioral Analysis

Behavioral Analysis

• Navigation patterns are analyzed using advanced machine learning techniques

• As a result of the analysis, users are grouped according to their course navigation pattern and a set of relevant variables (age, studies, knowledge of the topics covered by the course, etc.). iAdLearning establishes recommended navigation paths for each group of users based on the recorded course performance.

…

Recommended Navigation Paths

{ 1, 1.1, 1.2, 1.2.1, 1.2.2, 2}

{ 1, 1.1, 1.2.1, 2, 1.2.2, 1.2}

age, studies, previous knowledge, evaluation mark, etc.

RECOMMEND



Page 22

NEW USER

ClassificationClassification RecommendationRecommendation

1

1.2.2

2

1.1 1.2

1.2.1

1

2

3

4

• When a new user comes into the e-Learning platform, it is classified according to the criteria established by the behavioral analysis already performed on the navigation patterns of preceding users

• Based on the user classification, iAdLearning suggests an individual recommended navigation path targeted to maximize the user performance on the course

RECOMMEND



Page 23

STRUCTURALRELATIONSHIPS

SEMANTICRELATIONSHIPS

RECOMMENDEDNEXT NODE


3. iAdLearning Architecture


Architecture Principles

Page 25

All user actions are being logged for further analysis (logging intensive application)

Separate Application and “Logging” Information Flows

Individual Functions ScalabilityThe different components of the application must be individually scalable according to the real needs

The different parts of the application must be implemented in different logical entities that communicate through APIs

Function Separation via APIs

Redundancy / Resiliency All the system must be redundant

Cloud Based / Cloud Provider Independent

The system must reside on the cloudThe system must be independent of the cloud provider


iAdLearning Architecture (version 1)

Page 26


iAdLearning Architecture

Page 27

IMPORT

• Traffic coming from the Front End is redirected by NGINX into the Web Server

• The Web Server sends the documents to import to a pending jobs queue handled by RabbitMQ.

• Whenever there is processing power available, they are analyzed, fragmented and the “knowledge network” (graph) gets generated.

• Results of the import process are persisted into MongoDB, Neo4j, Elastic Search and Cloud Storage.


•Regular application flow is directed through NGINX into the Web Server

•The Web Server gets information to visualize from MongoDB / Neo4j

•When required, the front end component pulls content from the cloud storage

•Information persisted in MongoDB / Neo4j corresponds to:

• Training contents• Recommend paths through the content materials

iAdLearning ArchitectureVISUALIZE


•User “action events” come to NGNIX which redirects the traffic to the logging server.

•The logging server persists the user action events into Cassandra.

•Logs stored in Cassandra are periodically analyzed by a batch process which runs the analysis in Apache Spark.

•Results of the Analysis (Recommended Paths through the content) are persisted into MongoDB.

iAdLearning ArchitectureRECOMMEND


iAdLearning Architecture - Technologies

FRONT

PROGRAMMING LANGUAJES

DATABASES SEARCH ENGINES

ANALYTICS

CLOUD PROVIDERS

QUEUEING


iAdLearning Architecture – Technologies

Page 31

Members of DataStax Startup Program

DataStax Enterprise 4.8(DSE)

Spark 1.4

Cassandra 2.1

OpsCenter

DevCenter

Spark Cassandra Drivers


iAdLearning Architecture – Thinking points

Page 32

We are permanent newbies …

We live in a world of complex technologies continuously evolving

Solutions need to be simplified

There is always a new technology/system that may play a role in your solution

Management of solutions and cost need to be part of the equation

We miss “relational” …

Relational databases are great !!!

Sometimes you cannot use relational databases but they are still great !!!



Page 33

Security is important from day 1

Don’t treat security as a marginal element in your system until you go into production

Automate your deployments

How many times are you going to install the same type of node?

We use Chef

Networking is important from day 1

Don’t treat networking as a marginal element in your system until you go into production



Page 34

Reuse and go to the source

There are tons of libraries out there that may be used in your development

Go to the source … minimize the library over library effect ...



Page 35

1Simplify the way the front end is delivered to the user by using a Content Delivery Network

Content is static and the dynamic part (JavaScript) runs at the user browser

2 Reduce number of databases

A graph oriented database is not needed for the time being

3 Reduce the number of software components

Simplify the way importing of documents is performed (queuing part)

4 Introduce Deployment Tools and Continuous Integration

Reduce the deployment of new software cycle



Page 36


Thank you very muchiTopTraining

[email protected]