Analysing and supporting the process of co-designing inquiry-based and technology-enhanced

learning scenarios in higher education

Iolanda Garcia, Elena Barberà, Xavier Pujol & Mireia Usart (eLearnCenter, UOC)Anna Escofet, Marc Fuertes, Begoña Gros, Marta López & Ingrid Noguera (UB)

Meritxell Cortada & Marta Marimón (UVIC)

Design2Learn Project EDU2012-37537 Plan Nacional I+D+i.

http://design2learn.wordpress.com/

Content

• Overview and rationale of the project

• Aims• Methodology• Research instruments • Preliminary results• Discussion

Design2Learn project

Aims to study the development of learning scenarios that are more authentic, contextualised and focused on learners, through a co-design process involving students and teachers in the negotiation of the design principles of such scenarios

DESIGN PRINCIPLES

• Inquiry Based Learning • Technology Enhanced

Learning

The design is by nature iterative and collaborative. It requires discussion, reflection, critique and implementation, so it works better in teams in which there is a complementarity of skills and knowledge. Being a cognitively demanding task, it requires tools and representations that allow for abstraction to be managed and understood (Goodyear &Retalis, 2010)

Assumptions1. Co-design processes involving students and teachers can

facilitate the adoption of an inquiry-based learning model mediated by a more mature and autonomous use of technology by students in open and networked environments.

2. Students’ participation in the co-design process can integrate their perspective promoting deeper learning.

3. The use of tools for representing teaching and learning practice, can facilitate the co-design process.

RESEARCH QUESTIONS

RQ1. What is the role of representation instruments in supporting a co-design process (case stories, design patterns, storyboards and diagrams) of IB and TE/networked learning scenarios?

RQ2. How are participants’ roles and levels of intervention negotiated, assigned and managed throughout the co-design process?

RQ3. What are the stages and critical issues to consider in the process of co-design?

Design-Based Research

A systematic, but flexible methodology aimed to improve educational practice through iterative analysis, design, development and implementation, based on collaboration between researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories.

Wang and Hannafin, 2005

CONTEXTUALISE & EMPATHISE

PROBLEMATISE &DEFINE

DOCUMENT& IDEATE

CONCEPTUALISE & PROTOTYPE

IMPLEMENT & ASSESS

Identify problem/s

related with teaching/ learning practice,

define and operationalise

the design challenge to

address

Get involved to know the

participants’ context and needs, build

common ground and

understanding. Reflect and

share issues in your practice

Explore other experiences and decide

suitable design principles and pedagogical approach. Generate

variety of ideas to address the

design challenge

Conceptualize a learning

scenario able to solve the

design challenge and turn it into a visual and

tangible model that can be

implemented and tested

Implement the prototype in real context, monitor and

collect feedback about the learning

experience. Assess, reflect and improve the designed

scenario

iterate – document – reflect – refine – iterate …

Design-based research framework

Problem finding Problem solving Solution testing

Theory-practice loop

8 learning subjects• Two different

university models: online and blended

• About 4 UOC / 4 UB teachers

• About 16 students with different profiles

• Different disciplines

Teachers as designersas guides and facilitatorsas researchers Teacher professional development

Students as designers (listening to student voice) as producersas researchersStudent engagementLearning agency and control

Co-design context

Research plan

Reported period

Co-design process

Co-design process Mediating artefacts to represent practice

•Narratives•Concept and force mapping•Matrices and templates•Storyboards•LD tools •Diagrams•Design patterns

CONTEXTUALISE & EMPATHISE

PROBLEMATISE &DEFINE

DOCUMENT& IDEATE

CONCEPTUALISE & PROTOTYPE

IMPLEMENT & ASSESS

WORKSHOPS 3-4

Instruments:. Template design challenge. Forcemap. Matrix problems/solutions

WORKSHOPS 1-2

Instruments:. Case story. Matrix problems/design principles. Concept mapping. Vocabulary (IBL based). Guidelines reflective questions. Open visual representation

WORKSHOP 5

Instruments:. Matrix problems / solutions. Template IBL based scenario

WORKSHOPS 6-7

Instruments:. Storyboard (paper prototyping). Diagram Compendium LD (paper prototyping)

WORKSHOPS 8-11

iterate – document – reflect – refine – iterate …

September 2014 ----------------------------------------------- February --------------July

Problem finding Problem solving Solution testing

Theory-practiceloop

Research instruments and analytical framework

- Post-workshop questionnaire (broad open question + Likert)

- Observation

RQ1: data analysis

RQ1. What is the role of representation instruments in supporting a co-design process (case stories, design patterns, storyboards and diagrams) of IB and TE/networked learning scenarios?

WORKSHOP 1INSTRUMENTSCase story

WORKSHOP 1 Matrix problems/design principlesWORKSHOP 1 Concept MappingWORKSHOP 2/3 Vocabulary (IBL based)WORKSHOP 2/3 Guidelines /reflective questions (IBL / TEL based)WORKSHOP 2/3 Open visual representationWORKSHOP 3 Video (connectivism)WORKSHOP 4 Template (design challenge)WORKSHOP 4 ForcemapWORKSHOP 4 Matrix problems/tentative solutionsWORKSHOP 5 Template (IBL based scenario)WORKSHOP 6 Storyboard (paper prototyping)WORKSHOP 7 Diagram Compendium LD (paper prototyping)

CATEGORIESEase of use

Concept clarification

Meaning negotiation

Reflection own practice

Practice / problems sharing

Solutions elicitation / brainstorming

Expert knowledge sharing

Discussion and decision taking

Learning scenarios instantiation / prototyping

Learning scenarios sharing / communicating

Design principles integration IBL

Design principles integration NL/TEL

Qualitative analysis:

- Workshops observation- Perception questionnaire

RQ1: Findings

• Instruments were very useful to support the co-design process, although they need to be used in combination.

• Using too many instruments/guidelines in the same activity/session is counterproductive.

• Need to improve the instruments used in the ideation phase: brainstorming and ideas generation.

• Intruments need to address more directly the design principles, specially the TEL ones.

• The prototyping process helps to systematise practice, specially for less experienced teachers.

• Need to rethink prototyping session with Compendium LD in order to get a better value of visual representation.

RQ1: Teacher’s comments (perception questionnaire)

“I think the hassle of completing the story board is compensated by having an explicit, clear and comprehensive representation of all the elements involved”. (A)

“The definition of a set of scenes and a number of characteristics of each scene makes what needs to be defined perfectly dimensioned. On the other hand, let’s first outline the general structure to go deeper into each scene afterwards”. (X)

"The meeting was very similar to the previous one. I do not see much difference between information that is provided for each scene and the overview that gives each scene separately”. (R)

“The use of a graphical modelling system seems like a good idea and should facilitate the task. However, semantic relations among components are not clear and to what extent this tool complements or replaces the one previously employed.” (R)

RQ2 & RQ3: data analysis

Qualitative analysis

- Workshops observation- Perception questionnaire

RQ2. How are participants’ roles and levels of intervention negotiated, assigned and managed throughout the co-design process?

RQ3. What are the stages and critical issues to consider in the process of co-design?

A Co-design GROUP DYNAMICS B Co-design CORE

A1 Attitude B1 Key issues

A2 Intervention B2 Design principles

A3 Decision taking B3 Tasks

A4 Conflicts B4 Emerging concepts / ideas

A5 Moderation B5 Problems / Difficulties

A6 Role taking B6 Reflective practice

A7 Collaboration B7 Time management / Stages

RQ2 & RQ3: Findings

• Importance of group configuration, anticipating absences, etc.

• Group dynamics can be very different depending on participant teachers and researchers. At least 2 teachers involved per each context.

• The design always sits in a certain context and participants have difficulty in abstracting and thinking in more general terms.

• It is important to avoid keeping the focus on the subject content, especially in teamwork among teachers.

• It is useful to clarify concepts and practices associated with design principles and recall them frequently.

• Researchers must be knowledgeable of the dynamics and the correct use of the tools to redirect the process. Mistakes are not always discussed in sufficient depth or doubts not enough clarified.

RQ2 & RQ3: Findings

• Great opportunity for reflective practice and for problem sharing.

• The structure of the sessions is a key aspect to ensure progress: avoid spliting tasks between sessions, include a space for sharing and for reflecting on what was done.

• Short tasks with mixed groups work better. Tasks that are too long hinder productive discussion and emergence of new ideas.

• The final stage involves reflection and writing development outside the sessions and costs more to be completed.

• TEL design principles are more difficult to implement: the problem is placed in the virtual campus (UB / UOC).

“I think it's very appropriate to use concrete tasks with an allocated time as it gives flexibility and facilitates everyone’s participation”. (F)

“The time setting should be improved as when we started discussing the results of the previous week there were gaps in some aspects not remembered. It could have been avoided if we had finished in the previous session”. (F)

“The application of the IBL methodology proves to be a good tool to propose new ways of targeting teaching. Almost without realizing it, we are proposing highly innovative activities, in my case I am totally rethinking the orientation of a subject. This generates high degree of uncertainty”. (R)

“These important methodological changes inevitably lead to rethinking the role of the teacher. Perhaps there should be a more explicit alignment between the methodology and the teaching model”. (R)

RQ2 & RQ3: Teacher’s comments

“It has been very instructive to know other teaching experiences and highlight commonalities in the various stories, despite the diversity of subjects”. (X)

“The session has been very useful to analyze the real causes of some problems and the factors that may influence them. The methodology is particularly effective to reveal complex relationships between various elements that apparently might seem disconnected”. (R)

“To relate our experiences with specific principles (design principles) helps organize thinking, but it can also be somewhat forced”. (A)

“I would say I worked very naturally, and do not identify substantial differences in the way teachers design activities normally. I mean in the sense that the design principles, if we are incorporating them, we did it naturally, without worrying too much or remembering to check whether we were considering them”. (A)

RQ2 & RQ3: Teacher’s comments

Results RQ2 & RQ3 (perception questionnaire)

Evolution of the perception of UOC teacher across the workshops (F)

Likert scale (1-5)

Results RQ2 & RQ3 (perception questionnaire)

Evolution of the perception of UB teacher across the workshops (X)

Likert scale (1-5)

Results RQ2 & RQ3 (perception qüestionnaire)

Evolution of the perception of UOC teacher across the workshops (F)

Likert scale (1-5)

General conclusions

• Need to identify the design activities and instruments used and stimulate reflection on their function and usefulness in the different sessions.

• Difficulty of progressing in analysing data to fuel the co-design process as it develops.

• It is difficult for researchers to stand and reflect on their own role/performance in the co-design process.

• Importance of sharing/discussing findings with participants.

General conclusions

“The key tensions of co-design persist across phases: teachers never have enough time”.

“The social dynamic between teachers and researchers evolves over time, particularly with regard to agency and ownership. In the beginning, teachers did not begin with a strong sense of ownership in the project or a clear sense of the roles that they would play”.

“A key benefit of engaging the teachers in co-design was professional development. The co-design process offered teachers a chance to develop and refine their own ideas about teaching”.

(Roschelle & Penuel, 2006)

“Personally throughout the session I felt increasingly motivated and I think the other group members too. One expression of this is that we all felt that the time had passed so quickly”. (X)

“The main motivation is to detect an improvement in each session and begin to shape an activity applicable in the next academic year”. (X)

“This systematic reflection on our work is something we have little opportunity to do, surely we miss it with frequency, and when we do, it is in a very intuitive way, very focused on solving the specific problem right now, but without such a global view of the relationships between the different elements involved and the time to resolve them”. (A)

RQ2 & RQ3: Teacher’s comments

Thank you very much!

Iolanda Garcia & Begoña Gros igarciago@uoc.edu / bgros@ub.edu

Design2Learn Project EDU2012-37537 Plan Nacional I+D+i.

http://design2learn.wordpress.com/