Capturing Learning Paper

7/28/2019 Capturing Learning Paper

1/48

Abstract

This pilot study employs a design-based research methodology (Van den Akker, 2006) and

critical realist analytical framework (Danermarket al., 1997), to evaluate the design and

implementation of an online learning space, for a group of approximately 60 initial teacher

education students in their final year.

The inquiry aims to describe how learning happened in the online space, and explain how the

design of the online space influenced learning. Five contextually-sensitive design principles are

hypothesised, and these are interrogated through the lens of Mezirows transformative learning

theory (2009).

The design principles hypothesise that pubic yet safe online spaces have the potential to foster

student voice and the performance of professional identities. Is is suggested that transformative

learning has the potential to emerge if the technology affords an opportunity for reflexive

engagement with this re-performance of identity.

A conflict between the realist ontology and constructivist epistemology of the critical realism

meta-framework is highlighted as a major challenge for educational research, as well as the

complex nature of inquiry into socio-technological learning. The value of complexity theory

Contents

Abstract

Preface: digital scholarship

Introduction

Co ntext: educ ational turbulence

Context: online learning spaces

Appro ach: critical realist hacking

Approach: design-based research

Approach: integrated methods

Approach: critical realist analysis

Analysis: des cribing the des ign

Analysis: describing learning

Analysis: e xplaining learning

Analysis: theorising learning

Recontextualisation

Looking bac k: limitations

Looking forward: possibilities

Postscript: musings on authority

Search

CAPTURING LEARNING

Home About References MATHAGOGY PEPS MCCREA

PDFmyURL.com
http://pepsmccrea.com/capturing-learning-paper/#11http://pepsmccrea.com/capturing-learning-paper/#9http://pepsmccrea.com/capturing-learning-paper/#7http://pepsmccrea.com/capturing-learning-paper/#2http://pepsmccrea.com/capturing-learning-paper/#2http://www.pepsmccrea.com/capturing-learning-paper/http://www.pepsmccrea.com/capturing-learning-paper/?page_id=526http://www.pepsmccrea.com/capturing-learning-paper/http://www.pepsmccrea.com/capturing-learning-paper/http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://creativecommons.org/licenses/by/3.0/http://pepsmccrea.com/capturing-learning-paper/#17http://pepsmccrea.com/capturing-learning-paper/#16http://pepsmccrea.com/capturing-learning-paper/#15http://pepsmccrea.com/capturing-learning-paper/#14http://pepsmccrea.com/capturing-learning-paper/#13http://pepsmccrea.com/capturing-learning-paper/#12http://pepsmccrea.com/capturing-learning-paper/#11http://pepsmccrea.com/capturing-learning-paper/#10http://pepsmccrea.com/capturing-learning-paper/#9http://pepsmccrea.com/capturing-learning-paper/#8http://pepsmccrea.com/capturing-learning-paper/#7http://pepsmccrea.com/capturing-learning-paper/#6http://pepsmccrea.com/capturing-learning-paper/#5http://pepsmccrea.com/capturing-learning-paper/#4http://pepsmccrea.com/capturing-learning-paper/#3http://pepsmccrea.com/capturing-learning-paper/#2http://pepsmccrea.com/capturing-learning-paper/#1http://pepsmccrea.com/http://www.pepsmccrea.com/capturing-learning-paper/?page_id=1084http://www.pepsmccrea.com/capturing-learning-paper/?page_id=70http://www.pepsmccrea.com/capturing-learning-paper/?page_id=526http://www.pepsmccrea.com/capturing-learning-paper/http://www.pepsmccrea.com/capturing-learning-paper/


2/48

(Davis & Sumara, 2008), and the hacker ethos (Suiter, 2010) is explored in this context.

This study attempts to rise the the responsibilities of scholarship in a digital age, and has been

conducted using digital, networked and open practices where appropriate (Weller, 2012).

The online version of this report can be found at http://www.pepsmccrea.com/capturing-learning-

paper/ and a pdf version of the main body of the report can be accessed by clicking here.

Preface: digital scholarship

This is a report about my research into learning and design for learning in a digital age.

However, it also represents my developing understanding of what it means to be a scholar in these

times, and in particular the ethical choices facing contemporary researchers. In light of this, I have

chosen to approach both the doing of the research and the reporting of the research using the

following digital, networked and open practices:

I have been using Twitter to connect to networks of expertise, providing access to highly

current and relevant content, and as a way to share my thinking with others

I have been blogging about my research, which has encouraged me to reflect on and articulate

my thinking, and to create a space for discussion and feedback on my ideas

I have analysed data using tools that allow them to be represented digitally and shared openly

(after they have been anonymised) for reuse and scrutiny in meta-analyses and parallel studies

I am publishing this report openly on the internet with a Creative Commons license so anyone

(with the internet) can free ly access and build upon my work

I have built this report as a hyperlinked document to offer readers a connected experience,

where they can be transported directly to source material

I have designed this report so that readers can comment on it if they wish

I have linked this report to my digital identity so readers can easily evaluate my credibility as

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/fki3http://www.pepsmccrea.com/capturing-learning/wp-content/uploads/2012/03/www-capturinglearning-com-3.pdfhttp://www.pepsmccrea.com/capturing-learning-paper/


3/48

a researcher, and connect with me if they wish

As you can see from this list, digital scholarship is more than just using technologies for effective

collaboration and research it is as much about embracing open values and exploiting the

potential o f technology for the benefit o f the academy and so ciety (Weller, 2011). This includes:

championing the importance of sharing and being generous (Wiley, 2010); recognising the need

to practice inclusion on a larger scale (Meiszner, 2011); developing new systems

for licensing content; and exploring alternative models for resourcing learning.

The open movement is not new, but it does appear to be gaining momentum. At the time of writing

[21/12/11] the Directory of Open Access Journals had over 7000 periodicals in its

books. Furthermore , increasing numbers of academics are committing themselves to open

practices by signing the Open Access Pledge , and the UK government is developing policy to

encourage the publication of publicly funded research in Open Access Journals (Jha, 2011).

The open revolution also faces some significant challenges as it grows in scale: issues of quality,

identity and ownership (Weller, 2011). How does networked participation impact on

the democratisation of knowledge creation and dissemination (Veletsianos, 2011). Can more open

systems of publication match the academic rigor of established journals? What are the implications

for academics of having a significantly more visible digital footprint? How do these practice fit

with current institutional financial models of intellectual property? And how do we recognise and

ce lebrate intellectual contributions in more open forums?

One innovation that is attempting to address some of the structural issues outlined here is Creative

Commons. Creative Commons provides an free, public and standardised infrastructure and set of

licenses that offer a balance between stringent copyright law and the reality of (looser) internet

publication practices. Fig. 1 desc ribed some of the components of the licences:

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/fki3http://goo.gl/8R0Xrhttp://goo.gl/gaI9vhttp://goo.gl/6h9qxhttp://goo.gl/PqsmOhttp://goo.gl/TtJ1ghttp://goo.gl/gHA7phttp://goo.gl/HWRshttp://goo.gl/gaI9v


4/48

Fig. 1Creative Commons licence codes (UCT, no date)

www.capturinglearning.com is licensed by me, peps mccrea under a Creative Commons Attribution

3.0 Unported License. This means that anyone is free to copy, adapt and distribute this research, as

long as they attribute it to me. The image below (and on the top right) is a machine readable version

of this license:

Fig. 2Creative Commons licence for www.capturinglearning.com

I am keen to explore the publication of my thesis in a similar manner, and wish to use this pilot as a

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://creativecommons.org/licenses/by/3.0/http://creativecommons.org/licenses/by/3.0/http://goo.gl/yTjEShttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/12/What-is-a-Creative-Commons-license.jpg


5/48

test case to explore the position of open publishing within our institutional systems. I am not the

first person to do this for an interesting example check out Belshaws (2011) thesis on Digital

Literacies, and for a more detailed coverage of the area see Andrew et al.s (2012) SAGE

Handbook of Digital Dissertations and Theses.

Introduction

This is the report of a pilot project I am undertaking in preparation for my EdD thesis, exploring

blended learning in higher education. It is aimed at helping me understand the ro le and influence of

digital technologies on learning in the context of my practice as a secondary mathematics teacher

educator.

To help me do this I have designed and built a public, online discussion space using freely

available webspace building software (WordPress), and am encouraging the 60 (mostly

pos tgraduate) teacher trainees on my course to digest and comment on artefacts (articles, videos,

websites etc.) that I post each week during the first 9 weeks of their course. The online space is

called MATHAGOGY (link available on header bar at top of the page).

My approach uses a modified design-based research (Van Den Akker et al, 2006) methodology

and critical realist analytical framework (Danermark et al, 1997) to help me design, build and

evaluate an online learning space andanswer the below questions. Design-based research begins

with the premise that learning experiences can andshould be continually improved, and so my

rese arch questions are not built around an existing problem, but rather a potential oppo rtunity:

1. How do learners learn in the online space I have designed and made?

2. How does the design of the online space influence their learning?

My research and thinking about the world is located in a post-positive paradigm best described as

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.capturinglearning.com/?page_id=1084http://wordpress.com/http://goo.gl/pbhu2http://goo.gl/Wkl7b


6/48

a critical realism (Bhaskar, 1975). As a realist I will be seeking to accurately describe what the

MATHAGOGY learning experience looks like, as well as trying to develop a set of context-

dependent generalisations that explain how the structure of MATHAGOGY and the agency of the

learners using it are related. Critical realists describe these observable and non-observable links

between structure and agency as causal mechanisms.

A significant feature of the research design is an absence of initial conceptual framework. One of

the main purposes of this pilot to to support me in identifying an appropriate lens and set of

questions from which I can launch my thesis. Despite this predominantly inductive approach, I am

aware that I will not be conducting value-free research. This project (like all aspects of my

practice) are heavily influenced by my socio-cultural context, history, and pedagog ical values :

what I believegood learning and teaching looks like. This project provides a rich opportunity to

explicate my tacit assumptions about pedagogy, and develop a more visible understanding of my

pedagog ic practice something that educational des igners have found to be an invaluable by-

product of their engagement with des ign for learning (Swan, 2008 and Beetham & Sharpe, 2007).

This is significant for me in my currentcontext, but will be even more influential in my future. As

an early career academic I am still in the formative stages of developing a theoretically grounded

pedagog ical approach. However, considering that it is likely that I will s till be practicing in 2050

(Osborne, 2011) a secure grounding in digital literacies ( Belshaw, 2011) looks set to be equally

important.

Context: educational turbulence

Higher education in the UK is currently striving to adapt in a turbulently changing economic, social

and technological landscape: an era that is described increasingly as the digital age (JISC, 2009).

The rise of mobile devices and web 2.0 technologies mean that digital practices are becoming a

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/o8wq5http://goo.gl/2pCNghttp://goo.gl/sn4jphttp://goo.gl/X0OJx


7/48

taken-for-granted part of our daily blended lives, whilst the demand for digital literacies across a

wide range of sectors continues to grow (Horizon, 2011). Blended in this context refers to

mixed use of digital and non-digital tools. For a further exploration of digital literacies, please

refer to Belshaw (2011).

Accelerating technological innovation is not the only major influence on the digital age. Severe

constraints on spending (as in the recent Comprehensive Spending Review, Browne Report, andWhite Paper) are adding to the challenging context for higher education. This situation is forcing

institutions to explore alternative models of education, and technology has been touted as a

solution to cut costs, enhance marketing and meet the rising expectations of students. The UK task

force recently identified blended learning as a real opportunity for UK institutions to develop

responsive, engaging and interactive provision which, if offered at scale, can deliver quality and

cost-effectiveness and meet student demands for flexible learning (HEFCE, 2011).

Student expectations also appear to be changing. Experiences are being sought that are moreconsistent with their blended lives. Learners are increasingly demanding to learn whenever and

wherever they want: in essence, to have a more blended learning experiences ( Johnson, 2011).

However, the ECAR (2010) study suggests that institutions continue to make poor use of digital

technologies (see Fig. 3 below). While course management systems are indeed prevalent, this is

only narrowly exploiting the potential of technology for learning.

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.educause.edu/studentsAndTechnologyInfographichttp://goo.gl/GbuLkhttp://goo.gl/Mgwd9http://goo.gl/Y4nyphttp://goo.gl/bJ8Oshttp://goo.gl/ifmUhttp://goo.gl/GbuLk


8/48

Fig. 3Infographic of the ECAR ( 2010) national study of students and technology

The promise of technology for education is compelling. However, the uptake of blended learning

in higher education has been slow, and technology has not had the transformative impact on

learning that many had hoped (Conole, 2011). I suggest that one of the reasons for this is the huge

challenge of enabling a digitally literate academic staff body. Developing both a robust

understanding of technology and its role in learning requires sustained engagement, and a

commitment to changing practice .

HEFCEs (200 9) revised strategy for e -learning and JISCs (2009) Effec tive practice in a digital

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/pPnTwhttp://www.educause.edu/studentsAndTechnologyInfographichttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/09/Students-and-Technology-Infographic-EDUCAUSE-e1324713780823.png


9/48

age made a start on that agenda, challenging institutions to move beyond capital investment in

technology, towards the engagement of staff in the use of technologies and related

pedagog ies (UCISA, 2010). More recently, our current government has made hints that it will be

prioritising that agenda further. In a recent speech [01/12/11], Michael Gove clearly s ignalled the

importance of fostering digitally literacies within the profession (DfE, 2011) and promised policy

for the new year.

One of the important responsibilities of research in this area is to interrogate the technological

determinacy of solutions being advocated within educational policy, to ensure that we are not

building practice upon potentially naive conceptions of change. The disco urse surrounding

technology, like that of neuroscience, is highly compelling, and the academy has a particular

responsibility to play an initially sceptical role (Pickering & Howard-Jones, 2007) .

In summary, this context serves as a strong rationale for a project such as this. In addition to

contributing to a developed understanding of design for blended learning in higher education, itwill provide my students with an enhanced experience and perhaps most importantly : develop my

capacity as a digitally literate professional.

Context: online learning spaces

For this project I will be focussing on a particular aspect of blended learning: the design and

evaluation of an online learning space (MATHAGOGY). Virtual spaces differ from physical

spaces in several ways, most notably for this project: in theirasynchrony. This refers to a certain

degree of freedom from the constraints of time and affording learners access and interaction

whenever and from wherever they wish.

There are a wide range of online spaces that are currently used for learning. To help map this

territor I will describe these s aces accordin to the model o learnin the redominantl

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/WkwtShttp://goo.gl/l5aKN


10/48

espouse (acquisition, participation, or a mix), and the extent to which they are managed by

institutions. Fig. 4 below off ers a comprehensive visual representation of this typology.

Fig. 4Map of online learning spaces: click for acronym definitions (Pea-Lpez, 2010)

The first major category of space, commonly known as a VLE (Virtual Learning Environment), is

usuallyfully managed by institutions and predominately based on an acquisition model of learning,

offering students acce ss to c ourse materials (eg. powe rpoint presentations, readers, hyperlinks)

although these is evidence that this focus is beginning to broaden (UCISA, 2010).

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/l5aKNhttp://goo.gl/qexEqhttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/09/ple-shere-e1324904035405.jpg


11/48

One major criticisms directed at VLEs is that they tend to sustain existing models and perceptions

of learning rather than engaging with the possibilities for learning offered by the affordances of

new technologies (Laurillard, 2007). The widespread adoption of VLEs also appears to have

limited the discourse surrounding blended learning. In 2009, Ofsted reported that the best learning

environments gave learners the opportunity to reinforce aspects of their work as well as the

chance to catch up on missed material. This narrow pedagogical view of the effective use of

VLEs in higher education was characterised by a stark absence of referencesto participatory tools and models of learning.

The second kind of space, a PLE (Personal Learning Environment), is used more frequently by

learners in their everyday lives but is rarely referred to as a learning space. It is not formally

managed by institutions and draws on mixed modes of learning. A PLE is typically the set of tools

and spaces a learner uses to support their learning. The combination of tools used is based on

perso nal needs and prefe rences, and may include social networking sites , discussion forums,

bookmarking services , video curation, wikis, blogs, and so on. The possibilities are vast see Fig. 5 for an example PLE too lset.

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/jPH1K


12/48

Fig. 5Example PLE toolset(Villaret al, 2010)

One of the major problems associated with PLEs in education is that their effectiveness is

dependent on the autonomy and digital literacy of the learner, and that as a consequence: they

favour the academically prepared (Wiley, 2011). Another issue is that of tool fatigue humans can

only manage so many tools at one time. How can we decide what to choose and invest in amongst

the growing variety of alternatives?

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/FVO04http://goo.gl/53mLXhttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/12/PLE-e1324987999473.jpg


13/48

This leads us to the third kind of space, not yet clearly defined, but emerging in higher education.

This is a hybrid space a fusion between the VLE and PLE (Pea-Lpez, 2010) ,supported

rather than managed by institutions, and based on mixed models of learning as required. This may

be viewed as a way of scaffo lding learners in the development of their own PLE, and as a

consequence: their autonomy and self-directness as learners.

The big challenge raised by this third type of space is the immense complexity of its

implementation. Consider Facebook. I have been told that many of my students use Facebook to

complement their formal learning experiences. What would be the impact of an institutionalisation

of that space: on identity, on ownership, on agency, on autonomy? What would be the ethical

implications for higher education institutions of the corporate interface: of practicing in the public

domain; of the status of our data? And if instituions tried to set up an alternative social

networking service, within the walled garden of the VLE, would people ever really engage with

it?

It is precisely the complex nature of this third space that makes it the most appealing for me. It

raises emerging issues worthy of attention. Academics have access to a growing set of tools for

enhancing learning, and a responsibility to challenge dominant acquisition models of learning

(Laurillard, 2007). Furthermore, the design of this third space is accessible to individual academic

staff. Web 2.o is providing an abundance of free tools for every occasion. I have more choice than

ever in personalising learning experiences for my students. I choose the tools, set up the spaces,

facilitate the learning. Digital design becomes just another wing in my pedagogical repertoire, the

broadness of which impacts directly on the diversity of learning expe riences available to my

learners.

Approach: critical realist hacking

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/qexEq


14/48

As part of my EdD journey I have been working hard to understand both my assumptions about the

world (my ontology, epistemology and axiology), and how I have arrived at them. I refer to these

areas of my knowledge as myphilosophical position because in my experience they are fairly

cognitively structural. Like religious beliefs they are ideas upon which my understanding of the

world rests, which colour how I see the world. They are based on intuition as much as

reasoning, although I realise it is not quite this straightforward as I digest a body of literature that

reso nantes with my paradigmatic thinking, that process c hanges how I think, although to what extentthese c hanges are structural is beyond my grasp at the moment.

One of the things I have come to realise is that my philosophical position is heavily influenced by

the professional identities I have performed in the past, namely those ofengineer and teacher.

Bec ause I have spent time living and working within both natural and social sc ience paradigms, I

have found it a real struggle to reconcile the tensions between their worldviews, and identify a

clear philosophical standpoint.

Two major breakthroughs have really helped me in this pursuit: the discovery of critical realism

and hacker meta-theories. As I result I suggest that this project (and my research in general) is

bes t described as being underpinned by a realist ontolog y, a constructivist epistemology and a

hacker axiology. So what does this mean in practice? The post-positive ontology of critical

realism is based on the following three assumptions:

1. There is a single reality, which extends beyond the self, but which is only approximately and

no t wholly knowable (Krauss, 2005). Critical realism distinguishes clearly between ourknowledge about the world, and the world that is the object of that knowledge. This is in

contrast to the constructivist assumption that a different reality exists within each persons

conscious.

2. Reality is an open system with emergent properties. This means that it is not reducible to its

constituent parts. This differs form positivist positions which view the world as a deterministic

machine, or constructivist positions which espouse an ungrounded shifting sea of cultural

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01


15/48

meaning (Burgoyne, 2010).

3. The emergent nature o f reality means that that general laws which explain the s oc ial world

are unlikely to exist. Instead, critical realists aim to uncovercausal mechanisms that connect

structure and agency within our world. Some of these mechanisms are only available to the

researcher through the events that they cause at an observable level (Crawford & Wright, no

date). For me, structure refers to the enduring forms and outcomes of social practice, and

agency refers to a human capacity to choose and actupon those choices. Structure can enableor constrain agency (Scott, 2005).

So how do we come to know about the causal mechanisms that explain our reality? Critical realists,

like constructivists, believe that knowledge is constructed within our neural systems. As such,

people of fer diffe rentperspectives on reality, and so a large aspect of social inquiry concerns

itself with investigating qualitatively the meanings that people make of the world (Carter & New,

2004).

Although critical realists prioritise peoples descriptions of their experiences, they acknowledge

that these perceptions are historical, value-laden, situated (Carter & New, 2004), and cognitively

biased (Wikipedia, 2011). The plastic nature of our knowledge makes the an awareness of the

positionality of the researcher crucial (Krauss, 2005). It also means that we can never achieve

complete accuracy in our description and explanation of reality. Furthermore, because our

descriptions of the world have the capacity to change the world (and in doing so make themselves

redundant) we will always be at least one step behind the evolving, emergent and looping nature of

reality (Scott, 2005). Despite the risk posed to prediction by the fallibility of thispos ition, practical adequacy canstill be achieved when the resultant knowledge is judged useful

and effective in a contemporary context (Sayer, 1992 in Crawford & Wright, no date).

The values that I aim to espouse resonate with those practiced by the hacker community. When I

talk about hackers I am not referring to the narrowly defined stereotype promoted by the media: the

closet geek who endangers national security by cracking national security networks. This is a

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/S8b7X


16/48

cracker. A hacker is someone who explores systemic knowledge structures and learns about

them by making. They teach themselves and each another because they are at the bleeding edge

of knowledge about that system (Suiter, 2010). Hackers value playing, making, innovating and

sharing as much as knowing. Their art is bricolage re-imagined for a time when computers have

replaced magic (Suiter, 2010) and their products are tangible, functional examples of knowledge in

practice (Eraut, 1994).

Approach: design-based research

One of the challenges of this project (and my EdD journey so far) has been to find a methodology

that aligns with my philosophical position. In particular, I needed something that embraces the

hacker epistemology of making as knowing, and that can lend itself to the critical realist

investigation of agency and structure.

Design-based research is my best fit find so far. It is an approach that arose from the ashes of

failed design experiments in education during the last century (Van den Akker, 2006). It was

recognised that such a positivist approach was unlikely to yield much of value, but that

investigating design for learning in natural settings still had a lot of potential for the improvement

of theory and practice.

The approach gained momentum during 2003-2006, witnessing several dedicated journal special

issues (Educational P sychologist and Educational Researcher). Things then went a bit quiet in the

last 5 years, but is a new book being released in March (McKenney & Reeves, 2012). I am

looking forward to this in the hope that it will clarify the direction of design-based research.

Currently, proponents espouse a diverse range of positions within post-positivism, and the

approach lacks a clear sense of paradigmatic location. I suggest that it espouses a realist ontology

as it claims and generates transferrable design principles, although there is a lack of consensus

as to the context-dependent nature of these.

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/i6mX0http://goo.gl/zZT7Ihttp://goo.gl/n1401http://goo.gl/n1401


17/48

The epistemological assumptions underpinning current design-based research are not so

clear. However, for me (coming from a critical realist perspective) the approach is less about

isolating variables and generating context-free generalisations, and more of a kind

of interventionist ethnography (Dede, 2004). It is experimental, but not an experiment (Kelly,

2006).

I see the aim of design-based research as to improve learner experiences by deve loping theory,

artefacts and practices, through looking at the process of learning and the means that enable and

constrain it in naturalistic settings. It may be further characterised as:

Improvement oriented utilising pragmatic approaches aimed at making improvements rather

than being problem-oriented

Iterative following a repeating cycle of design, evaluation and theorisation

Process oriented seeking to understand what happens during the activity in question avoiding a black box model of input-output analysis

Learning foc ussed looking at how learning happens in the context of the intervention, and how

the intervention influences learning

Utility oriented evaluates the design according to its value for users in real contexts - ie.

impact on practice

Theory oriented design should be grounded in theory and evaluations build on, and adds new

insights to theory (Van den Akker e t al, 2006)

Pedagogically situated has a sharp focus on the underpinning pedagogies of design, andfosters the explication of learning and teaching value positions

Design situated recognises and values the (currently underrepresented) centrality of design

practices in education

In an over-simplification of the process: design-based researchers collaborate with educators to

design a theoretically informed artefact (resource) or process (activity), trial it in a natural setting,

PDFmyURL.com


18/48

evaluate it, theorise the f indings and then start all over again from a more informed position.

The principle reason this may be a wise choice of methodology in our current educational climate

is down to the potential impact factor of design-based research on policy and practice. I do not

have enough authority to evaluate the influence of educational research on policy, but the

literature sugges ts that it has been limited. I suspect this is in some part due to the different goals

of researchers and policy makers, but Dede (2004) suggest that it is also because educationalresearch is sometimes too theoretically focussed, and conducted with policy and practice only

loosely in mind.

Design-based research has been proffered as a vehicle to foster wider improvement, not just as

an end in itself. The visibility of its outcomes (well-designed artefacts in particular) together with

a strong sense of empirical inquiry could well make an appealing case for political

attention. However, at least equally important is the direct influence of design-based research on

practice . It creates a situation where educational des ign becomes infused with, and stee red bytheory. When contrasted with examples of poorly-grounded educational designs orretrospective

research, it seems an obviously sensible approach.

From a realist perspec tive, design-based res earch can generate plausible causal accounts because

of its focus on learning processes (DBcollective, 2003), and its location in natural settings. This

means that resulting designs are a good fit for the demands of practice, and theorisation has the

potential to provide more robust explanations (Sandoval & Bell, 2004). Fo r this project I will be

using critical realism and a hacker ethos to steer my design-based research methodology. Inparticular, I will be looking at the relationship betwe en the structure of the des ign and agency of

the learners. The hacker ethos suggests that intuition and imagination hold equal status to theory in

design, and so I will be using my own craft knowledge as much as theory to build the online

space.

Whilst one of the main objectives of design-based research is the development of theory, Amiel

PDFmyURL.com


19/48

& Reeves (2008) argue that this might only occur after long-term engagement with multiple

iterations. As a result I will need to be sensitive to the credibility of any claims I choose to make.

The investment (both in human capital and over time) required to make design-based inquiry

worthwhile could be seen as one of its greatest weaknesses. However, I suggest that the cost of

not infusing design with theory may be greater still. To balance this equation Edelson (2006)

suggests that any intervention must be sufficiently innovative to be worth the investment risk. This

immediately creates an ethical tension: is it permissible to experiment with learners experiences inorder to improve them? The whole of the education system is predicated on incremental

improvement through innovation and evaluation. If there is an issue here, it is endemic and beyond

the immediate scope of this report.

The final challenge faced by design-based research is the demand put on researchers when they

(as in my case) are also acting as a designer. Success requires the bridging of multiple domains

of expertise: research, design, pedagogy, and subject specialisms where appropriate. This mixed

positionality becomes increasingly complex as the researcher-designer endeavours both to makethe design work well at the same time as trying to evaluate it (Sandoval & Bell, 2004). To mitigate

this risk, I am aiming to be reflexive throughout the project.

Approach: integrated methods

This study is guided by the following two questions:

1. How do learners learn in the online space I have designed?


To help me do this I will be employing an integrated mixed methods framework (Castro et al,

2010). Mixed methods is a common approach for those conducting research using critical realism,

and strives to generate representative descriptions and explanations of reality, balancing demands

PDFmyURL.com


20/48

for accuracy, with the pursuit of generalizability. Integrated mixed methods is a particular

approach to using mixed methods that supports asimultaneous rather than sequential analysis of the

data generated from the different methods. This approach allows for a concurrent, integrated and

unified process of data analysis which places an equal status on both qualitative and quantitative

methods (Castro et al, 2010). However, because the epistemology underpinning critical realism is

constructivist, the methods selected will lean more towards the qualitative end of the spectrum of

choice.

Mixed methods have been gaining popularity over the last few years (Wheeldon, 2010). I am

aware that choosing to use multiple approaches to developing knowledge means that, for a finitely

resourced project such as this, I will be spreading my time and attention more thinly than

otherwise, and will likely use several methods more superficially than I would if I was using only

one. However, the realist demand for generalisation means that this is a sacrifice that has to be

made. Mixed methods will allow me to explore the consistency of findings obtained though

different instruments, and to clarify and build upon the results of one method with another(Wheeldon, 2010).

The group I have designed the online space for is made up of a mix of postgraduate and

undergraduate students in their final year of initial teacher education. There are about 60 learners of

a fairly demographically diverse nature, drawn together within the aim of becoming secondary

school teachers of mathematics. At the start of this year the group attend university based sessions

for the first nine weeks before they go into school on a full-time placement. It is during these first

couple of months that they will be encouraged to engage with the online learning space I havedesigned (MATHAGOGY), at the end of which time I will generate data for analysis, evaluation,

and possible theorisation.

For this project I have chosen to use a series of semi-structured focus groups together with a

qualitative survey. I have chosen to use focus groups rather than individual interviews because of

the inductive nature of this inquiry I anticipate that focus groups will encourage participants to

PDFmyURL.com


21/48

respond to and build upon each others contributions, in a similar way that elicitation techniques

are used in individual interviews (Cohen et al, 2007). I hope to get about six participants (10% of

the population) involved in the focus groups a sample size that will lead to a reasonable

likelihood of reaching saturation in my data analysis (Castro et al, 2010).

I recognise that there are some important disadvantages to using focus group methods. For this

project in particular, it means I will miss out o n the oppo rtunity to talk to participants alone and inconfidence. In a peer group situation they may be less likely to share information that could

damage their own, or other participants reputations or feelings. However, on balance I am

convinced that for the purposes of this study, the benefits of grouping participants outweigh the

disadvantages, partly because they will have an opportunity to contribute in confidence as

individuals in the project survey.

I have chosen to employ a compact, semi-structured, qualitative survey (Cohen et al, 2007) to

complement the data generated from the focus groups. The survey will be sent electronically to allparticipants (60 or so) soon after the focus groups are conducted. This means that anyone who

wants to say something more has the opportunity to do so without delay. The survey will pose two

fairly open questions and offer a large amount of space to respond in text. The phraseology is

designed to be open and familiar:

1. Have you learnt anything from MATHAGOGY?

2. How could we improve MATHAGOGY for the future?

One of the major limitations of this study is its lack of consideration of natural data during the

analysis stages. All participant comments are openly accessible and recorded on the learning space

it would be technically straightforward to retrieve this data, creating an additional perspective on

reality. However, I have chosen not to do this for two reasons. Firstly, this is only a pilot project

and has finite scope and resource. Secondly, because at the beginning of the project, students

were not informed that their comments would be analysed. The ethical issues surrounding the use

PDFmyURL.com


22/48

of open data are emerging and complex, but the analysis of natural data is something I do wish to

explore in further detail in preparation for the next stage of my EdD.

All participants were made aware from the beginning that the design and evaluation of

MATHAGOGY was the focus of a research project for my (their lead tutors) EdD. In addition,

those who attended the focus groups were made aware that I was recording the discussion, and the

primary data was dele ted so on after it has been processed.

What actually happened

Everyone in the group was invited to join me for focus group sessions on the ninth friday

afternoon following lectures. Nine students chose to attend (a 15% self-selecting sample) and I

saw them in groups of threes. The focus group was fairly loosely structured: I posed an opening

question: tell me about your MATHAGOGY experience, let them speak, and probed further

where I felt there was something interesting behind what they were saying, or if I neededclarification. The focus groups came to a natural conclusion each time between 30-40 minutes. I

made memos of my thinking during the sessions, and audio-recorded the discussions on my

smartphone fo r subsequent analysis.

Clearly these students are the ones who are more likely to have engaged with the online space and

so are not an unbiased sample of the population. However, for the focus groups this was not

significantly problematic ,as I was striving to generate data about those learning experiences

that did occur rather than how frequent they were. The survey was designed to explore therepresentativeness of the student experience. It had an approximately 75% response rate (n = 46)

which was enough to give me an insight into how the experiences of the focus group participants

extrapolated across the population.

A roach: critical realist analysisPDFmyURL.com


23/48

In order to maintain a coherent epistemological approach I will be using a critical realist analytic

framework to guide the analysis of my data. The end goal of this is to identify causal

mechanisms which explain how structure and agency in the context of this project are related. To

do this I will be using a modified version of Danermarket al.s (1997) six stage model of

explanatory realist research.

Stage 1: Description In the first phase of analysis I will be aiming to generate a rich description

of both the product (an analysis of the webspace design) and process (how people perceived they

learned through the webspace), using the variety of data available including: students responses

during the focus groups and survey; my field notes; and analytics available via the learning space

software. It is important to note that I am not trying to describe how people have actually learned

via MATHAGOGY at this stage (as a critical realist) I cannot claim anything beyo nd an

interpretation of theirperceptions.

In my analysis of the learning process I will use words and phrases used by the participants as

much as possible, to create authentic and rich descriptions in preparation for the next phase of

analysis.

Stage 2: Decomposition During this phase I will aim to dissolve the complex and composite into

a variety of dimensions. This is not the same as practicing reductionism emergent processes

are greater than the sum of their parts, and so the concept of dimensions rather than components is

more appropriate here.

I am aware that I will have a significant influence over how the data will be divided up, and where

the boundary of interest lies (Burgoyne, 2010). This is okay it is not possible to look the data

from every possible angle, particularly in an open social system such as education, where there

are limited clear boundaries. This is where my research questions will play an important role, in

steering the analytical process:

PDFmyURL.com


24/48



Stages 3&4: Abduction & Retroduction These two stages will be conducted concurrently as

the value here lies as much in theirinterplay, as discretely considered processes. Abduction refers

to the interpretation and redescription of the dimensions highlighted in the previous phase. It is theconstruction following the deconstruction phase, but now with the aim of building an abstracted yet

conceptually coherent model of reality.

As this phase progresses I will need to explore and exploit relevant conceptual and theoretical

frameworks in the development of a robust explanatory model. However, due to the inductive

nature of the project, it is not appropriate to pre-identify any of these, as it might risk

(further) limiting the scope of my imagination and reasoning significant aspects of researcher

thinking during the process (Hume in Stevens, 2009).

Retroduction refers to the process of highlighting various dimensions and asking myself questions

such as:

How are they related/unrelated?

What properties underpin them?

What else is going on here?

What causal mechanisms are at play?

Together, abduction and retroduction can result in the uncovering of underlying mechanisms (how

learning is happening/not happening in and because of MATHAGOGY) and a potential theorisation

of the process (Livock, no date). One of the main challenges here is to distinguish between cause

and correlation. Whilst we can observe correlation we cannot observe cause we infer cause.

Another is to sensitise myself to how the data and my values are intertwined (Carlsson, no date).

PDFmyURL.com


25/48

Reflexivity plays an important role in addressing my positionality I will be keeping one eye on

the analysis and one eye o n myself throughout.

A keen awareness of the fragile nature of any knowledge claim lies at the heart of the critical

realist paradigm. It is important to recognise that this is my first real attempt at trying to analyse data

in any depth. As a result, I am sceptical as to the credibility of any theorisation that I may claim to

achieve. Nonetheless, I am going to have a good crack at theory building as I hope to learn (atleast) as much from the engaging with theprocess as I do from the project outcomes.

Stage 5&6: Redescription & Recontextualisation In these final stages, the scene is set to look

at any theory that I develop through the eyes of existing theoretical frameworks, as well as

interrogating their explanatory power in the wider context of online learning spaces in higher

education.

Analysis: describing the design

This section of the report details the design of the online learning space (MATHAGOGY), and

briefly explores the mode l(s) of learning that it espouses, or rather thatI have espoused as the

architect. It is not a section that appears in a typical educational research report, but is an essential

requirement of the design-based methodology employed in this project.

The goal of critical realist research is to uncover the underlying causal mechanisms that link

structure and agency. Within this project, structure refers to the design of the learning space, my

role as a tutor and the culture that emerges within the space. Without an explication of these first

two elements there would be no structure to relate to in the structure-agency analysis. The third

element is a little different because it is emergent rather than designed, and so falls within the

analysis of participant experience. There is of course a much greater range of structures that

enable or constrain the agency of participants, but I have drawn a boundary around the

PDFmyURL.com


26/48

aforementioned aspects to provide the inquiry with a sufficiently sharp focus.

An obvious critique of this research design is that it deviates from suggested design-based

research methodolog y by notgrounding the learning space design explicitly in theory. As this was

the my first research iteration I decided to generate a design grounded foremostly in my own tacit

expertise (Eraut, 1994), and to use this as an opportunity to explicate my pedagogy (Swan, 2008).

This may limit the potential of theorisation during analysis, but the uncovering of my values holds agreater long term return-on-investment value in a pilot project such as this. As a result,

successive iterations will be grounded in both theory, and a clear understanding of my own

assumptions and practice as a designer.

There are four major components of the MATHAGOGY architecture: environment design; content

design; interaction design; and role of the tutor:

Environment design The space itself is a fairly formal, uncluttered, and focussed on a singlecore activity: the presentation of relevant artefacts with the opportunity to comment on them. The

use of a logo has been employed to give the space a more aesthetic appeal, arguably of equal

importance to functional effectiveness (Burkhardt, 2006).

Content design Content was curated based on my subject expertise. Artefacts (articles, videos,

websites etc.) were selected for both their mediaformat (a variety was deemed desirable) and

potential to challenge participant pe rceptions of what it means to be a maths teacher in the digital

age. The abundance of information available on the internet in our times means that finding andidentifying highly relevant artefacts requires a certain degree of digital literacy expert curation

is a digitally literate pedagogy underpinning this aspect of the design (Weller, 2011). Content was

deliberately presented neutrally, forcing participants to position themselves when commenting on

the artefac ts.

Interaction design One of the major design features of MATHAGOGY was the provision of a

PDFmyURL.com


27/48

space to comment on artefacts. This demonstrates the value I place on participation models in

addition to acquisition models of learning, where learning occurs through intellectual activity rather

than just via the absorption of information (Mayes & de Freitas, 2007). Commenting on and

collaboratively discussing are processes rooted in Vygotskyan (1978) and Bahktinian (1986)

notions of social constructivism and conceptual development.

Role of the tutor (me) My role in MATHAGOGY is characterised as: designer of the learningspace; curator of relevant content; and lurker in the interaction space. The absence of my voice

in the comments is a deliberate decision, not based on time constraints, but rather on the premise:

as soon as you have an expert in the room, people defer to them. Instead, participants were left to

figure out their positions together, learning from in each other, congregated by a relevant social

artefact (Littlejohn, 2011). This resonates with learning processes described by communities of

practice theory (Wenger, 1998).

Overall, the design is based on a mixture of acquisition and participation models of learning,valuing both the assimilation of a body of knowledge as well as the development of traditional

and digital literacies (Beetham & Sharpe, 2007), where participants can practice critical reflection

and perform their values (Adhikari, 2011) in an authentic professional setting (Eraut, 1994).

Analysis: describing learning

This section offers an overview of how I handled the data (generated by the focus groups, survey

and field notes), with an insight into the description phase of the analysis.

The first part of the process involved semi-transcribing the focus group recordings. I did this by

listening several times to the audio recordings, and each time capturing what I felt were significant

soundbites on Idea Sketch (a concept mapping tablet app). These were then supplemented by

soundbites harvested from the survey, and from my field notes, to produce the soundbite

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/vhd9G


28/48

overview illustrated in Fig.6 below:

Fig.6Screenshot of soundbite overview (click for larger image)

The reason I chose to only semi-transcribe the captured data is because I wanted to prioritise

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/photo-2-e1327047771392.png


29/48

the connection of concepts from early on in the process. Using concept mapping rather than

verbatim transcription meant I could amend and move around the soundbites with each listen. At the

end of this process I had a loosely in vivo themed overview of the content which I was able to

export to excel and tag with (what I felt were) appropriate dimensions (see Fig.7).

Fig.7Screenshot of the attributing dimensions process (click for full dataset)

Following this I was able to filter the dimensions and get a sense of recurring and/orsignificant

themes. A crude quantitative analysis of the attributed dimensions is illustrated in Fig.8 below

(Wordle uses word size to represent occurrence frequency).

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/Screen-shot-2012-01-20-at-08.42.38.png


30/48

Fig.8A Wordle illustrating the relative frequency of attributed dimensions

This crude quantitative analysis was used in conjunction with the earlier, more qualitative analysis

(where I filtered each dimension in excel, and then was able to further connect the soundbites

which were attributed the same dimension) to create a sense-making artefact (Siemens, 2005)

which I could then take my analytical knife to. This process of making as understanding is an

important part of the hacker epistemology which underpins and runs through this project. The

resulting artefact is presented below:

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/M8qRhttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/Screen-shot-2012-01-22-at-10.37.58-e1327655557735.png


31/48

Your tutor gives you a link

He says this is our forum and tells you to get involved

You want to impress him

So you have a look

It is a simple website with some writing

And videos and articles and links

All of its interesting

And it doesnt look like too much

So you read the first post

Hes asked you to comment

You go to write something

But youre not sure what you think

It takes you a while

You figure out where you stand

Youre not quite comfortable

But you give it a shot

Now youre writings out there

You feel a bit exposed

Everyone can read what youve written

What will they think?

Next week theres another post

You read other peoples comments

They change how you see things

PDFmyURL.com


32/48

You reluctantly continue

The ideas each week are great

Where does he get them?

Youd prefer just to read them

Commenting requires so much more thought

You start to feel more confident

Its like practicing something

MATHAGOGY feels like a safe space

But is it really?

Your workload is mounting

Youve no time to read or comment

But at least itll be there

If you want to look back

At this point I felt that I had described the perceived learning experiences of the participants in

enough range and detail to be able to begin deconstruction. The one thing I have felt uneasy with is

the centrality of my intuition and imagination during this process. My past identity as an engineer

screams use your reason man at me, but I take comfort in deferring to Hume (in Stevens, 2007)

who suggests that imagination plays a role of equal importance to reasoning, when conducting

social research.

Analysis: explaining learning

In this section I present an overview of the the abduction-retroduction process and its outcomes: a

set of causal mechanisms, framed as design principle hypotheses. The emphasis on design is an

PDFmyURL.com


33/48

important part of the design-based research methodology employed by this project the

hypothesis of contextually-sensitive transferrable principles allows for the theory emerging from

this analysis to be implemented, evaluated and refined in subsequent iterations (Van den Akker,

2006).

As discussedpreviously, the abduction and retroduction processes are conducted simultaneously,

in an attempt to identify connections between structural and agentic aspects of the perceivedlearning experience, with the aim of distilling a conceptual framework from which causal

mechanisms may be inferred. A snapshot of the retroduction process is captured here. During this

phase I pos ed and tackled questions such as:

What is happening here?

How are the dimensions related/unrelated?

What properties underpin them?

What else is going on here?What causal mechanisms are at play?

During this process certain relationships between dimensions began to emerge as significant. The

majority did not. The diagram below (Fig.9) represents the main dimensions identified and how

they are connected.

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/m9lShhttp://goo.gl/6kbEY


34/48

Fig.9Snapshot of the abduction process (click for larger image)

These connections were subject to substantial interrogation as to the nature of their relationship:

incidental, correlative or causal. Those that appeared to be causal were reframed as design

principles hypotheses . I have made hypothetical rather than theoretical claims at this stage because

neither my data analysis framework nor personal analytical aptitude warrant claims of any causal

certainty.

I used two frameworks to help guide me during this latter phase of analysis. The first was the

ro ect research uestions:

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/photo-3-e1327048540813.png


35/48



And the second was a loose typological framework developed by Illeris (2007) in his

comprehensive synthesis oflearning theory literature. As a result, I organised my design principle

hypotheses by how individual or (more commonly) combinations of structural dimensions enabledor constrained certain types of learning. Below is a list of the design principle hypotheses (DPH),

organised by learning type, supplemented with examples of soundbites harvested from the focus

groups.

Assimilative This refers to learning experiences that do not disrupt current schemas. New concepts

can be addedto existing f rameworks without significant overhaul.

DPH1 The greater the abundance of subject information, the greater the value of curation as apedagog ical activity

Where are you getting these amazing things?

I was able to acces high quality content that would have taken me years to find

DPH2 The greater the perceived subject expertise of the curator, the greater the perceived

value of the content, even if that curator is just repurposingexisting content (Siemens, 2005)

You take it seriously

More educational value than other blogs

Accommodative This refers to learning experiences that require making room for, where

existing framework have to be modified so that new concepts can be integrated into personal

knowledge networks (Illeris, 2007).

PDFmyURL.com


36/48

DPH3 The more the online learning space is perceived as being safe yet public, the more

likely participants are to perfo rm their professional identity

An opportunity to try ourselves out as maths teachers

Room to explore your thinking

You know other people are going to read it

DPH4 The greater the opportunity for participant voice, the greater the potential for identity

development

I spent an hour just thinking what to write

Where do I see myself fitting in with the ideas

It represents you

Transformative This refers to learning experiences that result in a capacity change. Existing

concepts and frameworks are reframed and reinterpreted, destabilising personal knowledge

networks, and allowing for future learning to happen in a different way (Mezirow, 2009).

DPH5 The greater the opportunity to be reflexive about their learning (including identity

development), the greater the potential for sustained capacity change (as learners)

Makes you a more critical personIve changed how I comment now when I read others views it make me think

differently eg. did I read that right?

Disclaimer As I write these I am acutely aware of the challenge presented by the original

contribution to knowledge criteria of the thesis phase of the EdD. What I have generated here are

hypotheses: suggested relations that require rigorous exploration before they could become

PDFmyURL.com


37/48

claims that could be generalised, even in a context-sensitive manner. There are at least three

significant challenges which I will need to get to grips with andfigure out if I am to fulfil this

criteria in the future.

The first challenge is to understand how to unpick the the influence of the design, when compared

with the influence of the implementation (Ravenscrof t, 200 7) of online learning spaces . In the case

in this study, the design is highly visible, but the implementation is much less so. If MATHAGOGY

was implemented on a similar course in a similar context, but different tutors and learners, how

would the experience differ? This leads me into my second challenge:

The enormous complexity of the general process of learning, and as a result: the daunting

challenge of identifying causal mechanisms that influence it. This complexity is confounded by the

limitless social, cultural and political forces that contextualise and influence education (Selwyn,

2011). And finally, the challenge of unpicking learning andperceived learning. With the methods

applied in this project what position am I in to make claims about actual learning? What value do

perceptions of learning hold? And how can my expertise as a choreographer o f learning help me

to evaluate learning in a meaningful way?

Interestingly, the last two of these appear to be largely the result of a conflict between the realist

ontology and constructivist epistemology of the paradigm that I am operating within these issues

are disc ussed in more detail in the Limitations chapter of this report.

Analysis: theorising learning

This section attempts to explore my analysis and DPHs (Design Principle Hypotheses) through

the lens of transformative learning theory (Mezirow, 2009), with the ultimate goal of building on,

or repurposing some o f that body o f understanding. At this s tage it is important to note that:

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.capturinglearning.com/?p=204


38/48

Due to the pilot nature of this s tudy any knowledge claims will be tentative and hypothetical.

I have yet to fully articulate what theory looks like within a critical realist paradigm. Is a set of

design principles a sufficient theoretical contribution (as would be accepted in the world of

natural science), or in a social science context does theory have to be somethinggreater?

As discussed previously, transformative learning refers to cognitive capacity change within the

learner, or what Mezirow (2009) would describe as a modification of our frame of reference(aka. mindsets, habits of mind, meaning perspectives). This can be contrasted with informative

learning, which happens more often as a result of assimilative and accommodative processes (as

described in the previous chapter) (Kegan, 2009). Epistemologically, these alternatives may be

thought of as the difference between what we know and how we know. Mezirow (2007)

argues that the latter is significant for learners, because new ways of knowing are more likely to

generate beliefs and opinions that will prove more true or justified to guide action (p.92).

This project may be able to contribute to transformative learning theory in two aspects, both ofwhich are under-represented in the literature I have been exploring. The first is in connecting

agency and learning within Mezirows (2009) framework, and the second in clarifying the

relationship and distinction between informative learning and transformative learning, with

acquisition and participation models of learning.

1. Connecting agency and learning within transformative learning

In my data analysis, the notion of identity performance emerged as a significant process infostering perceived learning via MATHAGOGY. I have positioned it

under accommodative learning processes (DPH3&4), but with the potential to become

transformative if performed reflexively (DPH5). What is significant here is that this

reflexiveopportunity is directly linked to the capacity of the learner to re-construct and re-

perfo rm their identity (o r self): a process that resonates closely with Levines (2005) description

of agency. I suggest that this reframing ofreflexive identity re-performance as agency may be a

PDFmyURL.com


39/48

useful theoretical contribution when considered alongside the growing body of structure-agency

theory, as it has the potential to help educators make more informed decisions when designing for

transformative learning.

For example, if I want to build a online learning space to foster transformative learning I can more

readily attend to the structural factors that may enable or constrain the opportunity for identity re-

perfo rmance, with a clearer vision of how this might impact on my learners abilities to act in

similar (and possibly professional) contexts. Furthermore, I would be able to utilise theoretical

frameworks such as structuration (Giddens, 1984) or morphogensis (Archer, 1988) to help me

understand how my design can lead to transformative experiences, and how these changes in

agency might then modify how the learning space is appropriated by the learners.

2. Connecting information, transformation, acquisition and participation

When I first began to explore the relationship between these concepts, I was unsure whether

information and acquisition were interchangeable terms. However, after consideration of these

concepts in the context of my DPHs I suggest that although they are similar in the process that they

describe, the use of acquisition is more helpful in a design for learning context as it considers

the activity of the learner rather than any epistemological outcome. This is significant because it

emphasises the affordances of particular pedagog ical approaches.

The relationship between transformation and participation is similar (although arguably much more

complex). Looking back at my DPHs in the previous chapter I can see an emerging correlation

between the degree of participation and potential for transformative learning. Again, this

distinction stresses the impact a particular pedagogical approach may afford for learning

outcomes. I suggest rather than seeing these relationships as discrete dichotomies, they are more

usefully conceptualised as being on competing ends of a spectrum, as illustrated in Fig.10 below.

This is clearly manifest when considering, for example: the role of identity performance in this

study.

PDFmyURL.com


40/48

Fig.10The information, transformation, acquisition and participation spectrum

Where does my learning during this inquiry sit on the above s pectrum?

This is a relevant question to address if I am to enact the participatory model of learning that I

espouse. In short, I would say that I have encountered a wide range of learning experiences, with

their relevant epistemological consequences. What might be more interesting to explore is how my

capacity for agency has developed, and to what extent I may be re-appropriating the facilitative

structures that have catalysed these changes. However, despite being

an important and interesting question, this lies beyond the scope of this study, and shall have to be

left for another time.

Recontextualisation

In this section I will reflect on my analyses, and consider how they fit with ideas discussed in

the earliercontextchapters. I will attempt to address the following question: how do my analyses

help us to understand the role HLEs (Hybrid Learning Environments) such as MATHAGOGY might

PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/02/Screen-shot-2012-02-17-at-14.19.221-e1329488385185.png


41/48

play in the unfolding of our turbulent educational landscape over the next fe w years?

To begin, it is worth clarifying whether we would wish for HLEs to play any role in higher

education at all. Is the impact of MATHAGOGY as hypothesised in the DPHs (Design Principle

Hypotheses) consistent with the values of the sector? Despite the lack of clear consensus

concerning the purpose of higher education (Schwartz, 2003) I suggest that the facilitation of

learning is one of its less disputed activities.

Based on the DPHs, MATHAGOGY appears to provide opportunities for a wide range of

higher learning activities (Garrison & Kanuka, 2004), including (based on an amended design)

transformative learning which I suggest offers a rich value proposition for students and the wider

stakeholders of higher education (employers, society etc.). Despite their promise, the literature

suggests that HLEs are more likely to enhance education by being integrated with traditional face-

t0-face experiences rather than replacingthem (Haythornwaite & Andrews, 2011).

In addition to enhancing the provision of learning experiences, I propose that HLEs like

MATHAGOGY have the potential to impact on higher education in two other significant yet

argulably less visible ways. Firstly, by enhancing learner digital autonomy. As students practice

learning in a hybid safe yet pubic space over a period of time (see DPH3), they become

increasingly exposed to the potential of PLEs (Personal Learning Environments) as tools that

can facilitate greater self-direction in their learning (Kop, 2011). In effect, well designed HLEs

can sc affo ld the learner in understanding both the how and the why of their own life-long learning.

Secondly, in their ability to disrupt pedagogy and practice, HLEs have the potential to transform

some aspects of higher education (Anderson & Shattuck, 2012). Technology has often been cited

as a catalyst for change (Veletsianos, no date), and in this instance the vehicle is increased

pedagogical awareness (realised through the explication of design decisions) and innovation. This

is in stark contrast to the impact of VLEs (Virtual Learning Environments) on institutions, where

innovations have often sustained existing pedagogies rather than disrupting them (Laurillard, 20 07).

PDFmyURL.com


42/48

One further potential consequence of the development of HLEs is contingent on who in involved

in their design. If academic staff are part of (or lead) the design initiative for online spaces for

learning, then the institutional capacity for digital literacies is likely to increase (McLoughlin &

Lee, no date). Again, this is in contrast to institutions that operate and rely on centralised VLEs,

where the opposite can happen: academic staff can become deskilled, both in their digital literacy

and aspects of their pedagogical practice, by being offered (and sometimes encouraged) to

utilise off-the-shelf digital tools for facilitating learning.

In summary, HLEs have the potential to be transformative for both the learner, the institution, and

the sector, but only where academic staff take responsibilityforand are participant in the design

process (Ravenscro ft, 2007). Ultimately, this could lead towards new foci for pedagog ical

practice (design specialists, content curators etc.), whist continuing to recognise the role of

teacher as expert (McLoughlin & Lee, no date).

Looking back: limitations

In this section I will explore the major ontological, epistemological, methodological and

axiological limitations and inconsistencies inherent in this study, in terms of research design and/or

implementation as appropriate.

One of the major problems of this research design has arisen from either: an inherent conflictbetween the realist ontolog y and constructivist epistemology within the critical realist meta-

framework, or my interpretation, understanding and implementation of these components. This is

most apparent when considering how learning was evaluated throughout the project, and in

particular the difference between actuallearning andperceivedlearning.

My realist ontology suggests that there is one true way that each of the participants this study

PDFmyURL.com


43/48

learned via MATHAGOGY. My constructivist epistemology suggests that the best way to find out

is by analysing their representations of their experience. However, this is

problematic because participants may not be aware, or be able to explicate what or how they

learned. This is not to criticise the respondents in any way they are a highly intelligent and

articulate group of people but rather to recognize that all humans have cognitive biases. For

example, in a study of Standford MBA students, 87% of them rated their academic performance

as above the median (Zukerman & John, 2001)! This dissonance between the aims andprocessesof this study means that it could be seen as lacking in coherence, and that even the tentative and

hypothetical nature of my claims may be ove r-ambitious.

Setting this inherent conflict to one side, I believe that my general approach to researching learning

during this study has been inadequate. After working through the process, I now understand the

extent to which research exploring the hugely messy human activity oftechnology mediated

learning, should be required to consider complexity theory (Morrison, 2006). Despite the

abundance of empirical inquiry that has explored the effect of technology on learning, theevidence is overwhelmingly inconsistent (Selwyn, 2011). Notions such as chaos, emergence, and

complex adaptive systems are increasingly being harnessed to help explain social activity,

allowing us to generate more holistic, rigorous, and ecologically sound theories of learning

(Davis & Sumara, 2008).

Another aspect of this study which is a major weakness is the lack of clear position of the role of

theory in design. I do not yet have a clear understanding of what this might look like, and as a

result the outcomes of this study are somewhat confused. Burkhardt (2006) suggests we should be

asking how far current theory is an adequate basis for design. Is a set of design principles for

learning as valid as a socio-cultural theory of learning? And how and who should decide? This is

something I am keen to pick up in the next stage of my EdD: an exploration of the role of theory

in design, design as research, and how these ideas fit with notions of rese arch impact.

I believe the concept of design as research is particularly relevant in our current times.

PDFmyURL.com


44/48

Haythornthwaite & Andrews (2011) argue that the pace of change, not only of technology, but in

the ecology of education , means that a different kind of research is required: forblended learning

as well as aboutblended earning. They suggest that research which runs hand-in-hand with (rather

than behind) development, which contributes to the design of new products, networks and models

of learning will have significant impact and currency over the coming years. This idea aligns

pleasingly with my hacker axiology: valuing making as knowing, with tools as concepts: to be

understood through use (Collins et al., in Mayes and Freitas 2007).

Looking forward: possibilities

In this section I will talk about the impact and potential of this study: on the field, my local context,

and my practice. Despite the limitations discussed in the previous section, there are several

aspects of this research which may be seen as value-generating. In the main, these are more a

consequence of theprocess engaged in, rather than any resulting findings. This is not necessarilya detriment of the study emerging models of connected learning (Belshaw, 2012) suggest that

in our information abundant context (Weller, 2012) learning experiences are at least as important as

learning outcomes: something which, from my perspective, the EdD lends itself neatly to (certainly

in the first phase).

Firstly, this study has highlighted the potential of online spaces to enable learners in higher

education to practice their voice, identity, and being part of a community. It has: described and

enacted a set of design principles, which have not to my knowledge previously been identifiedand/or described; and highlighted the challenges of figuring out and theorising online learning. In

particular, it has demonstrated how hybird learning environments can offer new avenues for

pedagog ical practice whilst continuing to recognize the role of the teacher as expe rt

(McLoughlin & Lee, no date), for example as: designers for learning, curators of content, and

choreog raphers of reflexivity.

PDFmyURL.com


45/48

It has already been an interesting case for discussion amongst colleagues in my institution: helping

us to challenge dominant face- to-f ace pedag og ies, and probe the liminal boundaries of

institutional policy. The power of this study lies in the combination of a tangible artefact

(MATHAGOGY), together with an analysis of that artefact. People can see the webspace: its

form, and how it has played out (all the comments are still there). These, along with my analyses,

may allow people to begin to envision how modifications in practice might lead to an impact on

student learning in new ways. It provides a lens for understanding how theory can be translatedinto practice in complex educational settings (DBcollective, 2003).

Above everything else, this study has had an invaluable impact on my own thinking and practice. In

addition to the ideas discussed above, it has enabled me to understand and redefine the balance

and scope of my pedagogical practice. I now view the creation of online spaces for learning as

an option available to me as a designer of learning. And this is particularly significant as the locus

of influence I have over the online environment is much greater than that of the physical. I cannot

move the walls of a classroom, but online I can bend both time and space.

In realising all this I have developed greater agency as an educator. I now have the capacity to

harness particular tools for particular purposes, rather than picking a stock tool off the VLE

(Virtual Learning Environment) shelf. This subtle difference mitigates against design blindness,

making me painfully aware of some of the structural and environmental aspects of the experiences

my learners encounter, over which I have less influence. However, it also mitigates against design

deskilling, as Ipractice making online learning environment

Documents

Capturing Learning Paper