Frameworks for New Media Literacy Assessments Margaret Weigel, bambini media || OLPC Workshop || April 5, 2011 | INTRODUCTION | LEARNING CONTEXT | LEARNING

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011

| INTRODUCTION | LEARNING CONTEXT | LEARNING CONTENT | MORE CHALLENGES |

Innovation in EvaluationApril 4th and 5th, 2011Cambridge, MA



Frameworks for New Media

Literacy Assessment

Margaret Weigel, bambini [email protected]



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


About me…



http://www.bambini-media.com

http://www.pz.harvard.edu

http://www.projectnml.org



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Project New Media Literacies…


-- one of the first three projects funded by the MacArthur Foundations $50M Digital Media and Learning Initiative (2005).



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011





-- Year One: Framework for new media literaciesPrototypes of potential work



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011








Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011






“Confronting the Challenges of Participatory Culture: Media Education for the 21st Century”



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


What constitutes a participatory culture?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



-- Low barriers to participation (i.e. access)



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011




-- Strong support for creating and sharing work with others



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011





-- Informal mentorship where experience and knowledge is passed down to novices



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011






-- A sense of meaning and purpose



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011






-- A sense of meaning and purpose

-- Some degree of social connection with others (individuals, a community)



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Two fundamental challenges…



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



-- What to teach



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



-- What to teach

-- How to teach it



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



-- What to teach

-- How to teach it

-- The Transparency Problem:educating students to identity how media shapes information via presentation…



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



-- What to teach

-- How to teach it

-- The Transparency Problem:educating students to identity how media shapes information via presentation…

-- The Ethics Challenge:educating students to behave as a responsible participant and community member….



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


What skills can be cultivated?

-- Play

-- Performance

-- Simulation

-- Appropriation

-- Multitasking

-- Distributed Cognition

-- Collective Intelligence

-- Judgment

-- Transmedia Navigation

-- Networking

-- Negotiation



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Play “the capacity to experiment with one’s surroundings as a form of problem-solving”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

Students are given ‘free time’ online to explore what interests them.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students use time online to chat with friends, skim the web, or browse inappropriate content.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students use time online to chat with friends, skim the web, or browse inappropriate content.

Is there an identifiable academic payoff?Is there sustained/deeper exploration?Does it captivate the student’s attention?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Simulation “the ability to interpret and construct dynamic models of real world processes”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

Students grow a functional society from the ground up by playing “SimCity.”

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students do not bother to discover what facilitates (or even breaks) the SimCity model.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students do not bother to discover what facilitates (or even breaks) the SimCity model.

Does the student understand processes better?Can the student deliberately alter outcomes by manipulating variables?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Performance “the ability to adopt alternative identities for the purposes of improvisation and discovery”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student adopts a professional role (I.e. architect) in a sustained fashion (epistemic games).

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student either chooses a role s/he is not interested in, and/or cannot sustain the role.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student either chooses a role s/he is not interested in, and/or cannot sustain the role.

Does the student demonstrate an understanding of this role?Can the student engage with others while performing this role?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Appropriation “the ability to meaningfully sample and remix media content”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student creates a collage out of photos of the Civil War found online.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student finds some random images and slaps them together in no particular order.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student finds some random images and slaps them together in no particular order.

Has the student found meaning in the content?Is this meaning new?Is it apparent to an external audience?Can s/he explain the choices made?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Multi-tasking “the ability to scan one’s environment and shift focus onto salient details on an ad hoc basis”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student works on a ‘breaking news’ blog, updating content as it is sent along.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student gets distracted by text messages and email, and does not return to the work at hand.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student gets distracted by text messages and email, and does not return to the work at hand.

Does the student focus on appropriate material?Can s/he pick up where s/he left off?Can s/he ignore stimuli as necessary?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Distributed Cognition “the ability to interact meaningfully with tools that expand our cognition.”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

Student teams play a roleplaying game using handheld devices, which allow them to share information with other team members.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students don’t share information, can’t or won’t operate the devices, and show no interest in solving the puzzle.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students don’t share information, can’t or won’t operate the devices, and show no interest in solving the puzzle.

Can the student operate the device?Does s/he use it appropriately?Can s/he understand/synthesize information?Is s/he engaged in solving the puzzle?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Collective Intelligence “the ability to pool knowledge and compare notes with others towards a common goal.”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

Students compile a list of local government resources, publish it in a wiki, and initiate dialogues with community residents.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students don’t share information with the broader community, or find it independently, that further the common goal.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students don’t share information with the broader community, or find it independently, that further the common goal.

Can the student interact with individuals and groups outside of school?Can the student identify existing information?Is the student engaged in the problem?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Judgment “the ability to evaluate the reliability and credibility of different information sources.”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student researches and finds several different cropped versions of a single news photo online.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student researches and finds several different cropped versions of a single news photo online.

?

A student uncritically accepts the validity of images and content found online, and forwards found content along to friends.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student searches and finds several different cropped versions of a single news photo online.

?

A student uncritically accepts the validity of images and content found online, and forwards found content along to friends.

Does the student look for markers of credibility?Does s/he seek out different perspectives?Does s/he look for the source of online content, and the perspective(s) of the author(s)?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Transmedia Navigation “the ability to deal with the flow of stories and information across multiple modalities.”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student uses a variety of platforms (instant message, PowerPoint, video camera) to create and share a story.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student uses a single modality to create a story, which is not shared online.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student uses a single modality to create a story, which is not shared online.

Can the student find and create content in various forms? Can s/he conduct a search across multiple modalities? Can s/he follow a narrative across different platforms?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Networking “the ability to search for, synthesize and disseminate information in a social network.”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

Students stage a citywide protest about a pending bill in the State House via MySpace.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students are upset about a pending bill in the State House, but can’t coordinate a protest.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

Students are upset about a pending bill in the State House, but can’t coordinate a protest.

Can a student network with peers?Can s/he contact adults and professionals via online tools?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


Negotiation “the ability to travel across diverse communities, discerning and respecting multiple perspectives, and following alternative sets of norms.”

+-

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-

A student successfully participates in a national online forum with a wide range of participants and opinions.

?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student does not participate, and/or is unable to respectfully listen to different perspectives and opinions.



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011



+-


?

A student does not participate, and/or is unable to respectfully listen to different perspectives and opinions.

Can a student respect alternate opinions?Can he express himself clearly?Can she argue her case ?Can he change his mind when appropriate?



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


More challenges…



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


More challenges…

-- standardized testing



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


More challenges…


-- synthesizing assessment into existing subject domains



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


More challenges…



-- shared standards across districts/states/nations



Fram

ew

ork

s fo

r N

ew

Media

Lit

era

cy A

ssess

ments

Mar

gare

t Wei

gel,

bam

bini

med

ia ||

OLP

C W

orks

hop

|| A

pril

5, 2

011


More challenges…



-- shared standards across districts/states/nations

-- combine assessment with taxonomy of skills, i.e. Bloom’s taxonomy

Documents

Frameworks for New Media Literacy Assessments Margaret Weigel, bambini media || OLPC Workshop || April 5, 2011 | INTRODUCTION | LEARNING CONTEXT | LEARNING