Adventure Based Training and Its Adventure Based Training and Its Impact on the Team Cohesion and Impact on the Team Cohesion and

Psychological Skills Development of Psychological Skills Development of Elite Netball PlayersElite Netball Players

Where are we headed with Where are we headed with this presentation?this presentation?

A report on my doctoral thesis 2002!– Rationale– A pictorial of “The Journey To The Nationals”– Methodology: Strong Isomorphism– Findings of Mixed Mode Investigation– Implications for practice & our Industry– Questions

What Can This Presentation What Can This Presentation Offer Our Industry?Offer Our Industry?

Feedback from our clients. We must ensure we are taking them in the direction they need and want to go?

The need for strong Isomorphism in our programs. (Ensuring the program is tailored for the clients needs and mirrors their reality)

A learning tool for new facilitators

The Connection Between Sport The Connection Between Sport Psychology and AdventurePsychology and Adventure

My own journey!

The Connection Between Sport The Connection Between Sport Psychology and AdventurePsychology and Adventure

Overcoming self doubt and fears

Blocking out distractions and focussing on task

Mental skills learnt during during my outdoor adventures Mental skills learnt during during my outdoor adventures transferred to my sport and life!transferred to my sport and life!

The question I often pondered was could I take others into the outdoors and teach them

mental skills to help their sporting endeavours?

Motivation for the StudyMotivation for the StudyInitial concerns for psychology skills training

““It was boring, it put us to sleep”It was boring, it put us to sleep” ““We just sat there and listened it was just We just sat there and listened it was just

like school”like school”Sport psychology’s lack of support for my

ideas ““Why take athletes away from their Why take athletes away from their

normal training venue?”normal training venue?”Dissatisfaction with previous adventure

based training interventions from sport coaches.

1. Athletes who received an adventure-based training program intervention, would show increased team cohesion when compared to a control group.

2. The duration of the intervention will see longitudinal improvements in all four sub-scales of team cohesion, when compared to a control group.

Research Questions?Research Questions?Quantitative HypothesesQuantitative Hypotheses

Qualitative Outcome QuestionsQualitative Outcome Questions From an athletes’ or coaches’ perspective, what were the major

outcomes of the adventure-based training program; and how did they impact most upon the team in the following areas:

– What new skills or knowledge about themselves or other teammates did individuals take away with them from the adventure-based training camp?

– How did the team or individuals within the team change as a result of their adventure experience? What new skills were developed that helped the team?

– Was there any direct evidence that psychological skills learnt during the adventure training camp were directly transferable to netball training or competition?

Qualitative Process QuestionsQualitative Process Questions From an athletes’ or coaches’ perspective, what

processes during the adventure-based training weekend had the most impact on the team?

– What elements of the training intervention had the most impact on athletes? Why was this significant for these athletes?

– How did this camp differ (if at all), from previous adventure-based training camps, which the athletes had been on in the past?

– How did the outdoor bush environment impact on the program? Was it an advantage or a disadvantage traveling away from their usual training venues?

NSW Netball: Needs AnalysisNSW Netball: Needs Analysis

A bunch of individually talented athletes with the ability to win the nationals but failed the previous year.

Cliques and negative behaviour toward each other on and off the court, was impacting on the team.

No mental toughness during the pressure of competition.

Lack of mental skills to handle this pressure

In order to address these issues the intervention was structured to ensure……

STRONG ISOMORPHISM STRONG ISOMORPHISM ((Priest & Gass, 1997)Priest & Gass, 1997)

The adventure experience The adventure experience had to had to mirror mirror thethe clients needs! clients needs!

Assessed, identified & ranked client goals [coach meetings] Created a metaphoric experience that possessed a strong isomorphic

relationship to playing as team at the national championships. Provided opportunities to explore how resolution during the adventure

would correspond to life at netball. Strengthened the isomorphic framework by using client language that

mirrored their reality as netball players. Constant review of activity selection to ensure clients were motivated.

(If it does not move group towards your program goals then don’t do it). Conduct the experience with revisions. Look out for that teachable

moment; the program is not static. Debrief focussed on how behaviours learnt would help or hinder netball

training or performance.

STEPS STEPS THAT GUIDED THE INTERVENTIONTHAT GUIDED THE INTERVENTION ISOMORPHISM ISOMORPHISM ((Priest & Gass, 1997Priest & Gass, 1997))

The Intervention The Intervention

Sequencing of activities: sequencing of activities can improve team cohesion of groups! (Bisson, 1997)

– Group formation activities (Forming) Ice breakers. Categories, Have you ever (Ronke, 1984)

– Group challenge activities (Storming) Games requiring the team to function together

– Group support activities (Norming) Climbing, abseiling, giant swing

– Group achievement activities (Performing) Bush walking, team campout, group swim, caving

Journey to the NationalsJourney to the NationalsFrom Storming to PerformingFrom Storming to Performing

Friday NightFriday Night

Psychological skills

development

Goal setting

Awareness of ideal state

Self-Monitoring

Self-Regulation

AN ATHLETE’S PERSPECTIVE AN ATHLETE’S PERSPECTIVE OF THE INTERVENTIONOF THE INTERVENTION

Hi! I am the leg of one of the under 17 NSW netball players. Let me share with you part of the journey that I had to endure through the

bush, cliffs, rivers and caves of Kangaroo Valley!

It taught me a lot.

We were getting wet in the first 5 minutes, but it seemed easy. Everyone was saying it

would be a piece of cake

First up was the mother of all bush

walks that lasted over 12 hours

Things soon changed!

All us legs had to begin to work

together!

We began to have to

think as a team

I am glad we are all helping each

other through the difficult sections!

NOT!

What is wrong with this picture? Are we working like a team of netballers? Or, are we individuals who are in the same netball team that don’t care

about anyone else?

Over a break we talked about how

we could overcome the challenges the

river posed!

The ideas were similar to what

we needed to do on the netball

court

All us legs were beginning to step in time!

A Focus On Personal Mental SkillsCentre, focus on yourself, don’t think about the things out of your control.

Bush WalkBush Walk

Metaphor of a journey to nationals.

“Through the physical & psychological challenges of a netball nationals campaign a team must stick together. During this section of the walk today you must work together all the way to the top. Don’t leave the slower people behind”.

Setting up a double bind!Setting up a double bind!

They’re too slow, lets just go off ahead!

Debrief: We had two choices today. To work as a team or to carry on our path of self destruction

where we play one out and forget about our team-mates. What kind of behaviour do you

want your team to have at the national championships?

Oh no, not more scratches! This hurts. We legs

were really complaining

now.

Remember team rule #1, No Complaining!

Just stay focused on the task. Don’t waste energies on

distractions that will take us away from our goals

Once at the top it gave us an opportunity however to sit and reflect on who we

were and where we were headed

By the time we got to the top of the

mountain we were aching and tired!

“Why did we have to come all the way up here to sort this out?”

We began the walk like a team of

individuals, but we started to pull

together near the top

A group in conflict with clique behaviour damaging

performance!A question for the audience to see if you

are still awake!

Used in this session were:Mini Solo Reflection Time

Questions were given to ponder; Strengths & weaknesses of team?

Sharing circle used to share ideasFacilitated problem solving session

After hiking to the top of this mountain, how would you utilise this location &

situation to address some of the negative team issues that were distracting this team

from their goals?

The power of the wilderness to heal problems!

This session was the beginning of us

coming together as a team

Here I am, you can see the big cut on my leg!

After a big day out we still were not off the mountain, instead of complaining we

finally worked together

No cabins, no tents Only a big tarp to sleep under. We

thought we would be sleeping in cabins! Showers from a billy. “We stank so bad!” Then we had to cook & clean. By the time we got to bed it was

midnight. This was all about learning to deal with

the unexpected situations that may occur in sport and being ready to deal with them!

Upon Return To Camp Upon Return To Camp The Pressure ContinuedThe Pressure Continued

5.30 AM The Next Morning As 5.30 AM The Next Morning As Twilight Filtered Through The Twilight Filtered Through The Morning MistMorning Mist

5.30am! Well that is the best alarm clock I’ve ever had.. NOT!

I don’t work at this time

of the day

The bagpipe alarm clockThe bagpipe alarm clock sounded sounded

This is crazy

Block out the negatives. If you’re

tired at Nationals you still have to play no matter how you are

feeling.

Okay team lets swim across, we can do

this together

Meditation

Visualisation

Learning to prepare mind and body for

competition

After the swim we had to sit for an hour preparing our minds for competition!

We then headed out for a day of

caving!

Its dark like when we

play at night

I feel so nervous like before a big

gameI feel like I do when I have to

sink a goal under pressure. My heart is racing

For me, I have come to the realisation that I will not be able to get out of this hole [cave] on my own, we will

have to do it as a team

It reminds of the night before a

big game, I feel really anxious

Can anyone please tell me what the hell going caving has

to do with playing netball?

I’m not sure but I think it goes somewhere

“You are kidding me aren’t you, there is no way we can fit through

that space”

You let go and I will kill you

Don’t worry I will support you

Ahhhhhhhhhhhhhh!

Its okay we have got you!The coaching staff were in position to

catch & support each player through the laundry shute!

I can’t believe I am doing

this

So in our earlier exploration to find our way down here you all

felt it was impossible to get through the squeeze. This was

with lights on……

No way, it is impossible to get out of here with out lights on! What? We have to hand our headlamps

in????

Rather than exhausting our energies on what is not possible, what would it look like if we all concentrated our energies on how we can work as a

team to get out of here?

What you just described to me then, sounds exactly like the skills a team would need to win a national netball championship. How

about we give it a try?

The whole weekend built a

bond that was the foundation from which we won

the national championships.

After this weekend nothing in our lives seemed impossible!

We are behind and we’re in a hole. Remember how we worked together to get out of that cave. We can

do the same here. “1, 2, 3, out of this hole”.

10 Seconds

to go

My body has

frozen

My heart is racing I feel like

when I was caving

Centre, breathe, relax,

visualise

Focus on the things

I can control

The goal goes in to

tie the match

The Isomorphic Metaphor!The Isomorphic Metaphor!The connection between netball & adventureThe connection between netball & adventure

We are a goal

behind

I can hear the crowd screaming

We have to either tie or win to go into the grand

final

Controlling the stress of caving Controlling the stress of caving transferred transferred to to netballnetball

Breathe deep, exhale, feel your body relax, focus on you!

These same skills are exactly what you need to do to maintain control and

concentration during a netball game

It does not matter if we are on the bench we are still

part of the team

TeamworkTeamwork

“Never doubt that a small group of thoughtful, committed people can change the world. Indeed, it is the only thing that ever has” Margaret Mead.

NSW NETBALL: AUSTRALIAN CHAMPIONS 1999NSW NETBALL: AUSTRALIAN CHAMPIONS 1999

I hope you like the guided tour I

gave you!

Any Questions Related To Any Questions Related To What We Have Just Seen?What We Have Just Seen?

MethodologyMethodology

Recommendations & processes that guided this Recommendations & processes that guided this research projectresearch project

Recommendations From Recommendations From Existing ResearchExisting Research

Neill (1988) lack of description from past studies– More info on instruction and facilitation. [Will be available

from Amazon.com in a paper back book by the end of the year ]

Ewert (1982) what’s happening inside the black box? It is no longer a mystery! Why & how did change occur?

Martens (1987) positivists stifling sport psychology research through the removal of the person from the process of knowing.

Dale (1996) existential phenomenology and sport psychology. Emphasizing the experience of the athletes.

MethodologyMethodologyOverview of quasi-experimental mixed Overview of quasi-experimental mixed

mode research designmode research designTreatment Group n= 23Treatment Group n= 23Control Group n= 11Control Group n= 11

Data Collection

0

25

Time 1(Week prior))

Time 2 (Endof W'end)

Time 3 (6weeks Post)

Time 4(Interviews)

PostNationals

Data Collection Times

Time LineIntervention

InstrumentationInstrumentation

The Group Environment QuestionnairThe Group Environment Questionnaire Was e Was Administered At 4 Time MeasuresAdministered At 4 Time Measures

(Carron, Brawley, Widmeyer, 1985)

Measures Group Cohesion on 4 sub scales– Individual attraction to group on task– Individual attraction to group Social– Group integration on task– Group integration social

Quantitative Survey ResultsQuantitative Survey Results

(A): Analysis of variance tests were conducted across all three groups for each time-period.

(B): Effect size testing examined changes within each group for times 1-2, 2-3, 3-4. TIME

4321

Means

Sco

res

36

35

34

33

32

31

30

GROUP

Control Group

U/19 Team

U/17 Team

A

B

TIME

4321

Me

an

s S

core

s36

35

34

33

32

31

30

GROUP

Control Group

U/19 Team

U/17 Team

Repeated measures analysis showing interaction between groups and time on the ATG-T sub-scale.

Individual Attraction To Group On TaskIndividual Attraction To Group On Task ATG-T Subscale

Lines considered not to be parallel thus interaction occurred

Individual team member’s feelings about their personal involvement with the group task, productivity, goals and objectives

Effect Size Change ATG-T SubscaleEffect Size Change ATG-T Subscale

Comparison of effect size change for three groups across time for ATG-T subscale.

Effect Size Description

Below .1 No detectable change

.1 to .2 Small Change

.2 to .3 Small-Med Change

.3 to .4 Med Change (Ave OE)

.4 to .5 Med Large Change

.5 to .6 Large Change

Above .6 Particularly large change

Individual Attraction to Group SocialIndividual Attraction to Group Social ATG-S SubscaleATG-S Subscale

TIME

4321

Me

an

Sco

res

43

42

41

40

39

38

37

36

GROUP

Control Group

U/19 Team

U/17 Team

Repeated measures analysis showing interaction between groups and time on the ATG-S sub-scale.

Lines considered not to be parallel thus interaction occurred

Individual team member’s feelings about their personal involvement, acceptance and social interaction within the group.

Effect Size Change ATG-S SubscaleEffect Size Change ATG-S Subscale

Comparison of effect size change for three groups across time for ATG-S subscale.

Effect Size Description

Below .1 No detectable change

.1 to .2 Small Change

.2 to .3 Small-Med Change

.3 to .4 Med Change (Ave OE)

.4 to .5 Med Large Change

.5 to .6 Large Change

Above .6 Particularly large change

Group Integration on TaskGroup Integration on Task GI-T SubscaleGI-T Subscale

TIME

4321

Me

an

Sco

res

46

44

42

40

38

36

GROUP

Control Group

U/19 Team

U/17 Team

Repeated measures analysis showing interaction between groups and time on the GI-T sub-scale.

Lines considered not to be parallel thus interaction occurred

Individual team member’s feelings about the similarity, closeness, and bonding within the team as whole around the groups task.

Effect Size Change GI-T SubscaleEffect Size Change GI-T Subscale

Comparison of effect size change for three groups across time for GI-T subscale.

Effect Size Description

Below .1 No detectable change

.1 to .2 Small Change

.2 to .3 Small-Med Change

.3 to .4 Med Change (Ave OE)

.4 to .5 Med Large Change

.5 to .6 Large Change

Above .6 Particularly large change

Group Integration SocialGroup Integration Social GI-S SubscaleGI-S Subscale

TIME

4321

Me

an

Sco

res

28

27

26

25

24

23

22

21

GROUP

Control Group

U/19 Team

U/17 Team

Repeated measures analysis showing interaction between groups and time on the GI-S sub-scale

Lines considered to be parallel thus no interaction

Individual team member’s feelings about the similarity, closeness, and bonding within the team as a whole around the group as a social unit

Qualitative ResultsQualitative ResultsExistential PhenomenologyExistential Phenomenology What was the athlete’s story? Dale (1996) recommended What was the athlete’s story? Dale (1996) recommended

hearing from those involved in the research as to whether hearing from those involved in the research as to whether the intervention was beneficial to them as individuals or as the intervention was beneficial to them as individuals or as a team. a team. – Was the learning transferable to Netball?Was the learning transferable to Netball?– Did it help in netball competition?Did it help in netball competition?

The participants told their story!The participants told their story!

I believe the athletes story has a very strong I believe the athletes story has a very strong message for our industry and how we conduct message for our industry and how we conduct programs!programs!

Qu

alit

ativ

e O

utc

omes

Qu

alit

ativ

e O

utc

omes

Group-Cohesion

Improved on court performance

Change outside of netball

Pro

cess

Facto

rs T

hat

Pro

cess

Facto

rs T

hat

Help

ed

Ach

ieve

H

elp

ed

Ach

ieve

Ou

tcom

es

Ou

tcom

es

Unfreezing

Moving

Refreezing

Lewin’s Lewin’s Change Change TheoryTheory

4 Years Later4 Years Later

Many of the athletes that participated in the intervention are now playing in the National Netball League with the Sydney Sandpipers. The Under 19 Coach in now the coach of this team.

Talking to her last week, she was telling me how the adventure experience from this intervention is still having a powerful impact on the athletes.

Maybe an area for follow up research!?

Implications for practiceImplications for practice Ensure programming addresses the needs of the client. Include clients in the

planning phase. Facilitate the program at a level appropriate for desired outcomes.

Sequence activities to suit stages of group development (Bisson, 1997). Activities need strong isomorphism!Activities need strong isomorphism!

The correct style of facilitation and metaphor development for the clients needs can greatly enhance the experience. Greater emphasis on training facilitators in our own programs and the tertiary sector can help ensure quality programming.

A “Change Agent” that can teach, inspire, motivate, and lead a group to a new level of personal performance is critical in the adventure process with athletic teams [and all clients].

Sport coaches and sport psychologists should consider utilising experiential teaching methods in their sports.

?Questions and Comments??Questions and Comments?

I can do it!

i.boyle@tsc.nsw.edu.auFor more Information contact:

THE ENDTHE END

[Under 19 Coach]: “At one point in the cave, I was asked to spot the girls as they came out that vertical hole. At that time all my instincts were telling me to get out of the cave, I felt like I could not breathe and I was losing control, and panicking. However, I focused, and realised that to learn to breathe, centre, relax, and gain control for the better of the team was more important.”

Putting the Team Before

Self

[Under 19 player] “I felt we were pushed both physically and mentally to the edge of what I could take when caving. I just wanted to get out of that cave and end it, but you couldn’t do a thing, you were stuck. Instead of freaking out you had to stay in control to help those around you so we all could get out. You had to trust yourself to get through it.”

Developing team-work in the face of extreme stress and anxiety.

[Under 19 player] “Our talk on the mountain made me feel that people would listen to me. I could share my feelings honestly, this helped me trust everyone. Up unto this point I felt like I was not part of the team, I did not connect with many in the team and felt like no one was interested in my feelings or what I had to offer. It was good to see we finally found a way to talk about the problems in the team.”

Improved Relationships

Decreased Dysfunction

[Extract from phenomenological interview with Under 19 player] “The game was so close and we were behind with several minutes to go. During a time out we had talked about how we had to really lift if we were going to narrow the other teams lead. The rest of the team just hustled, they chased down everything, and turned this into attack feeding me the ball. We were one goal down with only about ten seconds to go, and the other team had the ball down the other end of the court attacking. Marion however intercepted a pass and threw this giant bomb right at me, I caught it with only a few seconds left on the clock. My heart was just pounding; it felt like it was trying to jump out of my chest. My muscles felt heavy, like I could not control them, and I was all light headed. I had a flash back of lying in that squeeze section of the cave talking to you, the feelings I was having were the same, however this time I immediately knew what I had to do to gain control. I briefly closed my eyes, took a breath, and told myself to relax. I felt this unbelievable sense of calm. I saw myself using good shooting technique and the ball going into the net. I opened my eyes, and took the shot; it went in. We tied the match. We knew this was enough to get us into the final the next day.”

Transfer from the Adventure Transfer from the Adventure Environment to Netball was Environment to Netball was Clearly Evident in the DataClearly Evident in the Data

Improved Social Relations = Improved Social Relations = Platform for Improved Platform for Improved PerformancePerformance

[Under 17 coach] “To get away from the netball environment allowed the team to get to know each other in another dimension as people and not just netballers.”

Under 17 player] “Waking up at sunrise to the bagpipes and swimming across the lake, made us feel like we were all doing these hard things together, no one complained, we just got in and did it for each other.”

[Under 19 player] “During the caving, the coaches were crying and scared, they went through what we did. Instead of a “them and us” feeling between the coaches and players, we all felt as one, working toward the same goal.”

[Under 17 player] “Once we got out of the caves, we hugged her (the coach). After this we could talk to her, it made her more approachable and easier to talk to. We seemed to really trust her decisions and coaching much more.”

Team Cohesion: CavingTeam Cohesion: Caving to the Court to the Court

[Under 19 Team Captain] “I did not go into the cave with the girls because of my ankle, but I was there when they came out, it was amazing to see the emotion and relief when they made it out. Whatever occurred in that cave it helped us win the nationals without a doubt. The team came up with this call “out of this hole”, for me, I thought it was a bit silly to begin with as I could not relate to what it was supposed to mean. The others told me about it, and how they thought they were going to die, but because they all stuck together, they got out of the cave. Whenever we were in trouble we would use this to our advantage, and everyone would just put the extra in to get us over the obstacle. I can still vividly feel the power and enthusiasm of the others, it made a difference to the way we played.”

Transfer Of Skills Can & Does Occur

Changes Outside NetballChanges Outside Netball

[Under 17 coach] “During caving, I wanted to turn the light on when we were trying to get out of the cave with lights out. I was so scared. I felt I had no control whatsoever over my body, my thinking, I could not get it together. I got more out of this training than just things to do with netball. I was so emotional for days after this weekend. I would just keep breaking down in tears. I am someone who has always been able to do anything I tried, needing no help from others. Outside netball, I always try to take too much on. To get out of the cave I had to let go of the control I usually have, and rely on others to help me. This has taught me to ask for help, and that asking for help is okay.”

[Under 19 player] “Not only did the weekend teach us how to be good netballers, it taught us how to be the best at life.”

The Importance Of Facilitating To The Importance Of Facilitating To Client NeedsClient Needs

[Under 19 player] “The camp last year was just like fun games. However, this years camp we were just thrust into the thick of it. There were no easy options, like when we had to climb to the top of the mountain, we then had to remain together to get down again, otherwise we would not have got out. Last year, you were not put into a situation where you had to push yourself.”

[Under 19 player] “The first camp we went on, the leaders used all the buzz words about being a team, then they had us do low or high rope activities, like all stand on a log together and we were supposed to be a team. We didn’t have to push each other. This years camp was individual and group focused and we were all pushed hard, and we had to work together to get through it.”

[Under 19 Coach] “The adventure camp last year was at a superficial level, and I felt this is where our team was as well. This year was REAL, and it allowed us to get down to the deeper issues that I knew were affecting the team, and holding back performance on the court. I knew if we could solve the problems we would begin to see the talent, the girls really had.”

Wilderness As A Healing PlaceWilderness As A Healing Place

[Under 19 player] “The outdoors provided an environment where everyone was equal. The coaches, new and old players were all the same; you could say what you wanted to anyone without feeling threatened.”

[Under 19 player] “The outdoor environment provided an atmosphere to air grievances, which we had not been able to do back in Sydney.”

[Under 17 player] “Being out in the bush was new, and provided a safe place for us to communicate.”

[Under 19 player] “Why did we have to come all the way up here to sort this out? Couldn’t this of happened in Sydney?”

ReferencesReferencesBisson, C. (1997). The effects of varying the sequence of categories of adventure activities on the development of group cohesion. Unpublished doctoral dissertation. University of Northern Colorado. Greeley.

Priest, S. & Gass, M. (1997) Effective leadership in adventure programming. Human Kinetics. Champaign. Ill. USA (Back)

Rohnke, K. (1995). Quicksilver. Kendall Hunt Publishing.  Rohnke, K. (1989). Cowstails and Cobras II. Kendall Hunt Publishing.  Rohnke, K. (1984). Silver Bullets. Project Adventure Inc. Publishers.

Back to Friday Night

GEQ Sub Scale ExplanationGEQ Sub Scale Explanation Individual Attraction to Group – Task: [ATG-T] Individual

team member’s feelings about their personal involvement with the group task, productivity, goals and objectives. Eg: “I do not like the style of play of this team.”

Individual Attraction to Group – Social: [ATG-S] Individual team member’s feelings about their personal involvement, acceptance and social interaction within the group. EG: “some of my best friends are on this team.”

Group Integration – TaskGroup Integration – Task: [GI-T] Individual team member’s feelings about the similarity, closeness, and bonding within the team as whole around the groups task. EG: “Our team is united in trying to reach its goals for performance.”

Group Integration – SocialGroup Integration – Social: [GI-S] Individual team member’s feelings about the similarity, closeness, and bonding within the team as a whole around the group as a social unit. EG: “Members of our team do not stick to together outside of practices and games.”

Quantitative Data AnalysisQuantitative Data Analysis

Cohen (1994), significance testing “has not only failed to support the advance of psychology as a science but also has seriously impeded it”

Significance testing can lead to false conclusions, ie) type I or type II errors

– Type I errors falsely conclude an effect when there is none. Controlled by .05 error rate

– Type II errors falsely conclude no effect when there is an effect. Controlled by increasing sample size or increasing critical alpha to .10 or greater.

Traditional Significance TestingTraditional Significance Testing (Neill in press)(Neill in press)

Traditional Significance TestingTraditional Significance Testing 35% of outdoor ed studies may have reported false

conclusions through type II errors (Hattie, Marsh, Neill, & Richards, 1997)

Significance testing was designed to give binary answers, ie) either yes or no

Significance testing only tells whether an effect has occurred, but does not tell whether one program was more or less effective than another

Outdoor ed research could benefit more from understanding how much change occurred. (Neill, in press)

Effect SizeEffect Size

Effect size measures how much change (or difference) there is between two sets of scores

• A negative ES score indicates a reduced score

• A Score of 0 indicates no change

• A positive ES score indicates an increase in score

% Effect Size %% Effect Size %

Effect Size Description

Below .1 No detectable change

.1 to .2 Small Change

.2 to .3 Small-Med Change

.3 to .4 Med Change (Ave OE)

.4 to .5 Med Large Change

.5 to .6 Large Change

Above .6 Particularly large change

Effect Size DefinitionEffect Size Definition

The major new statistical tool, which is recommended for adoption by outdoor education researchers and evaluators, is called an “effect size” (ES). The ES measures “how much” change or difference there is between two sets of scores.

In mathematical terms, an ES is the average difference between two sets of scores in standard deviation units. An ES measures, for example, the standardized change in raw scores between the beginning and end of an outdoor education program.

Neill, (1999).