UNIT 3 & 4

PSYCHOLOGY

RESEARCH

METHODS TOOLKIT

Prepared by Lucie Young, Carey Baptist Grammar School

lucie.young@carey.com.au

Credit to Kristy Kendall ‘VCE Psychology research methods workbook’ for

some definitions included in this presentation.

STUDY DESIGN

• experimental research: construction of research hypotheses; identification and operationalisation of independent and dependent variables; identification of extraneous and potential confounding variables including individual participant differences, non-standardised instructions and procedures, order effects, experimenter effect, placebo effects; ways of minimising confounding and extraneous variables including type of sampling procedures, type of experiment, counterbalancing, single and double blind procedures, placebos, standardised instructions and procedures; evaluation of different types of experimental research designs including independent-groups, matched-participants, repeated-measures; reporting conventions as per American Psychological Association (APA) format

• sampling procedures in selection and allocation of participants: random sampling; stratified sampling; random-stratified sampling; convenience sampling; random allocation of participants to groups; control and experimental groups

• techniques of qualitative and quantitative data collection: case studies; observational studies; self reports

• statistics: measures of central tendency including mean, median and mode; interpretation of p-values and conclusions; evaluation of research in terms of generalising the findings to the population

• ethical principles and professional conduct: the role of the experimenter; protection and security of participants’ rights; confidentiality; voluntary participation; withdrawal rights; informed consent procedures; use of deception in research; debriefing.

TABLE OF CONTENTS

Hypotheses

Variables (IV, DV), Operationalisation of variables

Identification of extraneous and confounding variables: individual participant differences, non-standardised instructions and procedures, order effects, experimenter effect, placebo effects

Improvement of extraneous and confounding variables: sampling procedures, type of experiment, counterbalancing, single and double blind procedures, placebos, standardised instructions and procedures

Evaluation of Research Designs

Reporting conventions (APA format)

Sampling procedures: Convenience sampling, Random sampling, Stratified sampling, random-stratified sampling

Allocation procedures, control and experimental groups

Qualitative and Quantitative data

Case studies, observational studies, self reports

Descriptive statistics

Measures of central tendency: Mean, median and mode

Inferential statistics and p-values

Conclusions

Generalisations

ethical principles and professional conduct

View in slideshow mode-

everything is clickable!

Click the home button on

each page to return here.

RESEARCH HYPOTHESIS

A hypothesis is a testable prediction about the relationship between

two variables. An experiment looks to find out how the IV affects the

DV.

In any hypothesis, you must include 4 things:

The Population that the hypothesis will apply to

BOTH levels of the Independent Variable

The Dependant Variable

A specific prediction about what will occur

• Lets see what this actually looks like

HYPOTHESES

It is hypothesised that primary school aged children who drink

warm milk before bed will have fewer nightmares than children

who do not drink anything

Population

being tested

Both levels of

the IV (must

say both!)

The specific prediction:

the experimental group

will have less nightmares

than the control group

The DV: what is

being measured.

Here it is the # of

nightmares.

HYPOTHESIS TIPS

You only need to write a research hypothesis. This means you do not

have to operationalise each variable. You still need to be careful though.

Put in detail if you can!

Make sure you include BOTH levels of the IV

You do not need to justify WHY you are making that prediction within

the hypothesis itself. This is what the background info in an introduction

section of a report is for.

Make sure you include the 4 key things mentioned on the previous

slide!

INDEPENDENT VARIABLE

A variable is any condition that can change.

The Independent variable (IV) is the variable that the researcher

changes, manipulates, selects for or varies. This is to see the effect on

the other variable, the DV.

Usually in an experiment there is a control group and an experimental

group. By manipulating the IV differently between these two groups,

researcher can see the effect on the DV.

DEPENDANT VARIABLE

The dependant variable is the condition that is affected by the IV and is

also used to measure the effect of the IV.

Usually a measure of performance, a number or score.

Easy to identify as (usually) something that can be counted/quantified or

compared.

VARIABLES TIPS

If you struggle to remember which is which, look for the dependant

variable first.

This is the measure, the number, the count, or the quality being

investigated.

Its outcome depends on what the researcher does.

Once you have found the DV, figure out what the other key variable is.

This should be your IV.

IV DV EXAMPLES

Eating carrots will improve a persons eyesight

Eating carrots =

independent variable.

(note that ‘carrots’

alone is not

enough…what are you

doing with them????)

Improvement in

eyesight=dependent

variable. Something

you can measure!

OPERATIONALISATION

This is very important. You MUST be able to operationalise each of your

variables. Whenever you are asked for the IV and DV in an exam, 99%

chance it will be asking for the operationalised IV and DV.

Operationalisation just means ‘detail on each variable and how it is

administered or measured’.

You must dig deep in each scenario for as much detail as you can

muster!

OPERATIONALISATION

Remember the example from before? “Eating carrots

will improve a persons eyesight”

Lets pump this up with some more detail!

IV= eating carrots

DV= improvement in eyesight

Operationalised IV= Eating 6 standard sized carrots

daily for a period of 6 weeks

Operationalised DV= percentage improvement in

score on standard eyesight test from week 1 to week

6.

Detail about how many,

how long, what sort,

what you are doing

with them.

Detail about how you will measure

‘improvement’ and also what specific tests

will be used (if you know)

EXTRANEOUS AND CONFOUNDING

VARIABLES

An extraneous variable is any variable other than the IV than causes a

change in the DV.

This has an unwanted effect on the experiment, because it makes any

‘causal’ relationships between IV and DV hard to establish.

EV’s can be participant variables (ie. Intelligence levels), experimenter

effects (ie non-standardised instructions), or situational variables (ie

uncomfortable sleep lab).

A confounding variable is when an EV is not controlled for, it will have

systematic ‘confounding’ effects on the experiment. These will happen

time and time again until that variable controlled for. (look for things

within the design of the experiment)

INDIVIDUAL PARTICIPANT

DIFFERENCES

Each person brings with them to an experiment their own individual

differences.

If particular differences are not controlled for, these may cause a change in

the DV. (see Research Designs and how to control EV’s)

Examples could include: gender, age, intelligence, memory ability, brain injury,

health, experience on a particular task or occupation, sleeping patterns.

When looking for individual participant differences, you must choose the

most relevant and explicit examples if you want to state them as EV’s or

CV’s in the exam.

NON-STANDARDISED INSTRUCTIONS

AND PROCEDURES

EV’s can occur if there are differences in the way the experimenter (or

each experimenter if there is more than one) conducts their

experiment each time they run it.

This could be differences in the procedures carried out or the

instructions given to participants

If there are differences in administration each time, then this is an

example of the research being non-standardised.

This will make it hard to draw any conclusions because there is no

consistency in experimentation.

ORDER EFFECTS

An order effect might occur when (in a repeated measures design) the

sequence in which a person does the tasks affects their performance on

the tasks.

This could be seen through either improving performance on second

task due to practise, or worsening due to fatigue or boredom.

This would confound results, creating a false reading of the effect of the

IV on the DV.

See counterbalancing for ways to improve order effects

EXPERIMENTER EFFECTS

The experimenter effect occurs when there is a change in the

participants behaviour as a result of the interaction or influence with

the experimenter or researcher.

This could be treating the experimental group differently to the control

group, or dropping hints about what results they would expect to see.

See double-blind procedures as a way to minimise experimenter effects.

PLACEBO EFFECTS

The placebo effect is where participants will behave differently due to their

own expectations about the treatment they are receiving.

For example, if participants in the experimental group are given a smart pill

and the control group is given no pill, the experimental group may perform

better because they have been given something they believe will make them

smart. Likewise the control group may perform worse because they believe

they have been given no treatment.

See placebo and single and double blind procedures for ways to reduce placebo

effects

IMPROVEMENT OF EXTRANEOUS AND

CONFOUNDING VARIABLES

In an experiment, the researcher wants to minimise all extraneous

variables, hopefully to achieve good (statistically significant) results.

To do this they must be careful about their research and experimental

design, their sampling procedures, and the ways that they eliminate

potential problems in their experimentation

MINIMISING EV’S: SAMPLING

PROCEDURES

The type of sampling procedure must be appropriate and minimise the

chance of bias in the sample.

For example- random sampling will ensure that each person has a chance,

whereas convenience sampling is not based on chance and invites potential

bias.

For example- if there is a variable in the population that can impact on

results in that particular experiment (age, gender, ethnicity etc) than

stratified/random stratified sampling should be used.

Remember, your sample should reflect your population. How should you

sample to most accurately do this?

Exam tips: you may be asked to justify why a certain technique was used, or

to suggest the most appropriate sampling procedure to be used for a

scenario. Always think: what will most accurately reflect my given population?

See sampling procedures

MINIMISING EV’S: TYPE OF EXPERIMENT

The type of experimental design used must be carefully chosen to

reduce EV’s and CV’s

ie repeated measures experimental design eliminates differences in

participant variables, but adds the potential for order effects.

However if repeated measures is used, counterbalancing can be

put in place to reduce order effects.

The appropriate design could be different in each different

scenario. It depends on the experiment that needs to be

conducted.

Read your scenario carefully, or think carefully about how you

could design your experiment with as few problems as possible.

Exam tips: you may be asked to justify why a certain experimental

design was used, or to suggest the most appropriate experimental

design to be used for a scenario.

COUNTERBALANCING

Counterbalancing is used to control for the order effects in a repeated

measures design.

Counterbalancing is when the participants are divided into two groups. Each

group of which are exposed to both conditions, but are exposed in different

orders.

ie investigating effect of new training technique vs. old training technique on

number of goals scored.

Researcher would have half the sample participate in the old training

technique and then the new training technique

The other half of the sample would participate in the new training technique

then the old training technique.

(important note- the ‘groups’ are only in place to work out the order of

treatment. Each person still takes part in all conditions, therefore is still

repeated measures and not independent groups design…don’t get tricked!).

SINGLE AND DOUBLE BLIND

PROCEDURES

A single-blind procedure is where participants do not know which

group they have been assigned to (experimental or control groups).

This reduces the impact of participant expectations ( helps control for

the placebo effect—not eliminate!)

Note too: it is possible to have a single blind procedure where the

experimenter does not know which groups his participants are in but

the participants do know. This is much less common and unlikely.

SINGLE AND DOUBLE BLIND

PROCEDURES

A double blind procedure is when the participants and the

experimenter both do not know which group they have been assigned

to.

This is to help control for both the placebo effect and experimenter

effects (not eliminate!!).

This will be obvious in a scenario if there is a third party- ie research

assistant- who does the allocating and knows the groups, but does not

conduct the actual experiment.

PLACEBOS

A placebo is a fake or false (non-effective) treatment. This is given to the

control group. Neither group would know which treatment they

receive. (see single and double blind procedures)

This means that participant expectations will be less able to influence

results.

The placebo that is given will help to control (not eliminate) for the

placebo effect.

See placebo effect

STANDARDISED INSTRUCTIONS AND

PROCEDURES

Instructions to participants must be the same each time they are given.

These could be written down and given to the participants to read, or

read out like a script.

Each experimenter must conduct the procedure of the experiment

exactly the same each time it is done. This could be done by using a step

by step list of instructions.

Each participant must be treated in exactly the same way.

Control and experimental groups should not be treated differently,

apart from the specific variable being manipulated.

See non-standardised instructions and procedures

EVALUATION OF RESEARCH

DESIGNS

The experimental design is very important. The three we use in psychology are:

Repeated Measures

Matched Participant

Independent Groups

You have to ‘evaluate’, which means you need to know what they are, how to use them, AND the strengths and weaknesses of each (yes plural = at least 2!!). You will also need to know when each is appropriate to use and when it is less appropriate. You will need to be able to justify this.

INDEPENDENT GROUPS

Participants in the sample are randomly allocated to and have

an equal chance of being in to either a control group or

experimental group.

Strengths Limitations

Use of different participants in each

condition means there is no order effects to

control

May not be a representative sample of the

population as no differences between the

two groups have been controlled for

Use of different participants in each

condition means that the time required to

complete the study is often shorter, as two

conditions can be conducted at one time.

Inexpensive, quick, easier to administer.

Above is particularly evident if there is a

small sample size, so a disadvantage would

be that this design requires more individuals

to be used in order to reduce bias from

uncontrolled participant variables.

Use of different participants in each

condition means that there is less chance of

attrition or drop outs, compared with the

repeated measures design.

INDEPENDENT GROUPS

Use when the experiment does not require

particular participant variables to be

controlled (ie intelligence or memory ability

will not impact on results) and when you can

easily access large numbers of participants, to

quickly set up and administer the

experiment.

MATCHED PARTICIPANTS

After pre- testing, participants are paired together based on

their similarity to a certain characteristic researcher wishes to

control for. One from each pair is assigned to control and other

to experimental group.

Strengths Limitations

Participant variables are more constant across

conditions, controlling this as possible confounding

variables.

Matching participants on all characteristics is unlikely to

be able to occur. People still have individual differences.

Defining the characteristic properly is also difficult.

Use of different participants in each condition means

that the time required to complete the study is often

shorter, as two conditions can be conducted at one time

Matching participants is time consuming and costly. Twins

are best for this as they have the most characteristics in

common; however twins are uncommonly available in

large numbers.

Use of different participants in each condition which may

be conducted at the same time, means that there is less

chance of attrition or drop outs, compared with the

repeated measures design.

Pre-testing may create an order effect or an expectation

from the participants

Attrition (drop outs) can still be a problem, if one part

of the pair stops or discontinues, both sets of results

must be removed

MATCHED PARTICIPANTS

Use when you have a variable that is fairly

easily defined and participants can be

matched on ie age, gender, IQ score. Used

when the participant variables would impact

on the results. Use when order effects from

Repeated measures would occur and need

to be controlled.

REPEATED MEASURES

Repeated measures is when each member of the sample

participates in both the experimental and control conditions.

Strengths Limitations

Controls potential confounding variables arising

from individual participant differences, because it

uses the same participants in all conditions.

Order effects may occur from the participant

taking part in the first condition and then the

second condition. Performance on second task

may be enhanced or due to practise.

Performance may be decreased on the second

task due to fatigue or boredom.

Requires a relatively smaller number of

participants compared with other experimental

designs, as each participant acts in both the

control and experimental groups

Repeating the conditions with the same

participants takes time (particularly if delayed

time to eliminate order effects). This means that

it takes a lot longer for researcher to gather all

data. Participants may be more likely to drop out

between testing times. This means all of their

results would then be removed

REPEATED MEASURES

Used when the participant variables would

impact on the results and need to be

controlled. Use when a smaller number of

participants is necessary or the variable is

hard to define or control (ie brain damage)

APA REPORTING CONVENTIONS

APA stands for American Psychological Association and is the standard

format for all psychological reports.

This should be familiar to you as being the sections of your ERA reports.

Abstract: Summary

Title: contains IV and DV

Introduction: Contains background info, key terms, IV, DV, aim and hypothesis

Method: contains participants, materials, and procedure

Results: contains summarised data ie tables and graphs, and a statement of result

Discussion: contains support for hypothesis, discussion of research methodology,

implications, generalisations and conclusions.

APA REPORTING CONVENTIONS

• It is unlikely you will get a multiple choice or short answer question on the APA format.

• What is very likely is that in the extended response, you will be asked to write a section of an

ERA report, ie. the introduction, or the discussion, or parts of both.

• You need to know the things that goes in each section, the way the text should flow, and the

order that information would logically appear as per the reporting conventions.

• See ERA guidelines, or ‘How to write’ sheets, on CLASSe for more help

POPULATION AND SAMPLE

The group we wish to investigate and draw general conclusions about is

called the population

For practical reasons, we cannot test everyone in the population

We draw a smaller group from the population to test, called the sample

It is important to make sure that the sample is representative of the

population (shares similar characteristics) for the best research conclusions

to be made. We have different sampling techniques to do this.

CONVENIENCE SAMPLING

Convenience sampling is when subjects are picked based on their

availability at the time of the experiment. Quick, easy, cost effective.

Convenience sampling usually presents a biased sample. They may not

be representative of the population.

Looks like: people on the street, people in a class, people who volunteer,

first 10 people in the door

RANDOM SAMPLING

Random sampling is where every member of the

population has an equal chance of being selected

for the sample being used in the study.

Quick(er than stratified), relatively simple to

employ. Less chance of bias, however can be open

to bias if the population has under or over

represented groups in it.

Looks like: names of everyone in population being drawn out of hat,

being put into a computer random name generator, being given a number

and numbers drawn or rolled.

STRATIFIED AND RANDOM STRATIFIED

SAMPLING

Stratified sampling is to be used when there is diversity or under/over

represented groups in the population.

Stratified sampling involves breaking the population into groups or ‘strata’

based on the characteristics you wish to control for in the sample.

This could be age, gender, ethnicity etc.

Once the population is divided into strata, participants are selected for the

sample in the same proportions (ratios, percentages) that exist in the

population.

If random stratified sampling is used, this just means that the selection from

the already defined strata uses the equal chance methods (ie names in hat,

number generator)

EXAMPLE

Researcher interested in memory ability of adults.

Divides population into age brackets 21-30, 31-40 etc

Then selects participants from each age bracket, in the

same proportion that exists in the population

Ie, if there was 20% of adults aged 21-30 in the

population, there should be 20% of the sample also

aged 21-30.

Looks like: when researcher needs to keep proportions of something

controlled, look for gender, age, ethnicity control, selecting sample in

same proportions. May or may not then select using random methods.

ALLOCATION TO CONTROL

AND EXPERIMENTAL GROUPS

Once a sample is chosen, participants must be allocated to either the

control or experimental groups.

The experimental group is the group that is exposed to the

experimental condition where the IV is present. The control group is

the group that is not exposed to the experimental condition (the IV).

The control group provides the comparision or baseline performance

on the DV against which the performance of the experimental group

can be compared.

The best way to ensure that participant characteristics are evenly

distributed in both groups is through random allocation to groups.

Random allocation is where each person in the sample has an equal

chance of being in the control or the experimental group.

This could be done via the same methods as random sampling: names

in a hat, toss of a coin, numbers given then drawn out etc.

QUALITATIVE AND QUANTITATIVE

DATA

There are different types of data that can be collected in different ways.

Data is sometime called empirical evidence in psychology

Qualitative data are data that describes changes in the qualities of the

behaviour and are often expressed in words. Often subjective due to

different individual interpretations of the data.

Example could be words to describe someone’s facial expressions ie. happy,

sad, surprised, very shocked.

QUANTITATIVE

Quantitative data is that which takes a numerical or categorical form,

and can be statistically analysed and measured.

Often seen in numbers or counts

ie. quantitative data example from the previous slide could be the

number of times someone smiles while reading a page of comics.

Tends to be based on statistical data so seen as less subjective and more

objective.

CASE STUDIES

A case study is an in-depth analysis of an individual, group or situation.

They might include medical histories, interviews, observations, reports

and other pieces of information.

The benefits are that it allows researchers to gather a lot of information

on the one person/group

The limitations are that it may be quite specific to that individual and

findings may not be generalised to the population. It is also time

consuming.

OBSERVATIONAL STUDIES

An observational study involves an individual watching another person

or group, usually in their natural environment, and taking notes about

the observations made.

The benefits are that it allows a more natural environment eliminating

some EV’s. There is also less chance of someone responding in ways

they perceive as favourable (if they don’t know/forget they are being

observed)

The limitations are that the interpretation of the observations can be

subjective and open to observer bias.

SELF-REPORTS

A self-report method is used when an individual comments on their own

thoughts, emotions and perceptions when answering a series of questions

asked by the researcher.

Advantages are that it allows the researcher to gain valuable information

about things that are not overt.

A disadvantage is that participants may not answer truthfully.

The self reports could be in the form of a:

Survey: verbal/written questions. Can be open or closed questions

Questionnaire: written questions. Can be open or closed questions. Often uses a rating

scale.

Interview: usually face to face or telephone questions. Might be structured or

unstructured. Could also be both closed or open questions.

DESCRIPTIVE STATISTICS

A descriptive statistic is a way to summarise, organise and describe raw data, so that it can be more easily interpreted.

Be careful- if a descriptive statistic alone is given, you cannot draw any cause and effect relationships, and therefore limited conclusions and generalisations

See inferential statistics and p-values, conclusions and generalisations

The most common forms are:

Percentages

Graphs of all kinds

Tables

Measures of central tendency

MEASURES OF CENTRAL TENDENCY

Measures of central tendency involve calculations to show how scores fall in

a data set. The 3 most common ones used are mean, median and mode.

Mean: Also called the ‘average’. All scores in the data set are added up

(summed) and then the total score is divided by the number of pieces of

data in the set.

Median: Is the middle number in an ordered set of scores. You MUST make

sure the data set is firstly organised from smallest to largest entry. The

median is the middle number in this ordered set. If the data set is an even

number, you must take the mean (average) of the two middle numbers.

Mode: is the most commonly or most frequently occurring number in the

data set.

Each measure may be useful depending on circumstances. Often small

samples or outliers can make some less useful.

INFERENTIAL STATISTICS

Inferential statistics allow us to infer cause and effect relationship

between variables. They allow us to say that the IV has affected the DV.

They allow conclusion to be formed and generalisations to be made.

They are statistical calculations. One that you need to be able to

interpret is the p-value (usually from a t-test). You do not have to

actually calculate this in Yr. 12.

P-VALUE

An acceptable level of error is set by the researchers before the

experiment begins.

Convention is for this to be set at p<0.05. It is possible for it to be set

lower (ie. P<0.01) but you will be told of this.

p<0.05

The probability that

results occur due to

chance

Is less

than 5 %

P-VALUE: INTERPRETATIONS

If

p<0.05

Note this usually means less than or

equal to.

If

p>0.05

Results are statistically significant Results are not statistically significant (do

not say insignificant)

Results (changes in DV) are due to the

effects of the IV. The likelihood of chance

is less than 5%.

Results are likely to be due to chance

alone

Hypothesis can be accepted; conclusions

can be drawn

Hypothesis is rejected; no conclusions can

be drawn

Generalisations may be able to be made.

3 step method to p-value success:

1. Restate the obtained p-value

2. State if statistically significant or not

3. Explain what this means in terms of likelihood of chance

CONCLUSIONS

A conclusion is a decision or judgement about the research results

compared to the hypothesis.

For a conclusion to be made, there should be a statistically significant

result.

This means you can only draw conclusions if you have been given

inferential stats, not descriptive stats.

GENERALISATIONS

Generalising to a population is the main goal of

psychological research.

A generalisation is when a research finding can be

applied from the sample to the broader population.

A generalisation should only be made if the

following four things are all met.

1. The results must be statistically significant (inferential stats only!!!)

2. The sample is representative of the population you wish to generalise to

3. The sampling method was appropriate (unbiased)

4. Extraneous and confounding variables have been controlled for to the best

of ability (no obvious flaws)

ETHICS

the role of the experimenter is to protect their participants

psychological and physical welfare, to uphold the integrity of the

profession, to be fair and just towards all involved and to provide a

benefit mankind with their research.

protection and security of participants’ rights involves upholding 6 key

ethical principles.

ETHICAL PRINCIPLES (CVWIDD)

Confidentiality: researcher must collect, retain and dispose of all information related to participants in a manner that does not disclose their identity within the research. Participants must not be connected to their results or publically identified.

Voluntary participation: each participant has the right to choose to willingly take part in the research and must not be coerced, forced or tricked into taking part.

Withdrawal rights: each participant may leave the research at any time without negative consequences or pressure to continue, and may remove their results from the data set at any time.

Informed consent procedures: researcher must first fully explain to the participant about the true nature and the risks of the experiment, and then obtain written permission on a consent form in order to take part. If under 18 or incapable, parental/guardian consent should be given.

use of deception in research: deception should not be used in research unless it is really necessary. It may only be used when knowing true nature of the study would change the results, and when the deception would not cause harm to the participants. If deception occurs, participants need to be fully debriefed.

Debriefing: debriefing occurs after the experiment has concluded. During debriefing, a researcher must inform the participant about the results and the true nature of the experiment, and correct any misconceptions. The researcher must provide counselling if harm has occurred.

LAST WORDS

Remember that Research Methods is a big component of the exam, but it can also be fairly predictable!

Think about the likely questions you would be asked for each key word. Think about how each might look or present themselves in a scenario- what does a placebo look like? What clues are there to show they’re using stratified sampling?

Practice is really the key to success. Keep looking for different extended responses and research evaluations. After a while it will become second nature!

Good Luck!