Research Methodsrevision

Peer review

What do peer reviews achieve?• Allocation of funding: public bodies can peer review studies to gather whether or not they will be worthwhile

• Publication of research in scientific journals and books: this gives scientists the opportunity to share the results of their research. Peer review process can be used to eliminate incorrect or faulty date entering the public domain.

• Assessing the research rating of university departments: all university departments are expected to conduct research and this is assessed in terms of quality future funding. Good research = better funding.

What problems occur with peer reviews?

• Unachievable ideal: is not always possible to find an expert to review a research proposal. Poor research could therefore be passed on.

• Anonymity: gives people the chance to ‘settle old scores’ or bury rival research.

• Publication bias: researchers may publish something if it will prove their own research. Also, positive results tend to be published more often.

• Preserving the status quo: peer review has a preference for data that already follows what other current theories have found rather than dissenting. Peer review therefore can make the pace of development slower.

Specific research methods used:1. Experimental2. Observations3. Correlational analysis4. Surveys5. Interviews6. Case studies7. Pilot studies

Lab experiment ADVANTAGES DISADVANTAGES

within an artificial environment where the researcher has full control over the

variables

• Specialised equipment• Cause and effect• Easy to replicate

• Low ecological validity• Demand characteristics

Field experiment ADVANTAGES DISADVANTAGESis within a more natural environment but

researcher has some control.• Higher ecological validity• Lower demand characteristics

• Control not always maintained• Equipment not practical

Natural experiment ADVANTAGES DISADVANTAGESreal life environment where experimenter

has no control. • No/few demand characteristics

• Low control• Cannot establish cause and effect • Time consuming

1. EXPERIMENTAL

CONTROLLED ADVANTAGES DISADVANTAGES

Is where the conditions are contrived by the researcher. This type of observation may be carried out in a laboratory type

situation.

• Easy to replicate• Easy to analyse

• Low ecological validity• Researcher bias• Demand charac.

NATURALISTIC ADVANTAGES DISADVANTAGESIs when observation is within the participants natural environment.

• Higher ecological validity• Qualitative data

• Low validity• Too specific

PARTICIPANT ADVANTAGES DISADVANTAGESObserver is involved with the participants

he/she observes. • Qualitative date• Very in-depth• First hand experience.

• Costly• Hard to arrange – time consuming• Low validity

2. OBSERVATIONAL

NON-PARTICIPANT ADVANTAGES DISADVANTAGESObserver is NOT involved with the

participants he/she observes. • Not relying on memory

• Ethical issues

• This is the use of statistical correlation to evaluate the strength of the relations between variables.

3. CORRELATIONAL ANALYSIS

ADVANTAGES DISADVANTAGES• Shows there is a connection between two variables. This evidence can strengthen research and theories

• Shows a relationship

• Causality cannot be established as there is always going to be other variables (extraneous variables) involved.

• Linear graphs do not accurately represent variables.

• This is a set of questions that can be written out and delivered to participants in order for them to fill out.

4. SURVEYS

ADVANTAGES DISADVANTAGES• Can approach sensitive topics

• Sent them out to many people cheaply and quickly i.e. Post or email.

• Lots of data – both qualitative and quantitative.

• If badly worded, can effect participants responses. (i.e. Ambiguous or leading)

• Very low response rate (33%) – especially if via post.

•Demand characteristics

UNSTRUCTURED ADVANTAGES DISADVANTAGESThe interviewer has an idea of what they

will ask they interviewee but has no questions pre-prepared.

• Better validity• Speak openly• Detailed info• Flexible

• Difficult to analyse• Training for interviewer needed• Harder

STRUCTURED ADVANTAGES DISADVANTAGESThe interviewer has a pre-set list of

questions that they read from a sheet as they ask the interviewee.

• Less likely to deviate• Data analysis easier• Easy to generalise• Less training needed

• Lack of validity – formality of sit.• No further questions• Demand charac. And social desirability.

5. INTERVIEWS

SEMI-STRUCTURED ADVANTAGES DISADVANTAGESThe interviewer has some questions ready

but may ask more questions depending on interviewee’s answers already given

• Qualitative data• Keep focus but still gather more data.

• Training needed• Time, money and effort• Social desirability

• Focusing on one individual or a group of people and using different research methods to attain data.

6. CASE STUDIES

ADVANTAGES DISADVANTAGES• Lots of data attained – both qualitative and quantitative which will be very in-depth

• Can use case studies to challenge existing theories

• Researcher bias

• Retrospective data that is not always reliable

• Very narrow study if focus is on one individual

• Cannot generalise results to everyone

• A practice attempt before the real study to find if there is any problems with the upcoming experiment.

7. PILOT STUDY

ADVANTAGES DISADVANTAGES• Gives the researcher useful knowledge of how the task with be carried out.

• Eliminates any faults which will save money as they will not occur during the real thing

• Participants know it is a pilot study and may act differently to make it easier for the environment

• Participants may sabotage the research by giving the experiment false faults

Sampling...Opportunity sampling

Whoever is around at the time is chosen to take part in a study – based on availability.

ADVANTAGES DISADVANTAGES

• Quick and therefore the most common type used

• Convenient and easy

• Not representative – therefore any results can not be generalised to the public Volunteer sample

Participants see an advert for a study and sign up.

ADVANTAGES DISADVANTAGES

• Consent given instantly

•Motivated particpants

• social desirability

• Not representative – therefore any results can not be generalised to the public

Sampling...Random sample

When everyone is given an equal chance (i.e. Picked out from a hat.

ADVANTAGES DISADVANTAGES

• Convenient and easy

• More representative then a volunteer or opportunity sample

• Does not guarantee to be representative

• Time consuming . Stratified sampleEnsuring the ratio of

people is equal. 50:50ADVANTAGES DISADVANTAGES

• Most representative

• Can generalise from this data to the rest of the population

• Sorting out participants for the study can be time consuming

Experimental designs:Design description ADVANTAGES DISADVANTAGES

Independent groups: different people in different

conditions

• No order effects• Reduction in demand characteristics

• Different people – hard to make a comparison• Lots of different people – expensive and individual dif.

Repeated measures: same people doing all the

conditions

• Fewer participants• Comparing them with themselves. No individual differences

• Order effects – individuals becoming tired or bored•INTRODUCE COUNTERBALANCING

Matched pairs design: similar people doing same

condition

• Increase validity. Individual differences decreased• Lower order effects

• People never similar• Time consuming matching people

RELIABILITY• INTERNAL RELIABILITY Concerns the extent to

which there is consistency. Different parts should give consistent results throughout. i.e. IQ tests should all have questions of similar difficultly

Split-half method: split the test in half by odd and even numbers – this can assess questionnaires.

• EXTERNAL RELIABILITY Concerns the extent to

which there is a measure of something that is consistent over time. Test should produce consistent results regardless of when used.

This can be assessed with the test-retest method: repeating the test at a later date.

How do you improve reliability?

NUMBER OF MEASUREMENTS

When possible, make more than one measurement for each participant (avoid participant bias)

PILOT STUDIES Do it as a small scale test to make sure you are studying what needs to be studied

STANDARDISATION Have a set standard for the way researchers must collect data

INTRA RESEARCHER

Having more than one researcher and comparing the results that they both achieve

VALIDITY• INTERNAL VALIDITY Concerns the extent to

which the changing IV is entirely reliable for the DV and not an extraneous variable. Milgram’s study was valid as the participants believed the study was real.

Face validity: examines/assesses the test

• EXTERNAL VALIDITY The extent to which things

can be generalised to other people, times, situations etc. Milgram’s study did not have it as it was set in a lab, not in a natural environment.

Predictive validity – two sets of scores are obtained at different times. Allows accurate prediction of future behaviour.

How do you improve validity?

NATURALISTIC STUDY

Natural environment with a large and diverse sample

CONTROL EXTRANEOUS VARIABLES

Remove all the extraneous variables – then they will not effect what the researcher is studying. However, this is hard to do within a natural environment

AVOID DEMAND CHARACTERISTICS

• Single blind technique. Don’t tell the participant what they are doing – i.e. Use a placebo or a drug but do not tell the participant which they are receiving

GRAPHS!

BAR CHART

FREQUENCY POLYGON

HISTOGRAM

LINE GRAPH

SCATTER GRAPH

Significance levelsP < 0.01

Stricter level of significance. It means we are 99% sure the results are accurate and there is a real effect. Used more for drug studies as the results need to be accurate in order to be safe. This type of significance will lower the chances of a type 1 error (FAR) where you falsely accept the research hypothesis.

P < 0.05A conventional level of

significance. This means you are 95% sure that it is a real effect. This level would be used more for studies that have smaller samples and are not too important. This type of significance will also lower the chances of a type 2 error (FAN) falsely accepting the null hypothesis.

TYPES OF DATANominal

Where simple categories are used – i.e. Categorising people as smokers or non-smokers. A numerical value

often cannot be assigned. You can normally count up frequencies

within these categories (i.e. Number of women or number of men.)

OrdinalWhere numerical value is used,

but based on ranks or ratings. i.e. Doctors may be ranked in

preference by patients. Ordinal data is very subjective as based on personal opinions

of people.

Interval dataThis is when data is in the form of

equal units e.g. Score on an objective test. In such a case, if one participant has scored 20 on one test, and another has scored 10, then it is correct in

saying participant one performed twice as well as

participant two.

RatioInterval data with an absolute value of 0 i.e.

Distance or speed

Choice of statistical test

What level of measurement is the

data?

NOMINALChi-square

Is the study about difference between two sets of data or a

relationship between two variables?

RELATIONSHIPS (CORRELATIONAL)Spearman’s Rho

What type of design was it?

INDEPENDENT GROUP

Mann Whitney

REPEATED MEASUREWilcoxen