The Guide to Researching Audiences

The Guide to Researching AudiencesDr Rachel Quirk, Martin Olver, Dr Max Hammond & Dr Claire Davies

JISC Content

JISC, British Library, BBC, National Health Service, Becta, and Museums, Libraries and Archives Council working together to fully realise the potential of e-content for all users. For more information on the Strategic Content Alliance, please visit:

www.jisc.ac.uk/contentalliance

Produced and funded by JISC

Curtis+Cartwright Consulting Ltd Main Office: Surrey Technology Centre, Surrey Research Park, Guildford, Surrey GU2 7YG T: +44 (0)1483 685020 E: [email protected] W: www.curtiscartwright.co.uk

This report is available in alternative formats which can be found at: www.jisc.ac.uk/contentalliance

Contents PAGE 3

ContentsOverview of this Guide 4

1. Introduction 5

1.1 Why,what,who…? 5

1.2 Usingtheguide 6

1.3 Supportingdocuments 9

1.4 Seniormanagementcommitment 9

2. Describing and Defining the Target Audience 10

2.1 Theimportanceofthetargetaudience 10

2.2 Theconceptofthetargetaudienceinservicedevelopment 11

2.3 Definingthetargetaudience 11

2.4 Describingthetargetaudience 12

3. Planning Audience Research 15

3.1 Introduction 15

3.2 Objectivesofaudienceanalysis 15

3.3 Startingpointsfordefiningaudienceresearchobjectives 16

3.4 Choosingthemethodology 22

3.5 Researchingnon-usage 25

3.6 Recruitingsubjectsforaudienceresearch 27

3.7 Othersourcesofaudiencedata 28

3.8 Audienceresearchdoesnotneedtobeperfecttobeuseful 29

4. Collecting Audience Data 31

4.1 Undertakingresearch:in-houseorthirdparty? 31

4.2 Overviewofdatacollectionmethods 33

4.3 Prosandconsofthemethods 43

4.4 Indirectapproachestoaudiencebehaviour 45

5. Modelling the Audience 47

5.1 Introduction 47

5.2 Analysingquantitativedata 48

5.3 Makinguseofqualitativedata 51

5.4 Analysingaudiencesatisfactionforservicedevelopment 52

5.5 Personasanduserprofiles 52

6. Making Use of Audience Research 55

6.1 Introduction 55

6.2 Audienceresearchincontext 56

Appendix A: Glossary 58

Appendix B: Reasons to do Audience Research 60

B.1 Whydoaudienceresearch? 60

B.2 Theusesofaudienceresearch 61

Appendix C: Digital Audiences and Digital Services 64

C.1 Thedigitalrevolution 64

C.2 Audiencesinthedigitalenvironment 65

C.3 Digitalcontentisdifferent 66

C.4 Therelationshipbetweendigitalandnon-digitalworlds 67

C.5 Technologyshouldservetheaudience 67

C.6 Relationshipbetweendigitalandnon-digitalservices 68

C.7 Digitalaudiencesmaybeanalyseddifferently 69

Appendix D: Bibliography 70

D.1 Introduction 70

D.2 Generalaudienceresearch 70

D.3 Planningandanalysis 70

D.4 Methods 71

D.5 Servicedevelopment 72

D.6 Digitalcontext 72

D.7 Evaluationandimpact 73

The Guide to Researching Audiences

PAGE 4 Overview of this Guide

Overview of this Guide

PLANNING AUDIENCE RESEARCH

COLLECTION OF AUDIENCE

DATA

MODELLING THE AUDIENCE

MAKING USE OF AUDIENCE

RESEARCH

Section 3

Setting clear objectives for research activities, and planning activities to meet these objectives

Section 5

Analysing and interpreting the data collected to build up a picture of the audience

Section 4

The methods available for collecting audience data, and deciding how to conduct the research

Section 6

Using the research to meet the needs, wants and expectation of the audience

DESCRIBING AND DEFINING THE TARGET

AUDIENCE

Section 2

Defining who you would like your audience to be, and how to approach segmenting the audience

REASONS TO DO AUDIENCE

RESEARCH

BIBLIOGRAPHY

GLOSSARY

DIGITAL AUDIENCES AND DIGITAL

SERVICES

INTRODUCTION

SUPPORTING INFORMATIONCORE GUIDANCE

Section 1

An introduction to the toolkit, including an overview of how it might be used in practice

Annex A

A glossary of the key terms used in the toolkit (eg ethnographic research )

Annex B

The reasons for doing (and uses of) audience research, eg for service development

Annex C

The differences between digital and non-digital services and audiences

Annex DA bibliography of the information sources used for this toolkit, and suggested further reading

1. Introduction PAGE 5

1.Introduction

1.1 Why, what, who…?

Why do audience research?1.1.1 Itisincreasinglyimportantforpublicsectorservicesandresourcestobeabletodemonstrate

thattheyareusedandvaluedbyanappropriateaudience.Theusesofaudienceresearch

arediscussedindetailatAnnex B, and extendfarbeyondsimplydemonstratingacertain

numberofvisitorsorwebsitehitstotickaboxonaservicelevelagreement.Insummary,

audienceresearchcanbeusedto:

■■ developaproductorservice;

■■ evaluatetheservice(eg audiencesatisfaction);

■■ demonstrateaccountabilitytofunders(eg valueformoney,highlyvaluedbyaniche

audience);

■■ informlong-termplanning;

■■ attractsponsorshipandmarketing;

■■ informbusinessplanning(eg tofeedintoabusinesscase);

■■ informthemanagementofcontractualrelationships;

■■ supportprofessionaldevelopment.

Audienceresearchdoesnotneedtobeperfecttobeuseful

1.1.2 Audienceresearchshouldbeseenasanongoingprocess,ratherthananoccasional,one-

offevent.Evenasmallaudienceresearchprojectisworthwhile.Manytechniquescanbe

implementedquitecheaplyoradaptedtoashoestringbudget.Someinsightintoyour

audienceisvastlybetterthannoneatall.

1.1.3 ThisguidehasbeenproducedbyCurtis+CartwrightConsultingLtdonbehalfoftheStrategic

ContentAlliance.

1.1.4 AglossaryofthekeytermsusedintheguideisprovidedatAnnex A.


PAGE 6 1. Introduction

Who is this guide for?1.1.5 ThisguidehasbeenwrittenforpeopleintheUKpublicsectordeliveringonlinedigital

serviceswhowishtoresearchtheiraudiences.1

1.1.6 Itisprimarilyaimedatnon-expertswhodonothavemarketresearchskillsthemselves,and

whodonothaveaccesstodedicatedmarketresearchteams.Itisthereforelikelytobemost

usefulforpeopleinsmallorganisations/services,orthoseconductingsmallprojectsinlarger

organisations.Itmayalsobeusefulforpeopleprocuringserviceswhowanttoknowwhat

guidelinestoprovide,andforthosesettingpolicies.

What is in this guide?1.1.7 Thisguidesetsoutthebasicprinciplesofaudienceresearch.Thesecanbefollowed

regardlessofthetypeofserviceoraudience,andwillhelpyoutoconductaudienceresearch

moreeffectively(betterresults)andefficiently(lowereffort),withfewerproblemsand

unforeseencomplications.Theyprovidethebuildingblockstoenableyoutodesign,conduct

andapplyyourownaudienceanalysisresearch.Whatthisguidewillnotprovideyouwithisa

ready-madeaudienceanalysisprogrammespecificallydesignedforyourservice.

1.1.8 Thisisaguidetocurrentgoodpracticeandastartingpointforfurtherreading.Thereis

nothingveryradicalinthisguide–therearemanyotherresourcesavailableonaudience

analysisandmodelling,someofwhicharereferencedinthisguide(seethebibliographyat

Annex D).

1.1.9 Theguidehasaspecificfocusononlinedigitalservices.Thedigitalrevolutionhasresulted

inanexplosionofpossibilities:newandinnovativeservices;adelugeofcontent;availability

ofadditionalresearchtechniques(eg webstatistics);andpotentiallynewaudiences.

Relationshipsbetweendigitalcontentandnon-digitalcontent,betweendigitalservicesand

non-digitalservicesandbetweena‘digitalaudience’anda‘non-digitalaudience’areanew

andinterestingconsiderationforserviceprovidersandarediscussedinAnnex C.

1.2 Using the guide

1.2.1 Thebasicprinciplesarestructuredasstep-by-step activitiesinthelifecycleofaudience

research(alsorepresentedinFigure1-1):2

■■ Describing and defining the target audience (Section 2):ifyouwishtodelvestraight

intothepracticalguidance,thisistheplacetostart.Itdiscussestheimportanceof

knowingwhoyouwouldlikeyouraudiencetobe,andprovidesguidanceabouthowto

describeandsegmentyourtargetaudience.Thissectionwillbeespeciallyrelevantifyou

arecurrentlydesigninganewservice.However,itwillalsoberelevantifyouwouldlike

tostartfromthebasicswhenplanningresearchactivitiesforservicesthatarealreadyin

productionandthathaveanestablishedaudience.

1 Although it is focused on online digital services (ie those accessed via the Internet), it will also be useful for researching audiences reached via other channels.

2 Note that although these activities are presented as separate activities in a linear process, in practice this is not likely to be the case – it is likely to be an iterative process.



■■ Planning audience research (Section 3):thissectionistheplacetostartifyouhave

alreadydefinedyourtargetaudienceandwishtostartplanningyouraudienceresearch.

Itaddressestheimportanceofsettingclearobjectivesforaudienceresearchactivities,

andthingstoconsiderwhenplanningactivitiestomeettheseobjectives.Italsosetsout

examplequestionsabouttheaudiencewhichcandriveyourresearch.

■■ Collecting audience data (Section 4):thissectionprovidesanoverviewofthevarious

audienceanalysismethodsavailable(eg focusgroups,webanalytics).Italsoprovides

someinformationabouttheprosandconsofconductingtheresearchin-houseand

contractingtheworkouttoamarketresearchcompanytohelpyoudecidethebestroute

foryourorganisation/service.

■■ Modelling the audience (Section 5):thissectionfocusesonhowtoanalyseandinterpret

thedatathathasbeencollectedtoaddressthespecificresearchquestionsandbuildupa

betterunderstandingoftheaudienceandtheirinteractionswiththeservice.

■■ Applying the information (Section 6):thissectionbrieflydiscussesusingtheinformation

tobuildtheservicearoundtheneeds,wantsandexpectationsoftheaudience.

1.2.2 Howyouchoosetousetheguidancetoplanandconductaprogrammeofaudienceresearch

willbeinfluencedbymanyfactorsincluding:thematurityoftheservice,thebudgetavailable,

theamountofdigitalcontentheldandthebreadthofthetargetaudience.


PAGE 8 1. Introduction

Figure 1‑1an outline of the step-by-step activities, and structure and content of the guide

SECTION 3

Planning audience research

SECTION 4

Collection of audience data

SECTION 5

Modelling the audience

SECTION 6

Making use of audience research

Setting clear objectives for research activities, and planning activities to meet these objectives

Analysing and interpreting the data collected to build up a picture of the actual audience (and non-users )

Choosing the data collection methods and conducting the programme of audience research

Using the information to build the service around the needs , wants and expectation of the audience

Issues with analysing and interpreting data

Improving understanding of the audience

Audience analysis in context

Realising the full value of the research

Answering specific research questions

Defining research objectives

Combining qualitative and quantitative techniques

Recruiting subjects

Conducting the research in-house or buy it in?

Overview of the methods

SECTION 2

Describing and defining the target

audience

Defining who you would like your audience to be

Defining the target audience

Segmenting the target audience



1.3 Supporting documents

1.3.1 Thisguideissupportedbyaseparatedocumentwhichprovidesreal-world case studies

demonstratinggoodpracticeinaudienceresearch.3Thecasestudiesarefromboththe

publicandprivatesectors,andboththeUKandabroadandareusefultogainapractical

understandingofwhatcanbeachievedwhenaudienceanalysisisdonewell.

1.3.2 Anillustrative case studypresentingahypotheticalprogrammeofaudienceresearchtoput

thisguideincontextisavailableasaseparatedocument.4

1.4 Senior management commitment

1.4.1 Audienceanalysisresearchshouldbedonewithcommitmentandsupportfromsenior

management–withoutit,thefullbenefitsoftheresearcharelesslikelytoberealised.

1.4.2 Shoulditberequired,thisguideprovidesyouwithsomeofthebasicinformationneeded

towriteaninformalbusinesscaseforaudienceresearchtofacilitatebuy-infromsenior

management.

3 The Guide to Researching Audiences: case studies, Curtis+Cartwright Consulting Ltd. v1.0.5 December 2008.

4 The Guide to Researching Audiences: illustrative case study. Curtis+Cartwright Consulting Ltd. v1.0.5 December 2008.

PAGE 10 2. Describing and Defining the Target Audience

2.DescribingandDefiningtheTargetAudience

This chapter discusses the importance of the target audiences and looks at ways of defining the target audiences, and approaches to audience segmentation

Knowing who you would like your audience to be and what service you intend to offer is a vital part of business planning and service development.

Bothformalandinformalapproachestodefiningthetargetaudiencearepossible.

Audiencesegmentationcanbehelpfulwhendescribingthetargetaudience.Differentwaysofsegmentingtheaudiencewhicharediscussedinthischapterinclude:

■■ Demographicsegmentation.

■■ Segmentationforaccountabilitytofunders.

■■ Functionalortask-orientedsegmentation.

■■ Objectiveorientedsegmentation.

2.1 The importance of the target audience

2.1.1 Knowingwhoyouwouldlikeyouraudiencetobeandwhatservicesyouintendtoofferisvital

foranyservice.Itshouldbepartofbusinessplanningasthereareimplicationsformethods

offundingandpotentialroutestosustainabilityforadigitalservice.

2.1.2 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,the

functionalityprovided,meansofaccessetc.Itisimportanttorecognisethataservicemay

havemorethanonedistinctaudience.Needsandexpectationsofthedifferentaudiencesare

likelytobeonlypartlyoverlappingandtheimplicationsthishasforservicedeliveryshouldbe

considered.

2.1.3 Understandingtheaudiencecanbevitaltogiveasenseoffocusandidentitytoaservice.

Tryingtobeallthingstoallpotentialusersisdifficult.Differentgroupsofusersmayhave

incompatibleneeds(egexperiencedacademicresearcherswillprioritiseheavyweight

searchandanalysistoolswhenusinganarchive;leisureuserswillexpectaveryuser-

friendlyinterfacethatletsthemquicklyexplorethearchiveandgetaflavourofthecontents).

Thisdoesn’tmeanthatadigitisedarchiveshouldn’taimtoappealtobothaudiences,but

understandingthatthesearedistinctaudienceswithdifferentneedsshouldinformthedesign

anddevelopmentoftheservice.

2. Describing and Defining the Target Audience PAGE 11


2.2 The concept of the target audience in service development

2.2.1 Servicedevelopmentmaybeeitherresource-drivenoraudience-driven.Bothapproaches

arevalid,butdetailedservicedevelopmentshouldtakeintoaccounttheintendedaudience

fortheservice.Theservicemustappealtothem,sotheirhabits,needsandexpectationswill

haveanimpactonthenatureoftheserviceandthewaysinwhichitispromoted.

2.2.2 Resource-driven service developmentmightoccurbecausesomeonehasagreatidea,or

seesthepotentialforincreasingtheavailabilityanduptakeofaresourcethroughdigitisation.

Thisapproachisentirelylegitimate,butthedevelopmentworkshouldstillbedependenton

identifyingapotentialtargetaudiencewhowillbeservedbythenewresourceandconsider

thelikelyrequirementsofthistargetaudience.

Consider this example….

Ifanarchiveistobedigitised,thedesignoftheservicewillbeinfluencedbythenatureofthetargetaudienceandtheway(s)inwhichitisenvisagedthisaudiencewillusetheresource.Forexample,thetargetaudiencewillinfluence:

■■ thedesignoftheinterface;

■■ thetypesofsearchesenabled;

■■ availabilityofmaterialfordownloadingandre-use;

■■ choiceandpositioningofmetadata.

Audienceanalysisworkduringdevelopmentofthedigitisedarchiveshouldprobablyincludesomeaudienceengagementworkaspartofuserneedsassessment.Designoftheserviceshouldalsoideallyincludeusabilitystudieswithpotentialuserstoensurethattheserviceworksasintendedinthehandsofrealworldusersandmeetstheirexpectations.

2.2.3 Audience-driven service developmenttakesatargetaudienceasthestartingpointfor

servicedevelopment.Anorganisationmightwanttoextenditsofferingtoattractanaudience

forwhomtheirtraditionalnon-digitalofferingsareknowntohavelimitedappeal;alternatively

anorganisationmightwanttobuildonthegoodrelationshipithaswithaparticularaudience

byintroducingnewdigitalresourcestailoredtotheirwishesandexpectations.

2.2.4 Again,understandingthetargetaudiencewillbecriticaltothesuccessoftheservice.

Audienceresearchwillfocusonfindingoutthesortsofresourcesandservicesthatappealto

thetargetaudience,howtheywouldprefertoaccesstheservice,thetechnologiestheyuse.

Backgroundknowledgeaboutattitudes,lifestyle,socialinteractionsetc.isalsolikelytobe

usefuliftheresourceistobeprimarilyforleisureuseratherthanformallearning.

2.3 Defining the target audience

2.3.1 Definingyourtargetaudiencedoesn’tneedtobeadifficulttechnicalexercise.Agood

startingpointmaybetopictureanidealaudiencemember–oratypical,actualaudience

member.Howwouldyoudescribethisperson?Thedescriptorsyouusearegoodstarting

pointsforthinkingaboutcategoriesforaudiencesegmentation.



2.3.2 Amoreformalapproachtodefiningthetargetaudiencewouldbetodevelopaseriesofuser

profilesorfullyworkedpersonasasanaidtoservicedesignanddevelopment.Negativeuser

descriptions,profilesorpersonascanalsobehelpful:thekind(s)ofpeopleyourserviceisnot

intendedfor.

2.3.3 GuidanceondevelopinguserprofilesandpersonascanbefoundinSection5.

2.4 Describing the target audience

2.4.1 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,the

functionalityprovided,meansofaccessetc.Thefeaturesincludedinadescriptionofthe

targetaudiencewillvaryaccordingtotheservicebutmayincluderesidentialaddress,

occupation,gender,interests,technologicalexpertiseetc.Atargetaudiencemaybe

veryinclusive(egallthepeopleinManchester)orhighlyspecific(eg qualifiedmedical

professionalswitharesearchinterestinOsgood-Schlatter’sdisease).

Audience segmentation2.4.2 Audiencesareoftensegmentedintodifferentcategories.Publisheddata(egtelevision

viewingfigures,newspaperreadership)oftensegmentstheaudienceonthebasisof

demographiccategories.Academiclibrariesaremorelikelytocategoriseusersaccording

totheirrolewithinacademia.Themostappropriatewaytodefineatargetaudienceand

segmenttheactualaudiencewilldependontheserviceandtheobjectivesoftheaudience

analysisworkandseveraldifferentapproachestosegmentationarepossible.Themore

personaldatayoutrytocollectaboutusersthemorelikelyusersaretorefuseinformationor

failtocompleteyoursurvey,soprioritisinginformationneedsisimportant.

Demographic segmentation

2.4.3 Thisisaverycommonwayofsegmentingaudienceswhichisusedinpopulationsurveys.

Inconsideringthecategoriestobeusedyoumaywanttobearinmindthatsomeofthe

audiencemaybereluctanttousecertaincategoriesandfailtosupplycategorydata,refuse

toprovidedataaltogetherorprovideincorrectdata.Demographicsegmentationmight

includesegmentationonthebasisofgender,age,educationallevel,income,postcode,

occupationetc.

Segmentation for accountability to funders

2.4.4 Ifaservicelevelagreementspecifiesaparticularpriorityaudience(egparentswithpre-

schoolchildren)thensegmentationshouldincludethisasacategory.Furthersegmentation

mayormaynotbeusefulegfathers,mothers,othercarersforpre-schoolchildren,

responsibleformale,femaleorbothsexesofchildren.Thisadditionalsegmentationmightbe

beneficialiftheserviceprovidersuspectedorknewthattheneedsorexpectationsofthese

sub-segmentsweredifferent,orthoughtthattheservicewasunderusedbyasub-segment.

2. Describing and Defining the Target Audience PAGE 13


Functional or task‑oriented segmentation

2.4.5 Thismayoftenbemoreusefulthandemographicsegmentationbutitcanbedifficult

todefinenon-overlappingcategories,ortodefinecategoriesclearlyandensurethat

respondentscategorisethemselvesasyouanticipate.Ideallyaudiencesegmentsshould

bedefinedunambiguously;descriptionsofcategorieswhichseemlong,complexoroverly

technicaltorespondentswillbeoff-putting.


Digitalacademicservicesmightchooseafunctionalschemebasedontheroleoftheuser.Forexample,anacademic researcher:

■■ formallearner,ieastudentenrolledonacourseleadingtoarecognisednationalHEqualification;

■■ independentlearner;

■■ leisureuser.

Butthisschemedoesn’tcapturewhatmaybeimportantdistinctions,forexample:

■■ maturestudentsvs.otherstudents;

■■ graduateresearchersvs.experiencedresearchers;

■■ casualleisureusersvs.committed,knowledgeableleisureusers.

2.4.6 Segmentationshouldcapturethemostrelevantdifferencesbetweenusers.Forexample

ifatheserviceisinterestedincomparisonsbetweenmaturestudentsandotherstudents

becauseitisbelieved(orthereisaudienceresearchtoshow)thatmaturestudentstendto

havelowerconfidenceorskilllevelswhenusingIT,itmightbemoreappropriatetocategorise

usersintermsofself-ratedITskills.Theserviceshouldalsoconsiderwhetheritmattersthat

differentuserswillhaveadifferentideaofwhatconstitutesproficiencyinITskills.

Objective‑oriented segmentation

2.4.7 Undersomecircumstancessegmentationonthebasisofusers’goal(s)orobjective(s)maybe

mostappropriate.Ifyouareinterestedinlearningaboutaudienceneedsandexpectations,

andhoweffectiveyourserviceisinmeetingthoseneeds,itwillbeimportanttoknowwhy

youraudienceusedtheservice,whattheirobjectivewasandhowsuccessfultheywerein

achievingit.


Object-orientedsegmentationfortheaudienceofamuseum–who is your museum website for?andwhat goals will your users have?

■■ usersplanningavisittothemuseum–seekingbasicinformationaboutopeninghoursandtravel;

■■ userswhohavevisitedthemuseum–wantingtolearnmoreaboutmuseumexhibitsfollowingtheirvisit;

■■ researchers–withaprofessionalinterestinthecollections.



Understanding non-users2.4.8 Understandingnon-useofaserviceisimportant:publiclyfundedservicesmayhave

aresponsibilitytoserviceaparticularaudience(egseniorcitizens,ethnicminority

communities).Wheredigitalservicesareanalternativeorcomplementaryserviceitmaybe

desirabletoensurethattheyreachasegmentoftheaudiencewhofailtousethepre-existing

non-digitalservice.

2.4.9 Highnumbersofsatisfiedusersmaynottellthewholestoryaboutaservice–partsof

thetargetaudiencemayhavebeenunabletoaccessthedigitalservice.Theirviewsare

unrepresentedinusersurveysandthedigitalserviceisnotmeetingtheirneeds;thismaybe

animportantfailing.

2.4.10 Segmentingnon-userscanbeuseful.Animportantdistinctioncanbedrawnbetween

non-userswhoaren’tawareoftheserviceandnon-userswhoareawareoftheservicebut

havechosennottouseit.Non-userswhoareawareoftheservicecouldbefurtherdivided

intoex-users(thosewhohavesampledtheserviceatleastonce)and‘neverusers’(those

whohavenotsampledtheservice).Whatisthereasonfornon-usage(eg tooexpensive,too

difficulttoaccess,happywithanalternative,notinterestedetc.).

2.4.11 Drawingdistinctionsbetweennon-usersimmediatelyraisesfurtherquestionswhichcanbea

usefulstartingpointforaddressingtheproblemofnon-usage:

■■ whyex-usersnolongerusetheservice;

■■ whetherlowlevelsofawarenessindicateaproblemwithdiscoverability;

■■ howaccuratenon-users’perceptionsoftheserviceofferingareetc.

3. Planning Audience Research PAGE 15

3.PlanningAudienceResearch

This chapter looks at defining research objectives, selecting and combining appropriate methods, researching non-usage and recruiting subjects

Research need not be perfect to be useful – the key to good audience analysis is setting out clear objectives and planning appropriate research to achieve the objectives. Engaging non-users is important but presents special problems.

Objectivescanbebroadlydividedintotwocategories:assessmentofreachanduptake,andassessmentofqualityandimpact.

Startingpointsforplanningresearchinclude:

■■ Definingservicedevelopmentobjectives.

■■ Definingquestionsaboutthenatureandbehaviouroftheaudienceandnon-users.

Differentaudienceresearchtechniquesaresuitedtodifferentobjectivesandmethodscanbecombined.

3.1 Introduction

3.1.1 Planningaudienceresearchshouldstartwiththebasicquestion:what do you want to know

about your audience?Consideringwhy the knowledge is neededandhow it will be put to use

isalsoimportant.Understandingtheaudienceisnotanendinitself,butanintegralpartof

developingandmaintainingasuccessfulandsustainableservice.Understandingthedrivers

forandusesofaudienceanalysisworkwillhelptoensurethatitisalignedwithwiderservice

visionandobjectivesandmayhelptoobtainbuy-infromseniormanagement.

3.2 Objectives of audience analysis

3.2.1 Thekeytogoodaudienceanalysisissettingoutclearobjectivesandplanningappropriate

researchactivitiestomeettheseobjectives.Thereislittlepointinspendingtime,moneyand

staffeffortcollectingdataunlessitisgoingtobeuseful.

3.2.2 Audienceanalysiscanbedividedintotwobroad,interdependentcategories:

■■ assessmentofreachanduptake;


PAGE 16 3. Planning Audience Research

■■ assessmentofqualityandimpact.

3.2.3 Inplanningaudienceresearchitishelpfultodecidehowimportanteachaspectofthe

analysisis.

3.2.4 Analysisofreachanduptakeshouldbeapriorityifthesizeand/orcompositionofthe

audienceisimportant.Forexample,arecentlyintroducedserviceneedstodemonstrate

aminimumaudienceinordertoreceivefurtherfunding.Clearly,ifasegmentofthetarget

audienceisunder-representedoraudiencesizeissmallerthanexpectedquestionsaboutthe

qualityoftheservicewillformpartofthefutureinvestigationsofnon-usage.

3.2.5 Analysisofaudiencesatisfactionmeasureswillbeapriorityifmaintainingorenhancing

theserviceisimportantandtherearefewconcernsaboutthesizeandcompositionofthe

audience,egservicesforaclearlydefined,specialistaudiencewherethereisnodirect

competition.

3.3 Starting points for defining audience research objectives

3.3.1 Itisbeyondthescopeofthisguide,oranyother,toprovideaready-madelistofresearch

objectivesasthesemustbesetaccordingtoindividualserviceprioritiesandcontext.This

sectionprovidessomestartingpointsforidentifyingbroadaimsanddefiningmorespecific

objectives.

Service development objectives can drive audience analysis3.3.2 Oftentheservicedevelopmentcontextprovidesagoodstartingpointforframingquestions:

■■ Howcanweextendthereachofourservice?

■■ Howcanweenhancetheserviceforexistingusers?

■■ Howcanweencourageregularusersofourservicetobecomeregisteredusers?

■■ Howcanweappealtoalargeraudience?

■■ Whatadditionalcontent/features/serviceswouldouraudienceappreciate?

■■ Whatdoesouraudienceexpectfromourservice?

■■ Doesaudienceperceptionoftheservicematchwhatweareattemptingtoprovide?

■■ Isanypartofourserviceobsoleteorredundant?

■■ Isourredesignedserviceasuccessiehavetheobjectivesoftheredesignbeenmet?




Broadservicedevelopmentobjectivessuchas‘howcanweenhancetheserviceforexistingusers?’canbeusedtodefinespecificaudienceresearchquestionswhichwillprovidetheevidencerelevanttothebroaderquestions.

Specificresearchquestionstohelpanswer‘howcanweenhancetheserviceforexistingusers?’include:

■■ Howdousersratetheexistingservice?

■■ Aretheyawareofallthefeaturesoftheservice?

■■ Whatdotheythinkisbestabouttheservice?

■■ Whatdotheythinkcouldbeimproved?

■■ Whatadditionalcontentorfunctionalitywouldtheyuse?

■■ Whichaspectoftheserviceismostimportanttothem?etc.

Questions about the audience can drive audience analysis3.3.3 Allaudienceanalysisaimstoimproveunderstandingoftheaudience,bothserviceusersand

non-users.Thinkingaboutyouraudienceandwhatyouneedtoknowaboutthemprovides

analternativestartingpoint.Thesequestions(seealsoFigure3-1)arenotintendedasa

definitiveorprescriptiveset,butasastimulusfordiscussion.

Who are our audience?■■ demographics,egage,gender,socioeconomiccategories,geographicallocation,levelof

educationetc.;

■■ functional,egprofessional/researcher/otherspecialist,formallearner,independentlearner,

leisureuser,accidental/impulseuser,maturelearneretc.;

■■ goalorientationegdefinedobjective,browser,systematicexplorer,impulse/accidental

useetc.;

■■ preferredplatformetc.;

■■ domainexpertiseegforusersofElectronicBeowulf,howmuchtheyknowaboutBeowulf,

linguisticsandAnglo-Saxonliterature;

■■ platformortechnologicalexpertiseieexpertise,confidenceandfamiliaritywiththe

platformsandtechnologyfromwhichaserviceorresourcecanbeaccessed.

How do they use our service?■■ meansofaccess;

■■ frequencyofusageegregular,repeated,occasional,one-off;

■■ whattheyuseitfor;

■■ areanyalternativeservicesusedforthesimilarpurposes?

Why do they use our service?■■ inpreferencetoanon-digitalservice?



■■ isthereanalternative?

■■ quality;authority;comprehensiveness;convenience(speedandeaseofaccess);social

cache.

Does the service meet their needs and expectations?■■ expectations–includingexpectationsofextensionsandenhancement;

■■ needs;

■■ importance;

■■ quality:strengths,weaknesses,suggestedimprovements;

■■ reliability;

■■ authority;

■■ comprehensiveness;

■■ convenience.

How do users discover/reach our services?■■ howoriginallydiscoveredegsearchengine,referringsite,offlinereferral,recommendation

(who?),othermedia(advertisement?),event;

■■ howreachedinsubsequentusage.



Figure 3‑1questions about the audience

WH

O a

re t

he

y?

EX

IST

ING

AU

DIE

NC

E

WH

Y d

o t

he

y u

se

ou

r s

erv

ice

?

WH

O i

s n

ot

us

ing

ou

r s

erv

ice

?

DO

ES

th

e s

erv

ice

me

et

the

ir n

ee

ds

an

d e

xp

ecta

tio

ns

?

Ac

ce

ss

Ho

me

Pu

blic

Mo

bile

Fre

qu

en

cy

Re

gu

lar

Oc

ca

sio

nal

On

e-o

ff

Co

mp

eti

tors

Pu

bli

c

Pri

va

te

Inte

rna

tio

na

l

Wh

at

for

Co

nta

cts

Co

llec

tio

ns

Ho

w t

o f

ind

us

Fir

st

us

e

Se

arc

h e

ng

ine

Re

co

mm

en

dati

on

Refe

rrin

g s

ite

Su

bs

eq

ue

nt

us

e

Dir

ec

t

Se

arc

h e

ng

ine

Refe

rrin

g s

ite

Dir

ec

t

Re

as

on

Qu

ality

Au

tho

rity

Co

mp

reh

en

siv

e

Cla

rity

No

alt

ern

ati

ve

s

Alt

ern

ati

ve

s

Dig

ital

No

n-d

igit

al

De

mo

gra

ph

icF

un

cti

on

Go

al

Te

ch

no

log

y

Do

ma

in

Ag

e

Ge

nd

er

So

cio

eco

no

mic

cate

go

ry

Lo

ca

tio

n

Pro

fes

sio

nal

Re

se

arc

her

Ac

cid

en

tal/

imp

uls

e

Leis

ure

Bro

ws

ing

De

fin

ed

ob

jecti

ve

Sys

tem

ati

c

ex

plo

rer

Ex

pert

Co

nfi

de

nt

No

vic

e

Ex

pert

So

me

kn

ow

led

ge

No

kn

ow

led

ge

HO

W d

o t

he

y u

se

ou

r s

erv

ice?H

OW

do

th

ey

rea

ch

ou

r

se

rvic

e?

Se

pa

rate

dia

gra

mA

ud

ien

ce

sa

tis

fac

tio

n

WH

AT

are

th

e

lon

ge

r te

rm e

ffe

cts

of

the

se

rvic

e?

Imp

ac

t a

naly

sis

Qu

ality

Relia

bilit

y

Co

nve

nie

nc

e

Op

inio

n

Imp

ort

an

ce



3.3.4 Manyofthequestionsaboutusersarealsorelevanttonon-users,butsomeissuesare

particularlyrelevantwhenconsideringnon-users(seealsoFigure3-2).

Who are they?

■■ whichsegmentsoftargetaudienceareunder-represented?

■■ whichsegmentsofthegeneralpotentialaudienceareunder-represented?(anddoesthis

matter?)

Why don’t they use our service?

■■ awareness:

■− dotheyknowitexists?

■− weretheyabletodiscoverit?

■− weretheyabletoacessit?

■− havetheytriedourservice?

■■ Perceptions:

■− whatdotheythinktheserviceoffersandhowdotheyrateitforquality,valuefor

moneyetc.?;

■− whodotheythinkusestheservice?

■− howcouldtheybepersuadedtousetheservice?

What alternatives to our service do they use?

Why are these alternative services preferred?

■■ speed,easeofaccess,quality,authority,comprehensivecoverageetc.

How well do these alternative services meet their needs and expectations?



Figure 3‑2understanding non-users

Usage

Ex-users

Never

NON-USERS

WHY are they not

using our

service?

Awareness

Not aware

Aware

Reason

Not interested

Happy with

alternative

Too slow

WHY do they no

longer use our

service?

WHO are they?

Which segments of the target

audience are under-represented ?

Target audience – existing audience

= non-users

WHAT did they

think of our

service?

HOW could we

make them more

aware?

HAVE they ever

used our service?

WHY do they not

use our service?

Reason

Not interested

Happy with

alternative

Poor public

imageWHY are the

alternatives

preferred?

Reason

Better quality

Better coverage

Easier to

access

Audience satisfaction

Aware but

cannot accessWHY can they not

access it?

Reason

Technological

Authentication

required

Disabilities

Marketing



3.4 Choosing the methodology

Different approaches to audience analysis3.4.1 Audienceanalysismethodscanbecategorisedindifferentways,butthedistinctionbetween

quantitativeandqualitativemethodsisanimportantone.Quantitativemethodsareusually

moreappropriatewhenstatisticallyvalidconclusionsneedtobedrawnaboutthesize,

compositionandotherpropertiesofanaudience.Qualitatitivetechniquesmaybemore

appropriateforexploringattitudesandmotivationsindepth,discoveringpatternsofuser

behaviouranddevelopingarelationshipwithaudiencemembers.

Qualitative data

3.4.2 Smallsamplestendtobeusedandfindingscannotbetreatedasastatisticallyreliable

measureoftheaudienceasawhole.Howeverqualitativeresearchcanbeinvaluablein

exploringthecomplexitiesunderlyingaudiencebehaviour,helpingtoexplainwhyaudience

membersbehaveinthewaythattheydo.

3.4.3 Examplesofmethodsproducingqualitativedatainclude:

■■ focusgroups;

■■ one-to-one,in-depthinterviews.

3.4.4 Incertaincircumstances,informationcollectedviaaquantitativesurveycanbequalitativein

nature,egfreetextresponsestoopenquestions.

Ethnographic or observational data

3.4.5 Awidevarietyofdirectandindirectobservationaltechniquesareincreasinglyusedfor

studyingusabilityandpatternsofserviceuse.Ethnographicdataisoftenverydetailedand

mayincludecomputercollecteddatawhichisamenabletostatisticalanalysisaswellasa

qualitativerecordofuserbehaviour.

3.4.6 Examplesofmethodsproducingethnographicdatainclude:

■■ diarystudies;

■■ usergeneratingscenarios;

■■ task-focusedscenarios;

■■ accompaniedbrowsing.



Quantitative data

3.4.7 Quantitativedataprovidesinformationaboutquantitiesandfrequencies.Non-numericaldata

canbecodedorcategorisedtorenderitamenabletoquantitativeanalyticaltechniques.

Quantitativedataisgenerallycollectedfromasamplelargeenoughtopermitstatistical

analysis.Conclusionsfromstatisticalanalysiswillusuallybestatedwithadegreeof

confidence.5

3.4.8 Examplesofmethodsproducingquantitativedata:

■■ telephoneandface-to-facesurveys;

■■ postalsurveys;

■■ emailsurveys;

■■ onlinesurveys;

■■ webanalytics.

Representative data3.4.9 Samplesaresaidtoberepresentativewhenthecompositionofthesampleaudience–with

respecttoallrelevantattributes–isthesameasthatoftheaudienceasawhole.Insome

instancesitisimportanttoobtaindatafromarepresentativesampleoftheaudienceegto

assessthelevelofuptake,toprovideevidenceforattainmentofkeyperformanceindicators

(butseealsosub-section6.8.4).

3.4.10 Usingrepresentativesamplesdemandsaknowledgeofthefrequencyofdifferentsegments

inthetargetoractualaudience.Assessingtherepresentativenessofasamplerequires

informationaboutthesegmenttowhichsubjectsbelong.Itissometimesassumedthat

asampleisrepresentativeifthereisnoobviousevidenceofbias,orifthesamplesizeis

large.Unexpectedlylowresponseratesareapotentialindicationofsamplebiasandshould

generallybeinvestigated.

Selecting appropriate methods3.4.11 Thekindofinformationyouwantabouttheaudienceshouldhelptodeterminethemethods

youusebutpragmaticconsiderations(egtime,money,difficultyofrecruitingsubjects,staff

availabilityandexpertiserequired)willinevitablyplayapart.

3.4.12 Table3-1providesaguidetothemethodsmostlikelytobeusefulinresearchingdifferent

aspectsoftheaudience.

5 For example: based on this sample, 95% of the audience as a whole will fall between these limits; or, as a probability value expressing the likelihood of obtaining the same result by chance.



Table 3‑1Selecting appropriate methods (Key: – unlikely to be suitable; * may be suitable, but of limited use; ** often suitable)

Asp

ect

of

aud

ienc

e

Qua

ntit

ativ

e-su

rvey

Web

sta

tist

ics

Eth

nog

rap

hic

stud

ies

Focu

s g

roup

s

Ind

ivid

ual

inte

rvie

w

Info

rmal

Oth

er

app

roac

hes

Size ** ** - - - - -

Composition ** *moredetaileddataaboutregisteredusers

- - - * -

Motivations ** * - ** ** * -

Attitudes ** * - ** ** - -

Satisfaction ** * - ** ** - -

Prevalence of non-usage

** - - * - -

Reasons for non-usage

- - - ** ** - -

Patterns of usage - ** ** * * - -

Tasks service is used for

** ** **usertracking

** ** - -

Lifestyle * - ** * ** - -

Service usability * * ** * * - Heuristicevaluation6

User requirements - - ** ** ** * -

Combining methods in audience analysis63.4.13 Itiscommontouseacombinationofmethodsinaudienceanalysisprojects:thismakes

senseforanumberofreasons:

■■ differentsegmentsoftheaudiencemayberecruitedwithdifferentmethods;

■■ differentmethodshavedifferentadvantagesanddisadvantages;

■■ differentmethodsprovidedataondifferentaspectsoftheaudienceandtheirbehaviour.

3.4.14 Inplanninganaudienceanalysisprojectbearinmindhowdatacollectedearlyoninthe

projectcanbeusedtoadjustandimproveplansforthelaterresearch.Itiscommon

tocombinequantitativesurveyswithfocusgrouporinterviewwork.Whichshould

comefirst?Responsestoasurveymayraiseissuesthathadn’tbeenanticipatedand

6 See Section 4.4



suggestappropriatequestionsandthemesformoredetailedexplorationinfocusgroups.

Alternatively,youmayprefertocarryoutasmallnumberofinterviewsorafocusgroupand

distilkeyfindingssothatvaliditycanbetestedwitharepresentativequantitativesurvey.

Talkingtoaudiencemembersindividuallyoringroupsmightsuggesttopicsforinclusionina

survey.

3.4.15 Ifethnographicworkistobecombinedwithanintervieworshortsurveyquestionnairethe

samesubjectscouldbeusedforbothelementsoftheresearch.Itmightbehelpfultofollow

anobservationsessionwithaninterviewsothatyoucandiscusswhatwasobservedandask

aboutgoodandbadaspectsoftheuserexperience.Ontheotherhanditmaybeharderto

recruitusersforastudyrequiringalongertimecommitment.Youcouldaskintervieweesor

surveyrespondentsiftheywouldbewillingtoparticipateinfurtherresearch.Thentheorder

inwhichuserscompletethedifferentelementsofthestudymaybecriticalasyourquestions

mightinfluencetheuser’sbehaviourduringtheethnographicstudyortheuser’sexperience

oftheserviceduringtheethnographicsessionmayaffecttheirattitudeandratingofthe

servicewhenthey’reansweringsurveyquestionsorbeinginterviewed.Theseareissues

whichneedtobeconsideredinplanningtheresearch.

3.4.16 Anotherconsiderationisthedurationoftheresearch:

■■ one-off projectstoprovideasnapshotofanaudienceatasinglepointintime;

■■ repeated surveysieundertakingsimilarsurveysonaregularbasistomonitortrendsor

assesstheimpactofchange;

■■ longitudinal researchiethesamesampleofaudiencemembersistrackedoveraperiod

oftime(egtomonitortheimpactofdevelopmentactivities);avarietyofmeasuresmaybe

used.

3.5 Researching non-usage

The importance of research on non-users 3.5.1 Engagingwithnon-usersmaybedifficult,time-consumingandpotentiallyexpensive,

butitshouldbeviewedasanecessarypartofaudiencedevelopmentwork.Buildingup

relationshipswithnon-userswillpaydividends:

■■ non-userscanprovidevaluableinsightsforservicedevelopment;

■■ engagingnon-userscanhelpdemonstrateacommitmenttothataudiencesegmentand

increaseknowledgeaboutmarketingandcommunicationstrategiesthatwillbeeffective

forthatsegment.

Identifying non-users3.5.2 Reliabledataaboutactualusageofaservicecanbecomparedwithanaccuratedescription

ofthetargetaudiencetoderiveapictureofnon-usage,butthispictureofnon-usagewillonly

beasreliableasthedataaboutusersandtargetaudiencefromwhichitisderived.



3.5.3 Descriptionsofthetargetaudiencewilloftenbeinsufficientlydetailedegnoinformation

availableaboutthefrequencyorproportionofthetargetsegmenteitherbecausefunctional

segmentationhasbeenchosen,orbecauseappropriatedemographicdataisnotavailable.

Evenifagooddescriptionofthetargetaudienceisavailable,obtainingagoodestimate

ofthenumberandnatureofusersmaybedifficult,particularlyfordigitalservices(multiple

usersforoneIPaddress,thesameuserloggingonatwork,athome,inapublicsettingetc.).

Professionalexpertiseandinformalaudienceresearchmaybethebestcluestopatternsof

non-usage:thesecanbetestedandfollowedupwithmoreformalresearch.

Investigating non-usage

Reasons for non‑usage

3.5.4 Responsestoasurveymaynotprovideverymuchinformation.Engagingmoredeeplywith

non-usersthroughfocusgroups,discussionforaandinterviewingwillbenecessarytobuild

uparicher,moremeaningfulunderstandingofnon-usage.

Recruiting non‑users

3.5.5 Reachingnon-userscanbeproblematic–theiropinions,attitudesandhabitsarenot

representedinserviceusagestatisticsorsurveys.Peoplearelesslikelytobewillingtospend

timeansweringquestionsaboutaserviceorresourcethattheydon’tusethanonetheyvalue

highly.

3.5.6 Onewayofgaininganinsightintohowwidelyyourserviceisusedistoquestionthewholeof

yourtargetaudiencethroughawidelydistributedsurvey.Dependingonthesizeofyourtarget

audienceandhoweasilyyouareabletocontactthemthismaybedifficultandexpensive.

Possiblestrategiesinclude:

■■ Addingyourquestionstoasurveybeingdistributedto,orcarriedouton,thegeneral

population–marketresearchservicescarryoutbroadsurveysandlocalgovernments

mayalsoattempttosurveytheirelectorate.So-calledOmnibussurveyscanbeavery

cheapwayofsurveyinglargepopulations.Theyareaccessedonapay-per-questionbasis

soacoupleofwellconstructedandclearlyfocusedquestionscanbeansweredrelatively

cheaply(forinstance,relatingtoawarenessofaserviceandkeybarrierstouse).

■■ Carryingoutamoregeneralsurveyyourselforpayingamarketresearchcompanytodo

soonyourbehalf.

■■ Targetingaudiencesegmentsyoususpectareunder-representedbychoosingmethods

knowntobeeffectiveinreachingthatsegmentegchoiceofaparticularmedium,buying

acontactlistfromarelevantorganisation,enlistingthehelpofanorganisationwhich

representsorworkswiththesegmentofinterest.

3.5.7 Itmaybehelpfultoenlistthehelpoforganisationswhohavethetrustandconfidenceof

yournon-users.Non-usersarelikelytobemorewillingtoengageifyouareabletoconvince

themthattheirparticipationmayhaveapay-off,thattheirvoicewillbelistenedtoandtheir

commentsmayinfluenceserviceprovisioninthefuture.Directincentivesforparticipationare

usuallyofferedaswell.



3.6 Recruiting subjects for audience research

3.6.1 Recruitingsubjectsisaperennialproblemandtheeasiesttechniquemaynotalwaysbethe

best.Recruitingcommittedmembersoftheaudiencetotakepartinresearchtendstobe

easiestandthisisonepotentialcauseofanunrepresentative,biasedsample.Recruitment

willbemoredifficultifabigtimecommitmentisrequiredortheresearchisgoingtobe

inconvenientforsubjects.Incentivescanbeusedtoaidrecruitment,butcareneedstobe

takentoensurethattheincentiveisappropriatetotheaudienceyouseektorecruit.

Recruiting for quantitative research

3.6.2 Surveyscanbedistributedinanumberofwaysanditispossibletocombinemethodsto

improveresponserate.Eachmethodhasadvantagesanddisadvantages.Surveyscanbe:

■■ placedonawebsite;

■■ emailedtoknownusers,thewholetargetaudienceorasampleofthetargetaudience;

■■ posted;

■■ administeredbyaninterviewerbytelephoneorinperson.

3.6.3 Placingaquestionnaireonawebsiterequiresminimaladministrativeeffort,doesnot

requirecontactdetailsandischeap,butcombiningtheresponsedatawithwebstatisticsis

necessarytoobtainanestimateofresponserate.Responseratestendtobelowandthere

isnowayofcontrollingresponsebiasinthesample;sometypesofuseraremorelikelyto

respondthanothers.

3.6.4 Ifthecompositionoftheaudiencesampleisimportantthensurveysadministeredby

interviewermaybemostappropriateasitispossibletosetquotasforparticularaudience

segments.

3.6.5 Informationaboutresponsebias–systematicdifferencesbetweenrespondentsandnon-

respondents–maybeobtainedifanattemptismadetocontactnon-respondentsviaan

alternativemethodanddiscovertheirreasonsfornotresponding.Thismaybeattemptedifit

isparticularlyimportanttohavearepresentativesampleorwheretheaimwastosurveythe

completeaudience.

3.6.6 Surveydesigncanbeanimportantfactorinresponserateanddrop-outrate(theproportion

ofrespondentswhoceaserespondingpartofthewaythroughthesurvey).Higherresponse

ratesandlowerdrop-outaremorelikelyif:

■■ surveysareshort;

■■ questionsandpossibleresponsesareeasytounderstand;

■■ itispossibletosaveapartiallycompletedsurveyandreturnlater;

■■ surveynavigationiseasyandthesurveyisveryusable;

■■ questionsarevariedintheirstyle,formatandsubject(forexample,avoidrepeatedusage

ofbanksofsimilarlywordedandvisuallyundifferentiatedattitudestatements).



Recruiting for qualitative and ethnographic research

3.6.7 Recruitingsubjectsforqualitativeresearchpresentsslightlydifferentproblems.Thetime

commitmentrequiredistypicallygreaterandifparticipationintheresearchislimitedto

membersoftheaudiencewhoareavailableduringworkinghoursitwillbedifficulttoobtain

arepresentativeaudiencesample.Recruitmentislikelytobemoresuccessfulifpotential

subjectsarecontactedinperson.

Recruiting registered users

3.6.8 Digitalservicesmayofferusersincentivestoregister;buildingarelationshipwithatleast

partoftheaudienceinthiswaycanpaydividendsforaudienceresearch.Registeredusers

maybemorefrequent,moreloyalormorecommittedtotheserviceandthereforemore

willingtoparticipateinresearch.Ofcoursetheimplicationsofdifferencesbetweenregistered

usersandtherestofthecurrentandpotentialaudienceshouldbeborneinmind,butata

timewhenmoreandmoreorganisationsandaudienceresearchprofessionalsarereporting

‘surveyfatigue’amongstaudiencesthevalueofhavinganeasytorecruitaudiencesample

foraudienceanalysisworkshouldnotbeunderestimated.

3.6.9 Anextensionofthisideaistosetupresearchpanels.Theseconsistofasampleofaudience

memberswhohavebeenrecruitedspecificallyforthepurposeofregularly(ieonanongoing

basis)respondingtosurveys–qualitativeand/orquantitative.Theadvantageofthisapproach

isthatresearchcanbeorganisedrelativelyquicklyandinexpensivelysincethesampleis

alreadyinplace.Insomeinstances,priorknowledgeofandinvolvementinpreviousaudience

researchprojectscanbeadvantageous–althoughtherearealsopotentialdangersrelating

tolackofobjectivity.Familiaritywithissuescanhelpto‘fast-track’panelmemberstoalevel

ofthinkingaboutaservicewhichultimatelyresultsinaddedinsight.Inthepublicsector,

CitizensPanelshavebecomepopularwithlocalauthoritiesandhealthcaretrustsinparticular

butalsotoanextentwithavarietyofcommunityplanningorganisations.Theseare,ineffect,

largescaleresearchpanels.

3.7 Other sources of audience data

Using other people’s data3.7.1 Don’tneglectthepossibilityofincreasingyourunderstandingofyouraudiencebylookingat

researchdonebyotherserviceprovidersontheiraudienceorbyusingdatafrompopulation

surveyscarriedoutbymarketresearchorganisations,governmentagenciesoracademic

bodiesegtheOxfordInternetInstitutebiannualsurveyofinternetusage.7

3.7.2 Ifyouknow–orevenifyoujustsuspect–thatyouraudienceissimilartothatofanother

serviceyoucoulduseinformationaboutthataudience’sattitudesandpreferencesto

informyourownservicedevelopment.Ifyouknowthecompositionofyouraudiencethen

populationsurveyswiththerequiredsegmentationcanprovideausefulguidetoaspects

7 Oxford Internet Institute surveys www.oii.ox.ac.uk/microsites/oxis

http://www.oii.ox.ac.uk/microsites/oxis



ofyouraudienceegdataonmediaconsumptioncouldbeusedformarketingpurposes,

lifestyle-relatedquestionsmightprovidesuggestionsforserviceenhancement.Personasor

userprofilesfromasimilarservicemightbeusefulinservicedevelopmentwork.

3.7.3 Arelatedpointisthattheaudienceforthenon-digitalversionofaservice,whetherornot

theyoverlapwiththeactualortargetaudienceforthedigitalservice,maybeagoodsource

ofdataandthisaudiencemaybeeasiertorecruitforaudienceresearch:alternativesurvey

methodscanbeused,interviewerscanapproachvisitorsface-to-faceetc.

Informal sources of audience data3.7.4 Exploitinginformalknowledgeaboutyouraudienceisasmartthingtodoandcanbe

particularlyvaluableifresourcesforaudienceanalysisarelimited.Thesesourcesmight

include:

■■ callstoahelpdesk;

■■ emailstodigitalservicetechnicalsupport;

■■ observationsbystaffwhointeractdirectlywiththeaudience;

■■ unsolicitedcommentsorsuggestionsfromyouraudience.

3.7.5 Themainproblemisfindingawaytorecordandcollatethesedatasothattheycanbeused

inplanningformalresearch.

3.8 Audience research does not need to be perfect to be useful

3.8.1 Afrequentlyexpressedconcernisthatapieceofaudienceresearchwon’tbeanyuse

becauseitisn’tgoodenough.Someknowledgeabouttheaudienceisverymuchbetterthan

noneatallandtheseconcernsshouldnotdeteryoufromdoingaudienceanalysis.

Representative data are not always vital3.8.2 Acommonconcerninplanningaudienceresearchisthatanaudiencesamplewillbetoo

smallandnotnecessarilyrepresentativeoftheaudienceasawholeorthetargetaudienceas

awhole,butthismaynotalwaysmatter.

3.8.3 Itismore importanttoensurethatyouraudiencesampleiscompleteorrepresentative

when:

■■ reachanduptakearebeingassessedrelativetoaspecifictargetaudience;

■■ bigdecisionsaboutcommitmentofresourcesaredependentonthefindings;

■■ theserviceshouldbeuniversaliereach100%oftargetedaudiencesegments;

■■ findingswillbeusedasevidenceforaccountabilitypurposesegattainmentofkey

performanceindicators.

3.8.4 Itisless importantthatthesampleisrepresentativewhen:



■■ researchispreliminaryandexploratory;

■■ theaudienceishomogeneous;

■■ variabilityintheaudiencecharacteristicsorbehaviourunderinvestigationislow;

■■ researchwillinformservicedevelopmentforasubsetofthegeneralaudience;

■■ satisfyingacommittedaudienceismoreimportantthankeepingorrecruitingalarge

audience;

■■ alightweight,shortanalysisofamorerepresentativesamplecanbecarriedoutasa

follow-up(egself-completionsurveyratherthanone-to-oneinterviews;webstatistics

provideevidenceonusageofanewfeature;ashortersurveyforwiderdistributionfollows

upimportantorcontentiousissuesraisedbyadetailedsurveyofsmallsample).

4. Collecting Audience Data PAGE 31

4.CollectingAudienceData

This chapter provides an overview of different methods for collecting audience data and considers outsourcing audience research

Theprosandconsofconducingresearchin-houseandcommissioninganindependentmarketresearchagencytoundertakeitonyourbehalfarediscussed.

Datacollectionmethodscoveredinclude:

■■ Quantitativesurveys.

■■ Interviews.

■■ Focusgroups.

■■ Webstatistics.

■■ Ethnographic(observational)techniques.

Otherapproachestobuildingunderstandingoftheaudiencearebrieflydiscussed.

4.1 Undertaking research: in-house or third party?

4.1.1 Technologicaladvancesmakeitincreasinglytemptingtoconductresearchona‘DIY’basis,

ratherthancommissioninganindependentmarketresearchagencytoundertakeitonyour

behalf.However,itisworthconsideringanumberoffactorsbeforedecidingifthein-houseor

commissionedresearchrouteismostappropriate:


PAGE 32 4. Collecting Audience Data

Table 4‑1In-house research

Approach Pros Cons

In-house research

■■ Canoftenbetheleastexpensiveoption–especiallyifexpertin-houseresourcesareavailable.

■■ Knowledgeoftheaudience:expertknowledgeand/orexperiencecanhelpinplanningandconductingresearch.

■■ Greaterlevelofcontrol:amarketresearchagencymighthavedifferentideasabouthowtoresearchanaudience.

■■ Lackofexperienceinconductingresearchcanleadtopoorresearchdesignandotherproblemsthatanexperiencedresearchagencywouldhaveanticipatedandplannedfor.

■■ Questionnairedesignissurprisinglydifficulttogetright.Poorlydesignedquestionsresultinambiguousormisleadingdata.Experienceofquestionnairedesignisimportant.

■■ Similarly,unlessknowledgeofstatisticalandsamplingtechniquesisavailablein-house,mistakesarelikelytobemade–bothattheplanningandattheanalysisstages.

Table 4‑2Commissioned research

Approach Pros Cons

Commission market research company

■■ Ifcarefullyselected,aresearchagencywillbringexpertiseandexperiencetobear,resultinginamoreefficient(possiblyevenmorecost-effective)surveyandgreaterclarityofunderstanding.

■■ Theinvolvementofarespected,independentthird-partybringscredibilitytotheresearch.Thiscanhelpinachievingseniormanagementbuy-intothefindings;insomecircumstancesfundingbodiesmightlookmorefavourablyonacasefoundedonindependently-conductedaudienceresearch.

■■ Respondentsaregenerallymorelikelytorespondopenlyandhonestlytoathird-partythantheywouldiftheservice-providerasksthequestionsdirectly–especiallyiftheyarecriticalofaspectsoftheservicebutprefernottodamagetherelationship.

■■ Marketresearchagencieshaveaccesstospecialistresourcesrequiredtoconductresearch.Theseinclude:

■■ InterviewersandfocusgrouprecruitersinallareasoftheUK.

■■ Specialistquestionnairedesignandsurveyanalysissoftware.

■■ Commissionedresearchcanbeexpensive.

■■ Theprocessofidentifyingpotentialresearchagencies,writingaresearchbriefandbriefingtheselectedagencytakestime.

■■ Evenmarketresearchagenciesexperiencedinyoursectorwilllackyourdetailedknowledge.Bepreparedtospendtimeensuringthattheagencyunderstandsthenatureofyouraudienceandwhatyouaretryingtofindoutaboutit.



How do you decide which market research company to use?4.1.2 ‘TheResearchBuyer’sGuide’,8publishedbytheMarketResearchSociety,hassomeuseful

information,bothonthetypeofmarketresearchcompanytochoosefromandonthe

questionstobearinmindwhenselectingacompany.

4.1.3 Itincludesadviceonwhethertosub-contractsomeelementsofaresearchprojectto

consultants(eg quantitativeinterviewingorrecruitmentoffocusgroups),orbuy-inthe

fullservice(whereprofessionalswillhelptodesigntheresearchandcollect,analyseand

evaluatedata).Otheroptionsaretohireagenciestodofieldwork(theinterviewingprocess)

andtabulation(producingdatatablesfromasurvey)–thesetypesofagenciestypicallyhave

ateamofface-to-faceinterviewersand/oratelephoneinterviewingcentre,togetherwith

datacodingandprocessingresources.Therearealsodataprocessingagencieswhichdo

notundertakeinterviewingbutcanprovidecomputerdataprocessingandanalysisresources

andexpertise.

4.1.4 Whenselectinganagency,itwouldbeusefultoconsiderthefollowingquestions:

■■ Whichcompanybestunderstandsyourneedsandprojectbackground?

■■ Whichcompanyhasaddedtoyourthinkingabouthowresearchshouldbeplanned?

■■ Hasthecompanychallengedsuggestedapproaches,highlightingpossibledifficultiesand

solutions?

■■ Doesthecompanyhaverelevantexperience,intermsofindustry,sector,audience,

subject-matterandmethodology?

■■ Doestheapproachsuggestedbythecompanyoffervalueformoney?

■■ Wouldthecompanybecrediblepresentingtoseniormanagementandotherinternaland

externalstakeholders?

■■ Willexperiencedstaffbeallocatedtotheproject(andnotjustappearatthebeginningand

endofaproject)?

■■ Canyouformaproductiveworkingrelationshipwiththeresearchcompany’steam?

4.2 Overview of data collection methods

4.2.1 Informationondifferentmethodsofcollectingaudiencedataissummarisedinthefollowing

tables,providingpracticalguidanceforusingthesemethods.Thesetofmethodscoveredis

notintendedtobedefinitiveanddetailedinformationonhowtoimplementeachmethodis

notprovided.

4.2.2 Mostaudienceanalysismethodscanbeusedoradaptedsuccessfullybynon-specialists;

however,therearesomecircumstancesunderwhichitmaybeappropriatetoconsider

outsourcingoneormoreelementsofanaudienceanalysisproject.

8 Research Buyer’s Guide, UK and Ireland (2007); London: Market Research Society.



Web statistics4.2.3 Providingresourcesonlinebringsnewopportunitiestomeasuretheirusage,andto

understandthewaysthatusersinteractwiththem.Inthisdocument,wehavedividedthis

intotwomethods–‘traditional’webstatisticsandwebanalytics.Thisisasomewhatartificial

boundary,butonethatisnecessarytoconsiderthefullrangeofopportunities.Traditional

webstatisticsconsidersbulkinformationonaresource(ielooksatusagefromtheservice’s

perspective),whereaswebanalyticsconsiderstheactionsandpathwayschosenby

individualusers(solooksatusagefromauser’sperspective).

4.2.4 Boththesemethodsarequantitative,butthenumbersgeneratedmustbetreatedwith

care:avastrangeofstatisticsisavailable,buttounderstandwhicharemeaningfulrequires

acombinationoftechnicalandbusinessunderstanding.Forcommercialsites,thereis

oftenacleargoalforusers–aconcludedsale.Akeymeasureformanycommercialsites

istheconversionrate–theproportionofvisitorswhomakeapurchase.Thereisnodirect

equivalentforaresourcewhichisfreeatthepointofuse,anddrawinginsightfromthe

numbersrequirescreativeanalyticalthinkingandcarefulvalidation.

Registered users4.2.5 Increasinglybothdigitalandnon-digitalservicesmayrequireorpermitusersto‘register’

insomeway.Thisallowstheservicetocollectinformationaboutusers–subjecttothe

provisionsofthedataprotectionlegislation.Userregistrationcanbeexploitedeasily

bydigitalservicesforaudienceanalysis:requiringuserstoauthenticateallowsuser

characteristicstobeassociatedwithuserbehaviour.Althoughthiscanbeapowerful

analyticaltechniquetherearepotentialdrawbacks:

■■ unlessusersareauthenticatedbyatrustedauthoritythedatatheyprovideabout

themselvesmaynotbeaccurate;

■■ itisunlikelythatregisteredusersarerepresentativeoftheaudienceasawholeeg

theymaymakemorefrequentormorein-depthuseoftheservice,theymaybemore

committedtotheservice,theymaybemorelikelytobeexpertsinthedomain;

■■ requiringregistrationmaydetersomeofthepotentialaudience.



Data collection methods

Table 4‑3Focus groups

Focus groups

Description Focusgroupscanbeorganisedinmanywaysandinmanyformats.However,allformatsarebasedonthepremiseofasmallgroupofrespondentsdiscussingissuesguidedbyanexperiencedfacilitator(sometimescalledamoderator).

Uses Usefulforexploringattitudesandmotivationsandforgeneratingnewideas.Focusgroupsareoftenusedastheprecursortosomeformofquantitativesurveyresearch.

Outline of format ■■ ‘Standard’focusgroupstypicallycontain6-10respondentsandeachsessionlastsupto1½hours.

■■ A‘topicguide’–essentiallyanagendadetailingissuestobecovered–isusedbythefacilitatortoguidediscussions.

■■ Asuitablevenueischosen.Thiscanbeameetingroomor,ifobserverswishtoviewfocusgroups,adedicatedviewingfacilitycanbehired(andtherearemanyoftheseacrosstheUK).

Resources and timescales

■■ Relativelyshorttimescales:forexample,aprogrammeoffourfocusgroupsmighttakearound4weekstocompletefrominceptiontodeliveryoffindings.

Other details and considerations

■■ Initialrecruitmentquestionnairesareausefulwayofensuringthatyouobtainthedesiredhomogeneity/heterogeneityofgroups.

■■ Facilitatingfocusgroupsisaskilledtask.Itisthefacilitator’srolenotonlytoensurethatdiscussionsarekeptonsubject,butalsotocontrolthedynamicsofthegroup–forexampleensuringthatstrongercharactersdonotdominatediscussionsandundulyinfluencetheviewsofothers.

■■ Extendedfocusgroupsoftenlasting2½to3hourscanbeusefulforexploringmorecomplexand/ormorediverseissues.However,insessionsofthisdurationitisimportanttoincludenumerousbreaksandotherwaysofdividingdiscussionsintomoremanageablesections.

■■ Variationsonfocusgroupscanbeusefulforsomeaudiences.Forexample,anapproachoftenusedforinterviewingyoungpeopleistousefriendshippairs,triadsormini-groupstoovercomelackofconfidence.

■■ Focusgroupsrelyonasufficientnumberofaudiencemembersbeinglocatedincloseproximitytooneanother.Ifanaudienceiswidelydispersed,focusgroupscannotbeconsidered.

■■ Onlinefocusgroupsarearelativelyrecentdevelopment.Theyhaveseveralbenefits–notleastcost–butlackmanyofthekeybenefitsoftraditionalfocusgroupssuchastheabilitytoobserveinteractionandbodylanguage.

■■ Projectivetechniques–whichhavedevelopedoutoftechniquesusedinpsychoanalysis–aresometimeseffectiveinhelpingtouncoverunconsciousattitudesandmotivations.

■■ Becauseofthelevelofcommitmentdemandedofrespondents,itistypicaltooffer‘incentives’whichcanbeintheformofcash,vouchers,donationstoanominatedcharityetc.



Table 4‑4One-to-one in-depth interviews

One-to-one, in-depth interviews

Description In-depthinterviewsconductedonaone-to-onebasis,bytelephoneorinperson.

Uses Incommonwithfocusgroups,one-to-oneinterviewsareusefulinunderstandingtherangeofattitudesandmotivationswhichmightexistwithinanaudience.

Outline of format ■■ One-to-onein-depthinterviewscanbeconductedbytelephoneorface-to-face.

■■ Bothapproachescanbeusedtoobtainqualitativedata–althoughgenerallyface-to-faceinterviewsaremoreeffectiveindeliveringagreaterdepthofinformationandunderstanding.

■■ Face-to-faceinterviewstypicallylastforupto1hour.Telephoneinterviewsareusuallynolongerthan30-45minutes.

■■ A‘discussionguide’orsemi-structuredquestionnaireisusedasthebasisforinterviews.

■■ Inmostcasesrespondentswouldbeinterviewedintheirownenvironment(placeofwork/studyorhomeasrelevant).


■■ One-to-oneinterviews,especiallyifconductedinperson,tendtorequiresignificantlylongertimescalesthan,forexample,focusgroups.Geographicallydispersedaudiencescanmeanthatonly2or3interviewsmightbepossibleperday.Consequently,aprojectinvolving30interviewsmighttake6weeksormoretocomplete.


■■ One-to-oneinterviewsareespeciallyappropriateforsmall,geographicallydispersedaudiences(eg universitylibrarians).

■■ Theyarealsousefulifthesubjectmatterissensitive–inwhichcaserespondentsmightnotwanttodiscloseinformationtotheirpeerswhichwouldbethecaseinfocusgroups.

■■ Althoughtelephoneinterviewsareeasierandcheapertoarrange,thequalityanddepthofinformationobtainedfromface-to-faceinterviewsisusuallysuperior.

■■ Itisalsoworthrememberingthataface-to-faceformatallows‘stimulusmaterial’tobeused.Thisismoredifficulttoorganisefortelephoneinterviews,althoughmaterialscanbefaxed,emailedorpostedaheadoftheinterview.



Table 4‑5Quantitative surveys

Quantitative surveys

Description Quantitativesurveysinvolveusingasampleofanaudience/populationasthebasisforproducingastatisticallyreliablepictureofthewhole.

Uses Providingquantifiableinformationaboutanaudience,forexample,itsprofileanditspatternsofusageandbehaviour.

Outline of format Therearemanywaysofcollectingquantitativedataincluding:

■■ Postalsurveys:Audienceand/orprospectiveaudiencemembersaresentaquestionnairewhichtheyareinvitedtocompleteandreturn(usuallyinareply-paidenvelope).

■■ Emailsurveys:Audienceand/orprospectiveaudiencemembersareemailedasurvey(oremailedaURLlink)whichtheyareinvitedtocompleteonline.Contactemailaddressesareneededforthisoption.

■■ Web-basedonlinesurveys:Awebsiteorresourceextensivelyusedbythetargetaudienceisusedtopromotethesurveyandtohostalinkforrespondentstofollow.Thisoptiondoesnotrequirethatemailaddressesareknownbutisaninexactmeansoftargetinganentireaudience.

■■ Telephonesurveys:Audienceand/orprospectiveaudiencemembersarecontactedbytelephone,usingashort,structuredquestionnaire(generallylastingnolongerthan10-15minutes).Dependingonthecompositionoftheaudienceitissometimessensibletoset‘quotas’toensurethatdifferentsegmentsarerepresentedinthesample.

■■ Face-to-facesurveys:Forverylargeconsumeraudiences,on-streetandin-homeface-to-faceinterviewsareoftenappropriate.Thefewercontactsthatneedtobemadetofindeachqualifyingaudiencemember,themorecost-effectivethisapproachbecomes.Forexample,fornon-digitalservicesface-to-faceexitinterviews(egwithlibraryusers)offeracost-effectivemeansofsurveyinganaudiencebutexcludeorunder-representinfrequentandnon-usersofaservice.


■■ Telephoneandface-to-faceinterviewersrequireateamof(ideally)experienced,professionally-trainedinterviewers.

■■ Independentfull-serviceresearchagencieshaveaccesstointerviewersinallareasoftheUK.

■■ Theabilityofanorganisationtodevelopandhostonlinesurveysdependsonitsaccesstoappropriatetechnicalresources.

■■ Additionally,ameansofanalysingsurveydataisrequired.Mostindependentresearchagencieswillusespecialistsurveyanalysissoftwarewhichfacilitatesandintegratesallstagesoftheproject;sampleandquestionnairedesign;hostingofonlinesurveys;datacaptureandstatisticalanalysis.



Quantitative surveys


Sub-sections5.2and5.4provideanoverviewofhowtoanalysequantitativedata,includinglookingatthepotentialproblemswithdatainterpretation,sampling(includingsamplebias),theimportanceofquestionnairedesignandweightingresponses.Thereisalargeamountofinformationavailableinbooksandontheinternettohelpyouunderstandthesetopicsinmoredetail,including:

■■ AquestionnairedesignadvicesheetfromLoughboroughUniversity:www.lboro.ac.uk/library/sklls/Advice/QuestionnaireDesign.pdf

■■ Acomprehensivebookonmarketingresearchincludingusefulsectionsonquestionnairedesign,sampling,analysisandweighting:Marketing Research,Aaker,KumarandDay;JohnWiley&Sons,Inc.;8thed.(2004)

■■ Variousresearchtips,whitepapersandstudiesbyacademicsandpractitioners:www.sysurvey.com/tips/whitepapers.asp

Otherconsiderations:

■■ ‘Self-completion’formats–suchaspostalandonlinesurveys–areattractivesincetheyallowalargeaudiencetobesampledrelativelyeasilyandinexpensively.However,theresultingsampleisself-selecting(ierespondentsmakeaconsciouschoicetobepartofthesample).Aneffectoftenobservedinself-selectingsamplesisthatthepropensitytorespondisnotconsistentacrossallaudiencemembertypes.Inaudienceresearch,morefrequent,committedservice-userstendtobemorelikelytorespondthansporadicandinfrequentusers.Ifthereforetheserviceproviderisaimingtodevelopastrategytoincreaseusageamonginfrequentusers(andthereforeneedstolearnmoreoftheirattitudesandmotivations),self-selectingsurveyformatsarenotusuallyappropriate.

■■ Interviewer-ledformats(egtelephonesurveys)aremoreexpensivetoundertakebuthavetheadvantageofbeingmore‘random’(ietheoreticallyatleast,eachaudiencememberhasamoreequalchanceofbeingselectedforthesurvey).Quotascanalsobesettoensurecoverageofallsegmentswithinanaudience.

■■ Itisimportanttodistinguishbetweenaccuracyandprecision–youmaywellget1000responsestoyoursurveygivingstatisticallyprecisedata,butitcouldbeentirelyinaccurateifyouhavemissedsomekeysegmentsofyourtargetaudience.

http://www.lboro.ac.uk/library/sklls/Advice/QuestionnaireDesign.pdf


http://www.sysurvey.com/tips/whitepapers.asp



Table 4‑6‘Traditional’ web statistics

‘Traditional’ web statistics

Description Webserversgeneratelogfileswhentheyservearequest,whichistypicallydirectlyfroma‘useragent’–usuallyabrowsersuchasInternetExplorerorFirefox.Theselogfilesrecordtherequestwhichwasmade,thestatusoftheresponse(success,failureetc.)andsomebasicinformationabouttheuseragent.

Thismethodconsidersrelativelysimplewaysinwhichtheselogfilescanbeprocessedtogeneratequantitativeinformation(webstatistics)regardingtheuseoftheservice.

Uses Webstatisticscanhelptounderstand,onaggregate,thewaysinwhichanonlineresource(orcollectionofresources)isused.Forexample,itcanidentifywhichsectionsofthesitearemostused,whereusershavecomefrom(egfromasearchengineoralinkfromarelatedsite),thenetworkfromwhichtheyareaccessingthesite,thetypeofuseragenttheyareusingetc.

Inthecontextofaudienceanalysis,webstatisticshelptounderstandtherelativeusageofdifferentpartsofasite,andhowtheusagevariesovertime.

Outline of format Webserversgeneratelogfileswhichareperiodicallyanalysedtogeneratestatistics.Awiderangeoftoolsareavailabletoundertakethisanalysis,manyofwhicharefreeandopensource.

Differenttoolswillbeabletoundertakeanalysiswithdifferentlevelsofcomplexity.Forexample,abasictoolwillbeabletotellwhichresourcesreceivedthemostaccesses,ortheproportionofuserswhichcamefrom.ac.uknetworks.Morecomplextoolsallowcross-tabulation:dothoseusersat.ac.ukusedifferentresourcesfromthoseat.edu?Whichresourcesarediscoveredthroughsearchengines,andwhicharefoundbynavigatingthroughthesite?


■■ Dataiscollectedcontinuouslyduringserveroperation.Typically,statisticsgenerationisundertakenonaperiodicbasis,oftendaily.

■■ Generationofstatisticsisfullyautomated,andcommontoolshandlearchivingofpastinformationandgenerationofusefulreports.

■■ Awiderangeoftoolsareavailableforundertakingthelogprocessing,includingsimplefreeandopensourcetoolssuchasAnalog,AWstatsorWebalizer.Commercialpackagestypicallyoffergreaterfunctionality,andarebetterclassedaswebanalyticstools.

■■ Mostsitesexperiencehourly,daily,weeklyandseasonalvariationsinusage.Thiswillonlybecapturedwithtime.


■■ Webstatisticsrecordusageofthewebserver–theydonotfollowtheactivitiesofindividualusers.Eachrequesttoaresourceisessentiallyindependentofallotheractionsofthatuser.

■■ Itisexceedinglydifficulttoestimatethenumberofuniqueusersofasitefromthewebstatistics.Someproxymeasuresarepossible(forexamplethenumberofuniqueIPaddresses),butthesehaveverylowreliability.

■■ Itisusuallynecessarytoexplicitlyfiltersearchengineindexersfromtheresults.

■■ Relatingthenetworklocationofavisitortothecountrytheyarelocatedinispossibletoareasonablelevelofaccuracy,butonlythroughcommercialservices.

■■ Fordynamicwebsites,understandingresourceusageusingbasicwebstatisticspackagesdependsonthesitedesignusingpersistentanduniqueURLs.



Table 4‑7Web analytics

Web analytics

Description Webstatisticscanprovideusefulbulkinformationontheutilisationofresources,buthavesignificantlimitationswhenattemptingtounderstandtheusersofaresource.Webanalyticsisamoreadvancedmethodforinvestigatingtheactionsofusers.Webanalyticswillincludetheinformationavailablefromtraditionalwebstatistics,sotheseusesarenotrepeatedhere.

Whereaswebstatisticsisfocusedontheresource,webanalyticsisfocusedonindividualusers.Theactionsoftheseuserscanbecombinedtoprovidebulkinformationonaservice,butatafarhigherlevelofdetailthanthatavailablefromtraditionalwebstatistics.

Webanalyticscanbeparticularlyhelpfultosupportsitedesigndecisions,byunderstandingthedecisionsthatuserstakewhenusingaresource.

Uses Webanalyticscanbeusedtounderstandindetailthebehaviourofusersofasite.Itcansegmentusersmoreclearlythantraditionalwebstatistics,andcanidentifyuniqueuserswithfargreatercertainty.Itcanprovidemeasuresofvisitfrequencyandduration,andanunderstandingof‘clickstreams’whichrecordthepathsthatuserstakethroughthesite.

Thisinformationcanleadtoabetterunderstandingoftheuseofresources.Forexample,itmaybepossibletosegmentusersalonglinessuchasone-offvisitorswho‘land’onaparticularresourcefromasearchengine,andnevervisittherestofthesite,oroccasionalin-depthbrowserswhoarriveatasite,andthenspendsometimenavigatingandusingresourcesoruserswhoreturnoftenbutwhousealimitedsetofresources.

Thiskindofsegmentationcanleadtomakingbetterdecisionsaboutsitedesign,structureandcontent.

Outline of format Arangeofmethodscanbeemployedtotrackthebehaviourofindividualusers.Thekeyoptionsare:

■■ Enhancedloganalysis,wherebythestandardwebserverlogsareanalysedtoattempttoelucidate‘sessions’whichidentifyindividualusers;

■■ Server-sideusertracking,wherethewebserverisadaptedtoidentifyindividualsessionswithinitslogfiles.Thisisoftencombinedwithleaving‘cookies’totrackreturningusers;

■■ Arangeofwebanalyticsservicescanbeusedinan‘out-sourced’manner,wheretheserviceproviderembedscodewithineachpageoftheresource,whichisloadeddirectlyfromtheanalyticsprovider.Thisallowstheanalyticsservicetocollatedatawithoutaccesstotheserviceprovider’swebserverlogs;

■■ Mostcontentmanagementsystemswhichprovide‘library’resourcesonlinewillincorporatesomeusagemonitoringfunctionalitywithintheapplicationitself.Thisshouldincludetrackinganysearchtermsusedwithinthesite.

Followingdatacollection,thedatamustbeanalysed.Initially,thisisoftenbyaniterative,investigativeprocessratherthanbylistingindividualmetrics.Asservicesbecomemoreestablished,itmaybepossibletoelucidatequantitativeKeyPerformanceIndicators(KPIs)fromthewebanalyticsdata.



Web analytics


■■ Asforwebstatistics,dataiscollectedonlineandonanongoingbasis.

■■ Toolscanbeobtainedfornocost(egGoogleAnalytics),onaservicebasis(egWebTrendsOnDemand)orforapurchaseprice(egSawmill,ClickTracks).

■■ Theeffortrequiredtobegincollectingdataistypicallylowandembeddingthetoolswithintheserverworkflowisrelativelyeasy,especiallyforsimplehostingarrangements.

■■ Settingupreportstoanalysethedatacollectedisoftentime-consuming,requiringdetailedconsiderationofthesitestructure,thebehaviourofusers,andagoodunderstandingofthenatureoftheresources.Oncethesehavebeensetup,ongoingmonitoringshouldberoutine.


■■ Recordingextensiveinformationaboutthebehaviourofindividualusersraisesprivacyconcerns.Eveniftheservicedoesnotexplicitlyidentifyusers,thisinformationmaywellbepersonaldata(orevensensitivepersonaldata)withinthemeaningoftheDataProtectionAct1998.

■■ Webanalyticshelpstounderstandthebehaviourofusersofasite,butitisnecessarytovalidatefindings.Forexample,ifahighproportionofvisitorsonlyviewonepage,doesthismeanthattheyfoundwhattheywerelookingfor,orthattheydidn’tandtheydon’thavethepatiencetoinvestigatethesitemoredeeply?

■■ Itshouldbenotedwebanalyticsisalsoknownbyothernames.Forexample,‘DeepLogAnalysis’(DLA)isa‘brandname’forresearchundertakenbytheCIBERteamatUniversityCollegeLondon(UCL).DLAreferstotheanalysisofrawserverlogs,andisoftenconductedinconjunctionwithotheraudienceresearchtechniques(egquestionnairesandinterviews),todevelopanunderstandingofuserbehaviour.Althoughapotentiallypowerfultechnique,analysingweblogsinthisdepthisunlikelytobeachievablewithinthelimitedbudgetsofmanypublicsectororganisations,andnorisitalwaysnecessaryorappropriate.

Table 4‑8 User observation techniques (ethnography)9

User observation techniques (ethnography)

Description Ethnographyisatermusedtoencompassawiderangeoftechniquesandapproaches.Atitscoreisadesiretounderstandhumanculturebyobservingbehaviourbutthisbroadprincipleisincreasinglyusedincommercial,publicandsocialresearchcontexts.[Ethnographyis]‘anythingfromhangingaroundaskateboardpark,interviewingteenagersintheirbedrooms,askingrespondentstomakevideodiaries,tointerviewingtheirfriendsandfamilies,orvideoingthemgoingshopping–it’sallethnography.’9

Itisbroadlyacceptedbyacademicsandresearchpractitionersalikethatethnographyintoday’sworldisaboutdifferenttypesofobservation.

Uses Ethnographyissaidtominimise‘researcheffect’byobservingsubjectsintheir‘natural’context(whichmightforexamplemeanattheirdeskatwork,infrontofacomputerscreen,inagalleryormuseumorathome)andbyremovingbiasintroducedbyframingstructuredquestions.Inthecontextofaudienceresearchethnographyandobservationaltechniquescanofferaninsightintohowaudiencesuseandinteractwithservicesandresources.Fordigitalservicesverydetaileddataaboutusagecanoftenbecapturedunobtrusivelybycomputerloggingtechniques.

9 Viewpoint – Ethnography and market research,PhillyDesai;InternationalJournalofMarketResearchvolume49(6);MarketResearchSociety.



User observation techniques (ethnography)

Outline of format Therangeofformatsistoobroadtocoverindetailbutpossibleapproachesinclude:

User tracking studies:theaimistocaptureserviceusagebehaviourinarealworldcontext:

■■ subjectsmightbeaskedtokeepaphoto,audioorvideodiaryorweblogrecordingtheirday-to-dayinteractionwiththeserviceoveraparticulartimeperiod;

■■ subjectsare‘shadowed’andbehaviourrecordedbyanobserver,orusingspecialisedequipmentina‘viewinglab’.

Thesemethodscanbecombinedwithinterviewingtoallowaninterviewertoprobereasonsfordecisionsandactions.Interviewingmaytakeplaceaftertheuser-trackingactivityoratthesametimeegaccompaniedinternetbrowsing.

Use testing:aserviceistestedbyusersandthefinedetailofusagebehaviouriscapturedbycomputerorrecordedbyothermeans:

■■ user-generatedscenarios:usersareallowedtoexploreaserviceorresourcefreely,theiractivityandperhapstheircommentsontheuserexperiencearerecordedforsubsequentanalysis.Providesanaturalisticpictureofhowtheaudiencerespondtotheservice;

■■ task-focusedscenarios:usersareaskedtocarryoutapre-definedsetoftasksusingtheservice,theextenttowhichtheirpaththroughtheserviceisdictatedisvariable;aninteractiveformatispossibleietheuserrespondstoinstructionsfroman‘observer’whocanadapttheprotocolinrealtime.Theobservermaybewiththeuserorfollowingtheiractivityremotely.

Againthedetaileduseractivityisrecordedandanalysedtoallowassessmentofhowusabletheserviceisandhowwellitfulfilsitsfunction.


■■ Methodscanbecustomisedforsmallerbudgets.

■■ Useofspecialised‘viewinglabs’equippedwitheye-trackingandaudiovisualrecordingisexpensive.

■■ Foracomputer-basedservicedetailedactivitydatacanbecapturedquicklyandcheaplywithappropriateprogramming.

■■ Usetestingcanbeverytimeconsumingbutvariabilityacrosssubjectsislowsosmallsamplescanbeused.

■■ Analysingactivityandobservationaldatarequiressomeexpertise.


■■ Digitalservicescanbetestedbyusersatalocationandperhapstimeoftheirownchoosingwhichmaymakerecruitmenteasier.

■■ Serviceevaluationinanaturalisticsettingmayproducedifferentresultsfromtestinginalab.

■■ Ethnographicmethodscanbeadaptedtodifferentcircumstances,valuabledatacanstillbeobtainedifcheaper‘observational’techniquesareused.



4.3 Pros and cons of the methods

4.3.1 Acritiqueofeachofthemethodsisprovidedinthefollowingtablestohelpyoudecidewhich

techniquesareappropriateforyourresearch.

Table 4‑9Pros and cons of quantitative methods

QUANTITATIVE METHODS

Method Pros Cons

Face-to-face interviews

■■ Goodqualitydata.

■■ Limitvariationsinquestioninterpretation.

■■ Enablemorecomplexquestionstobeasked.

■■ Experiencedinterviewersareabletoproberesponsestoprovidefullerunderstandingofissues.

■■ Stimulusmaterialcanbeshown(egserviceoutlines;informationleafletsetc.).

■■ Relativelyexpensive.

■■ Requireahighlevelofrespondentcooperationandcommitment(mightneedtobeincentivised).

Postal questionnaires

■■ Relativelyinexpensive.

■■ Offeralargeproportionofrelevantaudienceachancetorespond–iftheychooseto.

■■ Demandlowlevelofcommitmentfromrespondents.

■■ Sampleisself-selecting(andthereforenotnecessarilyrepresentative).

■■ Noguaranteesofresponserates.

■■ Timingsaremoredifficulttocontrolandpredict.

■■ Responseratestendtobelow.

■■ Questionnairelengthandcomplexityislimited.

■■ Questioninterpretationcanbesubjective.

Internet/ email surveys

■■ Cost-effective.

■■ Offeralargeproportionofrelevantaudienceachancetorespond–iftheychooseto.

■■ Demandlowlevelofcommitmentfromrespondents.

■■ Canprovideveryimmediatefeedback.

■■ Sampleisself-selecting(andthereforenotnecessarilyrepresentative).

■■ Noguaranteesofresponserates.

■■ Notallpotentialrespondentswillhaveinternetaccessorwillbecomfortableusingit.

■■ Timingsaremoredifficulttocontrolandpredict.

■■ Questionnairelengthandcomplexityislimited.

■■ Questioninterpretationcanbesubjective.

Telephone interviews

■■ Goodbalanceofqualitydataandcost-effectiveness.

■■ Limitvariationsinquestioninterpretation.

■■ Enablemorecomplexquestionstobeasked.

■■ Experiencedinterviewersareabletoproberesponsestoprovidefullerunderstandingofissues.

■■ Stimulusmaterialcannotbeshown.

■■ Someaudiencesarebecomingover-researched–particularlybytelephonesurveys.



Table 4‑10Pros and cons of qualitiative methods

QUALITATIVE METHODS

Method Pros Cons

Focus groups ■■ Usefulforstimulatingdiscussionandnewideas.

■■ Consequentlyfocusgroupscanbeaverycreativeapproach.

■■ Thefocusgroupformatiswell-suitedtointroducingandgainingreactionstostimulusmaterials(egideasforserviceframeworks;promotionalandinformationliteratureetc.).

■■ Relativelyshorttimescales.

■■ Reliesonsufficientnumbersofaudiencebeingclusteredinageographicalarea–oratleastwithinaradiusthatrespondentsmightbepreparedtotravel.

■■ Respondentsmightnotbepreparedtodiscusspersonally,commerciallyoracademicallysensitivesubjectswithothers.

■■ Requireexperiencedfacilitators.

■■ Somegroupmemberscanbeundulyinfluencedbytheviewsofothers.

One-to-one in-depth interviews

■■ Easiertoarrangethanfocusgroups.

■■ Interviewercanvisitinterviewee.

■■ Geographicallydiverseaudiencescanberesearched.

■■ Respondentsarenotunderpressurefromotherstorespondinaparticularway.

■■ Arapportcanbebuiltbetweeninterviewerandinterviewee;thisinturncanleadtodisclosureofinformationwhichotherwisemightnothavebeendivulged.

■■ Formattendstobelessdynamicandcreativethanfocusgroups.

■■ Timescalesareusuallylonger(focusgroupscanprovideaccessto40individualswithinacoupleofdays;thesamenumbersinaone-to-oneformatwouldtakeseveralweeks).

Table 4‑11Pros and cons of web statistics

WEB STATISTICS

Method Pros Cons

‘Traditional’ web statistics

■■ Lowcostintimeandeffort.

■■ Informationcanbeavailableimmediately.

■■ Providesan‘at-a-glance’viewofhowbusyasiteis,andwhichsectionsofthesitearemostused.

■■ Limitedinsighttouserbehaviour.

■■ Verycrudemeasuresofdemographicofaudience.

■■ Itistemptingtoinfertoomuchfromtheinformationwhichisavailable.

■■ Basicpackagesdonothandlelibrary-typedynamicsiteswell.

Web analytics ■■ Informationcanbeavailableimmediately.

■■ Providesextensiveinformationonuserbehaviour.

■■ Canbeusedtosegmentusersbasedonarangeofbehaviours,andpossiblyondemographics.

■■ Commercialwebanalyticspackagesareexpensive.

■■ Settingupreportingistime-consuming,anddemandsgoodbusinessandtechnicalknowledge.

■■ Insightintouserbehaviourfromwebstatisticsmustbevalidated.



Table 4‑12Pros and cons of ethnographic methods

ETHNOGRAPHIC RESEARCH

Method Pros Cons

user tracking: diary studies, shadowing and contextualised interviews

■■ Usefulforbuildingupapictureofserviceusageinabroadcontext.

■■ Abilitytoquestionusersaboutbehaviourallowselucidationofunexpectedusagepatterns.

■■ Participationintheresearchshouldhaveonlyminimaleffectonthesubjects’normalroutine.

■■ Alowcost,loweffortwayofinvestigatingthefinedetailofusagebehaviour.

■■ Methodsofrecordingusagemayinterferewithnormaluserbehaviour.

■■ Diarystudies:qualityofdatacollecteddependsontheskillandcommitmentoftheparticipant.

■■ Shadowing:datacollectionmaybeintrusiveandresourceintensive.

use testing: user-generated activity sessions and task-focused activity sessions

■■ Veryhelpfulinidentifyingunspokenuserrequirements.

■■ Excellentwayofidentifyingproblemswithusability.

■■ Audiencebehaviourmaybeobservedandmeasureddirectly,eliminatinginaccuraciesthatoccurinself-reports.

■■ Capturesthefinedetailofuserbehaviour.

■■ Canprovideanaturalisticpictureofusagebehaviour.

■■ Variabilitybetweensubjectstendstobelowsosmallsamplesizescanbeusedsuccessfully.

■■ Analysinguseractivitydatarequiresconsiderableexpertise.

■■ Testingcanbeverytimeconsuming.

■■ Somesophisticateddatacapturetechniquesrequirespecialisedequipmentsosubjectmustattendatestingfacility.

■■ Toderivemaximumbenefittheremustbegoodcommunicationwithtechnicalsupportteam.

4.4 Indirect approaches to audience behaviour

4.4.1 Audienceanalysishasanimportantroleinservicedevelopmentandvariousethnographic

techniquesareincreasinglyusedtosupportuser-centreddesignandmanyofthese

techniquescanbeadaptedtosuitsmallerbudgets.User-centredservicedevelopmentmay

alsoincorporatemethodswhichdonotinvolvetheaudiencedirectly.Thesemethodsfall

outsidethescopeofthisguidebutaredescribedbrieflybelowbecauseoftheirutilityinthe

earlystagesofservicedevelopmentandasaquickanddirtyalternativetousertestingif

resourcesareseverelylimitedoruserrecruitmentisproblematic.

Heuristic evaluation

4.4.2 Expertevaluatorsassesstheservice’suserinterfaceagainstrecognisedusabilitycriteria.The

exactcriteriausedshouldreflecttheneedsofthetargetaudience,theplatformandtypeof

serviceoffered.

4.4.3 Thismethodofassessingusabilityischeapanddoesnotrequirealargeorrepresentative

sampleofpotentialusersfromthetargetaudience.Themethodcanbeusedevenatavery

earlystageindevelopmentifevaluatorsareabletoworkwithmock-ups.



Cognitive walkthrough

4.4.4 Expertevaluatorsplaytheroleofatypicaluser(userprofilesorpersonasmaybehelpful

here)andattempttoperformthetaskstheserviceisdesignedfor.Analysisoftheserviceis

animportantprerequisite:userobjectivesarespecifiedandbrokendowntoshowthesub-

goalsthatmustbeachievedinordertoattaintheobjective.

4.4.5 Thismethodprovidesanideaofhowwelltheservicefunctionsfromtheuserperspective,

butreliesontherole-playingskillsandjudgementsoftheevaluatorsratherthaninvolvingthe

targetaudiencedirectly.Aswithheuristicevaluation,themainadvantagesarethelowcost,

theavoidanceofrecruitmentproblemsandthepotentialtocarryoutevaluationsatanearlier

stageinservicedevelopment.

5. Modelling the Audience PAGE 47

5.ModellingtheAudience

This chapter deals with analysis and interpretation of audience data to address specific objectives and improve understanding of the audience

Datafromanaudienceanalysisprojectshouldimproveunderstandingoftheaudienceinseveralways,byproviding:

■■ Answerstospecificresearchquestions–orfurtherevidence.

■■ Newknowledgeabouttheaudienceandtheirinteractionswiththeservice.

■■ Objectivesforfutureaudienceresearch.

Topicscoveredinthischapterinclude:

■■ Approachestoanalysingquantitativeandqualitativedata.

■■ Potentialproblemswithdatainterpretation.

■■ Gapanalysisofaudiencesatisfaction.

■■ Developingpersonasoruserprofilesforservicedevelopment.

5.1 Introduction

5.1.1 Thissectiondealswiththeissuesinvolvedinanalysingandinterpretingthedatatoaddress

thespecificresearchquestionsandbuildupabetterunderstandingoftheaudienceandtheir

interactionswiththeservice.

5.1.2 Audienceanalysisprojectsshouldstartwiththerecognitionthatfurtherknowledgeabout

theaudiencewouldbehelpfulegforplanningfutureservicedevelopmentoroptimising

marketingstrategy.Thisleadstoframingthebroadobjectivesfortheaudienceanalysisand

settingoutspecificquestionstheresearchshouldaddress.Methodsforcollectingrelevant

datashouldbeselectedandthedetailsofthemethodologyestablished.Datacollectioncan

thenbecarriedout.

5.1.3 Theaudiencedatamustbeanalysedappropriately:thetechniquesusedwilldependon

thetypeofdataandtheaimsoftheresearch.Thedatashouldprovideevidencethatcan

beinterpretedtodrawconclusionsrelevanttothespecificobjectivesoftheresearch.Itis

importanttobesurethatthedatareallydoprovideevidenceforanystatementsyoumake


PAGE 48 5. Modelling the Audience

egandthatyouhaveconsideredallthepossibleinterpretationsofthedata.Fullanalysisand

interpretationofthedatashouldalsoprovidewiderinsightsintotheaudience,theirbehaviour

andtheirrelationshipwiththeservice.

5.1.4 Analysingthedatafromaudienceresearchwillbeeasieriftheresearchwaswellplanned

andcarriedoutproperly.Manyproblemsinanalysisandinterpretationcanbeavoidedifdata

collectionproceduresareselectedandimplementedwiththeanalysisinmind.Sometimes

atrade-offneedstobemadebetweensimplicity,convenienceandreliabilityofrecording

andconvenienceandpowerforanalysis.Paper-baseddatacollectionmightbeeasiestto

administer,butdatawillhavetobecodedandenteredontoacomputerbeforeanalysis.

5.1.5 Analysisofthedatafromanaudienceresearchprojectshouldimproveunderstandingofthe

audienceinseveralways,providing:

■■ answerstothespecificresearchquestions–oratleastrelevantevidence;

■■ newknowledgeabouttheaudiencemoregenerally;

■■ furtherquestionsabouttheaudiencetobeaddressedinfuturework.

5.2 Analysing quantitative data

5.2.1 Quantitativedatacanbeanalysedstatistically.Detailedexplorationofthemethodsisbeyond

thescopeofthisguideanditisrecommendedthatreadersconsultastatisticalhandbookfor

psychologyorthesocialsciencesegStatistical Methods in PsychologyDavidHowell(2006).

TheuserguidesforstatisticalsoftwarepackagesegSPSSareoftenhelpful.

5.2.2 Statisticaltechniquescanbeusedtoexplorerelationshipsbetweenvariables.Forexample,

toanswerquestionssuchas‘areregisteredusersmorelikelytodownloadfilesfroma

service?’andtouncoverhomogeneoussubsetsorsegmentswithinanaudience.

5.2.3 Quantitativesurveysmayincludequestionsinvitingorrequiringfreetextcommentsorsimply

acatch-allinvitationattheend,forexample,‘Pleaseusethisspacetomakeanyother

commentsaboutourservice’.Datafromtheseitemswillneedtobecodedforquantitative

analysisandmightbemoreappropriatelytreatedasqualitativedata.Thismaymeanmaking

thequalitativecommentsavailableinadifferentformattofacilitatecollationandanalysis.

Regression and correlation5.2.4 Regressionandcorrelationanalysisaremethodsfordefiningrelationshipsbetweenvariables.

■■ Correlation analysislooksatthestrengthandnatureoftherelationshipbetween

variablesegdoesaccesstobroadbandinfluencedownloadingbehaviour,does

goodaccesstobroadbandincreaseordecreasethefrequencywithwhichcontentis

downloaded,doesitincreaseordecreasetheaveragesizeoffiledownloadedetc.

■■ Regression analysisisusedtoderivepredictionsaboutthevalueofonevariable,

basedonanother;multipleregressionanalysisconsidersseveralvariables:however

one–adependentvariable–isgenerallypredictedorexplainedbymeansoftheother

independentvariable(s)andcovariates.



Factor and cluster analysis5.2.5 Boththesetechniqueslookatrelationshipsamongvariables.Theyarenotgenerallyusedin

prediction.Theresearchermustinterprettheoutputoftheanalysistoderiveamodelofthe

audiencewhichbestfitsthedata.

5.2.6 Inverysimpleterms,cluster analysisaimstofindrelativelyhomogeneousgroups

orclusters,themembersofwhichtendtobehaveandthinkinsimilarways.Different

mathematicaltechniquescanbeusedtoidentifyclusterswithinmulti-dimensionaldata.

5.2.7 Factor analysisisastatisticaltechniqueusedtoanalyserelationshipsbetweenalarge

numberofvariablesandtoexplainthesevariablesintermsofcommonunderlyingfactors.In

audienceresearchitisusedtosimplifythenumberoffactorsorcategoriesneededtoexplain

ormodeltheaudience(egfromalargeseriesofstatementsaboutsatisfactionwithaservice

thefollowingfactorsmightemerge:easeofaccess,qualityofcontent,valueformoney).It

isuptotheresearchertoinspectthewayinwhichvariablesaregroupedandassignuseful

namestothefactors.Factoranalysiscanbeusedtodiscoveraschemeforsegmentingan

audience.Ratherthandecidinginadvancewhatthedefiningcharacteristicsofeachsegment

are,factoranalysisletssegmentsemergefromthedata.

Potential problems with interpretation of quantitative data

Instructions and/ or responses are misunderstood

5.2.8 Mostoftheproblemsarisebecausetheaudiencesampledidn’treacttotheresearch

instrument–usuallyasurveyquestionnaire–astheaudienceresearcherintended:

■■ someoralloftherespondentsdidn’tinterpretthequestionsinthewaythattheaudience

researcherintended;

■■ someoralloftherespondentsdidn’tusetheresponseoptionsastheresearcherintended

egscalesusedthewrongwayround,ticksusedinsteadofnumericalpreferenceratings,

multipleresponsecategoriesselectedwhereonlyonewaspermittedetc.;

■■ researcherfindstheresponsesunexpectedlyambiguous;

■■ largenumbersofrespondentsfailtocompleteallthequestions.

Someoftheseproblemscanbeavoidedbypilotingaquestionnaireonasmallsampleor

usinginterviewerstocollectsurveydata.

Questionnaire design is important

5.2.9 Surveysoftenaskrespondentstorateaspectsoftheservice.Sometimesnumericalscales

areused,sometimesverbalorpictorialscalesareused;somescaleshaveaneutralmidpoint,

somequestionswillallow‘notapplicable’or‘don’tknow’asoptions.Thesedecisionsabout

questionnairedesigncanhaveimplicationsfortheinterpretationoftheresults.



■■ Theexactwordingofstatementsaboutsatisfactionwiththeservicecouldaffectthe

proportionofthesamplewhoappeartobecontentwiththeservice.Respondents

mayuse‘good’tomean‘acceptable,meetingtherequiredstandard’or‘betterthan

acceptable,exceedingtherequiredstandard’.Doyouknowwhatrespondentsmeant

whentheycompletedyourquestionnaire?

■■ Respondentstendtoavoidusingtheendpointsofanyscale,sothatveryfew

respondentswillappear‘extremelydissatisfied’or‘extremelysatisfied’withwhateveris

beingrated.

■■ Respondentswilltendtorespondrelativetowhattheythinkthedefaultoraverageisegif

askedtorateaccessibilityonaonetofivescaletheymightactuallybecomparingyour

servicetowhattheyperceivetobetheaverageforwhatevertheyconsidertobesimilar

services.

The sample of respondents is unrepresentative

5.2.10 Obtainingarepresentativesampleisparticularlydifficultwhensurveysareplacedonlineor

distributedtoanunknownsetofpotentialrespondentsegtoanemailaddresslistincludinga

lotofinvalidaddresses.

5.2.11 Sometimesitiseasytodetectabiasinthesampleegonly10%ofrespondentsarestudents

butyoualreadyhavereliableinformationthatatleast40%ofyouraudiencearestudents,but

iflittleisknownaboutthecompositionoftheaudiencethisismoredifficult.

5.2.12 Thesamplemaybebiasedbyanumberoffactors:

■■ thetypeofincentiveused;

■■ thetimeperiodoverwhichresearchwascarriedout;

■■ themethodofdistributionandpublicity/promotion;

■■ thekindsofquestionsasked.

5.2.13 Ifthesampleislikelytobeunrepresentativeitisimportanttoconsidertheimplicationsfor

interpretationofthedataandmakeclearthecaveatswhichapplytotheconclusions.Ifthere

isnoreasontobelievethatadifferentsamplewouldhaverespondeddifferentlythiscanbe

stated.Ifyouareawareofimportantdifferencesinthecharacteristicsofdifferentaudience

segmentswhichcouldaffectthefindingsthisshouldbeexplained.


Results from a survey suggest users would prefer money to be spent on making more programmes available to download rather than increasing the number of programmes available with subtitles.

Thismaybebecausethesurveywasmentionedbyapresenteron‘Livingwithhearingimpairment’andoverhalftherespondentssaidtheyconsideredthemselveshearingimpaired.

Itwouldbeusefultostatetheresultsfornon-hearingimpairedrespondentsseparately.

Itisimportanttodetectthebiasinthissamplebecauserelevantdemographicdatawerecollected,butitisoftennotsoobvious.



5.3 Making use of qualitative data

5.3.1 Interpretingqualitativedataisnecessarilyasubjectiveprocess,soifresourcespermititmay

beworthinvolvingmorethanonepersoninanalysisandpresentationofthedata.

5.3.2 Itispossibletocodeorcategorisequalitativedatasothatquantitativetechniquescanbe

appliedegassigningstatementsofattitudetowardstheservicemadeduringfocusgroup

discussionstocategories:verypositive,positive,neutral,somewhatnegative,verynegative.

Howeverdoingthisresultsinlossoftherichdetailwhichmakesqualitativedatasovaluable.

5.3.3 Presentationsofqualitativedatawillusuallyattempttosummarisethecontentandimportof

thematerial.Foragroupdiscussionthiswouldusuallyincludeajudgementaboutvariations

ofopinionbetweengroupmembersandcommentsonpossiblereasonsfordifferencesof

opinion.Abriefsummaryoftheflowofdiscussioncanbehelpfultoelucidatethecontextfor

theviewsandattitudesexpressedbygroupmembers.

5.3.4 Presentationsofqualitativedatawillalsotypicallyincludequotationsfromthematerial.If

quotationsaretobeuseditisimportanttoensurethattheyareaccurate,sointerviewsor

discussionsneedtoberecordedinfull.

5.3.5 Codedqualitativedatacanbepresentedintabularorgraphicalformatasasupplementto

textualdigestsofthematerial.Quantitativeanalysesofqualitativedatafromasmallsample

mightbeworthwhileifyoubelieveyoursampletoberepresentativeoftheaudienceasa

whole,butitwillneverbeasubstituteforarobust,statisticallysoundanalysisofalarger,

morerepresentativesample.

5.3.6 Perhapsamoreinterestingapplicationofquantitativetechniquestoqualitativedatauses

theindividualastheunitofanalysisandlooksatdistributionofbehavioursorcomments

forasingleindividual.Forexamplebrowsingbehaviourcouldbeassignedtocategories

egsearchingasite,reading,downloading,useofsearchengine,enteringdataetc.The

percentageoftimespentoneachactivitycanthenbecalculated.Similarlystatementsmade

duringinterviewcouldbecodedandthefrequencywithwhichaparticularaspectofthesite

orresourcewasmentionedcouldbecalculated.Statementscouldalsobecodedbystrength

offeelingtoassesswhichaspectoftheserviceprovokedmostsatisfactionorfrustration.

5.3.7 Theimportantpointinthistypeofanalysisisthatstatementsaremadeaboutanindividual,

ratherthanthegroupeg‘90%ofsubjectA’sreferencestositecontentweretostoriesabout

sport’.Ifalltheintervieweesreferredtosportmorethanothercontentitmightbealsobetrue

thattheaudienceasawholewouldrefermosttocontentaboutsport,butunlessthesample

sizeislargeenoughtopermitstatisticalanalysessuchaconclusionwouldbeunreliable.An

alternativepossibilityisthatwhilstnotalltheaudiencewouldrefermosttosportsstories,

theymightalltendtohaveafavouritetypeofcontent.Youmightliketoconsiderhow

qualitativefindingslikethesecouldbefollowedupwithquantitativeresearch.



5.4 Analysing audience satisfaction for service development

5.4.1 Weighted gap analysisisatechniqueforcomparingaudiencesatisfactionwitheachaspect

ofaservice,takingintoaccounttheimportanceattachedtoeachaspect,inordertoderive

prioritiesforserviceimprovement.Inoutline,thetechniqueisusedasfollows:

■■ respondentsarepresentedwithalistoffactorsegcost,valueformoney,easeofaccess,

breadthofcoverageetc.;

■■ respondentsgiveasatisfactionratingandanimportanceratingforeachfactor,usually

accordingtoanumericalscale;

■■ foreachrespondent,theimportancescoreforeachfactorissubtractedfromthe

satisfactionscoreforthatfactor:thisisthegap;

■■ thegapisweightedaccordingtotherelativeimportanceofeachfactor.

5.4.2 Thefollowingtableillustratestheprocess.

Table 5‑1weighted gap analysis

Factor Importance rating(1-10)

Satisfaction rating(1-10)

Gap Importance co-efficient

Weighted gap

Priority

Cost 9 6 3 90% 2.7 3

Value for money 10 8 2 100% 2 5

Ease of access 8 3 5 80% 4 1

24/7 access 5 7 -2(effectively,0)

50% 0 8

Online access 9 8 1 90% 0.9 6

Ease of website navigation

8 4 4 80% 3.2 2

Help and advice 7 4 3 70% 2.1 4

Innovative use of technology

6 5 1 40% 0.4 7

5.5 Personas and user profiles

Personas5.5.1 Personasareessentiallyanaidtoservicedevelopment.Itiseasiertodesignaserviceto

meettheneedsandexpectationsofonespecificuserthantryingtodesigntoabstract

criteria.Personascanhelptoprovidefocusforservicedevelopmentworkthathasfairlyhigh

level,genericobjectivesegtomakethesiteeasiertouseor‘makethesitemoreappealingto

schoolagechildren’.




The scenario

Anewspaperisredevelopingitswebsiteandneedstoknowwhowilluseitandwhattheywillwantfromit.Theteamemployedtodeveloppersonaslookatresearchonusersofotheronlinenewspapers,demographicdataaboutthenewspaper’sreadershipandpeopleintheregion.Theyalsocarryoutlengthyinterviewswithpeopleintownsandvillagesintheregion,askingabouttheirlifestyle,theirwork,howmuchtheyusecomputers,wheretheygettheirnewsandentertainmentfrom,petgripeswithtechnologyetc.Asaresultofthisprocesstheydevelopfourdistinctpersonas:oneisan‘uncommittedinformationsnacker’calledDan.

Persona #1: the uncommitted information snacker

Danisinhismid-30sandisManagerofHumanResourcesatGobbleandGookinLittleborough.Danisunmarried,butusuallyspend3or4nightsaweekathisgirlfriend’splace.TheygooutfordinneracoupleoftimesamonthandtheirfavouritethingtodoonSundayeveningisgetaDVDandphoneforpizza.ThesedaysDandoesn’tbuyanewspaper–hesometimeslooksatnewspapersonlinebutheoftenusestheBBCsiteashethinksthecoverageislesslikelytobebiased.Helikesbeingabletogetsportsresultsonlineandlikeshowquicklythewebsitecanbeupdated.Hecommunicateswithhismatesbytextmessageandemailshisbrotheraboutonceaweek.Hehasaworkemailaddressandapersonalemailaccount,he’snotthatbotheredaboutspamoraddressspoofing–hedoesn’tdomuchshoppingonlineandreckonstherisksareexaggerated.Heusestheinternetathometocheckfilmtimesandfindnewrestaurants.HeoccasionallybuysbooksfromAmazon.He’saRadioheadfanandthoughtbeingabletopaywhathelikedtodownloadthenewalbumwascool.Danrateshimselfasprettygoodwithcomputers,butisdefinitelynotageek.HeenjoysplayingfootballwithfriendsandhisgoalsinlifearetosaveenoughtobuyahouseandtraveltoSouthAmerica.

Service redevelopment to appeal to Persona #1

ThedesignteamwilldevelopthewebsitetoappealtoDan:

■■ thehomepageofthewebsitewillfeaturemoresportsstories;

■■ thehomepagewillfeaturea‘breakingnews’tickertape–clickingonthiswilltaketheusertothespecificnewsflash;

■■ reviewsforfilmsreleasedonDVDinthelast6monthswillbepostedonlineandthispagewillbelinkedfromboththeFilmpageandtheTVlistings;

■■ itwillbepossibletoviewfilmreviewsaccordingtolocalavailabilityandseescreeningtimesatlocalcinemas;

■■ somenewsstorieswillbetreatedintheroundwithdifferent‘perspectivepieces’toshowwhatthestorywillmeanfordifferentpeopleandcoveragefromothermediasourcestoprovidethe‘balancedcoverage’Danvalues;

■■ thewebsitewillincludea‘today’snewsinpictures’page,withquirkyphotographstoillustratestories–clickingtheimagewilllinktofurthercoverage.

5.5.2 Apersonaisanarchetype,withthepersonaltraits,lifestyle,aspirations,beliefs,needsand

goalsetc.typicalofasegmentoftheaudience.Thekeytosuccessfuluseofpersonasis

developingrichlydetailedpersonasthat:

■■ accuratelyreflectdifferentaudiences;

■■ actasabelievable,realisticstand-inforarealusersothatthedesignteamdevelopa

strongsenseofwhatwillbeappealingandusablefortherealaudience.



5.5.3 Theprocessofdevelopingapersonaisoftenquiteinvolved,includingdetailedinterviews

withpotentialusers,ethnographicstudies,novelactivities–anythingwhichwillhelptofoster

agoodunderstandingoftheaudience.Interviewingforpersonaresearchisaskilledand

lengthyprocess;theinterviewerneedstobuildagoodrapportwiththeintervieweeinorderto

elicitverydetailedandpersonalinformation.Constructingapersonafromalltheinformation

thathasbeengatheredisoftendoneasaworkshopactivity.

5.5.4 Differentpersonascanbedevelopedtorepresenttheaudiencesegmentswithdifferent

relationshipstoaservice.Thereshouldusuallybeaprimarypersona,representingthehigh

priorityaudienceforthedevelopmentactivitybeingundertaken.Sometimesitishelpfulto

includea‘negativepersona’,anarchetypeforwhomtheserviceisnotintended.

User profiles5.5.5 Ifresourcesdonotpermitthecreationoffullyfledgedpersonasitmaybepossibletoachieve

someofthesamebenefitsbydevelopinguser profiles–descriptionsoftypicalusers,

orientedtotheserviceunderconsiderationandfocusingoninteractionswiththeservice,

needs,expectationsandattitudestowardstheservice.


Alargeteachinghospitalisevaluatingitslibraryservicesanddevelopsaseriesofuserprofilestoensurethattheservicesmeettheneedsofthedifferentusers.

Oneoftheprofilesisfora‘medicalspecialistintraining’:

■■ needsaccesstospecialistjournalarticlesandtextbooks;

■■ preferselectronicresourcestopaper-basedresources;

■■ willwantaccesstocontentoffsite(forhomestudyandrevision);

■■ needsaccesstoresourcesandservicesoutsidenormalworkinghours;

■■ busyscheduleandchangingworkpatternsmakeflexibility,remoteandmobileaccessakeyrequirement.

6. Making use of Audience Research PAGE 55


6.MakingUseofAudienceResearch

This chapter reinforces the message that audience analysis should be seen in the context of the service as a whole

Audienceanalysiscanprovidemanybenefits,butanydecisionsonuseofthedatawilldependonthewiderservicecontextandpriorities.

Astrategyforrealisingthefullvalueofaudienceanalysisworkisdiscussed.

6.1 Introduction

6.1.1 Audienceresearchisatoolforservicedevelopmentandcanbeusedtoprovidethebenefits

discussedinSection3,howeveraudienceresearchisanaidtodecisionmakingandnot

asubstituteforit.Evidencefromaudienceresearchcanhelpserviceproviderstomake

informeddecisionsaboutspendingondigitisation,newservices,supportfordifferent

platforms,promotingtheservicetonewaudiencesetc.butitisnotasubstituteforbusiness

planningorthepoliticaldecisionsthatneedtobemadeaboutprioritiesforpubliclyfunded

services.

6.1.2 Becausedistinctaudiencesegmentshavedifferentrequirementsitisimportanttounderstand

therelationshipbetweenthemandprioritiseallocationofresourcesappropriately.Audience

analysiscanprovideinformationaboutaudiencesegments,theirrelativesizeandhowthey

use,valueandaccessaservice,butdecisionsaboutresourceallocationstilldependon

servicepriorities.

6.1.3 Decisionsabouthowtodevelopandpromotethewebsitearestillbusinessdecisions,not

leastbecauseuseofthewebsitewilldependonthequalityoftheresource,andthesuccess

ofthemarketingstrategy.Ofcourseaudienceanalysisshouldinformboththeseactivities.


PAGE 56 6. Making use of Audience Research


Amuseumhasdevelopedawebsitewhichprovidesdetailedinformationaboutitemsinthecollections,regularlyupdatedinteractiveonlineexhibitions,ablogaimedatthosewithaprofessionalinterestinthemuseumssectoranddigitisedimagesoftheirmostrenownedexhibits.Audienceresearchshowsthat98%ofvisitorstothesiteonlyviewpageswithtravelinformationandopeningtimes.

Weretheresourcesusedtodevelopthewebsitewasted?Shouldtheexistingwebsitebemaintained?

Theanswerstothesequestionsdependpartlyonwhothetargetaudienceforthewebsiteisandthisisapoliticalandbusinessdecision.Perhapsthemuseumrecognisedthatmostvisitorstothewebsitewouldonlywantbasicinformation,yetstillsoughttoattractasmallaudienceofacademicsandprofessionalswhowouldmakemoreextensiveuseofthesite.

Followupresearchorfurtheranalysisoftheexistingdatamightaddresstheseadditionalquestions:

■■ Hasthesitebeensuccessfulinattractingthisnicheaudience?

■■ Doesthesiteservetheneedsofthisaudience?

Alterationstotheserviceorachangeincommunicationsstrategymightbeneeded.

Otherquestionscannotbeansweredsolelyonthebasisofaudienceresearch:

■■ Shouldthesitebepromotedtovisitorstothephysicalmuseumasapost-visitresource?

■■ Canthemuseumusethewebsitetodevelopanongoingrelationshipwithitsaudienceegallowingthemtovoteforitemstogoondisplay,postcommentsaboutexhibitionsetc.?

6.2 Audience research in context

6.2.1 Attheconclusionofanaudienceresearchproject,whenthedatahasbeencollected,

analysedandinterpretedandconclusionsdrawnitisimportanttomakesurethatthefull

valueoftheresearchisrealised.Forexample:

■■ Theresultsoftheresearchshouldbedisseminatedtostakeholdersandotherinterested

parties.Thismayinvolvepreparingseveraldifferentpresentationsoftheworkfordifferent

audiences,egseniormanagement,funders,generalstaff,communicationsteametc.It

maybeappropriatetoprepareasummaryofthefindingsandfollow-upforparticipants;

peoplearemorewillingtotakepartinresearchiftheyfeeltheircontributionisappropriate

andtheresearchislikelytohaveanimpactontheservice.

■■ Everyonewhohelpedwiththeprojectshouldbethanked;thisincludesstaffwhohelped

tocarryouttheresearch,participantsandanyoneelsewhocontributed.Thiswillpay

dividendswhenfurtheraudienceanalysisworkisundertakenandcontributestoobtaining

buy-inforaudienceanalysisthroughoutaservice.

■■ Considertheimplicationsofthefindingsforservicedevelopmentandaudience

development.

■■ Lookattheconclusionsoftheresearch–werethespecificobjectivesmet?Whatlessons

canbelearnedforfuturework?

■■ Planfollow-upandfutureresearch.

6. Making use of Audience Research PAGE 57


■■ Assessthewiderimplicationsofthework:

■− istheservicebeingprovidedalignedwiththestatedservicemission,valuesandremit?

■− arethereanyimplicationsforlong-termstrategy?

■− isthebusinessplanorsustainabilitystrategyfortheserviceappropriate?

6.2.2 Itmaybeappropriatetosharesomeofyourfindingswithpartnersinthesamesector,other

publicsectororganisationsservingasimilaraudienceetc.

6.2.3 Itshouldalsobenotedthataudienceresearchprojectswilloftenresultinchange–whether

itbedirectlytotheserviceoffered,ortotheorganisationitself(eg asaresultofachangein

companystrategy).Changemustbecarefullymanaged,andwillentailthoughtfulplanning

andsometimessensitiveimplementation,andaboveallengagementwiththepeopleaffected

bythesechanges.Usefulfurtherreadingonthemanagementofchangeinaprogrammatic

contextisManaging Successful Programmes (MSP),OfficeofGovernmentCommerce(OGC),

2007edition.

PAGE 58 Appendix A: Glossary

AppendixAGlossary

Audience development Activitiescarriedouttoincreasetheaudienceforaserviceor

resource.

Audience satisfactionEstablishingiftheaudienceishappywiththeservicethatisbeing

provided.Forexample,isithappywiththequality/content/accessibility/comprehensiveness

oftheservice?Interrogationofaudiencesatisfactionisoftendoneintandemwithaudience

analysis.

Digital service and digital audienceInthisguidedigitalisusedtorefertoresourcesand/or

servicesexploitingdigitisedcontentordeliveredviaadigitalchannel.Inmostinstancesthis

meansaservicewhichisavailableonlinebutitalsoincludesdigitisedresourcesdistributed

onCDandcouldincludedigitaltelevisionserviceswhichsharesomeofthecharacteristicsof

onlinedigitalservices.Digital audiencereferstotheaudienceforadigitalserviceorresource.

Ethnographic researchResearchbasedonobservationsofhumanbehaviour,carriedoutin

a‘naturalistic’setting.Thiscaninclude‘observation’viaautomatedrecordingdevicesandis

sometimesextendedtoincludeobservationsofsubjectsmadeinalaboratory–thecrucial

pointisthatbehaviourinresponsetotherelevantstimulishouldbenormalandthatthe

presenceoftheresearcherortheobservationalequipmentshouldnotinterferewithoraffect

thesubject’sbehaviour.

Focus group Agroupofindividualsselectedandassembledbytheresearchertodiscuss

andcommentonthetopicunderconsideration.Subjects’personalexperienceofthetopic

andinteractionsamongstgroupmembersarethedistinguishingfeaturesofthetechnique.

Formative research Inthecontextofservicedevelopmentanddeliverythistermcovers

researchcarriedoutduringservicedevelopmentwork,aspartofthedevelopmentprocess,

toprovidefeedbackonthework.

Front end researchInthecontextofservicedevelopmentanddeliverythistermcovers

researchcarriedoutintheplanningandpreparationofaservice.

Impact analysisAnalysisofthelonger-termeffectsofaservice(orgroupofservices)on

itsaudience.Longer-termevaluationmethodsarerequiredtodetermineiftheimpactofa

serviceislasting.

Appendix A: Glossary PAGE 59


Reach Inthisguidereachisusedtorefertothenumberofpeopletowhomaserviceor

resourceisavailable.Differentfromuptake whichreferstousageoftheserviceorresource.

Segmentation Theprocessofdividinganaudienceintodifferentcategoriesorsegments

onthebasisofcriteriarelevanttotheserviceorresourceunderconsideration.Segments

arerelativelyhomogeneousanddistinctfromothersegmentswithrespecttothechosen

characteristics.

Summative research Inthecontextofservicedevelopmentthisreferstoevaluativeresearch

assessingtheextenttowhichaservicehasmetspecificcriteriaforsuccess.

Target audience Theintendedaudienceforaserviceorresource.

UsabilityAmeasureofthequalityoftheuserexperienceofaserviceorresource.Itgenerally

encompassesfactorssuchaseaseoflearning,efficiencyofuse,memorability,error

frequencyandseverityandsubjectivesatisfactionwiththeinterface.

User-centred design Adesignphilosophywhichplacesthegoals,expectationsandskillsof

usersatthecentreofthedesignprocess:theserviceorproductisdesignedtofittheuser.

Typicallyusersareinvolvedthroughoutthedesignanddevelopmentprocess,includinguser

testingoftheserviceorproductwhilstitisindevelopment.

PAGE 60 Appendix B: Reasons to do Audience Research

AppendixBReasonstodoAudienceResearch

This chapter sets out the reasons for doing audience analysis and some of the uses of the research

Experience and instinct are not a substitute for the information that systematic research can provide.

Theusesofaudienceanalysisinclude:

■■ Servicedevelopment.

■■ Serviceevaluation.

■■ Accountabilitytofunders.

■■ Long-termplanning.

■■ Sponsorshipandmarketing.

■■ Businessplanning.

■■ Managementofcontractualrelationships.

■■ Professionaldevelopment.

B.1 Why do audience research?

B.1.1 Itisincreasinglyimportantforpublicsectorservicesandresourcestobeabletodemonstrate

thattheyareusedandvaluedbyanappropriateaudience.Professionalsoftenfeeltheyknow

‘their’audienceandareconfidentthattheservicetheyprovideismeetingthataudience’s

needs.Experienceandinstinctareoftenasourceofdetailedknowledgeandvaluable

insightsintouserneeds,valuesandbehaviour,howevertheyarenotasubstituteforthemore

completepicturethatsystematicaudienceresearchprovides.

B.1.2 Itisimportanttoremainopen-mindedaboutwhataudienceresearchreveals,andalertto

changesinthenatureandexpectationsoftheaudience.Therapidpaceoftechnological

changemeansthattheaudiencesfordigitalservicesandcontent,andtheexpectationsof

theseaudiences,isalsoevolvingrapidly.

Appendix B: Reasons to do Audience Research PAGE 61


B.1.3 Itseemsself-evident,butit’sworthstatingunambiguouslythatthe more you understand

about your audience, the better you will be able to meet their needs – and demonstrate that

you are meeting their needs.

B.1.4 Asatisfieduserwillspreadthewordaboutagoodserviceandpromoteaudience

development.Satisfiedandengageduserscanbeavaluableresource:theymaybemore

willingtoshareinformationabouttheirattitudesandhabitsaswellastheirexperienceof

yourservice.Thisfeedbackcanbevaluableindevelopingandenhancingyourserviceinthe

future.

B.1.5 Evenasmallaudienceresearchprojectisworthwhile.Smallerservicesoperatingona

restrictedbudgetshouldnotbeintimidatedbytheconceptofaudienceanalysis.Many

techniquescanbeimplementedquitecheaplyoradaptedtoashoestringbudget.Some

insightintoyouraudienceisvastlybetterthannoneatall.

B.2 The uses of audience research

B.2.1 Theusesofaudienceresearchextendbeyondsimplydemonstratingacertainnumberof

visitorsorwebsitehitstotickaboxonaservicelevelagreement.Someoftheareaswhere

youmayexpecttorealisebenefitsaresetoutbelow.

Product and service developmentB.2.2 Thisisperhapstheareaofaudienceresearchwhichappealsmosttoprofessionalsandstaff

whoengagedirectlywiththeaudience.

B.2.3 Inplanninganewserviceitisimportanttoinvestigatewhoislikelytousetheservice,what

theyexpect,theirpreferredmeansofaccessetc.

B.2.4 Involvingtheaudienceinservicedevelopment–includingtheuser-centred designprocesses

–isincreasinglyviewedasanimportantwayofensuringthataserviceorresourceisusable

andusefulforthetargetaudience:directinvolvementofpotentialusersmakesiteasierto

ensurethattheservicewillmeetexpectationsandthatuserswillbeabletoaccomplishtheir

goalwhentheyusetheservice.

B.2.5 Involvingusersiniterativetestingofprototypesorpilotserviceswilloftenuncovernewand

unexpectedinformationaboutuserbehaviouranduserrequirements,orproblemswith

usabilityandaccessibilitythathadnotbeenanticipated.Gettingthisinformationwhilsta

serviceisbeingdevelopedallowsproblemstoberectifiedandimportantnewfunctionality

tobeadded.Itpotentiallyhelpstoavoidlaunchingaservicewhichisanexpensivefailure

becauseitfailstomeetaudienceneeds.


PAGE 62 Appendix B: Reasons to do Audience Research

Service evaluationB.2.6 Satisfiedusersaremorelikelytoreturntoawebsite,recommendaserviceorseekoutother

servicesandresourcesfromthesameprovider.Onlyifevaluativeresearchiscarriedoutwill

informationaboutthesize,natureandsatisfactionoftheaudiencebeobtained.Audience

researchonsatisfactionwiththeservice,perceptionsoftheserviceandattitudestowards

itareanimportantcomponentofevaluatingthe‘success’ofaservice.Audienceanalysis

shouldalsoformpartofananalysisofthelongertermorbroader‘impact’oftheservice.The

resultsofevaluativeresearchcanbefedbackintothenextiterationofservicedevelopment.

B.2.7 Collectingandanalysingdataonwhoisactuallyusingtheservice,whetherusersfindit

appealing,informative,valuableetc.isacriticalpartoftheservicedevelopmentcycle.For

example,itshouldpickupanyremainingtechnicalproblemsorconcernsaboutusabilitythat

needtobeaddressed,oritmightrevealthattheaudiencearefindingnovelwaystousethe

servicethatcouldbefurtherexploited.Evaluativeaudienceresearchwillprovideevidence

onhowsuccessfulyourservicedevelopmentprojecthasbeenandhowappropriateand

effectiveanyrelatedcommunicationsandmarketingeffortwas.

Accountability to fundersB.2.8 Publiclyfundedservicesareincreasinglyexpectedtoshowthattheyaredelivering‘valuefor

money’.Audienceresearchshouldprovideimportantevidence:numberofusers,frequency

ofuse,qualityoftheuserexperience,valueor‘impact’oftheserviceetc.Audienceresearch

shouldbetailoredtoshowthataparticularserviceismeetingitsownspecificobjectives.

B.2.9 Ifyouraudienceissmall,youmaynonethelessbeabletoshowthatyourserviceis

worthwhileegifaudienceresearchdemonstratesthatitis:

■■ usedbyahard-to-reachsegment;

■■ highlyvaluedbyanicheaudienceorbyparticularlyinfluentialusers.

B.2.10 Knowledgeoftheseaudiencescanbeusedmorewidelyegtoimproveservicedeliveryfor

thehard-to-reachsegmentorextendingserviceprovisionfornicheaudiencesthroughbetter

understandingoftheirspecialcontentand/ortechnicalrequirements.

B.2.11 Achievinggoodreachanduptakemaybeimportantforpubliclyfundedservices.Information

aboutwhonon-usersareandwhytheyarenotusingyourserviceprovidesusefulevidence

onwhichtobaseastrategytoincreasethereachoftheserviceandcanhelptomakeany

requestforadditionalfundingmorepersuasive.

Long-term planningB.2.12 Awiderangeofaudienceresearchmayberelevant,includingdataondemographicand

socialtrends.Longitudinalsurveydata,iethesameorverysimilarinformationcollected

atseveraltimepoints,isoftenpubliclyavailable.Thiskindof‘audienceanalysis’canbe

usefulforlong-termplanning.Itcanhelpservicesanticipatechangesinaudienceprofile,

lifestyle,habitsandpreferredplatforms(egincreasinguseofmobiletechnologies)andadjust

accordingly.

Appendix B: Reasons to do Audience Research PAGE 63


B.2.13 Aservice’sownlongitudinaldataareevenmoreusefulforspottingandtrackingemerging

trends.Ifadeclineinusebyaparticularaudiencesegmentisspottedatanearlystagethere

ismuchgreaterscopeforinvestigatingandpotentiallyreversingthedeclinebyadjustingthe

service.Similarly,longitudinaldatarevealingtheemergenceofanewaudiencecanbeused

toplanservicedevelopmenttailoredtotheirneedsifappropriate.Audiencesuggestionsand

responsestoqueriesaboutpotentialserviceenhancementcanbefollowedup.

Sponsorship and marketingB.2.14 Goodevidenceaboutyouraudienceisveryvaluablewhenitcomestoattractingsponsorship

ormarketingyourservice.Apotentialsponsorismorelikelytocommitifyouareabletoshow

thatyouraudienceprofileisalignedwiththesponsor’sclientbaseandvalues.Aneffective

marketingandcommunicationsstrategywilldrawoninformationaboutexistingandtarget

audiencestomakesuremessagesareappealingandpresentedviaappropriatemediaand

platforms.

Business planningB.2.15 Audienceresearch(egbasedonsurveysofatargetaudienceincombinationwithengaging

theexistingaudienceandstakeholders)isofgreatassistanceinassessingthefeasibilityofa

plannednewserviceorserviceenhancement.Informationaboutthelevelofinterestamongst

thetargetaudience,likelyaudiencelevelsandaudienceexpectationsshouldformapartof

thebusinesscaseaswellasinformingservicedesignandmarketing.

Management of contractual relationshipsB.2.16 Wheresomeaspectofserviceprovisionisoutsourced(egredesignofawebsite)itmaybe

helpfultospecifythatacertainsizeofaudience,orlevelofaudiencesatisfactionorimpact,

isachieved.Itwillbeimportanttosetoutclearlyhowaudiencesizeandexperiencearetobe

measuredandwhatthecriteriaforsuccessare.

B.2.17 Thesameprinciplecanbeappliedinternally.Definingobjectivesforaprojectintermsofthe

audienceorincludingaudience-relatedmeasuresofsuccessinevaluationsmaybeuseful

waysofimplementingtheservice-orientedvaluesofapubliclyfundedorganisation.

Professional developmentB.2.18 Professionals’richanddetailedknowledgeaboutserviceusersisvaluableandcanoften

betheinspirationfornewinitiativesandplayavitalpartinservicedevelopment.Formal

audienceresearchshouldbeseenasawayofsupplementingandenlargingprofessional

knowledge.Involvingstaffinaudienceresearchmayhelptofocusresearchandcanbe

empoweringforstaff.

PAGE 64 Appendix C: Digital Audiences and Digital Services

AppendixCDigitalAudiencesandDigitalServices

This chapter sets out some of the differences between digital and non-digital services and audiences, and discusses the implications for audience analysis

The size, composition, attitudes and behaviour of the audience may be different in a digital environment.

■■ Digitalcontentisdifferentfromnon-digitalcontentandthishasimplicationsfortypesofservicethatcanbeoffered.

■■ Therelationshipbetweendigitalandnon-digitalworldsisevolvingandservicesshouldbeawareoftheimplications.

■■ Adoptionofnewtechnologiesshouldbedrivenbyaudienceandserviceneedsandexpectations.

■■ Differenttechniquesmaybeusedforanalysisofdigitalaudiences.

C.1 The digital revolution

C.1.1 Thedigitalrevolutionhasresultedinanexplosionofpossibilities:newandinnovative

services,adelugeofcontent,andpotentiallynewaudiencesandnewusesforoldcontent

mademorewidelyavailablethroughdigitisation.Contentandserviceprovidersareonly

beginningtoexploreandunderstandthescopeandconstraintsofthedigitalenvironment.

Theapproachtodigitalresourcesvariesacrosssectorsandservicesandissubjectto

revisioninresponsetotechnicaldevelopmentsandchangingpatternsofuserbehaviour.

Relationshipsbetweendigitalcontentandnon-digitalcontent,betweendigitalservicesand

non-digitalservicesandbetweena‘digitalaudience’anda‘non-digitalaudience’areanew

andinterestingconsiderationforserviceproviders.

Appendix C: Digital Audiences and Digital Services PAGE 65


C.2 Audiences in the digital environment

C.2.1 Thetargetaudience(seeSection2)foradigitalserviceneednotbethesameasthe

audienceforanon-digitalequivalent.Adigitalserviceprovidesanopportunitytorecruit

anewaudienceaswellasencouragingsomeoralloftheexistingaudiencetoconvertor

extendtheirusage.Thedigitalenvironmentchangesboththedemographiccompositionof

theaudienceandthewaysinwhichtheyinteractwithaserviceorresource.Usersbehave

differentlyinadigitalenvironmentandhavedifferentexpectations:

■■ Acceptanceandfamiliaritywithdigitalservicesisnotuniversalandthisaffectsthe

effectiveavailabilityofdigitalservicesegelderlypeoplemaybeuncomfortableusinga

computertoaccessservices.

■■ Accesstotheplatformsusedtodeliverdigitalservicesisvariableandthereforeaccessto

digitalservicesisalsovariableegfewerlowincomefamilieshaveaccesstobroadband

internetathome,orlaptopsformobileaccess.

■■ Accesstophysicalservicesisunevenlydistributedacrossthepopulation.

■■ Theaudiencecanaccessdigitalservicesinmultiplecontexts eglaptop,mobiletelephone,

computerathome,computeratwork,computerinapubliclibrary.

■■ Communicatingwiththeaudienceisgenerallyeasierandcheaper.

■■ Socialnormsandattitudestoprivacyaredifferent.

■■ Usersareoftenless‘committed’toaresourcetheychoosetoview–perhapsbecause

accessisrapid,orbecausemultiple,apparentlyverysimilarresourcesrelevanttothe

user’sobjectiveareavailable.

■■ Resourcediscoverystrategiesfordigitalcontentarelesssophisticatedamongstcertain

users.Thisisnotwellunderstood,butstudieshavefoundthatstudentstendtorely

onresourcesthatcanbediscoveredquicklythroughGoogle,10whilstmoreexpert,

experiencedresearchershavehigherexpectationsofresourcequality,whichtheypursue

throughmoresophisticated,multi-layereddiscoverystrategies.

■■ Theaudienceforadigitalserviceisoftenoperatinginanenvironmentwhichmeansthat

supportislessreadilyavailableegathome,outsideofficehours,wherenohelpdeskis

immediatelyavailabletoprovideadviceandguidance.

■■ Concentrationspantendstobeshorterinadigitalcontext11.

■■ Competitionfortheattentionoftheuserisgreaterinanonlineenvironmenteginstead

ofbeingfocusedsolelyonselecting,findingandusingarticlesinaphysicallibrary,the

userissearchingonlineandcanbedistractedbyevents,ormaybeactivelyattemptingto

multi-task.

■■ Physicalconstraintsonaudiencesizenolongerapplyeguserscanaccessanonline

digitisedresourceatanytime,fromanylocation.

■■ Newconstraintsonresourceseglimitationsonbandwidthmayaffectqualityofservice.

■■ Itiseasiertoservenicheaudiences.

10 Information behaviour of the researcher of the future, CIBER briefing paper; Jan 2008

11 What the Internet is doing to our brains – is Google making us stupid? Nicholas Carr, Atlantic Monthly Jul/Aug 2008



C.2.2 Someusersofdigitalserviceswillembracethefullpanoplyofso-called‘Web2.0’

functionality:12theywillreadandperhapsevencontributetoblogs,sharepersonal

photographsonline,usesocialnetworkingsites,subscribetoRSSfeedsforfavouritesites

anduseandcreatetagsastheybrowsedigitalcontent.However,mostoftheaudiencewill

exploitonlyatinyfractionofthisfunctionalityandsomewillbeactivelyuncomfortableinan

environmentwhichseemstoexpectthesebehaviours:theywillnotfeelpartoftheintended

audiencefortheservice.

C.2.3 Itispossibletotailoradigitalserviceorresourcetomeetthevaryingneedsofdifferent

audiencesegments(seesub-section5.4).Althoughtailoringtheservicemightrequirethe

usertoself-identifyasaparticularclassofuser,itcansometimesbebasedoninformation

abouttheuserobtainedfromathirdparty.13

C.3 Digital content is different

C.3.1 Digitisingcontentpresentsanopportunitytodevelopandextendfunctionalitytoprovidea

muchricherresource,buttheremayalsobepenalties.

■■ Digitalresourcespresentnewresourcediscoveryproblemsegplacingmetadataabove

theauthenticationlevelisimportant;searchengineoptimisationcanhaveabigimpacton

thevisibilityofaresource.

■■ Digitalresourcescanbemoreflexible:powerfulsearching,filteringandordering

functionalitycanbeprovided.

■■ Digitisingresourcesfacilitatescomparisonsbetweendatasets.

■■ Althoughmorepowerfulsearchingtoolsareavailablefordigitalresourcesitisarguably

easiertobrowsenon-digitalcontent.Thisleadstotheslightlycounterintuitivepossibility

thatserendipitousdiscoveryofinterestingcontentmaybemorelikelywhenusingnon-

digitalcontent.14

■■ Digitalcontentmaybeplatformand/orsoftwaredependent.

■■ Digitalcontentcanusuallybeaccessedanytime,anywhere.

■■ Itiseasiertoupdateadigitalresource.

■■ Digitalresourcesaremoreamenabletointeractivefunctionalityegwikis,novel

combinationsofresourcestoprovideaddedfunctionality(iemash-ups).

■■ Undercertaincircumstancesthedigitalenvironmentcanbetreatedasanexperimental

laboratory:smalladjustmentstoresourcesorservicescanbemadeanddataonthe

effectcollectedeasilyandrapidly.Thishasledtothenotionthatwebservicescanbein

‘perpetualbeta’ieadigitalresourcecanundergocontinualevolution.

■■ Itispossibletocreatecertainclassesofdigitalresourcesverycheaply:whenresource

commitmenttoanideaforservicedevelopmentorenhancementislower,thepenaltyfor

failureislowerandamoreexperimentalapproachispossible.

12 What is Web 2.0? Ideas, technologies and implications for education Paul Anderson, JISC Technology and Standards Watch Feb. 2007

13 Developing personalisation for the information environment 2; JISC; 2008.

14 Great minds think (too much) alike? The Economist (17th July 2008).



■■ Licensingandcopyrightmaybedifferentforelectronicversionsofaresource(egaccess

toelectronicjournalarticlesmayberestrictedtoauthenticatedmembersofanHE

institution,althoughaccesstothepaperversionofthejournalispossibleforanyvisitorto

alibrarywhereitisheld).

C.4 The relationship between digital and non-digital worlds

C.4.1 Thedifferencesincontextmeanthatattemptingtoreplicateanon-digitalresourceorservice

inthedigitalenvironmentmaynotbeappropriateoreffective.Simplyreplicatingtheexisting

resourcealsoneglectspossibilitiesuniquetothedigitalenvironment.

C.4.2 Digitalaudiencesdonotnecessarilyoverlapcompletelywithnon-digitalaudiences,even

foranostensiblysimilarresource.Inthedigitalenvironmenttheaudiencehasdifferent

expectationsaboutresourcesandwillexpecttointeractwithresourcesindifferentways.


Membersofalibrarywhoaccessthesamepapertextmayhaveverydifferentexpectations.Forexampletheymaybe:lookingupaspecificreference,borrowingforrepeatedreferenceorextendedreading,skimreading,searchingforcontentonaparticulartopic,readingfromcovertocover,checkingpublicationdata,examiningillustrations,browsingwhilstwaiting,comparingwithalternativeeditionsetc.Thepapertextisnottheoptimalformatforallthesetasks,buttheusermustadapttotheformat;inthedigitalworldtheresourcecanalsobeadapted.

Adigitalresourcebasedonthetextcouldbecreatedtakingasthestartingpointtherangeoftasksthetargetaudiencewouldwanttocarryout.Themoreflexibledigitalresourcecanbepresentedindifferentwaysandthefunctionalityextendedtofacilitatetheusestowhichtheaudiencewouldliketoputtheresource.

C.4.3 Audienceresearchcanprovideevidenceofthefunctionalitythatuserswouldappreciate.

Consultingthetargetaudiencemayuncoverunanticipatedrequirements,orconverselymay

indicatethatfeaturestheserviceproviderisconsideringintroducingwouldnotbeused,or

wouldnotbevaluedenoughtojustifythecosts.

C.4.4 Digitalresourcesarestillintheirinfancyanditisalsocommontodiscoverthataresource

isbeingusedinwaysorforpurposesnotanticipatedbythecontentorserviceprovider.

Monitoringuseofadigitalresourceusingdifferentaudienceanalysistechniquesallows

serviceproviderstouncoverunexpectedpatternsofusageandtakeadvantageofany

opportunitiesforservicedevelopmentandaudiencedevelopment.

C.5 Technology should serve the audience

C.5.1 Contentandserviceprovidersoftenfeelanobligationtoexploitthenewpossibilitiesofthe

digitalworldbutareunsurehowtodosoeffectively.Technologyshouldbeusedwhereit

canenhancetheserviceandimprovetheaudienceexperience.Ratherthanaskinghowto

incorporateemergingdigitaltechnologies,applicationsetc,serviceprovidersshouldask

whichbenefitsofthenewtoolstheiraudiencemightappreciate.Digitalservicedevelopment



shouldbedrivenbythemissionoftheservice,ratherthanbyanobligationtoincorporate

emergingapplicationsandformats(egifyouraudiencewantinformationandaforumfor

discussiononhottopicsofthedaybloggingmightbetherightsolution).

C.5.2 Obviouslythisdoesn’tmeanserviceprovidersshouldn’texperimentwithnewpossibilities

(seesub-sectionC.4:experimentationmaybecheaperandeasierinthedigitalenvironment)

butthebusinesscaseshouldstateclearlywhenextensiontoaserviceisofferedonamore

speculativebasisandaudienceresearchwouldbecriticaltodeterminewhetherandhowto

pursuetheidea.

C.5.3 Thereiscurrentlyalotoftalkabout‘Web2.0’,withserviceproviderskeentoknowhow

torespond.RatherthanseeingWeb2.0asasetoftechnologies,itcouldbeviewedasan

approachtoserviceprovisioninthedigitalworld:userscanbecollaboratorsinresource

creation;useofaservicegeneratesdatathatcanbeminedforservicedevelopment;users

participateindesigningtheirexperienceofaservice.

C.6 Relationship between digital and non-digital services

C.6.1 Digitalservicescanbeanalternativeto,complementarytoorsupplementarytoanon-

digitalequivalent.Manyservicesandcontentprovidersnowhaveawebsiteinadditionto

theoriginalphysicalresourceandunderstandingtherelationshipbetweentheaudiences

forthesedifferentservicesisuseful.Ithassometimesbeenassumedthatanonlinesitewill

attractanaudienceforwhomthephysicalresourceisrelativelyinaccessible,butaudience

researchhasshownthatthisisnotnecessarilythecase.


Someserviceshavefoundthatmostvisitorstotheirwebsiteuseittoobtainsimpleinformationaboutopeninghoursandtraveltoassistinplanningavisit.Thissuggeststhatthewebsiteissimplyanalternativemeansofaccessinginformationalsoavailableonpaperorbytelephone.Butthewebsitemaystillbeattractinguniquevisitorstothemuseumievisitorswhowouldnothavediscoveredthemuseumifthewebsitewerenotavailableorwouldnothavechosentovisitwithouthavingaccessedthewebsite,eveniftheymakelittleuseofthewebsite.Uncoveringthiskindofinformationwouldrequireanintegratedaudienceanalysisstrategy,includingaskingtheaudienceforthephysicalmuseumabouttheiruseofthedigitalmuseumresource.

C.6.2 Servicesmaywanttouseaudienceresearchtoexploretherelationshipbetweentheirvarious

audiencesandtailortheiraudiencedevelopmentandservicedevelopmentworkaccordingly:

■■ Theaudienceforaphysicalresourcecouldbeencouragedtobecomepartofthe

audienceforarelateddigitalservice(egbyofferingfurtherinformationaboutexhibits

online,offeringaccesstomorematerialonline,offeringthechancetoinfluencethe

physicalresourceetc.).

■■ Thedigitalservicecouldbeusedtopromoterelatednon-digitalservices(eg anevent

organiseruseswebsites,perhapsincludingsocialnetworkingsites,toprovideinformation

aboutaneventandattractawideraudience).

■■ Itisrecognisedthatthedigitalserviceservesalargelyuniqueaudienceandtheserviceis

tailoredspecificallytotheirneeds,particularlythosethatcannotbemetthroughanon-

digitalservice.



C.7 Digital audiences may be analysed differently

Methods of audience analysis are discussed in more detail belowC.7.1 Audienceanalysisshouldfollowthesamebasicprinciplesregardlessofthetypeofservice

oraudience,howeveradditionalaudienceresearchtechniqueshavebecomeavailablewith

theadventofdigitalservices:webstatistics,onlinesurveysandamuchwiderrangeof

ethnographictechniques.Webstatisticsareavaluablenewtoolforaudienceanalysisasdata

ispotentiallyavailableonallusersofaresourceratherthanjustasmallsample.

C.7.2 Inthephysicalworldethnographicresearchinvolvesdirectobservationofanaudience

walkingthroughamuseum;askingtheindividualtokeepadiaryofusagetobeusedasa

promptinanintervieworcarryingoutacontextualisedinterview.Datacaptureisindirect:

ahumanbeing,eithertheresearcheroraudiencemember,wouldhavetoobserve,codeor

categoriseandperhapsalsorecallatalaterdatetheusagebehaviourofinterest.Thismeans

thatthefinedetailofusagebehaviourismissed,misrememberedorforgottenaltogether.

Additionally,themoreintrusivetheresearcherrecordingbehaviourthemorelikelyitisthathis

orherpresencewillinfluencetheaudience.

C.7.3 Intheeraofdigitalserviceswebanalyticscaptureuserbehaviourdirectlyanduser

observationstudieshavealsochanged.Studiesmaybecarriedoutinaspecial‘viewinglab’

orremotelywiththeusersinalocationoftheirownchoosing.Userscanbeaskedtocarry

outaparticulartaskwitharesource–eitheraccordingtoagivenprotocolorastheyprefer–

orsimplyinvitedtoexploretheresourcefreely.Users’behaviourastheynavigateawebsite

oruseanotherdigitalresourcecanbecapturedautomatically,accuratelyandtoamuch

greaterlevelofdetailbyspeciallywrittencomputerprograms.Navigationpathsthrougha

resource,searchingbehavioursandtheirsuccessorfailure,downloading,viewingtimesfor

filesandpagesetc.canbecapturedforlateranalysisinconjunctionwithusercommentsand

knowledgeoftheirobjectivesorthetasksattempted.

C.7.4 Webanalyticalmethodsareofonlylimiteduseindeterminingthesizeandcompositionofan

audience.Theyprovide,atbest,onlyanapproximateidentificationofusers:detaileddemographic

informationisnotavailableandanyothersegmentationhastobebasedoninference.

C.7.5 Thevalueofwebanalyticalmethodsisthattheycapturedataabouthowasiteisaccessed

andusedwhichallowsanaudienceanalysttomakeinferencesaboutaudiencebehaviour.It

mustofcourseberememberedthattheseareonlyinferencesandinorderfullytounderstand

whatnavigationpaths,pageviewtimesetc.reallymeanintermsofuserexperienceitis

stillimportanttotalktousersabouttheirexperience.Webstatisticscannottellyouauser’s

motivationsorattitudetoaresource.Forexample:

■■ Arepeatvisitcouldbeusedtoinfersatisfactionwiththeservice,butmightinsteadreflect

frustrationatafailuretofulfiltheoriginalobjective.

■■ Searchbehaviourswhichterminateinadownloadorprintrequestcouldbeusedtoinfer

success,buttheusermaystillbefrustratedthatthesearchtooktoolong;someusers

mayprefertoreadthematerialofinterestdirectlyratherthandownloadingorprintingit.

■■ Longpageviewtimesmightindicateinterestinaresourceorsimplybeanartefactof

tabbedbrowsing.Viewingonlyoneortwopagesonasiteissimilarlysubjecttomultiple

interpretations:thesitewasunappealing;thenatureoftheresourcewasn’twhattheuser

expected;theuserwascalledaway;thesitewasn’taccessiblefortheuseretc.

PAGE 70 Appendix D: Bibliography

AppendixDBibliography

D.1 Introduction

D.1.1 Thereisalargevolumeofpublicationsonaudienceresearch.Herewehavehighlightedthose

wehaveusedasasourceofinformationforthedevelopmentofthisguide,andsomewhich

areusefulforfurtherreading.Wealsoincludereferencestoareasnotcoveredinthisguide,

suchasimpactanalysis.

D.2 General audience research

Audience Research in the Australian Cultural Heritage Sector;EvaReussner,EVRSIG,

MuseumsAustralia;May2003

Understanding your Audiences;MLALondon;www.mlalondon.org.uk/renaissance/index.

cfm?NavigationID=401

User needs assessment for the Adaptive Management Portal; B. Banga,E.Landis,T.Tolle,

L.DecambreandF.Phillips;ACMInternationalConferenceProceedingSeries;Vol129

archive,Proceedingsofthe2002annualnationalconferenceonDigitalgovernmentresearch

Separate or Inseparable? – Marketing and visitor studies;http://archive.amol.org.au/evrsig/

pdf/mprpap.pdf

On the trail of the elusive non-user: what research in virtual reference environments reveals,

L.Connaway,M.RadfordandT.Dickey;BulletinoftheAmericanSocietyforInformation

ScienceandTechnology;Vol34(2),123-135;www.asis.org/Bulletin/Dec‑07

D.3 Planning and analysis

Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited

Resources;FreeManagementLibrary;www.managementhelp.org/evaluatn/outcomes.htm

http://www.mlalondon.org.uk/renaissance/index.cfm?NavigationID=401

http://www.mlalondon.org.uk/renaissance/index.cfm?NavigationID=401

http://archive.amol.org.au/evrsig/pdf/mprpap.pdf

http://archive.amol.org.au/evrsig/pdf/mprpap.pdf

http://www.asis.org/Bulletin/Dec-07

http://www.managementhelp.org/evaluatn/outcomes.htm

Appendix D: Bibliography PAGE 71


Knowing your audience and doing market research; www.usa.gov/webcontent/improving/

evaluating/audience.shtml

Statistical Methods in Psychology;DavidC.Howell;WadsworthPublishingCoInc;6thed.

(2006)

Discovering Statistics Using SPSS;AndyField;SagePublicationsLtd.(2005)

Why study users? An environmental scan of use and users of digital resources in humanities

and social sciences undergraduate education;D.Harley,J.HenkeandS.Lawrence;Center

forStudiesinHigherEducation,UniversityofCalifornia,Berkeley(2006)

Classifying response behaviors in web-based surveys;M.BosnjakandT.L.Tuten,Centerfor

SurveyResearchandMethodology(ZUMA);http://jcmc.indiana.edu/vol6/issue3/boznjak.

html

‘Classifying’ physical and online visitors and reflections on ‘flow’[sic];http://amarclk.

blogspot.com/2008/05/classifying‑physical‑and‑online.html

Personas: setting the stage for building usable information sites;A.J.Head;Infotoday.com

(July/August2003);www.onlineinc.com/online/jul03/head.shtml

D.4 Methods

Observing the user experience: a practioner’s guide to user research;MikeKuniavsky;

MorganKaufmannPublishing(2003)

Contextual Design: A Customer-Centered Approach to Systems Designs (Interactive

Technologies);H.BeyerandK.Holtzblatt;MorganKaufmannPublishers;1sted.(1997)

Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and

Techniques (Interactive Technologies),C.CourageandK.Baxter;MorganKaufmann

Publishers(2005)

Usability testing;www.hhs.gov/usability/refine/learnusa.html

Questionnaire design advice sheet;

www.lboro.ac.uk/library/sklls/Advice/QuestionnaireDesign.pdf

Marketing Research;Aaker,KumarandDay;JohnWiley&Sons,Inc.;8thed.(2004)

Technical Implementation of the MINES Survey Methodology;www.libqual.org/documents/

admin/ACRLMINESPlum040705.ppt

Non-visitor research: an important addition to the unknown;HermannSchaefer;Hausder

GerschichtederBundesrepublik,Bonn,Germany

A Guide to Designing and Conducting Visitor Surveys;J.Leones;www.ag.arizona.edu/pubs/

marketing/az1056

http://www.usa.gov/webcontent/improving/evaluating/audience.shtml

http://www.usa.gov/webcontent/improving/evaluating/audience.shtml

http://jcmc.indiana.edu/vol6/issue3/boznjak.html

http://jcmc.indiana.edu/vol6/issue3/boznjak.html

http://amarclk.blogspot.com/2008/05/classifying-physical-and-online.html

http://amarclk.blogspot.com/2008/05/classifying-physical-and-online.html

http://www.onlineinc.com/online/jul03/head.shtml

http://www.hhs.gov/usability/refine/learnusa.html


http://www.libqual.org/documents/admin/ACRLMINESPlum040705.ppt

http://www.libqual.org/documents/admin/ACRLMINESPlum040705.ppt

http://www.ag.arizona.edu/pubs/marketing/az1056

http://www.ag.arizona.edu/pubs/marketing/az1056



Oxford Internet Institute surveys;www.oii.ox.ac.uk/microsites/oxis

Knowing your audience and doing audience research;http://usa.gov/webcontent/improving/

evaluating/audience.shtml

Good Practice Guide for Developers of Cultural Heritage Web Services;www.ukoln.ac.uk/

interop‑focus/gpg/Usability

Focus Groups;LyndaKelly;AustralianMuseumAudienceResearchCentre;Museum

Methods(2001)

Viewpoint – Ethnography and market research,PhillyDesai;InternationalJournalofMarket

Research49(6);MarketResearchSociety

Tools for assessing web site usage;S.Anderson,T.Willard,H.CreechandD.Bakker;

InternationalInstituteforSustainableDevelopment(2001)

On interpreting access statistics: Why web usage statistics are (worse than) meaningless

www.goldmark.org/netrants/webstats

D.5 Service development

Customer-focused government: from policy to delivery;LyntonBarker;PublicServices

ProductivityPanel

Managing Successful Programmes (MSP);OfficeofGovernmentCommerce(OGC)(2007)

D.6 Digital context

Digital consumers: re-shaping the information profession; DaveNicholasandIanRowlands;

FacetPublishing(2008)

Visiting the virtual museum: art and experience online;LianneMcTavishinNewMuseum

TheoryandPractice:AnIntroductioned.J.Marstine;BlackwellPublishingLtd.(2006)

My Advice for Incorporation of Web 2.0 into Museums;http://museumtwo.blogspot.

com/2007/04/backwards‑interview‑my‑advice‑for.html

Why bother with digitisation? Users and using digital requirements;W.Kilbride(2004);http://

ahds.ac.uk/creating/information‑papers/why‑bother‑digitising

Do it yourself search engine optimization;www.theinternetdigest.net/archive/diy‑search‑

engine‑optimization.html

Great minds think (too much) alike?TheEconomist(17thJuly2008)

What is Web 2.0? Ideas, technologies and implications for education;PaulAnderson;JISC

TechnologyandStandardsWatch;Feb.2007

http://www.oii.ox.ac.uk/microsites/oxis

http://usa.gov/webcontent/improving/evaluating/audience.shtml

http://usa.gov/webcontent/improving/evaluating/audience.shtml

http://www.ukoln.ac.uk/interop-focus/gpg/Usability

http://www.ukoln.ac.uk/interop-focus/gpg/Usability

http://www.goldmark.org/netrants/webstats

http://museumtwo.blogspot.com/2007/04/backwards-interview-my-advice-for.html

http://museumtwo.blogspot.com/2007/04/backwards-interview-my-advice-for.html

http://ahds.ac.uk/creating/information-papers/why-bother-digitising

http://ahds.ac.uk/creating/information-papers/why-bother-digitising

http://www.theinternetdigest.net/archive/diy-search-engine-optimization.html

http://www.theinternetdigest.net/archive/diy-search-engine-optimization.html

Appendix D: Bibliography PAGE 73


Information behaviour of the researcher of the future;CIBERbriefingpaper;Jan2008

Developing personalisation for the information environment 2;JISC;2008

D.7 Evaluation and impact

The evaluation of public library online services: measuring impact;PeterBrophy;

thepeoplesnetwork;2002WorkshopSeriesIssuesPapersno.1

Public Value Test: Guidance on the conduct of the PVT;BBCTrust;www.bbc.co.uk/bbctrust/

framework/protocols/compliance.html

Digitised Resources: A Usage and Impact Study,JISC;www.oii.ox.ac.uk/research/project.

cfm?id=51

Evaluation of open access online courses: guidance developed by the Massachussetts

Institute for Technology (MIT);http://ocw.mit.edu/OcwWeb/HowTo/Evaluation‑

Measurement.htm

http://www.bbc.co.uk/bbctrust/framework/protocols/compliance.html

http://www.bbc.co.uk/bbctrust/framework/protocols/compliance.html

http://www.oii.ox.ac.uk/research/project.cfm?id=51

http://www.oii.ox.ac.uk/research/project.cfm?id=51

http://ocw.mit.edu/OcwWeb/HowTo/Evaluation-Measurement.htm

http://ocw.mit.edu/OcwWeb/HowTo/Evaluation-Measurement.htm



This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/uk/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA

This document is available in alternative formats. For more information: www.jisc.ac.uk/contentalliance

Document No: 670 Version 2.1, December 2009


Further information about JISC:Web: www.jisc.ac.ukEmail: [email protected]: +44 (0)117 331 0789

http://creativecommons.org/licenses/by-nc-nd/2.0/uk/

http://creativecommons.org/licenses/by-nc-nd/2.0/uk/

http://www.jisc.ac.uk/contentalliance

Documents

The Guide to Researching Audiences