ExtremeDialogue:SocialmediaTargetAudienceAnalysisandImpactAssessmentsinsupportofcounteringviolentextremism

Anabridgedsummaryreportoffindingsandlessonslearned

www.SecDev.foundation

www.preventviolentextremism.info

March,2016

2

IntroductiontoAbridgedReport1 CantargetedengagementsthroughsocialmediacontributetoreducingtheriskofCanadiansbecominginvolvedinviolentextremism?

Thisresearchquestionbroadlyframedajoint16-monthprojectconductedbytheInstituteofStrategicDialogue,andTheSecDevFoundation,supportedbyPublicSafetyCanada’sKanishkaProgramme.

Launchedin2014,thissocialmediaresearchprojectwasundertakeninsupportoftheInstituteforStrategicDialogue’sCVEeffort–ExtremeDialogue–aneducationalinitiativeinvolvingaseriesofawareness-raisingfilmsthataimsto“reducetheappealofextremismamongyoungpeopleandofferapositivealternativetoincreasingamountsofextremistmaterial…ontheInternet.”2

ThesocialmediaresearchcomponentoftheISD-SecDevjointprojectaimedtoundertakesocialmediaresearchtoaidindisseminationandtrackuptakeoftheExtremeDialogueCVEfilms.3Theeffortwasdefinedbytwoobjectives,namelytodevelopandapply:

• apracticalmethodologyforTargetAudienceAnalysis(TAA)thatwasinformedbySecDev’spriorexperimentationwithapplyingriskfactoranalysistosocialmediaaudiences;and,

• apracticalMeasuresofEffectiveness/ImpactAssessment(MOE/IA)insupportofcounternarrativeeducationresourcesdistributedthroughsocialmediaandaimedatonlinecommunities.

Thesummaryreporthighlightssomeoftheproject’skeylessonslearnedandfindingsinthedevelopmentandapplicationofTAAandIAmethodsandprinciples,asdevelopedinconjunctionwiththespecificExtremeDialogueeffort.

Theprojectwasambitiousandcomplex,involvedconsiderableactiveexperimentation,andreturnedsomeimportantlessonslearned.Perhapsthebiggestoftheseinvolvedthetensionsaroundenhancingtargetedonlinereachandresonance;and,bridgingtheonlineand‘real’worlds:

1. ThepotentialofsocialmediaassistedtargetinginPVEneedstobetemperedbyotherconsiderations.TheTAAmethodologyemployedbytheprojectfocusedononlinegatekeepersandintermediarycommunitiesasameansofidentifyingentrypointsontotargetaudiencecommunities(youth,educators,youthworkers).Theprojectinitiallyexperimentedwithacombinationofopensourcetoolsandmethodologies.However,thesetoolsprovedquitepowerful,capableofgeneratinginformationthatcouldbeusedtoprofileindividualsorcommunities.Theresearchers,concernedaboutprivacyissues,electedtousealesspowerfulapproach,namely,targetaudienceprofilesgeneratedandsoldbysocialmediaplatforms(i.e.,Facebook,YouTube,Twitter)asthesearedeliberatelymyopic,andthusdonotexposeindividualsorgroupstothesamedegreeasmoreadvancedmethodsandtechniques.

1ThisreportisanabridgedversionofamuchlongertechnicalreportproducedbytheInstituteforStrategicDialogueandtheSecDevFoundationasanoutputofthisresearcheffort.Researchersinterestedinadditionaldataandinformationarewelcometocontact:info@secdev.foundation2LaunchedacrossCanadainFebruary2015,ExtremeDialogueproducedaseriesofshortdocumentaryfilmstotellthepersonalstoriesofCanadiansprofoundlyaffectedbyviolentextremism.Thefilmsareaccompaniedbyasetofeducationalresourcesthatcanbeusedwithyoungpeopleinclassroomorcommunitysettings.3ThefilmstellthestoriesofDanielGallant(formerwhitesupremacist)andChristianneBourdreau(motherofDamianClairmontwhowaskilledwhilefightingwithISISinSyria).

3

Takeaway:Advancedsocialmediaresearchtradecraftandtoolsdefinitelyhavethepotentialtoyieldfine-tunedaudiencedataandmetrics,ofpotentiallygreatusefortargetedcampaigns,andforlatertrackingofcampaignresonance(uptake).However,theyalsoriskreturningtoomuchinformationwhichcouldviolaterightstoprivacyandraisewidersocietalconcerns.Existingguidanceandlegislationontheseissuesremainunderdevelopedandrequirere-doubledpolicyengagementandefforts(includingaroundissuesofprotectionofminorsonline).Forfurtherdiscussionoftheseissues,seeourworkunderanearlierKanishka-fundedeffort:Socialmedia,onlinenetworksandthepreventionofviolentextremism:LessonsLearned.

2. Virtualeffortswouldbenefitfromacloserunderstandingofsuccessfulorpromisingrealworldeffortsatcommunityengagementstrategiesandhowthesecanbeadaptedto,andaugmentedinonlinespace.

Thedecision,earlyon,topursueamoregeneralizedaudienceassessmentprotocolmeantthattheresearchambitionsaroundmeasuresofeffectivenesswerelikewisetoneddown.4TheresultsofthemoregeneralTAAwereusedtoinformthetypesofpoststhatwouldlikelybemosteffectivefordisseminationaroundthelaunchoftheExtremeDialoguefilms,andwasalsousefulforcontextualizingofflinenetworkswithinandbetweenvariouscommunitiesintargetedgeographicareas.However,morerefinedimpactassessmentexperimentationwouldrequirebothalongertimeframeandanabilitytofollow-upwithsurveysandassessmentsdeliveredoffline(forexamplecompareassessmentsfromsocialdatawithmoretraditionalsurveymethodsandinterviews).Inthisrespect,whiletheprojectcouldinformwhetheratargetaudiencewasengagedbythecontent,itcouldnotdemonstratetheresonanceorimpactofthatengagementintermsofchangesinattitudeorbehaviororovertime. Withthesecaveatsinmind,themoreconstrainedpilotefforttomaponlineaudiencesanddisseminationresultsforaspecificCVECampaign–ExtremeDialogue–deliveredanumberofkeyguidancepointsusefulforfuturesuchcampaigns,includingontheeffectivenessofdifferentplatformsandmediatypes,aswellaspaidadvertising,organicoutreachandmetrics.TheexperiencealsoyieldedsomerecommendationsforgovernmentsandsocialmediaplatformsonavenuestobettersupportfuturePVEefforts.Theseobservationsaresummarizedattheendofthisabridgedreport.

AboutthisReport ThisreportisanabridgedversionofafinaltechnicalreportsubmittedtoPublicSafetyCanada.Thisversionofthereporthasbeeneditedtohighlightthekeyfindingsandlessonslearnedthatmayhavevalueforawiderpublicaudience,specifically,identifyingengagementwiththemainmechanismsusedbyExtremeDialoguetoreachouttopotentialtargetaudiences.Thefindingsaremostlytacticalandshouldbeunderstoodaslessonslearnedandfindingsspecifictotheproject(anditsconstraints),andthechoiceofmaterialthatwasusedbytheExtremeDialoguewebsite.

Theintendedaudienceforthereportareresearchersandpractitionersthatareactivelyexperimentingwithapplyingriskreductionmethodstoaddressviolentextremismonline.

Forthoseresearcherswhoareinterestedinamoredetailedperspectiveontheroundsofexperimentationcarriedout,wewouldencourageyoutocontactusdirectly:

Kanishka.ISD@secdev.foundation

4SecDevdevelopedinteractivedashboardsthatcentralizedandvisualizedengagementdatafromExtremeDialogues’propertiesonthevariousonlinechannels,whichinformedthecampaign’sdecision-makingonadspendingandallocationoftimeforengagement.SecDevalsodevelopedananalyticaltooltounderstandthenatureofconversationsanddebatesononlinenewsarticlesrelatedtothecampaign.

4

TableofContentsINTRODUCTIONTOABRIDGEDREPORT..........................................................................................2

ABOUTTHISREPORT...............................................................................................................................3

TABLEOFCONTENTS......................................................................................................................4

INTRODUCTION..............................................................................................................................6ONLINECAMPAIGNOVERVIEW.................................................................................................................6PAIDADVERTISING..................................................................................................................................7BUDGET................................................................................................................................................8

SUMMARYFINDINGS.....................................................................................................................9WEBSITE:EXTREMEDIALOGUE.ORG............................................................................................................9

Overview.........................................................................................................................................9UserDemographics........................................................................................................................9UserGeography..............................................................................................................................9UserBehavior...............................................................................................................................10TrafficSources..............................................................................................................................10Findings........................................................................................................................................10

YOUTUBE............................................................................................................................................12Overview.......................................................................................................................................12ViewerRetention..........................................................................................................................12ViewsandViewerRetentionbyVideo..........................................................................................13ViewerDemographics..................................................................................................................13ViewerGeography........................................................................................................................13PlaybackLocations.......................................................................................................................13TrafficSources..............................................................................................................................13ViewerDevices.............................................................................................................................13ViewerEngagement.....................................................................................................................14FilmAnnotations..........................................................................................................................14YouTubeAdvertising.....................................................................................................................14AdTargeting.................................................................................................................................14AdContent....................................................................................................................................14ViewerGeography........................................................................................................................15AdType.........................................................................................................................................15UserAdExposure..........................................................................................................................15ViewerEngagement.....................................................................................................................16Findings........................................................................................................................................16

FACEBOOK...........................................................................................................................................17Overview.......................................................................................................................................17UserEngagement.........................................................................................................................17UserDemographics......................................................................................................................17UserLocations..............................................................................................................................18PostStats......................................................................................................................................18FacebookAdvertising...................................................................................................................18Reach&UserAdExposure...........................................................................................................18AdPlacement...............................................................................................................................19AdContent....................................................................................................................................19AdTargeting.................................................................................................................................19BoostedPostsvs.Ads...................................................................................................................20FacebookNetworkAssessment....................................................................................................20Findings........................................................................................................................................21

5

TWITTER.............................................................................................................................................22Overview.......................................................................................................................................22TwitterNetworkAssessment........................................................................................................23Findings........................................................................................................................................25

NEWSMEDIACOMMENTSANALYSIS........................................................................................................26NewsCommentThemes...............................................................................................................28Findings........................................................................................................................................29

YOUTUBECOMMENTS...........................................................................................................................30FACEBOOKCOMMENTS..........................................................................................................................31FAR-RIGHTREACTION............................................................................................................................31

CONCLUSIONS&RECOMMENDATIONS........................................................................................33CAMPAIGNOVERVIEW...........................................................................................................................33FINALCAMPAIGNASSESSMENT...............................................................................................................35CAMPAIGNINGBESTPRACTICE&LESSONSLEARNED...................................................................................37RECOMMENDATIONSFORGOVERNMENTS&SOCIALMEDIAPLATFORMS.......................................................39

GLOSSARY....................................................................................................................................40

EXTREMEDIALOGUE:PROJECT&PARTNERS................................................................................41

6

INTRODUCTION

The Extreme Dialogue awareness-raising film series was launched in Canada at the University ofCalgary on the 17th February 2015. ISD conducted an online dissemination campaign using acombinationofpaidandunpaidmethodstodistributethefilmsandattractuserstothewebsiteandsocialmediaaccountstoensurethattheprojectoutputswerewidelyconsumedandsharedonline.Aswell as encouraging activity and engagementwith content on socialmedia, the campaign alsoaimed to raise awareness of the project to awider audience through positivemedia coverage toamplifythecounter-narrativeembodiedinthefilmsthemselves.

Over the launch period ISD and SecDev tracked online engagement with the Extreme Dialoguecampaignacrossitswebsiteandsocialmediaaccounts.Usingthesocialmediaanalyticsdashboardcreated for the campaign (allowing for enhanced data collection and analytics capabilities), andanalytics informationprovidedby ExtremeDialogue’swebsite and socialmedia accounts, ISD andSecDevwereabletoanalysetheeffectivenessofthecampaign.Evaluatingthesuccess(andfailures)ofcounter-narrativecampaignsiscrucialaspractitionersinthisrelativelynewfieldarestillworkingto identify best practices and learning through active experimentation how to use onlinedisseminationandadvertisingtoolseffectively.

Bothquantitativeandqualitativemeasureswereusedinordertoeffectivelyassessthecampaign;

§ Reachandawarenessmetrics:aretargetaudiencesawareoftheproject, is iteasilyfoundonline,howmanyusersofthewebsiteandsocialmediachannelsandviewersofthefilmsarethere?

§ Engagementmetrics:howmuchdidusersinteractwithExtremeDialoguecontent,websiteandsocialmediaaccounts?

§ Impact indicators: does ExtremeDialogue content andmedia coverage encourage criticalthinkingandsparkdiscussionsaroundviolentextremismamongtargetaudiences?

Thisreportexaminesthefindingsofthecampaignanalysisintermsoftheeffectivenessofthevariousplatformsandthepaidandorganicdisseminationtacticsemployedingeneratingreachandawarenessanddrivingengagement. Itthenanalysesthequalitativedataavailabletoassessthe impactofthecampaignonaudiences,beforeconcludingwithafinalcampaignassessment,bestpracticesforfuturecounter-narrativecampaignsandrecommendationsforgovernmentsandsocialmediaplatforms.

OnlineCampaignOverviewPriortothelaunchoftheprojectISDdefinedaseriesofobjectivesthattheonlinecampaignhopedtoachieve:

1. Aminimumof10%ofviewsof,orinteractionswith,theprojectcontentstemfromthetargetaudiences.

2. Videocontentiswidelysharedandconsumedacrosssocialmedia.

3. Coverageinofflinemediaispositiveandhelpstoamplifythecounter-narrativesembodiedintheoriginalassets.

4. Sufficientdataiscollectedtoenablesolidconclusionstobemadearoundtheeffectivenessofplatforms,tacticsandspendinginreachingtheintendedtargetaudience(s),andtoallowforanalysisofengagementandimpact.

7

Therewereanumberofdifferenttargetaudiencegroups(bothcore&halo)thatwereconsideredkeytothesuccessoftheproject;

§ Core:Youngpeopleaged14-25.Thisagegroupwereconsideredtheprimarytargetaudienceoftheprojectastheyaretypicallythemost‘at-risk’toradicalisation.(Note:theeducationalresources are aimed at 14-18 year olds but the filmswere considered to have value as astandalonecounter-narrativeforolderaudiences)

§ Halo:TeachersandparentsacrossCanada.Teachersweretargetedastheprimaryintendedusersoftheeducationalresources,andparentsinordertoraiseawarenessofradicalisationandtoenlistpotentialadvocatesfortheuseofthefilmsandresourceswithyoungpeople.

It was also decided that the campaign would focus on specific cities within Canada that haveexperiencedproblemswithviolentextremisminrecentyears;

§ Ottawa(ON)§ London(ON)§ Windsor(ON)§ Calgary(AB)

AmorelimitedgeographicalfocusalsoenabledSecDevtotracktheflowofExtremeDialoguecontentthroughonlinesocialnetworks inLondon,WindsorandOttawabasedontheresultsof theTargetAudienceAnalysisphaseconductedpriortothelaunch.(Note:Calgarywasaddedasatargetareaatalaterstage)

TheprojectprimarilyusedFacebook,Twitter&YouTubeastheyarethethreemostpopularsocialnetworks inCanada.5Videocontentwashostedon theprojectYouTubechannelwithappropriatetitles, descriptions and keyword tags added toensure theyappeared in theappropriate searches.FacebookandTwitterwereusedprimarilytosharecontentandmediacoverage,advertisethelauncheventandinteractwithfollowerstobuildtheproject’sreachandonlinecommunities.

PaidAdvertisingPaid online advertising allowed ISD to directly reach target audiences using self-reporteddemographic, geographic, occupational, and interestsdataas target criteria. Paidadvertising tookplace on YouTube and Facebook as these have proven to be the most effective platforms fordisseminatingcounter-narrativecontentduringpreviousISDcampaigns.

Thefollowingtypesofadvertisingwereused;

§ YouTubeIn-displayads(videodisplayednexttootherYouTubecontentinside-bar)

§ YouTubeIn-streamads (videodisplayedpriortootherYouTubecontent, foracompulsoryminimumof5seconds)

§ Facebookboostedposts(postsbytheExtremeDialoguepagearepromotedwithintargetedaudiences’newsfeeds)

§ Facebookpaidads (text/imageadvertsappear intargetedaudiences’newsfeedsandside-bars)

5Alexa.com,“TopSitesinCanada”-http://www.alexa.com/topsites/countries/CA

8

BudgetThebudgetwasallocated toYouTubeandFacebookoverTwitter (andGoogle)adsbasedon theirrelativeeffectivenessandvalueformoneyduringpreviousISD-ledcampaigns.AsthecampaignwasmostlypromotingvideocontentYouTubewasprioritisedoverFacebook.ThebudgetwasallocatedevenlybetweenpromotingtheChristianneBoudreauandDanielGallantfilms.

Theadvertisingbudgetwasprovisionallyallocatedasfollows,althoughtheallocationswereflexiblein order to allow for alterations based on the relative success of different ad types during thecampaigningperiod;

ProjectedAdBudget(Audience) ProjectedAdBudget(AdType)

Youth60%

Teachers20%

Parents20%

FBBP7%

FBAd13%

YTIn-D40%

YTIn-S40%

9

SUMMARYFINDINGS

Website:extremedialogue.org

OverviewThewebsiteactedasacentralhubfortheproject,hostingthefilmsandresourcesandlinkingtothethreesocialmediaaccounts.

• Extremedialogue.orgwasvisited4460timesby3707usersoverthefirstweekofthecampaign.

ExtremeDialogueWebsiteUsers

• Trafficpeakedonthe18thFebruaryafterthelaunchitself,thenagainovertheweekendonthe20thand21st,beforetailingoffthefollowingweek.

• 82.3%ofvisitswerenew,while17.7%hadpreviouslyvisitedthesiteeitherduringtheweek(orthe“ComingSoon”holdingpagethatwasinplaceuntilthelaunchonthe17thFebruary).

• Thesevisitsresultedinatotalof21,657pageviewswhichequatestoanaverageof4.86pagespersession.

• Therewerealso1,415viewsofthefilmsonthewebsite.

• Anaveragesessiononthewebsitelasted02:51minutes.

UserDemographics• Fromtheavailabledatathesitewasmostpopularamong25-34yearolds(27%),but

attractedarangeofdifferentagedusers.

• Thesignificant35-64cohort(51%)shouldcontainasignificantproportionofparents.

• Thesmallernumberof18-24yearolds(9%),thelowestproportionofanyagegroup,issomewhatdisappointing.

UserGeography• Thesitewasvisitedbyusersinatotal95countriesworldwide,withthevastmajorityfrom

Canada(55%),theU.S.(24%)andtheUK(6%).

10

• ThemajorCanadiancitiesdominatedthemostpopularuserlocations,withCalgary(6.8%),Toronto(6.4%),Montreal(5.94%)andOttawa(5.1%)leadingtheway.

• Canadiansmadeatotalof3,727visitstotheExtremeDialogueWebsite.

• VisitorsprimarilyusedthewebsiteinEnglish(80.9%),withFrench(11.6%)representingasmallerbutsignificantproportionofvisitors.

• ThepercentageofnewsessionsfromCanadiansat98%wassignificantlyhigherthantheinternationalaverageat80.52%.

UserBehavior• Mostusersaccessedthesiteviacomputer(61.52%),mobilephones(25.70%)andtablets

(12.78%)justifyingthedecisiontobuildthesitewithmobilecompatibility.

• 4,460usersstartedatthehomepage(77.6%)andfilteredoutfromtheretootherpages.

• Thehomepagerepresentsthelargestproportionoftotalpageviews(44.5%).

• The“DanielGallant”(14.3%)andthe“About”(12%)pagesoutperformedthe“ChristianneBoudreau”(10.5%)and“GetInvolved”(7.9%)pages.

TrafficSources• 38%ofusersarrivedirectlyatthewebsiteviatheURL.

• Thoseoriginatingfromelsewherearriveviareferrallinksonexternalwebsites(27%),searchengines(20%)orsocialmedia(15%).

• Mostofthereferralsfromexternalwebsitesoriginatedfromlinksinmediacoverage(85%),withtherestcomingfromtheISDwebsite(15%).

• SearchenginetrafficwasdominatedbyGoogle(91%),withBing(5%)andYahoo(4%)trailingconsiderably.

• OnsocialmediaTwitter(59%)wasthetopperformer,followedbyFacebook(38%),withtheremaindercomingfromotherssuchasLinkedIn,YouTubeandGoogle+(3%).

• Thosethatcametothesiteviasearchesalmostexclusivelyusedthekeywords“ExtremeDialogue”(99.9%)orvariationsofit(i.e.addedspaces,“.org”orusedincorrectspellings)tofindthesite,suggestingthatthenameoftheprojectismemorableanduserslargelyknewwhattheywerelookingforandwereabletofindit.

Findings• Byattractingvisitorsfromover95countriesworldwidethewebsiteachievedsignificant

globalreachandexposurefortheproject.However,thewebsite’smostengageduserswereCanadian.

• Thebouncesratesanduserbehaviourstatisticsachievedbythewebsitesuggeststhatitishighlyeffectiveathookingusersandmaintainingtheirinterestorcuriosity,withthemajorityofusersvisitingthemajorityofpages.

11

• Amajorityofusersalsoknewwhattheywerelookingforandwereabletofinditeasily,withalargeproportionofusersarrivingdirectlyorviasearches(57.6%)pointtoeffectiveSEO.

• Theproject’ssocialmediaaccountsandlinksfromotherwebsites(42.4%)werealsoimportant.

• Thefactthat91%ofuserswere25yearsofageoroldersuggestthewebsitewasmosteffectiveforreachingthehaloaudienceofteachersandparents(aswellasotherssuchasjournalists,educationandCVEpractitioners).

• Thedecisiontobuildamobileandtabletcompatiblewebsitewasjustifiedbythevolumeofvisitorsthatmadeuseofthisfunctionality(38.5%).

• Overall,thefindingssuggestthatthewebsitewaseffectivelydesignedtopromotetheproject,encourageuserengagementandappearreadilyinsearchresults.

12

YouTube

OverviewYouTubeactedastheprimaryplatformfortheproject’svideocontent.ThefilmswerehostedonYouTubeandthenembeddedonthewebsite(andinsomenewsarticles)andsharedonsocialmedia.ThisapproachallowedISDandSecDevtotraceallviewsofthefilms,regardlessofwhethertheywereviewedonYouTubeorelsewhereonline.

ThefilmsontheExtremeDialogueYouTubechannelwereviewed50,673timesduringthefirstweekofthecampaign.Thisrepresentsatotalof2,933hoursor122daysofviewingtimeandanimpressiveaverageviewingtimeof3:30minutes(withfilmsrangingfrom7.5to9minutes).Asthegraphbelowshows,thevastmajorityoftheseviewswereaccruedduringthefirstfourdaysthatthefilmswereavailableandpaidadvertisingwasrunning.

ViewsofExtremeDialogueVideosonYouTube

ViewerRetention• Unpaidviewers,onaverage,watchedthefilmslongerthanthoseattractedthrough

advertising.

• Astotalviewsbegantodropoffafterthe18thFebruary(above),averageviewingtimesbegantoincrease(below).

AverageViewDuration(%ofav.filmlengthviewed)

13

ViewsandViewerRetentionbyVideo• ChristianneBoudreau”and“DanielGallant(Explicit)”werebyfarthemostpopularfilmsas

theyweresupportedbypaidadvertisingandthereforeaccountedfor92%oftotalviewsonthechannel.

• “ChristianneInterviewI”and“DanielGallant(Clean)”werethenextmostpopular,receiving3.5%and1.1%respectively.Theremainingfilmsallreceived0.5%orlessofthetotalviews.

• TheEnglishlanguagefilmsoutperformedtheirFrenchequivalentsacrosstheboardintermsofviews.

ViewerDemographics• ThevastmajorityofviewersinCanadafellintothe18-24category(78%)astheseviewers

weretargetedthroughpaidadvertising.

• 35-44yearsoldformedthenextlargestcohort(7.9%),followedby25034(6%)and45-54(3.9%).

• AsignificantmajorityofviewersontheYouTubechannelweremale(68%).

• Whilethefactthatonly1.5%ofviewscamefromthoseunder18isdisappointing,itissomewhatexpectedasitisnotpossibletotargetadvertisingatthatdemographic.

ViewerGeography• Duringthefirstweekofthecampaign92%ofviewers(46,760)werelocatedinCanada.

• TheremainingviewerswerespreadmostlybetweentheU.S.(4.1%-2,090)andtheU.K.(1.2%-616).

PlaybackLocations• OnYouTubefilmswereviewedpredominantlyoneithertheirown“watchpage”(i.e.the

uniquepage/URLthatcontainsthefilm)orviain-streamads.

• Theselocationsachievedaverageviewdurations(3.08&3.25respectively)similartotheoverallaverage(3.30)astheycollectivelyprovided95%ofthetotalviews.

• 5.4%ofviewsthatweregainedviaembeddedvideosonotherwebsites(primarilyextremedialogue.organdmediaorganisations)achievedaconsiderablyhigheraverageviewdurationof5.35,suggestingthatthoseviewersweremoreengagedwiththecontentofthefilmsthanthoseattractedthroughadvertising.

TrafficSources• Advertisingaccountedfor89%oftotalviewsduringthefirstweekofthecampaign.

• ThisproportionofpaidviewsalsocorrespondswithfindingsonpreviouscampaignsthatISDhasrun,whereadsaccountedfor88%oftotalviews.

• Thefactthattheproportionofminuteswatchedfromadvertisingwasonly86%reinforcesthefindingsthatviewsachievedviaadvertisingareofmarginallylower‘quality’thanunpaidviews,andthattheseviewersareslightlylessengagedwiththecontent.

ViewerDevices• YouTubeviewerslargelymirroredvisitorstotheExtremeDialoguewebsiteintermsofthe

devicesthatwereused.

14

• Computersaccountedfor60%ofYouTubeviews,withmobilephones(23%),tablets(14%)andgamesconsoles(2.2%)providingtherestthatcouldbeidentified.

ViewerEngagement• TheExtremeDialogueYouTubechannelattractedanettotalof73subscribers(78

subscribed,5unsubscribed)aswellas53likes,6dislikesand6favourites.

• Thefilmswereshared55times,addedbyuserstoplaylists43timesandreceived22comments.

• The“ChristianneBoudreau”filmwasthetopindividualperformer,garnering30likes,14commentsand25sharesalone.

• Facebookwasthedominantplatformforsharingthefilms,accountingfor81%ofallshares,withTwitter,Google+,Gmail,WhatsAppandTumblralsobeingused.

FilmAnnotations• Theannotationsaddedatthestartandendofeachfilmgarneredatotalof189clicksand821

closes,andonlyaccountedfor0.3%oftotalviews.

• This represents a 0.75% click-through rate and a close rate of 2.89%, suggesting that theannotations were not an especially effective method for driving additional views andviewerswerealmostfourtimesmorelikelytoclosetheannotationsthanclickthem.

YouTubeAdvertisingThemainfilmswereadvertisedtoyouthintargetcities(Calgary,London,Ottawa,andWindsor)andparentsandteachersacrossCanadausingbothIn-streamandIn-displayYouTubeads.Between16th-20thFebruaryatotalof24individualadswereused,enablingacomparisonofrelativeperformanceandanindicationofwhichaudiencesweremostreceptivetothefilms,perview.Theseviewsweretheresultofover2.3millionimpressions(i.e.thenumberoftimesadsappearedonusersscreens),andanaverageviewrate(i.e.thepercentageofpeoplewhowatchedyourvideoaftertheyfirstsawthead)of1.69.6Advertsalsogeneratedanadditional1,962earnedviews(i.e.whenauserviewsthevideoadandthenwatchesanothervideoorvideosontheExtremeDialogueYouTubechannel).

AdTargeting• Whilst viewvolumes largely correlatedwithad spending, interestingly theads targetedat

youthachievedahigheraverageviewrate(2.30%)thatthosetargetedatteachers(1.27%)andparents(1.26%).

• Thissuggeststhatyoungerusersweremorelikelytowatchthefilmsthateitherteachersorparents.

AdContent• Whencomparingthetwofilms,theChristianneBoudreaufilmachievedahighernumberof

viewsandimpressionsthantheDanielGallantfilm.

• Both were allocated the same original budget but this was exhausted quicker by theChristiannefilmastheadswereservedtomoreusers.Itremainsunclearwhyadsfordifferentcontent(inthiscasethetwofilms)butwithidenticaltargetingcriteriaareservedatdifferentrates.

6TherewasalargedisparitybetweentheviewratesfromIn-streamandIn-displayadvertisingsotheaveragerateissomewhatunrepresentative.

15

ViewerGeography• Ads aimedat youngpeople in the four target cities resulted in significantly different view

rates,with London (9.43%),Windsor (7.53%) and Calgary (5.05%) far outstrippingOttawa(1.04%)andtheadstargetedatparentsandteachersacrossCanada(1.2%).

AdTypeIn-stream ads proved considerablymore effective than In-display ads, accounting for 89% of alladvertisingviewsandadisproportionatelyhigh98%ofthetotalminutesusersspentwatchingthefilms.Thiswasreflectedintheaverageviewdurationof40.5%forIn-streamandadisappointing4.4%forIn-displayads.

In-displaysearchads In-displaysidebarads In-displaybanner/overlayads

TheviewratesforIn-streamadsrangedbetween11.83and25.89%,whereasIn-displayadsperformedextremelypoorlyincomparison,withviewratesbetween0.09and0.24%.In-displayadswerealsotwiceasexpensiveperview(£0.12),suggestingthatin-displayadsarebothmoreexpensiveperviewandlesslikelytoattractviewsandretainviewers.ThislargelyconfirmspreviousISDfindings.

In-streamads

UserAdExposure• Averageimpressionfrequencyisthenumberoftimesthatanaverageuserisexposedtoa

videoadoveragiventimeperiod.

• ExtremeDialogueadsaveragedanaverageimpressionfrequencyof2.99,meaningthatthetypicaluserwasservedanadalmostthreetimesduringthepaidadperiod.

• Thissuggeststhatadspendingwaseitherconcentratedovertooshortaperiodoftime,orthattheaudiencesthatweretargetedweretoosmallandthereforeover-saturated.

16

ViewerEngagement• Paidadvertisingproducedatotalof39subscribers,24sharesand12likes.These

represented53%oftotalsubscriptions,44%ofsharesand23%oflikes.

Findings• YouTubegeneratedanimpressivelylargenumberofimpressionsandviewsforacounter-

narrativecampaign.

• Althoughadvertisingaccountedfor89%ofviews,unpaidororganicviewsstillnumberedinthethousandswhichcomparesfavourablywiththeall-timeviewsofsomeothercounter-narrativeefforts.

• Thefactthatthemajorityofviewsoriginatedfromadvertising,whileunsurprising,reinforcesthedifficultyofattractingtargetaudiencesorganically,especiallyinashortperiodoftime.

• WhenadvertisingonYouTubeitistypicallybothmoreexpensiveandlesseffectivetotargetbroader,less-specificaudiences.

• Theover-exposureofadstosomeuserswasanunexpectedconsequenceoffront-loadingtheadvertisingbudgetduringthefirstfewdaysofthecampaign.

• AdsgeneratedadisappointinglevelofengagementwiththefilmsandtheYouTubechannelwhencomparedtoorganicviewers,although1,962usersdidgoontoviewmultiplevideos.

• Aswellasproducingadisproportionallysmallernumberoflikes,shares,commentsandchannelsubscriptions,adviewerswerealsomoredifficulttoretain.

• TheExtremeDialoguecampaignhasalsoemphasisedthefactthatviewscanbeamisleadingrepresentationofthesuccessofacampaignunlessqualifiedbyviewerretentionratesandengagementstatistics.

• Thequalityofviewsmustbeconsideredtofullyunderstandtheimpactofcounter-narrativecontentonYouTube.

• OverallYouTubeprovedtobethemostusefulplatformforreachingthetargetedyouthdemographicandrepresentsthemostpromisingavenuefordisseminatingcounter-narrativevideocontenttoayoungeraudience.

17

Facebook

OverviewFacebookwastheprimarysocialmediaplatformusedforbuildingandsustaininganaudienceforthe project. The ExtremeDialogue Facebook Page received946 likes during the firstweek of thecampaign.Thegraphbelowshowsthebreakdownofpaidversusorganicpagelikes,aswellaspageunlikesduring thatperiod.Theadvertisingbudget forFacebookwas spreadmoreevenly than theYouTubeallocation,resultinginamoreconsistentdistributionofengagementsduringthecourseoftheweek.

FacebookLikes(Paid&Organic)

UserEngagement• Likes(eitheroftheExtremeDialoguepageorcontent)occurredmostlyviaads(63.1%),

followedbypagelikes(13.9%),postlikesontheExtremeDialoguepage(13%)andpagesuggestions(6.7%).

• Theproportionofpaidlikes(63.1%)wasconsiderablylowerthantheproportionofpaidviewsonYouTube(89%),althoughtherewasasmallerbudgetallocatedtoFacebookadvertising.

• Themajorityoforganic,unpaidtraffictothepagethatoriginatedfromoutsideFacebookcamefromtheprojectwebsiteandanarticlethatlinkedtothepageinTheGlobeandMail.

UserDemographics• Disappointinglythe13-17agegrouprepresentedlessthan1%oftotalpagelikes,although

thedecisionwastakennottoadvertisetothisagegroupduetopotentialprivacyconcerns.

• ThemoreevendistributionintheagesofExtremeDialoguepagefansthanfilmviewersonYouTubesuggeststhatFacebookmaybemoreeffectiveatreachingtheproject’solderhaloaudiences–teachersandparents.Thepagereceivedthehighestproportionoflikesfromthe35-44(20%)agegroup.

• IncontrasttothemajoritymaleaudiencesonboththeExtremeDialoguewebsiteandYouTubechannel,anddespitethelackofgenderbiasintermsofadtargeting,66%ofthosethatlikedtheFacebookpagewerefemale.

18

UserLocations• ThevastmajorityofusersthatlikedthepagewerelocatedinCanada(981),withthe

UK(57)andtheUS(43)theonlyothercountriesrecordingdoubledigits.

• WithinCanadatherewasarelativelyevengeographicalspread,withthemostpagelikescomingfromToronto(62),Montreal(55)andCalgary(51).London,UK(33)wastheonlycityinthetop-10fromoutsideCanada.

PostStats• Data frompostson theExtremeDialoguepageagain reveals thecontrastingeffectiveness

betweenorganicandpaidmethodsindrivingengagementamongusers.

• Organicpostsrequiredamere26impressionsperuserengagement,whereasboostedpostsrequired1179impressionstogenerateasinglelike,commentorshare.

• This suggests thatwhileboostedposts generate significantnumbersof impressions theywerenotparticularlyeffectiveinconvertingthisreachintouserengagement.

FacebookAdvertising• FourdifferentFacebookadswereusedtoattractbothparentsandteachersacrossCanada

totheprojectFacebookpage,whileboostedpostscontainingeachfilmweretargetedatyouthaged18-24inthefourtargetcities.

• Facebookadswereservedover504,000timesto362,500uniqueusersduringthepaidadvertisingperiod,resultinginadisappointing1835clicks.

• Clicksonadsresultedin923actions(likes,sharesandcomments)from916uniqueusers,atanaveragecostof£1.19peraction.

Reach&UserAdExposure• Theaverageuserwasexposedto1.39ExtremeDialoguerelatedads.

• OnFacebooktheExtremeDialoguecampaignresultedinextensivereachduringandimmediatelyafterthepaidadperiodfrom16th-21stFebruary.

• Organicreachachievedahighof501usersforapostbeforethestartofpaidadson17thFebruary.

• Theinitiationofpaidadssawreachexplodeto106,485users,continuingatlevelswellhigherthan100,000until25thFebruaryandinthetensofthousandsuntil2ndMarch.

• This fourdayextensionofhigherviewershipafter thepaidadperiodceasedpoints to theaddedbenefitofenactingpaidads.

• Paidadsdidresultinincreasedlikesbybothmalesandfemalesinthe18-24agerange.On16thFebruarytherewerejustsixlikesoftheExtremeDialoguepagebyfemalesand18likesbymalesinthisagerange.Attheendofthepaidadcampaign,on25thFebruarythesenumbersincreased significantly, with 108 likes by females and 102 likes by males in the 18-24demographic.

• AggregatedFacebooklikesinCanada,byCanadiancity,andgenderinthe18-24demographic,allshowthelimitationsofpaidads.Thisinformation,likethatforYouTube,isrepresentativeoftheneedtoincreaseorganicreachandtodevelopapaidadstrategywhichisstaggeredovertime.

19

AdPlacementFacebookadscanbeplacedinthreedifferentlocations,indesktopandmobilenewsfeedsandtheright-handsidecolumnorsidebar.Althoughadsplacedinusers’newsfeedswereclickedonmoreoftenandresultedinahighernumberofactionsperclickthantheequivalentadsplacedintherightcolumn,theycarriedbothahighercostperclickandcostperaction.Thissuggeststhatusersaremorelikelytoclickoncontentintheirnewsfeeds(more“natural”advertisingi.e.includedalongsideotherstories) than the right column (reserved exclusively for ad content), and these users are morecommittalandmoreinclinedtolike,shareorcommentonthepageoradcontent.

Newsfeedadswerethereforemoreeffectiveatencouraginguserengagementbutdidcomeatahigherpriceandshould thereforebeusedsparingly.Rightcolumnads representbettervalue formoney, producing both lower costs per click and per action, but are more clearly presented asadvertisedcontentandthereforemaynotbeappropriateforallformsofcounter-narrativecontent.

AdContentAll theFacebookadsfeaturedtextand imagesrelatedtoeithertheproject itselforChristianneorDaniel’sstories,enablingacomparisonacrossthevariousdifferenttypesofcontent.AdsfeaturingChristianneprovedthemostsuccessful(andwerethereforeallocatedahigherbudget),withboththehighestclick-throughrate(0.67%)andthelowestcostsperclick(£0.53)andcostsperaction(£0.84).Forcomparison,adsfeaturingDanielhadaconsiderablylowerclick-throughrate(0.26%)andprovedfarmoreexpensiveingeneratinguserengagementintermsofcostsperaction(£2.13).

ThedatashowsthatadsfeaturingChristianneweremorecosteffectiveanddrovehigherlevelsofuserengagement.Thiscouldbearesultofthecurrenthighlevelsofinterestinthesubjectmatter(ISIS rather than far-right) and/or the response evoked in viewers by the film itself. Ads with adescriptionof theprojectperformedrelativelypoorly,suggestingthatpersonalstoriesaremoreenticingtousers.

NewsFeed&RightColumnFacebookAds

Whilstthere isanargumentforallocatingevenmoreresourcestothemosteffectiveads infuturecampaigns,thiswouldhavedisproportionatelypushedonefilmovertheother.And,whilethismayhaveprovedmorecosteffective,ExtremeDialogueisintendedtohaveacross-ideologicalscopeanditwasthereforeimportanttopromotestoriesthatmanyintheproject’stargetaudienceswouldhavelesspriorfamiliaritywith.

AdTargetingAsonYouTube,adsonFacebookweretargetedatyouth,parentsandteachers.Parentsrepresentedbettervaluethanyouth,generatingmoreengagementatlowercosts.Parentadshadaclick-throughrate of 0.46%, almost double the 0.26% achieved by youth ads. 55.6% of the clicks from parentsresultedinanaction,comparedtoonly41.8%ofclicksamongyouth.Adstargetedatparentshadalowercostperclick(£0.50)thanyouth(£0.77),andgeneratedactionsbyusersataroundhalftheprice

20

(£0.89and£1.85peractionbyparentsandyouthrespectively).Thedatasuggeststhat,at leastonFacebook,parentsasanaudiencewerecomparativelymoreattractedbyExtremeDialogueadsandthenmoreengagedbytheprojectand/orthefilms.

Incomparisontoparentsandyouth,adstargetedatteachersprovedsurprisinglyandprohibitivelyexpensive.Avarietyofdifferentapproachesweretriedtotargetteachers.Initiallyadsweretargetedatthosethathadself-reportedtheiroccupationasateacher(anestimatedpotentialreachof60,000).This approach was ineffective so ads were retargeted at those that had self-reported studying“Education”.Thechangefailedtohavethedesiredeffectsotheremainingbudgetforteacherswasthenreallocatedtowardsparentsandyouth.

Althoughclick-throughrates for teacherads (0.39%)wererelativelyconsistentwiththeyouthandparentads,theyweremuchmoreexpensiveperclick(£1.14)andperactiongenerated(£11.96).Theseresultssuggestthatalthoughtheteacheradsattractedclicksatasimilarratetotheothers,Facebookcharged amuchhigher premium to advertise to teachers (due to a considerably smaller pool ofpotentialusers)andthereforeitwasnotviabletocontinuetheads.Thehighcostperactionandlowpercentageofclicksresultingactions(9.5%)couldpointtoalackofinterestintheprojectand/orfilmsamongteachers,althoughthesamplesizewasconsiderablysmallerthaneithertheyouthorparentgroupssothismaynotbeasignificantresult.

BoostedPostsvs.AdsFacebookadsprovedmoreeffectivethanboostedpostsacrosstheboard.Theyhadahigherclick-throughrate(0.45%to0.22%)andresultedinahigherproportionofuserstakingactionsoncetheyhadclicked(53.2%to40.6%).Aswellasbeingmoreeffective,adsalsorepresentedbettervaluethanboostedpostswithlowercostsperclick(£0.58to£0.67)andcostsperaction(£1.09to£1.65).

FacebookBoostedPosts

Thecontentoftheposts(textandthumbnail image) isalsoan importantfactor,withprevious ISDstudies showing thatmore recognisable or controversial images (e.g. ISIS black flag vs. counter-narrativeprojectbranding/images)combinedwithprovocative textgenerate themost responsesfromusers.Ourabilitytocreatemoreenticingadswassomewhatconstrainedhowever,withcertainkeywords (e.g. Neo-Nazi, ISIS) being rejected during the automated Facebook ad review process.NeithertheExtremeDialoguebrandingnorscreenshotsfromthefilmswerelikelytobearecognisedbyusers,limitingtheeffectivenessoftheimagesavailableforuseinthecampaign.

FacebookNetworkAssessmentPreviously identified community groups on Facebookwere assessed for the presence of ExtremeDialogue content or discussion related to the issues of counter-extremism and de-radicalization.SecDevassessed233communitypagesfromLondon(47),Ottawa(83),Windsor(80)andCalgary(23),selected from previously conducted studies on relevant audiences and in the case of Calgary,additional qualitative analysis.7 These Facebook pageswere composed of community groups that

7OttawaFacebookAudience/CommunityAssessment,LondonFacebookAudience/CommunityAssessment,andWindsorFacebookAudience/CommunityAssessment

21

playedimportantrolesintheircommunities.Outofthe233pagesitwasdeterminedthatonlytwohadposts related to ExtremeDialoguewithin the past fivemonths and that noneof these pagesincludeExtremeDialoguecontent.

ThelackofExtremeDialoguecontenthighlightstheshiftinapproachtotargetingtakenpriortothelaunch.Insteadoffocusingadvertisingandorganicoutreachatyouthandcommunitygroupsitwasdecided that the campaign would instead directly target youth, parents and teachers. The 233Facebook pages identified in London, Ottawa, Windsor and Calgary are however broadlyrepresentativeofyouthandcommunityorganizationsineachcityandthelackofExtremeDialoguecontentonthesepagesordiscussionaroundthecampaignisthereforeadisappointment.Alternativestrategies would be required to reach these groups, primarily via organic outreach and directengagement,includingjointevents,educationalsessionsorsocialmediacampaigns.

Findings• Facebook was more effective for targeting older users, with ads targeted at parents

outperformingthoseaimedatyouth.

• The reasons for the difficulties experienced in targeting teachers remain unclear andwillrequireadditionalexperimentationandtesting infuturecampaigns inorderto identifythetargetingcriteriathatmakeadstargetedatteachersmoreviable.

• Ads in the right columnweremore cost effective than those in news feeds and shouldthereforebeusedunlessthereisaneedforcontenttoappearlesslikeadvertisingorthereare no budgetary constraints (i.e. with in-kind advertising support from social mediacompanies).

• Boosted posts proved less effective than ads and should therefore be avoided in futurecampaigns.

• IncomparisontoYouTube,Facebookprovedamoreappropriatesocialnetworkfororganicoutreach and engagement, perhaps due to the more ingrained culture of participation,interactionandsharingontheplatform.

22

Twitter

OverviewWhilsttheExtremeDialogueTwitteraccountwasnotsupportedwithpaidadvertisingitstillattracted222followersand119mentions intweetsfromotheraccountsduringFebruary,themonthofthelaunch.Thisengagementwastheresultof21,100impressionsand3,272profilevisits,generatedbyonly29tweetsfromtheaccount.

NumberofFollowersonProfile

Whilstadswerenotused,Twitterdatashowsarelatedincreaseindiscussionduringthelaunchandpaid ad period. Increased activity by the ExtremeDialogue Twitter account during the launch didhoweverresultinasubstantialincreaseinrepliesandmentions.

ExtremeDialogueAccountOutput(green)&Replies/MentionsfromOthers(gray)

WiththeendofthelaunchandpaidadperiodTwitteroutputsandassociatedengagementdecreasedsignificantly.Onaverage,engagementwasmostsuccessfulfrom12:00to22:00UTC,highlightingtheneedtotargetTwitteroutputsduringthistime.Bycomparison,anaveragetweetresultedinroughlysixtypesofengagement,whiletweetsmadeat12:00UTCresultedin70typesofengagement.

23

PeakEngagementwithAccount(TimesinUTC:-1fromUK,+4fromEST,+5fromMST,+7fromPST)

In termsofyouthengagement,Twitter resulted in the leastengagementwithusersaged29andunder.Usersunder24comprisedonly4.13%ofExtremeDialogue’sTwitteraudience,whilethose30andoveraccountedforthemajorityofviewers.

Figure6:AgeRangeofTwitterUsersfromtheExtremeDialogueDashboard

TwitterNetworkAssessmentForTwitterSecDevundertookanetworkanalysis,usingGephivisualizationsoftwaretodisplaytheresultsinadigestibleformatinordertounderstandwhichonlinecommunitieswerepenetratedbyExtremeDialoguecontentandwhetherthosecommunitieswererelatedtothecampaign’smandate.This allowed SecDev to understand the central users (represented by nodes) in each community(represented by clusters of nodes) and their activity (represented by lines) as they related to theExtremeDialoguecampaign.

24

TwitterCommunitiesSharingExtremeDialogueContent

SecDevidentifiedsixcommunitieswhichsharedExtremeDialoguecontentbetween16thFebruaryand16thMarchonTwitterusingsocialnetworkanalysis.Itwasdeterminedthatthemajorityofuserswhointeracted with Extreme Dialogue content came from the fields of counter-extremism and de-radicalization.Ofthesixcommunitiesthethreelargest:ExtremeDialogueCore;ChristianneBoudreauCentric;and,General Counter-Extremism, were the most engaged with Extreme Dialogue content. Thesecommunitieswere largelycomposedofmembers inExtremeDialogue’s largersocialnetwork.Theremaining three communities: CBCRadioOne Segment;USMilitaryCommunity; and,Middle EastAffairsweredominatedbyasingleExtremeDialoguestoryorrelatedtweet.Falsepositiveswerealsopresent,withtheUSMilitaryCommunityandMiddleEastAffairsusingtheExtremeDialoguehashtagintweetsthatwereunrelatedtothecampaign.Almostall thediscussionsurroundingtheExtremeDialoguecampaignwerefavourable,withonlytheGeneralCounter-ExtremismnetworkdisplayinganynegativitytowardsExtremeDialogue’sefforts.ConversationsonTwitterrelatedtoExtremeDialogueremainedinsularoverall.ThosediscussingtheinitiativewerewithinExtremeDialogue’snetworkor related institutions focusingon the issuesofcounter-extremismandde-radicalization.Discussionsoutsideofthesenetworkswerenotsignificantandhighlighttheneedtoinitiateconversationsaroundtheinitiativebeyondthosedirectlyrelatedtothesubjectmatter.

25

Findings• OverallTwitterprovedmoreusefulforinteractingwiththosethatarepotentiallyuseful

advocatesfortheprojectbutdonotfallwithintheprojects’primarytargetaudiences.

• The lack of younger users that interact with these networks perhaps suggests that CVEpractitionersingeneralneedtomakemoreeffortstoengageyouthonTwitter.

• Aswithotherplatformsexaminedinthisstudy,anorganicreachstrategythatincludesconstanttweetingaboutthecampaignandthemesrelatedtocounter-extremismandde-radicalizationwillbebeneficialtomovediscussionaroundExtremeDialoguetoawideraudience.

• TheseinitiativesshouldbestaggeredovertimeinordertocreatecontinuedengagementwithExtremeDialogueproducts.Furthermore,asshownpreviously,higherlevelsofoutputsbytheExtremeDialogueTwitteraccountresultsinhigherexposure.

• TothisextenttheTwitteraccountshouldremainactiveandengaging,discussingnotonlytheExtremeDialogueinitiativebutrelatednews,eventsandprograms.

26

NewsMediaCommentsAnalysisIntotalover240stories featuredordiscussedExtremeDialogueacrosstelevision,radio,printandonlinemediaplatforms,includingeightoutofthe10newsorganisationsthatfeatureinthetop100mostvisitedwebsitesinCanada.8Between13thFebruaryand6thMarch,newsarticlesthatincludedthefilmsembeddedwithinthetextgeneratedanimpressive5,319views,or9.8%ofthetotalviews(includingadvertising)duringthatperiod.

The project launch event at the University of Calgary was covered by various Canadian newsorganisations,withCBC,CTV,GlobalNewsandtheCalgaryHeraldCalgaryHeraldallinattendance.Aswellasthecoverageofthe launchevent,theprojectwaspreviewedinanationalprintexclusive intheTorontoStarpriortothelaunchandthefilmsandwebsitebeingreleasedpublically.OnthemorningofthelaunchISD’sRachelBriggsspoketoCTVNews’CanadaAMbreakfastnewsprogrammeandop-eds by Rachel and Christianne were published in the Calgary Herald and The Globe and Mailrespectively.

FollowingthisinitialinterestotherstoriesandinterviewsappearedthroughoutthedayinTheNationalPost,Yahoo!News,Metroandanumberofotherpublications.CBCRadio’scurrentaffairsdiscussionshow ‘The Current’ devoted a 22 minute segment to Extreme Dialogue and the difficulties ofcombatting extremist propaganda and creating effective counter-narrative content. Many storieswerealsorepublishedinlocalnewsoutletsacrossCanadaandtheprojectwasalsomentionedinUS,UK,Irish,Australian,DanishandCzechmedia.

Over the course of theweek the projectwas also featured inmajor online publicationsVice andHuffingtonPostCanada,bothofwhichembeddedthemainfilmsontheirwebsites,earning1000sofadditionalviewers.DuetotheattentiontheprojectreceivedduringISD’spresentationattheWhiteHouseCVESummit,ChristiannewasalsointerviewedlivebyCNN.

SecDevwasable to track thecommentsmadebyusersonnewsarticleswhich featuredExtremeDialoguecontent.Tounderstandtheimpactofpaidadsonnewscommentsthissectionprovidesananalysisofcommentsfrom13th-20thFebruary.Itthenmovesontoassesstheentireperiod,from13thFebruaryto6thMarch,inwhichcommentsweremadeonExtremeDialoguearticles.

8Alexa.com,“TopSitesinCanada”,http://www.alexa.com/topsites/countries/CA

27

Figure1:PaidadperiodcommentsrelatedtotheEDcampaign

Duringthepaidadperiodatotalof56commentsweremadeonnewsarticlesthatcoveredtheExtremeDialoguecampaign.Ofthesecommentsroughly66%wereinsupportofthecampaign.ManyofthesepositivecommentsduringthisperiodandparticularlythosethatdealtwiththecaseofChristianneBoudreaufocusedontheneedtopreventtheradicalizationofyouth.

Otherswhoexpressedanegativeviewoftheprogramduringthepaidadperiodbelievedeitherthattheprogramwasfutileandawasteofmoney,heldissuewithIslamanditsteachings,orthatradicalizedindividualswerementallyillandcouldnotbereformed.

Figure2:CommentsrelatedtotheEDcampaign

Therewereatotalof149newscommentsrelatedtotheExtremeDialoguecampaignfrom13thFebruaryuntil6thMarch.Theoverwhelmingnumberofthesecommentswereinsupportofthecampaign,whileaminorityofcomments,roughly25%,tookanegativeapproachtotheinitiative.ManyofthesesupportivecommentsappearedbelowthearticleISISrecruitment:formerwhitesupremacistDanielGallantfightsradicalizationonCBC’swebsite,pointingtotheimportanceofadoptingacross-ideologicalapproach.

Negative comments surrounding the Extreme Dialogue campaign seemed to take a hard line ondealingwiththosedeemedbysomecommentersas“terrorists”.Othersblatantlytookissuewiththe

Supportive37(66%)

Negative19(34%)

Supportive112(75%)

Negative37(25%)

28

facttheDanielGallantwaspreviouslyanextremist,whilesomecalledforastrongerlegalandpolicingapproachtoterrorismandlabelledanydialogueasfutile.

Newscommentsbothduringthepaidadperiodandfrom13thFebruaryto6thMarchhighlightsomeinterestingelements.ByandlargethosewhohadreadthearticlesexpressedapositiveviewoftheExtremeDialoguecampaign, the individuals featured ineachvideoand toward theadoptionofasofter approach to counter-extremism and de-radicalization. As noted above, many of thecommentersexplicitlymentionedthatthesearticleshadinfluencedthempositivelyandthattheycametoviewtheExtremeDialogueapproachasbeneficial,particularlywhencomparedwith themore legalistic and policing approach advocated by the current government. Overall, theoverwhelminglysupportivenatureofcommentsdirectlyreferencingtheExtremeDialoguecampaignshouldbeviewedasasuccess.ThosewhotookthetimetocommentonthecampaignlargelysawitasabeneficialapproachtodealingwiththethreatofextremismandwouldliketoseethespiritoftheinitiativeadoptedbytheCanadiangovernment.

Figure3:Abreakdownofnewscommenttopics

NewsCommentThemesArticlescoveringtheExtremeDialoguecampaignalsosparkeddiscussionandinmanycases,debate,overthemesrelatedtotheissuesofextremismandterrorism.From13thFebruaryto6thMarchatotalof1,184commentsweremade.Withinthesecommentsfourprimarythemesexist:domesticpolitics,domesticradicalization,internationalrelationsandreligion.Ofthesethemesdomesticradicalization,anissueclosetotheExtremeDialoguecampaign,madeupabouthalfoftheconversation.Negativeandneutralcommentscombineddominatedtheconversation,with28%ofcommentersexpressingapositiveviewonthefightagainstextremism.

Domesticpoliticsalsorankedhighamongcommenters,withafourthofindividualscommentingonthepoliticalprocessandthestancesofmajorpartiesinCanadaandtheUnitedStatesonterrorismanddomesticradicalization.ThemajorityofthesecommentatorshadanegativeviewofCanadianandAmericanpolitics,constitutinghalfofthetotalcomments,withtheremainderlargelyclassifiedasneutral.Thethirdmostprevalentthemediscussedinthecommentsection,discussedbyroughly15%ofcommentators,wasreligion.NegativeviewsaroundIslamandinsomecases,organizedreligion,dominatedthisconversation.Suchcommentsrepresentadisturbingtrendandtheneedtodevelopanopen,publicdialoguearoundradicalizationandinter-faithlearning.Internationalrelationsroundedonthecommentthemesandwasdominatedbynegativeandneutralcomments,at39each.

DomesticPolitics242

(21%)

DomesticRadicalization521(44%)

InternationalRelations85(7%)

Religion181(15%)

Unrelated155(13%)

29

Atotalof155comments,around13%ofthosecovered,wereunrelatedtoanyoftheabovecategoriesanddidnotholdanysignificantcommentaryontheExtremeDialoguecampaignorthemescoveredabove.Commentsnotrelateddirectlytothecampaignwereoverwhelminglynegative,withthebulkofothersbeingneutral,whichissomewhattypicalfornewscommentssections.Positivecommentswereonlyslightlydominate inthecaseofdomesticradicalizationtopics,with30%ofcommenterstakinganoptimisticviewcomparedtoalmost28%ofcommenterstakinganegativeposition.

Figure4:Newscommenttopicsduringthepaidadperiod

Duringthepaidadperiodandslightlybefore,from13th–25thFebruarytherewereatotalof400categorizedcomments,379ofwhichfitintothecategoriesofdomesticradicalizationreligion,internationalrelationsanddomesticpolitics.Aswiththeoverarchingperiodfrom13thFebruaryto6thMarch,thosecommentingatthistimealsogravitatedtowardsdiscussionsaroundtheissueofdomesticradicalization.Over50%ofcommentatorsdiscussedthisissue,withanequalnumber,61comments,takingpositiveandnegativeapproachestothetopic.Ofthenegativecomments,manytookissuewithasofterapproachtoradicalization.

Religionwasthesecondmostpopulartopicduringthepaidadperiod,withslightlyoverhalfofthecommentstakinganegativeapproachtotheIslamicreligionorreligioningeneralandonly10%takingafavourableposition.ThosecommentingoninternationalrelationstookanegativestanceonworldissuesandCanada’sinternationalrole46%ofthetime,withzeropositivecommentsrelatedtothetheme.Withregardstodomesticpolitics,outof37commentsrelatedtotheissue,therewere15negativecommentsandonlyonecharacterizedaspositive.

Aswith the entire period inwhich commentswere tracked, the paid period also experienced anoverwhelmingnumberof negative comments. The sole category todefy this trend, asmentionedabove,wasdomesticradicalization,withanequalnumberofpositiveandnegativecomments.

Findings• AnaccomplishmentoftheExtremeDialoguecampaignisthatoutsideofdirectreferenceto

theinitiativeitwasalsoabletosparkawiderdebatearoundtheissueofdomesticradicalization,oneoftheoriginalaimsofthecampaign.

• Also,giventhatmanyadmittedthatthecampaignhadchangedtheirviewofdomesticradicalizationorcreatedempathyforformerradicalsandtheirfamiliesshowedthattheExtremeDialoguechallengedthebeliefsofindividualswhopreviouslyheldstrongnegativeviewsonthetopic.

DomesticPolitics37(9%)

DomesticRadicalization214(54%)

InternationalRelations48(12%)

Religion80(20%)

Unrelated21(5%)

30

• Anundesirableaspectshouldalsobeacknowledged,inadditiontoconversationarounddomesticradicalization,ExtremeDialoguecoveragedidsparkunconstructivepartisandebateoverdomesticpoliticsandthehandlingofterrorism,inadditiontobigotedstatementsagainstIslam.

• Whilethisisnotanissuewiththecampaignitself,inthelattercaseithighlightstheneedformoreeducationaroundtheMuslimfaithandreligioningeneral,apossibleavenueforfuturedevelopedofExtremeDialogueorrelatedcampaignsthatwasalsohighlightedinfeedbackreceivedfromparticipantsinthetrainingsessionsheldinJune2015.

YouTubeCommentsSecDevcollectedcommentsfromExtremeDialogue’sYouTubechannelbetween13thFebruaryand6thMarch2015.UnsurprisinglythetwomainvideosontheYouTubechannelthatweresupportedbypaidadvertising,“ChristianneBoudreau”and“DanielGallant(Explicit)”,receivedthemostcomments.Ofthe33comments from22userscollectedacrossallvideos,17werecharacterizedaspositive,13werenegative,andthreewereneutral.ISDchosetomoderatecommentspriortoallowingthemtoappearunderneaththevideosprimarilytoinsulateyouthfromextremeoroffensivecommentsandfar-rightYouTubeusers.Commentsthatwerewithheldwerehoweverretained.

CharacterizationofCommentsonExtremeDialogueYouTubeVideos

Videos featuringChristianneBoudreau received the largestportionofpositive commentary,whilethose featuringDanielGallant received thevastmajorityofnegative commentary.Overall,videosfeaturingChristiannehadatotalof10positivecommentsand2negativecomments,whereasthosefeaturing Daniel had a total of 11 negative comments and seven positive comments. Positivecomments were liked 17 times by other users, with no dislikes, although many of the negativecommentswerenotpublishedsocouldnotbelikedordislikedbyotherusers.

Intermsofpositivecomments,individualstypicallydenouncedradicalinterpretationsofIslamandnotedtheirempathyandsupportofbothDanielandChristianneforovercomingtheirstrugglesassociatedwithradicalization.

Thenegativecommentslargelyconsistedofpersonalattacks,Islamophobia,anti-Semitismorremarksregardingthewhitesupremacistmovementor ISIS.Eight ofthe13negativecommentsoriginatedfromfourusersthatdirectlyexpressedextremefar-rightsentimentsorhadrecentlywatchedNeo-Naziorwhitesupremacyrelatedvideos.OneuserpostedmultiplecommentsattemptingtodiscreditDaniel,linkingtoablogpostfromtheperspectiveofamemberoftheCanadianwhite-supremacistmovement (see Far-Right Reaction). The negative comments received from extreme far-rightsupportersaresadlyrelativelytypicalofthereactiontoformerextremistsamongthosestillsubscribedtoextremeideologies.

Overall,onlyaverysmallpercentageofviewercommentedontheExtremeDialoguefilms.ThepositiveandnegativecommentsproducedbyYouTubeuserstypicallyexpressedstrongconvictions

Postive17(52%)

Negative13(39%)

Neutral3(9%)

31

fororagainsttheprojectandthesubjectsofeachvideo,suggestingthatthesewereamongthemostemotionallyengagedviewers.

FacebookCommentsSecDevcollectedcommentsfromtheExtremeDialogueFacebookpagebetween29thJanuaryand7thMarch.Ofthe50commentscollected16werecategorizedaspositive,zerowerenegativeand34wereneutral.

CharacterizationofCommentsonExtremeDialogueFacebookPage

In comparison to the commentsmade on news reports, Facebook comments tended to bemuchshorter,withabouthalfofthesecommentscontainingnotextasidefromanotificationthattheseusers had shared an Extreme Dialogue post. The gender breakdown of the commenters, with 24femalesand16males,alsopointstohigherinteractionwiththecontentbywomenontheFacebook.CommentsonFacebookdisplayedapersonalconnectiontothesubjectsofthevideosaswell.

TheChristianneBoudreauandDanielGallant videoswere themostpopularpostson theExtremeDialoguepage,withChristianneBoudreau’svideoliked46timesandshared14times.DanielGallant’svideoreceived45likesandwasshared6timesduringtheperiodofobservation.OneimportantpointtobenotedisthatsomeoftheFacebookcommentersarelikelytobethoseinvolvedorassociatedwith theExtremeDialoguecampaignor ISD,althougha largernumberwouldhavebeenattractedthrough advertising. This would partially help to explain the positive nature of comments whencomparedwiththosefromnewsreports.Newsarticlesreachawideraudienceoutsidesupportersofthe ExtremeDialogue initiative and its target audiences, and there tended therefore to be fewerpositivecomments.Likeotherplatformscoveredinthestudy,thisdatapointstotheneedtodeveloporganic reach beyond the insular community of counter-extremism and de-radicalizationprofessionals.

Far-RightReactionTheDanielGallantfilmswereembeddedbymembersonthefar-rightStormfrontand IronMarchforums,resultingindiscussionsofDaniel’sstoryandfilmviewsfromCanada,theUSandAustralia,allwithhigherthanaverageviewdurations.TheprojectalsotriggeredapostonaCanadianfar-rightblog,VinlandAwake!,thathasbeenviewed3,259timestodate.Thisblogpostwasalsolinkedtobyone of the four far-right influenced users that commented on the film directly on YouTube.Unsurprisinglythesentimentsexpressedinthepostsandcommentswereentirelynegative.Havingnotactively targeted thosealreadyexpressingextremeviewsonline, the fact that theprojectwaspickedupbyfar-rightblogsandforumssuggeststheprojectwasabletoachieveconsiderablereachbeyondtheprimarytargetaudiencesasaresultofmediacoverageandsocialmediaactivity.

Negative0(0%)

Positive16(32%)

Neutral34(68%)

32

ExtremeDialogueRelatedPostsonFar-rightWebsites

(Clockwisefromtopleft:IronMarch,VinlandAwake!&Stormfront)

33

CONCLUSIONS&RECOMMENDATIONS

CampaignOverviewThewebsiteprovedeffectiveatpromoting theprojectandencourageduserengagementwith thecontent.Itquicklyclimbedinsearchresultssuggestingthattheprojectwasmemorablynamedandthesitecontentcouldbeefficientlyindexedor“crawled”bysearchengines.Thewebsitewasmorepopularamongslightlyolderaudiences(25+),whereasyouthtendedtobefoundmoreonsocialmediaplatforms.

Conversely,YouTubewasthemosteffectiveforreaching18-24yearolds,withadvertisinggeneratingan impressively large number of impressions and views among the campaign’s primary targetaudience.Thedatadoeshoweverhighlightthedifficultyofattractingyoungeraudiencesorganically.When compared to organic viewers, ad viewers weremore difficult to retain and engage with adisappointingleveloflikes,commentsandsharesofthefilms.Thestrongabsoluteviewerdurationsdo however serve as a strong endorsement of the content of the films, with the average viewerwatchingseveralminutesoffootage.Facebook was the most effective platform for targeting older users as the ads targeting parentsperformedbetterthanthoseaimedatyouth.WhencomparedtoYouTube,Facebookalsoprovedthesocialnetworkmostsuitedtoorganicoutreachandengagement.AlthoughpaidadsonFacebookdidresultinextensivereachduringtheweek-longlaunchperiod,afterpaidadsceasedtheresidualeffectswerenotsustained.TheoverwhelmingsentimentsofcommentsontheExtremeDialoguepagewerepositiveorneutralbutdidnotsparkawiderdiscussion.Thecommunityorganisationsidentifiedduringthe Target Audience Analysis stage did not interact with the campaign, although this was to beexpectedduetochangesinthetargetaudiencesduringthecampaignplanningstage.TheExtremeDialogueTwitteraccountprovedmosteffectiveatinteractingwithusersfromoutsidethe project’s target audiences. On Twitter the conversation around the campaign remained quiteinsularwiththemajorityofusershavinganinterestintheprojectbutoriginatingfromexistingISDnetworks, the broader CVE Twitter community or journalists interested in covering the launch orreportingontheproject.High levelsofoutput fromtheaccountconsistentlyresulted in increasedengagementbutthissamelevelofoutputcouldnotbesustainedbeyondtheinitiallaunchperiod.Acrossplatformspaidadvertisingwassuccessfulinprovidingthecampaignwithinitialmomentum.Adsgeneratedahighnumberofviewsandlikesinthefirstfewdaysofthecampaign.Thismayhavecontributedtodrivingthehighlevelsofmediainterestwhichinturnexpandedtheproject’sreach,especiallyoutsideofthecampaign’spredefinedtargetaudiences.ThismomentumhoweverwasnotsustainedbysufficientunpaidororganicreachandengagementandtheresidualeffectsofadsonbothYouTubeandFacebookdisappointinglydidnotlastbeyondthefirstweekofthecampaign.Thecampaignhighlightedtheimportanceofhavingaflexiblebudgetinordertomaximisetheimpactofadvertising.BynotadheringrigidlytoapredefinedbudgetISDwasabletoavoidallocatingfundstoadsthatwerenotperformingwhilelargelymaintainingtherequiredbalancebetweenthedifferentplatformsand targetaudienceswhereneeded.Thechartbelow illustrateshowallocations shiftedtowardsthemoreeffectiveYouTubein-streamandFacebookads,andawayfromthelesseffectiveYouTubeIn-displayadsandboostedFacebookpostsduringthecourseofthecampaign.

34

ActualAdSpend(Audience)&(AdType)(Originalprojectedbudgetsinbrackets)

Asoutlinedabove,thebudgetcouldhavebeenmoreevenlyallocatedoveralongerperiodoftime.DifficultiesexperiencedwithtargetingonFacebookalsomeantthatfewerteacherswerereachedwithadsthanoriginallyplanned.Asaresulthowever,thecampaignreachedagreaternumberofparents(primarilymothers)instead.It isdifficult to fullyevaluate thecampaign in termsofvalue formoney.Adsgeneratedsignificantexposurebutnot sustained interaction. Spending resulted in short-termbrand recognition amongtargetaudiencesandthewiderCanadianpublic(viamediacoverage)butdidnotnecessarilyrepresentasolidlong-terminvestment.ThecampaignwasabletobuildabaseoffansandfollowersonFacebookandTwitterrespectively.Sincethelaunchcampaignendedhowevertheseaudienceshavenotbeenespeciallyactiveoutsideperiodsofprojectactivity(e.g.ExtremeDialoguetraininginJune2015)andadditionalmediacoverage.Itremainstobeseenwhethertheseaudiencesare“dormant”andcanbere-engagedthroughhigherlevelsofoutputfromtheprojectaccountsandnewfilmcontent.Itproveddifficulttomakethemostofthebudgetallocatedtothecampaignduetothebroadtargetingcriteriaemployed.Whentargetinglesswelldefinedtargetaudiencesthereisnoguaranteethattheaudiencehasinterestinthetopic.AsaresultthecostsperclickonFacebookandcostsperviewonYouTubewere considerably higher thanwould be expected for commercial users of socialmediaadvertising.Theywerealsomarginallyhigherthanthecostsassociatedwithprevious ISDcounter-narrativecampaigns,althoughthesecampaignsweretargetedatamorespecificaudiences.Targetingwaseffectiveinsofarasthecampaignachievedhighnumbersofviewsandlikesfromthetargetaudiencesofyouthandparents,andtoalesserextent,teachers.However,asstatedpreviouslyabove,thetargetedaudienceswerenotengagedsufficiently.Thecampaignshowedthatadvertisingismoreexpensiveandlesseffectivewhenusinglesswelldefinedtargetingcriteria.Infuturetargetingshould therefore be further refined andmakeuse of themore detailed criteria available on bothYouTubeandFacebook.ThelargenumberofnewsstoriesfeaturingExtremeDialoguewereeffectiveinamplifyingthemessageof the campaign and introduced the project to a national audience across multiple mediums.Embeddedvideosinnewsarticleswerethemost importantunpaidsourceofviewsotherthantheproject website, reinforcing the power maintained by traditional sources in the dissemination ofmedia on a large scale. The visibility of the project also led to an initial surge in requests fromjournalistsandeducators,aswellasoffersofsupport,feedbackfromthepublic,andrequestsforhelpfrom those concerned about radicalization or seeking support. The impact of the media shouldhowevernotbeoverstated.Whilespikesintrafficandviewshavecoincidedwithnewsstoriessince

Youth54%(60%)

Teachers16%(20%)

Parents30%(20%)

FBBP8%(7%)

FBAd22%(13%)

YTIn-D12%(40%)

YTIn-S58%(40%)

35

thelaunch,againthishasnotresultedinsustainableengagementwithnewsaudiencesonongoingbasis.SecDev’sanalysisofnewscommentsshowedthatthemajorityofcommentersthatreferencedtheprojectwere supportiveof theapproach takenandappreciatedhearing theperspectivesof thosepersonally impacted by violent extremism. The campaign was also ablemeet one of the originalobjectivesoftheprojectbycatalysingawiderdiscussionarounddomesticradicalizationinCanadaaswellasvariousotherissuesrelatedtoextremism,terrorismandsecurity.Onsocialmediacommentstendedtobemorefocusedontheprojectorpersonalmessagespraisingor criticisingChristianneandDaniel.Commentswerevettedprior topublicationonYouTubewithoffensive, extremeor threatening comments removed to protect the film’s younger viewers. Thishowever may have stifled discussions and proved counter-productive in terms of encouragingengagementandshouldthereforebereconsideredforfuturecampaigns.

FinalCampaignAssessmentThecampaignaimedtoamplifyhigh-quality,impactfulcounter-narrativecontent(ownedmedia)withanextensivepressandonlineadvertisingcampaign(paidmedia)toprovidethecampaignwithinitialmomentumandattractsignificantexposure,coverageandengagement(earnedmedia)aroundthelaunch.Whilethecampaignwassuccessfulintermsofgeneratingexposureandcoverage,itwaslesssuccessful in converting this into the regular levels of engagement hoped for beyond the launchperiod.

Theproject’sownedmediacontentperformedwelloverall.Theaverageviewingtimesandviewerretentionratesofthefilmssuggeststhataudiencesfoundthemengagingandthereispotentialfortheuseofmediumlengthcounter-narrativevideos(around4-10minutes)online.Similarly,theprojectwebsiteachievedsoliduserengagementfiguresintermsofaveragepagespersessionandtimespentonthesite.

EARNEDMEDIA• PRINT/ONLINENEWS• BROADCASTNEWS• SOCIALMEDIAENGAGEMENT• WEBSITETRAFFIC

PAIDMEDIA

• YOUTUBEADS• FACEBOOKADS• MEDIASUPPORT

OWNEDMEDIA

• FILMS• WEBSITE• YOUTUBE• FACEBOOK• TWITTER

36

Paidadvertisingdeliveredmixedresults,achievinglargenumbersofadimpressionsandviewsfromtheproject’stargetaudiencesbutfailingtodeliverengagementconsistently.ThepaidcampaignbuiltonpreviousISDeffortsandwassuccessfulinreinforcingunderstandingsofthemosteffectivetacticsin terms of ad type, placement and content on YouTube and Facebook. With the assistance ofprofessionalmediasupporttoinformrelevantpressaboutthelaunch,generatestoriesandmanageenquiries,thepresscampaignprovedhighlyeffective,resultinginsignificantearnedmediaintheformofnewscoveragethatwasbroadlypositiveorneutral.Questions remain as towhether it is cost-effective to target advertising at broad audienceswithcounter-narrativecontentinfuturecampaigns.ExtremeDialoguewastargetedatabroadaudiencebecauseitisaninherentlypreventativeinitiativethatisexplicitlyaimedatallyouthratherthanthosedeemed“at-risk”.Advertisingplatformsonsocialmediahoweveraredesignedtoofferhighlevelsofgranularityandaudiencesegmentation,andareattractivetocommercialadvertisersforthisreason.BynottargetinghighlyspecificaudiencesISDwasunabletotakefulladvantageoftheservicestheseplatformsoffer and targeting refinements shouldbemade in futureExtremeDialogue campaigns.Howeveritwouldcontradicttheethosoftheprojecttotargetmorespecific“at-risk”audiencesandthoseexpressingextremeviewsonline,andcouldpotentiallyprovecounter-productiveasthefilmswerenotmadewiththeseaudiencesinmind.Thecampaignalsohighlightedtheneedtoattainamorecompleteunderstandingofwhichapproachescouldbeusedtobetterengageyoungeraudiences.Someplatformsdonotallowadvertisingtocertainagegroupswhich,combinedwiththepotentialforinaccurateself-reportingofage,somewhatlimitsthescopeforeffectivelytargetingadvertisingandmeasuringresultstoyoungeraudiencesontheseplatforms.Theprimarydisappointmentduringthecampaignwasthelackofsocialmediaengagement,especiallyfromyouth,theproject’sprimarytargetaudience.Sharingbehaviour,seenasthemostvaluableformofuserengagement(orendorsement),didnotoccuratsufficientscaletomaintainthemomentumofthecampaign,pointingtothedifficultyoftargetingcounter-narrativecontentatabroadaudienceand remaining in the spotlight without content truly going “viral”. Despite currently occupying aprominent position in news coverage, issues surrounding extremism and radicalisation are stillperhapsdistantconcernstothemajorityofCanadians.ISDwasunabletosustainthelevelofinputrequired to engage consistently and effectively with audiences on social media through organicmeans. This has proved a challengewith previous counter-narrative initiatives, andwill remain asignificantobstacletosustainabilityforfutureeffortswithoutadditionalresourcing.Thereachandengagementdatacapturedduringthecampaignisextensiveandenablesforin-depthanalysisofmanyaspectsofuserdemographicsandonlinebehaviours.Paidadvertisingenablesfurtherinsightsintoaudiencepreferencesandprovidesalargerquantitativedatasetfromwhichtodrawsolidconclusions. In short, it is relatively straightforward to ascertain who (in the broadest sense) iswatching, liking or clicking, where they are doing it and for how long.What remains a challengehoweverisidentifyingtheimpact,ifany,thatcounter-narrativecampaignscouldhaveonaudiences’attitudes and offline behaviour. The lack of qualitative data available (with the exception of usercommentswhichwerebroadlyencouragingbutnotnecessarilyarepresentativesampleofusersortheirviews)severelylimitsthepossibilitiesformeasuringthistypeofimpactonlinewithoutadditionalresourcestoapplymorein-depthqualitativemethodologies.Whenevaluatingthecampaignagainsttheoriginalobjectivesitcomfortablymetthe10%targetofviewsandinteractionsfromtargetaudiences,althoughconsideringtheimpactofpaidadvertisingthisobjectiveinhindsightwasnotambitiousenough.Thesecondobjectivewaspartiallyachieved,withvideocontentbeingwidelyconsumedbutnotshared.Thecampaignwashighlysuccessfulinattractingpositive media coverage to build awareness of the project, meeting the third objective. Finally,although sufficientdatawas collected toenablean in-depthanalysis, considerablegaps remain intermsofthequalitativedatathatcanbegatheredandfurthertestingandinvestigationisalsorequiredinsomeareas.

37

SincethelaunchISDhasmadeeffortstoincreaseoutputfromtheproject’ssocialmediaaccounts,buildlinkswithsimilarprojectsandincreasedirectoutreachtoschoolboardsandeducationalbodies.Ithashoweverproveddifficulttosustainpublicinterestintheprojectwithoutnewcontentoutsideofthespikesassociatedwithintermittentnewscoverage.

CampaigningBestPractice&LessonsLearnedThereareanumberofessentialcomponents thatshouldbecentral toany futureonlinecounter-narrativecampaigns;

• Fundamentallyacampaignrequiresamemorablebrandandstrongcontent,ideallyinhighenoughqualityforrebroadcastorembeddinginnewsarticles.

• Campaigns should have a clearly defined strategy to meet set objectives, regardless ofwhetheritisaimedatraisingawarenessofanissue,resourceorsupportserviceorcompetingmoredirectlywithextremistpropaganda.

• Anystrategymusttakeintoaccountthecampaign’stargetaudiencesandconsiderhowbesttoreachthemintermsofplatforms,mediumsandtactics.

• Dependingonavailability,resourcesneedtobeappropriatelyallocatedbetweenpaidandorganicmethods,includingmediasupportifpossible.

• Thecampaignoughttohaveaframeworkforevaluationin-builtfromthestart,withanalyticsservicesestablishedtocollectdata.

• When completed the results should be examined to assess performance, inform futurecampaignsorongoingengagementandrefinestrategiesandtactics.

The analysis and evaluation of the Extreme Dialogue campaign did highlighted a number ofimprovementsthatcouldbemadetotheongoingpromotionanddisseminationstrategyforExtremeDialogue,butthatcouldalsobeappliedtootheronlinecounter-narrativeinitiatives;

• PaidAdvertising:Whenusingpaidadvertising,spendingshouldmoreevenlydistributedovertimeinordertokeepcontentrelevant,totakeadvantageofanyresidualeffectsintermsofuserreachandengagementandavoidover-exposinguserstocampaignads.Theimpressionfrequencyofadsshouldbecloselymonitoredwhenpaidadvertisingisrunningtoensurethatusersarenotbeingalienatedandspendingisallocatedeffectively.However,dependingonthetime-scalesandobjectivesofthecampaign,heavierspendingmayberequiredforashortperiod toprovide initialmomentumandattractmediacoverage.This shouldbecombinedwithgreateruseoforganicoutreachtacticstoencourageuserengagementamongadvertisingaudiencesduringtheinitialcampaignbutalsointhelong-term.

• OrganicOutreach:AswithpreviouscampaignsrunbyISD,higherlevelsofuserengagementhave proved elusive. This again points to the need for improved organic reach andmoreconsistent interactionwithusers,both in termsofqualityandvolume.Organicpromotionshouldbedirectlytargetedatinstitutionsandindividualsthatwouldbenefitfromorhaveaninterest in theprogramandstaggeredover time tomaintaina steady levelofactivityandretainaudiences.Resourceandtimeconstraintsshouldhowevertobeconsideredinordertoprioritisethetypesoftacticsthatarelikelytobemosteffective.

38

• Metrics:The“quality”ofcertainmetricsmustbeconsidered.Absolutenumbersofviewsorimpressionscanbemisleadingrepresentationsofthesuccessofacampaignunlesstheyarequalified by view or retention rates and other engagement statistics. Comparisons acrossplatforms should also be carefully considered as certain metrics are not necessarilyequivalent.Forexample,costsperviewonYouTubeandcostsperlikeonFacebookshouldnot be seen as comparable metrics hence the significant differences in prices during thecampaign.AviewofavideoonYouTubeisaoneoffinteractionwithauser,whereasFacebooklikesaremoreequivalent tosubscribersonYouTube in that theusercan thenbe reacheddirectlywithadditionalcontent in future.Asthis typeofbehaviour ismoreuncommononYouTube,Facebookadsrepresentthemoresustainablepaidavenueforbuildinganaudience,althoughasthecampaignshowsthisisnoguaranteeofconsistentlong-termengagement.

• Media:Media coverageprovidesexcellentexposurebutdoesnotnecessarily convert intoviewsandengagementonline.Themostusefulreports inthissensewherethoseincludingtheExtremeDialogue films.Users aremore likely to viewcontent that is readily availableratherthanproactivelyseekitoutelsewhereonline.Infuturecampaignsjournalistsshouldbeencouragedto includeembeddedvideowherepossible.Mediacoveragereceivedsincethelaunchhasresultedinspikesinonlineinterestonthewebsiteandsocialmediaplatforms.Projectsshouldthereforeaimtocontinueconductinginterviewsandreleasingnewcontentasoftenaspossibletomaintaininterest.

39

RecommendationsforGovernments&SocialMediaPlatformsISD’s experiences of counter-narrative campaigning to date have shown that building sustainableinitiatives remains a considerable challenge. There are a number of avenues through whichgovernmentsandsocialnetworkscouldsupportexistingandfuturecounter-narrativecampaignsbycivilsocietyorganisations;Thereisaneedforconsistentin-kindadvertisingsupportfromsocialmediacompaniessuitableforcounter-narrativecampaigningpurposes.Asmallmonthlyin-kindbudgetwouldallowcampaignstomaintain exposure and engagement at a moderate but consistent level. This could then besupplementedbyorganicmethods toensure thatusers attracted via advertising remainengaged.Thereareanumberofstrongexistingcounter-narrativecampaignsbuttoooftentheydonothavethenecessaryresourcesorfundingtodisseminatetheirmessageeffectivelytotheappropriateaudiences.Manyofthemorespecificaudiences,keywordsorURLsthatcouldbeusedtotargetcounter-narrativecontentthroughadvertisingareunlikelytoattractlargesumsinadvertisingrevenueastheywouldnotbeinhighdemandtocommercialadvertisers.Automated review processes for ads have caused issues on a number of platforms. Perhapsunderstandably,certainextremismrelatedtermsappeartobeblockedfrominclusioninads(whichhavetobeapprovedpriortogoing“live”).However,theseareoftenthetermsthataremostlikelytoattract viewsor clicks, and in the right context (e.g. “Former neo-Nazi reveals howhe renouncedextremism”)canpromoteapositivecounter-narrativemessage.Manyoftheadsusedinthecampaignhadtoberephrasedanddilutedandasaresultwerelesscosteffectiveingeneratingclicks,viewsandlikesthantheycouldhavebeen.The socialmediacompanies couldalsooffer technicaladvertising supportbyhavinga specificallyemployedmemberofstafftosupportcounter-narrativeinitiativesfromcivilsocietyorganisationsandindividuals.Thiscouldincludebothongoingtrainingandsupportduringcampaignstoensureadsareapproved and targeting criteria, ad content andplacement are cost-effective. Support fromotherprivate sector experts in campaigning, online advertising and media support would also help tomaximisetheimpactofcounter-narrativecampaigns.Nationalandlocalgovernmentandlaw-enforcementagenciesareoftenconstrainedintheirabilitytocontribute to certain types of CVE programs, including counter-narrative campaigns. However, asExtremeDialoguehasshown,itispossibleforgovernmentsandlaw-enforcementagenciestofund,supportorendorsemoreup-stream initiatives.Civil societycounter-narratives intended formorespecific audiences (thatmay not consider the state credible) can be supported through training,capacity building or the sourcing of private sector partners to develop technical, production,communications or campaigning skills. As previously mentioned, there are ongoing challengesassociatedwithqualitativelyevaluatingtheimpactofonlinecounter-narrativecontent.Governmentscouldthereforeinvestinresearchingmethodologicalapproachesthatcouldfillthisgaporprovidesupport to existing initiatives aiming to measure their impact online. Governments should alsocontinuetoworkwithsocialnetworkstostrengthencontentremovalprocesseswhenappropriateanddisrupt existingonlinenetworksof extremists, recruiters and their supporters andencourageparticipation in counter-narrative production and campaigning from the private sector morebroadly.

40

GLOSSARY

AverageImpressionFrequency:thenumberoftimesthatanaverageuserisexposedtoanadoveragiventimeperiod.

AverageViewRate:thepercentageofusersthatwatchedthevideoaftertheyfirstsawtheassociatedad.

BounceRate:thepercentageofsingle-webpagevisitsorvisitsinwhichtheuserleftthesitefromwithoutinteractingwiththepage.

Click-throughRate:thenumberofusersthatclickonanadoutofthetotalnumberofusersthatareexposedtothead.

Drop-offrates:thenumberofusersinteractingwithawebpageinsomeway(e.g.scrolleddownorclickedonembeddedcontent)andthenleavingthewebsite.

EarnedView:whenauserviewsavideoadvertandthenwatchesanothervideoorvideosontheassociatedYouTubechannel.

EmbeddedContent:contentthatishostedononewebsitebutisthenincludedonpagesofotherexternalwebsites(e.g.YouTubevideosorPrezis).

Engagement:thelevelofuserinteractionwithwebsites,onlinecontent,andsocialmediaaccounts.

FacebookAds:atextandimagebasedadvertthatappearsinpre-definedtargetaudiences’newsfeedsandside-bars.

FacebookBoostedPosts:poststhathavebeenpromotedtoappearinselectedtargetaudiences’newsfeeds.

Impressions:thenumberoftimescontentoradvertsappearedonusersscreens.

Organic:userreachorengagementonwebsitesorsocialmediaplatformsgeneratedfromsearchesand/orasaresultofunpaiddisseminationstrategiesandtactics.

Reach:thenumberofusersexposedtocontentoradvertisingonline.

SearchEngineOptimisation(SEO):theprocessofmanipulatingaspectsofawebsiteorwebpagetomaximiseitsvisibilityintheresultsofsearchengines(e.g.submittingthewebsitetodirectoryservices,designingwebsitearchitectureandcontentforsearchengine“crawlers”).

YouTubeAnnotations&BannerOverlays:Customisabletextand/orimagebased“pop-ups”thatcanbeaddedtoownedvideosorpaidforthroughadvertising.

YouTubeIn-displayAds:theadvertisedvideothumbnailandtextisdisplayedadjacenttootherYouTubecontentintheside-bar,searchresultsorinabanneroverlay.

YouTubeIn-streamAds:theadvertisedvideoisdisplayedpriortootherYouTubecontent,withviewersunabletoskipthevideoforacompulsoryminimumof5seconds.

YouTubeWatchPage:theuniqueYouTubewebpageorURLthatcontainsthevid

41

ExtremeDialogue:Project&Partners

LaunchedacrossCanadainFebruary2015,ExtremeDialogueaimstoreducetheappealofextremismamongyoung

peopleandofferapositivealternativetotheincreasingamountsofextremistmaterialandpropagandaavailableon

theInternetandsocialmediaplatforms.

AseriesofshortdocumentaryfilmstellthepersonalstoriesofCanadiansprofoundlyaffectedbyviolentextremism;a

formermemberoftheextremefar-rightandamotherwhosesonwaskilledfightingforISISinSyria.Thefilmsare

accompaniedby a set of educational resources that can be usedwith youngpeople in classroomsor community

settingsandareintendedtobuildresiliencetoextremismthroughactivediscussionandenhancedcriticalthinking.

Fundedby Public Safety Canada via the Kanishka Project, the project is a collaboration between the Institute for

StrategicDialogue, film-makersDuckrabbit, theeducationalcharityTimParryJohnathanBallFoundationforPeace

andtheSecDevFoundation.

TheInstituteforStrategicDialogue

TheInstituteforStrategicDialogue(ISD)isanindependent‘thinkanddotank’workingwithleadersingovernment,

business,civilsocietyandacademiatodevelopcross-borderresponsestothemajorgeo-strategic,socialandsecurity

challengesofourtime.

Combiningresearchandpolicyadvisorywork,with innovativedeliveryprogrammes,specialisedtask forces,cross-

sectorpartnershipsandnetworks,ISDworkstocounterglobalextremism,bridgeinter-communaldividesandenhance

Europe’scapacitytoacteffectivelyintheglobalarena.

TheSecDevFoundation

TheSecDevFoundationisaCanadian-basedthink-dotankthatworksatthecross-roadsofsecurityanddevelopment

–becauseyoucan’thaveonewithouttheother.

Weworkwithlocalstakeholdersincountriesandregionsaffectedbyconflict, insecurityandfragilityintheMiddle

East,LatinAmerica,Eurasia,AfricaandAsia.Weuseevidence-basedresearchtoextendandempowerlocalknowledge

andresilience.Ouraimistobuildabetterworldthroughopenintelligenceandempowerment