ABriefIntroductiontoQualtrics SurveyDesign

NicholasJacobsUniversityofVirginia

CONTENTSAccessingQualtrics

BasicSet-UpRandomization

PreviewingSurveyandLaunchingExportingData

AdvancedSurveyOptionsAdvancedQuestionOptionsInsertingGraphicsandVideoConditioningQuestions

Youcanseethemainsurveywe’llbedesigninghere:https://virginia.az1.qualtrics.com/jfe/form/SV_50FgnmItVrMCcOF

Caretoseetheresultsofthisactualexperiment?See:NicholasF.Jacobs.2017.“AnExperimentalTestofHowAmericansThinkAbout

Federalism,”Publius:TheJournalofFederalism47(4):572-598.

AccessingQualtricshttp://www.its.virginia.edu/software/qualtrics/login.html

AccountisalreadycreatedwithNetBadge

STARTHERE

BasicSet-UpCreatingNewSurveys

2

3

4

1

AddingQuestions

StartTyping

Question&AnswerOptions

AddingQuestions

DifferentTypesofQuestions

“Static”NoRespondentInteraction/Choice

OptionswithinOptions

Withexamples

AddingQuestions

FontandFormattingOptions

AddingQuestions

RandomizationCreatingBlocks

Questionsexistinside“blocks”

Youcanhaveasmanyorasfewblocksasyouprefer

Inordertorandomize,youwillneedtoplacetheEachcondition/manipulationinitsownblock

Therefore,theminimum numberofblocksyoursurveywillcontainistwo

CreatingBlocks

12

3

5

4

6

Intheend,wewillhaveFOURblockssofarinthisexamplesurvey

“SurveyFlow”

The“SurveyFlow”allowsyoutoseethewhole“logic”ofyoursurvey

AddingaRandomizer

NB:YoucanAdd“Blocks”throughthisinterfaceaswell

Clickanddragtheblocksintotherandomizer

AddingaRandomizer

Youwanttodragthe“block”ontopofthepurplerandomizerelement

Clickanddragtheblocksintotherandomizer

TorandomlyassigneachrespondenttoONEconditionmakesureyouenterthenumber“1”here

PreviewingSurveyandLaunching

Backtothe“SurveyFlow”interface:

Noticeafewthingsaboutthisparticularlogic:

1. Wearerandomlyshowingelementofthethreedifferenttreatmentgroups/“blocks”

2. Weareshowingeachrespondentasingledependentvariablelabeled“commondependentvariable”

3. Weareshowingeachrespondentasetoffourdemographicquestionsandasingledebrief

4. Weare“ending”thesurvey– thisisitsown“element”thatyoumustadd

5. Youmustalsomakesureto“saveflow”whenyoumakethesechanges.Thisreturnsyoutothequestionscreen.

Anewtabinyourbrowseropenswithamobileviewandadesktopview.Respondentswillseeoneofthesedependingon

theirdevice.NB:Youarealsogivenapreviewlink.

Previewing

Launching

NB:Onceyou“launch”yoursurveybycreatingalinkyouwillneedto“unlock”and“lock”yoursurveytosavechanges.

Automaticpromptscomeupwhenediting.

ExportingData

Qualtrics offersanumberof“in-interface”toolsformakingsenseofrespondents.Youarefreetousethese.However,inordertomeetformattingguidelines/expectationsforthefinalresearchpaper,youshouldexportyourresultstoadata-managementtoolyoufeelcomfortablewith(e.g.,Excel).

Exporting

ExportingasaCSVfilewillallowyoutousenumericvaluesor“choicetext.”

*Usenumericvalues(quantities)foreasieranalysisinExcel.

AdvancedOptionsSurveyOptions

Conventioninexperimentsistoprohibit

Dependsheavilyonhowyouplantodistributeaccess

SurveyOptions

Mindthe“Lock”wheneveryoumakechangesafter

launching

AnswerOptionsLet’ssayyouwantedtorandomizetheorderinwhichtheseCHOICESappearedforthisparticularquestion(makesurethatrespondentsare

notmorelikelytochoosethefirstoptionsurvey-to-survey)

InsertingGraphicsLet’ssayyouwantedtouseimagesastreatmentconditions.

Respondentswillallanswerthesamequestion.First,createhowevermanyblocksasthereareconditions(treatmentwithimage,treatment

withdifferentimage,treatmentwiththirdimage:3blocks)

Withineachblock,addaquestion.Choosequestiontype“Text/Graphic”

Clickbelowtoseewhatwe’redesigning:https://virginia.az1.qualtrics.com/jfe/preview/SV_3Q7lb70iYcLxSqp?Q_CHL=preview

InsertingGraphicsWithineachblock,addaquestion.Choosequestiontype“Text/Graphic”

InsertingGraphicsKeepas“Text”evenifyouareusingpictures(abletoeditsize)

1

2(doubleclick)

3(doubleclickimageinrichtexteditor)

Thisiswhereyoucanchangetheimagesizerelativetoothertextelements.The“padlock”willmaintainimagewidth/heightrelationship

650pt.heightisreasonable.

InsertingVideoLet’ssayyouwantedtouseVIDEOSastreatmentconditions.

Respondentswillallanswerthesamequestion.First,createhowevermanyblocksasthereareconditions(treatmentwithVideoOne,

treatmentwithVideo2,controlwithnovideo:3blocks)

Withineachblock,addaquestion.Choosequestiontype“Text/Graphic”

InsertingVideKeepas“Text”evenifyouareusingpictures(abletoeditsize)

1

EitheruseathirdpartyURL(e.g.,

YouTube)oruploadthevideo(sizerestrictions)

TimingQuestionsLet’ssayyouwantomakesuresomeoneSAWthewholevideo,orspenta

certainamountoftimelookingatthegraphic

2

1

3

TimingQuestions

Thistextwillnotappeartorespondents Thiswillkeeptheadvance

arrowatthebottomrightofthescreenhiddenfor

fivesecondsNB:Theoptionbelowautomaticallyadvances

thequestion

Thetimer“question”shouldcomeimmediatelyafterthestimulus/pageyouwanttorestrictor

advance

ConditioningQuestionsLetssayyouwanttoturnthisquestionaboutpartisanidentificationintoaseriesofquestionswheretheanswertoonequestionconditionswhatquestionandansweroptionsfollow:

CONDITIONEDPartisanIdentification

A: Independent

“Generally speaking, do you usually think of yourself as a Republican, a Democrat, an Independent, or what?”

A: Republican A: Democrat

Q Follow up: Would you call yourself a weak Republican or a

strong Republican?

Q Follow up:Do you think of

yourself as closer to the Republican or Democratic Party?

Q Follow up: Would you call yourself a weak

Democrat or a strong Democrat?

A: Strong Rep.

A: Weak Rep.

A: Rep.

A: Dem.

A: Neither

A: Strong Dem.

A: Weak Dem.

Rewritethefirstquestiontoappearasaboveandusethesmallgreencirclestoaddthreeadditional

questionstoappearimmediatelyafter

ConditioningQuestions

ForQuestions21-23,wearegoingtoadda“displaylogic”dependentonanswerchoicestoquestion13

NB:thenon-sequentialnumberingisduetotimelineofquestioncreation

ConditioningQuestions

ConditioningQuestions

SinceforthisquestionweareaskinghowstronglytheyidentifywiththeREPUBLICANParty,weonlywanttoaskthemthisquestioniftheyanswered“Republican”to

question13

ConditioningQuestions

ConditioningquestionsisaPOWERFULtool.Asyouscrollthrough

thevariousoptions,youwillseejusthowcustomizableyoucan

designsurveysbyfollowingthisbasiclogic.

BECAREFUL– Itiseasytoconditionyourwayoutoftrue

randomizationintotreatmentandcontrolconditions

Qualtrics offersanextensiverangeofsupportservicesandFAQs:https://www.qualtrics.com/support

Oftentimesfindingthe“right”answerislearningthelanguageofsurveyresearch.Assuch,a“google”

searchmighthelpyoutranslatewhatyouwanttodo

intosurveyorterminology.

“Right”isgettingittodowhatyouwanttodo,andthereareoftenmultiplewaystogetthejobdone!