JUNE 19, 2019
Revising your first paperDTGS 2019
Ilya MusabirovDenis BulyginAnastasiya Kuznetsova
nosoc.io/events/dtgs19
SHORT INTRODUCTIONTell about yourselves
Who are you?
What are your interests?
Why are you here?
You have 20 seconds
RESEARCH STRUCTURE
• Interesting, up-to-date and clear problem
• Knowledge of previous studies
• Evident contribution to the development of the field
• Reasonable analysis
What are the features of a good research?
TODAY’S WORKSHOP FOCUS
• Interesting, up-to-date and clear problem
• Knowledge of previous studies
• Evident contribution to the development of the field
• Reasonable analysis
What are the features of a good research?
STRUCTURING THE ARTICLE
TYPES OF CONTRIBUTIONS IN THE SOCIAL SCIENCES○ The contribution of empirical research - test theory on data
and analytics○ Methodological contribution - validation and evaluation of
methods and tools○ Theoretical contribution - clarification and / or discussion of the
theory○ Data contribution - data publication for the analysis and testing
of tools and methods○ Knowledge synthesis - review and meta-analysis of previous
works○ Contribution of opinion - an essay that convinces in something
• Abstract• Introduction• Literature review• Methodology• Results• Discussion• Conclusion
THE HOURGLASS STRUCTURE
https://umc.libguides.com/c.php?g=686833&p=4853772
• Abstract• Introduction• Literature review• Methodology• Results• Discussion• Conclusion
THE HOURGLASS STRUCTURE
https://umc.libguides.com/c.php?g=686833&p=4853772
A reader is constantly making a decision to choose what article to read and whether she should read articles at all
Our goal is to show the reader what we did and why this is important
Otherwise, our target audience won't find the article useful beforehand
Abstract is the easiest way to convey the short message of you research
The short form of introduction
Science as compete for attention
Questions to abstract (checklist)
❏ What is the research space?
❏ What is the problem or research gap?
❏ How are we fixing this problem in this paper?
❏ What did we find out? What are our results?
❏ How does this contribute to [field]?
“How to write CHI papers” (2018)
ExampleSound and virtual reality (VR) are two important output modalities for creating an immersive player experience (PX). While prior research suggests that sounds might contribute to a more immersive experience in games played on screens and mobile displays, there is not yet evidence of these effects of sound on PX in VR. To address this, we conducted a within-subjects experiment using a commercial horror-adventure game to study the effects of a VR and monitordisplay version of the same game on PX. Subsequently, we explored, in a between-subjects study, the effects of audio dimensionality on PX in VR.Results indicate that audio has a more implicit influence on PX in VR because of the impact of the overall sensory experience and that audio dimensionality in VR may not be a significant factor contributing to PX. Based on our findings and observations, we provide five design guidelines for VR games.
https://dl.acm.org/citation.cfm?doid=3173574.3173902
• Abstract• Introduction• Literature review• Methodology• Results• Discussion• Conclusion
THE HOURGLASS STRUCTURE
https://umc.libguides.com/c.php?g=686833&p=4853772
IntroductionContains answers to key questions that reviewers are searching for
● What problem from the real world is the author trying to solve?● Why is it so important to solve this problem?● What is the solution for this problem and how did they come to
it?● How do they understand that their solution is a good one?
These questions should help in assessing and analyzing the article
You can also ask them by yourself while writing this article
“How to write CHI papers” (2018)
IntroductionContains answers to key questions that reviewers are searching for
● What problem from the real world is the author trying to solve?● Why is it so important to solve this problem?● What is the solution for this problem and how did they come to
it?● How do they understand that their solution is a good one?
These questions should help in assessing and analyzing the article
You can also ask them by yourself while writing this article
“How to write CHI papers” (2018)
HOW TO DEFINE A RESEARCH PROBLEMWhat is the basis of the research problem?
● Field of research
Gap in research
➔ There is knowledge on topic X ➔ But thing Y is still not analyzed ➔ Analyzing thing Y is important because ...
Why is it important to close this gap in knowledge? What will change?
The knowledge gap is often based on existing literature
● Gap in research
RESEARCH PROBLEM: AN EXAMPLE“Understating why people purchase virtual goods is a pertinent practical issue for game businesses and virtual world operators. Virtual goods are primarily being operationalized as the source of revenue for otherwise free games (free-to-play) [3, 5, 6, 7] and virtual environments… However, only a minute percentage of registered users seem to purchase virtual goods (2% according to Forbes [9]).”
Therefore, to meet this looming research problem, we performed a literature review that provides an overview to the studies investigating the questions of what predicts virtual good purchases.
Hamari, Juho, and Lauri Keronen. "Why do people buy virtual goods? A literature review." In 2016 49th Hawaii International Conference on System Sciences (HICSS), pp. 1358-1367. IEEE, 2016.
RESEARCH PROBLEM: AN EXAMPLE“Understating why people purchase virtual goods is a pertinent practical issue for game businesses and virtual world operators. Virtual goods are primarily being operationalized as the source of revenue for otherwise free games (free-to-play) [3, 5, 6, 7] and virtual environments… However, only a minute percentage of registered users seem to purchase virtual goods (2% according to Forbes [9]).”
Therefore, to meet this looming research problem, we performed a literature review that provides an overview to the studies investigating the questions of what predicts virtual good purchases.
Hamari, Juho, and Lauri Keronen. "Why do people buy virtual goods? A literature review." In 2016 49th Hawaii International Conference on System Sciences (HICSS), pp. 1358-1367. IEEE, 2016.
SPLIT PROBLEM INTO PARTS
Knowledge gapUnderstating why people purchase virtual goods is a pertinent practical issue for game businesses and virtual world operators
The importance of this gapVirtual goods are … the source of revenue [3, 5, 6, 7] … However, only a minute percentage of users seem to purchase virtual goods (2%[9]).”
Contribution to the knowledgewe ... provide an overview to the studies investigating the questions of what predicts virtual good purchases.
SHOW KNOWLEDGE OF THE AREAKnowledge gapUnderstating why people purchase virtual goods is a pertinent practical issue for game businesses and virtual world operators
The importance of this gapVirtual goods are … the source of revenue [3, 5, 6, 7] … However, only a minute percentage of users seem to purchase virtual goods (2%[9]).”
Contribution to the knowledgewe ... provide an overview to the studies investigating the questions of what predicts virtual good purchases.
Clear research problem (checklist)❏ Knowledge gap❏ The importance of the gap❏ (Potential) Contribution to the
knowledge
Define research question
Research question is a statement of research problem / gap in the form of question
RP: We don’t know about thing Y
RQ: How thing Y is working?
Good research question (checklist)
❏ It can be answered❏ Nobody knows the answer to it (nobody cares)❏ Not very general (impossible to find the answer)
and not narrow (no one is interested)❏ Provokes discussion (not a yes / no answer
question)❏ Usually contains questions like, why, what,
under what conditions?
Good research question “we performed a literature review that provides an overview to the studies investigating the questions of what predicts virtual good purchases”
What predicts virtual good purchases?
✓ Is it possible to answer this question? Factors can be identified✓ Does anybody know the answer? A review of articles says that not✓ Is it wide enough? Interesting to the entire area of virtual
purchases✓ Is it specific enough? But focuses on one task✓ Does it provoke the discussion ? Discussion of the role of game
types and contexts✓ Answers questions in “Under what conditions?”, “In what
situations?”, etc.
BadWhich website is the
most visited in Russia?
Is Internet popular in
Russia?
Examples of research questions
BadWhich website is the
most visited in Russia?
Is Internet popular in
Russia?
Examples of research questions
BetterWhy VK.COM is the
most visited social network in Russia?
TRY TO REFORMULATE THE RQIs blockchain the future?
Analysis of the impact of social media on electoral behavior
Does the introduction of gamification increase CLV on the site carpenter.rf?
● It can be answered● The answer was not find yet (nobody cares)● Not very wide (impossible to find the answer) and not narrow (no one is
interested)● Provokes discussion (not a yes / no answer question)
● Usually contains questions like, why, what, under what conditions?
INDIVIDUAL TASKHere is the checklist for a good abstract
Try to formulate your own one in 10 minutes
❏ What is the research space?
❏ What is the problem or research gap?
❏ How are we fixing this problem in this paper?
❏ What did we find out? What are our results?
❏ How does this contribute to [field]?
“How to write CHI papers” (2018)
• Abstract• Introduction• Literature review• Methodology• Results• Discussion• Conclusion
THE HOURGLASS STRUCTURE
https://umc.libguides.com/c.php?g=686833&p=4853772
LITERATURE SEARCH
Part 1
BASIC WAYS OF LITERATURE SEARCH1. Citation databases: Google Scholar, WoS, Scopus2. Relevant and credible scientific journals and
conference proceedings on the topic3. Meta-analytical papers and literature reviews4. Analysis of bibliometric networks constructed in
VosViewer software based on WoS/Scopus data
Where do we search for literature - Google ScholarDo you already have some relevant papers on the topic? ->Look up what the paper cites and which other papers cite the one
Where do we search for literature - journals and conference proceedingsHow to check the quality of a journal:https://www.scimagojr.com/index.php (Impact Factors)https://www.scopus.com/sources (Quartiles)White and Black lists of journals in the university
Collections of conferences with a serious selection of applications:
● The ACM CHI Conference on Human Factors in Computing Systems (CHI)● The International AAAI Conference on Web and Social Media (ICWSM)● The ACM Conference on Computer-Supported Cooperative Work and Social
Computing (CSCW)
ACM Digital Library Publications https://dl.acm.org/
Where do we search for literature - meta-analysis and literature reviewsDoes Gamification Work? — A Literature Review of Empirical Studies on Gamification (Juho Hamari, Jonna Koivisto, Harri Sarsa)
● “This paper reviews peer-reviewed empirical studies on gamification”
● Collection of papers from different databases
● Detection of key gamification design-patterns
● Identification of the most popular contexts of gamification and its main effects
DISADVANTAGES OF SEARCH BASED ON SELECTED METHODS
Can miss influential papers
HOW TO DEAL WITH IT?
VosViewer maps can help in grasping the main research directions and understanding the relations between the subfields
Accurately formulated search query and properly constructed bibliometric networks allow extracting the most important directions of research on your topic, as well as identify influential papers and a space for your research question
Web of Science / Scopus + VosViewer
An example of literature review based on bibliometric networksRQ: It would be ideal to investigate also the process of creation, transfer and development of knowledge from a dynamic perspective, to unveil its evolution over time rather than just providing a static snapshot of it. The goal of this paper is to depict a landscape of the scientific literature on the Smart Factory concept.
https://hull-repository.worktribe.com/OutputFile/826612
1. Formulate a search queryWeb of Science / Scopus
2. Filter out irrelevant research fields and outdated publicationsVoS
Literature Review Step-by-Step
3. Construct bibliometric networks of ...
● co-occurrence of key terms
● co-authorship● co-citation...
4. Interpret generated network based on clusters detected
How to choose words for a search queryThe query shall be broad enough for covering a good share of papers but narrow enough for being focused on your research topic
An optimum number of output in WoS / Scopus is 1500-2000 papers
You can find the terms for a search query in the papers which you already know
Logical operators help a lot:
● AND -- a paper shall include both terms (narrowing the output)● OR -- a paper should include either of indicated terms (making the output
broader)● “ ” -- a paper contains a phrase formulated exactly this way (e.g. “reputation
system”● * -- matches any symbols after a combination before a space (e.g. “system*” will
return both system и systems)
“reputation system*” OR “reputation mechanism*”
2. Filters out irrelevant output (i.e. papers from non-related research fields, outdated publications)
3. Exports the data (title, keywords, abstract, authors + affiliations, references)
4. Inputs the data to VosViewer
An example of a search queryLet’s assume that we are interested in reputation systems
Formulated query:
“reputation mechanism*” OR "reputation system*"
Quotation marks plural and singular
extention
DEFINE KEY WORDSBased on abstracts
Choose 5-7 words which might be of use for a search query
Research Question: what are the effects of collaborative learning in programming
Exporting the output (based on WoS case)
other files formats
only 500 at a time
VosViewerWhat can be a vertex?An author | a paper | a journal| a term | a country | a university (organization)
What can be an edge based on?Citations | Co-authorship| Co-occurrence | co-citation
Download:http://www.vosviewer.com/downloadDocumentation:http://www.vosviewer.com/getting-started
A map of term co-occurrence
- the link between two vertices (i.e. terms) appears when both occur in the title, key words list or abstract of the same paper
- term co-occurrence network helps to understand what the research field covers and how the relations between research subfields look like
A map of co-citations- the link between two vertices means that
a paper refers to both publications- one makes use of this map when
there is a need to identify representatives of differentresearch direction withina field
A map of co-authorship- the link between two vertices appears
when two authors published a paper together
- co-authorship network helps to find the most important research teams and collaborations which work on the topic
- the authors can be replaced with institutions
https://www.elsevier.com/editors-update/story/practical-tips/how-to-generate-journal-insights-using-visualization-techniques
TASK - Try to interpret the map of term co-occurrence
6 selected Nursing journals (Scopus)
https://www.elsevier.com/editors-update/story/practical-tips/how-to-generate-journal-insights-using-visualization-techniques
statistics and experiments
education
surgery
lit reviews
LITERATURE REVIEW
Part 2
What should be in the literature review?
1. What are the main theories used to explain the phenomena and processes on your topic? What theories were used before, and what now?
2. What hypotheses did the authors put forward based on these theories?
3. What methods and data are used to test hypotheses?4. What empirical findings were made by authors in your field? Do
these findings contradict each other?5. Reflection and synthesis: what gaps in the research area have you
noticed? What methods and theories are outdated? In which direction is it important to continue research?
Examples“However, technology acceptance model [43], theory of planned behavior [44] and unified theory of acceptance and use of technology [45] were found in more than one study.“
“Among the two variables measuring behavior in the literature, purchase intention and behavior, intention to purchase virtual goods was adopted by clear majority of the studies”
“Clear majority of studies used structural equation modeling as their analysis method. Most of the studies employed covariance based SEM (13) while partial least squares SEM was also rather frequent (12)”
Hamari, J., & Keronen, L. (2016, January). Why do people buy virtual goods? A literature review. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 1358-1367). IEEE.
Checklist for the literature review
❏ The main works are listed (the oldest, most cited, most cited from the newest)
❏ State of the art is described (the current state of the research area
❏ Described how they feature our work❏ Described how they differ from our work❏ Marked a gap between existing works and our
Examples [2]“Based on the weighted means of more frequently studied paths, the strongest variable affecting purchase intention is satisfaction with virtual goods with path coefficient of 0.686. Moreover, both self-presentation and attitude towards purchasing virtual goods are rather strong with values of 0.467 and 0.437, respectively”
“The results of the review reveal that there has been a significant increase in the literature on the topic from year 2011 onwards till today. The body of literature has focused on a variety of environments of virtual goods purchase behavior such as social virtual worlds, social networking sites and different type of games. “
Hamari, J., & Keronen, L. (2016, January). Why do people buy virtual goods? A literature review. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 1358-1367). IEEE.
To Show Similarity Macey (2011) found that the average four-year-old in the U.S. watches fout hours of TY each day. Likewise, in Australia four-year-olds watch several hours of TY daily.
As in Australia, the average four-year-old in the U.S. watches more than three hours of TY each day (Macey, 2011).
Likewise, ...
As in X in Y . ..
Academic Writing for Graduate Students, 3rd Edition Essential Tasks and Skills, 3rd Edition, John M. Swales & Christine B. Feak
To Show Contrast Unlike Indonesians who have limited access to the internet, the majority of Japanese have easy access (Macey, 2011).
In developed countries internet access is viewed as a necessity. In most African countries, on the other hand, it is a luxury (Macey, 2011).
Unlike X Y ...
On the other hand, ...
Academic Writing for Graduate Students, 3rd Edition Essential Tasks and Skills, 3rd Edition, John M. Swales & Christine B. Feak
Templates for the overall description The research has focused on a variety of __________________
In discussions of X, one controversial issue has been _____________. On the one hand, argues ____________. On the other hand, contends ________________. Others even maintain _________. My own view is __________.
A number of data scientists have recently suggested that _________.
It has become common today to dismiss _______.
In their recent work, Y and Z have offered harsh critiques of for ____________.
Graff, Gerald, Cathy Birkenstein, and Russel Durst. "They say, I say." The Moves That Matters in Academic Writing (2006).
Templates for expressing agreementI agree that _________ because ____________ confirms it.
X is surely right about because, as she may not be aware, recent studies have shown that _____________.
X’s theory of is extremely useful because it sheds light on the difficult problem of __________.
Those unfamiliar with this school of thought may be interested to know that it basically boils down to ___________.
Graff, Gerald, Cathy Birkenstein, and Russel Durst. "They say, I say." The Moves That Matters in Academic Writing (2006).
Templates for expressing disagreementX’s claim that ________ rests upon the questionable assumption that _________.
I disagree with X’s view that because, as recent research has shown, _______.
X contradicts herself/can’t have it both ways. On the one hand, she argues _____.
On the other hand, she also says ________________.
By focusing on , X overlooks the deeper problem of _____________.
Graff, Gerald, Cathy Birkenstein, and Russel Durst. "They say, I say." The Moves That Matters in Academic Writing (2006).
Templates for partial agreement Proponents of X are right to argue that ___________.
But they exaggerate when they claim that _____________.
While it is true that , it does not necessarily follow that___________.
My view, however, contrary to what X has argued, is that ____________.
Adding to X’s argument, I would point out that _______________.
According to both X and Y, ________________.
Graff, Gerald, Cathy Birkenstein, and Russel Durst. "They say, I say." The Moves That Matters in Academic Writing (2006).
Do we we still
have time?
BONUS: how to name the paper Conferences
● Articles with question marks (?) in titles are less cited● Colons (:) increase citations
Journals
● Maximize citations with no colons, no question marks
Transactions
● More question marks in titles have more citations
VISUAL EFFECT
Part 3
Posters and diagrams
Think about sections, blank spaces, fonts, colors, visibility and usability
think like a designer!
CONSISTENT COLORS AND FONTS
- One pallet (2-3 colors), not too bright- Colors should match with each other (check for
colorblindness)- Light background (white), dark text - 72 size for title, 48 - subtitle, 24 - main text- Sans serif font-
https://colinpurrington.com/2012/02/example-of-bad-scientific-poster/
http://slideplayer.com/slide/11688869/
use simple diagrams which suits your data
Delete unimportant things
https://www.slideshare.net/MarkLawrence7/marktlawrenceibmdatavizadj10apr2014
FOCUS THE ATTENTIONMake your audience to see what you want them to see before they even know they’re seeing it
For your text
Nussbaumer Knaflic, Cole. "Storytelling with data: a data visualization guide for business professionals." (2015).