Pervasive Magazine 2015.01.01

Volume 14, Number 1 JanuaryMarch 2015

Telemedicine in the Cloud p. 54

Activity Recognitionp. 70

Thu Human Intranetp. 78

Privacy & Security

www.computer.org/pervasive

Contents | Zoom in | Zoom out Search Issue | Next PageFor navigation instructions please click here

Contents | Zoom in | Zoom out Search Issue | Next PageFor navigation instructions please click here

Although cloud technologies have been advanced and adopted at an astonishing pace, much work remains. IEEE Cloud Computing seeks to foster the evolution of cloud computing and provide a forum for reporting original research, exchanging experiences, and developing best practices.

IEEE Cloud Computing magazine seeks accessible, useful papers on the latest peer-reviewed developments in cloud computing. Topics include, but arent limited to:

3 Cloud architectures (delivery models and deployments),3 Cloud management (balancing automation and robustness with monitoring and

maintenance),

3 Cloud security and privacy (issues stemming from technology, process and governance, international law, and legal frameworks),

3 Cloud services (cloud services drive and are driven by consumer demand; as markets change, so do the types of services being offered),

3 Cloud experiences and adoption (deployment scenarios and consumer expectations),3 Cloud and adjacent technology trends (exploring trends in the market and impacts on

and infl uences of cloud computing),

3 Cloud economics (direct and indirect costs of cloud computing on the consumer; sustainable models for providers),

3 Cloud standardization and compliance (facilitating the standardization of cloud tech and test suites for compliance), and

3 Cloud governance (transparency of processes, legal frameworks, and consumer monitoring and reporting).

Submissions will be subject to IEEE Cloud Computing magazines peer-review process. Articles should be at most 6,000 words, with a maximum of 15 references, and should be understandable to a broad audience of people interested in cloud computing, big data, and related application areas. The writing style should be down to earth, practical, and original.

All accepted articles will be edited according to the IEEE Computer Society style guide. Submit your papers through Manuscript Central at https://mc.manuscriptcentral.com/ccm-cs.

If you have any questions, feel free to email lead editor Brian Kirk at [email protected].

IEEE Cloud ComputingCall for Papers

www.computer.org/cloudcomputing

_____________

qqMM

qqMM

qM

THE WORLDS NEWSSTAND

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

qqMM

qqMM

qM



MOBILE AND UBIQUITOUS SYSTEMS


Call for Papers:

Special Issue on Smart Food

Food, glorious food! There is currently a huge focus on food in todays culture: where it comes from, how its produced, how its transported, the culinary and dining experience, and, of course, if it is good for us. Additionally, the role of technology around food is growing.

The aim of this special issue is to explore technologies related to all aspects of food and agriculture in pervasive computing. Relevant topics for this special issue include, but are not limited to, the following

r Technology for restaurantsr Technology for enhancing the

dining experiencer Technology for the food supply

chainr Technology for food transportationr Tracking food from farm to tabler Technology for food provenancer Technology to support proper food

preparation, disposal, and waster Technology for knowing/

understanding nutrition information

r Technology to support the agricultural industry, such as sensing the health of plants and animals

r Technology for the social and sharing aspects of food

r Understanding how people use and experience technology for food

The guest editors invite original and high-quality submissions addressing all aspects of this fi eld, as long as the connection to the focus topic is clear and emphasized. Review or summary articles for example critical evaluations of the state of the art, or insightful analysis of established and up-coming technologies may be accepted if they demonstrate academic originality in discussing and highlighting academic relevance.

Guest Editorsr Anind Dey, Carnegie Mellon

Universityr Florian Pinel, IBM T. J. Watson

Research Centerr Eva Ganglbauer, Vienna University

of Technology

Submission InformationSubmissions should be 4,000 to 6,000 words long and should follow the magazines guidelines on style and presentation. All submissions will be single-blind anonymously reviewed in accordance with normal practice for scientifi c publications.

For more information, contact the guest editors at [email protected].

To present works-in-progress directly to the community, visit our Reddit community: www.re ddit.com/r/pervasivecomputing.

For author guidelines, see www.computer.org/pervasive/author.htm.

For submission details, email [email protected].

To submit an article, visithttps://mc.manuscriptcentral.com/pc-cs.

3XEOLFDWLRQ2FWz'HFf6XEPLVVLRQGHDGOLQH-DQ

ZZZFRPSXWHURUJSHUYDVLYH

___

_____________________________

________________

_____________

________________

qqMM

qqMM

qM



qqMM

qqMM

qM





ALL ARTICLES ARE PEER REVIEWED

page 12

Reuse Rights and Reprint Permissions: Educational or personal use of this mate-rial is permitted without fee, provided such use: 1) is not made for profit; 2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-party products or services. Authors and their companies are permitted to post the accepted version of their IEEE-copyrighted material on their own web servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the first screen of the posted copy. An accepted manuscript is a version that has been revised by the author to incorporate review suggestions, but not the published ver-sion with copyediting, proofreading, and formatting added by IEEE. For more infor-mation, please go to: www.ieee.org/publications_standards/publications/rights/paperversionpolicy.html.

Permission to reprint/republish this material for commercial, advertising, or promo-tional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or [email protected]. Copyright 2015 IEEE. All rights reserved.

Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per-copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.

WWW.COMPUTER.ORG/PERVASIVE

THEME:

Privacy & Security16 Guest Editors Introduction: Privacy and SecuritySunny Consolvo, Jason Hong, and Marc Langheinrich18 Engineering Gesture-Based Authentication Systems

Gradeigh D. Clark and Janne Lindqvist

26 Social Access vs. Privacy in Wearable Computing: A Case Study of AutismReuben Kirkham and Chris Greenhalgh

34 Context-Adaptive Privacy: Leveraging Context Awareness to Support Privacy Decision MakingFlorian Schaub, Bastian Knings, and Michael Weber

44 Security and Privacy Implications of Pervasive Memory AugmentationNigel Davies, Adrian Friday, Sarah Clinch, Corina Sas, Marc Langheinrich, Geoff Ward, and Albrecht Schmidt

7PMVNF/VNCFSt+BOVBSZo'FCSVBSZ

page 8

Cover art:Rob Magiera

Features54 Telemedicine in the Cloud Era: Prospects and ChallengesZhanpeng Jin and Yu Chen62 A Participatory Service Platform for Indoor Location-Based Services

Hyojeong Shin, Yohan Chon, Yungeun Kim, and Hojung Cha

70 Competitive Live Evaluations of Activity-Recognition Systems Hristijan Gjoreski, Simon Kozina, Matja Gams, Mitja Lutrek, Juan Antonio lvarez-Garca, Jin-Hyuk Hong, Julian Ramos, Anind K. Dey, Maurizio Bocca, and Neal Patwari

___________ ___________________________________

_____________

qqMM

qqMM

qM



qqMM

qqMM

qM





Submissions: Access the IEEE Computer Societys Web-based system, Manuscript Central, at https://mc.manuscriptcentral.com/pc-cs. Be sure to select the right manuscript type when submitting. Articles must be original and should be approximately 5,000 words long, preferably not exceeding 15 references. Visit www.computer.org/pervasive for editorial guidelines.

Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the authors or firms opinion. Inclusion in IEEE Pervasive Computing does not necessarily constitute endorsement by the IEEE or the IEEE Computer Society. All submissions are subject to editing for style, clarity, and length. For more information on any computing topic, please visit our Digital Library at www.computer.org/csdl.

__________

______

EDITOR IN CHIEFMaria R. Ebling

IBM TJ Watson Research [email protected]

ASSOCIATE EDITORS IN CHIEF

EDITORS IN CHIEF EMERITINigel Davies

Lancaster UniversityRoy Want

GoogleM. Satyanarayanan

Carnegie Mellon University

EDITORIAL BOARD

Anind DeyCarnegie Mellon

UniversitySteve Hodges

Microsoft Research

Jason HongCarnegie Mellon University

Marc LangheinrichUniversit della Svizzera

Italiana (USI)

CS MAGAZINEOPERATIONS COMMITTEE

Paolo Montuschi (chair), Erik R. Altman, Maria Ebling, Miguel Encarnao, Dave Walden,

Cecelia Metra, San Murugesan, Shari Lawrence Pfleeger, Michael Rabinovich,

Forrest Shull, George K. Thiruvathukal, Ron Vetter, and Daniel Zeng

CS PUBLICATIONS BOARDJean-Luc Gaudiot (VP for Publications),

Alain April, Laxmi N. Bhuyan, Angela R. Burgess, Greg Byrd, Robert Dupuis,

David S. Ebert, Frank Ferrante, Paolo Montuschi, Linda I. Shafer, H.J. Siegel,

and Per Stenstrm

Mary BakerHewlett-Packard Labs

Elizabeth M. BeldingUC Santa Barbara

A.J. Bernheim BrushMicrosoft Research

John CannyUC Berkeley

Sunny ConsolvoGoogle

Hans GellersenLancaster University

Mike HazasLancaster University

Stephen IntilleNortheastern University

Nayeem IslamQualcomm

Anthony JosephUC Berkeley

Yoshihiro KawaharaUniversity of Tokyo

Robin KravetsUniversity of Illinois at Urbana-Champaign

James LandayUniversity of Washington

Justin ManweilerIBM T.J. Watson Research Center

Cecilia MascoloUniversity of Cambridge

Kenton OHaraCSIRO

Joseph ParadisoMassachusetts Institute

of Technology (MIT)

Shwetak N. PatelUniversity of Washington

Bernt SchieleMPI Informatics

Albrecht SchmidtUniversity of Duisberg

Essen

James ScottMicrosoft

Research Cambridge

Rahul SukthankarGoogle Research

ADVISORY BOARDM. Satyanarayanan (chair)

Carnegie Mellon UniversityNigel Davies

Lancaster UniversityDaniel Siewiorek

Carnegie Mellon UniversityRoy Want

Google

Departments4 From the Editor in ChiefFrom Farming to Personal PrivacyUbicomps

Impact on SocietyMaria R. Ebling

8 Innovations in Ubicomp ProductsSocietal Discussion Required? Ubicomp Products beyond Weisers VisionAlbrecht Schmidt

12 Notes from the CommunityTechnology Tackles Safety, Eavesdropping, and Student LifeMary Baker and Justin Manweiler

78 SmartphonesThe Human IntranetWhere Swarms and Humans MeetJan M. Rabaey

84 ConferencesUbiComp 2014Alexis Hiniker, Seungchul Lee, and Mateusz Mikusz

1 Call for Papers: Smart Food33 Advertising Index43 IEEE CS Information53 How to Reach Us

qqMM

qqMM

qM



qqMM

qqMM

qM





4 PERVASIVE computing Published by the IEEE CS Q 1536-1268/15/$31.00 2015 IEEE

MISSION STATEMENT: IEEE Pervasive Computing is a catalyst for advancing research and practice in mobile and ubiquitous computing. It is the premier publishing forum for peer-reviewed articles, industry news, surveys, and tutorials for a broad, multidisciplinary community.

From the Editor in ChiefEditor in Chief: Maria R. Ebling Q IBM T.J. Watson Research Center Q [email protected]

W e have an exciting agenda planned for 2015, beginning with our current issue and its focus on privacy and security in pervasive sys-tems. Our AprilJune issue will look at interacting with smart spaces and the challenges we face as we try to make smart spaces more pervasive. Then, in our JulySeptember issue, we will look at the many ways reality can be enhanced with digital content, such as using digital recordings of life experi-ences to enhance our memories. Finally, our OctoberDecember issue will look at food and the food chain, all the way from the farm to your fork.

FOOD, GLORIOUS FOOD!In discussing our 2015 editorial calen-dar, the topic of how pervasive comput-ing is used in the food chain captured my attention, in part because my Girl Scout troop is currently working on the Sow What? journeya series focused on the food chain. I knew that years ago, ubicomp researchers had started looking into using sensor networks in the agricultural field, but I didnt know how that technology had been adopted by the agriculture industry.

What I learned in my exploration of this topic is that this type of technology

is really starting to gain a foothold. I found robots that can harvest certain crops, like strawberries (www.youtube.com/watch?v=RKT351pQHfI) or let-tuce (www.youtube.com/watch?v=_i62juq8Euk). Other robots navigate down rows of crops as people, sus-pended within the robot, weed the plants while laying downa much more ergonomic position than bend-ing over the plants (www.youtube.com /watch?v=AuGlYX XVpKk). There are also robots that automati-cally weed fields using image recog-nition technology to avoid chopping the desired plant (www.youtube.com/watch?v=NmeAAPLMSLw). I even found researchers working toward a vision of automated farm-ing (www.youtube.com/watch?v=aMF7EuCAVbI).

In addition to agricultural robots, I found information on precision agri-culture, using GPS as well as sensors embedded in the fields (www.youtube.com/watch?v=boCiBpWrggI). Sensors are also being used on equipment and the plants themselves to improve harvesting efficiency and quality (www.youtube.com/watch?v=ps3AcpZl0lY). And I found robotic beessmall fly-ing robots that might someday take the place of real bees in pollinating

c rop s ( h t t p : / /mode rn f a rmer.com/2013/08/5-robots-on-the-farm).1

This robobee project was particu-larly intriguing. It made me imagine robobugs designed to be the enemy of specific pests, going out to seek and destroy bugs that injure crops. Such robotic bugs could alleviate the need for harsh chemicals or genetic engi-neering. They could live in the soil and climb onto plants each season to protect the plants from specific bugs. They could get their energy from the sun. Were not there yetbut its fun to imagine what might be possible someday.

My journey also left me wondering where the field of robotics leaves off and the field of pervasive computing comes into play. Some of the technolo-gies I discovered in my (brief) foray into agriculture are clearly robots, such as the automated harvesters and robotic weeders. But other technolo-gies expand into the area of pervasive computing, with sophisticated sen-sors to measure soil composition and plant needs. The boundary is fuzzy and perhaps not critical given were all working toward similar goalsto make farming more efficient and pro-ductive with the least impact on our environment.

From Farming to Personal PrivacyUbicomps Impact on SocietyMaria R. Ebling, IBM T.J. Watson Research Center

_____________

_________________________

____

_____________________

________

________________________

___________

qqMM

qqMM

qM



qqMM

qqMM

qM





JANUARYMARCH 2015 PERVASIVE computing 5

2015 CHANGES: ITS NOW A DIGITAL WORLDSpeaking of a reduced impact on our environment, as I reported to you a year ago, IEEE Pervasive Computing has gone digital. Our primary channel for publication is now the digital edition, though the print edition is available as a premium service. Our move to digi-tal now gives us more opportunities for multimedia content. Everyone submit-ting a paper is encouraged to consider whether a video demonstration, a pod-cast, or other content would enhance their paper. Any digital content should be submitted along with your manu-script so that reviewers can view the entire submission at once.

There are also changes in the edito-rial board for 2015. IEEE Pervasive Computing would not exist without the hard work of our editorial and advi-sory board members, who help shape the magazines content and provide reviews using their extensive exper-tise. This month, were introducing three new board members and saying goodbye to two long-time contribu-tors to the magazine (see the Edito-rial Board Changes sidebar for more information). Were also bringing on a new Associate Editor in Chief. Anind Dey will be taking over for Sumi Helal (again, see the sidebar), so our four AEICs will each be responsible for the following topic areas:

rHCI and context awareness: Anind Dey;rHCI, usable security, and privacy:

Jason Hong;r hardware technologies and robotics:

Steve Hodges; andr privacy and security: Marc

Langheinrich.

We also have a number of departments, with regular contributions. The depart-ment editors are as follows:

rConferences: Elizabeth Belding;r Innovations in Ubicomp Products:

Albrecht Schmidt;

EDITORIAL BOARD CHANGES

I welcome three new board members and a new AEIC and bid farewell to two long-time contributors.

New Board MembersNayeem Islam is vice president of Qualcomm Research Silicon Valley, where he oversees research in location-based technolo-gies, mobile software and application-acceleration technologies, mobile security, and mobile cloud technologies. Islam has a PhDin computer science from University of Illinois at UrbanaCham-paign. Contact him at [email protected].

Yoshihiro Kawahara is an associate professor in the department of Information and Communication Engineering at the University of Tokyo. His research interests are in the areas of computer networks and pervasive and mobile computing. He is currently interested in developing tools and techniques to develop electrical circuits using off-the-shelf tools such as commodity inkjet printers. Kawahara received his PhD in information communication engineering from the University of Tokyo. Hes a member of IEICE, IPSJ, and IEEE. Contact him at [email protected].

Shwetak N. Patel is an associate professor in the departments of Computer Science & Engineering and Electrical Engineering at the University of Washington, where he directs his research group, the ubicomp lab. His research interests are in the areas of human-computer interaction, ubiquitous computing, sensor-enabled embedded systems, and user interface software and technology. His research focuses on developing new sensing technologies with a particular emphasis on energy monitoring and health applications for the home. He is also a MacArthur Fel-low, Microsoft Research Faculty Fellow, and Sloan Fellow. Contact him at [email protected].

New AEICAnind Dey will be stepping into the role of Associate Editor-in-Chief. Dey is an associate professor in the HCI Institute at Carnegie Mellon University, and serves as the Charles M. Geschke Director of the HCII. He is also the director of the Ubicomp Lab, which performs research at the intersection of ubiquitous com-puting, HCI, and machine learning, in the areas of mobile com-puting, health, and sustainability, among others. Dey received his PhD in computer science from Georgia Tech. Contact him at [email protected].

RetiringWith this issue, we give a special thanks to two of our founding members: Gaetano Borriello and Sumi Helal.

Borriello was a founding member of the magazine and served as editorial board member, AEIC, and most recently as a member of our steering committee. He has made innumerable contributions over the past decade. We will greatly miss collaborat-ing with him on the magazine and we wish him well!

Helal is stepping down as an AEIC of IEEE Pervasive Computing on 31 December 2014 so that he can step into the role of Editor-in-Chief of Computer on 1 January 2015. Helal was a founding member of our board and has made many contributions to the magazine over the years. We wish him all the best in his new role!

________________

_________________

_________________

___________

qqMM

qqMM

qM



qqMM

qqMM

qM





6 PERVASIVE computing www.computer.org/pervasive

FROM THE EDITOR IN CHIEF


rNotes from the Community: Mary Baker and Justin Manweiler;r Pervasive Health: Anind Dey, Jesus

Favela, and Stephen Intille;r Smartphones: Nayeem Islam; andrWearables: Bernt Schiele.

Note that Nayeem Islam has taken over for Roy Want for the Smartphones department. I thank Want for starting this department for Pervasive and I wel-come Islam as he takes on this new role.

In fact, I would like to take this opportunity to thank all of the board members, AEICs, and department edi-tors for their many contributions. I also thank the IEEE Computer Society staff, who continue to provide strong support for the magazine.

IN THIS ISSUEThis issue focuses on the privacy and security of pervasive computing. Sunny Consolvo, Jason Hong, and Marc Langheinrich have served as guest edi-tors for this issue, and I think you will enjoy the articles they selected. (We had planned on having a related Wearable Computing department on the topic of privacy, by Thad Starner and Annie I. Anton, but their research is still in the works. Look for their discussion in a future issue of Pervasive.) In addition to our theme articles, we also have three feature articles.

Zhanpeng Jin and Yu Chen examine the impact that mobile computing and cloud computing will have on medicine in their article, Telemedicine in the Cloud Era: Prospect and Challenges.

Telemedicine holds the potential to transform healthcare, but many techni-cal and legal challenges remain. These technical challenges present promising research opportunities for our com-munity. The legal challenges, though, are also important for this community. Although we might not be in a position to address these challenges directly, we can address them indirectly, and we might also influence the lawyers (or pol-iticians) as they tackle the legal aspects.

In A Participatory Service Frame-work for Indoor Location-Based Ser-vices, Hoeing Shin, Johan Chon, Youngun Kim, and Honing Cha pres-ent a system that makes it easier to create a Wi-Fi location map of a large indoor space. The system is bootstrapped by site trainers who initiate the creation of the indoor positioning service and perform a simplified training session to collect an initial map of the space. Then crowd users subscribe to the map and use various location-based services. In doing so, they also share their readings to improve the map database for future users. Although system accuracy suf-fers (3 m with traditional systems versus nearly 7 m with this system), the loca-tion map is generated with far less ini-tial investment (approximately 9 min-utes) in a large indoor mall. All in all, its a pretty impressive system.

Our final feature article is Competi-tive Live Evaluations of Activity-Recog-nition Systems, by Hristijan Gjoreski, Simon Kozino, Matja Gams, Mitja Lutrek, Juan Antonio lvarez-Garcia, Jin-Hyuk Hong, Julian Ramos, Anind

Dey, Maurizio Bocca, and Neal Patwari. The authors present a competition to evaluate activity recognition systems in a live trial. Their goal is to create a gold standard evaluation for measur-ing the quality of activity recognition by different systems. They present the competitive set-up and describe how the competition is run. Then they present the two strongest competitors in their most recent competition. They close with ideas for maturing this competi-tion and the field of activity recognition.

The Conference Department pro-vides a summary of Ubicomp 2014, which was held this past September in Seattle, Washington. One of the high-lights of the conference was its inclusion of remote participants who were able to attend sessions and interact with attendees via Beams robots by Suitable Technologies. Mateusz Mikusz, Alexis Hinker, and Seungchul Lee provide a good overview of the conference and the research presented there.

Mary Baker and Justin Manweiler bring us Notes from the Community to keep us informed about new develop-ments available in the wild. They begin with a discussion of technology to keep you and your loved ones safe, in the home and on the go. They also present home automation technologies that lis-ten to your every command. I guarantee they will make you laugh!

Jan Rabaey explores the concept of swarms of smart devices that inter-act opportunistically based on what happens to be nearby in the Smart-phones department. In this vision, the smartphone might be the central hub of a human-centered swarm, because its relatively rich in resources compared to devices such as jewelry or glasses. Other swarms might surround a car or physical location. The department pro-vides an interesting evolutionary vision for the future of smartphones and the Internet of Things.

In the New Products Department, Albrecht Schmidt reminds us to question the impact of our technologies on soci-ety and to include such conversations

SECURITY & PRIVACY MAGAZINE

This special issue focuses on privacy and security in pervasive computing. Readers with strong interest in this area will also appreciate IEEE Security & Privacy magazine (www.computer.org/security). One recent article from the Sept./Oct. 2014 issue is Can We Afford Privacy from Surveillance? by Jeffrey MacKie-Mason (http://doi.ieeecomputersociety.org/10.1109/MSP.2014.88). With much private information originating from pervasive computing devices, this article is particularly relevant to Pervasive readers. Another recent article, from the July/Aug 2014 issue, is Improving App Privacy: Nudging App Developers to Protect User Privacy, by Rebecca Balebako and Lorrie Cranor. The authors focus on ways to help app developers do a better job of improving user privacyagain, a topic particularly relevant to Pervasive readers (http://doi.ieeecomputersociety.org/10.1109/MSP.2014.70).

qqMM

qqMM

qM



qqMM

qqMM

qM







in our research. He points out how var-ious technologies from Mark Weisers seminar paper are now commonplace and how current technologies are mov-ing beyond that vision. Glasses and monitoring devices open new concerns about privacy and surveillance. Should we perhaps expect future submissions for publication to address the soci-etal cost and benefits of the proposed technology?

T he technologies that we see coming to our future bring incredible capabili-ties and possibilities. They can help keep us safe. They can help us feed more people using fewer resources. But as Albrecht reminds us, we must consider the impact these new capabilities will have on society. One of the profound impacts pervasive

computing has is on our privacy and on our security. I encourage you to dive into our theme for this issue.

REFERENCE

1. J. Hirsch, 5 Coolest Farm Robots, Modern Farmer, 20 Aug. 2013; http://modernfarmer.com/2013/08/5-robots-on-the-farm.

Maria R. Ebling is a director at the IBM T.J.

Watson Research Center. She manages a team

building systems capable of supporting a

Smarter Planet while not forgetting about the

people who use such systems. Ebling received

her PhD in computer science from Carnegie Mel-

lon University. Shes a member of the IBM Acad-

emy of Technology, a distinguished member of

the ACM, and a senior member of IEEE. Contact

her at [email protected].

IEEE Computer SocietyPublications Office

10662 Los Vaqueros CircleLos Alamitos, CA 90720

STAFF

Lead EditorBrian Kirk

[email protected]

Content EditorShani Murray

Assoc. Peer Review ManagerHilda Carman

Publications [email protected]

WebmasterJennie Zhu-Mai

ContributorsKristine Kelly, Keri Schreiner,

Dale Strok, and Joan TaylorDirector, Products & Services

Evan ButterfieldSenior Manager, Editorial Services

Robin BaldwinSenior Business Development Manager

Sandra BrownMembership Development Manager

Cecelia HuffmanSenior Advertising Coordinator

Marian [email protected]

IEEE Pervasive Computing (ISSN 1536-1268) is published quarterly by the IEEE Computer Society. IEEE Headquarters, Three Park Ave., 17th Floor, New York, NY 10016-5997; IEEE Computer Society Publications Ofce, 10662 Los Vaqueros Circle, PO Box 3014, Los Alamitos, CA 90720-1314, phone +1 714 821 8380; IEEE Computer Society Headquarters, 2001 L St., Ste. 700, Washington, DC 20036. Subscribe to IEEE Pervasive Computing by visiting www.computer.org/pervasive.

Postmaster: Send undelivered copies and address changes to IEEE Pervasive Computing, Membership Processing Dept., IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854-4141. Periodicals postage paid at New York, NY, and at additional mailing of fices. Canadian GST #125634188. Canada Post Publications Mail Agreement Number 40013885. Return undeliverable Canadian addresses to PO Box 122, Niagara Falls, ON L2E 6S8. Printed in the USA.

IEEE prohibits discrimination, harassment and bullying: For more information, visit www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Pervasi

ve Computing

seeks accessib

le, useful pap

ers on the la

test peer-

reviewed dev

elopments in

pervasive, m

obile, and

ubiquitous co

mputing. To

pics include h

ardware

technology, s

oftware infr

astructure, r

eal-world

sensing and

interaction,

human-comp

uter

interaction,

and systems

consideratio

ns, including

deployment,

scalability, se

curity, and p

rivacy.

Author guid

elines:

www.compu

ter.org/mc/pe

rvasive/autho

r.htm

Call for Ar

ticles

Further

details:

pervasive@

computer.org

www.

computer.

org/pervasive

__________

____

_____________

___________

______________________

______________

___

_______

_________________

qqMM

qqMM

qM



qqMM

qqMM

qM






Innovations in Ubicomp ProductsEditor: Albrecht Schmidt Q University of Stuttgart Q [email protected]

Societal Discussion Required?Ubicomp Products beyond Weisers VisionAlbrecht Schmidt, University of Stuttgart

M any of the devices and services envisioned and explored in the seminal ubicomp research project at Xerox PARC1 over 25 years ago have become mainstream. In particular, devices in the form factors discussed in the projectnamely boards, pads, and tabsare all around us now, and we hardly notice their presence. Schools and meeting rooms are equipped with large interactive displays (boards), tab-let computers (pads) are in widespread use in the work place and can be found in many living rooms, and smartphones (tabs) are ubiquitous.

Ubiquitous computing devices have become part of the fabric of everyday live for many of us. Just think about your last business trip. How would it have been different without your smartphone? Planning transportation ahead, printing out addresses and maps, not forgetting to take paper tickets before you leave, no social network to tell others where you are or that you were delayed, no online shopping while at the airport, and no last-minute presentation updates for the client.

We, researchers and users alike, pay little attention how our world has changed in the past 20 years through computing technologies. However, if you walk around with open eyes, you see it everywhere: paying with NFC at the caf, QR-coded airline tickets delivered on mobile devices, and pub-lic transport information on phones.

Theres also recommendation systems for restaurants combined with hand-held navigation systems, instant mes-saging, and social awareness through social media, and services are per-sonalized and contextualized to pro-vide information only when needed. Multimedia capture devices appear in different form factors, and we have access to factual knowledge and entertainment while on the go. These are just some examples that highlight the ubiquity of computing we take for granted.

As a community, we should be more aware of the impact and inspiration our research creates. Perhaps we might even dare to celebrate these changes.

PRODUCTS BEYOND THEUBICOMP VISIONMany recent products that have entered the market or have been announced are going well beyond the original ubicomp vision. Smart watches, head-worn com-puting devices, wearable cameras, and

tracking devices entering the market are meeting mixed responses. This isnt, however, surprisinglooking back, ubicomp technologies have com-monly been disputed (at least before a critical mass of users started participat-ing). But with new products, societal discussions might be even more impor-tant than before.

As computers come closer to the body, know more about the user (sometimes even more than the users know themselves), and have a greater impact on the users environment, it becomes more difficult to design these technologies. Google glass is one example thats receiving mixed responses. Besides solving techno-logical challenges, many upcoming devices require new agreements in society. As technologies enable new opportunities, we need to have a dia-log in society to address fundamental questions of whats required, desired, and allowed.

Technologies are enablers, and the following questions must be asked:

rWhat information can an individ-ual record and keep, and for how long?rHow much impact on others is

acceptable when a person records or shares information about themselves?rWill commercial entities (such

as health insurance providers) be allowed to require their clients to

As technologies enable new opportunities, we need to have a dialog in society to address fundamental

questions of whats required, desired, and allowed.

_____________

qqMM

qqMM

qM



qqMM

qqMM

qM






monitor their activity to provide a service or to receive discounts?rWill your employer be allowed to ask

youto prove that you have done a cognitive task (such as read a docu-ment) before youre allowed to make a decision on it?

Answering these questions can help us reach a wider agreement of whats acceptable.

IS SOCIETY READY?In the following, I discuss two new products as examples of technologies that have a chance to make a great and positive impact on users. At the same time, these products could create con-troversy in society and require open discussions on what we can do and what we should do.

Measuring Cognitive Activity with GlassesThe quantified-self idea looks at keeping quantitative records of our activities, which we can use for reflec-tion. Devices that measure physical activity have enabled many people to monitor themselves. An example is counting steps toward a personal goal (10,000 steps a day, for exam-ple). With new sensing techniques, the quantified self can be extended to the users cognition. In Activity Recogni-tion for the Mind: Toward a Cognitive Quantified Self, Kai Kunze and his colleagues describe how electro-ocu-lography (EOG) can be used to moni-tor eye movement, which in turn can be used to estimate cognitive activi-ties.2 In particular, EOG can be used to monitor a persons reading behav-iorestimating the number words a person has read, the type of text the person is reading (comics or scientific papers), and potentially determining how well the person understood what was read.

J!NS MEME is a pair of ordinary looking glasses that includes this tech-nology (www.jins-jp.com/jinsmeme/product). The frame includes three

electrodes for EOG, a three-axis gyro-scope and accelerometer, and Blue-tooth (see Figure 1). The device can provide information on eye movement, winking, as well as head movement.

Given that reading is linked to intel-lectual abilities, I can imagine users (or their parents) setting goals similar to setting goals for physical fitness. Goals could be reading 10,000 words in a foreign language or reading at least 90 minutes a day. Setting such goals could help individuals better track their activities and achieve their goalsjust as with tracking the number of steps walked.

However, such technologies could also be used to increase accountabil-ity. If you review a research paper, or check a contract, or claim time at work on reading on new trends, you could be asked to provide proof that you have done so. Imagine you get a research paper rejected from a con-ference, and you see that two of the three reviewers only skimmed your paper.

Its apparent that such a technol-ogy has a great potential and could revolutionize the way we track and quantify learning. Users will be able to better understand their cognitive activ-ities and will be able to set goals and

monitor their success. At the same time, such technologies might put pressure on individuals to opt-in.

Measuring Activities and Sleep with a StickerOver the last few years, many devices for tracking physical activities and for measuring physiological signals have become available.3 Devices for physical activity monitoring and fitness tracking in particular have been selling well. Nev-ertheless, wearable devices come with dif-ficultiesthey can be easily lost or acci-dentally thrown in the washing machine. For sensing physiological signals (such as heart rate, breathing rate, or ECG), device placement can be still tricky or uncomfortable (such as a chest strap).

Vancive has put activity monitor-ing into a water resistant adhesive plaster with a weight of 13 g (0.4 oz) and a battery and memory capac-ity for one weeks worth of recording (http://vancive.averydennison.com/en/home/technologies/metria/MetriaIH1.html). It includes a three-axis accel-erometer, a sensor for skin and near body temperature, and a galvanic skin response sensor. The disposable device is approximately 112 mm 66 mm u8 mm (4.40s u 2.59s u 0.31s) in size (see Figure 2).

(a) (b)

Figure 1. J!NS MEME eyewear. (a) The frame looks like an ordinary pair of glasses, and sensing is only directed at the user. (b) When wearing the glasses, basic cognitive activities can be tracked. (Photos curtesy of Kai Kunze; used with permission.).

______

___

__________________________

qqMM

qqMM

qM



qqMM

qqMM

qM






INNOVATIONS IN UBICOMP PRODUCTS

INNOVATIONS IN UBICOMP PRODUCTS

To use it, you stick it like an adhe-sive plaster on you upper arm, then switch it on and record data for a week. After just a few minutes, you forget youre wearing the deviceyou can even shower with it. After a week, you remove the plaster from your body, cut open the cover of the device (thereby destroying the cover) to expose the USB port (see Figure 2), and you can connect it to your com-puter. The device provides detailed

data about physical activity, calorie expenditure, and sleep duration and quality.

Here, too, the convenience and accuracy of the solution is a clear advantage. Its so much easier than other devices for sleep monitor-ing, because you wont forget to wear it. At the same time, a health insurance company might provide incentives for customers to stick it on, and unlike devices that are just

worn in a pocket, this device would let the insurer more closely monitor customer compliance.

Ubicomp devices are an integral part of our everyday environment. Once a significant portion of the population has bought into using a technology, were not giving it a second thought, reflecting little on what capabilities weve gained and the price paid. New devices bring more and exciting capa-bilities, which might require us to fur-ther discuss, on a broader scale, what we can do in society with these com-puters.

REFERENCES

1. M. Weiser, The Computer for the 21st Century, Scientific Am., vol. 265, no. 3, 1991, pp. 94104.

2. K. Kunze et al., Activity Recogni-tion for the Mind: Toward a Cognitive Quantified Self, Computer, Oct. 2013, pp. 105108; www.computer.org/csdl/mags/co/2013/10/mco2013100105-abs.html.

3. J. Meyer and S. Boll, Digital Health Devices for Everyone! IEEE Pervasive Computing, vol. 13, no. 2, 2014, pp. 1013; www.computer.org/csdl/mags/pc/2014/02/mpc2014020010-abs.html.

(a) (b)

(c) (d)

(e) (f)

Figure 2. The Metria IH1. (a) The adhesive plaster has (b) a built-in sensing device to be (c) attached to the (d) upper arm. The device must be (e) cut open to (f) access the USB port and read the data.

Albrecht Schmidt is a

professor of human-com-

puter interaction at the Uni-

versity of Stuttgart. Contact

him at albrecht.schmidt@

vis.uni-stuttgart.de.

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

__________

___________

_______________________

____________________________

___________________

qqMM

qqMM

qM



qqMM

qqMM

qM





Take the CS Library wherever you go!

IEEE Computer Society magazines and Transactions are available to subscribers in the portable ePub format.

Just download the articles from the IEEE Computer Society Digital Library, and you can read them on any device that supports ePub, including:

Adobe Digital Editions (PC, MAC) iBooks (iPad, iPhone, iPod touch) Nook (Nook, PC, MAC, Android, iPad, iPhone, iPod, other devices) EPUBReader (FireFox Add-on) Stanza (iPad, iPhone, iPod touch) ibis Reader (Online) Sony Reader Library (Sony Reader devices, PC, Mac) Aldiko (Android) &PYIVI6IEHIVM4EHM4LSRIM4SHXSYGL Calibre (PC, MAC, Linux)

(Can convert EPUB to MOBI format for Kindle)

www.computer.org/epub

qqMM

qqMM

qM



qqMM

qqMM

qM






T hank you for reading this quarters edition of Notes from the Com-munity. Recent contributions to our Reddit community focus on personal safety, data privacy, home listening devices, smartphone recycling, student life, and two futuristic projects.

TECHNOLOGY TO THE RESCUEDevice miniaturization brings us new experiences but also helps address age-old worries. One such worry is the personal safety of those we care about. Readers submitted links about two different approaches to keeping our friends and family safe.

Safety JewelryIt seems like a great solution to pin an alarm button to family members who live alone or walk through an empty parking garage after work. But what if those youre trying to protect care more about fashion? How will you get them to wear an alarm?

Long-established companies, such as MedicAlert (www.medicalert.org), have already solved this problem for static medical warnings by turning the

bracelets into jewelry. Unfortunately, for active alarms, such as the Life Alert necklace (www.lifealerthelp.com), the jewelry still looks like wearable hos-pital equipment.

Sense6 Design solves this problem by disguising the alarm button in an acces-sory called the Artemis (www.artem-isfashion.com). Tap three times on the Artemis pendant (see Figure 1), and it sends an alarm to a private security com-pany and the potential victims loved ones. The embedded sensing and com-munication device fits into your choice of a silver, gold, or diamond-studded palla-dium necklace pendant or a no-nonsense clip, both of which are shower proof. If the rather bulky pendant isnt your style, future options appear to include a large flat necklace or clunky bracelet. Maybe eventually something more delicate will be possible. For more information, see www.psfk.com/2014/11/artemis-stylish-smart-jewelry.html.1

iPhone HomeOr the PoliceA second approach is to attach an alarm to something most people already have handy: their smartphones. Lifeshel has

created a case called the Whistl that fits over a smartphone and has buttons on the outside of the case. If you press those buttons, the case emits a very loud sound (120 decibels) and starts up a strobe light. The case also talks to the phone via Bluetooth and causes it to send a distress notification to the police and your choice of pre-programmed contacts. It even causes the phone to start up video and audio recordings so there may be some helpful clarity about the incident afterward.

Of course, none of this is useful if the phone is in the bottom of a purse

Technology Tackles Safety, Eavesdropping, and Student LifeMary Baker, HP LabsJustin Manweiler, IBM T.J. Watson Research Center

JOIN OUR SUBREDDIT

This column offers a summary of interesting news and research in pervasive and mobile computing, with content drawn from submissions to a shared community on the social news site Reddit, at www.reddit.com/r/pervasivecomputing. We encour-age you to join our subreddit and spread the news of this site to others, so that together we can build a sustainable online community for all aspects of pervasive and ubiquitous computing. Mary Baker and Justin Manweiler

Section Title HereNotes from the Communityat

Figure 1. The Artemis safety pendant. (Source: Sense6 Design; used with permission.)

Editors: Mary Baker Q HP Labs Q [email protected] Justin Manweiler Q IBM T. J. Watson Research Center Q [email protected]

__________________________

_____________

_________

________

__________________________

qqMM

qqMM

qM



qqMM

qqMM

qM






or bag or in an awkward pocket. So now theres an excuse to keep that phone in hand! To read more about the device or the partnerships the company has established with victim-service agencies, see www.theatlantic.com/technology/archive/2014/11/applying-technology-to-the-problem-of-sexual-assault/382594.2

SAFETY FOR YOUR DATA TOONow that weve strapped on a variety of personal safety devices, we can pause to consider the safety of our data as well. With almost ubiquitous sensing, there are countless opportunities for sensi-tive data to escape our control. Readers brought our attention to two articles. The first describes privacy issues with self-tracking gadgets and apps. The sec-ond suggests that if were going to put personal data out there, we should be compensated for it.

Your Quantified, But Not Very Private, SelfThe Quantified Self or Lifelogging movement has generated many kinds of personal tracking. There are now gadgets and applications for us to track our physical activities, fitness, diet, sleep habits, alcohol and drug con-sumption, social interactions, mood, and more. Although having this data available might point out opportunities to improve our health and lives, it can also provide more opportunity for oth-ers to access our personal information.

Symantec recently performed a study to determine just how bad the situation really is. They built some relatively sim-ple portable Bluetooth network scan-ners, walked around with them, and attended sporting events with them. They found that all of the wearable fit-ness trackers transmitted unique hard-ware addresses that allowed Symantecs scanners to track the location of the devicesand thus the location of those wearing the products. They also found that 20 percent of tracking applica-tions failed to encrypt user credentials before transmitting them to the cloud,

leaving personal data vulnerable, even when stored in password-protected individual accounts. More than half of the self-tracking apps didnt even have privacy policies. To read more about the Symantec study, its findings, and its sug-gestions to prevent others from track-ing your trackers, see www.symantec.com/content/en/us/enterprise/media/security_response/whitepapers/how-safe-is-your-quantified-self.pdf.3

Payment for Your Lack of PrivacyWe frequently see stories about gov-ernments and large companies gather-ing comprehensive personal informa-tion about us, sometimes without our knowledge. Companies monetize this information by selling it to others or using it themselves for targeted advertis-ing. Quartz (qz.com) recently reported on companies that now consider your personal data so valuable theyre will-ing to pay you for more of it. While the long-established Neilsen ratings and supermarket loyalty cards already offer money or product discounts for data, this move to enrich online data gath-ering is still somewhat experimental, and pricing is likely to change. To hear more about this opportunity to give up your privacy for money, see http://qz.com/257950/a-new-way-to-track-your-data-with-your-permission-and-for-a-fee.4

MORE FUN WITH PHONESContributors to our subreddit brought our attention to three articles featuring smartphones. Two of these cover yet more data we can use our phones to senseboth cosmic and personal. The third article is all about what to do with our phones as they get old.

Your Phone and Cosmic RaysWe use our smartphone cameras to take pictures of Halloween costumes, our feet on vacation, crimes in prog-ress, what we cooked last night for din-ner, the pool our friends fell into, and of course cute kittens and hedgehogs. Some physicists at UC Irvine and UC

Davis hope to expand our photographic horizons by asking us to take pictures of ultra-high energy cosmic rays as well.

In a project reminiscent of SETI@home, the Crayfis (Cosmic RAYs Found In Smartphones) project aims to use smartphones around the world as a large-scale ground detector array. Air showers generated by cosmic rays cre-ate high-energy particles detectable by the CMOS sensors in our phone cam-eras. These sensors are small and inef-ficient, but there are potentially a lot of themover 1.5 billionto make up for that. Our phones also come equipped with GPS and network connections to label and upload the sensed data. If youre interested in being an astrophysi-cist, join their collaborative project by checking out http://crayfis.ps.uci.edu/about.html or read the teams paper at http://arxiv.org/pdf/1410.2895v1.pdf.5

Everything about College StudentsWhile physicists are sensing cosmic rays with smartphones, researchers at Dart-mouth College, the University of Texas at Austin, and Northeastern University are sensing studentseverything about them. The StudentLife Study used the smartphones of 48 Dartmouth under-graduate and graduate students over a college term (10 weeks) to gather information about the students mental health, physical activity, sleep habits, social activity, eating habits, academic performance, and reactions to aca-demic workload over the course of the term (see Figure 2). The collected data is rich, detailed, and very personal. For example, it includes location informa-tion, conversational information, and data about grades.

Going forward, the team hopes to add feedback and intervention to the phone app to help students live healthier lives, get better grades, and stay safe. One interesting finding is that there was no correlation between the students class attendance and gradesjust knowing that should reduce stress for some students! For more information about the study, see http://studentlife.

__________

__________________________

__________________________

_____________________

__________________________

_________________________

_______

______

__________________________

__________________________

____

qqMM

qqMM

qM



qqMM

qqMM

qM






NOTES FROM THE COMMUNITY


cs.dartmouth.edu. The website includes a link to the anonymized dataset (at least those parts the researchers have been able to anonymize).

Landfills of Past Sensing and FashionThe increasing performance and diver-sity of smartphone sensors should be a good thing, but it has a dark side too. Old phones dont have the sensing capabilities of new phones, and so we give up on them, stick them in draw-ers, or throw them away. Fashion also plays a role, because many people want to be seen with only the latest gad-gets. Chemical and Engineering Newspresents a variety of alarming statis-tics about smartphone waste, as well

as a description of new projects and chemical processes to help combat the problem.6

The world will purchase well over a billion new phones in 2014, but if his-tory is any predictor, owners will give up on most of them after a few years. Alas, only 3 percent of these phones are actually recycled, which is an economic and ecological calamity, especially given the large amounts of precious and poisonous metals in the phones. Theres a vastly higher concentration of gold in a phone than in ore from a mine! How is the world trying to address this problem?

Some organizations, such as Proj-ect Ara from Google (www.projec-tara.com) are creating more modular phones, so we can replace individual components without throwing away the whole phone. Other organizations are working to make recycling easier, more efficient, and cheaper. One approach, such as that taken by the England-based project Clever (Closed Loop Emotion-ally Valuable E-waste Recovery; http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/K026380/1), is to create phones whose materials are more easily recycled. Other organiza-tions strive toward better chemical processes for recycling existing materi-als. For example, a new process called

eVOLV might be able to recover up to 98 percent of precious metals in e-waste, and the process can be gradually scaled up in size. Gradual scaling is impor-tant, because it allows countries such as Japan, China, and the US (which dont have huge smelting operations) to enter the e-waste recycling fray. Read more on this important topic at http://cen.acs.org/articles/92/i35/Dialing-Back-Cell-Phone-Waste.html.6

WHEN THE WALLS HAVE EARSAudio sensing continues to find its way into more use cases and gadgets. Three submitted links involve audio sensing in the home, each with a different pur-pose: home automation, security, and ease-of-purchasing.

Ill Tell You What to DoHomey is a home automation hub (see Figure 3) that developers hope will eventually control everything in your home: lights, HVAC, entertain-ment system, kitchen appliances, and so forth. It is speech controlled, so youll be able to tell it that when you wake up you want your curtains to open and a ZZ Top album to play at high volume. Most of the functional-ity seems accessible via a phone app, so you can issue remote commands to ask the oven and hot tub to pre-heat themselves before you get home. Check out this successfully funded project at athom.nl.

Loud Sounds in the NightA Kickstarter project called Point hopes to offer a customizable home security device that is somewhat less invasive than others because it uses audio rather than video. (Apparently, we should feel comfortable with devices listening to us instead of watching us.) The device is a house sitter that can send you an alert if it hears a loud sound in a supposedly empty house. It can ask your Airbnb guests (www.airbnb.com) to turn down the TV late at night. It includes other sensors too, so it will let you know if guests are

Figure 3. Homey listens to your home automation commands. (Source: Homey; used with permission.)

Figure 2. A chart from the StudentLife Study shows that things generally are grim from midterms to the end of the term. (Source: Rui Wang; used with permission.)

0.16Gym

Mood

Stress level Mid-term

Deadlines

0.14

0.12

0.107 14 21 28 35

Spring term (days)

42 49 56 630

0.5

1.0

1.5

2.0

2.5

________________

_______________________

__________________________

____

____

______

_________

____________

______

qqMM

qqMM

qM



qqMM

qqMM

qM







smoking inside the house or if humid-ity levels are so high the walls will grow mold. To read more about this soft security device, check out its Facebook page (www.facebook.com/athomnl/photos_stream) or watch a video at www.kickstarter.com/proj-ects/830527119/point-a-softer-take-on-home-security?ref=popular.

Dont Name your Child AlexaPerhaps the most curious of the new home-listening devices is the Amazon Echo (www.amazon.com/oc/echo). This black cylinder will sit in your home and listen. If it hears the name Alexa, it will pay attention to what youre say-ing and do something. You can ask it questions, ask it to perform home auto-mation tasks, and of course, ask it to buy things from Amazon. At the time of writing, the Echo can be purchased by invitation only.

While readers didnt submit a link to the Echo directly, they did submit a link to an amusingly edited version of the Amazon Echo ad. For a small dose of pervasive computing humor, watch https://www.youtube.com/watch?v=GijLoiVkmYI. To their credit, Amazon has so far not squelched this parody. It is also the case that we might already have many other listening devices around us in our homes, includ-ing smartphones with speech-enabled personal assistants running on them.

THINGS WE WANT TOPLAYWITHLarge and small, readers point us to sys-tems and gadgets we cant wait to play with.

Introduction to CircuitsCircuit Scribe is a pen that writes with conductive ink so you can scribble cir-cuits on paper. The kit also comes with various components such as LEDs, power adapters, and buzzers that you can plop down on your circuits to make all sorts of cool things happen. Imag-ine creating your own wearables with paper, pen, and tapeor origami that

does more than look pretty! To see a demo of the pen, check out http://youtu.be/e0NM1jJbjrM.

Gamers ParadiseAs a follow-on from its IllumiRoom research project, Microsoft has com-bined video projectors and the Kinect to create RoomAlive, a more advanced prototype that extends the Xbox gam-ing environment to an entire room. The system tracks you throughout the room so you can interact with game objects in the middle of the room or via any of the rooms surfaceswalls, floors, and so on. As long as you dont trip over the coffee table, this seems like magic for gamers. Take a look at http://research.microsoft.com/en-us/projects/roomalive for more information.

REFERENCES

1. C. Stephens, Stay Safe Yet Fashionable with Artemis Smart Jewelry, PSFK, 14 Nov. 2014; www.psfk.com/2014/11/artemis-stylish-smart-jewelry.html.

2. J. Tierney, The iPhone Case That Can Call the Police, The Atlantic,11 Nov. 2014; www.theatlantic.com/technology/archive/2014/11/applying-technology-to-the-problem-of-sexual-assault/382594.

3. M.B. Barcena, C. Wueest, and H. Lau, How Safe is your Quantified Self? Symantic, 11 Aug. 2014; www.syman-tec.com/content/en/us/enterprise/media/security_response/whitepapers/how-safe-is-your-quantified-self.pdf.

4. E. Lopatto, A New Way to Track your Data: With your Permission and for a Fee, Quartz.com, 2 Sept. 2014; http://qz.com/257950/a-new-way-to-track-your-data-with-your-permission-and-for-a-fee.

5. D. Whiteson et al., Observing Ultra-High Energy Cosmic Rays with Smart-phones, ArXiv.org, 10 Oct. 2014; http://arxiv.org/pdf/1410.2895v1.pdf.

6. A. Scott, Dialing Back On Cell Phone Waste, Chemical & Engineering News, vol. 92, no. 35, 2014, pp. 3033; http://cen.acs.org/articles/92/i35/Dial-ing-Back-Cell-Phone-Waste.html.


Mary Baker is a senior research scientist at HP

Labs. Contact her at [email protected].

Justin Manweiler is a researcher at the IBM T.J.

Watson Research Center. Contact him at jman-

[email protected].

Figure 4. The RoomAlive gaming environment. (Source: Microsoft Research; used with permission.)

______________________

________________

_______________________________________________

____________

______________________

_____________________

__________

___________________

____________________

_________

______________________

___

___________

__________________

_________________

_____________________

__________________________

___________________

_____________________

_____

________

qqMM

qqMM

qM



qqMM

qqMM

qM






G U E S T E D I T O R S I N T R O D U C T I O N

Privacy and Security

From smartphones to wearable fitness trackers to in-car navigation systems, pervasive computing is becoming part of everyday life. Such systems have a great amount of potential benefit. They will help us and society in terms of sustainability, healthcare, transportation, and more. These systems need to work reli-

ably, helping authorized users understand whats happening with their data and provide them with adequate user con-trols. It must be easy for autho-rized users to access data and use services. However, it also needs to be extremely difficult for unauthorized users with bad intentions to do the same.

This special issue offers four articles that help us take a step forward. The first arti-

cle, Engineering Gesture-Based Authentication Systems, looks at the design space for using gestures (both 2D and 3D) as a way to log in to systems. The authors, Gradeigh D. Clark and

Janne Lindqvist, map out several dimensions for consideration and also discuss tradeoffs in terms of reliability, usability, and security.

The second article, Social Access vs. Privacy in Wearable Computing: A Case Study of Au-tism, looks at how Google Glass might be used as a social prosthesis to help individuals with high-functioning autism. The authors, Reuben Kirkham and Chris Greenhalgh, use this pros-thesis as a lens for discussing tensions that arise when there are conflicting goals and priorities. The right of these individuals to have support for their disability might be seen as conflicting with other peoples right to privacy.

The third article, Context-Adaptive Privacy: Leveraging Context Awareness to Support Pri-vacy Decision Making, looks at how sensing capabilities can be used to promote the applica-tion of contextually appropriate privacy pref-erences. In other words, instead of sensing be-ing diametrically opposed to privacy, what if it could be used to improve privacy? The authors, Florian Schaub, Bastian Knings, and Michael Weber, present several prototypes. Examples include blocking the display of photos or a

Sunny ConsolvoGoogle

Jason HongCarnegie Mellon University

Marc LangheinrichUniversit della Svizzera Italiana (USI)

qqMM

qqMM

qM



qqMM

qqMM

qM






personal calendar when certain people are in the room.

The final article, Security and Pri-vacy Implications of Pervasive Mem-ory Augmentation, is by Nigel Davies, Adrian Friday, Sarah Clinch, Corina Sas, Marc Langheinrich, Geoff Ward, and Albrecht Schmidt. They posit that ubiquitous displays and wearable de-vices can be used to help improve mem-ory recall. They consider how might such a system be built, and what are the likely privacy and security challenges. For example, if audio and video is cap-tured in a meeting room, how would the user know if it has been tampered with? What if the system tries to prompt selected memories to promote a more positive experience? What about by-standers that are incidentally recorded?

Privacy and security will con-tinue to be thorny issues for pervasive computing. This is-sues articles show that ad-dressing these concerns requires not only efficient algorithms and secure protocols but also usable interfaces and socially compatible designs. Most

of all, it will require researchers with a strong interdisciplinary interest to look for the non-obvious solutions. We hope that you find the articles use-ful, and we hope that they help foster the ongoing discussion in our research

community on how best to provide privacy and security in pervasive and mobile computing.

the AUTHORSSunny Consolvo is the lead and manager of Googles Privacy Research and De-sign team in Mountain View. Her research interests include usable privacy and security, persuasive technologies, ubiquitous computing systems, mobile com-puting, and Web technologies. Consolvo received her PhD in information sci-ence from the University of Washington. Contact her at [email protected].

Jason Hong is an associate professor of computer science at Carnegie MellonUniversity. His research interests include mobile computing and usable privacy and security. Hong received his PhD from University of California at Berkeley. Contact him at [email protected].

Marc Langheinrich is an associate professor at the Universit della Svizzera Italiana (USI), where he works on privacy and usability in pervasive computing systems. Langheinrich has a PhD in computer science from ETH Zurich. He is on the editorial boards of IEEE Pervasive Computing, Elseviers Personal and Mobile Communications, and Dagstuhls Open Access Series in Informatics. He is a mem-ber of IEEE and ACM. Contact him at [email protected].


IEEE Pervasive Computing explores the many facets of pervasive and ubiquitous computing with research articles, case studies, product reviews, conference reports,

departments covering wearable and mobile technologies, and much more.

Keep abreast of rapid technology change by subscribing today!

www.computer.org/pervasive

____________

__________

____________

qqMM

qqMM

qM



qqMM

qqMM

qM






Engineering Gesture-Based Authentication Systems

A uthentication has become an essential component in daily life. Increasingly, its the gateway to critical facets of the human expe-rience including work, communi-cation, and entertainment. To be effective, any authentication technique must be reliable, dif-ficult to compromise, and, above all, easy to use when people are focused on the activity behind the gateway and not the authentication itself.

Gesture-based methods have advantages over currently popular authentication methods

such as text entry, PINs, bio-metricsbecause gestures can be performed faster and are highly customizable,1 easier to remember,2 and potentially more secure.1 Gestures also re-quire lower concentration and accuracy compared to other methods, and thus have po-

tentially lower chances of error when used by stressed or distracted people. For example, an incorrectly entered text-based password yields an automatic rejection, whereas some inaccu-racy or deviation of the gesture password can still lead to a positive identification.

The difference between a recognizer and an authentication system is important here; the rec-ognizer is one aspect of an authentication sys-tem, which can consist of several components

(including the user interface). To the best of our knowledge, no comprehensive surveys of different recognition methods for gestures are available. Even more importantly, no criti-cal discussion exists on how we might com-pare these proposals or use them to design ro-bust and highly usable authentication systems. Prior studies have looked at basic issues, such as asking participants to generate secure and memorable gesture passwords with no other instructions.1 However, gesture authentica-tion is now at a stage where such work must be supplemented by a deeper understanding of us-ability and effectiveness. This isnt possible until a large, open dataset is created that will allow direct comparison of different gesture recogniz-ing methods.

Until such a dataset is available, we can quali-tatively analyze different approaches to gesture security. To this end, we offer here four con-tributions: a survey of common gesture recog-nizers; design considerations for gesture-based authentication systems; suggestions for compar-ing recognizers; and an evaluation of gesture-based authentication compared to text-based passwords.

Gesture TypesNo universally accepted terminology exists for gesture types. Often, different names are used for the same type of gesture. From a top-level

Gestures are a topic of increasing interest in authentication, but successfully implementing them as a security layer requires reliable gesture recognition. This survey presents and analyzes different methods of gesture recognition and offers design considerations for gesture-based authentication systems.

Gradeigh D. Clark and Janne LindqvistRutgers University

P R I V A C Y & S E C U R I T Y

qqMM

qqMM

qM



qqMM

qqMM

qM






view, gestures are divisible into two categories: touchscreen gestures and motion gestures (see Figure 1). These two gesture classes can be freeformthat is, created without constraints and cuesor predefined by a recognizers creator.

TouchscreenTouchscreen gestures are those cap-tured through a touchscreen. Single-stroke gestures use only one finger to perform a continuous input on the screen. Multistroke gestures are dis-continuous and allow for multiple stroke attempts at the screen before completion. Multitouch gestures use more than one finger to perform a con-tinuous gesture.

Motion GesturesMotion gestures are performed in 3D and can be divided into sensor-based and camera-based gestures. Sensor-based gestures use sensors other than a camera or touchscreen (such as a smartphones accelerometer). This di-vision is motivated by the input tech-niques challenges for recognizers, as well as the abundance of prior work. Camera-based methods represent the majority of gesture recognition pub-lications, and thus warrant their own category.

Threat Models: Attacks against GesturesAuthentication systems must be resil-ient against attacks. To successfully attack a gesture, an attacker must be capable of replicating the features ac-curately enough to fool the recognition algorithms into accepting the gesture as authentic.

A gesture password can be compro-mised in at least four ways: shoulder surfing, brute force, dictionary attacks, and storage leakage.

Shoulder SurngIn shoulder surfing, an attacker tries to memorize a password or se-cret via line-of-sight. Basic shoulder

surfing methods include the standardapproach, in which the attacker ob-serves the user from a vantage point that allows easy viewing of the users gesture. Another option is recording, in which the attacker records and later observes the users gesture. Finally, multiple attackers can work together from multiple vantage points to focus on specific parts of a password at dif-ferent times and reconstruct it later.

Brute ForceBrute force attacking is done by repeat-edly trying passwords to find the right one. A brute force approach can mea-sure the susceptibility of a recognizer to algorithmic attack. As we discuss later, an equivalent attack on text-based passwords would be trying to guess the password without having access to password hashes.

Dictionary AttackA dictionary attack is similar to a brute force attack except that the password attempts come from a set of more likely possibilities (such as datasets from user studies). To date, dictionary attacks have not been successfully demon-strated against gesture recognizers.

Storage LeakageStorage leakage can be a problem de-pending on how the device stores the gesture password. To successfully steal a gesture based on stored data, a thief would have to know both the recog-nizers structure and how to translate the stored values into gesture actions. Text-based passwords mitigate storage leakage by storing only the passwords cryptographic hash. Storage leakage is a serious issue for gesture passwords given that no two inputs will be exactly

(a)

(c) (d)

(e)

(f)

(b)

Figure 1. Different types of gestures. The example touchscreen gestures are (a) a single stroke gesture, (b) a multitouch gesture, (c) a multistroke gesture, and (d) a combination of multistroke and multitouch gestures (the hatch pattern can be drawn using two fingers and doing two strokes: one stroke with two fingers to get the horizontal portion, and a second stroke to form the vertical portion). Example motion gestures are (e) a sensor-based motion gesture, created by rotating a smartphone; and (f) a person being recorded for a camera-based motion gesture.

qqMM

qqMM

qM



qqMM

qqMM

qM






PRIVACY & SECURITY

alike, which makes comparing their hashes difficult.

Comparing Authentication SystemsBefore discussing recognition in full, its useful to outline aspects of gestures as an authentication scheme that make them viable as a replacement for (or supplement to) text-based passwords. The best way to do that is to evaluate gestures on three criteria:

r usability addresses a schemes viabil-ity from the users perspective,r deployability examines the infra-

structure a method requires to be us-able, andr security refers to the ability of a sys-

tem to resist attacks.

These three metrics follow from an exhaustive survey of authentica-tion methods,3 which unfortunately doesnt discuss gestures. None of the known alternatives to text-based pass-words offer the same range of features of such passwords,3 and gestures are no exception. Decades of infrastructure and development have gone into mak-ing text-based passwords ubiquitous. Because of this, despite their many dis-advantages, its difficult for any scheme to match all the benefits of text-based passwords. However, advancing the development of alternative schemes could allow for a real challenger to text-based passwords.

UsabilityGestures are potentially more memo-rable than text-based passwords be-cause human recall is better for picto-rial concepts than for strings of text,2

although no definitive measure exists for a passwords memorability. We dont yet know whether complicated gesture passwords are more memorable than complicated text-based ones, but evidence from human psychology2 and user studies1 supports the assertion.

Gesture-based passwords are as easy to adopt as text-based passwords. Most

people have used gestures to commu-nicate silently or have drawn pictures to explain something to another per-son. As such, little additional training is required to teach people how to use them. The introduction of the Android 3 u 3 grid-based graphical password can be thought of as a primer to us-ing gesture-based passwords on touch-screen devices.

Latency and error rates are natural usability related concerns; the former can be reduced with proper recognizer design, but error rates are harder to minimize. However, complicated ges-ture passwords could have better er-ror rates than complicated text-based passwords.

Password recovery and the ability to reset passwords are necessary for us-ability. Gestures are equal to text-based passwords in this wayas we describe later, the same systems that recover text-based passwords can be applied to gesture passwords. The password recovery flow for gestures will be dif-ferent than with text-based passwords because a gestures features can be used to recover the password. A simple ex-ample would be to ask users to trace a set of characters and see how the re-sult correlates to their past behaviors. Proper users can either be shown a pic-ture of their gesture or given steps or hints as to how it can be replicated. The effort required to authenticate depends on the population under consideration. Given text-based passwords ubiquity, user effort can be higher when start-ing out with gestures because more users are more comfortable using keyboardsthough this might be less an issue for Android graphical pass-word users. However, differently abled users (such as paraplegics) who cant naturally interact with a smartphone or touchscreen, or properly motion to a camera are at a disadvantage.

DeployabilityWith current technology, gesture passwords cant be used by all people who can use text-based passwords.

Thedifferently abled can face issues using gestures, especially if they suffer from a loss of sight or motor function.

Users dont require additional tools to authenticate. This doesnt mean that gestures are compatible with all current systemsrather that they integrate well into existing infrastructures. Although touchscreen tablets and phones are al-ready prominent, laptops and monitors with touch capability are starting to be-come more commonplace as well. Al-ternatively, gestures could be generated with a mouse or using laptops touch-interactive mousepads. A negative is that, for motion gestures, desktop us-ers might require a separate webcam component.

Gestures arent directly remote-login compatible; this is attributable to both their infancy as an authenti-cation scheme and the proliferation of text-based passwords. However, some devices use biometrics as a master pass-word, allowing integration with remote login servers. A gesture could be used as a master password in much the same way. This would improve remote-login compatibility, but its not a perfect solu-tion due to the inability to reliably com-pare the hash of two different gesture inputs. Gestures can be integrated into Web browsers. HTML5 or mobile web-sites can have a gesture capture area for touchscreen, while allowing a browser access to the camera would enable camera-based gestures. A mobile plat-form could observe a motion gesture on behalf of the browser. Using gestures might require additional hardware on desktops (such as a webcam or external sensors, but these are becoming default equipment for desktops, too).

SecurityGestures can be more resistant to shoulder surfing attacks than text-based passwords, depending on the amount of features used in recogni-tion. Replication of the exact way a gesture is performed can be more dif-ficult than assembling all the characters of a text-based password, depending on

qqMM

qqMM

qM



qqMM

qqMM

qM






the passwords length. Similar to bio-metric systems, personal knowledge doesnt yield clues that could reveal a users gesture password. Comparing the security of text-based passwords to gesture passwords is an open problem. Its possible to quantify a gesture pass-words security based on a surpris-ingness factor.1 This score allows for password creation policies similar to text-based passwords (such as rejecting simple passwords and instructing the user to try again).

If a system restricts the number of failed attempts, then it becomes diffi-cult to compromise the password. As with text-based passwords, gesture passwords can be stolen if attempts are unlimited and the attacker is properly trained. The lower accuracy required for gestureswhich is an advantage for usabilityis a disadvantage here. This demonstrates the need for proper gesture-password creation policies like the ones used for text-based passwords.

Storage leakage is a problem because we currently lack a way to store gestures such that two similar inputs would have the same cryptographic hash. A hashing approach was explored in the Draw-A-Secret graphical password sys-tem,4 where an input is a drawing over a 25-grid space. Inputs are compared by checking the order of grid boundar-ies that a drawing crosses (for example, up in grid five, right in grid four) and concatenating those into a string. The hash function is then applied to this string. An approach like this does not work for free-form gestures, consider-ing that the act of discretization causes a severe loss of information. This re-mains an open and important problem in gesture authentication and one that is well worth examining further.

Designing Recognizers for AuthenticationSome design considerations can be gleaned from prior work on recog-nizers,5 although such efforts focus on recognition, not authentication. Additionally, some aspects of reliable

recognition5such as location and scale invariancedont translate to re-liable authentication.

Our work extends previous r

Documents

Pervasive Magazine 2015.01.01