Broader Impacts

Slide 1

Broader ImpactsResearch you can useJudith OlsonUniversity of California Irvine

Thank you, Elizabeth. And thank you for taking the time and effort to organize the nomination. I am deeply honored by this award. And, this is not just an award to me. It is to all of us. This is an ACM award recognizing the core value of HCI in Computing. If it doesnt work for people, it doesnt work!

I have chosen today to talk about an issue that has been growing in my attention over the last two years. Broader Impacts.developing research that is useful and usable.

1Today Im going to cover.Three recent events that inspired this call to armsYour role in this discussionWhat it means to have impactKinds of impact

Recap of what it means to have impactScope, Cost, TimelineYour pledge about making an impact

Collaboration Success WizardTheory based on The literature on teamsOwn own observations and interviews of over 50Science CollaborationsCorporate virtual teamsTo verify theoryNeed dataOnline survey with advice to motivate participationThey get the help and we get the data

This Wizard has a history. We have been studying long-distance collaboration for nearly 20 years. There is a lot of literature on social dynamics of teams. We have a lot of experience in Science Collabarations, called Collaboratories. We have built tools, observed the interactions, done interviews, etc. We have done the same in corporate teams, what are called there virtual teams.3Collaboration Success WizardWeb accessible assessment tool AssessesStrengthsChallengesHow to overcome the challenges

How many of you work with people who are not in the same building with you? Good. Lots of you. This is for you.. The Collaboration Success Wizard is a recently built web-accessible tool that, through a series of questions, assesses where a collaboration. Theres a page from the question part, and on the right is a report that comes out at the end about our assessment of where you are strong, where you are vulnerable and what to do about it.

4We are having an impactNSFHad us give a talk to Federal funders in generalI have needed this for the last 10 years! Thank you.Teams who were assessed welcomed adviceIt drew out patterns in the way our members work that we were not conscious of, confirmed some of our impressions, and allowed us to hear frankly from our members.useful as an independent evaluation tool not tied to a funding agency or other review panel.

In addition, being sponsored by the National Science Foundation, we reported our results at a meeting in Washington, whereupon our program director, realizing the potential impact of this work and the Collaboration Success Wizard in general, asked us to speak at several meetings of program officers from other federal agencies. We passed out a handout of the factors, on the left, and the many ways we might think of success, on the right. One program officer blurted out, Ive needed this for the last 10 years! Thank you.

5Clinical and Translational Science Awards (CTSA)Object of study

to speed the translation of laboratory discoveries into treatment for patients.from bench to bedside.

National Institute of Health: 60 CTSA awards in 30 states plus DC

FROM A DIFFERENT FIELD: the idea of Clinical and Translational Science! Upon a bit more investigation, we found that their goal is to speed the translation of laboratory discoveries into treatment for patients, from bench to bedside. This NIH effort has, to date, funded 60 such grants (and they are not small) to organizations in 30 states plus the DC. They are making progress across a broad range of diseases and conditions, including cancer, diabetes, neurological disorders and heart disease. If something is known from the research, you sure would like it applied to your case.6Clinical and Translational Science Awards

National Institutes of HealthSince 2006$733 MNSF Dear Colleague

While most researchers know what is meant by Intellectual Merit, experience shows that many researchers have a lessthan clear understanding of the meaning of Broader Impacts.Third source: Policy8ImpactMany of us came to this field to change the digital worldTechnology had gone awry

Many early people attracted to HCI were Children of the 60s

The CHI community was formed in the early 1980s by people who had done some work in the area, were beginning to be established in the field. Presenters at the first CHI included Tom Moran, Marilyn Tremaine, Don Norman, Terry Roberts, Mary Beth Rosson, and many more still active today. The main motivation for joining the CHI community was to make a better world. Technology was too hard; there had to be a better design. Basically, we were carrying the fervor of the Children of the 60s.

9ThenOur careers were caught up in the reward structuresIndustryCreate new productsDisincentive to makefindings available to othersAcademiaPublish new findingsStay on topic, build a reputation

We found our careers caught up in a reward scheme that included creating new products where you dont share your knowledge with your competitor. And in academics, the reward was from publishing new findings in journals or conferences, staying on topic, building a reputation. Some of us wondered Where have all the impacts gone? (long time passing.)

10Worry..Where have all the impacts goneLong time passing

How we will proceed.I will describe what I think it means to have impactI will list a number of ways we do and can have an impactYou pledge

The card on your seatWhat other ways can you have impactHow are you going to have an impactCollected by SVs at the door as you leave

SO.nowHow do WE have an impact. Heres how I want to proceed. In what follows, Im going to outline a number of ways we can and DO have an impact. What I want you to do is to think carefully about how YOU can have a greater impact than you do now. You have two 3.5 cards on your seats. We are going to use the WHITE ones for the standard procedure about asking questions of me at the end. As you think of a question, write it on the card and pass it to the aisle for an SV to pick up. The BLUE card is for you to report on how YOU are going to have an impact in the future. Hold on to these, make notes, comments, etc. and then turn them in to the SV as you leave the session. OK? White cards are for questions, blue cards for pledges of impact.

12What it means to have impactWhat countsTheory gets usedDownloads/viewsProfitsDegrees/EducationTechnologiesLives changed..Who is impacted?StudentsDevelopersConsultants

Specific populationsThe general publicWhat it means to have impactScopes differYou affect some people directlyInterventions, teachingYou enable others to be better at making better productsToolkits

You set policyAffect a large number of people

What it means to have impactTime scales differNowe.g. Action research1-3 yearse.g., Publications20-30 yearse.g., Theory Assessment Tools40-50 yearse.g., Cyberinfrastructure development?e.g., Policy (like SOPA/PIPA)

What it means to have impactAccess?FreeToolkitsWizard.FeesCommercial Assessment ToolsProductsEducational degree.Decisions you have to make in deciding what kind of impact you will make.16Kinds of ImpactsTheoriesAssessment toolsTechnological innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materialsWhat else?Which kinds of impacts will YOU make?Here are the sevencategories of kinds of impacts I would like to review. If you think of more, please put them on the blue cards as well. I dont pretend to be exhaustive, and my examples within these categories couldnt possibly give a shout-out to ALL the people who have made impact. These are examples, with the intent of helping you find a way you can have impact.

17TheoriesThere is nothing so practical as a good theory Kurt Lewin

He who loves practice without theory is like the sailor who boards ship without a rudder and compass andnever knows where he may cast Leonardo Da Vinci

Theory is the rudder and compass..says where you can go (e.g. you cant go up!)18TheoriesWhoOther researchersConsultantsTool developersHowRead and build on/test theoryScopeSmall at firstTime scale1-3 or more yearsAccessFree

Kinds of ImpactsTheoriesAssessment toolsTechnological innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materialsWhat else?Which kinds of impacts will YOU make?Here are the sevencategories of kinds of impacts I would like to review. If you think of more, please put them on the blue cards as well. I dont pretend to be exhaustive, and my examples within these categories couldnt possibly give a shout-out to ALL the people who have made impact. These are examples, with the intent of helping you find a way you can have impact.

20Theories delivered as Assessment ToolsCollaboration Success Wizard

GlobesmartMyers-Briggs Personality AssessmentCogTool

I begin with Assessment tools, because, of course, of the Collaboration Success Wizard. There are three examples Id like to talk about: GlobeSmart, CogTool and a bevy of personality assessment tools.

21Assessment Tools: GlobeSmartBased on academic theories of cultural differencesDavid MatsumotoHandbook of Culture and PsychologyLike the Wizard, they collect data to adjust their assessmentsRecent upgrade used data from 400,000 users from over 60 countries

This is an example of research you can use, and one that continues to be a research endeavor. The original questions and assessment were based on a theory by David Matsumoto, but through the assessment tool, they have collected a lot of more up-to-date data. In this case, a revision of the assessment came out this last fall, based on the responses from 400,000 users. They revised two things, based on these data: some questions were reworded to fit more business users rather than student users, and, importantly, one of the dimensions no longer discriminatedthe long-term/short-term focus went away. It turns out that with increasingly fast-paced world, more and more people who were traditionally long-term in their thinking have moved to a more short-term focus.

22Assessment Tools: Globesmart

Within a country you can get information on various aspects of communication and business, like establishing relationships, and negotiating.

23Assessment Tools: GlobeSmart

The assessment form asks a number of questions like #5, I must be prepared to sacrifice my personal goals in order to achieve the goals of the team as a whole.


What comes back is a profile, here in black, this one for Ted Dale, one of the co-founders of GlobeSmart. You can see that hes moderately indendent (which is common for Americans), totally egalitarian, a risk taker, quite indirect in his speech, a mix of task and relationship focused, and moderately short-long-term oriented. If you compare him to Megan Slater, in blue, you see that though they are similar on the first two dimensions, they differ on the rest. Megan is more risk averse, more direct, very task oriented, and looks at things in the long-term. You can see that they might disagree on some decisions coming down the road (long-term vs short-term) and Ted might be offended by how direct she is. You can also see the contrast of Ted with the generic Indian, having to do with almost every value he holds. We take this generic profile with caution, of course, because individual personalities differ greatly. We are on more solid ground if the actual person you have to interact with takes the assessment, like Megan and Ruth.


This view shows the specific advice to Ted about Megan, noting not only that shes more direct, but then what to do about it.

26Assessment Tools: Myers BriggsBased on the work of Carl JungDeveloped further by Myers and Briggs

Like the GlobeSmart you can see differences in values and habits with people you interact withDimensions of discussionSome professional help

Like GlobeSmart, the value is in comparing your profile with that of people you work with to understand and then find middle ground in which to work successfully. Myers-Briggs was based on the work of Carl Jung and has been used for many years hence. There are some controversial issues surrounding the data collected from the inventories, but the good thing is that people are arguing from data, not opinion.

27Assessment Tools: CogToolBased on work of Bonnie JohnBased on Card, Moran, & NewellGOMS and the Model Human Processor

A general purpose UI prototyping toolIt automatically evaluates your design with a predictive human performance modelA cognitive crash dummy

Another example of an assessment tool that is having some impact is CogTool, based on the work of Bonnie John, on leave from Carnegie Mellon, working on CogTool at IBM. CogTool allows you to evaluate a human-computer interaction interfaces on a set of tasks to find out which are more efficient, both in time to perform and memory/thinking load. She calls it a cognitive crash dummy which I love.

28Assessment Tools: CogTool

The evaluator inputs story boards of the interaction layout, identifying the action areas

29Assessment Tools: CogToolYou can compare expert use task time without recruiting participantsAn excellent choice for completely new systems that dont already have experts.

Specify the task sequence by demonstration and it will tell you the results based on the model human processor of Card-Moran-and Newell fame. It builds performance based on various parameters of behavior from the literature, tested thousands of times in a myriad of tasks. Then, as she says, You can compare expert task time without experts.

30Assessment ToolsWhoGeneral publicHowTake the assessmentScopeCould be hugeTime scaleImmediateAccessSome are free; some cost money


32Technical Innovations: Alice

3-D programming environmentFor telling a storyPlaying an interactive game

Teaching tool for introductory programmingFormally shown to improve learning and performance

Randy PauschFor IMPACT, I cant imagine anyone with more impact than Randy Pausch. His Last Lecture has been viewed over 14 million times. But here I want to point to his work on Alice, a programming language that teaches programming concepts like iteration and branching without typing code. It focuses on story telling in an interactive game. It works. It has been shown to improve learning and performance in a number of settings.

33Technical Innovations: Alice

Caitlin Kelleher,2006 Here is the interface to a version of Alice where 1) the world view of the environment the student will control, here what looks like an ice skater, 2) a list of the objects in the virtual world, 3) the methods like move, turn, roll, think, say, etc. that the characters and objects can do, and 4) a place to order the various methods into a pleasing sequence with branching and iteration and other control structures are embodied in boxes. This particular embodiment was done by one of Randys Ph.D. students, Caitlin Kelleher, now a professor at Washington University in St. Louis.34Technical Innovations: AliceUsing storytelling to make computer programming attractive to middle school girls

Storytelling Alice users spent 42% more time programming were more than three times as likely to sneak in extra time to continue working on their programs

Caitlin Kelleher She in particular focused on getting middle school girls interested in computer programming. She found that in students who used StoryTelling Alice spent 42% more time programming compared to those using a similar interface without the story. And, they were more than 3x more likely to sneak extra time to continue working on their program.

35Technical Innovations: Alice10% of the nations colleges now use AliceAn accompanying textbook, lessons, test banks

88% of at risk students who had Alice in a pre-CS1 course were retained through CS23.03 GPA

As for impact, 10% of the nations colleges use Alice. A community has grown around it that has produced teaching materials: A textbook, lessons, test banks etc. Importantly, in a study of at risk youths, those who had Alice in a pre-CS1 course stuck with it through CS2 with a solid grade point average. Others without Alice did not.

36Which then inspired.iMuseA requirements engineering environment where both developers and stakeholders could understand the flow

Kristina Winbladh

Technical Innovations: More...Hypertext Transfer Protocol HTTPHTTP/1.1 specFielding, Gettys, Mogul, Frystyk and Berners-LeeWebDAV extensionArchitecture of the WebFielding and Taylor

As for other research that has impact because it fits a basic human need, on our own faculty at Irvine are people who were involved in the development of the internet, in particular with the development of HTTP and WebDAV, the extension that allows collaboration among users in editing and managing documents. Of course the granddaddy of all impact is Vince Cerf for creating the internet. While Vince is not in our community, Fielding and Taylor were in Informatics at the time of the development of HTTP and WebDAV, so we can count them as cousins, and very impactful.38Technical Innovations: More.Aspect Oriented ProgrammingDifference lies in the power, safetyand usability of the constructs provided

Original article downloaded6,681 times16,600 articles in Google Scholar with Aspect Oriented Programming

Crista Lopes

Also in Informatics, in a branch of software engineering, was Crista Lopes development of Aspect Oriented Programming, which as it say increases the power, safety and usability of the constructs provided. The original article was downloaded over 6,000 times, and the term Aspect Oriented Programming appears in over 16,000 scholarly articles available to Google Scholar. As an aside, Cristaand others in our field were consulted by Mattel about what Computer Engineer Barbie should be dressed in. (She wears a neon-colored T-shirt with a binary code pattern and carries a smartphone and a Bluetooth headset. Her hot pink glasses will come in handy during late nights coding on her hot pink laptop. Before any one begins complaining, Mattel points out that her accessories were chosen with the help of the Society of Women Engineers and the National Academy of Engineering.)39Technical innovationsWhoStudentsThe general publicOther developersHowUse the technology that makes things possibleScopeHugeTime scale5-10 yearsAccessOften free (though products cost money)


41Guidelines, templates, and patternsAll provide conventionsSo there is little new to learn

Where things go, what they look like

Sometimes task flow guide

Many of the guidelines that exist help with users not having to learn too many new things in order to use an application. It guides people, for example in designing for an Android phone, where to put things and what the various icons look like. Some guidelines also include things that look like task flows, boxes and arrows about what state precedes each new state. For those of us who were around in the early CHI days, these look somewhat like Generalized Transition Networks, that David Kieras introduced, something more detailed than site maps, but containing a static view of the flow of interactions from one state to another. 42Guidelines, templates, and patternsWhat are they based on? Are they consistent?

(Human Interface Guidelines)In Wikipedia under the heading of Human Interface Guidelines, is this list. But the question is where did they come from? Are they based on sound principles of human behavior? A ripe dissertation topic that could have impact revolves around the question, Are these consistent? How do you tell whether one is good or not? How do you change a guideline for the better?43Guidelines, templates, and patternsPrinciples, patterns and practices for improvinguse experience

Early instance:

Christian Crumlish & Erin Malone

Christopher Alexander

A form of guideline called a pattern is a template of recommendations for design for a particular task in a particular situation. One of the best known sets of patterns for interfaces for social interactions is the Yahoo! Design Pattern Library, which was developed by Crumlish and Malone at Yahoo. The design pattern library continues; Crumlish and Malone have published a book on these patterns, under joint publishers Yahoo! and OReily.

44Guidelines, templates, and patternsTheir effectiveness depends onThe research they are based onThe context in which they aroseTheir fit to the context they are being applied to

Patterns come in the form of a Problem, Context, Solution, and Special cases., with examples to illustrate its use. Their effectiveness has to be examined with respect to the research they are based on, the context in which they arose, and their fit to the context that are being applied to. They can be an effective way to have impact on design.

45ToolkitsUI Development environmentsWith extra featuresHighly interactiveGraphicalDirect manipulationAutomatic undoSupport for animationGesture recognitionAmulet - C++Garnet Common Lisp, X11, and Mac

Brad Myers

One of the major figures in the development of toolkits is Brad Myers from CMU. These toolkits provide the user with ready-made modules in the language they are programming in to make various functions easy to incorporate. For Amulet and Garnet, for example, programmers could easily add graphical features, direct manipulation, support for animation, etc. Instead of having people program common features from scratch, components were available to them to select and modify. There are many more toolkits, too many to mention. But the idea is to make certain good designs available in modules to be incorporated into a program, a well designed piece to merely adopt rather than design from scratch.

46Standards

Keeping these up to date.WhoDevelopersEnd usersHowFind and use relevant templates.ScopeSpeeds development, makes software consistent Time scaleImmediateAccessFreeGuidelines, templates, patterns, toolkits and standards

Kinds of ImpactsTheoriesAssessment toolsTechnological innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materialsWhat else?Which kinds of impacts will YOU make?Here are the seven categories of kinds of impacts I would like to review. If you think of more, please put them on the blue cards as well. I dont pretend to be exhaustive, and my examples within these categories couldnt possibly give a shout-out to ALL the people who have made impact. These are examples, with the intent of helping you find a way you can have impact.

49PoliciesStop Online Piracy Act (SOPA)/Protect Intellectual Property Act (PIPA)Network neutralityParticipatory design in ScandinaviaOpen access vs. commercial production of educational materialsData sharing policiesWhoEveryoneHowDictates whats possibleScopeHugeTime scale?AccessWho gets to be in the conversation?Policies


52New Media Dissemination Total views = 14,660,471

Ive already mentioned Randy Pauschs 14 million views of the Last Lecture. Here I want to show you two more examples of what impact can be had. The first is Johnny Lees somewhat amateur videos of what you can do if you take apart a wii controller and sensor and reconfigure them in other ways. This speaks to the people who used to read Popular Mechanics: Take apart something and make it even more useful. He made a video whiteboard, a head tracking input device for desktop Virtual Reality displays, and even a multi-touch whiteboard interface. His work was published in the more traditional venues, of course, but the impact came when he posted videos on YouTube. They have been viewed over 14 million times. 53New Media DisseminationAll videos viewed 6,813,795 times

Hans Rosling

A Swedish medical doctor, academic, statistician and public speaker. He is Professor of International Health atKarolinska Institute[2]and co-founder and chairman of theGapminder Foundation, which developed theTrendalyzersoftware system.

I have to also mention a Ted Talk favorite of mine, Hans Rosling. How many of you have seen his animated information visualization of the change in mortality and various other factors in different countries over the last century? You know what I mean about impact. The message is at least as much about the power of computer based, animated information visualization as it is about the policy and history of these social movement. His suite of videos have been viewed at least 6 million times. Thats impact.

54WhoThe publicStudentsHowYouTube, TedTalks.ScopeHugeTime scaleImmediateAccessFreeNew Media Dissemination


56Action ResearchHelping teachers of autistic children assess behavioral incidents

Helping caretakers and clinicians of preterm infants monitor their movement and other key factors

Gilllian HayesWhoTarget population starting with a small groupHowNew technologies to help critical situationsScopeSmall at first, larger as results are generalizedTime scaleImmediateAccessFreeAction research


59Teaching and Teaching MaterialsUndergraduate teaching 6,970 students in a careerPh.D. students ~40A multiplier because they go on to teach

Teaching materialsBooks for classesCases, exercises

All of us professors have impact in the classroom. In my 42 years of teaching (yes!) I figure I have taught nearly 7,000 students. I figure Ive had at least a small effect on about half of them.given the number I saw that were either looking out the window, asleep or, today, thumbing an important status message to their friends on their iPhones, telling them about the amazing new insights I was imparting on them. More importantly, I have probably chaired nearly 40 Ph.D. students dissertation committees and served on countless others as a committee member. Many, not all, of these graduate students go on to academic careers where they then multiply my efforts to also teach, and affect their own 7,000 students in their careers. Of course, those of you who write textbooks have an impact on educating the next generation, as well. There are a number of excellent textbooks, only two of which I have shown here, that help structure courses in HCI and provide lesson plans, exercises, quizzes, exams, and project topics. Clearly there is impact in doing this.

60Teaching and Teaching MaterialsOnline resourcesthat educate

UsabilityFirst.comHcibib.org Useit.com

In this online world, too, there are a number of good sources for teaching material. I have long advocated for the hci.bib, the resource centered at Ohio State University as a public service to our community. And there are others sprouting up, like usability first.com, originated by Diamond Bullet, and now hosted by Foraker Labs. There are a lot of good resources there about various analysis methods, techniques for user testing, and places where you can further your education in this area. Even jobs.

61Teaching and Teaching MaterialsTeaching or action kits

National Center for Womenin Information TechnologyNCWITThere are also a number of easy to use materials formed for a particular purpose. The National Center for Women in Information Technology (NCWIT), for example, has a number of Programs in a Box, or kits to accomplish various things.

62Teaching and Teaching MaterialsNCWIT

Heres one of them on offera Box about unplugging your curriculum, a set of activities to help people understand some basic concepts. There are others on how to mentor a woman computer scientist, one for in academia, one for corporate positions.

63Teaching and Teaching Materials

Particularly exciting to me is the new Kahn Academy. I learned about this resource from my daughter who is a high school physics teacher. She said there is a whole movement to turn the school-day on its head. The idea is that there are a lot of good teaching materials out on the internet, excellent well crafted lectures. Instead of having students hear a half good lecture from their teacher and then struggling alone to do the problem sets, the Kahn Academy says have the students hear and take notes on the well-crafted lectures available on line, and then do the problem sets in the classroom where help is available. It is an opportunity for collaborative learning and help and instructionjust in time facts and concepts from the teacherwhen the students need it. Kahn Academy has 2,700 video lectures on line already, mostly in the sciences at the high school/undergraduate level. There are to date 289 practice exercises as well, a great resource for getting good learning experiences to students. The offerings in computer science are thin23 lectures mainly on Python. So, there is a big opportunity here, both to use materials and to create materials for others.

Controversy has to do with mere film strip lecture and drill techniques.

64Teaching and Teaching Materials

There are a number of movements to make course material accessible. MIT Open Courseware is the best known of them. These are university level courses. Here there are many more courses in core computer science, with a few on computational cognition and mobile computing. Again, there is a big opportunity here, both to use materials and to create them for others. The site additionally needs some good UX help as well to guide students through a curriculum given their goals. 80% of the visitors rate OCSs impact as extremely positive or positive; 96% of the educators say the site has/will help improve their courses. Its good to see that Open CourseWare is gathering data to see how theyre doing. Open CourseWare is supported by grants and foundations, and also sports a donate button on its pages.65Teaching and Teaching Materials

David Evans &Sebastian ThrunJust the other daynote Jan 23, 2012a friend posted this: A professor from Stanford wants to have broader impact in what hes teaching, so developed on online course in parallel with his classroom teaching. 160,000 students enrolled in the online version; he had to scale up his offerings of exercises and grading and participation. He had all students take the online version, and he was present in class to have discussions and answer questions and encourage independent thinking. He is doing the Kahn Academy at the Undergraduate level. There were more students in his course from Lithuania alone than there are students at Stanford altogether. There were students in Afghanistan, exfiltrating war zones to grab an hour of connectivity to finish the homework assignments. There were single mothers keeping the faith and staying with the course even as their families were being hit by tragedy. And when it finished, thousands of students around the world were educated and inspired. Some 248 of them, in total, got a perfect score: they never got a single question wrong, over the entire course of the class. All 248 took the course online; not one was enrolled at Stanford. So, how did he fare after this experience? He developed a start-up, Udacity, offering low-cost online classes. His goal is to enroll 500,000 for his first course, which happens to be on how to build a search engine.

66WhoStudents HowExposed to lectures, exercises, assessmentsScopeDigital media is the multiplierTime scale1-2 yearsAccessSometimes free; sometimes requires tuitionTeaching and Teaching Materials


68Recap on what counts as impactsWho is impactedHowScopeTime ScaleAccess

Decisions you have to makeWhat it means to have impactWhat countsTheory gets usedDownloads/viewsProfitsDegrees/EducationTechnologiesLives changed..Who is impacted?StudentsDevelopersConsultants

Specific populationsThe general publicWhat it means to have impactScopes differYou affect some people directlyInterventions, teachingYou enable others to be better at making better productsToolkits

You set policyAffect a large number of people

What it means to have impactTime scales differNowe.g. Action research1-3 yearse.g., Publications20-30 yearse.g., Theory Assessment Tools40-50 yearse.g., Cyberinfrastructure development?e.g., Policy (like SOPA/PIPA)

What it means to have impactIs it free?YesKhan AcademyOpen KnowledgeStandards, toolkits, patternsWizardNoUdacityMeyers BriggsGlobeSmartTextbooksDegree programsProductsDecisions you have to make in deciding what kind of impact you will make.73A continuing dialogHow to translate our research to have broader impacts?

How to guarantee quality?E.g. evidence based medicine

How to make it accessible?

How to evaluate impact?Indirect but important impactHow careers are advanced nowProduct innovationPublications

Future+ Impact

It takes the evaluators to change the systemPromotion policyWhat impact will you make?TheoriesAssessment ToolsPopular technologies that become standardsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction ResearchTeaching and teaching materials

Student Volunteers will collect on the way outSo there you have it. Not complete, but a starter. There are a lot of ways to have impact from your research or experience. Build assessment tools, developing amazing technical solutions to things people need, develop guidelines, templates, pattern; design and deliver toolkits. Create videos of your findings. Teach big, and develop teaching materials for others to also teach big. What kinds of impacts will you make?76Thank you

In the interest of potential impactA video of this will appear on the ACM website and the ACM-W [email protected] forget to report back to me what impacts you will make. This is like a pledge of your effort in this direction. And, this talk is being videorecorded and will appear on both the ACM and the ACM-W website.Thank you.

77

Documents

Broader Impacts