Last update: 10/23/14

1

SYLLABUS AS OF OCTOBER 23, 2014 PLEASE CONSULT STELLAR FOR MOST UP-TO-DATE

INFORMATION AND READINGS Instructors Professors Sinan Aral & Erik Brynjolfsson Classroom E51-376 Class times Tuesdays 4-5:30pm (September & October)

Workshops 3-6pm, September 11 & 10am-4pm, December 5 Teaching Assistant Noel Sequeira noel.sequeira@sloan.mit.edu Project Mentors

Chuck Gibson cgibson@mit.edu; Paul Grigas pgrigas@mit.edu; Hossein Ghasemkhani hosseing@mit.edu; Frank MacCrory maccrory@mit.edu, Renee Richardson Gosline rgosline@mit.edu; Shachar Reichman shachar@mit.edu; Deborah Soule dsoule@mit.edu; George Westerman georgew@mit.edu

Administrative Assistant Office Hours

Susan Young susany@mit.edu By Appointment

Email: sinan@mit.edu; URL: http://web.mit.edu/sinana/www/ Email: erikb@mit.edu; URL: http://digital.mit.edu/erik/

Summary and Objectives: Action learning seminar on analytics, machine learning and the digital economy The unprecedented growth in big data and analytics is driving a revolution in management decision-making, operations, marketing, finance, and product innovation. Businesses across the world are wrestling with challenges and opportunities that call for the application of analytics. We are on the cusp of a second machine age a digital age comparable to the advent of the steam engine, the internal combustion engine, and electricity. This will be an era that holds opportunities and challenges for both individuals and the economy. Workers and professionals in all fields are racing to acquire the skills and capabilities necessary to survive and thrive in this digital revolution. The purpose of the Analytics Lab (A-Lab) is to match student teams with leading-edge projects involving big data, analytics, or digital technologies such as machine learning as they apply to business questions and problems. The particular focus of the projects is on the business rather than the technical aspects.

15.S06 Fall 2014 Analytics Lab:

Analytics, Machine Learning & the Digital Economy Professors Sinan Aral & Erik Brynjolfsson

Last update: 10/23/14

2

Project teams of three or four will be formed which mix levels and types of experience. Project topics will be matched to team interests. Project proposals have been received from the following organizations, and more may come: Amazon, BASF, Capgemini, Center for Digital Business, Enlitic, Fusion, GE, Houghton Mifflin Harcourt, IBM Watson, Imagitas/Pitney Bowles, Marathon Data Systems, Northwestern Mutual, Open Source Media, Thomson Reuters, WOOX Innovations, and Zensar. Course Principles and Expectations: The primary criterion for projects is to provide a learning experience for the students. In addition, the projects should be of high relevance and interest to a particular organization and senior managers and professionals in it. Each project team will have an MIT-associated faculty or research mentor to provide guidance and assistance and a link to outside project sponsors. Students will be expected to attend all the weekly 1.5 hour class sessions in September and October, some of which will have guest speakers from proposing organizations listed above. In November and December, each student team should plan to meet weekly with their research mentor. In addition,there will be a special session on September 11 for matching team interests with corporate sponsor proposals. On December 5 the seminar culminates with an all-day workshop of presentations by each of the teams, with invited guests including sponsoring corporate project representatives. Class Meetings and Activities: 1. Regular class meetings: Tuesdays from Sept 9 through October 28 (except SIP Week) from 4:00 pm -

5:30pm; wrap up session December 9. 2. First Session, on September 9, will be followed by an informal pizza-provided session to facilitate

team formation. 3. Match Day session: Thursday, September 11, 3:00 pm - 6:00 pm. We will meet jointly with the

representatives from each project proposing company. They will briefly describe their project as proposed, and students will have an opportunity to circulate to tables to meet each of them and informally mix with them and fellow students. The Session will be followed by a reception for all.

4. OPTIONAL: Friday, October 10, 6:00 - 7:30pm: Conference on Digital Experimentation (CODE),

Fireside Chat on Experimentation and Ethical Practice location and further details to be announced.

5. Final Workshop: Friday, December 5, 10:00 am - 4:00 pm: final presentations, all students entire

session.

Last update: 10/23/14

3

6. Milestones and due dates:

- September 16: team formation and project selection preferences due by 9:00 pm; each team should submit one document to Noel Sequeira ; in the following days, faculty mentors will work out assignments of projects to teams, subject to review by the proposing company. - September 23: final resolutions will be communicated to students by September 23. - September 30: project plan due by 9:00 pm; each team should submit one document to Noel Sequeira . - In November and December, project teams are expected to work independent of regular class meetings, with advice and assistance available from the instructors and mentors, with the opportunity for travel as may be required. Project sponsoring organizations will cover costs of travel and lodging, if any. - December 10: final report due by 9:00 pm (10 pages maximum, word-limit of 3000 words); report should consider feedback received during final presentations on December 5. Each team should submit one document to Noel Sequeira .

Grading:

30% final presentation content team-wide 30% final presentation delivery team-wide 20% timeliness and quality of milestones team-wide 20% contribution to class discussions and team project enablement individual

Required Reading: Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking, Foster Provost and Tom Fawcett. 2013. OReilly Media Inc.

Last update: 10/23/14

4

Class Schedule

Date Time Session Lecturer S1 9/9 4-5:30pm The Economic Payoff From

Analytics Erik Brynjolfsson

S2 9/11 3-6pm Match Day: Meet Project Proposers -- S3 9/16 4-5:30pm Social Analytics: A Deep Dive Sinan Aral S4 9/23 4-5:30pm Team Meetings S5 9/30 4-5:30pm Machine Learning at Scale IBM Watson Team

S6 10/7 4-5:30pm Experimentation and AB Testing Brooks Bell S7 10/14 4-5:30pm The Analytics Advantage Jeremy Howard 10/21 SIP - NO CLASS S8 10/28 4-5:30pm The Art of Data Science

(rescheduled from 9/23) Foster Provost

S9 11/4 4-5:30pm Team Meetings With Mentors S10 11/11 4-5:30pm Team Meetings With Mentors S11 11/18 4-5:30pm Team Meetings With Mentors S12 11/25 4-5:30pm Team Meetings With Mentors S13 12/2 4-5:30pm Team Meetings With Mentors S14 12/5 10am-4pm Team Project Presentations Student Teams S15 12/9 4-5:30pm Final Wrap Up Erik and Sinan

Last update: 10/23/14

5

Reading List Session 1: The Economic Payoff from Analytics (9/9): 1. Chapter 1: Introduction: Data Analytic Thinking Provost, F. and Fawcett T. 2013. Data

Science for Business, OReilly Media Inc.; http://library.mit.edu/item/002221893. 2. Big Data: The Management Revolution Brynjolfsson, E. and McAfee, A. 2012. Harvard

Business Review, 90(10); October: 60-68. Optional Reading: 3. Strength in Numbers: How Does Data-Driven Decision Making Affect Firm Performance?

Brynjolfsson, E., Hitt, L., and Kim, H. 2011; http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1819486.

4. "The Future of Prediction: How Google Searches Foreshadow Housing Prices and Sales" Wu, L. and Brynjolfsson, E. Economics of Digitization (A. Goldfarb, S. Greenstein, and C. Tucker, Eds), Univ. of Chicago Press, 2014 (in press). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2022293.

5. Three-Way Complementarities: Performance Pay, Human Resource Analytics, and Information Technology Aral, S., Brynjolfsson, E. and Wu, L. 2012. Management Science, 58(5); May: 913-931.

Session 2: Match Day: Meet Project Proposers (9/11): 6. Carefully review all project proposals Optional Reading: 7. Chapter 2: Business Problems and Data Science Solutions Provost, F. and Fawcett T.

2013. Data Science for Business, OReilly Media Inc. Session 3: Social Analytics: A Deep Dive (9/16): 8. Chapter 3: Introduction to Predictive Modeling: From Correlation to Supervised

Segmentation Provost, F. and Fawcett T. 2013. Data Science for Business, OReilly Media Inc.

9. The Problem with Online Ratings Aral, S. 2014. MIT Sloan Management Review, 55(2); January: 47-52.

10. What Would Ashton Do - And Does it Matter? Aral, S. 2013. Harvard Business Review, 91(2); May: 25-27.

Optional Reading: 11. Social Influence Bias: A Randomized Experiment. Muchnik, L., Aral, S., and Taylor, S.

2013. Science, August 9: 647-651. 12. Identifying Influential and Susceptible Members of Social Networks. Aral, S. and Walker,

D. 2012. Science, July 20: 337-341.

Last update: 10/23/14

6

13. Creating Social Contagion through Viral Product Design: A Randomized Trial of Peer Influence in Networks. Aral, S. and Walker, D. 2011. Management Science, 57(9); September: 1623-1639.

14. Distinguishing Influence Based Contagion from Homophily Driven Diffusion in Dynamic Networks, Aral, S., Muchnik, L., and Sundararajan, A. 2009. Proceedings of the National Academy of Sciences (PNAS), 106(51); Dec. 22: 21544-21549.

Session 4: The Art of Data Science (9/23): 15. Chapter 4: Fitting a Model to Data Provost, F. and Fawcett T. 2013. Data Science for

Business, OReilly Media Inc. 16. Chapter 5: Overfitting and Its Avoidance Provost, F. and Fawcett T. 2013. Data Science

for Business, OReilly Media Inc. Session 5: Machine Learning at Scale (IBM Watson) (9/30): 17. Chapter 6: Similarity, Neighbors and Clusters Provost, F. and Fawcett T. 2013. Data

Science for Business, OReilly Media Inc. 18. Chapter 7: Decision Analytic Thinking I: What is a Good Model? Provost, F. and Fawcett

T. 2013. Data Science for Business, OReilly Media Inc. Optional Reading: 19. IBM is Betting that Watson Can Earn Its Keep Hardy, Quentin. 2014. New York Times,

Jan 8. http://www.nytimes.com/2014/01/09/technology/ibm-is-betting-that-watson-can-earn-its-keep.html

Session 6: Experimentation and AB Testing (10/7): 20. Chapter 8: Visualizing Model Performance Provost, F. and Fawcett T. 2013. Data Science

for Business, OReilly Media Inc. 21. Practical guide to controlled experiments on the web: listen to your customers not to the

HiPPO R Kohavi, RM Henne, D Sommerfield. 2007. Proceedings of the 13th ACM. http://www.exp-platform.com/Documents/GuideControlledExperiments.pdf

22. Kohavi, Ron, Alex Deng, Roger Longbotham, Ya Xu. Seven Rules of Thumb for Web Site Experimenters http://www.exp-platform.com/Pages/SevenRulesofThumbforWebSiteExperimenters.aspx

Optional Reading: 23. Chapter 9: Evidence and Probabilities Provost, F. and Fawcett T. 2013. Data Science for

Business, OReilly Media Inc. 24. Are You Letting a Groundhog Dictate Strategy? DeFranza, David. 2014. Brooks Bell,

February 6. http://www.brooksbell.com/blog/are-you-letting-a-groundhog-dictate-strategy/ 25. The Surprising Thing Brand New and Highly Advanced Testing Programs Have in

Common DeFranza, David. 2014. Brooks Bell, June 19.

Last update: 10/23/14

7

http://www.brooksbell.com/blog/surprising-thing-brand-new-highly-advanced-testing-programs-common/

Session 7: The Analytics Advantage (10/14): 26. Chapter 13: Data Science and Business Strategy Provost, F. and Fawcett T. 2013. Data

Science for Business, OReilly Media Inc. 27. Browse: http://www.enlitic.com Optional Reading: 28. Competing on Analytics Thomas Davenport. 2006. Harvard Business Review, 84(1);

January: 98-107.

Session 8 - Session 14: Team Meetings With Mentors: NO CLASS-WIDE ASSIGNED READINGS: Reading, interviews and preparation based on team project requirements Session 15: Final Project Presentations (12/5): Team Project Reports (Penultimate Drafts)