Embed Size (px)
Citation preview
1
18-847F:SpecialTopicsinComputerSystems
FoundationsofCloudandMachineLearning
Infrastructure
2
Lecture1:IntroductionandLogistics
FoundationsofCloudandMachineLearning
Infrastructure
GraduateSeminarClass
3
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject(NoExams!)
LearningObjectives
4
o Knowthestate-of-the-artframeworksincloudandmachinelearningandtheirtheoreticalfoundations
o Readandprovideconstructivecriticismofresearchpapers
o Presenttoanaudience,andanswertheirquestions
o Docreative,collaborateresearch
WhystudyCloudandMLinfrastructure?
5
Whatarethelargestwordsafter‘BigData’?
BigDataGoldRush
6
WhogotrichintheCaliforniagoldrush?
BigDataGoldRush
7
WhogotrichintheCaliforniagoldrush?
IntheBigDatarush,it’stheinfrastructurecompanies
TopicsCovered
8
CloudCompu)ng DistributedStorage
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
a b a+b
TopicsCovered
9
CloudCompu)ngo SchedulinginParallelComputing
o MapReduce,Spark
o StragglerReplication
o TaskReplicationinQueueingSystems
o CodedMapReduce
TopicsCovered
10
DistributedStorageo Codingforlocality/repair
o Systemsimplementationofcodes
o Reducinglatencyincontent
download
a b a+b
TopicsCovered
11
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
o SGDandSupportVectorMachines
o Backpropagation,LeNet,AlexNet,GoogleNet
o DistributedGradientDescent
o Hyper-parametertuning
o GANs,Deepreinforcementlearning
TopicsCovered
12
CloudCompu)ng DistributedStorage
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
a b a+b
Instructor:GauriJoshi
13
SM+PhD2010-2016
B.Tech+M.Tech2005-2010
ResearchStaffMember2016-2017
AssistantProfessorFall2017-
Internships
Haveworkedinalltheseareas
14
CloudCompu)ng DistributedStorage
MachineLearning
Modelreplica
PARAMETERSERVERw’=w–αΔw
Modelreplica
Modelreplica
w Δw
a b a+b
StudentIntroductions
15
o Name?
o Department?
o MastersorPhD?
o Previousrelatedclasses(ifany)?
o Whatyouarelookingtolearnfromthisclass?
ClassHoursandWebsite(s)
16
o When:Mon,Wed4:30-6:00pm
o Where:ScaifeHall222
o ClassWebsite(Readings,Schedule):https://www.andrew.cmu.edu/user/gaurij/18-847F-Fall-2017.html
o CanvasSite(Readings,Assignments,Projects):https://canvas.cmu.edu/
ReadingMaterial
17
PaperswillbepostedontheclasswebsiteoronCanvaso Bookchapters
o Surveypapers
o Theorypapers(Scheduling,Queuing,Coding,Optimization)
o Systemspapers(Cloud,MachineLearning)
Additionalreferencebookslistedinthesyllabus
InstructorandOfficeHours
18
Prof.GauriJoshi,ECEDeptEmail:gaurij[AT]andrew.cmu.eduOfficeLocation:CIC4105OfficeHours:Wed2:00-3:00pmorbyappointment
GraduateSeminarClass
19
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
Lectures
20
o Nextweek:DeeperOverviewofcoursetopics
o 3-4Guestlecturesduringthesemesterbyauthorsofpapersrelevanttothisclass
GraduateSeminarClass
21
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
Homeworks(45%)
22
o Classdividedintotwogroups;readthepaperassignedtoyourgroup
o Submitpaperreview(due9:00ambeforeclass)o Tworeviewsperweek(advise:finishthemtogether!)
o Discussionwithyourgroupisokay,butwritereviewsinyourownwords.Listcollaboratorsinthehomeworksubmission
PaperReviewFormat
23
o BriefSummaryofthepaper
o Reflectsyourunderstandingofthepaper
o Nojudgments(positiveornegative)here
o High-leveltechnical/writingcomments
o Significance&correctnessofresults(Don’tbemean!)
o Paperorganization
o Low-leveltechnical/writingcomments
o Smallerclarifications,corrections,typos
o DiscussionQuestionsforClass(atleast2)
o Confusionsaboutthepaper,openresearchdirections
TentativeGradingRubric(Total:10pts)
24
o Clarity,Organization(1pt)
o Understandingofthepaper(5pts)
o High-levelcomments(2pts)
o Low-levelcomments(2pt)
GraduateSeminarClass
25
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
ClassPresentations(20%)
26
o Onestudentfromeachgroupsignsupforpresentation,atleast1weekinadvance
o Eachstudentwillpresent~2timesinthesemester
o 20minpresentation,followedby25mindiscussiono MotivationandRelatedworko Summaryofmainresultso Yourviewsonthepaper
TentativeGradingRubric(Total:10pts)
27
o Motivation(2pts)
o Clarity(2pts)
o Correctness(2pts)
o Engagingtheaudience(2pts)
o Extraresearch,goingbeyondthepaper(2pts)
GraduateSeminarClass
28
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
TentativeGradingRubric(Total:5pts)
29
o Discussionquestions(2pt)
o Attendanceandattention(1pt)
o Speakingupinclass(1pt)
o InsightfulQuestions/Comments(1pt)
GraduateSeminarClass
30
(Almost)nolectures
Readingresearchpapers
Studentpresentations
ClassDiscussions
FinalResearchProject
ResearchProject(25%)
31
o Groupsof2or3
o Originalresearchonatopicofyourchoiceo Topicsalignedwithyourresearchallowedandencouragedo Ifyoucan’tthinkoftopics,cometalktome!
o PossibleProjectTypes:o Newtheoreticalanalysiso Implementationusingoneoftheframeworksdiscussedo In-depthliteraturesurveyofaparticulartopic
Timeline
32
o 1-pageproposaldueSept27
o Publishablequalityreport(max5pg)inACMformato Initialdraftdue:Nov22o Finalreportdue:Dec8
o Peer-review2otherreportso Last2weeksofclass:Presentations(20mineach)
TentativeGradingRubric(Total:20pts)
33
o Originality(1pts)
o ReviewofRelatedWork(1pts)
o WritingandOrganization(2pts)
o TechnicalResults(5pts)
o Peer-ReviewofOtherreports(1pts)
o Finalpresentation(10pts)
InSummary..
34
o PaperReadingo SubmittingReviewso ClassPresentations(~2inthesemester)o FinalProject
Mightseemlikealotofworkbut..o Youwillgetfastandefficientatreadingpaperso Theprojectwillbeafun,collaborativeexerciseo Noexams!
TODO
35
o Formgroupsforclassreadings
o Sign-upforpresentation
o Formgroupsforclassprojects
o Startreadingthepapers
o Startthinkingaboutprojects
NextClass
36
Historyandoverviewofcloudandmachinelearninginfrastructure
Willgiveyouadditionaltimeforthefirstpaperreviewsandpresentations
