Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Lecture0:Introduction
CSE252C:AdvancedComputerVision
ManmohanChandraker
CSE252C,SP20:ManmohanChandraker
Virtualclassrooms
• VirtuallecturesonZoom
– Onlyhostsharesthescreen
– Keepvideoturnedoffincaseofbandwidthissues
– Microphonesmutedunlessspeakingtoreducenoise
– Butpleasedospeakup(remembertounmute!)
– Slidestobeuploadedonwebpagejustbeforeclass
– WhiteboardthroughcameraandiPadtowritethings
• LecturesrecordedanduploadonKaltura
– Availableunder“MyMedia”onCanvas
CSE252C,SP20:ManmohanChandraker
Virtualclassrooms
• VirtualinteractionsonZoom
– Askandanswerplentyofquestions
– Trytohavein-classdiscussions
– “Raisehand”featureonZoomwhenyouwishtospeak
– InstructororTAscanunmute
– Postquestionsonchatwindow
– TAwillhelpkeeptrackofraisedhandsandchatwindow
– Ifinstructorcannotnotice,TAwillinterruptatlogicalpause
• Happytotryothersuggestions
CSE252C,SP20:ManmohanChandraker
Enrollmentlogistics
• Waitlist
– Thereisalongwaitlistfortheclass
– Departmenthasnotagreedtoaddmorestudents
– Youarewelcometoattendlecturesevenifonwaitlist
– TolimitTAworkload,wecangradeonlyenrolledstudents
• Canvas
– Allenrolledandwaitlistedstudentsshouldhaveaccess
• AllannouncementswillbepostedonPiazza
– SendemailtoTAs(CCinstructor)ifdidnotgetPiazzainvite
CSE252C,SP20:ManmohanChandraker
ComputerVision
CSE252C,SP20:ManmohanChandraker
Definingcomputervision
Wall-E: Fact and Fiction (Minh Do, Princeton University)CSE252C,SP20:ManmohanChandraker
Studyingcomputervision
Source:2Pmedia
• Visionisafundamentalinterfacetotheworld
CSE252C,SP20:ManmohanChandraker
Studyingcomputervision
Source:2Pmedia
• Visionisafundamentalinterfacetotheworld
CSE252C,SP20:ManmohanChandraker
Humans
Machines
Virtual
(Cyborgs!)
• Visionisafundamentalinterfacetotheworld
• Weareallusersofcomputervision
Studyingcomputervision
Transportation Design
Communication E-CommerceCSE252C,SP20:ManmohanChandraker
Studyingcomputervision• Visionisafundamentalinterfacetotheworld
• Weareallusersofcomputervision
• Deepandattractivescientificproblems• Howdowerecognizeobjects?
• Whydonewbornbabiesrespondtoface-likeshapes?
[Farroni etal.,2005]
CSE252C,SP20:ManmohanChandraker
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourproblemsbecamemorecomplex
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourambitionsbecamehigher
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourconceptschangedovertime
Definingcomputervision
CSE252C,SP20:ManmohanChandraker
Ourtoolsadvancedovertime
[Mahony etal.,2019]
Gainingperspectiveoncomputervision
CSE252C,SP20:ManmohanChandraker
ImportantforAutopilot:Donothitapolice car!
Gainingperspectiveoncomputervision
CSE252C,SP20:ManmohanChandraker
ImportantforAutopilot:Donothitapolice car!
ImportantforRobots:Stayclearofhumans!
[Wired.com]
Gainingperspectiveoncomputervision
CSE252C,SP20:ManmohanChandraker
ImportantforAutopilot:Donothitapolice car!
ImportantforRobots:Stayclearofhumans!
[Wired.com]
Autonomousdriving
Source:WiredCSE252C,SP20:ManmohanChandraker
AutonomousdrivingWhereisourcar?
StructurefromMotion
VisualSLAM
Whereareotheragents?
Objectdetection
3Dlocalization
Wherearesceneelements?
Semanticsegmentation
Whatisasafepath?
Behaviorprediction
Pathplanning
CSE252C,SP20:ManmohanChandraker
Afewtopicsinthiscourse
• StructureandMotion
• FacesandHumans
• ObjectsandStuff
• MaterialandLighting
• BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
StructureandMotion
CSE252C,SP20:ManmohanChandraker [Agarwaletal.,ICCV2009]
StructureandMotion
CSE252C,SP20:ManmohanChandraker[SongandChandraker,CVPR2015]
FacesandHumans
CSE252C,SP20:ManmohanChandraker
[Blanz andVetter]
[Microsoftblog, 2018]
FacesandHumans
CSE252C,SP20:ManmohanChandraker [Kanazawaetal.,CVPR2019]
ObjectsandStuff
CSE252C,SP20:ManmohanChandraker [Choi,ICCV2015]
ObjectsandStuff
CSE252C,SP20:ManmohanChandraker
[Wangetal.,CVPR2019]
[Leeetal.,ICCV2017]
MaterialandLightingInputimage Augmented image
CSE252C,SP20:ManmohanChandraker [Lietal.,CVPR2020]
MaterialandLighting
CSE252C,SP20:ManmohanChandraker [Li,Yeh,etal.,CVPR2020]
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
Caucasian African-American East-Asian
Proportion in datasets: 80% Proportion in datasets: 10% Proportion in datasets: 5%
High accuracy Low accuracy Low accuracyTraining on biased data without domain adaptation
High accuracy High accuracy High accuracyUsing domain adaptation to address dataset bias
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
Noadaptation
Withadaptation
Labelsrequire1.5hours perimage!
Sourcedomain:goodweather,with labels
Targetdomain:rainyweather,no labels
Trainonsource,apply ontarget
Trainonsource,adapt totarget
[Tsaietal.,ICCV2019]
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker
America
Collect
data
Label
data
Train
modelDeployExisting
solution
Newmarket
Newmarket
....
....
....
....
Asia
Collect
data
1week
Label
data
Adapt
model
1week
Deploy
Africa
Collect
data
Label
data
Adapt
modelDeploy
1week 1year 1week
1week 1week
BiasandPrivacy
CSE252C,SP20:ManmohanChandraker [Xiaoetal.,AAAI2020]
Overallgoalsforthecourse
• Introducethestate-of-the-artincomputervision
• Studyprinciplesthatmakethempossible
• Getunderstandingoftoolsthatdrivecomputervision
• Enableoneorallofseveralsuchoutcomes
– Pursuehigherstudiesincomputervision
– Joinindustrytodocutting-edgeworkinAI
– GainanappreciationofmodernAItechnologies
CSE252C,SP20:ManmohanChandraker
CourseDetails
CSE252C,SP20:ManmohanChandraker
Coursedetails• Eachclasswillcovertopicsincomputervision
• Examplesoftopics
• Correspondence
• Stereo
• Opticalflow
• Structurefrommotion
• Facerecognition
• Humanposeestimation
• Materialandlighting
• Semanticsegmentation
• Objectdetection
• Tracking
• Actionrecognition
• Domainadaptation
• Privacyandfairness
CSE252C,SP20:ManmohanChandraker
Coursedetails
• Topicsstructuredintoafewmodules
– Background
– StructureandMotion
– FacesandHumans
– ObjectsandStuff
– MaterialandLighting
– AdaptationandBias
CSE252C,SP20:ManmohanChandraker
Coursedetails
• “Lightning”presentations
– Fourstudentstopresentinoneclass
– Timelimit:5minutes
– Paperstobeassignedbyinstructor
– Orderofpresentation:alphabetic
• Sendpresentation1daybeforeclass
– Well-practicedandfluentpresentation
– Includenarrationifasynchronous
– Askandanswerquestionsafterpresentation
CSE252C,SP20:ManmohanChandraker
Coursedetails
• Presentationformat(1slideforeach):
1.Motivationandproblemdescription
2.Priorwork
3.Methodoverview
4.Methodanalysis
5.Experiments
6.Futureworkanddiscussion
CSE252C,SP20:ManmohanChandraker
Coursedetails• Classwebpage:
– http://cseweb.ucsd.edu/~mkchandraker/classes/CSE252C/Spring2020/
• Instructoremail:
• TAs:Zhengqin LiandYou-YiJau
– Emails:[email protected] [email protected]
• Grading
– 10%presentation
– 60%assignments
– 30%finalexam
• Aimistolearntogether,discussandhavefun!
CSE252C,SP20:ManmohanChandraker
MyInterests
CSE252C,SP20:ManmohanChandraker
Sceneunderstandingforself-drivingLearningtosimulate
Large-scale,real-time,monocularSFM.
Accuracycomparabletostereosystems.
Teacher
CNN
Student
CNN
ImageHint
loss
Student
Detector
Teacher
Detector
Distillation
loss
GT
loss
Distillationnetworksforfastandaccurateobjectdetection
DistillationforcompressedCNN(student) tomimicuncompressed
CNN(teacher),toachievegreateraccuracyatthesamespeed.
MonocularSFMLearningtopredictuncertainfuturebehavior
DESIRE:DeepStochasticIOCRNNEncoder-Decoder
• DeepCVAE(autoencoder) togeneratediversehypotheses.• RNNtorankpredictions basedonmotion,sceneandinteractions.• Deepinversereinforcementlearningforlong-termfuturerewards.
Reinforcementlearningforsimulations.
Ensuringdiversity andcoverage.
NeurIPS 2017
CVPR2017CVPR2015
ICLR2019
CSE252C,SP20:ManmohanChandraker
Geometricandsemantic3Dreconstruction
WeaklysupervisedsemanticreconstructionMetriclearningforcorrespondence
OcclusionreasoningandlargetransformationsDeepsupervisionforocclusion-reasonedparts
CVPR2016
CVPR2017,PAMI2018
NeurIPS 2016,ECCV2018
ECCV2018,CVPR2019
Input
Occlusion-reasoned
top-viewlayout
CSE252C,SP20:ManmohanChandraker
Physically-basedlearningforshapeandmaterial
Materialsand
global
illumination
Refractive
interfaces
Shapeand
motion
WarpNet ColorNet
SIGGRAPHAsia2018
WACV2018,2019
ICCV2017ICCV2017CSE252C,SP20:ManmohanChandraker
UnsupervisedadaptationtonewdomainsReconstructionwithunaligneddataFacerecognitionforprofileinputs
FromrainytogoodweatherCarrecognitionacrosscameraandlightingconditions
CVPR2019
3DV2017
ICCV2017
CVPR2018CSE252C,SP20:ManmohanChandraker
Take-homemessage
• ComputervisionisakeybranchofAI
• Enablesseveralmodernapplicationsaroundus
• Alotofhighlyvisibleandhigh-impactactivity
• Hugeindustryinterest
• Thisisagreattimetostudycomputervision!
CSE252C,SP20:ManmohanChandraker