In cooperation with

Computer Vision NewsThe Magazine of The Algorithm community

A publication by

Women in Computer Vision

PresentationsToday’s Picks

Interview withOctavia Camps

Interview withJitendra Malik

FridayOctober 14

For Friday 14

2ECCV Daily : Fr iday

Miaomiao’s & Buyu’s Picks

Miaomiao Liu (left in thepicture) works as a researchscientist for Data61, CSIRO inCanberra Australia. She isworking on 3D vision, 3D scenemodeling and understanding.

Buyu Liu (right in the picture)works as a Post-doc in CalvinVision Group at University ofEdinburgh in the UK. She isworking on holistic videounderstanding and human poseestimation.

Morning:9:00 to 10:00 O-4A-2 Page 85 of the Pocket Guide• Single Image 3D Interpreter Network

10:00 to 10:30 S-4A-6, A-4A-9 Page 85 of the Pocket Guide• Learning Image Matching by Simply Watching Video• Learning a Predictable and Generative Vector Representation for

Objects11:00 to 12:30 P-4A-20 Page 87 of the Pocket Guide• Semantic 3D Reconstruction of HeadsFor some reasons, Buyu forgot to tell us that today she also presents aposter in this session, so we remind you for her. Please visit also:P-4A-19 Learning Dynamic Hierarchical Models for Anytime Scene Labeling

Afternoon:16:00 to 17:30 O-4B-4, P-4B-23, Pages 79-81• Look-ahead before you leap: end-to-end active recognition by

forecasting the effect of motion• Where should saliency models look next?• For some reasons, also Miaomiao forgot to tell us about her poster in

this session, so we recommend that you visit also her. Find her at:P-4B-43 Building Scene Models by Completing and Hallucinating Depth andSemantics

Dear Reader,

This is the 4th and final issue of ECCV Daily forthis year. It is the right moment to think aboutwhat we've achieved with this first edition of thismagazine and what we, the 1701 participants,have experienced this week here in Amsterdam.

The 4 ECCV Daily magazines (including this one)had the great privilege of exposing the work ofmore than 100 scientists. In particular, Ipersonally enjoyed the honor of interviewingOctavia Camps 🔗 , Michael Black 🔗 and JitendraMalik 🔗 for ECCV Daily: besides fulfilling three ofmy professional dreams, I have learned from allthree of them an unforgettable lesson of graciouskindness, patience, positive curiosity, openaccessibility and dedication. Our community is solucky to have them.

Besides the exceptional level of the award-winning works and the success of all technicalprograms, hundreds of excellent papers werefeatured at the conference. Through as manypresentations, we have unveiled the remarkabletalent of many researchers. ECCV Daily hasmanaged to expose in a few days the work ofmore than 100 scientists. In particular we areproud of our choice of believing in EmmaAlexander and the Focal Flow team, thedeserving winners of the Best Paper Award: theonly time we named a section "Our Pick forToday" was on Wednesday for Emma: read againthe 4 pages we had dedicated to her work.

One of the our most successful sections has beenWomen in Computer Vision, hosting 4 brilliantwomen scientists to offer an open microphoneand compensate asymmetries with respect towhat is still a minority in our community. Meetone of them now at page 12 🔗.

See you all again at ECCV 2018!

Ralph AnzarouthEditor , Computer Vision NewsMarketing Manager, RSIP Vision

Miaomiao’s & Buyu’s Picks: 2

Editorial and Summary 3

Interview Jitendra Malik 4

PresentationsYoni Kasten 7

Interview Octavia Camps 8

Women in Computer Vision Cheng Zhang 12

PresentationsLisa Anne Hendricks 18

Bosch 20

PresentationsSimone Meyer 21Guy Lev 22

ECCV DailyEditor: Ralph AnzarouthPublisher: RSIP Vision

Copyright: ECCV & RSIP VisionAll rights reservedUnauthorized reproductionis strictly forbidden.

ECCV Daily’s editorial choices are fully independent from ECCV

Dear reader,

Would you like tosubscribe to ComputerVision News andreceive it for free inyour mailbox everymonth?

FREE

SUBSCRIPTION

Subscription Form(click here, it’s free)

Welcome 3ECCV Daily : Fr iday

4E CCV Da i l y : Fr i da y

Interview with Jitendra Malik

Ralph Anzarouth: Jitendra, youhave taught computers to do somany things. Is there one thing thatcomputers haven’t learned from youyet and you would like to teachthem?

Jitendra Malik: In the context ofcomputer vision, I think one of thekey problems that we have not yetsolved is prediction: given a certainsituation, what will happen next.Usually, the benefits of perceptionare in guiding action. One of theways in which it comes about is thatin any situation we know what islikely to happen next. If we see atiger moving, we kind of know thatit is going to jump at us orwhatever… or cars moving.

But sometimes, the behavior ismore sophisticated. Let’s say youare sitting in a restaurant, and youare waiting at a table. Then youhave an expectation that a waiterwill come to you. So that’s aprediction. If you are playing somesports, then you have a predictionabout whether somebody will passthe ball to somebody else, whethersomebody is trying to score a basketand whether that will succeed ornot.

What I like about the visualprediction problem is that itcombines understanding aboutobjects, agents, behavior, shortterm, long term, and it brings insome amount of cognition, but it isa very visual problem. You must useyour own perception of the scene todo this.

Ralph: There is also some randomcomponent in it.

Jitendra: Yes, so it’s not that you willalways have the right answer. Youhave a probability distribution overa set of possible futures.

“In the context of computer vision, I think one of the key problems

that we have not yet solved is prediction”

Ralph: You have conducted somany projects, and you havehelped your students in countlessother projects. After all of thiswork, where can you still find newideas for new projects and newdirections?

Jitendra: For me, human vision andbiological vision are a source ofgreat insight. biological vision are asource of great insight. I always liketo think about all of the range ofproblems that are solved in humanvision and biological vision. Untilwe attain that level ofsophistication, which we have not,there always remain problems tobe tackled.

When we come to solving theseproblems, we don’t need to imitatebiology in detail. It terms of posingproblems, thinking about whathumans can do has been the mainsource of insight for me.

Ralph: We are pretty much fromthe same generation. We see andhear young engineers who arequite different from what we wereat their age. If there was onefeature that these young people ofthis generation have that you couldtake for ours, what would it be?

Jitendra: I think they have a muchgreater sense of confidence thatthese problems can be solvedbecause they are operating at atime when computing power, data,and so forth are much more readilyavailable. They have already seensome successes. That gives themthis confidence that many of theseproblems can be solved, but theytend to think of it in thisengineering solution way.

Interview with Jitendra Malik 5ECCV Daily : Fr iday

“I always like to think about all of the range of problems that are solved

in human vision and biological vision. Until we

attain that level of sophistication, which we have not, there always remain problems to be

tackled”

“I think the young generation has a much

greater sense of confidence that these

problems can be solved”

6E CCV Da i l y : Fr i da y

Interview with Jitendra Malik

In an earlier generation, we didn’tthink of it so much as a set ofengineering problems to be solved.We thought of it more as scientificchallenges. I think that’s the trade-off. The current generation is toomuch engineering and too littlescience, and the old generationwas too much science and too littleengineering, because we didn’thave any successes in practicalsettings.

Ralph: Did you see any of yourcolleagues or students whomanaged to have both?

Jitendra: I think now everyone istrying to do both. I would expectthat the field evolves. Mechanicalengineering is grounded in physics,but a lot of the stuff that was onceupon a time subjects that physicistslike Newton and so forth studied, isreally in the province of

engineering. Physics professorsdon’t study that anymore. Fields gothrough this transition…sometimes over hundreds of years.In our field, maybe it’s over thenext 50 years. 50 years havepassed and there will be another50 years to go.

The field as a whole has to evolvefrom science to engineering, but itdoesn’t happen simultaneously forall of the problems in the field.Some problems are more at thescience stage, some problems aremore at the engineering stage.What I try to do is to think aboutwhen a problem becomes verymuch in the engineering state,then I want to get out of thatproblem. I want to focus on theproblems which are in the earlierstages where it’s really aboutscientific research, because once itbecomes more of an engineeringproblem, at that point it reallyshould be done by industry.

Ralph: So at this point you aremore fascinated by science thanengineering?

Jitendra: Yes.

Keep reading us also after ECCV 2016. Subscribe now for free at Computer Vision

News(click here)

“The current generation is too much engineering and too little science, and the old generation was too much science and too

little engineering”

“once a problem becomes more of an engineering problem, at that point it really should be done by

industry. ”

Yoni Kasten prepared this projecttogether with Shmuel Peleg andMichael Werman who areprofessors at the Hebrew Universityalong with Gil Ben-Artzi, theadditional author.

The work that they presented findsthe fundamental matrix betweentwo views with wide angle whenthey recorded videos of multipleobjects moving inside.

The main novelty of this work, withrespect to previous work, is that thiswork can deal with multiple objects.Previous methods could deal withone object moving in such a videoscenario in cases where camerashave a wide angle between them.When using cameras without wideangles, you can use feature points inorder to find the fundamentalmatrix. If the angle is wide, youcannot do that because thebackground is very different.However, you can use the motion inthe video sequences in order to findthe fundamental matrix using theconvex hull of the foreground objectin each view, which can not be usedwhen there are multiple objects inthe scene.

This model has very important andspecific practical applications. Forinstance, there are surveillancecameras that recorded the samescene and the cameras can be

calibrated using this method. Themodel has been tested on bothsynthetic and real datasets includingPETS 2009, which is a verychallenging dataset. Fundamentalmatrices were found with highaccuracy. The method is based online motion barcodes, descriptors forlines in a video sequence which areinsensitive to the wide angles ofcameras. Follow the link to thisvideo to better understand whatmotion barcode is.

Fundamental Matrices from Moving

Objects Using Line Motion Barcodes7

ECCV Daily : Fr iday

Yoni Kasten is a student at HebrewUniversity of Jerusalem having justcompleted his Master’s degree.

Ralph: I am interviewing OctaviaCamps. For me it is a doublepleasure, since it follows thechance I had of interviewing for thelatest CVPR Daily one of herstudents, Angels Rates Borras, whotold me so many nice things aboutOctavia. So thanks to ECCV formaking this happen and thank youOctavia for being with us.

Octavia: First of all, thank you forthe interview. It is a pleasure to behere.

Ralph: You have done a lot of workby yourself in creating so manymodels, and you have supervisedyour students to invent even moremodels. After so many years, whatdrives your curiosity to keepinventing? Do you have momentswhen you lose your sense ofcuriosity?

Octavia: What I really strive for isto have something that is going tobe really useful and have an impactin our daily lives. One of the worksthat I’m most interested in rightnow is visual surveillance. Whatsurprised when I started talking topeople at airports, for example, isthat most cameras are therewatching people, but there isn’tany video analytics going on. Thereis not much automatic processingof that video. It’s just relying onpeople.

We’re doing such great things hereat the conference. Everything isgetting flashier and better, butwhen you try to apply that to realworld problems with poor qualityvideo from surveillance cameras, itdoesn’t work as much. I really wantto think of work in the real world,not in the curated data basis thatwe are collecting. We have come a

“We include an anatomy-based eyeball

model and computer animation techniques”

8E CCV Da i l y : Fr i da y

Interview with Octavia Camps

“It’s not about one more paper”

long way to be more and morerealistic, but we are not thereyet. When we try to apply this tothe real environment, we still havea long way to go.

Ralph: If I understand you, whatdrives your curiosity is the fact thatyou are making an impact in thereal world?

Octavia: Exactly.

Ralph: That is what gives you thewill to continue and invent? Only ifthe solution helps improve people’squality of life?

Octavia: Absolutely, it’s not aboutone more paper.

Ralph: You talked aboutsurveillance. What do you thinkabout all of the privacy issues thatthe huge amount of video camerasaround us arises?

Octavia: Absolutely, it’s a difficultquestion. There are manyimplications about it. Immediately,when somebody talks aboutsurveillance, the first thing thatthey think of is Big Brotherwatching you. On the other hand, Ithink that we have already given upprivacy in many places like publicspaces or airports. We are not

surprised that cameras arewatching us. In those cases I thinkthe potential benefit of preventingsomething really bad fromhappening is bigger than the loss ofprivacy.

Ralph: So you are still positiveabout it?

Octavia: Yes, but we have to becareful of course with how it’shandled.

Ralph: The problem is who is goingto handle that. Who can you trust?Can we know?

Octavia: No, but that’s true foreverything, right?

Ralph: That’s true. Do you knowany of your colleagues thatregretted a model that theybrought up?

Octavia: No, I don’t knowfirsthand, but I’m sure there aresome.

Ralph: One of the words I hear themost in this community is“overcommitted”. What drivesthese young people to committhemselves to more than they areable to and make their life moredifficult than what it already is?

Interview with Octavia Camps 9ECCV Daily : Fr iday

“Immediately, when somebody talks about

surveillance, the first thing that they think of is Big Brother watching you”

“I think the potential benefit of preventing something really bad

from happening is bigger than the loss of privacy”

10E CCV Da i l y : Fr i da y

Interview with Octavia Camps

Octavia: I think it’s excitement. Welike the things that we do, and wealways think that we can domore. Then we look on thecalendar and say I should have saidno to this. Yet it’s such aninteresting project, and I want towork on it.

Ralph: Sometimes you regret it?

Octavia: Of course, but you have todo it, right? Because you made acommitment. When you do ityou’re enjoying it too. You need todo a lot of juggling.

Ralph: It’s funny because you havedone this many times with manyprojects, and still you say it’sexciting. Are you still as passionateas when you started?

Octavia: Yes, even more because ifyou compare what we were able todo in computer vision when Istarted as a PhD student and thethings that we are doing today, it’sjust amazing how much the field

has grown. It keeps being exciting.The closer we are to applyingthings to real life problems andhaving an impact, it becomes evenmore exciting.

Ralph: That’s extraordinary,because in many jobs you becomeless and less excited as the years goon. Do you think that things willcontinue to get better?

Octavia: Sure, There are somethings that I hope don’t getforgotten through all of thisprogress. We were talking abouthow great it is that deep learning isdoing so much for the field, butsometimes I worry about theyounger generation neglecting thebasics of geometry and physics ofvision because it’s so exciting to bein this new deep learningmovement.

Ralph: Of course, because that’sthe hype.

Octavia: Right, so there are someconcerns.

Ralph: Hype is attractive.

Octavia: Sure, it is attractive andkeeps pushing the field. It makespeople want to be in the field.

“We like the things that we do, and we

always think that we can do more”

Ralph: You know already thesection that our publications,Computer Vision News and nowECCV Daily, dedicate to Women inComputer Vision. Do you know ofyoung women who found it difficultto be a woman scientist and gaveup just because of that?

Octavia: No, I don’t know ofanyone. I think that we have comea long way. When I startedcomputer vision and then startedcoming to the conferences, youhad maybe 3, 4, 5 women in thecommunity. That was it. We are stilla minority, but you see a lot ofyoung women. I, personally, I haveto say, I never felt discriminatedbecause of being a woman.

Ralph: Are you embarrassed to bethe only woman in aman’s team?

Octavia: You get used to it. In mycountry, when I did my undergrad,we were like 300 students, andthere were maybe 5 women in thatgeneration, but I always was “oneof the guys”… [she smiles]

Ralph: Did you notice somediscomfort among younger ladies?

Octavia: Yes, but I think the morewomen there are in advising rolesor as role models, of course thathelps them feel more comfortablein the room.

Ralph: Something that comes upagain and again is a lack ofconfidence. What would you adviseto a young woman starting todayto help acquire more confidence inthis setting?

Octavia: I think at things that weare already doing: networking,having women talk to otherwomen, seeing that there aresuccessful women, that they canbe confident, and they areaccepted. That’s important. Alsohaving a mentorship.

Ralph: Is there anything that youwould like to add to what youalready said?

Octavia: We need to keep workingon problems that are relevant, thathave real application, and thathave an impact. It shouldn’t be justabout one more paper, but reallywhat are we trying to do with all ofthis to help us.

Ralph: Are you optimistic for thefuture?

Octavia: Oh yes.

Ralph: What will science look like ina few years from now?

Octavia: Who knows? That’s thebeauty of it, right? If people wouldhave said 10 years ago what we aredoing today, we couldn’t dream ofit. We could dream about it, but itwas hard to believe it wouldhappen so quickly. The limit is thestars. Who knows?

Interview with Octavia Camps 11ECCV Daily : Fr iday

“When I started computer vision and then started

coming to the conferences, you had maybe 3, 4, 5

women in the community”

ECCV Daily: First, congrats for yourPhD, received only two weeks ago!

Cheng: Thank you!

ECCV Daily: Cheng, yours is a veryspecial personal path. Can you shareit with our readers?

Cheng: That’s a long story. I neverknow exactly what I like, so mystrategy is to try different things. Formy Bachelor’s, when I chose myprogram, I thought I wanted tobecome an engineer. I wasn’t clearabout the difference betweeninformation theory and so on. Atthat time, I didn’t even know aboutcomputer vision or machinelearning. I was only 17. I chosemechanical engineering andautomation because it soundedcool.

ECCV Daily: Where was it?

Cheng: In Beijing, China

ECCV Daily: Did you grow up in thecapital?

Cheng: No, I didn’t, but my family isfrom Beijing. I actually grew up inTaiyuan which is a city about 3 hoursaway from Beijing by train and afterthat I went back to study.

At the time in high school, I neededto choose a program. I knew that Iliked engineering so I chose that.Then I started and realized there areso many different programs. So I didmy second degree, at the economicsand management school. I have twoBachelor’s degrees.

I took the courses there, but they

were not challenging. It’s not thekind of problems that I want tosolve. There they solved interestingproblems like market analysis, but Ireally felt like I wanted to solvetechnical problems.

ECCV Daily: Why do you want tosolve these kind of problems? Is thissomething that gives yousatisfaction?

Cheng: Yes, exactly. I want to makesome impact in the world. Big orsmall, I want to leave somethinguseful.

ECCV Daily: Were you a gifted child?

Cheng: Well, it’s hard to define,right? [she laughs] I don’t think so. Ithink I’m just a normal person, but Inever had problems with math or

Cheng Zhang

12ECCV Daily : Fr iday

Women in Computer Vision

physics in high school. Some saythat I’m lucky, but I’m not in thenational team competing for theOlympics. It’s a little bit anoverstatement to say, “I’m gifted”.

I was not really encouraged to gointo a technical field. For me,everything was the same. I didn’thave particular interest in finance orhuman resources at that time. I alsodidn’t have pressure from family orfriends. I think they knew that I likedtechnology.

ECCV Daily: So you were notencouraged, but you didn’t haveobstacles?

Cheng: Exactly, that was mysituation. So I did it my way. I trieddifferent programs, but then Irealized I wanted to go into thetechnical field. Then I got the chanceto go to Sweden to study technologyfor my Master’s degree.

ECCV Daily: Why in Sweden? Whyleave China?

Cheng: There are differentperspectives in life, right? In mystudies, I wanted to try differentthings, and so it’s the same in life. Iwanted to try different cultures. Iread magazines about Swedensaying it was a quiet, beautifulcountry with a lot of forests. I alsolisten to a lot of music from Sweden.

ECCV Daily: Who do you listen to?

Cheng: I really like Swedish deathmetal like Arch enemy, Grave,Entombed, and Amon Amarth. Ithink most people don’t know thesebands [she laughs]. Anyway, thereare many reasons why I chose to go

abroad to KTH University. Sweden isa country I wanted to check out, andKTH is a fine university.

ECCV Daily: Did you have chances togo to other places?

Cheng: I didn’t actually apply. InChina, we have a special programfor recommended Master’sstudents. The universityrecommends the top 10% or 15% ofthe students to a Master’s programwithout going through theexaminations. We had theopportunity to choose without evenapplying. There were a few options,but I always liked Sweden.

ECCV Daily: How did you feel whenyou were accepted?

Cheng: I was really happy to start anew life!

ECCV Daily: Did you feel like you hada high chance of being accepted?

Cheng: I was pretty confident withmyself.

ECCV Daily: What gave you thatconfidence?

Cheng: There were always somerequirements from the universitysuch as English skills. My English is

Women in Computer Vision 13ECCV Daily : Fr iday

not perfect, but when I was inuniversity in China, I wasn’t afraid ofspeaking English with other people.The interview involved a lot ofEnglish which I was prettycomfortable with, but a lot ofstudents got super nervous. Ireceived awards for my Englishskills. Looking back, I don’t think Iwas that good, but in China, I waspretty confident.

ECCV Daily: Knowing Swedish wasnot a requirement?

Cheng: No, it’s not. I also checkedthat out, and Swedish people speakthe best English as a non-Englishspeaking country.

ECCV Daily: You felt confidentbecause you met the requirements,and logically you thought that your

chances were very high. Is this whyyou felt confident? It wasconfidence based on logic.

Cheng: Exactly. I have my opinion,and I had evidence to support that. Ihad confidence in myself, andreasons for it.

ECCV Daily: How long after thatdecision did you go to Sweden?

Cheng: It was 3 months later. It waspretty fast.

ECCV Daily: What was it like whenyou arrived in Sweden?

Cheng: When I arrived, it wassummer, and there is no night. Ithought, wow, now I have so muchtime to do so many things. Therewere so many involved students andlots of student activities. I made alot of friends from all over theworld. Before I went abroad, Ithought foreigners were different.The first thing I realized is we’rereally all human.

ECCV Daily: Did you feel the sameopenness among men and womentoward you?

Cheng: In general, yes, but there area couple of situations. There weresome reactions when I got the PhDposition where people didn’t knowwhy I got the position and theydidn’t. I think that’s jealousy. Theythink that maybe it’s therecruitment process or maybe it’sbecause they want to hire morefemale PhD students. But it’s nottrue! I am that good!

ECCV Daily: How do you react tothis?

14ECCV Daily : Fr iday

Women in Computer Vision

Cheng: Talking definitely helps. Mysupervisor is a female as well. Hername is Hedvig Kjellström. Thehead of our lab is also a female.People have their own freedom tothink what they want. It’s notgeneral. Not everyone thinks likethat, but it happens. For example,some people from certain countrieswhere females have an even worsesituation may think differently; theyhave different life experiences soyou cannot stop it.

ECCV Daily: Did you ever feel thatkind of jealousy toward others?

Cheng: Maybe I can feel jealous, butI think when a male is jealous ofanother male, they will just thinkthat they are lucky or it was theright time for him. This can happenbetween people. When it is a maletoward a female, the reason can bea bit nastier. I find jealousy issomething that happens all of thetime whether it’s milder or becauseit’s unfair because she is female.

ECCV Daily: Do you think thatpeople feel that way because youare female?

Cheng: I think people in generalhave jealousy toward people whoget a good position, but when it’stoward a female, the jealousy isstronger.

ECCV Daily: When your teacher toldyou not to think about, did it help?

Cheng: Yes, it helped because theyall shared their experiences. Theyhad similar experiences before aswell.

ECCV Daily: No one succeeded in

hurting you?

Cheng: Maybe for a day or two, butit went away pretty fast. It happens.It’s a bad day. It’s not a bad life.

ECCV Daily: Aside from that, haveyou ever met any obstacles as afemale?

Cheng: I wouldn’t say obstaclesbecause people in my labs like me. Idon’t feel like that’s an actualobstacle. It’s more of a mentalobstacle so it depends on how I dealwith it.

ECCV Daily: Did it prevent you fromdoing something or were you alwaysable to do it your way?

Cheng: I think for me I always do itmy way. I also know people whoreally care about what other peoplethink, and try to be moreconservative in some way.

Women in Computer Vision 15ECCV Daily : Fr iday

ECCV Daily: Have you seen otherwomen dealing with this sort ofissue without being able toovercome it?

Cheng: People have similar feelings,but I don’t see any females aroundme being stopped by this.

ECCV Daily: Have you felt that youwished to be a man so this wouldn’thappen to you?

Cheng: No, it never happened tome. I’m happy to be a woman. Ifyou are jealous, it’s your problem,not mine.

ECCV Daily: What do you want toachieve?

Cheng: As I said in the beginning, Iwant to impact the world in apositive way. I also like technology. Iask, why do we have technology? Tohelp humans reach a higherstandard of living. That’s my goal, tocontribute to science andtechnology in a way that can makepeople happier and healthier.

ECCV Daily: Most of thesetechnologies help those who arealready happy and healthy, andthere is a big imbalance betweenthose who are happy and healthyand those who are not.

Cheng: No, I don’t think so becauseI’m one of the people who actuallythink about it. If you come to myposter this afternoon, I have apaper here which is solvingcomputer vision problems. I made amodel. I talked to doctors in clinicsand asked them about what theydo. They explained the problemsthat their patients have by showing

me drawings of the human body.That’s an image, right? A veryexperienced doctor can inspect thepatient and draw some conclusions;maybe you have a nerve infection,etc. Then you have text data ordescriptions. Combining images andtexts help doctors solve medicalproblems, for example.

That’s how I can apply my model tohelp people so a doctor can makebetter recommendations on theproblems that their patients have byreferring to the drawing. It’s arecommendation system for thedoctor. In that way I can helppeople.

In Sweden, you have to wait a longtime to meet a doctor. You have toschedule early in advance and if youhave small problems, it can getdelayed. In this way, a lot of peoplewho have problems can get helpfaster.

16ECCV Daily : Fr iday

Women in Computer Vision

ECCV Daily: You are incrediblyconfident. I always hear fromwomen that they feel inadequate.Do you have this problem?

Cheng: There are always days likethat. I don’t think that it’s aparticular problem for women.Sometimes, things don’t work thefirst time. That happens, and I mightfeel stupid, but it happens. We allhave those days.

ECCV Daily: Doesn’t it consume youinside?

Cheng: There are definitely ups anddowns. I think in the end, most ofthe time I overcome it. There aretimes when I doubt whether it’s theright thing to do or not, but I keepgoing.

ECCV Daily: Have you seen womenwho are not as confident as you thatyou would like to advise? Maybethey seem nervous when they do apresentation. How do you feel whenyou see that?

Cheng: First of all, I am nervous aswell when I do a presentation,although it’s not as though I startshaking. I did have friends who didnot feel comfortable givingpresentations. I talked with them. InSweden, it’s very open to talk withprofessors and other people, but noteveryone that comes from otherplaces and cultures feel comfortabletalking. So in the lab, I invite themfor lunch or for a coffee to talk. Ifthey ask my opinion, I am honestand say what I feel.

ECCV Daily: If you had not become ascientist, what would you havebecome?

Cheng: I have wanted to be a lot ofthings [she laughs again]. When Iwas 3 years old, I wanted to work ina zoo because I liked animals.

ECCV Daily: Which animals?

Cheng: Cats, dogs… I also like tigers,lions… [she laughs]

Then when I got a bit older, I learnedpainting and drawing. I was thinkingto become a fashion designer. Since Igrew so tall, I thought to become amodel.

ECCV Daily: Did you ever model?

Cheng: Not professionally, butsometimes friends tookphotographs. I’ve never modeled ina show, but for friend’s pictures.

ECCV Daily: Do you have no regretsof not becoming a painter, a model,or working a zoo?

Cheng: All these things are cool. Istill draw for fun. I think being ascientist is even cooler. I’m happywith what I do. My hobby and mywork are the same thing. How coolis that? As far as the other things Ilike, I can do them too. No one isstopping me, right?

Women in Computer Vision 17ECCV Daily : Fr iday

“My hobby

and my work are the same

thing. How cool is that? ”

Deep networks have excelled on avariety of vision tasks, but they aregenerally opaque and do notprovide any insight as to why aparticular output is appropriate.Unlike opaque systems, systemsthat can explain their output aremore likely to be trusted by a user[1]. In their work GeneratingVisual Explanations Lisa Anne

Hendricks and her team propose amethod to output justification textto help explain a deep network'soutput. The visual explanationsgenerated by their model describeelements in an image which areclass discriminative, and thus helpus understand why a particularclassification decision isappropriate for a particular image.

Generating Visual Explanations

18E CCV Da i l y : Fr i da y

Presentations

Lisa Anne Hendricks is a third year graduate student at UC Berkeley studyingcomputer vision with Professor Trevor Darrell. Before, she was anundergraduate at Rice University. Lisa Anne was already our guest at the latestCVPR, in Las Vegas, where she told us about Deep Compositional Captioning.

Presentations 19ECCV Daily : Fr iday

In order to generate explanations, Lisa Anne's team propose a new lossfunction for training text generation models. Instead of simplyminimizing cross entropy loss between ground truth and predictedwords as is usually done when training text generation models, theyinclude a "discriminative" loss. During training, sentences are sampledand given a reward based on how class discriminative they are.

More class discriminative sentences are given a higher reward. Becausethe loss operates over sampled sentences, REINFORCE [2] is employedto backpropagate through the sampling mechanism.

Results on a fine-grained bird classification dataset demonstrate theeffectiveness of the proposed approach. The explanation model isevaluated on a variety of metrics and is shown to outperform otherbaselines, such as a normal description model. Qualitatively, thegenerated explanations discuss

the generated explanations discuss more class discriminative attributesthan descriptions. For example, in the image below, a descriptionmodel mentions attributes which are common across bird classes (e.g.,"black"), whereas the explanation model mentions attributes which arespecific to the White Necked Raven (e.g., "white nape").

As the community continues to use deep networks, providingexplanations for network predictions is becoming more important. LisaAnne and her team envision that future explanation models will providemore insights into the exact mechanism of deep networks and will beimportant for the adoption of sophisticated AI systems.

[1] Biran, O., McKeown, K.: Justification narratives for individualclassifications. In: Proceedings of the AutoML workshop at ICML 2014.

[2] Williams, R.J.: Simple statistical gradient-following algorithms forconnectionist reinforcement learning. Machine Learning (1992)

We have visited the booth of Boschand they told us that even thoughpeople normally associate Boschwith washing machines or powertools, it is actually one of the largestcar suppliers in the world. Boschattended ECCV to show what otherexciting computer solutions theywork on, while also getting to knowthe community better, and allowingthe community to know thembetter as well.

As the biggest car supplier, Boschbuilds cameras and topic sensorsfor the automotive industry, Eventhough they have not publicize a lotabout it, they are one of the leadingcompanies in that area. The Boschresearch team uses deep learningtechniques combined with classicalmethods.

After attending the event, theyrealized that many people didn’trealize that Bosch does so muchwork with computer vision. Thefeedback helped the companyunderstand how to better promote

their computer vision work in thefuture. In response, attendees ofthe event are excited about Bosch’smany projects, such as automateddriving and its other computervision applications.

The people from Bosch wereexcited to see so many bright,young scientists together withsenior experts presenting theirinnovative ideas on new topics. Theconference gives them a greatopportunity to meet researchers inthe field, both young and senior,and help offer them exciting, newprojects for the future. They alsohope to recruit new experts to thecompany.

ECCV featured many greatdiscussions, especially with youngresearchers. The Bosch team hereat ECCV 2016 were impressed bythe kind of impact that deeplearning has had these days andwith the number of papers showingstunning results from deep learning.

20E CCV Da i l y : Fr i da y

Bosch

On this project work Simone Meyer,PhD student on a collaborationproject between ETH and DisneyResearch, Alexander Sorkine-Hornung, a Senior ResearchScientist of Disney Research, and Dr.Markus Gross, a Computer Scienceprofessor at ETH Zurich. AtECCV2016, they presented Phase-based Modification Transfer forVideo. The project modifies anentire video by editing the firstframe, proposing an efficientmethod to propagate thismodification from the first frameacross the whole video rather thandoing so manually on each frame.

The novelty of the project is in theapproach they are using. Thetraditional approach, for example,would be computing optical flowand transferring it. Instead, whatthey are doing is observing howeach pixel changes. The value theyobserve is the phase which doesn’tneed any explicit matching acrossthe frame.

The challenge is that it’s a newmethod which is not well exploredleading to unpredictable outcomes.It’s using an old method, but it’srevisited for new applications. It’salso a mathematical formulation.

To solve the challenge, the team hasexperimented by trying out newand different algorithms andcombining the information. Theonly information they have is phaseand amplitude and how to combinethe two for the application.Sideways, they have the amplitudeand the phase. The shift is themotion, but it can implicitlyrepresent the phase of this curve.For the algorithm, as a basis theyuse steerable pyramiddecomposition which gives thephase and amplitude value.Thereon, it’s its own algorithm.

The practical application is that itcan be used for any frame editing.It’s really suitable for high framerate video because it can handleonly tiny motion. It needs frameswith small motions in between. It’salso suited for high resolution databecause it works so fast.

Presentations 21ECCV Daily : Fr iday

“The practical application is that it can be used for any

frame editing”

Phase-based Modification Transfer for Video

In many machine-learningtasks, it is needed to representa sequence of variable length bya fixed-length vector. Forexample, representing a videowhich is a sequence of frames,or a sentence - a sequence ofwords. In this paper, the teamproposes the RNN Fisher Vectoras an effective representationfor sequences. Themethodology they use is basedon Fisher Vectors, where RNNsare the generative probabilisticmodels (instead of GMM whichis used in the traditional Fisher

Vector). The team shows thatthis representation can becomputed effectively usingbackpropagation, and reportstate-of-the-art results obtainedin two central but distant tasks:video action recognition andimage annotation. The projectis conducted by a teamcomposed by: Guy Lev, GilSadeh, Benjamin Klein and LiorWolf.

Visit their poster, it’s today(Friday) at 11:00-12:30 [posterP-4A-30 ]

RNN Fisher Vectors for Action Recognition and Image Annotation

22E CCV Da i l y : Fr i da y

Presentations