Importance of region-of-interest on image difference metricsMarius Pedersen

The Norwegian Color Research Laboratory

Faculty of Computer Science and Media Technology

Gjøvik University College, Gjøvik, Norway

Marius.pedersen@hig.no http://www.colorlab.noSupervisors: Jon Yngve Hardeberg and Peter Nussbaum

Thesis presentation, 7. June 2007, Gjøvik

2

Outline

Background Research questions Experimental setup

Psychophysical experiment Image difference metrics Region-of-interest Images Workflow

Results Questionnaire results How do we look at images? Image difference metrics

Conclusion

3

Background

When we print an image we want the output to be as close to the original as possible.

How perceivable are changes made to an image by the observers?

Image difference metrics have been developed to answer this question, their goal is to predict the perceived image difference.

The image difference metrics used today do not predict the perceived image difference very well.

When observers view an image some regions are more important than others.

4

Research questions

Question 1: - Can region-of-interest improve overall image difference metrics in complex images?

Question 2: - How do observers look at images given different tasks?

5

The experiment

A psychophysical experiment using 4 different scenes was carried out with 25 observers.

Using an eye tracker to record the gaze position of observers. 4 different image difference metrics

- ΔE*ab

- S-CIELAB- SSIM- iCAM

Different region-of-interest- Freeview- Psychophysical experiment- Gaze marking - Observer marked

6

Images

Changes to images made only in lightness. 4 global changes and 4 local changes. 3 and 5 ΔE*ab globally and 3 ΔE*ab locally.

7

Experiment workflow

Freeview task- Observers were told to look freely at the images.

Psychophysical experiment- Choose the image most similar to the original in a pair comparison task.

Gaze marking- Look at the regions important for your decision in the experiment.

Observer marking- Observers marked important regions on paper with a pen.

Questionnaire

8

Questionnaire results

25 observers ranging from 20 to 38 years, with a mean age of 24.

Recruited from the school 56% experts and 44% non-

experts. 24% had participated in

psychophysical experiments.

9

Psychophysical experiment results

Small global changes are rated better than higher global changes.

Overall results show that regions are rated generally better than global changes

Highly visible changes in small regions are given a low score.

10

How do we look at images

Difference between experts and non-experts when it comes to marking important areas. - Expert mark smaller and more precise areas.

Same observations made with observers with psychophysical experience.

Experts use longer time to evaluate difference.

11

How do we look at images

Region-of-interest change when observer are given different tasks.

2-D correlation coefficient used as a measure of similarity between groups and maps.

Freeview Psychophysical experiment Gaze marking Observer marking

12

Image difference metrics results

In the normal computation S-CIELAB, ΔE*aband the hue angle algorithm outperform SSIM and iCAM.

Pearson product-moment correlation coefficient used a measure of performance.

Scene 3 has a small but highly visible region, all metrics have problems here.

13

Area based image difference

In metrics performing well only a minor improvement is found.

While in metrics with a lower performance a bigger improvement is found.

Also the mean squared difference from the regression line supports the finding of improvement in the low performing metrics.

14

Conclusion

Q1: Can region-of-interest improve overall image difference metrics in complex images?

- Region-of-interest can improve overall image difference metrics, especially in metrics with a low performance.

Q2: How do observers look at images given different tasks?

- Observer have different region-of-interest in different tasks.* In a freeview task semantic regions as faces draw attention* In a pair comparison task attention is drawn toward other areas where the observer locates difference but faces still draw attention.* Gaze marking cannot replace region-of-interest marking by hand. * Manual marking only reflects some areas of the gaze.

15

Questions?

Thanks for your attention.