An automated image prescreening tool for a printer qualification process

by†Du-Yong Ng and ‡Jan P. Allebach

† Lexmark International Inc.‡ School of Electrical and Computer Engineering, Purdue University

Synopsis

Anatomy of a formatter-based EP laser printer Overview of a printer qualification process Motivation Examples of artifacts Image fidelity metrics – prior work Prescreening tool Experiment Results Conclusion

Anatomy of a formatter-based EP laser printer

A standalone network device

Print Engine

Formatter

Control Panel

Overview of a printer qualification process

Phase I The formatter (hardware + firmware) and the print engine are

developed in parallel. The hardware portion of the formatter is not ready. The firmware of formatter is tested using a simulator .

Test image (PDL)

Newly developedfirmware + simulator

Firmware of an earlierproduct + simulator

Master Current

Digital outputsmatch qualitatively?

Overview of a printer qualification process (cont.)

Phase II Preproduction print engines are either scarce or not available yet. Preproduction formatter hardware is available. The formatter (firmware + hardware) is tested with a print engine

emulator. Test image (PDL)

Newly developed formatter + print engine emulator

Formatter of an earlierproduct + print engine emulator

Digital outputsmatch qualitatively?

Master Current

Overview of a printer qualification process (cont.)

Phase III Preproduction printers (formatter + print engine) are available.

Test image (PDL)

Newly developed printerEarlier printer model

Hardcopiesmatch qualitatively?

Master Current

Overview of a printer qualification process - summary

Simulator without actual formatter/ emulator with actual formatter

Test image(PDL)

Softcopy current image

Comparison masters

Softcopy

Formatter/firmware

development teams

Debug

Error flagged

Actual formatter Hardcopy current imagePrint engine

HardcopyDebug

Motivations Image screening (comparison of master-current image pairs)

is mostly performed by trained observers. is needed for thousands of softcopy and hardcopy image pairs

throughout the printer development process. is very labor intensive. is often performed only on a fraction of test suites before the

product is rolled out due to» a relatively short development time.» a large number of test images in the test suites.

Our goal is to develop a automated tool to reduce the workload of trained

observers and increase the volume of tests the by screening out softcopy image pairs with

» highly objectionable errors (failed).» visually insignificant errors (passed).

Trained observers only need to focus on image pairs which could not be screened out by the tool (further evaluation)

Examples of master-current image pair

Master Current

Different halftone algorithms

Missing pixels

Master Current

Examples of master-current image pair (cont.)

Different in character size

Master

Current

Sporadic difference

CurrentMaster

Image fidelity metrics – prior work First category

Examples» Peak-signal-to-noise ratio (PSNR), root mean square error, and ∆Ea*b*

Pros» They are fast and easy to compute, and produce a single number.

Cons» Averaging effect destroys local information and spatial interaction of

pixels is ignored.

Second category Examples

» Wu’s color image fidelity assessor (HVS model) and structural similarity image metric (first and second order statistics the luminance channel of a local window)

Pros» Spatial processing model is included.

Cons» The algorithm is computational expensive and does not produce a

single number (Wu’s color fidelity assessor) » SSIM produces a single number but suffers from the averaging effect.

Prescreening tool - requirements needs to work reasonably fast. classifies the master-current pairs into one of the categories:

‘passed’, ‘failed’, and ‘further evaluation required’. must adapt automatically as the spatial resolution of the images

changes in dpi. has to be capable of processing any image content. needs to handle both halftone and continuous-tone images. will process full color, indexed color, grayscale, and bilevel

images consistently. will detect and ignore small spatial shifts in content and

differences in orientation

Master image Current image

PASSEDFAILED

The prescreening tool

FURTHER EVALUATION

13

Preprocessing

Compute 2D error map

Cluster error map

Compute error metric

Thresholding

Prescreening tool - preprocessing

Determine shift in content

Master Current

• Determine image type• Perform quick diff• Determine resolution• Correct orientation• Correct spatial shift

Other processing

Con

tent

size

Imag

e si

ze

Check dimensionRotated CWRotated CCW

Master Current Current

Check orientation

14

Prescreening tool – compute and cluster 2D error map (illustration)

Master M i

j

i

j

Current C

i

j

Binary Error Map

j

iClustered Error Map

Clustering

kth cluster

(k+1)th cluster

(k+2)th cluster

Prescreening tool – compute error metric (contrast sensitivity function CSF component)

Master i

j

i

j

Currentcorrespond to the lth pixel of the kth cluster

Mkl

Ckl

Average filtered pixel value for the kth cluster

Master :

Current:

Error for the kth cluster:

CSF error component for the image pair:

MMkll

kk N

1

CCkll

kk N

1

To CIE L*a*b*Mk

Ck

CMkk

CSFkbaE

**

k

CSFkbak

tot

CSFba EN

NE ****

1

Prescreening tool – compute error metric (contrast sensitivity function CSF component)

(cont.)

Prescreening tool – compute error metric (visual acuity filter VAF component)

Master i

j

i

j

Currentcorrespond to the lth pixel of the kth cluster

Mkl

Ckl

Average filtered pixel value for the kth cluster

Master :

Current:

Error for the kth cluster:

VAF error component for the image pair:

MMkll

kk N

1

CCkll

kk N

1

To CIE L*a*b*Mk

Ck

CMkk

VAFkbaE

**

k

VAFkbak

tot

VAFba EN

NE ****

1

Prescreening tool – compute error metric (visual acuity filter VAF component)

(cont.)

Prescreening tool – compute error metric (overall error)

Combined error

Error metric value for the image pair

maperror thein pixels of # total1 where

**

totf

NVAFCSFba

NN

E f

Ea*b*{ }CSF+VAF = ΔEa*b*{ }

CSF⎡⎣ ⎤⎦N p

+ ΔEa*b*{ }VAF⎡⎣ ⎤⎦

N p

{ }

1N p

where N p = 1+ 2 ⋅tanh(max( ΔEa*b*{ }CSF , ΔEa*b*{ }

VAF ))

Experiment There are 147 image pairs. Test image pairs are of

300, 600 and 1200 dpi. binary, grayscale, color bitmap and full color.

Six expert observers classify the image pairs into 3 categories Acceptable (passed). Need further evaluation. Objectionable (failed).

These image pairs are processed with our prescreening tool. The peak signal to noise ratio (PSNR) metric and structural similarity

image metric (SSIM) are also computed for each preprocessed (to ensure fair comparison) image pair.

21

Results – error metric 22

A zero error metric value indicates a perfect match. Decision thresholds exist to pass and fail image pairs. The prescreening tool is able to screen out as many as 35% of the image

pairs tested.

Results - PSNR A larger PSNR value (PSNR = ∞ for a perfect match) indicates a closer

match. Only decision thresholds to pass image pairs exist. The PSNR metric is able to screen out only as many as 4% of the image

pairs.

Results - SSIM 24 A larger SSIM value (SSIM = 1 for a perfect match) indicates a closer

match. Only decision thresholds to fail image pairs exist. The SSIM metric is able to screen out only as many as 3.4% of the

image pairs.

Conclusion

We have successfully developed an automated prescreening tool along with an image fidelity metric for a printer qualification process.

This tool works for a wide range of image types, content, and image resolutions.

It requires no training and it is able to reduce the workload of expert observers by a substantial amount.

25