18
A formulation for patenting content-based retrieval processes in digital libraries q Hideyasu Sasaki a,b, * , Yasushi Kiyoki a a Keio University, 5322 Endo, Fujisawa 252-8520, Japan b The Third Judicial Department, New York State Bar, Albany, NY, USA Available online 10 May 2004 Abstract In this paper, we present a formulation and case studies of the conditions for patenting content-based retrieval processes in digital libraries, especially in image libraries. Inventors and practitioners demand a formulation of the conditions for patenting data-processing processes as computer-related inventions in the form of computer programs. A process for content-based retrieval often consists of a combination of prior disclosed means and also comprises means for parameter setting that is adjusted to retrieve specific kinds of images in certain narrow domains. We focus on requirements for technical advancement (nonobviousness) in the combination of data-processing means, i.e., processes and specification (enablement) on the means for parameter setting in computer programs. Our formulation follows the standards of patent examination and litigation on computer-related inventions in the US. We confirm the feasibility and accountability of our formulation by applying it to several inventions patented in the US. Ó 2004 Elsevier Ltd. All rights reserved. Keywords: Multimedia digital libraries; Content-based image retrieval (CBIR); Patent law; Parameter setting; Intellectual property 1. Introduction In this paper, we present a formulation and case studies for patenting content-based retrieval processes in digital libraries, especially in image libraries (Sasaki & Kiyoki, 2002a, 2002b, 2001). A content-based retrieval system generates indexes to images by visual feature extraction and also classifies those generated indexes to perform a certain retrieval function. Data-processing means, i.e., processes, here for content- based retrieval, constitute computer-related inventions in the form of computer programs. Those inventions often combine prior disclosed means, and also comprise means for parameter setting as is adjusted to retrieve specific kinds of images in certain narrow domains. Inventors and practitioners demand a for- mulation of the conditions for patenting those data-processing processes. We formulate the conditions on patentability of processes for performing content-based retrieval in combining the prior disclosed means q An earlier version of this paper was presented at ICADL (Singapore, December 2002). * Corresponding author. Address: 1223-4 Minaminakano, Minuma, Saitama 337-0042, Japan. Tel.: +81-48-685-2467; fax: +81-466- 49-1245. E-mail addresses: [email protected] (H. Sasaki), [email protected] (Y. Kiyoki). 0306-4573/$ - see front matter Ó 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.ipm.2004.04.002 Information Processing and Management 41 (2005) 57–74 www.elsevier.com/locate/infoproman

A formulation for patenting content-based retrieval processes in digital libraries

Embed Size (px)

Citation preview

Information Processing and Management 41 (2005) 57–74

www.elsevier.com/locate/infoproman

A formulation for patenting content-based retrievalprocesses in digital libraries q

Hideyasu Sasaki a,b,*, Yasushi Kiyoki a

a Keio University, 5322 Endo, Fujisawa 252-8520, Japanb The Third Judicial Department, New York State Bar, Albany, NY, USA

Available online 10 May 2004

Abstract

In this paper, we present a formulation and case studies of the conditions for patenting content-based retrieval

processes in digital libraries, especially in image libraries. Inventors and practitioners demand a formulation of the

conditions for patenting data-processing processes as computer-related inventions in the form of computer programs. A

process for content-based retrieval often consists of a combination of prior disclosed means and also comprises means

for parameter setting that is adjusted to retrieve specific kinds of images in certain narrow domains. We focus on

requirements for technical advancement (nonobviousness) in the combination of data-processing means, i.e., processes

and specification (enablement) on the means for parameter setting in computer programs. Our formulation follows the

standards of patent examination and litigation on computer-related inventions in the US. We confirm the feasibility and

accountability of our formulation by applying it to several inventions patented in the US.

� 2004 Elsevier Ltd. All rights reserved.

Keywords: Multimedia digital libraries; Content-based image retrieval (CBIR); Patent law; Parameter setting; Intellectual property

1. Introduction

In this paper, we present a formulation and case studies for patenting content-based retrieval processes

in digital libraries, especially in image libraries (Sasaki & Kiyoki, 2002a, 2002b, 2001). A content-based

retrieval system generates indexes to images by visual feature extraction and also classifies those generated

indexes to perform a certain retrieval function. Data-processing means, i.e., processes, here for content-

based retrieval, constitute computer-related inventions in the form of computer programs. Those inventions

often combine prior disclosed means, and also comprise means for parameter setting as is adjusted to

retrieve specific kinds of images in certain narrow domains. Inventors and practitioners demand a for-mulation of the conditions for patenting those data-processing processes. We formulate the conditions on

patentability of processes for performing content-based retrieval in combining the prior disclosed means

qAn earlier version of this paper was presented at ICADL (Singapore, December 2002).* Corresponding author. Address: 1223-4 Minaminakano, Minuma, Saitama 337-0042, Japan. Tel.: +81-48-685-2467; fax: +81-466-

49-1245.

E-mail addresses: [email protected] (H. Sasaki), [email protected] (Y. Kiyoki).

0306-4573/$ - see front matter � 2004 Elsevier Ltd. All rights reserved.doi:10.1016/j.ipm.2004.04.002

58 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

and/or comprising the means for parameter setting from the practical standpoints of technical advancement

(nonobviousness) and specification (enablement). Our formulation follows the standards of patent exam-

ination and litigation on computer-related inventions in the US. We confirm the feasibility and account-

ability of our formulation by applying it to several inventions patented in the US.

1.1. Motivation: parameter setting components for content-based retrieval

Memory storage capacity has expanded tremendously to enable large-scale storage of image data indigital libraries. The digital library community demands an automatic and scalable solution for retrieval,

especially in image libraries. After elaboration on keyword-based (or text-based) image retrieval, content-

based image retrieval (CBIR) is introduced to the digital library community. CBIR is appreciated for its

automatic indexing to digital contents stored in multimedia digital libraries. CBIR also promotes a new

technology, i.e., ‘‘domain-specific approach’’, which is a type of content-based retrieval focused and

optimized to specific target domains, e.g., a field of brain images. In the domain-specific approach, a search

engine classifies images stored in a digital library by its thresholding function that evaluates structural

similarity of visual features of those stored images.A component for ‘‘parameter setting’’ takes the form of a computer program and realizes the thres-

holding function with a set of ranges of parametric values. In CBIR, those parametric values determine as

thresholds which candidate image is similar to an exemplary requested image by computation of similarity

of visual features. A set of visual features of structural similarity has its corresponding class of images of

semantic similarity in certain specific domains.

The parameter setting component is familiar in mechanical inventions. Its typical example is a patented

thermostat invention for crude petroleum purification, which automatically monitors and fixes crude oil

temperature at a certain range under its explosive point. Another exemplary patent is a thermostatic devicein an automatic temperature controller in air cargo, which adjusts the body temperature of raw fish inside

the cargo in the proper range so they do not get frozen but keep chilled during the course of transportation.

The parameter setting components in computer-related inventions define the scope of patent enforcement

on computer programs with clear boundary.

1.2. Background: content-based image retrieval as patentable computer-related invention

A keyword-based retrieval system provides images stored in a digital library with pre-fixed index terms.

Its search engine performs keyword-based retrieval by pattern matching of those index terms pre-fixed to

candidate images with a certain requested keyword. A keyword-based (or text-based) retrieval system is

often implemented in current Web search engines, e.g., Google (Brin & Page, 1998, Brin, 1998).A content-based retrieval system extracts visual features out of images and generates indexes out of those

extracted features, classifies those generated indexes based on similarity computation and finds certain

images, which are similar to a certain requested image, from candidate images stored in a digital library.

Fig. 1 outlines its data-processing in a hypothetical example of geometric figure retrieval. Its pre-processing

step extracts each feature of the images, e.g., color, shape, edge, contour, etc. All images are converted to a

set of several (four, in the example) binary data values representing visual features extracted out of images.

Images are then indexed by grouping those extracted features. The example indexes in the figure take the

form of decimal data. Its final processing step dynamically classifies those automatically generated indexesby computing correlation of the values of those indexes every time users request a search. The retrieved

images as data-processing results are ranked based on the structural similarity of their classified indexes.

The structural similarity represents semantic similarity of those retrieval results in the certain selected

domain.

DL

FEATUREEXTRACTION

INDEXING

CLASSIFICATION

1stRank

2ndRank

RETRIEVALRESULTS

EXTRACTEDFEATURES

09050311 INDEXES0905010904050800

1001010100111011

0100010110000000

1001010100011001

TARGET IMAGE ORSKETCH BY USERSCANDIDATE IMAGES

Fig. 1. Data-processing outline of content-based image retrieval.

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 59

CBIR has two types of retrieval approach in its applied domains: the domain-general approach and the

domain-specific approach (Smeulders, Worring, Santini, Gupta, & Jain, 2000, Rui, Huang, & Chang,

1999). The domain-general approach is applied to broad domains that include various kinds of visual

features extracted out of images. Its typical implementation includes Virage Image Retrieval (Bach et al.,1996): US Pat. # 5,893,095 invented by Jain et al. (1999), QBIC (Flickner, Sawhney, & Niblack, 1995): US

Pat. # 5,647,058 invented by Agrawal, Equitz, Faloutsos, Flickner, and Swami (1997), WebSeek (Smith &

Chang, 1997), and also, VisualSEEK (Smith & Chang, 1996). The domain-specific approach is applied to

narrow domains in which several specific visual features extracted out of images represent the semantics of

the target domains. Its typical implementation includes medical image retrieval systems, e.g., US Pat. #

6,125,194 invented by Yeh, Lure, and Lin (2000) and fingerprint image retrieval systems, e.g., US Pat. #

6,356,649 invented by Harkless, Potter, Monro, and Thebaud (2002).

Broad domains describe ‘‘their semantics only partially’’, though specific domains restrict their ‘‘vari-ability’’ on the extracted features in the ‘‘limited and predictable’’ scope (Smeulders et al., 2000). Fig. 2

outlines the data-processing of the domain-specific approach using a hypothetical example of starfish image

retrieval. Its pre-processing step is almost the same as the data-processing outlined in Fig. 1, but for the

processing of restricted selection of features and classification comprising the means for ‘‘parameter set-

ting’’, i.e., the means for selecting and/or adjusting parametric values on operating parameters. Its classi-

fication step determines whether the classified indexes fall within certain pre-defined ranges of parametric

values on the parameter setting (e.g., for similarity computation of the indexes). The hypothetical ranges of

values take ±125 in the double-lined box of Fig. 2 as approximated ranges of parametric values forthresholding of each generated index in the form of decimal data.

1.3. Issues: requirements for patentability of content-based retrieval processes

CBIR realizes its retrieval function by performing a number of ‘‘processes’’, i.e., methods or means

embodied in computer-related inventions that take the form of computer programs. Those processes for

CBIR consist of combinations of data-processing means, some of which are prior disclosed inventions.

Also, those processes comprise the means for parameter setting that is adjusted to retrieve specific kinds of

images in certain narrow domains as in the domain-specific approach. The domain-specific approach

implements its best mode by selecting parametric values, such as weights, on parameter setting that indexand classify the extracted features in certain specific narrow domains.

DL

FEATUREEXTRACTION

INDEXING

CLASSIFICATION

EXTRACTEDFEATURES

±±±± 0001±±±± 0011±±±± 0111±±±± 0011

PREDEFINEDRANGES OF VALUES(on parameter setting)

09050311 INDEXES09050109

±±±±125

04050800

RETRIEVALRESULT

1001010100111011

0100010110000000

1001010100011001

TARGET IMAGE OF STARFISHOR SKETCH BY USERSCANDIDATE IMAGES

Fig. 2. Data-processing of the domain-specific approach of CBIR.

60 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

Inventors and practitioners demand a detailed formulation of the conditions for patenting those data-

processing processes, here, content-based retrieval processes in digital libraries, as computer-related

inventions in the form of computer programs as combinations of processes with parameter setting com-

ponents.

The scope of this paper is, nonetheless, limited to several requirements for patentability: nonobviousness

(technical advancement) on a combination of prior disclosed computer programs and enablement (speci-

fication) on parameter setting components in the domain-specific approach of CBIR.The rest of this paper is organized as follows. In Section 2, we formulate the conditions on patentability.

In Section 3, we provide case studies to confirm the feasibility of the formulated conditions. In Sections 4

and 5, we conclude with discussions on our formulation.

2. Conditions on patentability

In Section 2, we formulate the conditions on patentability of processes for content-based retrieval in

digital libraries, i.e., the processes for performing CBIR functions as computer-related inventions in the

form of computer programs in combining prior disclosed means, and also in comprising the means for

parameter setting.Those formulated conditions are described in the block diagram of Fig. 3 with the list of conditions in

Table 1. The double-lined-boxes in the block diagram represent the most critical conditions concerning the

combination of means and the parameter setting components. Our formulation follows the standards of

patent examination and litigation in the US practice. We provide the formulation with a simulation

example, i.e., ‘‘Virage Image Retrieval’’.

2.1. Procedural diagrams of the formulated conditions

The formulated conditions consist of the following three requirements for patentability: ‘‘patentable

subject matter’’ (entrance to patent protection), ‘‘nonobviousness’’ (technical advancement) and ‘‘enable-

ment’’ (specification). Patentable subject matter is ‘‘the issue of which types of inventions will be consideredfor patent protection’’. Nonobviousness is ‘‘the technical advancement reflected in an invention’’. In

I. PATENTABLE SUBJECT MATTER OFPROCESSES FOR CBIR FUNCTIONS

DO CLAIMS COMPRISE THE MEANS FORPARAMETER SETTING ?

yesno

PATENTABLE NOT AS CBIR INVENTIONS,BUT AS THE PROCESSES RELATED TOGENERAL RETRIEVAL FUNCTIONS (out)

II. NONOBVIOUSNESS(TECHNICAL ADVANCEMENT)

(1) DO THE PRIOR ARTS PREDICATE ACOMBINATION OF THE MEANS FORPERFORMING THE INSTANT FUNCTIONALPROCESS OF CBIR ?

(2) DO THE FUNCTION REALIZEQUANTITATIVE AND/OR QUALITATIVEADVANCEMENT ?

no

yes

yes

no

OBVIOUS(NOTPATENT-ABLE)

III. ENABLEMENT(CLEAR SPECIFICATION)

(1-a) DO THE DESCRIPTIONS SPECIFY THEFORMULAS FOR PARAMETER SETTING ?

yes no

(2-b) DO THE DESCRIPTIONS DISCLOSE EXAM-PLES OF VALUES ON PARAMETER SETTING ?

(I) WORKING OR PROPHETIC EXAMPLESOF INITIAL VALUES OR WEIGHTS ONPARAMETER SETTING

(ii) WORKING EXAMPLES OF THE RANGEOF VALUES ON PARAMETER SETTING

yes

DO THE PROCESSES HAVE ANY IMPROVED FORMULASFOR PARAMETER SETTING BASED ON PRIORDISCLOSED MEANS FOR CBIR ?

(1-b) DOES THE DISCUSSED INVENTIONHAVE ANY CO-PENDING APPLICATIONTHAT SPECIFIES THE ABOVE FORMULAS ?

no

(out)

yes

yes

(2-a) DO THE PROCESSES REALIZE ANEW FUNCTION BY COMBINING THEPRIOR DISCLOSED MEANS ?

nono

(out)

yes

no

(out)

PATENTABLE AS A DOMAIN-GENERAL APPROACH OF CBIR

PATENTABLE AS A DOMAIN-SPECIFIC APPROACH OF CBIR

Fig. 3. A block diagram for the formulated conditions on patentability of the processes for performing CBIR functions.

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 61

general, nonobviousness is the ultimate condition on patentability, though enablement has received

increasing attention as ‘‘the technical specification requirement’’ since the late eighties of the previous

century in the US practice (Merges, 1997).

2.1.1. Requirement for patentable subject matter in the form of computer programs

The processes for performing CBIR functions are patentable subject matter when their patent appli-cations claim the means for parameter setting which perform certain retrieval functions.

Otherwise, the discussed processes are considered not as specific inventions of the data-processing

processes for performing CBIR functions but as the inventions that are peripheral or just related to general

retrieval functions.

In the US practice, a data-processing process or method is patentable subject matter in the form of a

computer-related invention, i.e., a computer program (US Patent Act, 2003, Jakes & Yoches, 1989) as a

‘‘means-plus-function’’ (In re Trovato, 1995) as long as the ‘‘specific machine . . . produce(s) a useful,concrete, and tangible result . . . for transforming . . .’’ physical data (‘‘physical transformation’’) (In reAlappat, 1994). A process must ‘‘perform independent physical acts (post-computer process activities)’’,

otherwise, ‘‘manipulate data representing physical objects or activities to achieve a practical application

(pre-computer process activities)’’. A process must not ‘‘merely manipulate abstract idea or solve a purely

Table 1

A list of the formulated conditions on patentability of the processes for performing CBIR functions

Box Conditions Determinations: Go to

Yes No

I Patentable subject matter of processes for CBIR functions

(1) Does its patent application claim the means for parameter

setting?

Go to II Go to I (2)

(2) Patentable or not as an invention related to general retrieval

functions

Out of the block diagram

II Nonobviousness (technical advancement)

(1) Do the prior arts predicate the instant combination of prior

disclosed means for performing the instant functional process?

Not patentable Go to II (2)

(2) Does the function realize quantitative and/or qualitative

advancement?

Go to III (1-a) Not patentable (out)

III Enablement (clear specification)

(1-a) Do the descriptions of the instant process specify the formulas

for parameter setting?

Go to III (2) Go to III (1-b)

(1-b) Does the discussed patent have any co-pending application that

specifies the above formulas?

Go to III (2) Not patentable (out)

(2) Does the process have any improved formulas for parameter

setting?

Go to III (2-b) Go to III (2-a)

(2-a) Does the process realize a new function by combination of the

prior disclosed means?

Patentable as a domain-

general approach of CBIR

Not patentable (out)

(2-b) Do the descriptions of the process give examples of the values

on parameter setting in the descriptions?

Patentable as a domain-

specific approach of CBIR

Not patentable (out)

Box numbers correspond to the previous block diagram.

62 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

mathematical problem without any limitation to a practical application’’ (US Patent & Trademark Office,

1996a, 1996b).

The processes for performing CBIR functions are patentable subject matter because they comprise the

means for parameter setting and those means perform physical transformation of data, i.e., image retrieval

processing. CBIR functions generate indexes as physical results on a computer and a memory, and also

require pre- and post-computer process activities through data processing between feature extraction andindexing, also between indexing and classification, as indispensable procedure. In the case of no claiming of

the means for parameter setting, those processes do not realize any specific advancement on content-based

retrieval functions but facilitate certain functions that are related or just peripheral to data processing of

general retrieval functions.

Finally, the US practice demands that inventions be of ‘‘technological arts’’. That requirement does not

limit patentability of computer-related inventions because technological arts are fully equivalent to, in a

broad sense, the concept of useful or practical arts (Merges, 1997).

2.1.2. Requirement for nonobviousness on data-processing means of combination

The processes for performing CBIR functions are nonobvious from the prior arts when they

(1) comprise combinations of prior disclosed means to perform certain functions that are not predicated

from any combination of the prior arts, in addition,

(2) realize quantitative and/or qualitative advancement.

Otherwise, the discussed processes are obvious so that they are not patentable as the processes for per-forming CBIR functions.

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 63

First, a combination of prior disclosed means should not be ‘‘suggested’’ from the disclosed means ‘‘with

the reasonable expectation of success’’ in the US practice (In re Dow Chemical Co., 1988). Second, its

asserted function must be superior to conventional functions that are realized in the prior disclosed or

patented means. On the latter issue, several solutions for performance evaluation are applicable. Studies(M€uller, M€uller, Squire, Marchand-Maillet, & Pun, 2001, Manchester Visualization Center, 2003) proposebenchmarking of CBIR functions. The patent application for US Pat. # 6,259,809 invented by Maruo

(2001) reports a suggestive solution for performance evaluation by comparing the computational order of

its instant inventive process with one of a conventional disclosed process. Another general strategy is

restriction of the scope of claims to a certain narrow field to which no prior arts have been applied. This

claiming strategy is a kind of local optimization of application scope.

2.1.3. Requirement for enablement on parameter setting components

The processes for performing CBIR functions must be described clearly enough to enable those skilled in

the arts to implement the best mode of those claimed processes by satisfying the following conditions:

(1-a) The descriptions of the processes must specify the formulas for parameter setting. Otherwise,

(1-b) the disclosed inventions should have its co-pending application that describes the formulas indetail. In addition,

(2-a) the processes must perform a new function by a combination of the prior disclosed means.

Otherwise,

(2-b) the processes should have the improved formulas for parameter setting based on the prior disclosed

means for performing CBIR functions and also give examples of the values on the parameter setting in the

descriptions.

1-a and 1-b determine whether the discussed processes are patentable combinations of the prior disclosed

means for performing CBIR functions. 2-a determines whether the discussed processes are patentable as adomain-general approach. 2-b determines whether the discussed processes are patentable as a domain-

specific approach when the processes specify the improved formulas for parameter setting based on the

previous disclosed means.

For 2-b, the processes must specify the means for parameter setting by ‘‘giving a specific example of

preparing an’’ application (Autogiro Co. of America v. United States, 1967, Unique Concepts, Inc. v.

Brown, 1991) to enable those skilled in the arts to implement their best mode of those processes without

undue experiment. The US Patent and Trademark Office (USPTO) suggests in its Guidelines and Training

Materials that a process comprising the means for parameter setting should comply with the requirementfor enablement by giving examples of specific values on parameter setting (US Patent & Trademark Office,

1996a, 1996b).

The processes comprising the means for parameter setting must disclose at least one of the following

examples of values on parameter setting:

(i) Working or prophetic examples of initial values or weights on the parameter setting;

(ii) Working examples of the ranges of values on the parameter setting.

The ‘‘working examples’’ are parametric values that are confirmed to work in the laboratory or as

prototype testing results (In re Strahilevitz, 1982). The USPTO has also accepted in practice so-called

‘‘prophetic examples’’, especially in the area of biotechnology after the decision on In re Strahilevitz (1982).

Those prophetic examples are given without actual work by one skilled in the art. Fig. 4 describes the

hypothetical example that provides the example range of parametric values, here weights on parameter

setting, with, e.g., ±125. Its active range of values, e.g. ±100, falls within the example range of parametric

values.

Main {int xxx ; …

.

.

}

USER REQUEST BY TARGETIMAGE OF STARFISH

1050 >xxx > 950

CBIR FUNCTIONS

within the example range

DL

FEATUREEXTRACTION

INDEXING

CLASSIFICATION

±±±± 0001±±±± 0011±±±± 0111±±±± 0011

EXAMPLE RANGE OFPARAMETRIC VALUES

(on parameter setting)

±125

±±±±100THE ACTIVERANGE OF

PARAMETRICVALUES

±

Fig. 4. A conceptual description of parameter setting components.

64 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

The parameter setting components defines the scope of patent enforcement of computer programs as

process patents. It is a critical problem to define the scope of equivalent modification of process patents. Its

typical case is found in the computer programs comprising parameter setting components. The scope of

patent enforcement extends to the equivalents of the structure of means-plus-function claims as far as they

are predictable from those claims and descriptions (Pennwalt Corp. v. Durand-Wayland, Inc., 1988, GraverTank&Mfg. Co. v. Linde Air Products Co., 1950, Laitran Corp. v. Rexnord, Inc., 1991). Parametric values

in parameter setting are easy to modify and adjust at application. A process for performing CBIR functions

must specify its parameter setting by giving exemplary parametric values in order to define the clear

boundary of its scope of equivalent modification. Especially, the domain-specific approach for performing

CBIR functions must distinguish a claimed invention with examples of parametric values from other im-

proved formulas for parameter setting which are based on prior disclosed means. Those examples define the

scope of the equivalent modification of a patented process within a certain specified scope as suggested from

those exemplary parametric values.

2.2. A simulation example for the formulated procedural diagram

‘‘Virage Image Retrieval’’ (VIR) was developed in the early 1990s as a typical content-based retrieval of

visual objects stored in digital image libraries. VIR is an indexing method for an image search engine with

‘‘primitives’’, which compute similarity of visual features extracted out of typical visual objects, e.g., color,

shape and texture of images. VIR evaluates similarity of images with ad hoc weights, i.e., parametric values,which are given to the parameter setting components for correlation-computation, by user-preference. Its

claims consist of ‘‘function containers’’ as means-plus-functions for feature extraction and similarity

computation. Its first claim, as described below, constitutes the primitives as the means-plus-functions.

Those primitives realize a domain-general approach of CBIR by the formulas on parameter setting with

‘‘weighted sums’’.

VIR Claim # 1. A search engine, comprising: a function container capable of storing primitive functions; . . .a primitive supplying primitive functions . . . . . ., wherein the primitive functions include an analysis function. . . . . . of extracting features from an object. . . .

I. PATENTABLE SUBJECT MATTER

DO CLAIMS COMPRISE THE MEANSFOR PARAMETER SETTING ?

II. NONOBVIOUSNESS (TECHNICAL ADVANCEMENT)

(1) DO THE PRIOR ARTS PREDICATE THEINSTANT COMBINATION OF THE MEANS ?

(2) DO THE FUNCTION REALIZEQUANTITATIVE AND/OR QUALITATIVEADVANCEMENT ?

Claimed components generate pre/post-computeractivities on computer memories.

YES

Claimed functions go beyond a merecombination of conventional processesand databases.

NO

Claimed functions bring in a brand newintegration of visual feature extractionand automatic indexing with similarityanalysis of images.

(1-a) DO THE DESCRIPTIONS SPECIFY THEFORMULAS FOR PARAMETER SETTING ?

YES

DO THE PROCESSES HAVE ANY IMPROVEDFORMULAS FOR PARAMETER SETTING ?

(2-a) DO THE PROCESSES REALIZE ANY NEWFUNCTION BY COMBINING THE PRIORDISCLOSED MEANS ?

NO

PATENTABLE AS A DOMAIN-GENERAL APPROACH

YES

III. ENABLEMENT (SPECIFICATION)

Claimed components disclose theirsimilarity metrics in the form ofequations.

Claimed processes introduce thedomain-general approach of CBIR.

Claimed inventions discloses itsinnovative functions as featurerepresentation, extraction, andcomparison function.

Fig. 5. A simulation example (VIR) applied to the formulation.

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 65

Fig. 5 outlines the application of our formulation to the VIR invention. First, on its patentable subject

matter, its retrieval processes consisting of the formula for parameter setting are to be determined as

patentable subject matter in the form of computer programs. Those data-processing processes generate

physical transformation on a specific machine, i.e., a computer memory with certain classification results.

Second, on its nonobviousness, those data-processing processes are inventive steps that consist of combi-

nations of the prior arts on thresholding functions as implemented in the integration of classification based

on similarity computation, visual feature extraction and automatic indexing techniques. Those combina-

tions are not predicated from any conventional keyword-based retrieval technique. Third, on its enable-ment, VIR’s description of preferred embodiments gives its clear specification on the formulas for

parameter setting that realizes a domain-general approach of CBIR that was a brand new technology at the

time.

VIR Description. . . .. . .For primitives having multiple dimensions, . . .. . ., An equation for an exemplaryEuclidean metric is as follows:

1 In

Street

66 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

sj ¼Xj

ðfeature vector1ðjÞ

� feature vector2ðjÞÞ2!1=2

Primitive design. A primitive encompasses a given feature’s representation, extraction, and comparison

function. . . .. . . The constraints are as follows:Primitives, in general, map to cognitively relevant image properties of the given domain.

The formulation should take advantage of a threshold parameter (when available), . . .. . ..

3. Case study

In the rest of Section 3, we confirm the feasibility and accountability of the formulated conditions on

patentability of data-processing processes for performing CBIR functions in digital libraries by applying

the formulated conditions to several instances of patents disclosed in the US.

The ‘‘USPTO Patent Search Database’’ (US Patent & Trademark Office, 2003) responded to search

terms ‘‘image(s)’’, ‘‘retriev(al, e, es, ed, ing)’’ and ‘‘database(s)’’ with 385 disclosed patents betweenSeptember 5th, 2000 1 and May 28th, 2002. Those search terms are wide enough not to miss inventions

performing CBIR functions but described in terms of pattern recognition. Table 2 lists eight patents that we

selected from the above 385 patents for case study by screening out the other peripheral data-processing

processes that do not perform CBIR functions. The first case in the list is a typical one of the latter

peripheral inventions for general retrieval functions. Among the remaining seven cases, five cases take the

domain-general approach while the remaining two cases take the domain-specific approach.

3.1. Patentable subject matter cases

Seven patents claim the means for parameter setting in their processes as patentable subject matter of

technological arts. The remaining single case is an exception that is not patentable as a process for per-

forming content-based retrieval functions. It is an invention that is just peripheral or related to general

retrieval functions. US Pat. # 6,253,201 invented by Abdel-Mottaleb and Wu (2001) realizes a data-pro-

cessing process for listing the pre-indexed image identifiers, which are based on the prior arts of color and/

or edge detection for performing fast classification, as described in its first claim. That invention is

applicable to any process for general retrieval operations so that it is not patentable as an essential processfor performing content-based retrieval functions.

Claim # 1 (Abdel-Mottaleb and Wu (2001)). A method of image retrieval, comprising the steps of:

partitioning a target image into a plurality of content-independent partitions, characterizing each parti-

tion of the plurality of content-independent partitions to form an index value associated with each par-

tition, obtaining a list of image identifiers associated with the index value, accumulating counts of each

image identifier in the list of image identifiers associated with each partition of the plurality of content-

independent partitions,

and retrieving at least one image associated with at least one of the image identifiers, based upon the

counts of the at least one of the image identifiers. . . .. . .

the year of 2000, the USPTO changed its practice on computer-related inventions after the US Supreme Court decision on State

Bank & Trust Co. v. Signature Financial Group, Inc. (1998).

Table 2

A list of applied patents for case study

Classification of inventions US Pat. No. (Inventor(s)) Description

Patentable not as an essential process for per-

forming content-based retrieval functions but as

an invention related or just peripheral to general

retrieval functions

US 6,253,201

(Abdel-Mottaleb & Wu)

A process for listing the pre-indexed image

identifiers, which are based on the prior arts of

color and/or edge detection for performing fast

classification

Patentable as the processes by combining the

prior disclosed means to realize the domain-

general approach for performing CBIR functions

US 6,115,717

(Mehrotra et al.)

A process for performing image recognition based

on objects and regions of images, which are stored

in open spaces of text and image commingling

multimedia

US 6,263,089

(Otsuka et al.)

A process for retrieving, by the Hough transfor-

mation, the appearing and disappearing motion

pictures based on texture and edge of images, e.g.,

images of clouds for weather forecasting

US 6,259,809

(Maruo)

A process for performing edge detection of the

rotated, magnified or minified images of linear

components, e.g., semiconductor wafers, by the

Hough transformation and thresholding on sim-

ilarity computation

US 6,240,423

(Hirata)

A process for performing object-based CBIR by a

twofold processing of region detection and

boundary detection, which is based on another

process patented by the same inventor

US 6,246,804

(Sato et al.)

A process for performing edge and color based

region detection in the compound or over-wrap-

ping movable regions of images by combining the

conventional arts of boundary line detection

Patentable as the processes that have the im-

proved formulas for parameter setting based on

the prior disclosed means for performing CBIR

functions and also that give the examples of the

parametric values on parameter setting in the

descriptions

US 6,356,649

(Harkless et al.)

A process for performing CBIR functions of

fingerprints that rotate or dilate, translate or

distort by improving the formulas disclosed in

another prior patent granted to one of the instant

inventors, Thebaud. The claimed invention dis-

closes working examples of the ranges of values

on the thresholds, i.e., the assumed ranges of

rotation and dilation

US 6,125,194

(Yeh et al.)

A process for performing CBIR functions of

radiological images of lung nodules by re-screen-

ing once diagnosed negative images in a neural

network. The claimed invention discloses the

working and prophetic examples of initial weights

on its back-propagation

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 67

3.2. Nonobviousness cases

The processes for performing CBIR functions must realize quantitative and/or qualitative technical

advancement with a combination of prior disclosed means.

3.2.1. Quantitative advancement

A typical case of quantitative nonobviousness is the US Pat. # 6,259,809 invented by Maruo (2001), as

described in Table 3. Its process realizes, as an inventive step, a new function of edge detection of the

Table 3

US Pat. # 6,259,809 (Inventor: Maruo)

Conditions Determinations

Does its patent application claim the means for parameter

setting?

Yes (e.g., means for calculating the mean value of picture

element values, thresholds for providing binarization of image

data, representative point calculation means for determining the

coordinates of a representative point of each group, parameter

space with weight)

Do the prior arts predicate the instant combination of prior

disclosed means for performing the instant functional process?

No. It realizes the instant functional process for detecting the

images of flexible rotation or dilation angles. It is not predicated

from any combination of the conventional means

Does the function realize quantitative and/or qualitative

advancement?

Yes. Both advancements are realized by the processing

Do the descriptions of the instant process specify the formulas

for parameter setting?

Yes

Does the process have any improved formulas for parameter

setting?

No

Does the process realize a new function by combination of the

prior disclosed means?

Yes. Patentable as a domain-general approach of CBIR

68 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

rotated, magnified or minified images of linear components, e.g., semiconductor wafers, without pre-determined angles for rotation, etc. by combining the prior arts on the Hough transformation and thres-

holding on similarity computation. Its description asserts its superior performance by comparing the

computational order of their instant inventive method with ones of conventional disclosed methods, as

described in its detailed description of the preferred embodiments.

Detailed Description (Maruo (2001)). . . .. . .As a consequence, the total volume of computation needed in thepresent invention which uses the Hough transformation is as given below.

The total volume of computation needed according to the conventional technique which applies the

pattern matching technique is as given below.

It will be seen from the above illustration that the first or the fourth embodiment is by a factor of about13 more advantageous than the pattern matching technique in respect of the volume of computation. The

Addition/subtraction 2N þ 360� 2� ð8000þ 1600ÞMultiplication 9N þ 360� 2� 8000Memory access 360� 2� 8000TOTAL 11N þ 360� 2� 25600 � 83N

ðN � 262000Þ

Addition/subtraction 2N � ðN þ 360� NÞMultiplication 9N þ 360� NMemory access 0

TOTAL 11N þ 1080N ¼ 1091N

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 69

second or the fifth embodiment dispenses with edge detecting procedure, and thus reduces the volume of

computation, which results in an advantage by a factor of about 14 as compared with the conventional

technique . . .. . .

3.2.2. Qualitative advancement

A typical case of qualitative nonobviousness is the US Pat. # 6,240,423 invented by Hirata (2001),

as described in Table 4. Its process realizes an inventive step that provides drawbacks of conventionalmeans for CBIR with a solution that is a new function of object-based CBIR. That inventive step consists

of a twofold processing of region detection and boundary detection, which is based on another process

patented by the same inventor. Its description asserts its superior property by comparing it with leading

systems, here, Virage and QBIC, as described in its below detailed description of the preferred embodi-

ments.

Detailed Description (Hirata (2001)). . . .. . .In the registration (image indexing) and matching phases, Viragedoes not identify objects contained in the image. . . .. . . Thus, it is hard to introduce analysis at an objectlevel, such as object-based navigation and object-based clustering. The image matching in QBIC is based on

features such as color, texture, and shape, similar to Virage. . . .. . .The instant inventor has assisted in developing a Content Oriented Information Retrieval engine (the

COIR engine) to . . .. . . evaluate the position of the objects and overlapping areas in order to correlate theobjects with each other. . . .. . . (T)he present invention is directed to . . .. . . provide a way to integrate regionbased image matching with boundary based image matching. . . .. . .

3.3. Enablement cases

The processes for performing CBIR functions must specify the formulas for parameter setting in co-pending applications and/or with exemplary parametric values.

Table 4

US Pat. # 6,240,423 (Inventor: Hirata)

Conditions Determinations

Does its patent application claim the means for parameter setting? Yes (e.g., means for calculating local correlation between

the local blocks, global correlation between the local

blocks as weighted summation of the local correlation, an

average and standard deviation of the global correlation

for each member of the first set of similar images)

Do the prior arts predicate the instant combination of prior disclosed

means for performing the instant functional process?

No. It realizes the instant functional process of object-

based CBIR by a twofold processing of region detection

and boundary detection, which is based on another

process patented by the same inventor

Does the function realize quantitative and/or qualitative advancement? Yes. Both advancements are realized by the processing in

comparison with conventional CBIR systems

Do the descriptions of the instant process specify the formulas for

parameter setting?

Yes

Does the process have any improved formulas for parameter setting? No

Does the process realize a new function by combination of the prior

disclosed means?

Yes. Patentable as a domain-general approach of CBIR

Table 5

US Pat. # 6,115,717 (inventors: Mehrotra et al.)

Conditions Determinations

Does its patent application claim the means for parameter setting? Yes (e.g., average, variance, range of the color component

pixel values, means for depicting properties of open space)

Do the prior arts predicate the instant combination of prior disclosed

means for performing the instant functional process?

No

Does the function realize quantitative and/or qualitative advancement? Yes (Both: Automatic metadata indexing realizes faster

and more precise retrieval functions)

Do the descriptions of the instant process specify the formulas for

parameter setting?

No

Does the discussed patent have any co-pending application that

specifies the above formulas?

Yes (US Ser. 08/786,932)

Does the process have any improved formulas for parameter setting? No

Does the process realize a new function by combination of the prior

disclosed means?

Yes (Realized as image recognition based on the objects

and regions of the images stored in open spaces of text-

image-commingling multimedia)

70 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

3.3.1. Clear specification on formulas for parameter setting in co-pending applications

Its typical case is US Pat. # 6,115, 717 invented by Mehrotra, Warnick, and Romer (2000), as described

in Table 5 with its detailed description of the invention. That invention performs image recognition based

on visual objects and regions of images stored in open spaces of text and image commingling multimedia by

automatic metadata indexing. First, that process does not specify any formulas for parameter setting,

though their formulas are disclosed in its co-pending application of the US Ser. No. 08/786,932. Second,

that invention realizes new functions of open space metadata indexing and region recognition by combining

the prior disclosed means.

Detailed Description (Mehrotra et al. (2000)). . . .. . .Image Storage Component

. . .. . . The present embodiment employs the open space detection method described in the co-pending andcommonly assigned patent application Ser. No. 08/786,932 entitled ‘‘Method and System for Detection and

Characterization Open Space in Digital Images’’ to detect open space, generate open space maps and

compute associated open space metadata. . . .. . .

3.3.2. Clear specification on the formulas for parameter setting with exemplary values

A certain domain-specific approach not only performs a new function by a combination of prior dis-

closed means but also has certain improved formulas for parameter setting based on the prior disclosed

means in order to perform CBIR functions. In this case, the descriptions of the process for the domain-specific approach must give its parameter setting with examples of parametric values that are the working

or prophetic examples of initial values or weights, otherwise, the working examples of the ranges of

parametric values. Its typical case is US Pat. # 6,356,649 invented by Harkless et al. (2002), as described in

Table 6.

That invention performs retrieval of fingerprint images which often rotate or dilate, translate or distort.

Its thresholding functions evaluate similarity of images by correlation-computation, as described in its

sixteenth claim.

Table 6

US Pat. # 6,356,649 (Inventors: Harkless et al.)

Conditions Determinations

Does its patent application claim the means for parameter setting? Yes (e.g., assumed rotation and dilation, and distortion of

images, means for rationing the respective ridge-spacing

and orientation values, means for correlating the two

transformed power spectral densities, a normalized spatial

correlation value at similarity correlation)

Do the prior arts predicate the instant combination of prior disclosed

means for performing the instant functional process?

No

Does the function realize quantitative and/or qualitative advancement? Yes (Precise qualitative performance by rotation and

distortion detection)

Do the descriptions of the instant process specify the formulas for

parameter setting?

Yes

Does the process have any improved formulas for parameter setting? Yes

Do the descriptions of the process give examples of the values on

parameter setting in the descriptions?

Yes (With working examples of the ranges of values on the

thresholds, i.e., the assumed ranges of rotation and

dilation for correlation computation)

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 71

Claim # 16 (Harkless et al. (2002)). The apparatus . . .. . ., wherein the evaluating means further comprise:means for forming a composite measure of . . .. . . subregions in the common area; and means for thres-holding said composite measure to make said decision. . . .. . .

Those processes for thresholding functions have several estimated weights as their exemplary ranges

of parametric values on parameter setting as below. The data-processing processes of the instant inven-

tion have the improved formulas for parameter setting based on the prior disclosed means as claimed in

US Pat. # 5,909,501 invented by Thebaud (1999), who is one of the instant inventors. The instant pro-cesses detect the rotated or dilated images of fingerprints without pre-determined angles for translation

of those images. The discussed invention, Harkless et al., discloses its detailed embodiments in the appendix

of its descriptions by giving the working examples of the ranges of values on parameter setting, i.e.,

the thresholds as the assumed ranges of rotation and dilation for correlation computation, though they

are not disclosed in the prior patent application by Thebaud. The working examples provide the instant

retrieval processes with the assumed ranges of rotation and dilation values that are indispensable to

implementation of the best mode of the patented data-processing processes, as described in its below

description.

Detailed Description (Harkless et al. (2002)). . . .. . .Stage 2 of Processing

III. Overview of the global search procedure:

E. Determine the optimum rotation and dilation to apply to the template window . . .. . .:1. FFT the b-grid template and candidate windowed data fields, then truncate to Nyquist. . . .. . .3. Correlate these two new fields in the hypothesis range (currently) of )18� to 18� of rotationand ±10% of dilation to find the best rotation/dilation value for the template window region.. . .. . .

72 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

4. Discussions

Our formulation is unique in the sense that it is based on process patents of computer-related inventions

in the form of computer programs which receive attention as patents of the ‘‘methods of doing business’’,i.e., business model patents. CBIR is promising for retrieving images in a large-scale digital library. In

particular on application, its domain-specific approach has received attention and elaboration in several

areas including the following fields: medical image retrieval systems of radiological or MRI images of

nodules in brains, abdomens, intestines, etc.; product deficit image retrieval systems in artificial manu-

factures including semiconductor wafers; and authentication systems based on human skin and pupil

patterns, etc.

Our formulation focuses on parameter setting components that determine, with their thresholding

functions, whether a candidate image is similar to an exemplary requested image in certain specific do-mains. Inventors as patent applicants should provide parameter setting with example ranges of parametric

values, otherwise exemplary initial values or weights.

Our formulation is limited to several requirements for patentability: patentable subject matter, non-

obviousness and enablement. Novelty and other procedural issues should be consulted with general patent

law research.

5. Conclusions

In this paper, we have formulated the conditions on patentability of data-processing processes for

performing content-based retrieval functions as combinations of prior disclosed means and also on pro-

cesses comprising the means for parameter setting. There exist two critical issues in satisfaction of

requirements for patentability on those processes. One is nonobviousness on a combination of the prior

disclosed means. The other is enablement on parameter setting that realizes a domain-specific approach of

content-based retrieval. Our formulation follows the standards of patent examination and litigation oncomputer-related inventions that are practiced in the US. The case study shows the feasibility and

accountability of our formulated conditions on patentability that facilitate patenting those functional

processes of CBIR in the US.

We are preparing a paper that formulates systematic digital library protection by using patentable

content-based retrieval processes (Sasaki & Kiyoki, 2003, 2004).

Acknowledgements

We would like to thank Koichi Furukawa of Keio University for his suggestions on the fish cargo patent,

which has been referred to give an instance of parameter setting in Section 1.

References

Abdel-Mottaleb, M., & Wu, M. (2001). United States Patent 6,253,201: Scalable solution for image retrieval. Issued on June 26, 2001.

Filed on June 23, 1998.

Agrawal, R., Equitz, W. R., Faloutsos, C., Flickner, M. D., & Swami, A. N. (1997). United States Patent 5,647,058: Method for high-

dimensionality indexing in a multi-media database. Issued on July 8, 1997. Filed on February 28, 1996.

Autogiro Co. of America v. United States (1967). 384 F.2d 391, 155 U.S.P.Q. 697 (Ct. Cl. 1967).

Bach, J. R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Jain, R., & Shu, C. F. (1996). The Virage image search engine: an open

framework for image management. In Proceedings of SPIE storage and retrieval for still image and video databases, IV (pp. 76–87),

San Jose, CA.

H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74 73

Brin, S. (1998). Extracting patterns and relations from the World Wide Web. In P. Atzeni, A. O. Mendelzon, & G. Mecca (Eds.),

Proceedings of the 6th international workshop on the web and databases (WebDB’98) at EDBT’98, Valencia, Spain (pp. 172–183).

Brin, S., & Page, L. (1998). The anatomy of large-scale hypertextual Web search engine. In Proceedings of the 7th international World

Wide Web conference, Brisbane, Australia (pp. 107–117).

Flickner, M., Sawhney, H., & Niblack, W. (1995). Query by image and video content: the QBIC system. IEEE Computer, 28(9), 23–32.

Graver Tank&Mfg. Co. v. Linde Air Products Co. (1950). 339 US 605.

Harkless, C. R., Potter, R. E., Monro, J. A., & Thebaud, L. R. (2002). United States Patent 6,356,649: Systems and methods with

identity verification by streamlined comparison and interpretation of fingerprints and the like. Issued on March 12, 2002. Filed on

April 11, 1997.

Hirata, K. (2001). United States Patent 6,240,423: Method and system for image querying using region based and boundary based

image matching. Issued on May 29, 2001. Filed on April 22, 1998.

In re Alappat (1994). 33 F.3d 1526, 31 U.S.P.Q.2d 1545 (Fed. Cir. 1994) (en banc).

In re Dow Chemical Co. (1988). 837 F.2d 469, 473, 5 U.S.P.Q.2d 1529, 1531 (Fed. Cir. 1988).

In re Strahilevitz (1982). 682 F.2d 1229, 212 U.S.P.Q. 561 (C.C.P.A. 1982).

In re Trovato (1995). 42 F.3d 1376, 33 U.S.P.Q.2d 1194 (Fed. Cir. 1994), cavated & remanded, 60 F.3d 807, 33 U.S.P.Q.2d 1194 (Fed.

Cir. 1995) (en banc).

Jain, R., Horowitz, B., Fuller, C. E., Gupta, A., Bach, J. R., & Shu, C. F. (1999). United States Patent 5,893,095: Similarity engine for

content-based retrieval of images. Issued on April 6, 1999. Filed on March 28, 1997.

Jakes, J. M., & Yoches, E. R. (1989). Legally speaking: Basic principles of patent protection for computer science. Communications of

the ACM, 32(8), 922–924.

Laitran Corp. v. Rexnord, Inc. (1991). 939 F.2d 1533, 19 U.S.P.Q.2d (BNA) 1367 (Fed. Cir. 1991).

Manchester Visualization Center (2003). CBIR evaluation [On-line]. Available: http://www.man.ac.uk/mvc/Research/CBIR/ (last

modified March 10, 2003).

Maruo, K. (2001). United States Patent 6,259,809: System and method for recognition of image information. Issued on July 10, 2001.

Filed on August 31, 1998. Foreign Application Priority Data: August 29, 1997[JP] 9-234264.

Mehrotra, R., Warnick, J., & Romer, D. M. (2000). United States Patent 6,115,717: System and method for open space metadata-

based storage and retrieval of images in an image database. Issued on September 5, 2000. Filed on June 30, 1997.

Merges, R. P. (1997). Patent law and policy: cases and materials. Contemporary legal education series (2nd ed.). Charlottesville, NC: The

Michie Company.

M€uller, H., M€uller, W., Squire, D.M., Marchand-Maillet, S., & Pun, T. (2001). Automated benchmarking in content-based image

retrieval. In Proceedings of the 2001 IEEE international conference on multimedia and expo (ICME 2001), Tokyo, Japan (pp. 321–

324).

Pennwalt Corp. v. Durand-Wayland, Inc. (1988). 833 F.2d 931, 4 U.S.P.Q.2d 1737 (Fed. Cir. 1987) (en banc), cert. denied, 108 S. Ct.

1226 (1988).

Rui, Y., Huang, T. S., & Chang, S.-F. (1999). Image retrieval: current techniques, promising directions and open issues. Journal of

Visual Communication and Image Representation, 10(4), 39–62.

Sasaki, H., & Kiyoki, Y. (2001). The patent application methodology of a multimedia database search engine comprising parameter

settings for optimizing data processing mechanisms as a patentable computer-related invention. Transactions on Databases of the

Information Processing Society of Japan, 42(11), 22–38 (in Japanese).

Sasaki, H., & Kiyoki, Y. (2002a). Patenting advanced search engines of multimedia databases. In S. Lesavich (Ed.), Proceedings of the

3rd international conference on law and technology (pp. 34–39). Cambridge, MA: International Society of Law and Technology

(ISLAT).

Sasaki, H., & Kiyoki, Y. (2002b). Patenting the processes for content-based retrieval in digital libraries. In E.-P. Lim, S. Foo, C. Khoo,

H. Chen, E. Fox, S. Urs, & T. Costantino (Eds.), Proceedings of the 5th international conference on Asian digital libraries

(ICADL)––digital libraries: people, knowledge, and technology, Lecture Notes in Computer Science (No. 2555, pp. 471–482),

Singapore.

Sasaki, H., & Kiyoki, Y. (2003). A proposal for digital library protection. In Proceedings of the 3rd IEEE–CS+ACM joint conference

on digital libraries (JCDL) (p. 392). Houston, TX: IEEE Computer Society Press.

Sasaki, H., & Kiyoki, Y. (2004). Multimedia digital library as intellectual property. In S. Foo & Y. L. Theng (Eds.)Design and usability

of digital libraries: Case studies in the Asia Pacific Idea Group Press.

Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000). Content-based image retrieval at the end of the early

years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1349–1380.

Smith, J. R., & Chang, S. F. (1996). Querying by color regions using the VisualSEEK content-based visual query system. In M. T.

Maybury (Ed.), Proceedings of the international joint conference on artificial intelligence (IJCAI): Intelligent multimedia information

retrieval (pp. 23–42).

Smith, J. R., & Chang, S. F. (1997). Visually searching the Web for content. IEEE Multimedia, 4(3), 12–20.

State Street Bank & Trust Co. v. Signature Financial Group, Inc. (1998). 149 F.3d 1368, 47 U.S.P.Q.2d 1596 (Fed. Cir. 1998).

74 H. Sasaki, Y. Kiyoki / Information Processing and Management 41 (2005) 57–74

Thebaud, L. R. (1999). United States Patent 5,909,501: Systems and methods with identity verification by comparison and

interpretation of skin patterns such as fingerprints. Issued on June 1, 1999. Filed on September 9, 1996.

Unique Concepts, Inc. v. Brown (1991). 939 F.2d 1558, 19 U.S.P.Q.2d 1500 (Fed. Cir. 1991).

US Patent Act (2003). Title 35 US C. §§101, 103, 112.

US Patent and Trademark Office (1996a). Examination guidelines for computer-related inventions, 61 Fed. Reg. 7478 (February 28,

1996) (‘‘Guidelines’’) [On-line]. Available: http://www.uspto.gov/web/offices/com/hearings/software/analysis/computer.html (last

modified November 16, 2003).

US Patent and Trademark Office (1996b). Examination guidelines for computer-related inventions training materials directed to

business, artificial intelligence, and mathematical processing applications (‘‘Training Materials’’) [On-line]. Available: http://

www.uspto.gov/web/offices/pac/compexam/examcomp.htm (visited January 8, 2004).

US Patent and Trademark Office (2003). USPTO patent full-text and image database [On-line]. Available: http://www.uspto.gov/patft/

index.html (last modified August 28, 2003).

Yeh, H. Y. M., Lure, Y. M. F., & Lin, J. S. (2000). United States Patent 6,125,194: Method and system for re-screening nodules in

radiological images using multi-resolution processing, neural network, and image processing. Issued on September 26, 2000. Filed

on February 4, 1998.