Haptic Characteristics of Document ConservationTasks

Rainer Leuschke∗, Regina Donlin∗, Marc Claus†, Maria Nugent‡, Dianne van der Reyden‡, Blake Hannaford∗,∗BioRobotics Laboratory

Department of Electrical Engineering

University of Washington

Seattle, Washington 98195

Email:{leuschke, rdonlin, blake}@u.washington.edu† Delft University of Technology (TUDelft), The Netherlands

Email: marcclaus@gmail.com‡Preservation Division

Library of Congress, Washington D.C.

Email:{lnug,dvan}@loc.gov

Abstract— Conservation of historic documents is often nec-essary to preserve their cultural value for future generations.An important component of the skill of document conservationis delicate hands-on manipulations. As with medical proceduraltraining, there is a great need for better ways to train documentconservators in these skills. This paper reports initial measure-ments of forces and torques at the interaction point betweentools and mocked up documents. Five conservators used theirpreferred tools (such as scalpel, needle, brush, and microspatula)to remove material adhered to the samples. We analyzed videoand 100 Hz force and torque recordings in the time and frequencydomain to gain understanding of the nature of these tasks. Theresults can inform design of training simulators for documentconservation skills.

I. INTRODUCTION

Conservation of historic documents, including books,

manuscripts, maps, prints, and photographs, often requires

examination and treatment of important works that can’t

otherwise be used. This is because such documents may be

dirty, torn, stained, brittle, poorly repaired or damaged in any

number of ways, as a result of handling or natural aging.

The United States Library of Congress has over 130

million books and documents of various types, and many

millions require individualized treatments, including surface

or mechanical cleaning, and removal of brittle backings,

acidic tapes, thick adhesives, extraneous attachments, etc.

Conservators in the Library’s Preservation Directorate de-

vote tens of thousands of hours annually to undertake in-

dividualized treatments for thousands of documents. (See

http://www.loc.gov/loc/lcib/0705/directorate.html )

To learn their profession, conservators study theory and

practice. Some conservators earn masters degrees following

2-4 years training at university, plus 16-30 months practical

internships. These conservators are trained in materials science

in order to understand the nature of substrates and media

such as leather, parchment, paper, inks and photographic

emulsion, etc. They are also trained to use analytical and

practical techniques and tools originally developed for the

medical profession, since many are adaptable to the treatment

of documents.

For example, microspatulas can be used to split or scrape

acidic and brittle back boards; scalpels may be used to slice,

score, or breakup and remove unstable accretions; tweezers

can tease off cellophane tapes when adhesives are softened by

heat or solvents; swabs and brushes may apply solvents for

dissolution of adhesives; and syringes may deliver controlled

amounts of consolidants to reattach flaking ink. van der

Reyden has recently developed a training manual derived from

a compilation of standard techniques used in the document

conservation field[1], [2].

Traditionally, conservators are trained by working under

the guidance of an instructor on actual collection objects

or on mock-ups. In the first case, when working on actual

collection items, instructors must constantly monitor trainees

to make sure they make no wrong move that would lead, for

instance, to slicing a scalpel through a delicate architectural

tracing, weakening and devaluing a piece or worse. In the

second case, when using mock-ups, a trainee can practice

alone and even make mistakes, but significant resources are

required for making and properly storing the mock-ups. In

both cases, a great deal of time is invested in only one range

of problems. For example, there are a wide range of backing

materials used to mount photographs. The variety includes thin

or thick acidic, brittle, or colored paper; Bristol, corrugated,

orange or grey cardboard; and starch, protein, gum or synthetic

adhesives that were brushed, sprayed, or thermally applied.

Any combination of these may have been used to back a

wide range of photographic prints, including albumen, gelatin,

collodion, platinum, etc. And to complicate the challenge, how,

to what extent, and under what circumstances an object has

aged or worn down can also effect its sensitivity and reaction

to treatment. Consequently the variables a conservator might

encounter seem infinite.

As a result, there has long been interest to develop a way

to ensure a trainee encounters the broadest range of theseSymposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems 200813-14 March, Reno, Nevada, USA978-1-4244-2005-6/08/$25.00 ©2008 IEEE

383

variables in a systematic manner. Capturing such a range of

variables can be done most effectively by adapting virtual

reality or haptic technology, used to train medical interns in

treatments of humans, to train conservators in the treatment of

objects.At the Library of Congress, interest in simulation based

training for document conservation skills is four-fold. The Li-

brary has an extensive training program, and such technology

would expedite training and raise the standard of hand skills to

a measurable level. It could increase the variety of experiences

to which a trainee would be exposed. It would provide a

way for senior conservators to practice before undertaking

techniques not used for awhile, or to practice or try out newly

developed techniques, without huge resources of time, space

or materials. Finally, senior conservators with scores of years

of experience are approaching retirement. Haptic technology

will enable us to capture their well-honed expertise and ensure

its availability for training students both onsite and remotely,

via far-flung satellite workstations in countries across the

globe. This in turn frees instructors and students to devote

discussion time and monitoring to treatments that are not so

easily simulated.In the late 1990’s, Camberwell College in the UK embarked

on a program to develop a haptic system that would simulate

the feel and appearance of a backing being removed from a

document[3], [4]. Some of the Camberwell team’s findings in-

cluded the time required and best means to develop expertise.

However, the model parameters of their simulation were tuned

after the fact for apparent realism.Surgical Data Recording: The delicate manipulations and

even the tools of document conservation are reminiscent of

surgery. Surgical simulators with force feedback have an

increasing base of measurements to draw from. Force sensors

have been applied to surgical instruments for characterization

of surgery by several groups over the last 10 years [5], [6], [7],

[8], [9]. These types of studies have recorded multi-axis force

and torque information from sensors placed in the surgeon’s

tools and performed various types of analyzes on the resulting

multi-axis data.In both fields of application, this data can be used to benefit

training simulations in at least four ways:

• A histogram can be collected of force magnitudes. With

this information, haptic device actuators can be selected

which will support a given fraction of the measured forces

(i.e. 95%) without being oversized.

• Frequency data can be computed to get an estimate of

the bandwidth required of the haptic device.

• Force/torque data can be combined with independent

measurements of displacement to create accurate mod-

els of the mechanical properties of the materials being

manipulated.

• Statistical or other techniques can be derived to create

measures of task skill from the recorded signals. Data

must be collected from various groups with presumed

skill levels to establish norms.

In this paper we will dwssribe initial results pertinent to the

first two of these benefits.

II. METHODS

Instrumentation: In earlier work, we developed a device

to measure, record and evaluate tool/tissue interaction data for

eye surgery [10]. To collect data for document conservation

tasks we utilize a similar approach. A platform supporting the

work piece, or in this case a mocked-up sample, is placed

on top of a 6-axis force/torque sensor (Nano 17 from ATI

Industrial Automation). The samples for our study were fixed

securely to the platform using double stick tape. The base of

the sensor is mounted to a large piece of backing board that

the subject can hold down and reposition as needed to access

the work area. With its placement directly below the mock-up

workpiece, the 6-axis sensor tracks direction and magnitude of

resulting forces and torques applied to the sample throughout

the treatment. The coordinate system of the sensor data is

fixed to the platform with the z-axis pointing up, perpendicular

to the paper plane (Figure 1). This arrangement allows the

subject to utilize all their tools unmodified for data recording

purposes. However due to the sensor height the sample is

elevated about 20mm above the regular work surface. Subjects

hold and reposition the sample by moving the base (without

touching the upper platform so as not to interfere with force

measurements). Figures 1 and 2 show this arrangement.

In addition to the platform setup, we developed an instru-

mented tool holder for tool based measurements. Figure 1

shows the tool holder with a scalpel blade attached. A 6-axis

force/torque sensor (Nano 17) is used to measure tool-sample

interaction. A handle modeled after a standard scalpel handle

(No. 5) is mounted to the base of the sensor and an interface

bracket is attached to the tool side of the sensor. Any standard

scalpel blade and a few other modified tools can be mounted

to the bracket. For this study we can mount a cotton swab,

a microspatula and scalpel blades No. 11 and No. 15. The

coordinate frame of the acquired data is fixed to the tool with

the z-axis extending along the long axis of the tool. Subjects

can hold and manipulate the handle as usual, however the

integration of the sensor results in an additional offset of about

20mm between tool tip and hand.

Because of the time required to change tips on this tool and

the frequent tool changes used by the conservators, the data

reported below was collected with the platform.

For both the tool based and platform arrangements, the

force/torque sensor is connected to a laptop equipped with a

data acquisition card. This allows for real time recording and

display of the data. A firewire digital video camera imaged

the subject’s hand, tool, and workpiece. The camera interface

permitted frame capture to be controlled by software.

Video and force/torque data were captured by software

programmed with Labview. The video frame rate was set to

25 fps and data rate was set to an integer multiple of the

frame rate. The video and data are software synchronized.

During a treatment, data and video were recorded to a file to

allow off-line analysis. Data were generally collected at 100

384

Fig. 1. Video still showing instrumented scalpel for paper tape removal task.Coordinate systems for platform and scalpel are illustrated.

Fig. 2. Instrumentation setup for document conservation experiments. Forcetorque sensor could be mounted either in tool handle or work support platform.Firewire video camera collected close up imagery of tool use. Most tasks wereperformed using the microscope with video monitor.

Hz with some additional episodes recorded at 1000 Hz for

task bandwidth validation.Test materials and task definitions: Five conservation

tasks were selected by the Library of Congress conservation

staff to represent a range of manipulation skills in paper

document conservation (summarized in Table I):

1) Paper Tape Removal Samples were prepared of blue

binder board to which paper tape with water-based

adhesive (Lineco Inc. Type 533-0751) was affixed and

allowed to set. Water was applied with a small fine brush

to soften the adhesive and a scalpel was used to pick and

peel up the tape as the adhesive softened.

2) Linen Tape Removal Samples were prepared of blue

binder board to which cloth tape with water-based

adhesive (Lineco Inc. Type 533-1010) was affixed and

allowed to set. Water was applied with a small fine brush

to soften the adhesive and a scalpel was used to pick and

peel up the tape as the adhesive softened.

3) Old Label Removal A set of identical documents of no

historical value dated 1879 were obtained with original

identical small red office stickers affixed. Water was

applied with a small fine brush to soften the adhesive, a

scalpel was used to pick and peel up the sticker as the

adhesive softened. In tasks 1, 2 and 3, some conservators

scored or scratched the surface of the sticker to speed

absorption of the water.

4) Mechanical Surface Cleaning with Eraser CrumbsA piece of matboard was marked by rolling graphite

across its surface. A pile of Staetdler Mars Plastic eraser

crumbs was poured onto the document surface, and the

subject used two or three fingers to rub the powder

around on the surface.

5) Mechanical Surface Cleaning with Eraser BlockMarked as above, but after cleaning with eraser crumbs,

the matboard was rubbed with a new Staetdler Mars

Plastic eraser.

Protocols: Five male and female subjects between the ages

of 39 and 60 were studied. All of the subjects under study

were right-handed. Average years of document conservation

experience of the subjects was 20. The experimental protocol

was approved by the University of Washington Human Sub-

jects Committee. Each subject was asked to perform the five

conservation tasks described above in order.Common tools used in document conservation were pro-

vided to the subjects. The tools provided to them were: a

scalpel with No. 11 and No. 15 blades, tweezers, microspatula,

small fine brush, cotton swabs, a pin-like tool and a micro-

scope. The subjects were allowed to bring their own preferred

tools for use in the experiment if desired. The solvents

available to the conservators to soften the adhesive were water,

ethanol and methylcellulose, but all the conservators chose to

use only water. The subjects were under no time constraint to

complete each task.For the Paper and Cloth Tape removal tasks, the subjects

were asked to peel back the tape to a marked location

approximately 3/8 inch from the corner of the sample.

385

Task Substrate Accretion Sample Preparation Actions1 Modern blue binder board (Con-

servation by Design Limited, Su-perior Millboard, 1.9 mm thick)with a smooth, hard surface, ma-chine made with chemical pulpfibers

Water-based Archival GummedPaper Hinging Tape (Lineco Inc.Type 533-0751)

Tape was dampened by a porce-lain wheel with DC tap waterand applied to the rough side ofthe board, placed under polyweb,blotters and 1lb weight to dry for0.5 hour

Tape removal with pin, scalpel,microspatula, brush, water, abso-lute ethanol and/or methylcellu-lose (Bookmakers Grade A4M,Viscosity 4000)

2

”Water-based Archival GummedLinen Hinging Tape (Lineco Inc.Type 533-1010)

” ”3 Vintage rag paper, cream-colored

with blue ruled linesOld red-bordered gummed label Unknown (samples are dated

1879)Label removal with pin, scalpel,microspatula, brush, water, abso-lute ethanol and/or methylcellu-lose (Bookmakers Grade A4M,Viscosity 4000)

4 Modern white Matboard(Archivart), 4-ply

Graphite soiling Soiling was done by rolling a #7(Papermate) graphite stick later-ally across the smooth side of theboard

Soil removal using mechanicalcleaning with Staetdler MarsPlastic Eraser (vinyl) crumbs

5

” ” ”Soil removal using mechanicalcleaning with Staetdler MarsPlastic Eraser (vinyl) block

TABLE I

DETAILS OF CONSERVATION TASKS STUDIED IN THIS EXPERIMENT

For the Mechanical Surface Cleaning tasks, the subjects

were asked to clean a document that was prepared with

graphite rubbed on the surface. Typical mechanical cleaning

involved two phases. Fresh eraser crumbs were poured on the

surface of the sample in a small pile about 1cm thick. The

subjects used two or three fingers to rub the crumbs around

on the surface. Second, the subjects rubbed the sample using a

Staetdler Mars Plastic Eraser until they indicated that they had

completed the task. These two phases were analyzed separately

as tasks 4 and 5.

Data analysis: The collected data and video were recorded

for off-line analysis using custom LabView software. After

the experiment, data points were tagged to label tool use.

Tags were manually applied by stepping through frames of

the video recording of the experiment. The LabView app was

used to play back the video for each task by all subjects. The

experimenter selected a tag number associated with the tool

used in each video frame, which was synchronized with the

data. An additional column was added to the data to include

the tag associated with each data point. The tags used for the

data analysis are:

• No Activity

• Scalpel #11

• Scalpel #15

• Microspatula

• Eraser

• Eraser crumbs

• Tweezer

• Brush + Tweezer

• Swab

• Swab H2O

• Brush

• Brush H2O

• Pin

• Fingertips

• Moving Platform

Tagging the data points enabled us to perform analysis on

force and torque associated with each tool.

Histograms and Fourier analysis were computed based on

the magnitude of the force and torque vectors:

Fm =√

F 2x + F 2

y + F 2z , τm =

√τ2x + τ2

y + τ2z

RMS value was computed from the magnitudes according

to

FRMS =

√√√√ 1N

N∑i=1

Fmi2

where i is the sample number, and N is the total number of

samples. In making the RMS value calculation, we edited the

data to ignore most non-contact data segments except those

during rapid periodic tool contacts, because periods of zero

force (which can be arbitrarily long as subjects change tools

or take breaks) lower the RMS value.

Although we have begun analysis of the torque signals, it

is important to realize that most of the tasks were essentially

point contact and thus generate no torque. Furthermore, the

torque signals recorded by the sensor, τsensor contain a term

proportional to both the contact torque, τC , and contact force,

FC , according to

τsensor = τC + R × FC

where R is the vector from the sensor coordinate system origin

to the contact point. Although we measured R for each task,

R continually changed due to the subject moving the contact

point. Thus, accuracy of correcting the sensor torque was

limited. Beyond Figure 3, we will not further report torque

data in this paper.

386

Fig. 3. Typical force and torque records (magnitude of x,y,z components)for the linen tape removal task (task 2) plotted as a function of time. Subtaskbeing performed and tool being used were identified from the videos and areplotted.

III. RESULTS

A typical record of forces and torques during one task

is shown in Figure 3. Bursts of force above noise level

correspond to periods of contact between tool and workpiece.

A frame from the video record, selected during tool-work

contact is shown in Figure 1.

The frequency of each force level for the Paper Tape

Removal, Cloth Tape Removal, and Mechanical Cleaning tasks

was analyzed by computing histograms of the magnitude of

force (Figure 4). The RMS value of the forces was calculated

for all periods except the pauses for tool changes. The his-

tograms are not normalized to the length of the various data

sets.

RMS force computed for each tool type and task are given

in Table II. RMS force ranged from 0.05N (light use of the

cotton swab on Paper Tape) to 0.8N (heavy rolling use of the

cotton swab). RMS force for Mechanical Cleaning was 1.7N

for the fingertips and 2.1N for the eraser.

RMS value of the sensor background noise was 0.1N. It

should be noted that the RMS force for the cotton swab

(0.05N) is less than the noise level. The reason for this is

that the noise (defined as force signal when there was no tool

contact) had two components. First a “white noise” pseudo

random component, and second, a DC offset which changed

at discrete intervals over time. We are still investigating the

source of this DC offset which sometimes changed in response

to transient contact, but other times changed spontaneously

without contact. The DC offset did not appear during the

Cotton swab data but the background RMS force value was

computed from all parts of the entire data set which were

tagged “no-activity”.

Considering the background noise in the sensor, force

signals from the fine brush and light use of the cotton swab (as

in the Paper Tape task) are not distinguishable. The noise level

0 0.2 0.4 0.6 0.8 1 1.20

5000

10000

Force (N)

Num

ber o

f Sam

ples

Paper Tape Experiment

0 0.2 0.4 0.6 0.8 1 1.20

5000

10000

Force (N)

Num

ber o

f Sam

ples

Cloth Tape Experiment

0 0.2 0.4 0.6 0.8 1 1.20

5000

10000

Force (N)

Num

ber o

f Sam

ples

Antique Sticker Removal Experiment

Fig. 4. Histograms of force information for three different conservation tasksby all subjects.

Fig. 5. Force magnitude for tool-work contact phases for 8-second samplesfrom three tasks: Paper Tape Removal, Cloth Tape Removal, and MechanicalCleaning. Note different Y-axis scales for |F |.

of the commercially available ATI nano force/torque sensor

was not sufficiently small to be able to clearly resolve the

most delicate tasks like applying water with a small fine brush.

The DC offset component of noise may be an artifact of our

platform and we will continue to work on it. The conservation

tasks chosen for study featured highly intermittent contact.

Three segments of data, consisting only of tool-work inter-

action, were selected for further analysis (Figure 5). These

consist of eight seconds of data taken during the use of

the #11 Scalpel, the microspatula, and Finger tip rubbing.

All show periodic behavior characteristic of the repeated

nature of the picking/rubbing actions observed in the video.

Their amplitudes (note differing scales for each subplot) seem

roughly consistent with the values in Table II, but their signal

to noise ratio appears quite good.

A Fourier analysis was performed on these three samples to

387

Tool Paper Tape Cloth Tape Old StickerBackground 0.10 0.10 0.10Scalpel #11 0.32 0.16 0.33Scalpel #15 0.16 0.16 0.33Microspatula 0.46 0.34 0.38

Tweezer 0.26 0.27 0.20Cotton Swab 0.05 0.25 0.81Fine Brush 0.12 0.08 0.11

Pin 0.59 0.40 —

TABLE II

RMS FORCE MAGNITUDES (NEWTONS) FOR SEVERAL TASKS AND TOOLS. PIN WAS NOT USED ON THE OLD STICKER TASK.

−1

100

101

102

10

−50

−40

−30

−20

−10

0

10

20

Conservation tasks: Frequency Domain Signal

Frequency (Hz)

|For

ce| (

dB)

Paper

Cloth

Mechanical Cleaning

Fig. 6. Fourier analysis of Force magnitude for three tasks: Paper TapeRemoval, Cloth Tape Removal, and Mechanical Cleaning. Each plot has beennormalized by the total signal energy.

determine the frequency content of the force magnitude signals

for the Paper Tape, Cloth Tape, and Mechanical Cleaning

Tasks (Figure 6). The frequency below which 95% of the

energy was contained was 25, 39, and 25 Hz for the three

tasks respectively.

IV. DISCUSSION

We have presented characteristics of haptic interactions

between experienced document conservators and test samples.

We encouraged the conservators to use their normal preferred

techniques and constrained them as little as possible. There

are many aspects of this data for further analysis including

directional information, clarification or filtering of the DC

noise terms, and refinement of the Fourier analysis parameters.

These data supply reference information with which to

synthesize design of possible simulation training systems for

the hands on skills of document conservation. Although much

of the skill of document conservation consists of analysis of

material properties and condition and deciding which methods

to apply to precious documents, the correct application of

techniques requires hands-on training of the highest skill level

possible. Haptic simulation training technology, aided by data

like that of this paper, hopefully will ensure attainment of high

skill levels and increase the speed of training and treatment of

damaged and deteriorating documents in the race against time

to save them.

V. ACKNOWLEDGEMENTS

The authors would like to acknowledge support of the

Library of Congress, Library Services administration and

the Preservation Directorate staff, particularly conservators

Tamara Ohanyan, Lage Carlson, and Elmer Eusman, and NSF

grant IIS-0713028.

REFERENCES

[1] V. Authors, The Paper Conservation Catalog. American Institute forConservation Book and Paper Group, 1993.

[2] D. van der Reyden, “Recent scientific research in paper conservation,”Journal of the American Institute for Conservation, vol. 31, no. 1, pp.117–139, 1992.

[3] A. Geary, “A haptic virtual reality training tool for paper conservators,”International Conference on Computer Graphics and Interactive Tech-niques, pp. 1–1, 2003.

[4] A. Geary and M. Sandy, “A haptic training simulation for paperconservation: Preliminary results,” Eurohaptics Proceedings, 2004.

[5] A. Bicchi, G. Canepa, D. De Rossi, P. Iacconi, and E. Scillingo, “Asensor-based minimally invasive surgery tool for detecting tissueelasticproperties,” Robotics and Automation, 1996. Proceedings., 1996 IEEEInternational Conference on, vol. 1, 1996.

[6] A. Morimoto, R. Foral, J. Kuhlman, K. Zucker, M. Curet, T. Bocklage,T. MacFarlane, and L. Kory, “Force sensor for laparoscopic babcock.”Stud Health Technol Inform, vol. 39, pp. 354–61, 1997.

[7] J. Rosen, B. Hannaford, C. Richards, and M. Sinanan, “Markov mod-eling of minimally invasive surgery based on tool/tissue interaction andforce/torque signatures for evaluating surgical skills,” IEEE Transactionson Biomedical Engineering, vol. 48, no. 5, pp. 579–591, May 2001.

[8] S. Greenish, V. Hayward, V. Chial, A. Okamura, and T. Steffen,“Measurement, Analysis and Display of Haptic Signals During SurgicalCutting,” Presence: Teleoperators and Virtual Environments, vol. 11,no. 6, pp. 626–651, 2002.

[9] S. DiMaio and S. Salcudean, “Needle insertion modeling and simula-tion,” Robotics and Automation, IEEE Transactions on, vol. 19, no. 5,pp. 864–875, 2003.

[10] R. Leuschke, A. Bhandari, B. Sires, and B. Hannaford, “Low cost eyesurgery simulator with skill assessment component,” in ProceedingsMMVR 2007, Long Beach, CA, Feb 6-9 2007.

388