42510011 0010 1010 1101 0001 0100 1011

Hazards of price-per-use comparison

in e-journal managementJason S. Price, Ph.D.Claremont Colleges’

LibrariesLos Angeles, California

Are they any use?

Plenary Session 430th Annual UKSG Conference, 16-18 April 2007University of Warwick, Coventry, UK

4251

0011 0010 1010 1101 0001 0100 1011

General hazards -- Broad Strokes1. Defining use narrowly2. Vagaries of user

behavior3. Different dissemination

styles in teaching

* Ejournal-centric approach in academic institutions

4251

0011 0010 1010 1101 0001 0100 1011

G1. A narrow definition of useCOUNTER JR 1: Full text article views

Additional use-related measures: • A-Z list click throughs/Web log files• Times cited at your Inst. in recent papers ACS

Livewire 8:2• Impact/Usage Factor• # of papers published by local researchers by

journal • Faculty/Researcher Surveys• Print Use?• Emerging measures? (Bollen & Van de Sompel 2006)

– Page rank (vs. Impact factor) & who’s using what– Viewing structural patterns in usage data

4251

0011 0010 1010 1101 0001 0100 1011

G2. Vagaries of search/use habits• Users may check for full-text

before judging relevance from abstracts (or even titles!)

• The prevalence of this approach may vary among disciplines or packages

• Google Accelerator

4251

0011 0010 1010 1101 0001 0100 1011

G3. Dissemination style in teaching

Prof. A downloads 1 pdf, makes copies for students↓ under-counts 1 use for many

Prof. B sends link to publisher PDF to her 40 students↑ over-counts many uses of 1 article

Prof. C posts pdf on Electronic Reserve site↓ under-counts 1 use for many

Prof. D uses ‘FREE’ full text from PubMedCentral ↓↓ Way under-counts 0 use for many

Prof. E repeatedly uses publisher site to retrieve his own article↑↑ Way over-counts many uses for 0

http://www.springerlink.com/content/ygv3dbb9j8jj2vaq/

4251

0011 0010 1010 1101 0001 0100 1011

Specific Hazards

1. Determining cost2. Comparison to ILL cost3. Comparison across

Publishers4. Ignoring ‘by-title’ data5. Lack of benchmarks

4251

0011 0010 1010 1101 0001 0100 1011

COUNTER briefing

Counting Online Use of NeTworked Electronic Resources

-A standard & code of practice that enables comparison of usage statistics from different vendors

Components:Terminology & DefinitionsLayout & Format of reports (for journals & databases)

Processing of user inputDelivery frequency & availability periodTesting & Regular auditswww.projectcounter.org tinyurl.com/nxqvv

COMPLIANT

4251

0011 0010 1010 1101 0001 0100 1011

S1. Determining Cost

Overall cost per view=1 year’s cost / 1 year’s views

e.g. $58,600 publisher E-access fee 35,700 article views

$1.64 Cost per view? * $420,000 mandatory cost of subs (to

agent) for a subset of these same titles

$420K + $ 58.6K = $478,600 / 35,700 = $13.40

4251

0011 0010 1010 1101 0001 0100 1011

Overall cost per view by Subs Type

Cost per view by access type

$30.51

$0.81

$13.41

$0

$5

$10

$15

$20

$25

$30

$35

All Titles Subscribed Titles[n=192]

Unsubscribed(Leased) Titles

[n=345]

Access Type

Co

st p

er v

iew

[n=537]

$1.64

4251

0011 0010 1010 1101 0001 0100 1011

S2. Comparison with ILL

Package CPV = $13.40

What does this tell us?

• Is it High? Low?

• Better than ILL?

• How does it compare with other packages?

4251

0011 0010 1010 1101 0001 0100 1011

S3. Cross-package comparison

So Pkg 1 is a better value than Pkg 3?

pkgIDTotal Use SubsCost UnSubsCost Overall PPV1 140048 $1,652,000 $182,000 $13.102 20341 $333,000 $10,000 $16.863 13572 $282,000 $21,000 $22.33

CPV

It might not be…

4251

0011 0010 1010 1101 0001 0100 1011

Variation in use by format

Davis and Price, 2006

4251

0011 0010 1010 1101 0001 0100 1011

html to pdf Ratios vary widely for these packages

352 5684066

13004

32688

48047

0

10000

20000

30000

40000

50000

1 2 3Package

# o

f vi

ews

html viewspdf downloads

1:1.3 1:23

1:12

How many pdfs in Pkg 1 are duplicates of html views?

4251

0011 0010 1010 1101 0001 0100 1011

Live Link

4251

0011 0010 1010 1101 0001 0100 1011

S3. Package value revisited

pdf requests only tell a different story!

pkgID Est. pdf Use SubsCost UnSubsCost Overall PPP1 83469 $1,652,000 $182,000 $21.972 18734 $333,000 $10,000 $18.313 13287 $282,000 $21,000 $22.80

CPP

pkgIDTotal Use SubsCost UnSubsCost Overall PPV1 140048 $1,652,000 $182,000 $13.102 20341 $333,000 $10,000 $16.863 13572 $282,000 $21,000 $22.33

CPV

vs.

4251

0011 0010 1010 1101 0001 0100 1011

Response: COUNTER filterA unique article filter provides

new metric:number of successful unique article

requests in a session Vendor % Reduction

(Group 2) Publisher

A25.14%

Publisher B

25.50%

Publisher C

21.40%

Publisher D

35.65%

Publisher E

47.36%

Need to be applied to Specific institutions/

interface configurations

4251

0011 0010 1010 1101 0001 0100 1011

Reality Check

Should we expect cost per use to be equivalent among packages?

• Quality• Business Model

– For Profit vs Cost Recovery

• Exposure in Google Scholar• Title list accuracy• Backfile access ASSUMPTIONS

4251

0011 0010 1010 1101 0001 0100 1011

S4. Ignoring by-title data

4251

0011 0010 1010 1101 0001 0100 1011

0

100

200

300

400

500

600

700

800

900

1000

0 100 200 300 400 500 600 700

Titles (ordered by use for each institution)

Fu

ll t

ext

arti

cle

view

s

CUC SubColl

LLU SubColl

PEP SubColl

CLU

Selected SCELC Subject Collection use (2003)

Cutting off the long tail…

4251

0011 0010 1010 1101 0001 0100 1011

1

10

100

1000

0 100 200 300 400 500 600 700

Titles (ordered by use for each institution)

Fu

ll t

ext

arti

cle

view

s

CUC SubColl

LLU SubColl

PEP SubColl

CLU

Selected SCELC Subject Collection use (2003)

Before…

4251

0011 0010 1010 1101 0001 0100 1011

1

10

100

1000

0 100 200 300 400 500 600 700

Titles (ordered by use for each institution)

Fu

ll te

xt a

rtic

le v

iew

s

CUC SubColl

LLU SubColl

PEP SubColl

CLU

CUC STL

PEP STL

LLU STL

CLU STL

Selected SCELC STL vs. Subject Collection use

After…

4251

0011 0010 1010 1101 0001 0100 1011SCELC SD STL members

Proportion of titles containing 80% of Use

19.4

19.3

25

16

13.9

15.9

11.4

9.8

34.9

34.6

32.6

36.5

33.0

38.8

32.1

26.4

0 10 20 30 40 50 60 70 80 90 100

CUC

LLU

PEP

UOP

USD

CLU

MLS

MSM

Mem

ber

Proportion

SubCollSTL

Before Collaboration After Collaboration

4251

0011 0010 1010 1101 0001 0100 1011SCELC Package 'W' Overall Price per Use

$0.00

$10.00

$20.00

$30.00

$40.00

$50.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Member (Sorted by decreasing spend)

Pri

ce p

er f

ull

text

art

icle

vie

w

Use

dat

a n

ot

aval

iab

le

Consortium

S5. Lack of Benchmarks

4251

0011 0010 1010 1101 0001 0100 1011SCELC Package 'W' Overall Price per Use

$0.00

$10.00

$20.00

$30.00

$40.00

$50.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Member (Sorted by decreasing spend)

Pri

ce p

er f

ull

text

art

icle

vie

w

Use

dat

a n

ot

aval

iab

le

Consortium

S5. Lack of Benchmarks

4251

0011 0010 1010 1101 0001 0100 1011SCELC Package 'W' Overall Price per Use

$0.00

$10.00

$20.00

$30.00

$40.00

$50.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Member (Sorted by decreasing spend)

Pri

ce p

er f

ull

text

art

icle

vie

w

Use

dat

a n

ot

aval

iab

le

Consortium

S5. Lack of Benchmarks

4251

0011 0010 1010 1101 0001 0100 1011SCELC Package 'W' Overall Price per Use

$0.00

$10.00

$20.00

$30.00

$40.00

$50.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Member (Sorted by decreasing spend)

Pri

ce p

er f

ull

text

art

icle

vie

w

Use

dat

a n

ot

aval

iab

le

Consortium

S5. Lack of Benchmarks

4251

0011 0010 1010 1101 0001 0100 1011

Consortial benchmarking

SCELC Package 'E' Price per Use by Member

$0.00

$5.00

$10.00

$15.00

$20.00

$25.00

$30.00

$35.00

$40.00

$45.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Member

Pri

ce p

er f

ull

tex

t ar

ticl

e re

qu

est

20052006

4251

0011 0010 1010 1101 0001 0100 1011

Recommendations

1. Unsure you have the right cost2. Be wary of cross-publisher

comparison– Consider both overall and pdf use

3. For single package evaluation:– Look at patterns at title level– Benchmark vs Consortium or

Peers

4251

0011 0010 1010 1101 0001 0100 1011

Support from COUNTER

o Indication of subs type (Subs vs Lease)

Unique article filter to mitigate interface & linking effects

o Separation of backfile dataBy title dataSingle Password consortium access to

aggregate and by-institution statisticsMuch more…

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

A ‘local’ analogy

4251

0011 0010 1010 1101 0001 0100 1011SCELC SD USE 2005-2006

0

50

100

150

200

250

300

350

400

450

BRN CLU COH CUC LLU LMU MLS MSM PEP SLK SRI UOP USC USD WST

Th

ou

san

ds

Member

Fu

ll t

ext

arti

cle

view

s

2005

2006

4251

0011 0010 1010 1101 0001 0100 1011

SCELC SD Elsevier PPU 2005-2006

$-

$5.00

$10.00

$15.00

$20.00

$25.00

BR

N

CL

U

CO

H

CU

C

LL

U

LM

U

ML

S

MS

M

PE

P

SL

K

SR

I

UO

P

US

C

US

D

WS

T

ME

AN

Member

Pri

ce

pe

r fu

ll te

xt

art

icle

vie

w

2005

2006

Pri

ce d

ata

not

yet

avai

labl

e

Pri

ce d

ata

not

yet

avai

labl

e

Pri

ce d

ata

not

yet

avai

labl

e

Pri

ce d

ata

not

yet

avai

labl

e

4251

0011 0010 1010 1101 0001 0100 1011SCELC Package 'W' Overall Price per Use

$0.00

$10.00

$20.00

$30.00

$40.00

$50.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Member (Sorted by decreasing spend)

Pri

ce p

er f

ull

text

art

icle

vie

w

Use

dat

a n

ot

aval

iab

le

Consortium

S5. Lack of Benchmarks

4251

0011 0010 1010 1101 0001 0100 1011

New ways to answer classic questions1) Which titles should be in our

collections?

2) Which titles should we cancel?

3) (Which titles should we add?)

4) Is this collection a good value?

4251

0011 0010 1010 1101 0001 0100 1011

Q1. Which titles should be in our collection? Big Deal’ E-journal package benefit:

added titles

– Pre-packaged subject collections?

– Consortial unique title list?

– eUsage-based consortial shared title list•Includes highest use unsubscribed titles

from each institution

•List can be adjusted periodically to meet changing needs and use patterns

•Returns title-by-title control to libraries

4251

0011 0010 1010 1101 0001 0100 1011

Which titles should be in our shared collection?

Building the list:1) Compiled e-Usage by institution2) Removed Subs title use from each

institutions use data3) Sorted by total use & calculated

cumulative use (Road Hazard)

4251

0011 0010 1010 1101 0001 0100 1011

Example of Cumulative Use

4251

0011 0010 1010 1101 0001 0100 1011

Which titles should be in our shared collection?

Building the list:1) Compiled e-Usage by institution2) Removed Subs title use from each institutions use

data3) Sorted by total use & calculated cumulative use 4) For each institution, guaranteed inclusion of:

1) A set representing a big chunk of cumulative use (66-80%)

2) Every title viewed more than x / month (1-4)

5) As a group, agreed on further title cuts based on price per consortial view

Result: Libraries saved from 10-60% on the collection though a couple experienced price increases

4251

0011 0010 1010 1101 0001 0100 1011

Q1. Which titles should we share? A: not the Unique Title List…

For more detail see: http://tinyurl.com/lte96

4251

0011 0010 1010 1101 0001 0100 1011

Q2. Which titles should we cancel?

2003 JR1479 rows

2006 JR1799 rows

2005 JR1671 rows

2004 JR1552 rows

Remove Duplicated

Titles (backfile entries, sub-

titles, incomplete

splits, backfill ISSNs)

2003 JR1465 titles

2006 JR1545 titles

2005 JR1530 titles

2004 JR1485 titles 2003-2006

# of FT VIEWS545 Titles

Subscribed Title List with price48 Titles

Select Query

Add titles cascading

backward to allow

complete 3yr use queryA

CT

ION

1560

85

2003 JR1545 titles

2006 JR1545 titles

2005 JR1545 titles

2004 JR1545 titles

Unique Title identifier

NE

ED Every title

represented every year

48 Subs Titles

with 3-yr Use & Price

Find UnmatchedQuery

Combined Data

497 Un-Subs

Titles with 3-yr

Use

Select Query to join stats

from all years

4251

0011 0010 1010 1101 0001 0100 1011

Q2. Which titles should we cancel?

2003 JR1479 rows

2006 JR1799 rows

2005 JR1671 rows

2004 JR1552 rows

Remove Duplicated

Titles (backfile entries, sub-

titles, incomplete

splits, backfill ISSNs)

2003 JR1465 titles

2006 JR1545 titles

2005 JR1530 titles

2004 JR1485 titles 2003-2006

# of FT VIEWS545 Titles

Subscribed Title List with price48 Titles

Select Query

Add titles cascading

backward to allow

complete 3yr use queryA

CT

ION

1560

85

2003 JR1545 titles

2006 JR1545 titles

2005 JR1545 titles

2004 JR1545 titles

Unique Title identifier

NE

ED Every title

represented every year

48 Subs Titles

with 3-yr Use & Price

Find UnmatchedQuery

Combined Data

497 Un-Subs

Titles with 3-yr

Use

Select Query to join stats

from all years

4251

0011 0010 1010 1101 0001 0100 1011

Q2. Which titles should we cancel?

2003 JR1479 rows

2006 JR1799 rows

2005 JR1671 rows

2004 JR1552 rows

Remove Duplicated

Titles (backfile entries, sub-

titles, incomplete

splits, backfill ISSNs)

2003 JR1465 titles

2006 JR1545 titles

2005 JR1530 titles

2004 JR1485 titles 2003-2006

# of FT VIEWS545 Titles

Subscribed Title List with price48 Titles

Select Query

Add titles cascading

backward to allow

complete 3yr use queryA

CT

ION

1560

85

2003 JR1545 titles

2006 JR1545 titles

2005 JR1545 titles

2004 JR1545 titles

Unique Title identifier

NE

ED Every title

represented every year

48 Subs Titles

with 3-yr Use & Price

Find UnmatchedQuery

Combined Data

497 Un-Subs

Titles with 3-yr

Use

Select Query to join stats

from all years

4251

0011 0010 1010 1101 0001 0100 1011

(Q3. Which titles should we add?)

• What do turnaways mean?• Pay-per-view by title (not separate

from licensed?)• Degree of ‘rights transparency’ will

affect• Don’t know that counter can help

much here –except through enabling consortial/peer benchmarking

4251

0011 0010 1010 1101 0001 0100 1011

Thoughts on the COUNTER standard

• Librarians manage subscribed & unsubscribed collections separately, we need to be able to divide easily

• Usage should be reported by paid units– Since backfiles paid separately, require separate

(or at least distinguishable) reporting– If split titles are subscribed as a unit, then report

that way• Aggregation of multi-year data is a challenge• Caution is critical when comparing acrosscollections: linking tools may skew the statistics