Crossmark for Researchers

Cite With Certainty!15 February 2014

AAAS Chicago, IL Business Development & Marketing

@meyercarol #AAASmtg

Carol Anne Meyer

Hello, welcome, thank you for coming today, etc. I am... Today I will to introduce and explain the CrossMark service from CrossRef, the people who bring you DOI reference linking in scholarly articles. This session will give you information on what CrossMark and how it can help you cite with certainty

So in order to explain how CrossMark has come about, I’m going to start with a couple of fairly simple and statements.

Content changes

So in order to explain how CrossMark has come about, I’m going to start with a couple of fairly simple and statements.

Content changesWhen it does, readers need to

know

So in order to explain how CrossMark has come about, I’m going to start with a couple of fairly simple and statements.

ADD BITS ON WHAT IS CROSSREF

Content changes

It’s easy to think that once something is published that’s it - the version of record is out there and that’s how it’ll stay. But we know it’s not as simple as that....

“The Web is by nature an interactive environment, yet online journals are mostly static, befitting their traditional role as a never-changing scholarly record.”

The idea that journals should be published and preserved in amber doesn’t work in a web world.

“The Web is by nature an interactive environment, yet online journals are mostly static, befitting their traditional role as a never-changing scholarly record.”

ADD NEW AMBER PICTURE HERE !The idea that journals should be published and preserved in amber doesn’t work in a web world.

When content changes, readers need to know

and when this content changes readers need to know that it has changed. It could be that an update adds extra data or background information to an article, but it could potentially be more serious, with corrections to information that could alter follow-on research or even treatment. So it’s important that this information gets out there.

The way we are communicating corrections to readers is archaic. This is a real photo of a correction notice for a book from a bulletin board!

Nature 478, 26-28 (2011) Science publishing: The trouble with retractions http://dx.doi.org/10.1038/478026a

There is also the big issue of trust for scholarly literature. Changes in content, if handled correctly, can enhance trust but if they aren’t handled well, they can undermine trust. There has been quite a bit in the news lately about how scientific fraud may be on the rise, with suspected increases in plagiarism and other types of misconduct. An some studies have confirmed this. A news item in Nature last year confirmed the numbers and highlighted a 10-fold increase in retraction notices which far outpaced the growth in the number of articles.

errata corrigenda

updates enhancements

withdrawals retractions

new editions protocol updates

notices of concern, etc.

But it’s not just retractions that are a concern. Corrections are more common, and in the online world there are growing opportunities to enhance content, perhaps by adding source data or supplementary material after publication of the original article. It’s fair to say that the majority of content won’t change, but some of it will.. So two main points that CrossMark is addressing - First: it’s time for the idea of the “final version” of an article to rest in peace. There is no final version for content. Readers, and often publishers themselves, have a mindset that once the “version of record” is published that’s the end of the story and the end of the publisher’s role. In all the recent debates about whether and how scholarly publishers add value I didn’t see any mention of the role publishers have in providing ongoing maintenance and stewardship of content.

ADD LINK TO NISO DOCUMENT. !released a working paper on Journal article Versions. Publishers are concerned with versions starting with the Version of Record and beyond.

Version of RecordEnhanced VoRCorrected VoR

ADD LINK TO NISO DOCUMENT. !released a working paper on Journal article Versions. Publishers are concerned with versions starting with the Version of Record and beyond.

It’s time for the idea of a “final version” of an article to rest in peace.

How Does a Researcher Know What to Trust?

• Author name

• Affiliation

• How many times something has been cited

• On a reputable web site (publisher or journal brand)

• Ratings, Comments

To determine the accessibility of retracted articles residing on non-publisher websites and in personal libraries. Non-publisher websites provided 321 publicly accessible copies for 289 retracted articles: 304 (95%) copies were the publisher' versions, and 13 (4%) were final manuscripts. PubMed Central had 138 (43%) copies; educational websites 94 (29%); commercial websites 24 (7%); advocacy websites 16 (5%); and institutional repositories 10 (3%). Just 15 (5%) full-article views included a retraction statement. Personal Mendeley libraries contained records for 1,340 (75%) retracted articles, shared by 3.4 users, on average.

To determine the accessibility of retracted articles residing on non-publisher websites and in personal libraries. Non-publisher websites provided 321 publicly accessible copies for 289 retracted articles: 304 (95%) copies were the publisher' versions, and 13 (4%) were final manuscripts. PubMed Central had 138 (43%) copies; educational websites 94 (29%); commercial websites 24 (7%); advocacy websites 16 (5%); and institutional repositories 10 (3%). Just 15 (5%) full-article views included a retraction statement. Personal Mendeley libraries contained records for 1,340 (75%) retracted articles, shared by 3.4 users, on average.

http://dx.doi.org/10.3163/2F1536-5050.100.3.008

To determine the accessibility of retracted articles residing on non-publisher websites and in personal libraries. Non-publisher websites provided 321 publicly accessible copies for 289 retracted articles: 304 (95%) copies were the publisher' versions, and 13 (4%) were final manuscripts. PubMed Central had 138 (43%) copies; educational websites 94 (29%); commercial websites 24 (7%); advocacy websites 16 (5%); and institutional repositories 10 (3%). Just 15 (5%) full-article views included a retraction statement. Personal Mendeley libraries contained records for 1,340 (75%) retracted articles, shared by 3.4 users, on average.

A Little Quiz: Does this article have corrections?

A Little Quiz: Does this article have corrections?

Does this article have corrections?

One of these is consistency. This article in PNAS has a correction. It’s flagged over here to the right of the abstract so that you can spot it.

Does this article have corrections?

One of these is consistency. This article in PNAS has a correction. It’s flagged over here to the right of the abstract so that you can spot it.

But this one is a bit more subtle.  There’s nothing in the left of right hand columns, but instead this publisher has chosen to site the correction up at the top of the article here.  

But this one is a bit more subtle.  There’s nothing in the left of right hand columns, but instead this publisher has chosen to site the correction up at the top of the article here.  

And what about this one? Nothing obvious at the top of the page, or in the tool bars on the right...

...but if you scroll down the page a bit here’s a correction located under the “related articles” heading.

...but if you scroll down the page a bit here’s a correction located under the “related articles” heading.

And then you have content that’s being held offline - here’s the PDF of the article we were just looking at. If you’ve downloaded this to your laptop or device you’ve got absolutely no means to know that there’s a correction that has been issued for this article. You could go back to look at it weeks or months later and you’d be completely oblivious to any updates or changes in its status.

Which leads to a second problem, which is that there is often more than one version of an article available. Here we have an article from the Journal of Surgical Research which was retracted because it was found to contain plagiarised material. On the publisher’s site it’s flagged pretty clearly as retracted up here in the article title...

Which leads to a second problem, which is that there is often more than one version of an article available. Here we have an article from the Journal of Surgical Research which was retracted because it was found to contain plagiarised material. On the publisher’s site it’s flagged pretty clearly as retracted up here in the article title...

If you search for this article in Google Scholar, however, the publisher’s site isn’t the first to appear - in fact it’s the fourth listing

The first result is an information sharing site for doctors where someone has posted the abstract, and here there’s no mention of the retraction....

Researcher Concerns

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• Users not always clear which version of a document they are reading.

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• Users not always clear which version of a document they are reading.

• Google and Google Scholar search results return multiple versions.

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• Users not always clear which version of a document they are reading.

• Google and Google Scholar search results return multiple versions.

• Librarians do not have time to track post-publication changes at the article level.

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• Users not always clear which version of a document they are reading.

• Google and Google Scholar search results return multiple versions.

• Librarians do not have time to track post-publication changes at the article level.

• Readers might cite “incorrect” versions instead of the maintained Version of Record

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• Users not always clear which version of a document they are reading.

• Google and Google Scholar search results return multiple versions.

• Librarians do not have time to track post-publication changes at the article level.

• Readers might cite “incorrect” versions instead of the maintained Version of Record

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Researcher Concerns• UPDATE THESE WITH RESULTS FROM RESEARCH FOCUS

GROUPS

• Users not always clear which version of a document they are reading.

• Google and Google Scholar search results return multiple versions.

• Librarians do not have time to track post-publication changes at the article level.

• Readers might cite “incorrect” versions instead of the maintained Version of Record

• on point 3: They do teach patrons the importance of such notifications and where to look for them.

Specific Correction Problems Mentioned

• Interlibrary loan

• Downloaded PDFs

• Printed articles.

Lack of awareness of corrections or updates particularly problematic in medicine and related fields.

The second is PubMed, and the retraction has made it on to the Pub Med copy, although it’s not as obvious as it is on the publisher’s site - it’s not part of the article title but a separate link below.

The second is PubMed, and the retraction has made it on to the Pub Med copy, although it’s not as obvious as it is on the publisher’s site - it’s not part of the article title but a separate link below.

But what if you’d come across the abstract somewhere else? Maybe through CiteULike, where again there’s no mention of the retraction.

Or there could well be a copy in the author’s institutional repository... With all of these options there’s a reasonable chance that the reader isn’t necessarily going to see the correction or retraction that the publisher has issued. Also PDF problem.

Or there could well be a copy in the author’s institutional repository... With all of these options there’s a reasonable chance that the reader isn’t necessarily going to see the correction or retraction that the publisher has issued. Also PDF problem.

These are all problems that we’re looking to address by launching CrossMark

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

A logo that identifies a publisher-maintained copy of a piece of content

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

A logo that identifies a publisher-maintained copy of a piece of contentClicking the logo tells you

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

A logo that identifies a publisher-maintained copy of a piece of contentClicking the logo tells you

Whether there have been any updates

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

A logo that identifies a publisher-maintained copy of a piece of contentClicking the logo tells you

Whether there have been any updates

If this instance is being maintained by the publisher

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

A logo that identifies a publisher-maintained copy of a piece of contentClicking the logo tells you

Whether there have been any updates

If this instance is being maintained by the publisher

Where the publisher-maintained version is

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

A logo that identifies a publisher-maintained copy of a piece of contentClicking the logo tells you

Whether there have been any updates

If this instance is being maintained by the publisher

Where the publisher-maintained version is

Other important publication record information

What is CrossMark?

So CrossMark. At its simplest it’s a logo that publishers will apply to content that they publish. When a reader clicks on the logo they will quickly and easily be able to tell: The best way to explain it is to show some examples.

I’ll start with the most useful common scenario. We’re looking at a PDF from the Journal of Applied Crystallography. This came from my hard drive. Or I downloaded it from the author’s web site. Or was it my university’s institutional repository? Maybe somebody emailed me a copy? No, wait, I think this was from my Mendeley account. And when was that? !At any rate, you see there is a CrossMark logo in the upper left corner. Providing I am online, when I click on the logo it will pop up a webpage...

with a pop-up dialogue box giving the latest status. This is what most people will see - confirmation that the document is up to date, the CrossRef DOI link that will always point to the publisher-maintained copy, and a link to the publisher’s policies. There are no updates. This time.... (click) And the box also tells the reader that Future updates - if any - will be listed below, so getting them used to the idea that if changes happen, this is where they can find them. But what if there had been a correction?

Here’s another PDF. (For the record, this example is a mock-up from the fictitious Journal of Psychoceramics.) The CrossMark logo in a different place (click), though it works the same way. I just click on the logo. ..

Here, clicking on the logo brings up the same CrossMark dialog box... but I discover that there was a clarification issued for this document. It gives a link to the correction. !All I need to do is click on the CrossRef DOI link (click) to go to the update.

Of course, these work the same way on publisher’s HTML pages. (Click) You see a real article from The Proceedings of The Royal Society B on Royal Society’s website. The CrossMark logo appears just above the article title here.

Of course, these work the same way on publisher’s HTML pages. (Click) You see a real article from The Proceedings of The Royal Society B on Royal Society’s website. The CrossMark logo appears just above the article title here.

Rolling your mouse over the logo brings up a text box that says Click to get updates and verify authenticity. And then when you click on the logo

You see the CrossMark dialog box. And this is what most people will see - confirmation that the document is up to date, the CrossRef DOI link that will always point to the publisher-maintained copy, and a link to the publisher’s policies. The box also tells the reader that Future updates - if any - will be listed below, so getting them used to the idea that if changes happen, this is where they can find them.

The second example is of a corrected article from another of our pilot publishers, the International Union of Crystallographers. Here, clicking on the logo brings up the same CrossMark dialog box...

The second example is of a corrected article from another of our pilot publishers, the International Union of Crystallographers. Here, clicking on the logo brings up the same CrossMark dialog box...

..but with information that alerts the reader to changes. Updates are available for this document. It says that there is a correction and gives a link to the correction.

You may have noticed in that previous example that there is an additional tab appearing in the dialogue box at the top here - the record tab.

You may have noticed in that previous example that there is an additional tab appearing in the dialogue box at the top here - the record tab.

This is where you can show additional metadata about the piece of content if you choose to do so. The publisher decides what to put here and can use these fields to define publication practices. You don’t have to populate this tab at all if you prefer not to, and if you don’t supply an additional metadata the tab simply won’t show. The fields are defined and labelled by the publisher, and there can be as many or as few as you choose. This particular data from another of our pilot participants, the International Union of Crystallography, and you can see that they are sharing some really useful information on the copyright, review process and publication history. Also useful for FundRef!

This is where you can show additional metadata about the piece of content if you choose to do so. The publisher decides what to put here and can use these fields to define publication practices. You don’t have to populate this tab at all if you prefer not to, and if you don’t supply an additional metadata the tab simply won’t show. The fields are defined and labelled by the publisher, and there can be as many or as few as you choose. This particular data from another of our pilot participants, the International Union of Crystallography, and you can see that they are sharing some really useful information on the copyright, review process and publication history. Also useful for FundRef!

ADD FUNDREF DESCRIPTION

What kind of Publication Record information could be available?

Try to find visual examples of these. These are a few of the other possible pieces of information that have come up when talking with publishers. CrossRef isn’t going to advise on what publishers should display in the record box, but we expect that communities of interest may develop guidelines or best practices within different areas. There’s already a group of publishers discussing how best to display funding and grant information, for example.

What kind of Publication Record information could be available?

Funding disclosures

Conflict of interest statements

Publication history (submission, revision and accepted dates)

Location of data deposits or registries

Peer review process used

CrossCheck plagiarism screening

License types

and more...

Try to find visual examples of these. These are a few of the other possible pieces of information that have come up when talking with publishers. CrossRef isn’t going to advise on what publishers should display in the record box, but we expect that communities of interest may develop guidelines or best practices within different areas. There’s already a group of publishers discussing how best to display funding and grant information, for example.

CrossMark data will be freely available and machine readable and query-able, so could potentially be used in search results to flag content that has status verification and possible additional information, although this is something that we’ve just seen implemented in Microsoft Academic Search. We have had some conversations with librarians about using CrossMark data to populate link resolvers by pulling back relevant information, and also with other third parties such as bibliographic management systems who might be able to pull status updates into users reference lists and personal libraries.

CrossMark data will be freely available and machine readable and query-able, so could potentially be used in search results to flag content that has status verification and possible additional information, although this is something that we’ve just seen implemented in Microsoft Academic Search. We have had some conversations with librarians about using CrossMark data to populate link resolvers by pulling back relevant information, and also with other third parties such as bibliographic management systems who might be able to pull status updates into users reference lists and personal libraries.

Microsoft  Academic  Search  implemented  CrossMark  on  their  platform  in  early  February  and  are  displaying  the  CrossMark  logo  on  relevant  content  within  their  index.  This  is  a  useful  development  in  terms  of  being  able  to  publicise  CrossMark  to  affiliates  and  show  ways  in  which  the  CrossMark  data  can  be  used  to  identify  the  publisher  version  of  a  piece  of  content.  Inera’s  eXstyles  have  also  just  announced  that  they’re  supporting  CrossMark.  If  you’re  using  that  tool  for  your  references,  eXtyles will now provide a warning if a reference has a CrossMark record that indicates it has been “retracted,” “withdrawn,” or “removed.”  !

Microsoft  Academic  Search  implemented  CrossMark  on  their  platform  in  early  February  and  are  displaying  the  CrossMark  logo  on  relevant  content  within  their  index.  This  is  a  useful  development  in  terms  of  being  able  to  publicise  CrossMark  to  affiliates  and  show  ways  in  which  the  CrossMark  data  can  be  used  to  identify  the  publisher  version  of  a  piece  of  content.  Inera’s  eXstyles  have  also  just  announced  that  they’re  supporting  CrossMark.  If  you’re  using  that  tool  for  your  references,  eXtyles will now provide a warning if a reference has a CrossMark record that indicates it has been “retracted,” “withdrawn,” or “removed.”  !

CrossMark logos will not be applied to prepublication content —no longer true

• Anything with a CrossRef DOI can have a CrossMark logo

CrossMark logos will not be applied to prepublication content —no longer true

• Anything with a CrossRef DOI can have a CrossMark logo

•

CrossMark logos will not be applied to prepublication content —no longer true

• Now

UPDATE THESE NUMBERS

•Over 385,000 CrossMark deposits since launch with 3,700 plus updates

UPDATE THESE NUMBERS

•Over 385,000 CrossMark deposits since launch with 3,700 plus updates

•Working with over 35 publishers on CrossMark implementation

UPDATE THESE NUMBERS

What do Researchers have to do?

So now let’s take a look at what you actually have to do to use CrossMark.

ClickSeriously….that’s it

1. CrossMark Policy Page

Explains CrossMark, commitment to maintain the content

Explain publisher policies on corrections, retractions, etc.

Define any fields for the Publication Record Tab

Has a CrossRef DOI for persistent linking

A CrossMark policy page on the publisher’s website explains that the publisher is a CrossMark member committed to maintaining the content that it publishes. It will link to or explain its policies on corrections and retractions, and define any of the Publication Record in the Record Tab. This page will have its own CrossRef DOI so that it can be linked to.

Here’s an example from Wiley -, it has links to the publisher’s guidelines for authors

5. Viewing the CrossMark Logo

REWRITE FOR RESEARCHER AUDIENCE You need to display the CrossMark logo on the article landing page, ideally situated as close as possible to the article title, and outside of access control. On the CrossMark pilot site you will find information on the widget that can be inserted into HTML pages to add the logo and link it to the CrossMark dialogue box. The CrossMark logo should also be added to PDFs. Black and white versions of the logo are available if required. We have another tool on the Support site that will help you to add the logo and link to PDFs. We’re not asking pilot publishers to add logos to PDFs just yet because there’s an element of permanence once something is in a PDF and downloaded so we want to complete final testing, but CrossMarks in PDFs will be a very important part of the service once it’s launched. There is no requirement to add CrossMarks to backfile if the publisher doesn’t want to do so.

5. Viewing the CrossMark Logo

On HTML article landing pages

REWRITE FOR RESEARCHER AUDIENCE You need to display the CrossMark logo on the article landing page, ideally situated as close as possible to the article title, and outside of access control. On the CrossMark pilot site you will find information on the widget that can be inserted into HTML pages to add the logo and link it to the CrossMark dialogue box. The CrossMark logo should also be added to PDFs. Black and white versions of the logo are available if required. We have another tool on the Support site that will help you to add the logo and link to PDFs. We’re not asking pilot publishers to add logos to PDFs just yet because there’s an element of permanence once something is in a PDF and downloaded so we want to complete final testing, but CrossMarks in PDFs will be a very important part of the service once it’s launched. There is no requirement to add CrossMarks to backfile if the publisher doesn’t want to do so.

5. Viewing the CrossMark Logo

On HTML article landing pages

In PDF articles

REWRITE FOR RESEARCHER AUDIENCE You need to display the CrossMark logo on the article landing page, ideally situated as close as possible to the article title, and outside of access control. On the CrossMark pilot site you will find information on the widget that can be inserted into HTML pages to add the logo and link it to the CrossMark dialogue box. The CrossMark logo should also be added to PDFs. Black and white versions of the logo are available if required. We have another tool on the Support site that will help you to add the logo and link to PDFs. We’re not asking pilot publishers to add logos to PDFs just yet because there’s an element of permanence once something is in a PDF and downloaded so we want to complete final testing, but CrossMarks in PDFs will be a very important part of the service once it’s launched. There is no requirement to add CrossMarks to backfile if the publisher doesn’t want to do so.

5. Viewing the CrossMark Logo

On HTML article landing pages

In PDF articles

REWRITE FOR RESEARCHER AUDIENCE You need to display the CrossMark logo on the article landing page, ideally situated as close as possible to the article title, and outside of access control. On the CrossMark pilot site you will find information on the widget that can be inserted into HTML pages to add the logo and link it to the CrossMark dialogue box. The CrossMark logo should also be added to PDFs. Black and white versions of the logo are available if required. We have another tool on the Support site that will help you to add the logo and link to PDFs. We’re not asking pilot publishers to add logos to PDFs just yet because there’s an element of permanence once something is in a PDF and downloaded so we want to complete final testing, but CrossMarks in PDFs will be a very important part of the service once it’s launched. There is no requirement to add CrossMarks to backfile if the publisher doesn’t want to do so.

How much does all this cost?

• for End Users and Librarians

• CrossRef Members underwrite the cost

How much does all this cost?

EXPLAIN BACKFILES?CURRENT

There is no requirement to add CrossMarks to your backfile if you prefer not to. !

...the first of the pilot participants to have CrossMark live was VGTU and you can see CrossMarks on their journal Business Theory and Practice.

The Royal Society has implemented CrossMark on all of their titles, going back to the start of 2011.

CrossMark is running on 1250 journals on Elsevier’s Science Direct.

http://www.crossref.org/crossmark/AboutParticipatingPubs.htm

Who? Participating publishers include:!!Amsterdam University Press!Scholar Science Journals!The Rockefeller University Press!World Bank!International Union of Crystallography (IUCr)*!Science Reviews 2000 Ltd!Vilnius Gediminas Technical University (VGTU)* !Philippine Association of Institutions for Research (PAIR)!Cambridge University Press !Mednet!Philosophy Documentation Centre*!John Wiley & Sons*!Riga Technical University!F1000 Research*!AIP*!Elsevier*!Turkish Online Journal of Design, Art and Communication (TOJDAC)!Bioscientifica!Michael Joanna Publishers!M2 Communications*!American Diabetes Association!The Royal Society*!Royal Society of Chemistry!International Engineering Journal!BMJ Journals*!Proceedings of the National Academy of Sciences!The Korean Physical Society!Medical University of Lublin!!!!

UPDATE THIS LIST

Any Questions?

cmeyer@crossref.orghttp://www.crossref.org/crossmark