University of Groningen Computerized adaptive testing in ...€¦ · 523226-L -bw-Bebber Processed on: 27-8-2018 PDF page: 3 Computerized adaptive testing in primary care: CATja PhD

University of Groningen

Computerized adaptive testing in primary care: CATjavan Bebber, Jan

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.

Document VersionPublisher's PDF, also known as Version of record

Publication date:2018

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):van Bebber, J. (2018). Computerized adaptive testing in primary care: CATja. University of Groningen.

CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.

Download date: 27-03-2021

https://research.rug.nl/en/publications/computerized-adaptive-testing-in-primary-care-catja(f8d3f5c4-844a-48c7-bb30-1eb9456b8b8f).html

523226-L-bw-Bebber523226-L-bw-Bebber523226-L-bw-Bebber523226-L-bw-BebberProcessed on: 27-8-2018Processed on: 27-8-2018Processed on: 27-8-2018Processed on: 27-8-2018 PDF page: 1PDF page: 1PDF page: 1PDF page: 1

Computerized adaptive testing in primary care: CATja

Jan van Bebber


The research presented in this thesis was funded by the Friesland Mental Health

Services (GGZ Friesland).

All rights reserved. No part of this publication may be reproduced or transmitted

in any form by any means, without permission of the author.

ISBN: 978-94-034-0914-6 (print version)

ISBN: 978-94-034-0913-9 (electronic version)

„Ach,“ sagte die Maus, „die Welt wird enger mit jedem Tag. Zuerst war sie so breit, dass ich Angst hatte,

ich lief weiter und war glücklich, dass ich endlich rechts und links in der Ferne Mauern sah, aber diese

langen Mauern eilen so schnell aufeinander zu, dass ich schon im letzten Zimmer bin, und dort im Winkel

steht die Falle, in die ich laufe.“ – „Du musst nur die Laufrichtung ändern,“ sagte die Katze und fraß sie.

(Die Kleine Fabel, Frank Kafka)1

1 With kind permission of Elena Moreno Sobrino, Calambac Verlag. Franz Kafka: Kleine Fabel. Graphic Novel. Illustriert von Elvira Calderón nach Vorlagen von José Guadalupe Posada. Dreisprachige Ausgabe: Deutsch/Englisch/Spanisch. ISBN 978-3-943117-79-0, S. 26.


Computerized adaptive testing in primary care: CATja

PhD thesis

to obtain the degree of PhD at the

University of Groningen

on the authority of the

Rector Magnificus Prof. E. Sterken

and in accordance with

the decision by the College of Deans.

This thesis will be defended in public on

24 September 2018 at 09.00

by

Jan van Bebber

born on 24.03.1979

in Xanten, Germany


Supervisors

Promotor

Prof. dr. R. R. Meijer

Copromotores

Dr. A. Wunderink

Dr. J.T.W. Wigman

Assessment committee

Prof. Dr. M. Y. Berger

Prof. Dr. R. Schoevers

Prof. dr. B. Veldkamp


Musipals

Amon Adonai Santos de Araújo Tobin (aka Amon Tobin, Cujo, & Two Fingers),

Vaughn Robert Squire (aka Sixtoo),

Simon Green (aka Bonobo),

Daniel Dumile (aka MF Doom),

Rio Reiser (Ton Steine Scherben),

Jason Swinscoe (The Cinematic Orchetra),

Joshua Paul Davis (aka DJ Shadow),

Hideaki Ishi (aka DJ Krush),

Kieran Hebden (aka Four Tet),

Matt Black and Jonathan More (aka Coldcut),

Ed Kowalczyk (Live),

Smudo & Thomas D,

Mike Lévy (aka Gesaffelstein),

Eddie Vedder,

Mike Skinner (The Streets),

Jack White,

James Di Salvio (Bran van 3000),

Otis Jackson Jr. (aka Madlib),

Jimmy Hendrix,

Jon Hokins,

Crispian Mills (Kula Shaker),

Robert Plant,

Lou Rhodes (Lamb),

Leonard Cohen,

José Padilla (Café del Mar).

Keith Caputo (Life of Agony),

Lewis Allan Reed (aka Lou Reed),

Robert Del Naja (Massive Attack),


Sascha Ring (aka Apparat),

Andrew Carthy (aka Mr Scruff),

Mykel Paul,

Justin Sullivan (New Model Army),

Nils Petter Molvær,

Mike Patton (Faith No More),

Vadim Peare (aka DJ Vadim & One Self),

José-Manuel Thomas Arthur Chao (Manu Chao & Mano Negra),

Beth Gibbons (Portishead),

Roger Waters (Pink Floyd),

Bobby Gillespie (Primal Scream),

Tom Yorke,

Ali Friend (Red Snapper),

Rodney Hylton Smith (Roots Manuva),

Saul Stacey Williams,

Bradley Nowell (Sublime),

Jeff Keith (Tesla),

Chris Robinson (The Black Crowes),

Tom Rowlands and Ed Simons (The Chemical Brothers),

Liam Howlett (The Prodigy),

Jake Wherry & Ollie Teeba (The Herbalizer),

Billy Corgan (The Smashing Pumpkins),

Karl Hyde & Rick Smith (Underworld),

Philippe Cohen Solal (GoTan project),

and all the other ninjas out there.


Contents

Chapter 1 ........................................................................................................................................... 1

Introduction ....................................................................................................................................... 1

1.1 Recent developments in western mental health care ........................................................................ 1

1.2 The mental health care system in the Netherlands ............................................................................ 2

1.3 Notification, recognition of mental illnesses, and treatment outcome .............................................. 3

1.4 Rationale for this thesis and the relation to existing eHealth diagnostic tools .................................. 3

1.5 Description of the online test battery CATja ....................................................................................... 4

1.6 Technical details CATja ........................................................................................................................ 7

1.7 Outline thesis ....................................................................................................................................... 7

1.8 References ........................................................................................................................................... 8

Chapter 2 ......................................................................................................................................... 13

The Prodromal Questionnaire: a case for IRT-based adaptive testing of psychotic experiences? ......... 13

2.1 Introduction ....................................................................................................................................... 14

2.1.1 At risk for psychosis .................................................................................................................... 14

2.1.2 The Prodromal Questionnaire .................................................................................................... 15

2.1.3 Computerized Adaptive Testing (CAT) and Item Response Theory (IRT) ................................... 15

2.1.4 Aims of this study ....................................................................................................................... 16

2.2 Methods ............................................................................................................................................ 16

2.2.1 Data collection design ................................................................................................................ 16

2.2.2 The two-parameter logistic model (2-PL) and its assumptions .................................................. 18

2.2.3 Model fit 2-PL and Local Dependence (LD) ................................................................................ 19

2.2.4 Differential item functioning (DIF) ............................................................................................. 19

2.2.5 Simulation of CAT-properties based on item parameters and observed response patterns: RDS ............................................................................................................................................................. 19

2.2.6 Criterion Validity ......................................................................................................................... 20

2.2.7 Instruments ................................................................................................................................ 20

2.2.8 Predictive Validity ....................................................................................................................... 21

2.3 Results ............................................................................................................................................... 21

2.3.1 Model fit 2-PL and Local Dependence ........................................................................................ 21

2.3.2 Differential Item Functioning (DIF) ............................................................................................. 23


2.3.3 Real Data Simulations ................................................................................................................. 23

2.3.4 Criterion Validity ......................................................................................................................... 24

2.3.4 Predictive Validity ....................................................................................................................... 25

2.4 Discussion .......................................................................................................................................... 26

2.5 Appendix ............................................................................................................................................ 28

2.5.1 Model fit revisited ...................................................................................................................... 28

2.5.2 Settings for Real Data Simulations in Firestar ............................................................................ 28

2.5.3 Usage of IRT-scores in linear models .......................................................................................... 28

2.5.4 A brief introduction to the CAT operation procedure ................................................................ 36

2.6 References ......................................................................................................................................... 37

Chapter 3 ......................................................................................................................................... 43

Identifying levels of general distress in first line mental health services: Can GP- and eHealth clients’ scores be meaningfully compared? ................................................................................................... 43

3.1 Introduction ....................................................................................................................................... 44

3.1.1 Background ................................................................................................................................. 44


3.2 Methods ............................................................................................................................................ 46

3.2.1 The Four-Dimensional Symptom Questionnaire (4DSQ): background information and existing research ............................................................................................................................................... 46

3.2.2. Participants ................................................................................................................................ 47

3.2.3 The Generalized Partial Credit Model (GPCM) ........................................................................... 47

3.2.4 Differential Item Functioning (DIF) and Multiple Group IRT analysis (MGIRT) .......................... 48

3.2.5 MGIRT Analyses .......................................................................................................................... 49

3.3 Results ............................................................................................................................................... 50

3.3.1 Sample Descriptives for both groups ......................................................................................... 50

3.3.2 IRT-analyses: GP clients .............................................................................................................. 51

3.3.3 IRT-analyses: EHealth clients ...................................................................................................... 53

3.3.4 Differential Item Functioning (DIF) ............................................................................................. 53

3.3.5 Summary..................................................................................................................................... 56

3.4 Discussion .......................................................................................................................................... 57

3.4.1 Main findings .............................................................................................................................. 57

3.4.2 Strengths, limitations and future research ................................................................................ 57

3.5 Appendix ............................................................................................................................................ 59


3.5.1 Technical information on the GPCM .......................................................................................... 60

3.5.2 Technical details DIF tests .......................................................................................................... 62

3.5.3 Detailed information on MGIRT analyses................................................................................... 62

3.6 References ......................................................................................................................................... 62

Chapter 4 ......................................................................................................................................... 67

Searching for the optimal number of response alternatives for the distress scale of the Four-Dimensional Symptom Questionnaire ............................................................................................... 67

4.1 Introduction ....................................................................................................................................... 68

4.1.1 The Four-Dimensional Symptom Questionnaire (4DSQ) ............................................................ 68

4.1.2 Optimal number of response alternatives: Existing research .................................................... 68


4.2 Methods ............................................................................................................................................ 70

4.2.1 Participants ................................................................................................................................. 70

4.2.2 Instruments ................................................................................................................................ 71

4.2.3 Item response theory ................................................................................................................. 72

4.2.4 Statistical analyses ...................................................................................................................... 74

4.3 Results ............................................................................................................................................... 75

4.3.1 Model fit and measurement precision ....................................................................................... 75

4.3.2 Convergent and discriminant validity ......................................................................................... 76

4.3.3 Predictive validity ....................................................................................................................... 77

4.4 Discussion .......................................................................................................................................... 78

4.4.1 Main findings and conclusions ................................................................................................... 78

4.4.2 Strengths and limitations ........................................................................................................... 79

4.4.3 Directions for future research .................................................................................................... 79

4.5 Appendix ............................................................................................................................................ 80

4.6 References ......................................................................................................................................... 83

Chapter 5 .........................................................................................................................................

Application of the Patient-Reported Outcomes Measurement Information System (PROMIS) item parameters for Anxiety and Depression in the Netherlands ...............................................................

5.1 Introduction ....................................................................................................................................... 9

5.1.1 The Patient-Reported Outcomes Measurement Information System ....................................... 9



5.2 Methods ............................................................................................................................................ 9

5.2.1 Participants ................................................................................................................................. 9

5.2.2 Instruments ................................................................................................................................ 9

5.2.3 Statistical analyses: Fit of item parameters ............................................................................... 9

5.2.4 Statistical analyses: Real Data Simulations ................................................................................ 9

5.2.5 The latent distributions of anxiety and depression in the Dutch general and Dutch clinical population ........................................................................................................................................... 9

5.3 Results ............................................................................................................................................... 9

5.3.1 Fit item parameters for the PROMIS Anxiety item bank ............................................................ 9

5.3.2 Fit item parameters for the PROMIS Depression item bank ...................................................... 9

5.3.3 How serious is misfit for practical decisions? Results Real Data Simulations ............................ 9

5.3.4 The latent distributions of anxiety and depression in the U.S. general population, the Dutch general population, and the Dutch clinical population ..................................................................... 10

5.4 Discussion ........................................................................................................................................ 10

5.4.1 Summary of main findings ........................................................................................................ 10

5.4.2 Practical implications and recommendations .......................................................................... 10

5.4.3 Strengths and limitations ......................................................................................................... 10

5.4.4 Directions for future research .................................................................................................. 10

5.5 Appendix .......................................................................................................................................... 10

5.6 References ....................................................................................................................................... 10

Chapter 6 ....................................................................................................................................... 11

Predicting Relapse and Outcome in First Episode Psychosis: Impact of Negative Symptoms, and Personalized Low Dose versus Maintenance Antipsychotic Treatment ............................................. 11

6.1 Introduction ..................................................................................................................................... 11

6.2 Methods .......................................................................................................................................... 11

6.2.1 Subjects .................................................................................................................................... 11

6.2.2 Assessments and definitions .................................................................................................... 11

6.2.3 Statistical analyses .................................................................................................................... 11

6.3 Results ............................................................................................................................................. 11

6.3.1 Number of relapses and functional outcome .......................................................................... 11

6.3.2 Predicting relapse ..................................................................................................................... 11

6.3.3 Predicting functional outcome ................................................................................................. 1


6.3.4 Disentangling the effects of baseline negative symptoms and treatment strategy on relapse risk ..................................................................................................................................................... 1

6.4 Discussion ........................................................................................................................................ 12

6.4.1 Main findings ............................................................................................................................ 12

6.4.2 Our results in the light of existing literature and implications for treatment of FEP patients. 12

6.5 References ....................................................................................................................................... 12

Chapter 7 ....................................................................................................................................... 1

The development of a smart screening device for primary health care ............................................. 1

7.1 Introduction ..................................................................................................................................... 13

7.1.1 Background ............................................................................................................................... 13

7.1.2 Aims of this study ..................................................................................................................... 13

7.2 Methods .......................................................................................................................................... 13

7.2.1 Developmental approach ......................................................................................................... 13

7.2.2 Computerized adaptive testing (CAT) ...................................................................................... 13

7.2.3 Content of the alpha version of CATja ..................................................................................... 13

7.2.4 Sample characteristics .............................................................................................................. 13

7.2.5 Statistical analyses .................................................................................................................... 13

7.3 Results ............................................................................................................................................. 13

7.4 Discussion ........................................................................................................................................ 13

7.5 Appendix .......................................................................................................................................... 13

7.6 References ....................................................................................................................................... 13

Chapter 8 ....................................................................................................................................... 1

General Discussion ......................................................................................................................... 1

8.1 Main findings of this thesis .............................................................................................................. 1

8.2 Generalizability of research findings ............................................................................................... 14

8.2.1 Measurement invariance between application modes: GP clients/paper and pencil versus eHealth clients ................................................................................................................................... 14

8.2.2 Relevance of baseline negative symptoms for relapse prevention and long-term functional outcome ............................................................................................................................................ 14

8.3 Lessons learned ............................................................................................................................... 14

8.3.1 Evaluation of model fit and Differential Item Functioning ....................................................... 14

8.3.2 Never underestimate people’s resistance to change when implementing new tools............. 14


8.4 Limitations ....................................................................................................................................... 14

8.5 Future Research............................................................................................................................... 14

8.5.1 The incremental value of CATja: Improved placement and recovery? .................................... 14

8.5.2 Future research and improvements 4DSQ ............................................................................... 14

8.5.3 The intercorrelations of CATja’s domains of psychopathology and constructs of positive psychology ......................................................................................................................................... 14

8.6 The future of CATja.......................................................................................................................... 14

8.7 References ....................................................................................................................................... 1

Samenvatting ................................................................................................................................. 15

Dankwoord .................................................................................................................................... 16

Curriculum Vitae ............................................................................................................................ 16

Documents

University of Groningen Computerized adaptive testing in ...€¦ · 523226-L -bw-Bebber Processed on: 27-8-2018 PDF page: 3 Computerized adaptive testing in primary care: CATja PhD