Public Consultation on the Draft CAMHS - ISD Scotland · steps for implementation. While the consultation is now closed, the combined publication draft remains available via the ISD

1

FindingsfromPublicConsultationontheDraftCAMHSWaitingTimes,Workforce,and

BenchmarkingPublication

2

Background Currently the quarterly CAMHS Waiting Times, the CAMHS Workforce, and the CAMHS

Benchmarking Toolkit publications are produced separately. The CAMHS Workforce project is

funded by NHS Education for Scotland (NES) and is a long‐term collaboration between the NES

Psychology Directorate and NHS NSS’s Information Services Division (ISD). The CAMHS Waiting

Times and CAMHS Benchmarking Toolkit are produced by ISD.

As these three publications are highly interrelated, it is logical to combine them into a single

publication to guide any comparison that may be made between the three areas. ISD and NES

are committed to producing information and intelligence that best meets the needs of our

stakeholders, customers and the public. Therefore modernising the presentation, improving

the usability and enhancing the accessibility of national statistics and publications is integral

to this commitment.

A draft version of a combined CAMHS publication was developed by ISD. NHS, Scottish

Government and members of the public were invited to read the draft version of the

combined publication and provide feedback via a consultation questionnaire (a copy of this

can be found in Appendix 2 from page 10). The consultation questionnaire contained 62 items

in total; fifty questions, eleven spaces for respondents to provide written feedback, and an

area for the respondent to note their organisation (where applicable). The consultation period

originally lasted for six weeks, between 5th September and 17th of October 2017, with an

extension until the 5th of January. This approach is in line with the UK Stats authority Code of

Practice for Official Statistics, which encourages user engagement to ensure that publications

are of value and user needs are met.

The key areas consulted on were:

Merging the current CAMHS Waiting Times and CAMHS Workforce publications

Including information currently published within the CAMHS Benchmarking Toolkit

Providing information within an open data platform

Improving data visualisations

This summary report presents the key findings from the consultation and outlines the next

steps for implementation. While the consultation is now closed, the combined publication

draft remains available via the ISD website. If you wish to discuss this further please contact

the CAMHS Waiting Times team via [email protected], or Workforce team via

[email protected].

The combined CAMHS publication is scheduled to be published in June2018.

3

Key Findings

The consultation document was made available on the ISD website. Communications were

issued to a variety of users of the statistics via email and social media. ISD received eleven

responses, consisting of ten individuals from various Health Boards and one Integrated Joint

Board.

Based on the responses provided, the key findings are as follows:

There is support to combine the CAMHS Waiting Times, Workforce, and Benchmarking

publications

The flow, clarity, and utility of tables and charts in the new combined CAMHS

publication were rated favourably by the majority of respondents

The majority of respondents thought there might be some issues around

interpretation, and useful feedback was received on how this can be improved

Respondents identified specific areas in current publications that are less useful to

them

Few respondents reported combining CAMHS datasets for their own analysis

There is support for CAMHS information to be made available via an interactive

dashboard

There is support to continue releasing the combined publication on a quarterly basis

Various suggestions were made around CAMHS waiting times data that cannot be

implemented at this stage as this information is only available at aggregate level

A full summary of the consultation responses is presented in Appendix 1 on page 5.

4

Implementation

The focus of this consultation was on gaining feedback on the combined CAMHS publication

draft so that the final version of the combined CAMHS publication would meet users’ needs as

well as possible when it is published in June 2018.

Following consultation, the following changes will be made to the publication document:

Inpatient activity will be consistently presented before workforce information

The benchmarking sections will be clearly labelled

Run charts will be removed from publication document, but included in supplementary

excel files made available online

Sections on ‘unadjusted completed waits for people seen’ and ‘unadjusted completed

waits for people waiting’ will be removed from the body of the document, but retained

in supplementary excel tables made available online

Sections on ‘number of sessions that CAMHS specialist staff have contributed to the

delivery of evidenced‐based parenting programmes for 3‐4 year olds (current data

quality issues)’ and ‘number of peer reviewed publications with a CAMHS theme

(current data quality issues)’ will be removed from the body of the document, but

retained in supplementary excel tables made available online

While the full publication will continue to be released quarterly, supplementary open

data will be supplied on a monthly basis

Longer term changes are likely to include:

Clarifying which CAMHS services in each board provide waiting times data and whether

these match with the services included in the workforce data

Making direct comparisons between CAMHS waiting times and workforce data

Collecting data to compare the ratio of new patients to return patients

Developing an interactive dashboard for CAMHS data

If you wish to discuss any of the above points further, please contact:

Michelle Kirkpatrick

Principle Information Analyst

0131 2756458

[email protected]

Lynne Jarvis

Principle Information Analyst

0131 2756424

[email protected]

5

Appendix 1 ‐ Results Table

Table 1: Summary of Joint Publication Consultation Responses

No. Items and Responses No. Agree1

% Agree

Overview

1.1 What benefits do you see in this proposal?

Useful to have all CAMHS info in one document, emphasis on clarity and comparability

N/A N/A

2.1 Do you think the new draft publication flows well? 10/11 91

2.2 Summary of respondents’ comments on previous:

Inpatient activity before workforce

CAMHS waiting times tables before workforce summaries

Further commentary (e.g. trends for individual boards)

Clarify which boards have inpatient units (Chart 10, p. 25 of the draft publication)

N/A N/A

3.1 Do you find it easy to differentiate between waiting times, workforce and benchmarking content?

7/9 78


Rationale explained well

Benchmarking should be clearer marked and included in table of contents

Addition of funding sources (e.g. NES: workforce)

N/A N/A

4.1 Are the tables and charts useful in the new proposed form? 10/10 100


More clarity around sources of NES data

More clarity about inclusion of Excel data tables

Changes to wording around run charts and Chart 9

Run charts fairly divisive

N/A N/A

5.1 Do you envisage any issues arising with the interpretation of Waiting Times and Workforce data together? Are there any specific caveats which should be considered? If so, what are these?

4/10 40


Need for care when comparing workforce and referrals

Clarify which services within boards provide data and whether these are the same for waiting times and workforce

Demand for discrete services within boards

Discussion on deferred referrals

More accurate picture of demand

N/A N/A

Use of current CAMHS publications

Waiting Times No. Useful

% Useful/V. Useful

6.1.1 Adjusted completed waits for people seen 10/10 100

6.1.2 Adjusted waiting times for people waiting 10/10 100

6.1.3 Unadjusted completed waits for people seen 7/10 70

1 N.B. The denominator in the fraction presented represents the number of respondents who answered the specific question, rather than the total number of respondents. This is to account for questions that were not answered by respondents.

6

6.1.4 Unadjusted waiting times for people waiting 7/10 70

6.1.5 Number of referrals 10/10 100

6.1.6 Distribution of waits 10/10 100

6.1.7 Number of did not attends (DNAs) 10/10 100

Workforce

6.2.1 Clinical staff employed by professional group 9/10 90

6.2.2 Clinical staff employed by gender 8/10 80

6.2.3 Clinical staff employed by contract type 9/10 90

6.2.4 Clinical staff employed by contract term 9/10 90

6.2.5 Clinical staff employed by NHS Board 9/10 90

6.2.6 Clinical staff employed by pay band 8/10 80

6.2.7 Clinical staff employed by headcount and whole time equivalent (WTE)

9/10 90

6.2.8 Clinical staff employed by age profile 8/9 80

6.2.9 Clinical staff employed by target age 6/10 60

6.2.10 Clinical staff employed by area of work (e.g. learning disabilities) 9/10 90

6.2.11 Clinical staff employed by inpatient/community working 8/10 80

6.2.12 WTE of clinical staff (per 100,000) total population by board 9/10 90

6.2.13 WTE of clinical staff (per 100,000) of children and adolescents, by board

9/10 90

6.2.14 Retention of graduates: MSc applied psychology for children and young people

6/10 60

6.2.15 Retention of graduates: Doctorate in Clinical Psychology (CAMHS aligned)

6/10 60

6.2.16 WTE of vacant posts currently advertised for recruitment, including a breakdown of which are for new and replacement posts

9/10 90

6.2.17 WTE of vacant posts approved for recruitment but not yet advertised

6/9 67

6.2.18 Headcount of staff currently in training on key training courses 6/9 67

Benchmarking

6.3.1 Completed CAMHS waits where patients have been identified as having a learning disability

7/9 88

6.3.2 Mental health bed days of all under 18s 8/8 100

6.3.3 Mental Health admission numbers of all under 18s 8/8 100

6.3.4 Number of new referrals to specialist CAMHS 8/8 100

6.3.5 Percentage sickness absence of dedicated NHS CAMHS clinicians 7/9 88

6.3.6 Non inpatient CAMHS wages budget 5/7 71

6.3.7 Number of sessions that CAMHS specialist staff have contributed to the delivery of specific targeted evidenced based parenting programmes for 3‐4 year olds (current data quality issues)

4/8 50

6.3.8 Number of peer reviewed publications with a CAMHS theme (current data quality issues)

5/8 63

6.4 Respondents’ comments on previous:

Compare WTE by professional group to activity/waiting times by professional group

Completed CAMHS waits where patients have been identified as having a learning disability has data quality issues – guidance on criteria needed

N/A N/A

7

7.1 Do you currently combine any CAMHS datasets for your own analysis? If so, which datasets?

4/10 40


DNA rates, adjusted waits, median waits, workforce (headcount), RTT

Local data combined for Workforce and Demand & Capacity Activity

N/A N/A

8.1 What extra information would you like ISD to publish on CAMHS, either waiting times or workforce?

More analysis related to age, sex and deprivation on an annual basis for public health purposes,

New to return ratios

Workforce information on supply and demand

Reason for DNA and action taken in different boards

Waiting times by diagnosis/condition

N/A N/A

How would you like ISD to make information available to you?

9.1.1 Publication summary highlighting the main points 9/10 90

9.1.2 Publication report with narrative content 9/10 90

9.1.3 Excel file – with data tables/charts 9/10 90

9.1.4 Open data 7/10 70

9.1.5 Interactive dashboard – allowing you to view the data in different ways

10/10 100

9.1.6 More granular information 4/10 40


Open data and interactive dashboard plans should be reviewed with the National Workforce Planning Group (NES are working on something similar).

The CAMHS data are NES funded and are involved in this process – this should be made clear

N/A N/A

How often would you like ISD to make information available to you?

10.1.1 Annually 3/9 30

10.1.2 Biannually 5/9 56

10.1.3 Quarterly (as is currently) 8/9 89

10.1.4 Monthly release of waiting time data with less frequent commentary

4/10 40

10.1.5 Monthly release of workforce data with less frequent commentary 4/10 40

10.1.6 Monthly release of benchmarking data with less frequent commentary

4/10 40


Monthly reporting a good idea

Monthly reporting potentially dangerous

Quarterly reporting a requirement for workforce

N/A N/A

11.1 Respondents’ closing comments:

Combining is a good idea

Can we combine all waiting times data per specialty

NES should be involved in draft, review and signoff (national

workforce planning group)

Would be good to separate inpatient and outpatient data as

N/A N/A

8

this makes comparing health boards difficult (if not possible,

explain why)

Update definitions of professions (e.g. dietetics)

The report focuses on quantity rather than quality of service

12.1 Please identify the type of organisation that you work for (where applicable)

7 regional boards

2 national boards

1 Integrated Joint Board

N/A N/A

9

Appendix 2: Consultation Questionnaire

An example of the consultation questionnaire is presented in the following pages.

Consultation on Publication

CAMHS in Scotland: Waiting Times, Service Demand

& Workforce

Responses required by

17 October 2017

Education for

Scotland

Consultation on the Publication of CAMHS in Scotland: Waiting Times, Service Demand & Workforce

Introduction

The Information Services Division (ISD) of NHS National Services Scotland is committed to producing infor-mation that best meets the needs of our stakeholders, customers and the public. Modernising the presenta-tion, improving the usability and enhancing the accessibility of Child and Adolescent Mental Health Services (CAMHS) Waiting Times and Workforce Information are integral to this commitment. Your input will help to inform the content of future releases of this information. This is in line with the UK Stats Authority Code of Practice for Official Statistics, which encourages user engagement to ensure that publications are of value. The key improvements we wish to consult on are:

• Merging the current CAMHS Waiting Times and CAMHS Workforce publications • Including information currently published within the CAMHS Benchmarking Toolkit • Providing information within an open data platform • Improving data visualisation

Background Currently three separate publications are produced quarterly, the CAMHS Waiting Times, the CAMHS Workforce the CAMHS Benchmarking Toolkit. The CAMHS workforce data collection and publication is produced collaboratively with ISD and NHS Education for Scotland (NES). The CAMHS Workforce project is funded by NHS Education for Scotland (NES) and is a long term successful collaboration between the NES Psychology Directorate and National Services Scotland (NSS). The CAMHS waiting times shows progress towards the 18-week waiting time target set by the Scottish Government. We are planning to merge the three publications into one, in order to present the workforce and waiting times information together, along with CAMHS activity data where available and producing some new ways of presenting the data. This will provide a fuller picture of CAMHS services in Scotland in terms of demand and capacity. The current publications can be found at the following links: http://www.isdscotland.org/Health-Topics/Workforce/CAMHS/

http://www.isdscotland.org/Health-Topics/Waiting-Times/Child-and-Adolescent-Mental-Health/ http://www.isdscotland.org/Health-Topics/Quality-Indicators/National-Benchmarking-Project/child-and-adolescent-mental-health/ Both CAMHS waiting times and CAMHS workforce publications are badged as National Statistics publications, which publishes service management and target information aggregated to Health Board level.

Page 2


Open Data Open data is the idea that some data should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control. Data has a value for any organisation in collecting and holding, but we can enhance the value of this data when there is license given to be able to use, modify and share for any purpose, as well as freely access on any platform in a form that can be useful. We are looking to provide CAMHS data as open data which will accompany the publication; the benefits of this include, but are not limited to, generating live dashboards; utilising new technologies such as application program interface (API) to distribute automatic management reports for specified geographies; and more transparency of CAMHS in Scotland. All data currently released by ISD into the public domain are technically ‘open’, but some data are more ‘open’ than others, as described by the open data 5* deployment scheme. ISD is working towards legislation that has been agreed across Scotland’s public bodies that they will aim to publish 3* open data as a minimum, meaning it is available in a structured, machine-readable, non-proprietary format and accompanied by an open government license which gives permission to freely use. As part of this ISD have created a new open data portal which will become a data store for Scotland’s health data which will go to support innovation of the health industry.

Consultation Questions

1. What benefits do you see in this proposal? Comments:

2. Do you think the new draft publication flows well?

Yes No Comments:

Page 3


3. Do you find it easy to differientiate between waiting times, workforce and benchmarking content?

Yes No Comments:

4. Are the tables and charts useful in the new proposed form? More useful Less useful

About the same

Comments:

5. Do you envisage any issues arising with the interpretation of Waiting Times and Workforce data

together? Are there any specific caveats which should be considered? If so, what are these?

Yes No

Comments:

Page 4


Your use of ISD’s current CAMHS publications

6. Of the information ISD has published on CAMHS please identify what is useful to you:

Very useful to me Useful to me Not useful to me Waiting Times

Adjusted completed waits for people seen

Adjusted waiting times for people waiting

Unadjusted completed waits for people seen

Unadjusted waiting times for people waiting

Number of referrals

Distribution of waits

Number of did not attends (DNAs)

Workforce

Clinical staff employed by professional group

Clinical staff employed by gender

Clinical staff employed by contract type

Clinical staff employed by contract term

Clinical staff employed by NHS Board

Clinical staff employed by pay band

Clinical staff employed by headcount and whole time equivalent (WTE)

Clinical staff employed by age profile

Clinical staff employed by target age

Clinical staff employed by area of work (eg learning disabilities) Clinical staff employed by inpatient/community working WTE of clinical staff (per 100,000) total population by board WTE of clinical staff (per 100,000) of children and adolescents, by board Retention of graduates: MSc applied psychology for children and young people Retention of graduates: Doctorate in Clinical Psychology (CAMHS aligned) WTE of vacant posts currently advertised for recruitment, including a breakdown of which are for new and replacement posts

Page 5


Very useful to me Useful to me Not useful to me

WTE of vacant posts approved for recruitment but not yet advertised Headcount of staff currently in training on key training courses

Benchmarking

Completed CAMHS waits where patients have been identified as having a learning disability

Mental health bed days of all under 18s

Mental Health admission numbers of all under 18s

Number of new referrals to specialist CAMHS

Percentage sickness absence of dedicated NHS CAMHS clinicians

Non inpatient CAMHS wages budget

Number of sessions that CAMHS specialist staff have contributed to the delivery of specific targeted evidenced based parenting programmes for 3-4 year olds (current data quality issues) Number of peer reviewed publications with a CAMHS theme (current data quality issues)

Comments:

7. Do you currently combine any CAMHS datasets for your own analysis? If so, which datasets?

Yes No Comments:

Page 6


8. What extra information would you like ISD to publish on CAMHS, either waiting times or workforce (noting the limitations of the data that are collected)?

Comments:

9. How would you like ISD to make information available to you?

Very useful to Useful to me Not useful to me Don’t know / notme sure

Publication summary highlighting the main points Publication report with narrative content

Excel file – with data tables/charts

Open data

Interactive dashboard – allowing you to view the data in different ways

More granular information

Other:

Comments:

Page 7


10. How often would you like ISD to make information available to you?

Very useful to Useful to me Not useful to me Don’t know / notme sure

Annually Biannually

Quarterly (as is currently)

Monthly release of waiting time data with less frequent commentary Monthly release of workforce data with less frequent commentary Monthly release of benchmarking data with less frequent commentary

Other

Comments:

Page 8


Submit Response Thank you for taking the time to provide your feedback to our consultation. Your input will be used to inform how CAMHS is published and developed by ISD and NES in the future. A report on the consultation findings will be available on our website at the end of this consultation.

If you have any further comments about the merging of publications, please add these here.

Comments:

Please identify the type of organisation that you work for (where applicable).

Please select organisation type

SUBMIT

A report on the consultation findings will be on our website by 17 October 2017.

Page 9

Documents

Public Consultation on the Draft CAMHS - ISD Scotland · steps for implementation. While the consultation is now closed, the combined publication draft remains available via the ISD