PAEDIATRIC PATIENT- REPORTED OUTCOME AND EXPERIENCE MEASURES (PROMS AND PREMS) IN ROUTINE CLINICAL PRACTICE Current developments and challenges

To explore current challenges surrounding the development and application of PROMs/PREMs in paediatric healthcare, a workshop was held for healthcare professionals and parent representatives from University College London (UCL) Institute of Child Health (ICH) and two clinical partners, Great Ormond Street Hospital and Moorfields Eye Hospital. The workshop aimed to establish an academic-clinical-service user collaborative network to support local development and translation into clinical practice of paediatric PROMs/PREMs. The workshop report summarises invited presentations, small group discussions and key local issues.

In order to meet these objectives we invited academic, clinical and other professionals from different GOSH specialties and from Paediatric Ophthalmology at MEH.We also invited parent representatives of the families of patients attending GOSH; those attending the workshop had participated in pre-workshop focus groups with their children held especially to inform the workshop through service user perspective (see section 6, presentation by Rehana Ahmed).

Workshop format
The morning programme included keynote presentations from invited speakers outlining the background and methodology for development and application of paediatric PROMs and PREMs respectively.These helped delineate the two types of measures and associated constructs (health outcome and patient experience respectively).This was followed by a presentation on the patient and family perspective on PROMs and PREMs at GOSH.The afternoon programme comprised showcase presentations from academic and clinical colleagues across UCL ICH, GOSH and MEH.Contributors outlined their experiences of using PROMs or PREMs in their departments and speciality areas, the impact that applying or developing these has had on their practice and/ or the associated challenges encountered.
Within breakout groups, which balanced clinical, academic and parental input, delegates discussed their own experiences of the challenges in using PROMs and PREMs, solutions that they had developed locally, and their expectations for a new PROM/PREM collaborative network that would further facilitate this work.
The workshop programme is provided in Appendix 1 and the workshop delegate list in Appendix 2.

Summary of invited presentations
The morning programme focussed on the distinction between patient reported outcomes and patient reported experience and associated measures, PROMs and PREMs respectively, as well as the overlap between the methodologies and considerations when developing measures for children and young people.Both external presentations highlighted that capturing children and young people's own perspective of their health and healthcare is both necessary and feasible.Developing and applying sensitive innovative approaches to engaging children and young people to develop age-appropriate PROMs and PREMs can be challenging, but is worthwhile and critical to ensuring that children and young people's needs are considered in healthcare decision making that concerns them.The presentation on patient and family perspective on PROMs and PREMs highlighted some key issues important to patients and their families attending GOSH services with regards to completing PROMs and PREMs.As a patient and public involvement officer at GOSH, Ms Rehana Ahmed has an important role working across the Trust with patients and their families with an aim of improving the patient experience of everyone using GOSH services.Ms Ahmed fed back the key findings from two parallel focus groups held with a small group of children and young people who are patients of GOSH and adult family members (mainly parents).

'Patient Reported Outcome Measures (PROMs) for children and young people' by
These focus group consultations were held as a first step to exploring patient and parent views of PROMs and PREMs in order to inform the workshop.One aim of the focus groups was also to identify and invite parent representatives interested to attend the workshop and contribute service users' perspective to the professional dialogue (two fathers subsequently attended).The issues highlighted by patients and their families in these focus groups with regards to completing PROMs and PREMs were: the need for adequate information provision to allow them to understand the purpose of PROMs and PREMs they are asked to complete; the importance of receiving feedback following their completion in terms of how this information contributes to their child's individual care or the health service in general; patient and parent preferences for the completion of questionnaires in routine clinical practice; and the issues, such as confidentiality and differences in perspective, related to obtaining parental and child patient report (the full report of the focus groups is available on http://www.gosh.nhs.uk/health-professionals/clinical-outcomes/.experience of using NEUGEN Quality tool and PedsQL as part of a pilot project aimed to evaluate the success of the process of using PROMs, the resources required and the quality and relevance of the information within the Neurology service at GOSH.He outlined some preliminary findings as the pilot data has not been analysed yet.He also highlighted issues arising from the project regarding routine PROM use in clinical services for children, e.g.issues of using non-validated tools such as NEUGEN, applicability of a single PROM in a diverse and complex range of conditions, ages and treatments and significant resource requirements for on-going collection of PROMs data routinely.

Summary of showcase presentations
Ms Susan Maillard, Clinical Specialist Physiotherapist (GOSH) reported the experience of using a range of PROMs and PREMs in Paediatric Rheumatology at GOSH.She outlined how these are used to inform inpatient and outpatient service development (e.g. by capturing patient and parent reported service delivery and timing of appointments), assess patient satisfaction (through self and proxy report satisfaction questionnaires), monitor disease (e.g. using known PROMs like Childhood Health Assessment Questionnaire-CHAQ, Childhood Health Questionnaire-CHQ, Independent Home Activity Score-IHAS), PedsQL-pain and fatigue and Visual Analogue Scores for pain, fatigue and general wellbeing) and develop research.
Dr Christina Liossi, Senior Lecturer in Health Psychology (University of Southampton) and Honorary Paediatric Psychologist (Pain Control Service, GOSH) presented a talk on PROM use in services for children with epidermolysis bullosa (EB), as part of 'pain in EB' project at GOSH.A pain specific PROM, as well as a generic health related quality of life instrument QoL and parental distress measure (using the Hospital Anxiety and Depression Scale) are used to document the prevalence of pain in various types of EB along with associated physical symptoms, anxiety and mood disorders and impact on quality of life.This will help develop and evaluate EB specific psychological interventions for the management of EB related pain as well as make widely available EB specific pain management interventions.

Summary of break out group sessions
There were 4 break out groups (with composition balancing clinical, academic and service user input), with allocated rapporteurs and with the discussion being guided by the following questions: 1. What are the key barriers to using/increasing the use of paediatric PROMs and PREMs in your practice or department?The feedback by four rapporteurs is summarised in Appendix 2, Table 1, but we present here the summary of key points.
The key barriers to using or increasing the use of paediatric PROMs and PREMs in the clinical departments at

Summary of Key Issues
1.There is much enthusiasm for quality paediatric PROM and PREM research and for applying such measures in routine practice.
2. Most current development and use is around PROMs, rather than PREMs.It is important to distinguish between these as outcome and process measures.It is also important to remember that PREMs are not 'patient satisfaction' measures.The Picker Institute Europe demonstrates good child-centred methodology for ensuring that these are developed to inform service development and that changes are measured/ audited.When developing and applying a PROM it is important to know what it measures.Although potentially related, outcomes such as HRQoL, functional status, symptom severity and participation are conceptually distinct constructs that require separate PROMs.Many PROMs are developed as Health Related Quality of Life (HRQoL) instruments, but PROMs can include additional aspects (such as functional status, functional vision, participation, symptoms).
3. Much research work is going into developing high quality, disease-specific PROMs where these are lacking, but should we also be evaluating use of existing/generic instruments in different settings?It would be helpful to develop a more standardised methodology for evaluating this approach and for testing whether a generic measure is adequate/appropriate for the new setting.
4. The key is completing the "measurement-action" cycle -there is a need for being clear about why a PROM or PREM is being implemented and for ensuring there is a follow-through to use the results and enable improvements.This means being explicit about whether the PROM is to monitor individual care/ outcomes of group/service outcomes.5. PROM and PREM development and application is resource-intensive.However, clinical colleagues should use the opportunity of the academic groupings available (such as our grouping at our MRC centre) that can support developing and using PROMs and PREMs.
6.There is a need for greater resources for PROM and PREM development and application and it is recognised that the main necessary resources have to come from the Trusts involved.

Next steps
Following the workshop, as the first step towards acting on the key issues raised by participants, we have set up a webpage (www.ucl.ac.uk/childproms), which will provide key information and resources regarding PROMs and PREMs.The aim of this website is to allow information provision and sharing amongst colleagues at ICH, GOSH and MEH.We have also set up a pilot LISTSERVE mail group to test feasibility and effectiveness of an interactive interface between academic and clinical professionals involved in paediatric PROM and PREM initiatives at UCL ICH, GOSH and MEH in the first instance, but with an aim to extend wider to service users -patients and parents alike -in the future.However, as recognised by participants, the key to implementation of paediatric PROMs and PREMs into routine clinical practice, supported by high quality research, is higher level leadership and resources, which can only come from our clinical partners.We urge colleagues to feedback this to their own Trusts in order to highlight the current gap in the NHS priority agenda concerning PROMs and PREMs for children and young people.

Appendices
The workshop organising team were researchers at the MRC Centre of Epidemiology for Child Health, UCL Institute of Child Health: − Dr Val Tadić, Research Associate, v.tadic@ucl.ac.uk − Dr Rachel Knowles, Senior Clinical Research Fellow and Public Health Doctor, rachel.knowles@ucl.ac.uk − Ms Ailbhe Hogan, Research Assistant, ailbhe.hogan@ucl.ac.uk − Professor Jugnoo Rahi, Professor of Ophthalmic Epidemiology and Consultant Ophthalmologist, j.rahi@ ucl.ac.ukThe workshop was funded by the MRC Centre of Epidemiology for Child Health and an MRC Public Engagement grant.
Dr Chris Morris, Senior Research Fellow, University of Exeter Dr Chris Morris leads the Peninsula Cerebra Research Unit (PenCRU) research unit undertaking a broad range of health services research dedicated to improvement of health outcomes for children.He has experience of PROM methods and measurement, having developed the Oxford Foot and Ankle Questionnaire for children with orthopaedic problems.With the Oxford PROM group he has co-authored the Department of Health commissioned report on feasibility of PROMs in routine practice for children.Dr Morris outlined the background for PROMs within the NHS, the methodology of development and application of such measures for children and the opportunities and challenges facing paediatric PROM research currently and in the future.He outlined his current CHildren's oUtcome Measurement Study -the CHUMS project aims to inform the development of the NHS Outcomes Framework and will gather the views of children, parents and professionals to examine whether existing generic PROMs could be used to measure the NHS and wider health outcomes of children with neurodisability.'Patient Reported Experience Measures (PREMs) for children and young people' by Mrs. Amy Tallett and Mrs. Bridget Hopwood, Picker Institute Europe Mrs Bridget Hopwood is a Director of Surveys and Mrs Amy Tallett is the Senior Project Manager within the Children and Young People Research Team at the Picker Institute Europe, which is a not-for-profit healthcare research charity and an approved survey contractor to the NHS.The Picker Institute recognises that children and young people have very specific needs when giving feedback on their healthcare and has been involved in the development and testing of paediatric inpatient, outpatient and emergency department surveys and overseeing the implementation of these on an annual basis for NHS Trusts in England.The speakers outlined the background to PREMs, their purpose in measuring experience of care beyond 'patient satisfaction', and the methodological considerations when developing PREMs with children.They outlined the findings of postal paediatric outpatient department surveys across 8 NHS Trusts in the UK and presented examples of using outpatient PREMs to drive improvement of healthcare provision in response to the self-reported experiences of children and young people.'PREMs & PROMs: Is Anybody Listening?What the patients and their families say' by Ms Rehana Ahmed, Patient and Public Involvement and Experience Officer, Great Ormond Street Hospital (GOSH)

i
Dr Val Tadić, Research Associate (MRC Centre of Epidemiology for Child Health and Ophthalmology, GOSH and MEH) gave an overview of a multidisciplinary research programme to develop 2 PROMs specifically for children and young people with visual impairment -a Vision Related Quality of Life Questionnaire and a complementary Functional Vision Questionnaire.She outlined 3 key challenges in developing such measures.These are a) the lack of an established theoretical framework for development of such measures whereby distinct but related concepts targeted by vision-related PROMs (e.g.visual function, functional vision and vision-related quality of life) are often used interchangeably, b) the increasing trend of low participation rates and ethnic and socio-economic bias in health services research involving children and c) lack of understanding about feasibility and effectiveness of PROMs in routine clinical practice for children with ophthalmic disorders, where PROM research and application is still in its infancy.Dr Jo Wray, Research Health Psychologist (Cardiorespiratory, GOSH) and Senior Research Fellow (Centre for Nursing and Allied Health Research, GOSH) outlined the use of Pediatric Cardiac Quality of Life Inventory (PCQLI) and Paediatric Quality of Life Inventory (PedsQL) in routine clinical services for children with cardiac disease.She showed how these are used, alongside parent-reported PREMs, as part of a Standardised Clinical Assessment and Management Plans (SCAMPs) initiative that combines elements of research, clinical practice guidelines and audit to inform indicators of quality.She also outlined the challenges for ongoing use of PREMs and PROMs, such as lack of resources, frequency and sustainability of PROM/PREM data collection, access to data and getting research evidence into clinical practice.Dr Naomi Dale, Consultant Clinical Psychologist, Head of Psychology (Neurodisability, GOSH) reported development of a novel outcome measure for assessing parental understanding of neurodisability -Parents' Understanding of Neurodisability Questionnaire (PUN_Q).The measure was developed by the Neurodisability services at GOSH, which serves to provide consultations to parents/referrer's, diagnosis of the child's neurodevelopmental status and recommendations for management.To develop the measure, the team consulted parents of children and young people with autism spectrum disorders.PUN_Q has undergone preliminary validity and its ratings are associated with parental stress and self-efficacy levels, which are likely to have behavioural consequences on the child.Hence, the group envisages clinical utility of the measure in routine clinical practice for evaluating effectiveness of service interventions.Dr Mandy Bryon, Consultant Clinical Psychologist and Joint Head of Paediatric Psychology Service at GOSH provided an update of using the child self-report form of Cystic Fibrosis Questionnaire (CFQ) with children with cystic fibrosis at GOSH as part of their annual review.The CFQ is relatively quick to complete as part of routine clinical care.It has been highlighted as helpful in providing patient input in highlighting problematic areas of a patient's quality of life, although the majority of patients would not want to complete this PROM more than once a year.Dr Prab Prabhakar, Consultant Paediatric Neurologist (Neurology, GOSH) outlined the Neurology Department's

2 .
Have you developed or come across any good solutions?What are your top tips for a) developing and b) implementing PROMS and PREMS as part of routine clinical care for children? 3. How would you like to take this multidisciplinary network forward?What would you find most useful?
GOSH and MEH were reported to be: − Lack of time and dedicated resources for both development and validation of PROMs and PREMs − Introduction of PROMs and PREMs appears currently to emphasise speed of development over quality − The PROMs and PREMs initiative is being led by small groups working independently rather at a high level strategic initiative − Lack of information, leadership and co-ordination from the higher levels of management is leading to fragmentation of effort and approach across different specialties The solutions and tips to developing and implementing PROMs and PREMs as part of routine care were reported as follows: Instruments − Questionnaires should be clear and concise enough to be integrated into course of routine clinical work − Given the lack of resources available, we should use validated tools, rather than develop new ones Context for use − There is an increased value of PROMs if they form part of routine clinical care (e.g. in diagnostic process) − There should be standard guidelines on using PROMs and PREMs on a Trust basis − There should be a network and support group (including experts) that is available hospital wide Involving patients and families − We should strive to provide clear information to patients and carers about data collected and ensure confidentiality (especially PREMs) − We should devote attention to feeding PROM and PREM information back to parents The ways of taking this multidisciplinary network forward that would be useful to the attendees were reported as follows: − A dedicated website for further communication and information sharing − Tangible ways of enabling practical support (e.g.teaching and drop-in sessions and future workshops), although it is recognised that this requires allocated resources − Continued patient and parent involvement − Involvement by the senior management teams at the Trust

7.
There is a need to further develop informative and sophisticated Patient and Public Involvement (PPI).Many parents currently involved in these initiatives are drawing on experience of measures designed to monitor individual care, but we now need to include PPI in developing and implementing measures with other goals/aims.8.There is a need for better communication and co-ordination of activities across different specialties and research/clinical institutions to ensure development work is being used in most effective and efficient ways, with minimal duplication of effort.The focus for leadership in translation and implementation should be clinicians and Trusts.9.There is a need for a communication and support platform in the form of a network for academic and clinical colleagues undertaking PROM and PREM work at our centres.10.PROMs directed at children have special and unique features.The workshop attendees form a body of clinical and academic expertise in 'child-focused' PROMs that should be supported and developed as an important resource within the PROMs and PREMs field.

11. 1
Appendix 1: Workshop Programme A multiprofessional workshop at UCL Institute of Child Health Hosted by the MRC Centre of Epidemiology for Child Health 'Paediatric patient-reported outcomes and experience measures (PROMs and PREMs) in routine clinical practice: Current developments and challenges' Date and venue: Wednesday 3rd October 2012 UCL Institute of Child Health, Wolfson Centre (Room B), Mecklenburgh Square,

3 Appendix 3: Content of the break out group discussions What are the key barriers to using/ increasing the use of paediatric PROMs and PREMs in your practice/ department? Have you developed or come across any good solutions? What are your top tips for: a) developing PROMS and PREMS. b) implementing them as part of routine clinical care for children. How would you like to take this multidisciplinary network forward? What would you find most useful?
Dr Val Tadić, Research Associate, MRC Centre of Epidemiology for Child Health and Ophthalmology, GOSH and Moorfields Eye Hospital (MEH) * Registered but not able to attend 14 11.