Lancaster medical school final visit report 2013–14 Lancaster University medical school This visit is part of the GMC's remit to ensure medical schools are complying with the standards and outcomes as set out in Tomorrow’s Doctors 2009. For more information on these standards please see: Tomorrow's Doctor's (2009) R eview at a glance About the School Medical school Lancaster medical school University Lancaster University Dates of visit 11 June 2014 (Lancaster medical school) 10 July 2014 (Year 3 objective structured clinical exam [OSCE]) 16 July 2014 (Year 2 exam board) 23 July 2014 (Year 3 exam board) Programme investigated MB ChB Area explored Quality management strategy: quality management of the programme, general practice (GP) placements, hospital and community placements; curriculum development and mapping to Tomorrow’s Doctors 2009; development and QA of assessment items, delivery of assessments, blueprinting and standard setting; University Hospitals of Morecambe Bay NHS Foundation Trust (UHMBNFT) reconfiguration; contingency planning and decoupling; Cumbria Partnership NHS Foundation Trust (CPNFT) clinical placements and involvement. Number of students 268 (2013 Medical School Annual Return [MSAR]) Number of LEPs 2 acute hospitals, 2 mental health trusts and 34 general practices Local Education and Training Health Education North West Board Last GMC visit 2013 QIF North West of England regional review Significant Patient safety or Educational concerns identified None Has further regulatory action been requested via the Responses to Concerns process No Executive sum m ary Sum m ary of key findings 1 Lancaster University has delivered an MB ChB programme in partnership with the University of Liverpool since 2006. Students enrolled in this programme have followed the Liverpool curriculum and currently graduate with a Liverpool primary medical qualification (PMQ). However, a process is underway to decouple the medical schools. Lancaster is developing its own curriculum and taking over responsibility for managing its own processes. The students entering Years 1, 2 and 3 in September 2014 are registered as Lancaster students; and, if the GMC agrees to add Lancaster to the list of bodies who can award PMQs, these students would be awarded a Lancaster degree upon successful completion of the programme. 2 We have been visiting Lancaster and Liverpool since 2011 and will continue the rolling cycle of annual quality assurance visits to Lancaster medical school (the School) following the first cohort of students through to graduation and their first year of practice. The School has made much progress since our last visit in November 2013 and we were pleased to find that many of the previous requirements and recommendations that we had set for the School have now been met. We also identified a number of additional areas of good practice. 3 There remain a number of challenges and risks for the School that have been explored during recent quality assurance activities and will continue to be a keen focus over the coming year. Not least, the uncertainty surrounding the reconfiguration of healthcare services in the area, and the breadth of clinical experience therefore available to students, along with the ongoing concerns about the School’s main local education provider, University Hospitals of Morecambe Bay NHS Foundation Trust (UHMBNFT). The School considers its main risk to be the recruitment of a suitable successor to the current head of school. It is vital that all of these risks are effectively managed and mitigated to ensure the long term success of the School. Good practice Tom orrow ’s Good practice Doctor Report paragraph paragraph 1 43, 48; 51; 83, 101 The involvement of the Cumbria Partnership NHS Foundation Trust (CPNFT) in the development and delivery of the programme. 25, 30, 4344 2 49,54; 85; 89-90 and 115 The use of technology, including the eAssessment system and tablet computers to enhance the development and delivery of assessments, including standard setting, quality assurance of assessment items, feedback to students and reasonable adjustments. 15, 56, 73, 76, 79 3 39; 49, 54; 43, 48 The extent of the School’s responsiveness to student evaluation with examples of improvements being made in response. 14, 19-20, 27-29 Requirements Tom orrow ’s Requirements Doctor paragraph Report paragraph 1 28a The School must ensure students are provided with written guidance so they are clear which clinical procedures they are able to perform when on clinical placements. 1 2 44 The School must formulate detailed plans to address the two long standing open requirements concerning the appointment of a successor to the Head of School and ensuring the clinical experience of students is not compromised by the reconfiguration of healthcare services in the area. 32 Recommendations Tom orrow ’s Recommendation Doctor Report paragraph paragraph 1 51 In developing the quality management of general practice placements, the School should check the current Care Quality Commission (CQC) status of the practices, and link to postgraduate quality management systems for the postgraduate training practices where possible. 22 2 51 The School should make the quality markers for general practices mandatory requirements; this includes completion of equality and diversity training by the GPs. 23 3 41, 50 The formal responsibility for approval of new general practice placements should be reviewed to ensure there is an appropriate governance structure and transparent delegated authority. 26 4 43, 48 The role of patients in the quality management of the programme should be clearly defined. 31 5 44 The School should review and update the risk template to reflect how the risk level or score 33 has changed over time. 6 100 The School should review the prescribing and therapeutics workshops to enhance the learning opportunities. 47 The Report Domain 1: Patient safety 26. The safety of patients and their care must not be put at risk by students' duties, access to patients and supervision on placements or by the performance, health or conduct of any individual student. 27. To ensure the future safety and care of patients, students who do not meet the outcomes set out in Tomorrow's Doctors or are otherwise not fit to practise must not be allowed to graduate with a medical degree. Acting within competence (TD28a) 1. The School is partially meeting this standard. The Year 3 students confirmed they have been able to take blood while on placements in obstetrics and gynaecology and general practice, under close supervision. Despite this, they were uncertain whether or not they are permitted to undertake this procedure, and they had heard that there is a list of procedures assigned according to year of study but they had not seen this yet. The students said the medical educators at UHMBNFT will allow students to do venepuncture with supervision, but that the School has not been clear whether this was permissible. See requirement 1. 2. The Year 5 students said they have never been asked to complete a task that was outside their competence. Systems in place to minimise harm to anyone taking part in training (TD30) 3. The School is meeting this standard. We met with representatives of CPNFT, which provides placements in acute mental health for Year 3 and 4 students and community care services for Year 2 students as well as a ‘selective in advanced medical practice’ (SAMP) for students in Year 5. We heard that an information leaflet is provided in units to inform patients that medical students are present and what their role is. It was explained that nursing staff select the patients with whom it is appropriate for the students to meet and the patients give consent. 4. CPNFT advised that students must complete one case for their logbook with a patient who does not have capacity, and a foundation doctor supports the student in doing this. At induction, students are informed of how to approach patients with mental health problems including where to sit when meeting with a patient and wearing an alarm. They are advised of the types of questions that some patients may find distressing and what to do if this occurs. We heard that there is a protocol in place for students to inform the nurse in charge if a patient discloses suicidal thoughts or other information that suggests the patient is at risk. Students are also informed what to do if they hear distressing information, such as reports of abuse or symptoms of illness. 5. CPNFT confirmed they receive information about students from the School in advance of the academic year. This includes student names, photos, special circumstances, and any particular needs they must be aware of and accommodate. Clinical supervision (TD31, 35) 6. The School is meeting these standards. The students we met from Year 3 confirmed they are supervised at all times on GP placements. They also told us there are two community clinical teachers who they are able to contact. Students are aware of a lead consultant at Furness General Hospital with a remit to work with undergraduate medical students. Overall they feel well supervised. 7. CPNFT provides a half day induction for students at the beginning of their placement (see also paragraph 4). We were informed by the Trust that students are given clear instructions on how to end interviews if a patient becomes distressed, and how to manage professional boundaries. Students are provided with an alarm to carry when in clinical units. Home visits to patients are always undertaken with a clinician from the Trust; never by a student alone. UK Departments of Health guidance on accessing patients (TD32) 8. The School is meeting this standard. The Year 3 students had good awareness of where to find further guidance and information. They told us that they are informed about GMC guidelines at the beginning of the year and that they are signposted to further policies and procedures, guidelines and local polices, with links available on the Moodle system. Identifying concerns about medical student conduct (TD28c) 9. The School is meeting this standard. CPNFT said its first point of contact for concerns about students would be the Undergraduate Education Manager or the Director of Medical Studies. We heard that potential concerns about students are discussed, but they only knew of one occasion when a formal ‘concern’ form has been completed. We heard that any concerns about performance or professionalism are recorded in the students’ portfolios. Fitness to practise (TD28d, 36, 37) 10. We believe the School is on track to meeting these standards. Over the past year, Lancaster has taken over responsibility from Liverpool for running its fitness to practise procedures. The first full fitness to practise panel was scheduled to run the day following our visit in June 2014 and we will follow this up during future quality assurance activities. Raising concerns (TD28e) 11. The School is meeting this standard. We set a requirement for the School following the November 2013 visit to ensure all students are aware of and able to follow the process for raising patient safety concerns when on clinical placements. The School and UHMBNFT have collaborated to produce two leaflets specifically for students, one about raising patient safety concerns and whistleblowing and the other about safeguarding. We viewed these leaflets shortly before the visit in June 2014. The students we met with were not yet familiar with these but were aware of the appropriate ways to raise concerns. We heard from the School that it plans to include these leaflets every year at induction. The School advised it has held further discussions with students to make sure they know what to do if they have a concern. 12. The Year 3 students were informed about whistleblowing at the beginning of the year and said they are encouraged to raise concerns and given different routes by which to do so. They told us about an example of a student raising a concern about a problem they encountered at a care home, first discussing it with the teacher and later with the Director of Community Studies. The Year 5 students clearly described the process for raising a patient safety concern, which requires them to alert their medical supervisor immediately, subsequently following up with the School management if necessary. This requirement is now closed. 13. A significant event process is in place to deal with concerns arising in community placements. We heard that a student had recently raised a concern, through a community clinical teacher, about an incident. The teacher completed a significant event incident form, this was sent to the Director of Community Studies who then raised the concern with the practice involved. This was documented and reviewed by the Learning and Teaching Committee and feedback was given to the student to confirm action had been taken. Domain 2: Quality assurance, review and evaluation 38. The quality of medical education programmes will be monitored, reviewed and evaluated in a systematic way. Quality management framework (TD39) 14. The School is meeting this standard. Following the November 2013 visit to the School, we set a requirement to develop an overarching quality management document, that sets out a clear framework or plan for how it organises quality management and control including who is responsible for this. The document ‘Quality Management Framework for the MB ChB programme’ was provided in advance of the visit. The document sets out clearly defined responsibilities and reporting mechanisms, and the use of multiple sources of information. We note the extent of student evaluation that is detailed in this document. This requirement is now closed. See good practice 3. Assessment of students (TD49, 54) 15. The School is meeting this standard. Since the last visit, we have received monthly updates from the School about the development and quality control of assessment items and papers. The School provided a demonstration of the Rogō system that is used to create, quality control, and deliver online assessments. We viewed the year 1 summative assessment paper 2. We viewed some example items, including some ‘hotspot’ questions and could see that the quality control check is recorded with the individual items in the system. A response to the quality control check can also be recorded with the item. An audit trail in Rogō for the individual assessment items records who completed the quality control check and when this was done. See good practice 2. 16. A written assessment is mapped to learning objectives by question and each learning objective often links to multiple questions. Once a written paper is assembled it is made available to the external examiners to review online and submit their reports. Internal quality control of the whole paper is also undertaken to make sure there are no overlaps or over-assessment of particular modules. 17. The assessment team explained that the external examiners get a similar view of the papers. There was initially an issue with the external examiners not being able to view the hotspot images and this was worked around with PowerPoint slides being provided to them. The issue has now been rectified. 18. An away day has been organised for September 2014 for assessment staff to debrief and plan for the upcoming academic year. We heard the changes that have been made to the OSCE for years 1 and 2 were based on learning from previous quality activities. A further event is planned for the external examiners. 19. We heard about changes that have been made to assessments as a result of student evaluation following the formative exams for Years 1, 2 and 3. For example, although the exams are now completed electronically on Rogō, the students felt it would be beneficial also to have a paper copy of the questions so they had the option to read on the page instead of the screen. The Year 3 students told us the onscreen assessment is not especially popular but they accept that electronic assessments will become more common and were pleased that the School had responded to their concerns by providing a paper copy. 20. The School also asked students to vote on whether they would prefer two shorter papers, or one long paper for one of the examinations, with a narrow majority favouring one long paper; therefore one long paper was set. See good practice 3. Clinical and vocational placements (TD51) 21. We are satisfied the School is meeting our standards in this area. Following the last visit cycle, we set a requirement for the School to formalise and monitor the quality management procedures for community placements, and to ensure the outcomes of these are recorded and shared with the practices involved. We received a report on the development of this system prior to our visit in June 2014, including a summary of the visits that had been undertaken at this time and reports from the individual practice visits. The system was presented at the School’s recent Learning and Teaching Committee meeting. We note the good progress made in developing the system. There is a visit schedule in place for the next three years, with flexibility to allow practices to be reprioritised if necessary. 31 out of 34 practices have been visited in the 12 months up to June 2014. The Director of Community Studies currently undertakes the visits, but consideration is being given to training Community Clinical Teachers to complete future visits. 22. A master spreadsheet captures key information about the 34 active practices, with student evaluation, significant events and practice visits clearly shown. Attendance at the Training in Practice course is recorded, although this is not yet mandatory. It would be helpful to see that in the future this system checks the current CQC status of the practice, and that it will permit liaison with postgraduate GP training quality systems. See recommendation 1. 23. The Director of Community Studies informed us that they have worked with Leeds medical school and Liverpool medical school to share good practice and to develop quality markers for community placements. One further area of development planned for the coming year is longitudinal analysis of student evaluation of placements. Not all of the quality markers are currently mandatory requirements for the practices; for example, completion of equality and diversity training by the GPs is not. This should be reviewed. See recommendation 2. 24. A significant event process (see paragraphs 12 & 13) is in place to deal with concerns arising in community placements. We heard that a student had recently raised a concern, through a community clinical teacher, about an incident. The teacher completed a significant event incident form, which was sent to the Director of Community Studies who then raised the concern with the practice involved. This was documented and reviewed by the Learning and Teaching Committee and feedback was given to the student to confirm action had been taken. 25. We also discussed the quality management systems in place with CPNFT. We heard the Trust has a very good relationship with the School, with key contacts enabling concerns to be shared and resolved. The Trust receives evaluation on its placements and has developed the student questionnaires to improve the extent of the evaluation to identify areas for improvement. For example, ‘Did you get information on arrival?’ has been expanded to ask whether it was the right information and if there was anything further that the students needed. See good practice 1. Agreements with providers (TD41, 50) 26. The School is meeting these standards. However, we recommend the School reviews the process for approving general practices. The Director of Community Studies completes a new practice proforma and decides whether or not to approve a new practice to teach students. The contract agreements make the requirements of practices clear; nonetheless, the formal line of responsibility for approval does not seem to be clearly defined at present and should be reviewed to ensure there is an appropriate governance structure and transparent delegated authority. See recommendation 3. Student, Patient and Employer Involvement in Quality (TD43, 48) Students 27. The School is meeting this standard. Online surveys capture students’ evaluation of placements. The School reviews the data in order to identify any immediate concerns. The information is collated and feeds into the quality management visits and is shared with the relevant local education provider. The online student survey system allows student evaluation to be easily collated. The completion of surveys is not compulsory at present, but students are reminded of their professional responsibility to do this. We heard that completion rates in Years 1 and 2 are high, at around 80-90%, although this reduces in the later years. 28. The School told us about how student evaluation can lead to developments, for example, positive student evaluation of dermatology clinic experience during a GP placement led to the incorporation of this area into the Year 3 logbook for the coming academic year. 29. The Year 3 students told us the school is receptive to their evaluation and responds. A recent example was provided of an extension being given for an assignment as it clashed with revision time. See good practice 3. 30. CPNFT informed us it completes an end of year report to evaluate delivery of the curriculum and identify opportunities and areas for improvement. The report includes information about induction, student evaluation of teaching, timetabling, and the sign off of clinical experiences in student logbooks to assess how easily students are able to complete the required cases. We heard that student evaluation is generally positive. See good practice 1. Patients 31. The School is partially meeting the standards for involvement of patients in quality. We heard that patient satisfaction surveys are required in year 5 for GP placements and also that there are plans to increase expert patient involvement in community teaching. We have previously reported on the introduction of real patient problem based learning (PBL) cases in year 3 of the curriculum. However, although patient groups are represented at the School’s Board of Studies, the role of patients in the quality management of the programme does not appear to have been clearly defined. See recommendation 4. Identifying and managing risks and concerns (TD44) 32. The School is not meeting our standards for identifying and managing risks and concerns. While the key risks facing the School over the coming years have been identified and recorded, the management and mitigation of these risks could be more effective. We recognise that some of the risks arise from external factors; the School does need to formulate more detailed plans to ensure these risks are mitigated and, if necessary, contingency plans are in place. In particular, there remain two open requirements that we set for the School following visits in 2012. These relate to appointing a successor for the Head of School (see also paragraph 86); and ensuring the clinical experience of students is not compromised by the reconfiguration of healthcare services in the area. Addressing these requirements is crucial to ensuring the long term success of the medical school. See requirement 2. 33. The Faculty of Health and Medicine requires the School to review the risk register twice per year. The School’s template, which is provided to the GMC, does not show how the risk level or score has been adjusted over time and it would be helpful to capture changes in this way. See recommendation 5. 34. Reconfiguration of healthcare services in the area has been under consideration for some time and we heard that this is now moving forward, with the strategic outline case submitted at the end of June 2014 and the business case due to be completed by the end of August 2014. The Head of School advised the risk register will be reviewed in light of this. 35. UHMBFT is the only acute trust currently providing core placements for Lancaster students. 36. The day prior to the visit, it was announced that the Chair of UHMBFT had resigned. The CQC visit report was published shortly after this. An interim Chair of UHMBFT has been appointed. The media reported that the Medical Director of the Trust had also announced his resignation; however, his appointment last year was for a fixed term and the resignation will not take effect for another year. The Trust plans to recruit a new medical director who will work alongside the current Medical Director until this time. Nonetheless the CQC report is critical of the leadership and notes a sense of disconnect between the front line staff and the executive board. 37. UHMBFT is under enhanced monitoring by the GMC, with ongoing concerns about postgraduate medical education at Furness General Hospital, especially in emergency medicine, obstetrics and gynaecology and paediatrics. There is also a public inquiry into the deaths of mothers and babies at the hospital, as well as an ongoing criminal investigation in relation to the same. 38. The CQC report was published on 26 June 2014 and subsequently the Trust has been placed in special measures. The School informed us it does not have concerns about its undergraduate placements at Furness General Hospital and the students we met did not inform us of any patient safety concerns. Nonetheless, we are concerned about the potential impact this report could have on organisational stability in view of the reconfiguration agenda and will continue to monitor this closely over the coming year. Domain 3: Equality, diversity and opportunity 56. Undergraduate medical education must be fair and based on principles of equality. Local education provider commitment to equality and diversity (TD69) 39. The School is partially meeting our standards in this area. CPNFT informed us that equality and diversity training is mandatory within the Trust and a separate course is also completed for those with an educational role that is monitored by the Medical Education Facilitator. See also paragraph 23 and recommendation 2; regarding the quality management of general practices. Domain 5: Design and delivery of the curriculum, including assessment 81. The curriculum must be designed, delivered and assessed to ensure that graduates demonstrate all the 'outcomes for graduates' specified in Tomorrow's Doctors. Student selected components (TD94-99) 40. The School is meeting this standard. We viewed a sample of the Year 2 student selected module reports, and noted the quality of work produced by the students, including a good range of contemporary topics and primary research as well as literature review. The external examiners at the Year 2 exam board also commented on the overall high standard of the work. See also paragraphs 57 & 58 for information about marking and feedback. Curriculum design and structure (TD83, 101) 41. The School is meeting this standard and has continued to develop its curriculum. This year it has delivered a revised Year 1 as well as some further developments and minor changes for Years 2, 3 and 5. The revised curriculum for Years 2 and 3 has been planned and will be delivered in the 2014/15 academic year. Changes to Year 5 include a new lead for the patient safety programme and a management and leadership week, which included the involvement of the Chief Executive of Health Education England. Changes to Year 4 are expected to be implemented in 2015/16. 42. Student involvement in the curriculum review has included focus groups for each year group. In addition, all student evaluations have been used to feed into the reviews for each year. 43. Changes have been made to the Year 3 women and child health block including additional involvement of CPNFT, with some community based paediatrics, as well as obstetrics and gynaecology. There are further potential opportunities for clinical placements with the introduction of a community based eating disorders service. 44. CPNFT told us of its involvement in a full review of the mental health aspects of the programme, which is currently nearing completion. Previously, it was felt that cover across the five years was patchy, and the revisions will introduce spiral elements to the curriculum. The Trust is not part of the core curriculum group but is included in a sub-group specifically looking at mental health. See good practice 1. 45. Another area that is being evaluated is the inclusion of some longitudinal cases in order that students can see a full patient journey. During four to five months in general practice, students will follow approximately four patients long term. There are no current plans to implement this further until it can be shown to enhance the learning experience. Balance of learning opportunities (TD100) 46. The School is meeting the standards for ensuring a balance of learning opportunities. Extra-curricular debates have been introduced to support student learning about ethics and law. We heard that students were asked to suggest ethical issues they were concerned about. One such debate was held on discharge against medical advice. 47. The Year 3 students told us their sexual health teaching is very good, but the prescribing and therapeutics workshops are less helpful and would be improved by greater structure. For example, they felt the inclusion of foundation doctors in learning how to complete prescriptions would be beneficial. See recommendation 6. Clinical Placements and Experience (TD84) 48. We remain concerned about the breadth of experience the students gain from their clinical placements and will continue to monitor this in light of the potential impact of reconfiguration. The Year 3 students expressed concerns about the learning opportunities in a number of specialities. 49. The Year 3 students told us the paediatric units at both Royal Lancaster Infirmary and Furness General Hospital are small and the cases seen during the seven week attachment are limited in both range and volume. Complex cases presenting at Furness General Hospital tend to be transferred to Royal Lancaster Infirmary. The students told us they have problems completing the required cases in their logbooks; and on some occasions had to complete a theoretical case based discussion with a consultant as they had not seen a relevant case. 50. The Year 3 students reported that they had some concerns about the breadth of their clinical experience to date but the Year 5 students were more positive and reported feeling well prepared to begin the foundation programme. However, they said there is limited exposure to cardiology, neurology and renal medicine and would welcome additional opportunities in these specialties and are aware that the School continues to explore possibilities with other local education providers. During the next cycle of visits we will focus our quality assurance activity to enable a conclusion to be drawn as to whether or not there is sufficient breadth of clinical experience. 51. CPNFT told us of possible opportunities to extend student experience in some of the specialist community services such as end of life/palliative care and neurology. There are ongoing changes to the high level leadership and strategic planning for the Trust; however, they do not consider there to be any risks to education opportunities for students. Likewise, CPNFT is not aware of any risks posed by the reconfiguration of acute services to its own services and therefore the placements it provides the School. The Year 3 students told us they were aware of reorganisation within the Trust but had not been directly affected by it. Nonetheless, CPNFT has kept them informed of changes and has seemed to manage it smoothly. The students were positive about their placements with CPNFT. 52. The School has made some progress in agreeing further placements with Blackpool Teaching Hospitals NHS Trust, with further SAMPs to be offered in cardiothoracic surgery for Year 5 in 2014/15, following positive student evaluation this year; and also some placements included during the Year 3 women and child health block. Mandatory clinical placements (TD106) 53. As described in paragraph 34 it is expected that acute care services will be reconfigured, with details to be set out in a business case in August 2014. The School maintains there is no problem with the range of experience delivered to its students at present. We must establish clearly whether we agree that this is the case, and we will continue to monitor developments closely to ensure each student receives sufficient experience including in all of the mandatory specialties. 54. We heard that of the current Year 5 cohort, seven of the 54 students were planning to complete their foundation training at UHMBNFT, based at Royal Lancaster Infirmary rather than Furness General Hospital, with 63% staying in the North West region for foundation training (44% in North Western and 19% in Mersey). The School has set up a formal study by a postgraduate student to look at the reasons behind the choice of where to complete the foundation programme with Year 5 students and foundation doctors. The Year 3 and 5 students we met told us they would probably not choose to complete foundation training at UHMBNFT, and, although the reasons for this varied, we were told by several that they wish to have a broader range of experience, or particular specialist experience in another local education provider. The Year 5 students told us they would not be against returning to UHMBNFT for core or specialist training. 55. The School thinks the options for foundation training in the area have improved this year as the track has changed to allow the doctors to complete both years at either the Royal Lancaster Infirmary or Furness General Hospital rather than rotating as in previous years. The Head of School sits on a working group at UHMBNFT to look at increasing the number of foundation doctors training in the Trust. Feedback to students on their performance (TD85) 56. The School is meeting this standard. We previously set a recommendation for the School to give students more detailed feedback on the formative OSCE and this has been enabled via the use of tablet computers and the development of detailed feedback sheet for each station. Year 3 students told us this has been the best year so far in terms of feedback on assessments, in particular the very detailed feedback provided on the formative OSCE. All students who fail the formative OSCE receive individual feedback in person from a clinical skills assessor. Rogō provides individual feedback sheets to students on their written exams, with scores on topics rather than individual questions. This recommendation is now closed. See good practice 2. Identification of strengths and weaknesses (TD111) 57. The School is meeting this standard. A new assessment mark sheet has been introduced for the student selected modules. This includes an overall percentage mark that is based on the university’s marking system. The assessment lead told us that so far, the evidence suggests the new mark sheet is clearer and fairer. Prior to the Year 2 and Year 3 exam boards we viewed a sample of SSM reports and results, including the convenor and second marker feedback sheets. We noted a difference in the quality and consistency of the feedback, with the new mark sheet and descriptors appearing to provide greater clarity in marking and reduced variability in feedback. 58. Although assessor feedback provided with the new mark sheet was generally good, it was not of a consistent standard and there were some examples of very brief, general feedback, which was also highlighted by the external examiners. The exam board acknowledged the inconsistency and outlined plans to improve this. The final Liverpool student selected modules have now been delivered, so going forward all student selected modules will be Lancaster’s own. Consequently, the consistency might improve as the assessors become more familiar with the mark sheet and further training is provided. Fitness for purpose of assessments (TD86) 59. The School is partially meeting this standard and we note the significant progress and improvement made since the 2012/13 visit cycle, when we set a requirement for the School to provide regular monitoring and reporting of its progress in this area. We have closely monitored the development of assessment items and papers, and were pleased to find that all the milestones and plans were met. 60. We heard that the current assessment lead is working on the psychometrics of the assessments; in particular, she is looking at improvements to individual questions where necessary. 61. This year, the School has delivered electronic assessments through Rogō for Years 1 and 2. The development and delivery of the Year 3 papers has been shared with Liverpool, with each school producing a written paper. 62. We heard of contingency plans in place to manage risks of delivering electronic assessments. The School notifies University’s information security department of the dates and times of exams so it can avoid restarting the systems during this time. A member of the team checks in advance that all the computers to be used for assessment have had updates run. 63. The Cronbach’s Alpha 1 reliability scores for the assessments for both year 2 and year 3 were below 0.7 in five of the seven assessments. The School’s information states that a score in the range 0.7-0.9 is good. We will continue to monitor progress in assessment development, including reliability, over the next cycle of quality assurance activities. Assessment of Tomorrow's Doctors outcomes including practical procedures (TD112) 64. The School is partially meeting this standard. We raised a requirement following the November 2013 visit for the School to ensure the contingency plans for the decoupling from Liverpool University medical school continue to be feasible; and we required that the Lancaster curriculum be mapped to the new Liverpool curriculum to ensure no students are placed at a disadvantage for assessments. Both Schools have signed the variation of the consortium agreement. Liverpool is preparing to roll out a new curriculum and Lancaster confirmed it has received an overview of the timetable for the five years of the programme, but is awaiting the learning outcomes. 65. Lancaster has documented all changes to its curriculum and has mapped it against Tomorrow’s Doctors 2009. Lancaster advised it is awaiting Liverpool to do the same and it will then be able to map the curriculum to the new Liverpool curriculum. The Director of Medical Studies is confident any gaps would be identified and remediated if necessary so that students would not be disadvantaged if they had to undertake Liverpool assessments. We will continue to monitor developments over the next cycle of visits. 66. We set a requirement for the School following the 2012/13 visit cycle to ‘provide an assessment blueprint that shows in detail how the assessments used map against the individual objectives rather than higher level themes. This should show sampling strategy and define how validity will be demonstrated and how a reliable result will be arrived at’. Blueprints for written assessments have been provided as well as a clinical skills map for Years 1-5. Design and delivery of assessments (TD113) 67. The School is meeting our standards in this area. This year, the School had planned to run its own Year 3 OSCE for the first time, using a combination of existing Liverpool stations and some new stations it had developed itself. However, Liverpool made changes to the Year 3 OSCE following a review in the months preceding the assessment. Lancaster were informed of this late in the planning cycle and after consideration it concluded that it had to run the new Liverpool OSCE in its 1 Cronbach’s Alpha is the most commonly measured aspect of reliability of a test, checking for internal consistency. The closer to 1.0, the more reliable the assessment. entirety, to ensure equivalence as current Year 3 students have dual registration and will receive a Liverpool degree upon graduation. We were very concerned that, only four weeks before the OSCE, Lancaster had not yet received full details of the new OSCE and could not assure us that the assessment would be fit for purpose. 68. The students felt anxious about the changes. We fed back our concerns to the School on the day, and subsequently held a conference call with the Heads of School for Lancaster and Liverpool, to discuss the work needed to ensure that the Year 3 OSCE would be fit for purpose. We requested and received additional documents and assurances from both Schools that the information and paperwork required to deliver the OSCE had been completed and shared. We attended and observed the OSCE and the exam board at Lancaster. Despite the late changes, the OSCE was well organised and well run by Lancaster and there did not appear to be an impact on the day. Nonetheless, the Cronbach’s alpha reliability score for the OSCE was only 0.53. From next year, Lancaster will be running its own Year 3 OSCE rather than using Liverpool’s so such a situation with late changes should not reoccur. 69. Some of the OSCE stations we observed could be improved; in particular, we felt the ‘headache’ station could include an element of probable diagnosis following the consultation with the patient. The School’s report on the OSCE showed that similar feedback was given by one of the examiners as there was no assessment of global diagnosis in the mark scheme for this station. 70. We also heard from the Year 3 students about a switch to a Liverpool Year 3 written paper, including unexpected assessment item format types (extended matching items). We were informed this would also include some content which the students had previously been informed would not be included (Individuals, Groups, and Society). Following our visit the School confirmed it had provided the students with a copy of the formative exam items that were taken by the Liverpool students. 71. We heard an example of a perceived inconsistency from the formative Year 3 OSCE, in which one student reported having been marked down in one station for re-sheathing a needle, but another student received the opposite feedback for the same station, and was marked down for failing to do so. At the Year 3 OSCE we observed the examiners behaving appropriately and keeping to their brief. The simulated patients used were of a high standard and were consistent in following the instructions. Guidance about assessments (TD87) 72. The School is meeting this standard with students being provided with comprehensive information about each element of their assessments, which we viewed prior to the Year 2 and Year 3 exam boards. As a result of the School having to run the Liverpool OSCE for Year 3, and the need to ensure equivalence with Liverpool, these students were provided with prior information about the stations that would be appearing. We consider that having this prior knowledge may result in students focussing on one skill to the detriment of the development of others and could significantly reduce the value of sampling across skills in OSCEs. The external examiner also commented on the negative impacts of providing station information ahead of the assessment, and the School confirmed at the exam board that there are no plans to provide such information for any future OSCEs. Assessment criteria and marking guidance (TD115) 73. The School is meeting this standard. At the Year 3 OSCE we observed an appropriately informative briefing for the examiners prior to the assessment. The assessment lead advised that all assessors have completed training to support marking the OSCEs with tablet computers. The examiners we met with on the day were positive about the use of the tablets and we observed how detailed individual feedback and notes were recorded for each individual student at each station. Although there were no issues with the tablets on the day, risks associated with use of technology have been well managed. This includes having paper mark sheets available as back up, chargers for the tablets at each station as well as spare tablet computers and a central charge station. See good practice 2. 74. This year the School introduced a professional practice, values and ethics (PPVE) case analysis for Year 3. Students are required to identify and write about an ethical dilemma witnessed on a clinical placement and apply ethical theories to this. We viewed a sample of the work along with the mark sheets and feedback. Each case was marked by a pair of assessors – one academic and one clinician. There was a variation in the scores awarded by the first and second markers, with the final mark being an average of the two, unless there was a difference of 10 marks or more, in which case the assessors would meet to discuss each section. The external examiners also noted this variation and the School advised that feedback has been provided to all of the assessors and some examples of good quality feedback have been produced to support their training. The descriptors are being reviewed with the aim of bringing them in line with the quality of the students selected module mark sheets. Standard setting (TD89) 75. The School is meeting this standard. As we have previously heard, the School uses Modified Angoff to standard set the written exams. Borderline regression is used to standard set the Year 1 and 2 OSCEs and for the majority of Year 3 OSCE stations this year. For 2013/14, responsibility for the Year 3 assessments was shared with Liverpool and so the common elements (the OSCE and one of the written papers) were jointly standard set. An external examiner, who is also an external for Liverpool, considered that there was equivalence between the two schools. 76. A webpage is available for students to provide evaluation of the assessments. Statistics from Rogō support the standard setting meetings. Individual questions are discussed and student evaluation is also considered. We observed during both exam boards the good extent to which the analysis of the results and the evaluation was deliberated. The exam board discussed and agreed decisions to remove particular questions and in one case changed the standard setting method for an OSCE station that did not perform as expected. See good practice 2. External examiners (TD116) 77. The School is meeting this standard and has appointed all of the external examiners it currently requires. External examiners attend all of the exam boards. We heard that a one day meeting with all external examiners was held in January 2014, which the School considered useful and allowed discussion of the curriculum. At the Year 2 and Year 3 exam boards, we observed that the external examiners were appropriately involved in the process, and the insightful feedback they provided was clearly valued by the School. The School welcomed the advice offered by the external examiners about how it might change its processes in future. QAA Code of Practice (TD119) 78. The School is meeting this standard. We previously set a requirement for the School to provide anonymised documentation at the Year 5 final exam board, as, when we observed the exam board during the 2012/13 visit cycle, documentation provided to the members of the exam board displayed student names. The School has assured us that all summary documentation provided at the meeting is now fully anonymised and so individual students cannot be identified. We observed that all documentation at the Year 2 and Year 3 exam boards was anonymised. This requirement is now closed. Reasonable adjustments (TD90) 79. The School is meeting this standard. We heard that Rogō allows changes to be made for individual students as required, for example, the font size, type and colour can all be adapted to meet individual students’ needs. The system retains the changes so that each time a student undertakes an assessment the appearance is automatically changed for them. See good practice 2. Domain 6: Support and development of students, teachers and local faculty 122. Students must receive both academic and general guidance and support, including when they are not progressing well or otherwise causing concern. Everyone teaching or supporting students must themselves be supported, trained and appraised. Guidance about the curriculum (TD123) 80. The School is meeting the standard for providing guidance on the curriculum. The curriculum lead explained that each cohort receives a lecture about the upcoming curriculum, which includes an opportunity for questions and answers as well as a ‘You said, we did’ briefing. All of the changes are detailed at the lecture as well as on Moodle. The student parliament continues, with two meetings per year. The students confirmed the student parliaments are well run and that changes are implemented as a result. 81. CPNFT provides a half-day induction on the first day of each placement. The students are oriented to the unit and introduced to key staff. They also talk about risk management and safety for themselves and patients. Students receive a timetable to attend particular aspects of the service and the mid-point review with the medical education facilitator provides an opportunity to change the timetable if needed. Feedback sessions are also held with the students at the end of the placements. Academic and pastoral support (TD124) 82. The School is meeting this standard. We heard of improvements to pastoral support for students completing placements at Furness General Hospital. There is now a flat provided, so that if a student is feeling unwell or for other reasons is unable to travel, they can stay at the flat if needed. The students were clear whom in the School or local education provider they can contact for support when required. 83. As on previous visits, the students we met told us they like the School and feel the small size is beneficial in many ways. In particular, they feel they get more contact time with senior staff at the School as well as clinicians, and they know individual students well. They also know the other year cohorts and are able to help each other. Overall the students would recommend studying at the School and those in Year 5 had no doubt students in the earlier years of the programme would be getting a good experience based on their evaluation and the changes implemented by the School. Training the trainers (TD148) 84. The School is meeting this standard. CPNFT representatives informed us that most of those involved in undergraduate teaching are also trainers for foundation and other doctors in training. They told us they are kept up to date with changes to the curriculum, meet with the Head of School once a year and also receive regular emails from School. Attendance at the Training in Practices and Education day events for general practices is now tracked via the quality assurance master spreadsheet. Domain 7: Management of teaching, learning and assessment 150. Education must be planned and managed using processes which show who is responsible for each process or stage. Management plan (TD151) 85. The School is currently meeting this standard. Professor Neil Johnson has been appointed as the new faculty dean and took up post officially from 1 August 2014. Professor Johnson is currently Pro-Dean of Education at the Warwick Medical School and a General Practitioner. There have been no other significant changes to the staffing and leadership at the School. 86. The School continues to search for a replacement for the Head of School, who plans to retire in the coming years. There is an open requirement for the School to ensure that there is clarity and action around succession planning to ensure the future expert leadership and management of education strategy, development and delivery required to establish an independent medical school. We will continue to monitor this closely and recognise that the School perceives this as its major risk. See paragraph 32 and requirement 2. Teacher involvement in curriculum management (TD152) 87. The School is partially meeting this standard. We previously reported on the dissociation of Furness General Hospital from Royal Lancaster Infirmary and the School, and recommended consideration be given to the use of alternative means of inclusion such as using video conference facilities to maintain relationships across sites and with the School. We heard that video conferencing was discussed and trialled but did not receive positive feedback, probably as the School’s system did not have good compatibility with the NHS system. The CQC report published on 26 June makes reference to the ‘cultural disparity between the hospital sites’ and found that ‘staff felt that they were not always listened to and that Trust’s Executive Team needed to be more visible’. We will follow this up during the next cycle of quality assurance visits. 88. The School informed us that an anaesthetist based at Furness General Hospital has been appointed as the deputy lead for Year 5. The School also delivered an education day at both Furness General Hospital and Royal Lancaster Infirmary this year. 89. CPNFT said it has a good relationship with the School and participate in student selection and admission, PBL, as well as assessments including the LOCAS (Liverpool Objective Clinical Assessment System) and OSCEs. We heard the Furness General Hospital is keen to continue to build a strong partnership with the School. Agreements with local education providers about curriculum delivery (TD157) 90. The School is meeting this standard. CPNFT told us that like many other local education providers, job planning for consultants’ educational roles is imperfect and historically reliant upon good will. It is currently in the process of drawing up educational agreements with all undergraduate and postgraduate trainers. This will form part of the appraisal and job planning process. There is a separate educational appraisal completed for those with educational roles in the Trust. Domain 8: Educational resources and capacity 159. The educational facilities and infrastructure must be appropriate to deliver the curriculum. Clinical skills facilities (TD166) 91. The School is partially meeting this standard and there have been positive developments with plans agreed for new clinical skills laboratories and an improved location for the OSCEs. During 2014/15 a temporary improved location for the OSCEs will be provided at Royal Lancaster Infirmary, with a new purpose built facility on site at the hospital expected to be completed in time for the 2016/17 assessments. 92. This is a very positive development, as the current OSCE location at Queen Victoria Hospital is not fit for purpose. The venue is small and there is noise pollution between the stations, for example, there was a mental health station curtained off on the corridor that received some ambient noise. Appendix 1: Visit team Dates of visits 11 June 2014 (Lancaster medical school) 10 July 2014 (Year 3 OSCE) 16 July 2014 (Year 2 exam board) 23 July 2014 (Year 3 exam board) Team Leader Professor Paul O’Neill Visitor Dr Steve Ball Visitor Mrs Sue Hobbs Visitor Dr Matt Kirkman Visitor Dr Will Owen GMC Staff Trish Steele, Jennifer Barron and Emily Saldanha Appendix 2: Document register Document number Document name Description Publication date and version Source Dec 13 doc 1 Monthly assessment update December 2013 Update on progress in developing, quality assuring and delivering assessments December 2013 Lancaster Medical School Feb 14 doc 1 Monthly assessment update February 2014 As above February 2014 Lancaster Medical School Mar 14 doc 1 Monthly assessment update March As above March 2014 Lancaster Medical School 2014 Apr 14 doc 1 Monthly assessment update April 2014 As above April 2014 Lancaster Medical School May 14 doc 1 Monthly assessment update May 2014 As above May 2014 Lancaster Medical School May 14 doc 1a Safeguarding leaflet Guidance for students on their responsibilities and reporting procedures for safeguarding issues May 2014 Lancaster Medical School/ UHMBNFT May 14 doc 1b Patient Safety Guidance for leaflet students on procedures for raising concerns about patient safety and whistleblowing May 2014 Lancaster Medical School/ UHMBNFT May 14 doc 2 Current risk register School’s risk register May 2014 Lancaster Medical School May 14 doc 3 Minutes of decoupling meeting - 6 February 2014 Minutes of meeting with Liverpool medical school May 2014 Lancaster Medical School May 14 doc 4a GP placement Quality Assurance Policy Quality assurance for community and primary care placements at Lancaster Version 3.0 Lancaster Medical School May 2014 Medical School May 14 doc 4b GP placement Update report Quality Assurance update for 2013-14 May 2014 Lancaster Medical School May 14 doc 4c GP placement Quality Assurance master spreadsheet May 2014 Lancaster Medical School May 14 doc 4d GP placement Reports from QA visit individual QA reports visits to general practices May 2014 Lancaster Medical School May 14 doc 5a Quality Management Framework Quality Management Framework for the MB ChB programme May 2014 Lancaster Medical School May 14 doc 5b QM Framework Appendix 1 - May 2014 Lancaster Medical School May 14 doc 5c QM Framework Appendix 2 - May 2014 Lancaster Medical School May 14 doc 6 Summary of the placements provided by Cumbria Partnership NHS Foundation Trust - May 2014 Lancaster Medical School June 14 doc 1a-b Emails from UHMBNFT to 2 x Emails regarding changes to 11 June 2014 Lancaster Medical School the School leadership at the local education provider June 14 doc 2 Learning objectives for Year 1 formative paper one Example of the 11 June 2014 learning objectives sampled by a specific paper Lancaster Medical School June 14 doc 3 Minutes of the decoupling meeting 2 June 2014 Minutes from the sixth joint decoupling meeting between Liverpool and Lancaster medical schools 2 June 2014 Lancaster Medical School June 14 doc 4 Year 1-5 clinical skills map Mapping of clinical skills against TD09 outcomes showing when assessed May 2014 Lancaster Medical School June 14 doc 5 Year 3 Assessment Including presentation slides, copies of emails and information relating to the individuals, groups and societies element of the Liverpool formative exams June 2014 Lancaster Medical School Outline of information provided to June 2014 Liverpool Medical School Summary of information sent to students June 14 doc 6 Responses to queries the GMC following conference call on 15 June 2014 June 14 doc 7 Year 3 general examiner briefing Slides from the Summer Year 3 OSCE 2014 examiner briefing Liverpool Medical School June 14 doc 8 Changes to the format of the Year 3 OSCE Details and June 2014 background of the changes to the 2014 OSCE Liverpool Medical School June 14 doc 9 Email to students Copy of email to students outlining changes to the OSCE June 2014 Liverpool Medical School June 14 doc 10 Genetics station Documentation June 2014 about the genetics station including old and new weightings, mark sheets and instructions to illustrate the changes to the OSCE Liverpool Medical School June 14 doc 11 Psychosis station Documentation June 2014 about the psychosis station including old and new weightings, mark sheets and Liverpool Medical School instructions to illustrate the changes to the OSCE June 14 doc 12 Comment sheets Sample of examiner comment sheets from other summative OSCEs 2014 May/June 2014 Liverpool Medical School June 14 doc 13 Student briefing video Video viewed by students before the OSCE Summer 2014 Liverpool Medical School June 14 doc 14 Year 3 14 complete preOSCE information PowerPoint presentation provided to students ahead of the summative OSCE Summer 2014 Liverpool Medical School June 14 doc 15 Year 3 Global descriptors for OSCE performance descriptors for standards expected of a third year medical student Summer 2014 Liverpool Medical School July 14 doc 1 Examiner brief Year 3 summative 2014 Presentation slides from OSCE examiner briefing July 2014 Lancaster Medical School July 14 doc 2 OSCE map 3rd year Summative map 1314 FINAL July 2014 Lancaster Medical School July 14 doc 3 OSCE info for students y3s 14 complete pre- July 2014 Lancaster Medical School OSCE information Lancaster version July 14 doc 4 Global descriptors year 3 Global Descriptors for OSCE performance July 2014 Lancaster Medical School July 14 doc 5 Examiner info Year 2 exam board paperwork July 2014 Lancaster Medical School July 14 doc 6 Student briefing Student brief Year 3 Summative OSCE July 2014 July 2014 Lancaster Medical School a) Agenda b) Results for board c) Year 2 standard setting report 2013-14 d) Year 2 summative exams 201314 feedback for students e) Year 2 summative OSCE report f) Year 2 exam board report paper 3 2014 g)Year 2 summative exams reliability a) Agenda b) Results for board c) Year 3 standard setting report July 2014 Lancaster Medical School July 2014 Lancaster Medical School July 14 doc 7a-g July 14 doc 8a-h Year 3 exam board paperwork 2013-14 d) PPVE Case Analysis Coursework Summative Exam Board 2014 e) Reliability statistics f) Year 3 Summative OSCE Report 2014 g) Year 3 Summative Exams: Student Feedback h) Year 3 Summative Exams: Reliability Appendix 3: Abbreviations CPNFT Cumbria Partnership NHS Foundation Trust CQC Care Quality Commission GMC General Medical Council GP General practice/practitioner LEP local education provider LOCAS Liverpool Objective Clinical Assessment System MB ChB Bachelor of Medicine and Surgery MSAR Medical school annual return NHS National Health Service OSCE objective structured clinical examination* PMQ Primary medical qualification PPVE professional practice, values and ethics QIF Quality Improvement Framework QAA Quality Assurance Agency SAMP selective in advanced medical practice UHMBNFT University Hospitals of Morecambe Bay NHS Foundation Trust *See glossary (in appendix 4) for definition. Appendix 4: Glossary OSCE A type of examination to test clinical skill performance and competence in skills such as communication, clinical examination, medical procedures or prescription, exercise prescription, joint mobilisation or manipulation techniques, radiographic positioning, radiographic image evaluation and interpretation of results. Rogō University of Nottingham e-Assessment system that is used to create, quality control, and deliver online assessments. Moodle Virtual learning environment. A tool used by staff to add resources for students to access online to enhance teaching and learning efforts. Lancaster Medical School Response to GMC final visit report 2013-4 We were pleased to receive this report and to note that the visiting team have considered we have made much progress since their visit in November 2013 and that they have identified a number of additional areas of good practice. Response to requirements • The School must ensure students are provided with written guidance so they are clear which clinical procedures they are able to perform when on clinical placements. We had already recognised that this was an area of ambiguity for students as well as for the clinical teachers and had been working on a document to make this clearer. This document is now complete, has been through the committees of the Medical School and is already incorporated into the appropriate year log books for all students from the beginning of this academic year. The information is also on the Trust Intranet and has been disseminated to all GP practices who take our students. The document is appended to this response. • The School must formulate detailed plans to address the two long standing open requirements concerning the appointment of a successor to the Head of School and ensuring the clinical experience of students is not compromised by the reconfiguration of healthcare services in the area. The post of replacement Head of the Medical School is about to be re-advertised We understand that there are two areas of concern regarding our clinical placements a) that our students have expressed concerns about whether or not they are exposed to a sufficient breadth of clinical experience (paras 48-50) b) concerns around the local Acute Trust as identified by CQC and the GMC (paras 36-38) Furthermore we recognise that these two areas of concern could potentially be worsened by the Trust’s plans for reconfiguration of their clinical services. However we also consider that reconfiguration as proposed by UHMB Trust may well offer significant opportunities for providing a curriculum that is reflective of future models of healthcare delivery rather than old models. As regards the first area of concern, we are confident that our students do receive sound clinical experience. This confidence is based on the evidence that we have now graduated four cohorts who have gone on to fill Foundation posts across the country and the anecdotal reports we receive from both our graduates and their employing Trusts is that they are well prepared for their role as Foundation doctors. In addition, the feedback we have from HENW is that, while the numbers are small, our students have a lower incidence of ‘doctors in difficulty’ than average As regards the second area, we are confident our students receive a good educational experience at the Trust as evidenced by the feedback we monitor from their placements and by their performance in the exams which they sit jointly with Liverpool students and in which they perform comparably. Nonetheless, we recognise the importance of mitigating the concerns raised by the visiting team and are currently working on • • • • developing plans with other local Acute Trusts for our students to undertake specialised placements working with UHMBFT, and other Trusts involved with them in the plans for new patterns of service delivery in Cumbria and North Lancashire, so that as they develop, our students will be able to experience these new models of care provision maintaining a very close working relationship with the local health economy so that the clinical placements are not affected by changes in service provision and that changes in service provision are reflected in the curriculum monitoring the situation through our regular meetings of the Curriculum Co-ordination Group, student feedback and the School Learning and Teaching Committee. Should there be any evidence from our monitoring, or from any other quality monitoring processes, that our students are being affected by the situation in the Trust, the we will develop, as a matter of urgency, contingency arrangements to deal with this.
© Copyright 2024 ExpyDoc