The Faculty of Pain Medicine welcomes the publication of the independent review of all examinations delivered by the Royal College of Anaesthetists, conducted by Professor John McLachlan in 2022. We are committed to delivering fair, robust, and high-quality examinations, and this report will inform our developmental roadmap for the future.
We understand that postgraduate examinations are high stakes assessments that have a significant impact on candidates’ personal and professional lives. We also recognise the hard work of candidates, trainers, examiners and the RCoA exam team in making the FFPMRCA exam successful as a measure of attainment against stringent standards of knowledge, skill, attitude and critical thinking inherent to a high performing pain physician. The review sought to capture the experiences and opinions of all these stakeholders, and we are grateful to everyone who has participated in the review process, and to Professor McLachlan for conducting the review.
We will implement changes with care over the next few years, taking into account the needs of candidates. We will ensure exam candidates are given at least 12 months’ notice of any significant changes to the exam in order to facilitate good preparation and support. Therefore, doctors in training and prospective candidates should not change the way they prepare for the current sets of exams until formally notified of such changes.
Scope of the independent review
The Royal College of Anaesthetists commissioned the review to provide independent assurance and insights into the development and delivery of all its assessment processes across the FRCA, FFICM and FFPMRCA exams. It includes consideration of an error in the delivery of the September 2021 FRCA Final Written Exam and an unexpectedly low pass rate for the October 2021 FFICM Objective Structured Clinical Examination (OSCE).
We welcome the all-encompassing nature of this review and acknowledge the recommendations. While there is no explicit mention of the FPM exam, the recommendations provide an opportunity to improve processes and methods to further develop the purpose and validity of the FFPMRCA exam and improve the candidate experience.
We have published an Executive Summary and proposed timetable for implementation of the changes resulting from the review, and we highlight some key themes relevant to the FFPMRCA exam here.
Our programme of improvement
The FFPMRCA, now mature at 10 years old, has been subject to a process of continued quality assurance, and while rightly proud of its performance to date, we recognise that this is a timely opportunity to develop and improve the exam. In considering the recommendations from the review and implementing any subsequent changes, we will:
- Invest additional resources in our examinations function to implement the recommendations of the review and strengthen our capacity to deliver exams.
- Work closely with the RCoA and the Faculty of Intensive Care Medicine in delivering relevant changes to the FFPMRCA exam through a newly formed group, the Exam Development and Assurance Group. This group will lead the development, quality and alignment of exam processes across all exams delivered by the RCoA – where appropriate – to ensure they are delivered in line with best practice in assessment for each medical specialty. We believe this group, tasked with implementing expertly informed recommendations, will help the FFPMRCA exam develop and remain world class for the future.
- Give candidates and doctors in training a greater role in our assessment processes, including more representation on College and Faculty committees engaged in assessment. These representatives will also play a central role in supporting communication with candidates.
- Review the purpose of our assessments in accordance with the changes made in the curriculum and the changing nature of clinical practice in our specialties. We will consult with members and stakeholders to define this.
- As part of the Examinations Development and Assurance Group, we will contribute to the implementation of a unified approach to the use of, and training in, standard setting across all exams delivered by the RCoA.
- Use the findings of new research undertaken to inform our assessment design process. For example, in relation to validity and the impact of gender, ethnicity and educational background on exam performance.
We understand that members may have questions about the independent review, the applicability to FPM and the actions we are taking as a result. We welcome your feedback and encourage you to attend our Let’s Talk member engagement online event on 8 March at 19.00.
Members can also share their feedback in writing using our online form. This feedback will be shared with the new Examinations Development and Assurance Group.
Dr Lorraine de Gray, Dean of the Faculty of Pain Medicine
Frequently Asked Questions
Candidates and trainers do not need to change their revision/ study plans for the current exams. While it is likely that changes will be made to the exams’ formats in future, all our current examinations remain valid, reliable, fit for purpose, and are approved by the GMC. We will ensure exam candidates are given at least 12 months’ notice of any significant changes to exams in order to facilitate good preparation and support. Prospective candidates should not change the way they prepare for the current sets of exams until formally notified of any changes.
The College and Faculty exams continue to be valid, reliable, and fit for purpose. The structure, format and standard setting methods for these exams have been approved by our regulator, the General Medical Council (GMC), and we have maintained close contact with them throughout the review process. Any changes we make to these aspects of the exams will require approval from the GMC.
Best practice in examinations changes over time in line with research in assessment, assessment theory and changes in medical education. We are committed to the continuous improvement of our examinations, and the recommendations from both the internal review of the FRCA and the independent review conducted by Professor McLachlan provide us with a roadmap for further development and improvement.
We very much welcome trainee and candidate participation in our exams. Trainee representatives are present on the Examinations Committee as well as College Council and Faculty Board. The Examinations Department will continue to engage with the Anaesthetists in Training Committee, the Anaesthetists in Training Representative Group (ATRG) and the FICM StR Sub-Committee with updates on exams. Trainees have also supported us in the development of training materials for examiners and for exam candidates. All candidates receive a post exam survey to feedback on examination delivery and content and this feedback is reviewed by the operational teams for Primary, Final and the Faculties, and during moderation and item analysis meetings, as is any ad hoc feedback received.
We will provide updates on changes to the exams by direct email to the membership as well as on our website and social media channels. Regular updates to the Anaesthetists in Training Committee, the Anaesthetists in Training Representative Group (ATRG) and the FICM StR Sub-Committee will ensure that representatives from these groups also support the feedback of changes to trainees and our wider membership.
The assessment blueprints for the FRCA examinations are located in the core level and intermediate training syllabuses for Primary and Final on the RCoA website. Revised syllabuses for the 2021 curriculum will be available from summer 2023 and relevant key capabilities from the 2021 curriculum are already referenced in feedback to candidates for the written examinations.