FPM Exam Handbook

Why was the FFPMRCA examination introduced?
Following discussion with RCoA Council, it was agreed in 2008 that an examination needed to be created to award post-nominal Fellowships of the Faculty.  The Faculty Board introduced the examination, in line with other international pain examinations, to further drive up the standard and standing of Pain Medicine in the UK and of UK Pain Medicine internationally.

At the MCQ, I informed the invigilator of a question I considered unclear/contentious.  What happened next?
All questions are reviewed by the MCQ Core Group before and again after each sitting of the MCQ paper.   All queries about questions raised by candidates sitting the MCQ paper are discussed in detail at the MCQ Core Group’s meeting held a week after the MCQ paper.  Any questions assessed as being contentious or unclear are discounted from the marking of the paper.

What is the revision syllabus for the examination?The syllabus for the examination is the Stage 1, Stage 2, Stage 3, and Stage 3 Special Interest Area (SIA) Pain Medicine domains of the CCT in Anaesthetics curriculum.

A trainee who took and passed the examination has also written an article for Transmitter with a full library of suggested reading. The Training & Assessment Committee has provided a curriculum expansion document.

How is the examination fee set?
The Examination fee is set by the RCoA and the FPM and is designed to break even on paper (and in reality lose money once overheads are included). The Examination is subsidised by Fellowship subscriptions in order to not fully pass the cost burden onto examinees. The FPM appreciates that the cost is higher than the FRCA (although still lower than other Colleges) – holding a high standard, well designed, quality assured and continuing improving examination is not an inexpensive endeavour and the comparatively small numbers undertaking the exam mean we benefit less from economies of scale.

Does the FPM review the examination’s impact on the overall cost of training?
Yes.  The Royal College of Anaesthetists (RCoA), Faculty of Pain Medicine (FPM) and Faculty of Intensive Care Medicine (FICM) ensure that trainees, members and fellows are provided with an appropriate breakdown of costs of supporting trainees and in the provision of the examinations that form part of the relevant training programmes.  For the purposes of this analysis, the income and expenditure for FRCA, FFICM and FFPMRCA examinations have been combined, as the vast majority of the functions are managed centrally by the RCoA examinations department to improve economies of scale.

Each department within the College and each Faculty has its own budget which is managed by the relevant head of function and the overall directorate budget is the responsibility of the relevant director.
Further details on the cost of training are available.

Why is the MCQ almost as expensive as the SOE, which is a more complex examination to run?
It is entirely understandable for those taking the exam to perceive the actual examination days themselves as the major part of the exam process. In reality, each exam sitting, whether MCQ or SOE, takes a considerable amount of development, both in times of physical days (Standard Setting Groups, Core Groups for question writing, training etc.) and remote development (IT systems, engagement with the Academy/GMC on national assessment changes). A difference in fee between the written and oral components of the exam has been agreed to reflect the different costs of the actual exam days, in similar ratio to that of the FRCA.

Is there an MCQ booklet along the lines of the FRCA?
The FRCA has been in existence for a considerable time longer than the FFPMRCA and therefore has a bank of question many times bigger, allowing a number of used questions to be released for revision purposes. So far the FFPMRCA has released a small sample of questions (see below). We will continue to release example questions at a rate made possible by question writing. The Court of Examiners is already considering ways to increase question writing to support this.  Please see the example questions document in the FFPMRCA examination preparation section.

What do I do if I have passed the MCQ but have been unsuccessful in the SOE?
If you are in this situation, it may be worth considering a guidance interview, the details of which are in chapter 8 of this guide.

How is the pass mark decided?

Quality assurance and standard setting are based upon the established processes which have been approved by the GMC. The questions are benchmarked for difficulty and relevance to Pain Medicine.  The general principle is that we will be looking for the standard of a doctor ready to make the transition to the level of a competent and knowledgeable Pain Medicine consultant.

For the MCQ
The pass mark for each MCQ exam is set using a modified Angoff method. An Angoffing group, comprising examiners and independent Pain consultants (including junior consultants who have recently attained the Fellowship), grade each question in the paper according to difficulty using the proxy of the likelihood of a borderline candidate knowing the answer. A week following the exam a meeting takes place in which the group spends several hours discussing the scores given, with particular regard to the questions where there is disparity in the responses. Once an agreed score for each question has been reached a raw score for the paper can be calculated. The raw score is adjusted to allow for the potential for guessing the correct answers.

The pass mark is set as the adjusted raw score further reduced based upon the Standard Error of Measurement (SEM), a statistic relating to the reliability (Kuder-Richardson 20 score) of the exam. This resulting figure is then rounded down to the nearest whole number to give the pass mark. This is applied to the final question set following scrutiny of every single question leaf, and removal of questions where the group have reason to believe that the answer is either incorrect or the question is ambiguous, and therefore not reliable.

For the SOE
At least two standard setting tools are employed in setting the pass mark for the SOE component of the exam. Prior to the exam, examiners are asked to consider the relevance and difficulty of all individual SOE questions. This ensures that exams of broadly similar difficulty and relevance overall are used- review of this data has indicated that this metric is stable, that is all exams are similar in this regard. Prior to the exam, all SOE questions are presented and discussed in detail, to clarify the standard expected.

The questions are asked in a standardised way in the exam itself. During the exam, in addition to candidates being marked on each question, performance at the exam is graded by examiners on a global ratings scale. This global rating is then cross-referenced against performance in individual questions using a linear regression method. The suggested pass marks generated by these two methods (which, to date, have been the same figure) are then used as a basis for discussion between examiners.

The individual performance of all candidates whose scores are just above or just below the suggested pass mark are reviewed by the court of examiners in coming to a decision regarding whether they have demonstrated sufficient knowledge and understanding to pass the exam. Discussion of these borderline candidates’ performances’ determines the final decision of where the pass mark should be set. The pass mark has demonstrated remarkable stability over time with these measures, with little variability, found to be either 31, or 32, out of 40.