Poor performance in GPs and revalidation

Cox, S., & Holden, J. (2009). Presentation and outcome of clinical poor performance in one health district over a 5-year period: 2002–2007 British Journal of General Practice, 59(562), 344-348 DOI: 10.3399/bjgp09X420527

I’ve been ruminating about revalidation for UK doctors over the past week. This was precipitated by a meeting I attended last week on work-place based assessment. Revalidation is essentially a slimmed down, palatable work-place based assessment (WBPA) and most of the usual suspects are there: patient questionnaires, 360° feedback, significant event analyses, audit etc.

Several of the doctors at the meeting, including representatives from the PCT and the Deanery, all seemed to make it clear that those GPs that have concerns raised by their revalidation will have to sit the Applied Knowledge Test (AKT) and the Clinical Skills Assessment (CSA). I nearly fell off my chair. Pulse suggested that this could be as much as 5 to 14% of GPs. My first reaction is that this is shockingly flawed. Knowledge tests like the AKT and the CSA don’t measure the same elements as an ‘enhanced’ appraisal. While I’m on the subject, where is the evidence that clinical competence can be fully measured through the work-place based assessment that is proposed? It isn’t regarded as being adequate for GP trainees. The Royal College mentions this about WPBA:

WPBA is a developmental process. It will therefore provide feedback to the [GP trainee] and drive learning. It will also indicate where a doctor is in difficulty. It is learner led: the [GP trainee] decides which evidence to put forward for review and validation by the trainer.

Revalidation purports to make a learner driven developmental tool into a reassuring summative measure of doctor professional competence and behaviour. And there seems to be the bonkers idea mooted that this can work both ways. There has been speculation that GPs could opt to do the exams rather than jump through the other hoops. This is even more crackers. Work-place based assessment has limitations but I also appreciate that there is far more to ‘poor performance’ than the exams can measure.

Presumably the aim of all this is to pick up poor performance by GPs. The BJGP has just published a paper looking at performance issues: Presentation and outcome of clinical poor performance in one health district over a 5-year period: 2002-2007. What can it contribute to this morass?

The study was based in St Helens (oop north if you were wondering) and there were 102 individual presentations of issues which raised significant concerns. These 102 presentations were spread over 37 individual clinicians. However, only 27 of these were GPs and the others were dentists, nurses and community pharmacists. So what’s the denominator? St Helens PCT covers 35 practices and there was an average of 130 GPs on their list over this period.

The paper has documented the source of these presentations. Interestingly they found that whistleblowing was the major source and it accounted for 43/102 cases. Given the recent furore with whistleblowing it highlights that whistleblowing exists on many different levels. National scandals which will jeopardise your registration (and where you feel duty-bound to invite in a Panorama film crew) are rare beasts. Complaints were next on the list with 18/102. There were several other sources but notably GP appraisal highlighted just 3/102 cases. The paper suggests that:

“Combinations of attitude, behaviour, and probity issues were seen in 84% of the more serious cases.”

They quote another paper based in Tyneside where only 25% of cases had clinical care as a cause for concern but they highlight GMC findings that 46% of cases had seriously deficient levels of knowledge. They also note that:

“In this study of primary care poor performance cases, the less serious locally-managed cases showed a much greater preponderance of knowledge, skills, or medicines-management issues.”

As the authors acknowledge, the study is limited to one area. A major factor in this district was that there had been a high profile case with the conviction of a criminal GP who had sexually assaulted female patients. The numbers in the area seem high but the authors suggest in their discussion that:

“A robust, developmental but non-punitive approach could have led to this apparently high incidence of performance concerns, which is probably more reassuring than a misleadingly, artificially low incidence.”

Indeed. Out of the 27 GPs there were 23 that had their cases managed by the group and are all now working locally in unrestricted practice. Does this suggest some hysterical over-reporting of concerns in a group of nervy GPs scarred by the experience of having had a criminal GP in their midst? Well, this study certainly can’t answer that question.

So let’s consider some key points, and in particular, the key points to inform revalidation. Well, information on the incidence and prevalence of poor performance is poor but it may be as high as about 20% according to this study. If revalidation further raises the number of concerns in GPs there is going to have to be a spectacular amount of resources put in place to manage the aftermath. St Helens took an average of 19 months to manage their cases and attempt to remedy problems. This took several weekly sessions of work from a veritable army of doctors, nurses and pharmacists plus a full-time administrative assistant and data analyst time.

How many of these presentations would be elicited by revalidation? If you look at the 102 presentations I suspect only patient complaints (18) and GP appraisal (3) would be the ones picked up by the proposed revalidation process. I find it difficult to believe that 360° feedback is going to have the same effect as whistleblowing to the PCT. Arguably, the patient complaints would go directly to the PCT anyway.

It’s a crucial point from this paper that the presentation of poor performance would not obviously be picked up through revalidation. Whistle-blowing, patient complaints and meds management will all go on regardless. The WBPA that GP trainees go through also involves observations by trainers who have a close individual relationship to the trainee.

This study did not directly address this issue but it seems unlikely that revalidation would add to the number of GPs in St Helens with performance issues. Indirectly, it begs the question: what is the point of revalidation? The formative, development and constructive aspects of appraisal will be marginalised and rendered useless while this study could infer that it won’t even add to the identification of performance issues anyway.