How does the National Joint Registry's work benefit me? How can I become a part of the NJR's Regional Support network? Who are the working groups the NJR works with? We aim to answer all your questions whether it be in person at a conference, in our Annual Report and weekly reports or here on out website

The NJR Accountability and Transparency Model
FAQs

NJR Accountability and Transparency Model

Why does the NJR do outlier analyses?

Monitoring surgeon performance outcomes is a key function of the NJR, as part of the ongoing drive for greater transparency within the NHS to improve standards and safety. The NJR surgeon outcomes monitoring process identifies surgeons with revision ratios (an indication of implant failure or surgical performance) that are higher than expected and triggers a defined process to liaise with and support these surgeons so that an appropriate review of their practice can be taken to identify possible causes and actions. The NJR does not publish details of individual surgeon revision rates publicly via the NJR Surgeon and Hospital Website.

What outcomes does the NJR use?

The two outcome measures that the NJR uses in outlier analyses are: revision surgery of a primary hip or knee replacement; and mortality in the first 90 days following surgery for a primary hip or knee replacement. These events are used in the calculation of risk-adjusted standardised mortality ratio (SMR) and risk-adjusted standardised revision ratio (SRR).

Why do we risk adjust analyses?

Analyses are risk adjusted in order to make a fair comparison between different surgeons. For example, if one surgeon exclusively operates on patients with hip fracture we might expect them to have an increased revision rate due to peri-prosthetic fractures, compared to a surgeon who operates on younger patients with primary osteoarthritis. The intention of using risk-adjustment is to make a fairer comparison between surgeons, whilst accounting for differences in the case mix of patients.

When does the NJR review data?

The NJR reviews surgeon-level data twice yearly, in the months of June and November. Currently outlier analysis is undertaken for hip and knee, for revision and mortality.

What time-period is used for NJR outlier analysis?

Outlier analysis for hip and knee revision uses the last 5 years and the last 10 years of data to identify potential outlier surgeons. A surgeon could become an outlier over the last 5 years, the last 10 years or both 5 and 10 years for their hip or knee replacement revision measure.

For hip or knee mortality outliers, the NJR looks at the last 5 years of data only.

How does the NJR identify outlier surgeons?

To identify a surgeon who is an outlier for hip or knee revision surgery, the NJR uses standardised revision ratios (SRR).

To identify a surgeon who is an outlier for hip or knee mortality, the NJR uses standardised mortality ratios (SMR).

Outlier (also known as Alarm) level surgeons have an SRR or SMR which has crossed the 99.8% control limit, ~3 standard deviations from the expected number of revisions or deaths. This can be viewed on a funnel plot available from NJR Connect – Data Services.

For the November 2022 surgeon performance data review, the identification criteria and notification process for surgeons at alarm level for knee revision surgery will change:

  • Surgeons who reach alarm level for only their total knee replacement (TKR) data and/or their unicompartmental (uni) knee replacement data will be notified, and the medical directors (MD) of their hospital(s) will be notified six weeks later.
  • Surgeons identified as at alarm level for their entire knee practice only (i.e not for their uni or TKR practice in isolation) will be notified by the NJR, but in most cases this will no longer trigger a notification to their MD six weeks later.

In order to address the possibility that the overall knee practice data for a surgeon might indicate an imminent outlier status in the subgroups, for every surgeon whose overall knee practice demonstrates outlier status the SPC Chair will examine that surgeon’s results individually and assess whether there is a “case to answer”.

As a result of this examination, should it be felt that there is a potential problem then the surgeon and the MD would be notified in the same way as has happened up to now. More information on this is available on the NJR website at Changes to NJR alarm surgeon identification and notification procedures for knee

What is the SRR and how is it calculated?

The standardised revision ratio (SRR) is the ratio of observed revisions to expected revisions. An SRR of 1.0 means the observed number is as expected. Greater than 1.0 means it is more than expected. Thus an SRR of 2 means twice as many revisions than expected, while an SRR of 0.5 means half the number expected.

What do we adjust SRR for?

We adjust the expected number of revisions for gender, age of the patient (at the time of the primary procedure), ASA (American Society of Anaestheologists) grade, and the primary indication for the operation.

What counts as a primary procedure?

For hip replacement, conversions of hemiarthroplasties (which the NJR records on a H1 form) are excluded from this calculation, all other primary procedures are included.

For knee replacement, all primary procedures are included.

What counts as a revision?

A revision is defined as any operation where one or more components are added to, removed from or modified in a joint replacement or if a debridement and implant retention (DAIR) with or without modular exchange is performed. Capturing DAIR with or without modular exchange commenced with the introduction of MDSv7. Prior to this DAIR with modular exchange was included as a single-stage revision but DAIR without modular exchange was not captured. For surgeon, unit and implant performance analysis DAIR without modular exchange is not currently included as a revision outcome. However within NJR Annual Reports, DAIRs with and without modular exchange are both included in the analysis as a revision.

Examples of procedures which count as a revision episode are secondary patella resurfacing, second partial knee replacement or posterior lip augmentation device (PLAD). It remains mandatory for all these procedures to be entered into the registry using the appropriate revision form.

For historical reasons conversion of a hemiarthroplasty to a total hip is classified as a primary replacement, however these procedures are now excluded from outlier analysis.

As a consultant which procedures are attributed to me?

Each patient in the NHS is assigned a Consultant in Charge (CiC). That Consultant is ultimately responsible for that patient’s treatment and care. For this reason, we have always reported outcomes relating to the CiC, as well as for that individual when they are named as the lead surgeon for the procedure. Those as CiC will include all cases where they are the “named consultant”, such as those carried out by trainees, fellows and non-consultant career grade surgeons (NCCGs). Reporting these two metrics makes it clear what a surgeon’s individual outcomes are as lead surgeon, as well as those for which they are ultimately responsible as CiC.

It is important to remember that it is not just the revision cases that are attributed to the consultant in both of these measures, but the unrevised primary cases too.

Over the years we have found that the majority of consultants have very similar results as CiC and lead surgeon, with just a few consultants having their results either improved or worsened by inclusion of their trainees’ and other surgeons’ cases for which they are CiC.

It is possible to filter and view your data as both CiC and lead surgeon on the NJR Connect – Data Services portal. We also report outcomes as both CiC and lead surgeon as part of the Consultant Level Report (CLR) which we recommend should be used to inform discussion during your annual appraisal.

Some CiC surgeons do feel that the cases where they are not lead surgeon should not be attributed to them for the purposes of outlier analysis.  It is important in this context to remember that:

  • the data for ALL Consultants are treated in the same way in this regard.
  • removing some of the cases which have been revised will only provide a “balanced” assessment if the same thing is done for all other surgeons with whom the consultant is being compared.
  • removal of the revised cases performed by someone else gives a distorted view unless all of the unrevised cases performed by other surgeons are also removed from the calculation.
  • all surgeons have the data of their own lead surgeon cases provided, so they can easily demonstrate the difference to their appraiser and others if there is a significant discrepancy between the revision rates in the two roles.

How is the SMR calculated?

The standardised mortality ratio (SMR) is the ratio of the amount of observed deaths to expected deaths, measured at 90 days from the date of surgery.  An SMR of 1.0 means the observed number is as expected. Greater than 1.0 means it is higher than expected.

What do we adjust SMR for?

We adjust the expected number of deaths for gender, age of the patient at the time of the primary procedure and ASA (American Society of Anaestheologists) grade.

Are any procedures excluded from analysis?

There are a number of different procedures excluded from mortality analyses.

There are three principle exclusions:

  1. Primary procedures due to trauma (including conversion of hemiarthroplasty to THR) and cancer cases are excluded from the calculation of the SMR. The reason for this exclusion is because of substantially higher mortality rate in these patients. We also note the reason for higher than expected mortality in these patients is multifactorial and is monitored by the Falls and Fragility Fracture Audit Programme.
  2. If same-day primary bilateral hip or knee replacements are performed, only one procedure is used in calculation of the SMR. We exclude one procedure to avoid counting as two deaths, i.e. a left and right hip replacement, against one consultant/surgeon in relation to one patient death.
  3. If different day primary bilateral hip or knee replacements are conducted within 90 days of the index procedure, by the same consultant/surgeon, the second procedure is excluded from the analysis. We exclude the second procedure to avoid counting as two deaths, i.e. a left and right hip replacement, against one consultant/surgeon in relation to one patient death.

Note: If a different day bilateral procedure is conducted within 90 days of the index procedure, by different consultant/surgeon, the second procedure is included in the analysis and one death is attributed to each consultant/surgeon in the calculation of the SMR.

What is the difference between an Alert and Alarm outlier?

The NJR notifies Alert (borderline) surgeons when either their standard revision ratio (SRR) or standard mortality ratio (SMR) falls between the 95% and 99.8% control limits, (approximately 1.96 and 3 standard deviations) from their expected number of revisions or deaths. At this time, we do not contact hospital medical directors directly although we recommend that surgeons review their data and reflect on any potential changes they could make to their clinical practice.  Surgeons are asked to acknowledge receipt of an Alert letter to confirm they have received it, and once receipt of this has been acknowledged the NJR takes no further action unless a surgeon reaches Alarm level.

The NJR notifies an Alarm (outlier) surgeon when their SRR or SMR crosses the 99.8% control limit, approximately 3 standard deviations from their expected number of revisions or deaths. Six weeks later, their responsible officer and the responsible officer of any other NHS trust and/or any independent sector hospitals where they practice will be notified. A surgeon, working together with their responsible officer, should review their data and formulate an action plan to address any issues identified.

What are the main factors affecting revision?

There is no one single factor that explains higher or lower revision rates. However, factors such as case selection, implant selection, surgical technique and the revising surgeon’s revision threshold can all impact on a surgeon’s SRR. Reaching either alert or alarm status is often multifactorial. Following receipt of either an alert or alarm notification, the first step is for the surgeon to download revision data from NJR Connect and audit these cases. This audit should be to check the data quality and to review the causes of revisions to identify any causes which may not have been entered accurately at the point of data entry and could be corrected.

In the NJR Connect portal, surgeons are able to filter their data to review cases where they were the lead surgeon or cases where they were the Consultant in Charge. The NJR Consultant Level Report (CLR) also provides information broken down by procedure type, for example by cemented/cementless knee and hip replacement and unicompartmental knee procedure.

It is important to note that NJR outlier analysis uses all cause revision as the end point for all surgeons. Therefore, it is not possible for a particular group of revisions, for example secondary patella resurfacing to be excluded from an individual surgeon’s revision count to establish if they would therefore drop below the current threshold. The reason for this is that if it was done for one surgeon, it would need to be done for all other surgeons and in doing so, would alter the baseline revision rate. Often, this exclusion exercise makes no difference anyway to the outlier status.

What should a surgeon do if they receive an alarm level letter?

When a surgeon receives an alarm level notification, they also receive a recommended action plan to provide guidance with next steps. The first step should be to log into the NJR Connect portal and to download the relevant data. This data should then be audited to establish if there are any issues with the data quality, eg missing procedures or procedures attributed to the wrong Consultant in Charge.

Data should also be reviewed to establish what factors may have impacted on outcomes. This could include factors such as implant choice, procedure type, case selection, infections and revision threshold. It is recommended that this review is undertaken with a clinical director. Following this audit, surgeons should develop an action plan to address any factors which may have impacted on outcomes. Some examples of potential actions as a result of reviewing these include:

  • Changing to use a different implant
  • Referring patients to colleagues if outcomes are poor for a particular procedure
  • Additional training and support from peers

More widely, the NJR encourages surgeons to share their outcome data as part of regular department meetings and as part of their appraisal using the Consultant Level Report (CLR).

For advice and support, surgeon may wish to contact their NJR Regional Clinical Coordinator.

How can a surgeon update their data?

Following any audit of data, any errors found can be updated with the assistance of the NJR regional compliance officers. The NJR would only be able to review an adjusted standardised revision ratio (SRR) at the time of the next data cut, so changes need to be made by the end of February and by the end of August each year.

Since April 2019, surgeons have received email notifications if a new revision or mortality event is linked to one of their procedures. All revisions of primary joints will be included in calculation of the SRR (we do not count revisions of previous revisions). Surgeons are notified of any deaths recorded within the NJR, however deaths following joint replacement surgery performed for trauma or cancer cases, are not included in the calculation of the mortality statistics, neither for units nor for individuals.

Details of all revision and mortality cases can be reviewed within the NJR Connect portal. If there are any errors, regional NJR compliance officers can assist with updating the details that are recorded on the registry.

What happens if a surgeon does not respond to an NJR letter?

The NJR contacts the responsible officer of a surgeon’s designated body and the responsible officers of any other hospitals where a surgeon has practiced as part of the standard process. In cases where a surgeon does not respond to an NJR letter to acknowledge receipt, we would contact their responsible officer to inform them of this.

Why does a surgeon’s expected number of revisions stay the same when they have an increase in the number of primary procedures?

The expected number of revisions is calculated based on a variety of factors which include: the number of primary procedures performed, the time at risk of revision of those procedures and the characteristics of the patient and the type of operation. The principal reason that a change in the number of primary procedures may not result in a marked change in the expected number of revisions is that the risk of revision in the new cohort of patients is lower than the previous cohort, so any increase in the expected number of revisions created by an increase in the number of primaries is being balanced out by other factors such as the period of time those primaries have been followed up for, the operation type or the characteristics of the patients operated on.

The surgeon performance outcome monitoring process now looks at a rolling window of 5 and 10 years. Therefore, even if the number of procedures between monitoring points is the same, the patients and risk of revision may be different. In the case of a previous 10-year monitoring window, patients who received their primary procedure between 9 and 10 years ago are not included in the current 10-year monitoring window, as they received their procedure between 10 and 11 years ago and are therefore now outside the capture of the window of analysis.

If the number removed from the beginning of the rolling 10-year window is greater or less than the number added in the latest year, that modifies the ‘expected’ number of revisions accordingly.

Why is the overall expected number of revisions different from the expected number of revisions when it is broken down by type of joint replacement?

It is important to note that the “all” category for hips includes all types of hip replacement (cemented, uncemented, hybrid and resurfacing). The expected number of revisions is therefore a function of the observed overall revision rates for all the different types of procedure recorded. Similarly, the “all” category for knees includes TKR, unicondylar and patellofemoral knee replacements, so the expected revision rate is a function of the observed overall revision rates for all the different types of procedure recorded. 

Two different scenarios are presented below:

  • A hip surgeon, who exclusively uses cemented THR, looks at their data in the “Hip all” SRR Funnel Plots. They have an overall expected number of revisions 15, an observed number of revisions of 10, and the standardised revision ratio is 0.67, putting them below the green line. The same surgeon looks at their data in the “Cemented Hip procedures” category, which compares this surgeon’s performance against the rates for cemented THR only (excluding uncemented, hybrid and resurfacing). The expected number of revisions that the observed number is compared against is lower as the revision rate is lower in cemented THR than for all hip procedures. The expected number of revisions is now 10 but their observed number of revisions remains the same i.e. 10, and they have standardised revision ratio 1.0 which puts them on the green line, higher than they appeared in the “Hip all” figure. The reason for the increase in standardised revision ratio is because of the fall in expected number of revisions in the cemented THR only analysis. The use of other hip types in the “Hip all” analysis results in a higher expected number of revisions. This is because as a class, cemented THR are associated with some of the lowest observed revision rates.
  • A knee surgeon, who exclusively  performs cemented TKR, looks at their data in the “Knee all” SRR Funnel Plots. They have an overall expected number of revision of 20, an observed number of revisions of 20, and a standardised revision ratio of 1. This puts the plotted point on the green line which indicates a standardised revision ratio of 1. The same surgeon looks at their data in the “Total Knee replacement” analysis. Here, the expected number of revisions is calculated from TKR only and hence is lower as the higher revision rates of unicondylar knees and patellofemoral knees are not included. They hence now have an expected number of revision of 10, their observed number of revision remains the same i.e. 20, and they have standardised revision ratio 2.0. This results in their point now being plotted further up the y-axis of the figure and may lead to them being above the 95% or 99.8% upper confidence intervals indicating either alert or alarm status. The reason for the increase in standardised revision ratio is because of the fall in expected number of revisions in the TKR only analysis.  The use of unicompartmental (including unicondylar and patellofemeoral replacement) prosthesis in the “overall” analysis results in a higher expected number of revisions. This is because as a class, unicompartmental knee replacements are associated with higher revision rates.

  •  
  •  
  •  
  •  
  •  
  •  
Scroll to Top