Could computers help GPs diagnose cancer early?
Picture the scene. A patient visits their GP after losing weight and having chest pain that wouldn't go away.
The GP has just 10 minutes to consider many different factors, from symptoms to medical and family history, to work out which of thousands of different conditions and diseases this particular patient may have.
And now consider that the average GP see hundreds if not thousands of patients a year with potential cancer symptoms – fewer than 8 of whom will actually have cancer. It's a difficult job.
Help may be at hand. Over the last few years, researchers have developed a series of mathematical models that use the latest evidence on symptoms and lifestyle to work out the likelihood that a patient may have a specific disease.
And to help GPs use them, Macmillan Cancer Support, partly funded by the Department of Health, has developed two of them into software – known as an electronic Cancer Decision Support Tool (CDS) – that are compatible with GPs' computer systems.
After a trial run last year with volunteer GPs around England, the software is now being improved and made available to GPs around the country.
But does it actually help? To find out, Cancer Research UK independently helped to evaluate how GPs use it during the trial run.
We've now published a summary of our evaluation, and we've found some encouraging signs that they did influence decisions about whether to refer patients.
But there isn't yet definitive evidence to show that it helps GPs diagnose patients more accurately or at an earlier stage.
How do these programmes work?
There's a growing pool of evidence about how a range of different symptoms and lifestyle factors affect the likelihood that a patient will develop or have a disease.
Using that evidence, researchers have developed a number of different mathematical models (often called algorithms) that aim to estimate the overall chances that a given patient, with particular symptoms and other risk factors, will have a particular disease. The aim is that by turning these abstract algorithms into materials or software that GPs can use, it's easier for them to make use of the evidence available.
The algorithms themselves tend to be (but aren't always) linked to specific diseases – and two focusing on cancer are the Risk Assessment Tool (RAT) and QCancer, which were developed by Professor Willie Hamilton and Professor Julia Hippisley-Cox respectively.
RAT focuses on the symptoms a patient has, while QCancer looks at symptoms alongside things like a patient's medical history and risk factors (e.g. whether they smoke, their weight and age).
These algorithms cannot diagnose patients; they only predict the chance that a patient may have cancer based on the information they share with their GP.
GPs can then use this information to help discuss things with their patients, and to help make decisions about their care, such as whether to refer them for further tests.
This can be particularly helpful when GPs are working out whether to refer a patient whose symptoms don't fit into the guidelines on urgent referrals for suspected cancers (often called 'Two Week Waits' in England).
GPs have previously been able to use these algorithms in both printed and online formats. Here's an example of a printed chart for GPs, based on the Risk Assessment Tool – it allows GPs to look up the likelihood that different combinations of a patient's symptoms could be worth investigating (in technical terms their 'positive predictive value').
What's new about these programmes?
Macmillan worked with GPs and GP software providers to develop RAT and QCancer into two new electronic software programmes. Making them computer-based in this way means they can potentially be used more widely, and allows certain features to be automated.
The CDS software consists of 3 main features to help GPs spot patients at higher risk of having (or developing) 5 types of cancer: lung, bowel, oesophago-gastric (i.e stomach and oesophagus cancers combined), pancreatic, and ovarian cancers:
- An automated prompt that uses the information added to a person's medical records to alert the GP when the person they're seeing has a cancer risk above a certain threshold.
- A symptom checker that provides a cancer risk score based on symptoms the GP enters into their medical records or into the programme directly.
- A 'risk stratification list' generated by the software for the GP, based on the relevant parts of the medical records of people registered with them. This helps the GP identify and liaise with anyone at increased risk (e.g. to offer further investigations or a referral).
Now, it seems logical that the software should help GPs work out who to refer or investigate for suspected cancer.
But anything that influences how doctors care for their patients needs to be proven as practical and beneficial in the real world, to make sure there are no negative unintended consequences. That's why an independent evaluation was needed and Cancer Research UK was asked to coordinate it.
Putting the CDS software through its paces
During Macmillan's test run, GPs from 439 practices across England had access to the software between March and November 2013.
These GPs were divided into two groups, with one using a version that showed results from the Risk Assessment Tool and the other using a version that showed results from QCancer.
The software recorded the scores produced for patients by the two algorithms to see if there were differences between them.
Teams here at Cancer Research UK then analysed the data collected. One of the Department of Health's Policy Research Units also gathered feedback from interviews with patients and GPs, to get more detail on how they felt about the CDS software.
Our evaluation comes with some caveats – for example, participation and providing feedback was voluntary. That means that the data on how the programmes were used isn't comprehensive and may not be representative, so the results can't necessarily be generalised to all GPs.
And it wasn't possible to get data on key things like the number of cancers diagnosed, or the stage at which they were diagnosed, during the trial run. So we don't know if using the software meant that patients were diagnosed at an earlier stage and did better in the long run.
Despite this, the evaluation has provided vital insights into GPs and patients' views and experiences of using the programmes, including how they could be improved.
What's the verdict?
Overall, our evaluation suggests the CDS systems can:
- raise GPs' awareness of cancer symptoms;
- both alert and remind GPs when patients are potentially at risk;
- influence the decisions GPs make about how to care for patients.
In fact, in nearly a fifth (19%) of cases GPs said that, had they not used the symptom checker, they would not have referred/investigated the patient.
Looking by cancer type, GPs said that using the lung cancer symptom checker was most likely to influence them; in a third (33%) of cases it lead them to refer/investigate patients that they wouldn't otherwise have done. The oesophageal and pancreatic cancer symptom checker was least likely to influence them (9% of cases).
However, there was no strong evidence that access to the software increased urgent referrals for suspected cancer (the two week waits) for the cancers the CDS programmes covered.
Interestingly the risk calculated by the symptom checker was different from the GP's perceived level of risk nearly half (46%) of the time. This highlights the potential of the software to help GPs identify patients at risk. But it could also reflect the fact that the GPs sometimes felt they were in a better position to assess their patient's risk.
GPs also expressed several concerns, including that:
- they didn't always agree with or understand how to interpret the risk scores produced for their patients;
- the risk scores produced by the algorithms used in the software rely on data recorded in a patient's medical records – which may not necessarily be 100% accurate. This could flag patients who aren't actually at risk or miss others who are;
- they could end up with 'prompt fatigue' both because of how regularly the programmes prompt them, and because other systems on their computers also flag various things;
- in a 10-minute consultation it was sometimes difficult to use the symptom checker function properly;
- looking at their computers to use the CDS software meant GPs felt they weren't focusing on interacting and talking to their patients.
What did patients think?
Patients echoed some of these concerns that GP's would be over-reliant on the computer, and they would tend to look at the screen rather than at them, leaving them feeling excluded.
Despite those concerns, overall the patients did generally think that it was a good idea to find ways for GPs to make use of the RAT and QCancer algorithms in some form.
The key will therefore be to ensure that GPs are trained in how to use the CDS programmes software in collaboration with patients, and in a way that doesn't reduce the quality of the interaction.
What's next?
Training will be vital – During the trial run, Macmillan worked with local cancer networks to try to offer all participating GPs comprehensive training on how to use the software and interpret its results.
Despite this, 17% of GPs who fed back after using the symptom checker said they had not had any training on how to use the CDS programmes.
CDS software isn't the only show in town – One of their key benefits appears to be that they can make GPs 'think cancer' when they might not otherwise, and increase their awareness of cancer-related symptoms, symptom combinations and cancer risk factors. But this can be achieved through other means too, and work to enable GPs to examine and improve all aspects of both their own work and that of their practice must continue (such as audits and sharing GP practice cancer profiles). It is a combined approach alongside the GP's clinical judgement and expertise that's likely to produce most benefit for patients and doctors alike.
Further evaluation is a must – For example, the data we collected showed that the two different algorithms – RAT and QCancer – gave quite different risk scores for around a third (34%) of patients. GP practices will still be receiving one of at least two versions of the software meaning there's potential to cause variations between practices.
Can CDS software help the GP and patients?
We know that being diagnosed at an early stage gives most cancer patients a much better chance of surviving the disease. Sadly only half (54%) of patients in England are diagnosed at an early stage – and for some cancers, such as lung, that number is much lower.
There's no single issue that leads to people being diagnosed with cancer late – we need people to know what's normal for them and be confident about getting unusual or persistent changes checked out by their GP; GPs need to be supported to spot symptoms that could be a sign of cancer; and the NHS needs to have the people, systems, and tools to be able to diagnose and treat patients as fast as possible.
Our evaluation shows that using the CDS systems would help some GPs to suspect cancer and refer or investigate their patients – but it's not all positive and this is just one piece of the jigsaw.
So, with another tool added to the GP's toolkit, the work must continue to ensure more patients are diagnosed at an early stage and survive cancer long term.
