On the Mark: Computer-Aided Detection

14 March 2010 (Last Updated March 14th, 2010 18:30)

Computer-aided detection (CAD) is increasing the chances of finding more cancers. While it is generally accepted that CAD improves detection rates, there is an inevitable trade-off between sensitivity and specificity that keeps sceptics in doubt. Breast imaging expert Dr Robyn Birdwell and CT colonography specialist Dr Stuart Taylor offer Elly Earls an insight into the future of these systems.

On the Mark: Computer-Aided Detection

CAD can signify two different procedures, computer-aided detection (CADe) and computer-aided diagnosis (CADx). While computer-aided diagnosis has not yet been applied in non-experimental settings, its forerunner computer-aided detection has been clinically proven to help radiologists find cancer and its precursors.

The technology has been used for the early discovery of breast cancer in mammograms since 1998 when the R2 ImageChecker became the first FDA-approved device and, according to prospective studies, the use of CADe has improved rates of breast cancer detection by between 7%-19%.

Before the advent of CADe, research showed that if two radiologists read a mammogram, the second would help the first find 3%-15% more cancers. With the new technology, CADe acts as a computerised second reader, giving similar benefits. The radiologist looks at their mammogram and comprehensively analyses it before engaging the system. Its software algorithms then generate identifiable marks, which are indicative of either a suspected mass (which some systems mark with an asterisk) or bright spots that may be calcifications (which some systems mark with a triangle).

Dr Robyn Birdwell, an associate professor of radiology at Harvard Medical School, who has been using CADe for ten years, emphasises that the radiologist controls the decision-making process. "CADe simply focuses their attention on a given spot, but the decision completely rests with the radiologist as to what does or does not need to be done," she explains.

"CAD can signify two different procedures, computer-aided detection (CADe) and computer-aided diagnosis (CADx)."

A similar technology has more recently been developed to assist in the early detection of colon cancer, which identifies polyps, the precursor to the disease. Patients undergo a CT colonography during which the radiologist will read the CT scan without CADe, write down their observations, turn CADe on and make a note of their additional thoughts. Colon CADe algorithms are particularly effective at detecting the protuberant "bowler hat" shaped polyps, which are the most frequent.

Although this system is yet to be approved by the FDA in the US, it is becoming increasingly popular in Europe where companies such as iCAD and MedicSight have created CADe solutions that use advanced algorithms to highlight suspicious polyps warranting closer review by a radiologist.

The sensitivity is over 90% for clinically significant polyps and, according to Dr Stuart Taylor, a reader at University College London, this is the target of the technology. "Most radiologists say they want to see polyps of more than 6mm in size and most CADe systems have 85%-90% sensitivities for this."

Sensitivity and specificity

CADe techniques differ significantly between colonography and mammography but there is one key point that Birdwell and Taylor categorically agree on: there is a trade off between the advantages of sensitivity and the decrease in radiologists' specificity.

The sensitivity of CADe technology is variable depending on the version of the system but breast imaging experts hope for a background of no more than an average of two marks per four view mammogram. "If there is a mark every 2mm, the system is completely unhelpful. You probably will find cancers but the reader will not benefit because they will see nothing but marks," Birdwell claims. "Vice versa, if not enough things are marked, some cancers will be missed by the system."

The expectation is to find between four and ten cancers in 1,000 mammograms but at least 2,000 marks will be found across such a sample. "This means that there are a tremendous number of marks that need to be dismissed," Birdwell explains. "If you called every mark back, this would be a completely useless tool.

"The mark isn't necessarily telling you there's cancer. In fact, nowhere near the majority of marks are going to be indicative of that. CADe should really be thought of as a spell check. Neither what it is looking for is perfect nor is what it finds necessarily accurate. Radiologists need to figure that out."

Often the colon CADe will mark the folds of the bowel or retained bowel content, while the mammography system frequently misinterprets tissue overlaps as calcifications, but patients are generally willing to tolerate a large number of false positives if it leads to finding a real cancer. "They are inclined to say that they will stand for ten, 15 or 20 false positive cases if it means finding another important polyp," Taylor confirms. "Patients are very keen to get the benefits of the sensitivity and put up with the drop in specificity."

Safety first

CADe is essentially a safety net and one that is particularly valuable for newly qualified radiologists. Indeed, many papers have found an improvement in detection rates of up to 10% with less experienced readers. But, according to Taylor, no matter how knowledgeable the radiologist is, a safety net is always a useful tool. "It takes away from the fatigue of reading the CT colonography," he explains. "It's a difficult exam to read with 1,000 images per data set and if you're doing five or six of those in an hour, you're going to get pretty tired. Even if CADe doesn't help radiologists every single time, it's invaluable for the one case where they had overlooked something."

"CADe simply focuses attention on a given spot, but the decision on what to do completely rests with the radiologist."

As CADe technology gains prevalence, the training of radiologists is becoming an increasingly important issue. "They have to have training to understand what a true polyp looks like," Taylor begins. "CADe points out where it thinks a polyp may be – seven times out of ten that will be a real lesion but three times out of ten it will be a false positive. In terms of how the system will be implemented, it will have a beneficial effect overall but it will not replace training."

Birdwell, who is the section head of Brigham & Women's Hospital's Division of Breast Imaging, agrees, describing CADe as an adjunct, merely another attempt to help us focus our eye. Yet there is such variability in the individual radiologist's acceptance of the technology that teaching is a real challenge faced in the industry; some radiologists are convinced that it is of no help while others look too quickly at their exam, hoping that CADe will do the work for them. "They push the button, see no marks and feel that's enough to leave the case as a negative, but that is not how it is supposed to be used," Birdwell declares. "We need to make trainees aware of what pushing that button is really about."

Many sceptics remain unconvinced by the merit of the fast-developing system and although studies have shown that CADe does improve the detection rates of breast cancer, the advances are undeniably small. "Only two more cancers were found in our study at Stanford," Birdwell acknowledges. "The advances are important but they are small so it is hard to show statistical differences, which means that sceptics continue to be sceptical."

Limited appeal

Both Taylor, who is a reader in clinical radiology, and Birdwell recognise that the software does have inherent limitations. There are three potential ways of using CADe: as a standalone technology, concurrently, or as a second reader. The first method would not involve a radiologist and is currently purely theoretical, the second means engaging CADe from the beginning of the reading, and the final technique, which involves turning CADe on only at the end of the process, is the generally recommended practice. While this should ensure that the radiologist gives his or her full attention to their initial reading, it also adds another minute or so to the CT read process. "If you're reading 200 exams, that little bit of time adds up," Birdwell notes.

Readers also need to be comfortable with dismissing the majority of marks. "You should never allow yourself to call something back just because CADe marked it. Nor should you ignore something that you're worried about because CADe doesn't mark it," Birdwell emphasises, adding that it takes more time to adjust to the system than many early users expected. "If your call back rate is, say, 7% and then you add CADe to your system, you should be very aware that your rate will increase. It takes as long as 18 months to come back to just about the level of your initial call back rate – maybe a little higher because CADe has found a few things that warrant a call back."

CADe is not a fail-safe programme by any stretch of the imagination. In mammography, a spiculated mass is much more likely to be marked than a smooth margined mass, while amorphous calcifications are not identified as often as linear branching calcifications. Moreover, colon CADe algorithms find it much easier to detect rounded polyp shapes than larger, more angular cancer masses.

"iCAD and MedicSight have created CADe solutions that use advanced algorithms to highlight suspicious polyps."

But moving forward, manufacturers are endeavouring to further develop the software's algorithms with the hope of catching a larger range of cancers in CADe's net. "People want a product that detects polyps but doesn't miss the cancers," Taylor confirms. "There is now some research on training CADe to pick up the masses and cancers as well as the smaller polyps. A mass is like a wall-thickening and is much harder for CADe to detect than a polyp, which sticks into the lumen. The final hurdle is detecting the flat polyps, which are only shown as a slight raise above the wall of the bowel."

Similar advancements are hoped for in the arena of breast imaging. While the versions of CADe have steadily improved over the past 12 years with manufacturers attempting to refine the algorithms and balance sensitivity against specificity, there is certainly more to come. "The system does not compare last year's and this year's exams," Birdwell notes, "and that would be a nice additional function."

The only way to calm the sceptics, however, will be to reduce the number of false positives identified by CADe, and, in the field of colonography at least, research in recent months has focused on this. "In one area of study you literally do the CT read, have six or seven prompts to scroll through and are given a little check at the end of the process to make sure you haven't missed anything," Taylor clarifies.

Given the relatively minor benefits of the system and its reliance on the radiologist's final verdict, executing statistical studies on CADe's efficiency is a challenging task and direct comparisons between different versions of the software are equally complex. Yet, as manufacturers endeavour to hone the delicate balance between sensitivity and specificity and the technology expands into new areas, future research can only further enlighten the industry on the capabilities of computer-aided detection.