CAD provides mixed benefits for DR lung exams

By Brian Casey, AuntMinnieEurope.com staff writer

March 8, 2008 -- VIENNA - A pair of presentations on Saturday at the 2008 European Congress of Radiology (ECR) on computer-aided detection (CAD) for digital radiography reported mixed benefits for the software in detecting lung nodules in thoracic exams.

Detecting lung nodules on radiography exams has historically posed a problem for radiologists, and as a result CT has become the preferred modality for lung pathology. But radiography is still an attractive option, in large measure due to its lower cost compared to CT.

The recent arrival of digital radiography (DR) gives new options to radiologists who still see a role for radiography in lung imaging. In addition, several vendors have developed CAD software that offer the hope that radiography can achieve a diagnostic performance closer to that of CT.

In Saturday's ECR sessions on reporting and CAD, several presenters discussed their experiences with a commercially available CAD product and its impact on radiologist performance in analyzing DR lung studies.

In the first presentation, Dr. Elmar Kotter of the University of Freiburg in Germany discussed his group's experiences with the software (xLNA Enterprise, Philips Healthcare, Andover, MA). Kotter's group in particular reviewed the software's impact on the diagnostic performance of three radiologists in analyzing nodules in the CAD software's sweet spot in analyzing lesions 5 mm to 15 mm in diameter.

The group started with a patient population of 4,512 chest x-ray studies, which were compared to a reference standard of 1,114 chest CT exams. Of this group, 117 patients fulfilled the requirements of the study by having both exams within four weeks of each other, from January to April 2007.

Another 75 patients in this group were excluded because they had nodules that were either too large or too small for the CAD algorithm to operate effectively, leaving the group with a total of 42 patients. This group had 66 nodules, with a mean diameter of 7.5 mm.

The group found the sensitivity of the CAD system to be 39%, compared to a range of 18% to 30% for the three radiologists. The system generated a total of 288 false positives, for an average of 2.6 false positives per image, Kotter said.

The group found good agreement among the radiologists for detecting nodules later proved to be malignant on CT, while agreement was not so strong between radiologists and the performance of the CAD software. Kotter said this lack of agreement actually could be seen as a benefit, as it could be implied that the CAD software was highlighting nodules that might have been overlooked by radiologists.

One potential problem generated by the software occurred when more than one nodule was found in the region of interest. "It can be confusing for the radiologist, as he doesn't know what to look for," Kotter said.

Another difficulty occurred when nodules larger than 15 mm were found in images. The CAD software is not designed to highlight lesions this large, and this prompted some of the readers in the study to question the accuracy of the system when they were not highlighted. This effect can be counteracted with education and training of radiologists when using the software, according to Kotter.

"We think, but we have yet to prove this at this time, that the CAD system detects more of the smaller nodules than larger nodules," Kotter said. "Radiologists are better at seeing larger nodules and miss more of the very small nodules."

Kotter concluded by stating that the incremental value of CAD as a second reader for radiologists in DR chest studies -- its intended purpose -- remains to be defined.

The Vienna study

In the second paper presented, Dr. Martin Uffmann of the Medical University of Vienna specifically focused on the question of using CAD as a second reader. Like Kotter's group, they also studied the xLNA Enterprise software.

The group's patient population consisted of 87 patients, 37 of whom had lesions while 50 did not. All patients also received CT lung studies, and images were read by four readers, two of whom were expert radiologists and the other two were first-year residents.

The radiologists read images using three different methods: first normal images, then normal images with a grayscale reversal postprocessing technique applied, and then with CAD markings as a second reader.

The researchers found that the sensitivity for the inexperienced readers using CAD was 56%, compared to 52% when reading grayscale reversal and normal images. The two experienced readers benefited most from the grayscale reversal technique, which brought their sensitivity to 68%, compared to 63% sensitivity for both CAD and normal reading (63%).

In specificity ratings, CAD actually decreased the specificity of the novice readers, which was 22% with CAD, compared to 29% with normal reading and 17% with grayscale reversal. On the other hand, CAD improved the specificity of the expert readers, which was 65%, compared with 59% for grayscale reversal and 58% for normal reading.

As with Kotter's group, Uffmann's team found wide variations in how CAD was used by its readers. For example, the group found major differences in how radiologists took advantage of CAD markings, with some accepting them for positive nodules while others rejected them. In another similarity with Kotter's study, Uffmann's group had to train the readers regarding the CAD software's difficulty in recognizing obvious lesions, such as those larger than 15 mm.

Uffmann said that the group in general felt that CAD turned in a low performance in the study, which he acknowledged could have been due to a difficult patient population of smokers with "dirty lungs" that might make it more difficult to differentiate nodules. Still, he noted the technology was useful for DR lung exams.

"Computer-aided detection of lung lesions is a feasible tool in projection radiography, and it should be used as an assistive tool," Uffmann said.

By Brian Casey
AuntMinnie.com staff writer
March 8, 2008