AJR: Breast imaging report errors more likely with speech recognition software
September 26, 2011
by Brendon Nafziger, DOTmed News Associate Editor
Breast imaging reports generated with speech recognition software were six times more likely to contain a major error than ones created with a traditional transcription service, according to a study published in the October issue of the American Journal of Roentgenology.
Of the hundreds of reports analyzed, nearly one-quarter of reports generated with automatic speech recognition had a major error, compared with fewer than one in 20 for reports made by teams transcribing a radiologist's dictations, the study said.
Breast MRI reports had even higher error rates. According to the study, nearly one-third of ASR breast MRI reports had a major error, compared with only 7 percent of conventional reports.
The researchers said the higher error rate for breast MRI was likely caused by the modality's greater complexity.
The study included 308 ASR reports, and 307 conventional ones. All reports were created between Jan. 2009 and April 2010, and were for patients whose results were talked about during weekly multidisciplinary tumor board rounds at two of the five university-affiliated hospitals in the authors' network.
Even though the software is based on voice recognition, error rates did not differ substantially between reports created by native English speakers and non-native English speakers, likely because the software "learns" the way the speaker talks.
"We thought that there may be a higher error rate for non-native English speakers because the software works with voice recognition, but that didn't happen," Dr. Anabel Scaranelo, the corresponding author, who's with the University Health Network in Toronto, said in a statement.
Nor were errors higher among trainees than among staff radiologists, the researchers said.
"Even though most errors detected in our study did not alter patient care because all cases had been discussed in multidisciplinary team meetings, the presence of major errors tends to make reports confusing and difficult to read," the authors said.
They recommended doctors carefully edit ASR-generated reports to minimize errors in breast imaging reports.
Defining errors
Errors discovered in the reports were classified independently as "minor" or "major" by an undergraduate medical student and a radiologist with 17 years of experience in breast imaging.
"Major" errors were held to be those that affected the interpretation of the result. For instance, in the example given by the researchers, a major error would be reporting: "There is no definite evidence of suspicious enhancement in either breast after the duodenum administration," instead of "There is no definite evidence of suspicious enhancement in either breast after the gadolinium administration."
Of the hundreds of reports analyzed, nearly one-quarter of reports generated with automatic speech recognition had a major error, compared with fewer than one in 20 for reports made by teams transcribing a radiologist's dictations, the study said.
Breast MRI reports had even higher error rates. According to the study, nearly one-third of ASR breast MRI reports had a major error, compared with only 7 percent of conventional reports.
The researchers said the higher error rate for breast MRI was likely caused by the modality's greater complexity.
The study included 308 ASR reports, and 307 conventional ones. All reports were created between Jan. 2009 and April 2010, and were for patients whose results were talked about during weekly multidisciplinary tumor board rounds at two of the five university-affiliated hospitals in the authors' network.
Even though the software is based on voice recognition, error rates did not differ substantially between reports created by native English speakers and non-native English speakers, likely because the software "learns" the way the speaker talks.
"We thought that there may be a higher error rate for non-native English speakers because the software works with voice recognition, but that didn't happen," Dr. Anabel Scaranelo, the corresponding author, who's with the University Health Network in Toronto, said in a statement.
Nor were errors higher among trainees than among staff radiologists, the researchers said.
"Even though most errors detected in our study did not alter patient care because all cases had been discussed in multidisciplinary team meetings, the presence of major errors tends to make reports confusing and difficult to read," the authors said.
They recommended doctors carefully edit ASR-generated reports to minimize errors in breast imaging reports.
Defining errors
Errors discovered in the reports were classified independently as "minor" or "major" by an undergraduate medical student and a radiologist with 17 years of experience in breast imaging.
"Major" errors were held to be those that affected the interpretation of the result. For instance, in the example given by the researchers, a major error would be reporting: "There is no definite evidence of suspicious enhancement in either breast after the duodenum administration," instead of "There is no definite evidence of suspicious enhancement in either breast after the gadolinium administration."