to test chatgpt’s knowledge of cirrhosis and liver cancer, the team presented the chatbot with 164 frequently asked questions that were divided into five categories: basic knowledge, diagnosis, treatment, lifestyle or preventive medicine. two liver transplant specialists graded the results.
according to investigators, chatgpt answered about 77 per cent of questions correctly, with high levels of accuracy in over 90 per cent of questions from different categories. however, the liver specialists who graded the answers found that 75 per cent of responses relating to basic knowledge, treatment and lifestyle categories were correct but inadequate. the proportion of answers that were “mixed with correct and incorrect data” was as follows: basic knowledge (22 per cent), diagnosis (33 per cent), treatment (25 per cent), lifestyle (18 per cent) and preventive medicine (50 per cent).
chatgpt also a tool for doctors
although the software was able to offer useful advice to patients and caregivers dealing with cirrhosis and liver cancer, the study made it clear that superior information comes from a doctor.
“more research is still needed to better examine the tool in patient education, but we believe chatgpt to be a very useful adjunctive tool for physicians — not a replacement — but adjunctive tool that provides access to reliable and accurate health information that is easy for many to understand,” spiegel said. “we hope that this can help physicians to empower patients and improve health literacy for patients facing challenging conditions such as cirrhosis and liver cancer.”
in an essay published in the new england journal of medicine late last week, peter lee, microsoft’s corporate vice president, research and incubations, said society has a choice to make when it comes to using chatgpt in the health care industry.
“we can try to slow down or shut down this technology, because we don’t understand the risks completely,” lee said. “a second argument is more fatalist, saying: “it’s unstoppable. we might as well give it a try.
4 minute read
