A new study published in Communications Medicine looked at predicting which patients with cancer will see a psychiatrist or counsellor from their initial oncology consultation document using natural language processing.
“Our study is about using artificial intelligence to predict which patients with cancer will see a counsellor or psychiatrist,” study author Dr. John-Jose Nunez told us. “We do this by using AI models to read the document an oncologist writes after the first appointment with a patient. The AI is then able to predict which patients will go on to see a psychiatrist or counsellor with accuracy above 70%.”
The research team hypothesized that their models would be able to achieve around 65% accuracy, based on similar studies in mental health.
“As a cancer psychiatrist and researcher, I know that the mental health needs of patients with cancer can sometimes go unaddressed, or can be addressed later than would be optimal,” Nunez told us. “Developing a tool like this may help patients with cancer receive mental health supports earlier before illnesses worsen, and may help ensure more patients receive the care they need.”
The research team trained and evaluated their AI models on prior data to ensure they were working well. They used a ‘holdout set’ to test the AI on data it had never seen before. The accuracy was above 70%, with their models achieving performance metrics that indicated quite good performance, with their best models achieving a balanced accuracy around 75%, and area-under-curve around 0.80.
The researchers were surprised with the results.
“The results were better than we expected based on prior studies in mental health research,” Nunez told us. “Our study suggests that non-mental health medical documents can be used to predict mental health needs. This is potentially exciting, as mental health can commonly affect patients with physical illness, but it is usually not possible to have mental health clinician see all patients with physical illness. Our tools may be able to detect which patients with physical illness may benefit from mental health supports in a variety of medical settings.”