With AI bringing yet another significant advancement for medical diagnostics, PathChat 2, a new pathology-specific large language model (LLM), is transforming how pathologists identify and diagnose tumours.
As reported by Taryn Plumb in a recent VentureBeat article, PathChat 2 offers a new level of precision and interaction in the field of computational pathology.
One outperforming existing models in both accuracy and versatility.
Unmatched Diagnostic Precision
PathChat 2 was tested alongside other state-of-the-art LLMs like GPT-4V (vision), LLaVA, and LLaVA-Med.
Each were presented with an image of a potentially serious eye tumour. These were the results:
While GPT-4V provided a vague and inaccurate response, and LLaVA and LLaVA-Med failed to correctly identify the tumour’s location, PathChat 2 accurately pinpointed the tumour in the eye and highlighted its potential to cause vision loss.
This superior performance showcases PathChat 2’s capability to serve as a reliable consultative tool for human pathologists.
Breakthrough in Computational Pathology
PathChat 2, described by Richard Chen, founding CTO of Modella AI, as “a multimodal large language model that understands pathology images and clinically relevant text,” significantly surpasses its predecessors.
By adapting a vision encoder specifically for pathology, combining it with a pre-trained LLM, and fine-tuning with visual language instructions and question-answer turns, this innovative approach allowed PathChat 2 to achieve 78% accuracy on image-only prompts and 89.5% accuracy when additional clinical context was provided .
Comparative Performance
In rigorous testing, PathChat 2 outperformed models like GPT-4V, LLaVA, and LLaVA-Med.
According to Taryn Plumb, PathChat 2 “performed with 78% accuracy (on the image alone) and 89.5% accuracy (on the image with context).”
When compared to its competitors, PathChat 2 scored over 52% better than LLaVA and 63% better than LLaVA-Med in image-only evaluations. In clinical context scenarios, it performed 39% better than LLaVA and nearly 61% better than LLaVA-Med .
Faisal Mahmood, associate professor of pathology at Harvard Medical School, however, emphasises the broader implications of this development.
“PathChat moves us one step forward towards general pathology intelligence, an AI copilot that can interactively and broadly assist both researchers and pathologists across many different areas of pathology, tasks, and scenarios,” Mahmood told VentureBeat .
Practical Applications and Future Directions
PathChat 2’s capabilities extend beyond simple image analysis.
In one scenario, the model accurately diagnosed lung adenocarcinoma in a 63-year-old male patient based solely on a chest X-ray and clinical symptoms. In another case, it identified a liver tumour as metastatic melanoma, demonstrating its ability to handle complex differential diagnoses and tumour grading tasks .
Mahmood highlights a critical shift in AI pathology research, noting that PathChat 2’s training on comprehensive pathology knowledge allows it to adapt to various downstream tasks without requiring extensive labelled training data.
This adaptability marks a departure from traditional models that are typically limited to specific diseases or tasks .
Future Enhancements and Broader Implications
While PathChat 2 presents a breakthrough, challenges such as hallucinations remain.
Researchers suggest that reinforcement learning from human feedback (RLHF) could improve model accuracy. Mahmood envisions expanding PathChat 2’s capabilities to other medical imaging specialties and integrating it with digital slide viewers and electronic health records to enhance its utility further.
Looking ahead, Mahmood’s team plans to collect extensive human feedback to refine the model and align it more closely with human intent. They also aim to integrate PathChat 2 with existing clinical databases to retrieve relevant patient information, enhancing its diagnostic accuracy.
“We plan to work with expert pathologists across many different specialties to curate evaluation benchmarks and more comprehensively evaluate the capabilities and utility of PathChat across diverse disease models and workflows,” Mahmood stated .
Get More AI Content!
PathChat 2 represents a significant leap forward in computational pathology.
It offers pathologists a powerful new tool for diagnosing and understanding tumours and other serious conditions and its development marks a pivotal moment in the integration of AI and medical diagnostics, promising a future where AI-assisted pathology becomes the norm rather than the exception.
