Evaluating LLMs in non-metastatic melanoma care: a comparative analysis
Leading LLMs demonstrate potential for high‑quality melanoma management but exhibit inconsistent performance influenced by architecture and case complexity, with reduced reliability in advanced stages.
Background: Malignant melanoma is an aggressive skin cancer with rising incidence. Accurate early staging and standardized treatment are crucial for prognosis. This study evaluates seven large language models (LLMs) – including GPT‑5.2, Gemini‑3.1, and medically enhanced models – in assisting non‑metastatic melanoma management amid clinical complexities. Methods: Employing a prospective, simulated expert‑blinded design, 59 virtual cases across TNM stages, ages, and comorbidities were assessed. Multiple senior oncologists independently evaluated model outputs using a 6‑point Likert scale for staging accuracy, treatment rationality, and protocol standardization. Results: GPT‑5.2 (5.56 ± 1.12) and Gemini‑3.1 (5.25 ± 1.3) achieved the highest staging accuracy. Performance declined significantly in complex Stage III cases. GPT‑5.2 and Gemini‑3.1 also led in treatment rationality, showing stability, whereas model performances converged in early stages but diverged in advanced ones. Gemini‑3.1 excelled in protocol standardization (5.17 ± 0.57), though some models posed risks like insufficient surgical margin recommendations. Conclusion: Leading LLMs demonstrate potential for high‑quality melanoma management but exhibit inconsistent performance influenced by architecture and case complexity, with reduced reliability in advanced stages. Future tools require risk‑stratified guidelines and real‑world validation to improve patient outcomes.
The study highlights the variability of advanced AI models in making diagnostic and therapeutic decisions for non‑metastatic malignant melanoma, underscoring the need for risk‑stratified guidelines and real‑world validation before clinical adoption.
Evidence level: Számítógépes vagy elméleti. Modellből vagy adatbányászatból származó jel.
Related signals
- Beyond the Five-Year Milestone: Long‑term Survivorship of Melanoma Patients Treated Off‑Trial with anti‑PD‑1
- Uveal Melanoma – Trials in Disease – Overview of Information and Clinical Research
- A phase 1/2 trial of an immune-modulatory vaccine against IDO/PD-L1 in combination with nivolumab in metastatic melanoma
- Melanoma Research
- The Clinical Trial Landscape for Melanoma Therapies