Artificial Intelligence and the Accuracy of Disease Diagnosis

John Leddo

doi:10.46609/IJSSER.2026.v11i09.027

Title: Artificial Intelligence and the Accuracy of Disease Diagnosis
Authors: John Leddo
Journal: International Journal of Social Science and Economic Research (IJSSER) ISSN: 2455-8834 Volume: 11 Issue: 9 Publication Date: September 2026 DOI: 10.46609/IJSSER.2026.v11i09.027
\|\| \|\|
Taruna Agarwal, John Leddo, Dev Agarwal, Ashna Melanathuru, Rishaan Prabakaran, Mihika Mamidi, Srishti Kallu, Shree Garg, Dhairya Garg, Heidi Vineet, Yousef Malik, Tatva Duggineni, Anika Dubey, Maahika Challagulla, Vihaan Penumatsa and Mahathi Girla MyEdMaster, LLC Virginia, US
MLA 8 Leddo, John. "Artificial Intelligence and the Accuracy of Disease Diagnosis." Int. j. of Social Science and Economic Research, vol. 11, no. 9, Sept. 2026, pp. 4887-4903, doi.org/10.46609/IJSSER.2026.v11i09.027. Accessed Sept. 2026. APA 6 Leddo, J. (2026, September). Artificial Intelligence and the Accuracy of Disease Diagnosis. Int. j. of Social Science and Economic Research, 11(9), 4887-4903. Retrieved from https://doi.org/10.46609/IJSSER.2026.v11i09.027 Chicago Leddo, John. "Artificial Intelligence and the Accuracy of Disease Diagnosis." Int. j. of Social Science and Economic Research 11, no. 9 (September 2026), 4887-4903. Accessed September, 2026. https://doi.org/10.46609/IJSSER.2026.v11i09.027.
References [1] . Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Rodman, A., & Chen, J. H. (2024). Large language model influence on diagnostic reasoning: A randomized clinical trial. JAMA Network Open, 7(10), e2440969. https://doi.org/10.1001/jamanetworkopen.2024.40969 [2] . Hirosawa, T., Kawamura, R., Harada, Y., Mizuta, K., Tokumasu, K., Kaji, Y., Suzuki, T., & Shimizu, T. (2023). ChatGPT-generated differential diagnosis lists for complex case-derived clinical vignettes: Diagnostic accuracy evaluation. JMIR Medical Informatics, 11, e48808. https://doi.org/10.2196/48808 [3] . Ito, N., Kadomatsu, S., Fujisawa, M., Fukaguchi, K., Ishizawa, R., Kanda, N., Kasugai, D., Nakajima, M., Goto, T., & Tsugawa, Y. (2023). The accuracy and potential racial and ethnic biases of GPT-4 in the diagnosis and triage of health conditions: Evaluation study. JMIR Medical Education, 9, e47532. https://doi.org/10.2196/47532 [4] . Johnson, A., Bulgarelli, L., Pollard, T., Celi, L. A., Mark, R., & Horng, S. (2023a). MIMIC-IV-ED (version 2.2). PhysioNet. RRID:SCR_007345. https://doi.org/10.13026/5ntk-km72 [5] . Johnson, A., Pollard, T., Horng, S., Celi, L. A., & Mark, R. (2023b). MIMIC-IV-Note: Deidentified free-text clinical notes (version 2.2). PhysioNet. RRID:SCR_007345. https://doi.org/10.13026/1n74-ne17 [6] . Mitchell, A. J., Vaze, A., & Rao, S. (2009). Clinical diagnosis of depression in primary care: A meta-analysis. The Lancet, 374(9690), 609–619. https://doi.org/10.1016/S0140-6736(09)60879-5 [7] . Newman-Toker, D. E., Peterson, S. M., Badihian, S., Hassoon, A., Nassery, N., Parizadeh, D., Wilson, L. M., Jia, Y., Omron, R., Tharmarajah, S., Guerin, L., Bastani, P. B., Fracica, E. A., Kotwal, S., & Robinson, K. A. (2022). Diagnostic errors in the emergency department: A systematic review (Comparative Effectiveness Review No. 258; AHRQ Publication No. 22(23)-EHC043). Agency for Healthcare Research and Quality. https://doi.org/10.23970/AHRQEPCCER258 [8] . Rutledge, G. W. (2024). Diagnostic accuracy of GPT-4 on common clinical scenarios and challenging cases. Learning Health Systems, 8(3), e10438. https://doi.org/10.1002/lrh2.10438 [9] . Salinas, M. P., Sepúlveda, J., Hidalgo, L., Peirano, D., Morel, M., Uribe, P., Rotemberg, V., Briones, J., Mery, D., & Navarrete-Dechent, C. (2024). A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis. npj Digital Medicine, 7, Article 125. https://doi.org/10.1038/s41746-024-01103-x [10] . Shallcross, L. J., Rockenschaub, P., McNulty, D., Freemantle, N., Hayward, A., & Gill, M. J. (2020). Diagnostic uncertainty and urinary tract infection in the emergency department: A cohort study from a UK hospital. BMC Emergency Medicine, 20, Article 40. https://doi.org/10.1186/s12873-020-00333-y [11] . Singh, H., Daci, K., Petersen, L. A., Collins, C., Petersen, N. J., Shethia, A., & El-Serag, H. B. (2009). Missed opportunities to initiate endoscopic evaluation for colorectal cancer diagnosis. The American Journal of Gastroenterology, 104(10), 2543–2554. https://doi.org/10.1038/ajg.2009.324 [12] . Singh, H., Hirani, K., Kadiyala, H., Rudomiotov, O., Davis, T., Khan, M. M., & Wahls, T. L. (2010). Characteristics and predictors of missed opportunities in lung cancer diagnosis: An electronic health record–based study. Journal of Clinical Oncology, 28(20), 3307–3315. https://doi.org/10.1200/JCO.2009.25.6636 [13] . Wang, T.-W., Hong, J.-S., Chiu, H.-Y., Chao, H.-S., Chen, Y.-M., & Wu, Y.-T. (2024). Standalone deep learning versus experts for diagnosis lung cancer on chest computed tomography: A systematic review. European Radiology, 34, 7397–7407. https://doi.org/10.1007/s00330-024-10804-6
ABSTRACT: Artificial intelligence (AI) and large language models (LLMs) have demonstrated strong diagnostic performance in narrowly defined medical tasks, but less is known about their accuracy when diagnosis is open ended and spans diverse disease categories. The present study evaluated GPT-4o as a general diagnostic system using 1,000 real-patient cases derived from the MIMIC-IV clinical database. Clinical information from each case was parsed and assembled into a structured summary, and GPT-4o generated a ranked list of five possible diagnoses. Three prompting strategies were compared: zero-shot, few-shot, and chain-of-thought prompting. Performance was evaluated at both the disease level and the broader medical-category level using Top-1, Top-3, and Top-5 accuracy. Zero-shot prompting produced the strongest diseaselevel performance, with Top-1 accuracy of 55.7%, Top-3 accuracy of 69.5%, and Top-5 accuracy of 73.0%. Few-shot prompting yielded corresponding accuracies of 52.1%, 65.9%, and 69.6%, while chain-of-thought prompting produced substantially lower accuracies of 34.2%, 46.7%, and 50.3%. At the category level, zero-shot Top-1 accuracy was 72.3%, compared with 69.4% for few-shot and 56.2% for chain-of-thought prompting. Zero-shot performance varied considerably across disease categories, with particularly high Top-5 disease-level accuracy for musculoskeletal (98%) and endocrine/metabolic (91%) cases and lower accuracy for cardiovascular (66%) and other (63%) cases. Accuracy also appeared to stabilize after approximately 300–400 evaluation samples. These findings suggest that GPT-4o can identify the correct diagnosis within a short differential diagnosis for a substantial proportion of heterogeneous real-patient cases, but performance depends on prompting strategy and disease category. Further research should address possible information leakage from dischargesummary framing, automated diagnosis-matching limitations, unequal category sample sizes, and prospective evaluation under conditions that more closely approximate real-time clinical diagnosis.

IJSSER is Member of