International Journal of Social Science & Economic Research
Submit Paper

Title:
The relative effectiveness of a self-assessment chatbot and Chat GPT for teaching neurological diseases

Authors:
Ashna Melanathuru and John Leddo

Journal: International Journal of Social Science and Economic Research (IJSSER)
ISSN: 2455-8834
Volume: 11
Issue: 9
Publication Date: September 2026
DOI: 10.46609/IJSSER.2026.v11i09.016

|| ||

Ashna Melanathuru1 and John Leddo 1,2,3
1. MyEdMaster, LLC
2. METY Technology, Inc.
3. METY Foundation

MLA 8
Melanathuru, Ashna, and John Leddo. "The relative effectiveness of a self-assessment chatbot and Chat GPT for teaching neurological diseases." Int. j. of Social Science and Economic Research, vol. 11, no. 9, Sept. 2026, pp. 4721-4734, doi.org/10.46609/IJSSER.2026.v11i09.016. Accessed Sept. 2026.
APA 6
Melanathuru, A., & Leddo, J. (2026, September). The relative effectiveness of a self-assessment chatbot and Chat GPT for teaching neurological diseases. Int. j. of Social Science and Economic Research, 11(9), 4721-4734. Retrieved from https://doi.org/10.46609/IJSSER.2026.v11i09.016
Chicago
Melanathuru, Ashna, and John Leddo. "The relative effectiveness of a self-assessment chatbot and Chat GPT for teaching neurological diseases." Int. j. of Social Science and Economic Research 11, no. 9 (September 2026), 4721-4734. Accessed September, 2026. https://doi.org/10.46609/IJSSER.2026.v11i09.016.

References

[1] . Ahmad, M. and Leddo, J. (2023). The Effectiveness of Cognitive Structure Analysis in Assessing Students’ Knowledge of the Scientific Method. International Journal of Social Science and Economic Research, 8(8), 2397-2410
[2] . Anderson, J.R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369-405.
[3] . Bekkari, V., and Leddo, J., (2023). Cognitive Structure Analysis: Assessing Elementary School Students in Math to Determine the Types of Knowledge They Have. International Journal of Social Science and Economic Research, 8(10)
[4] . Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational researcher, 13(6), 4-16.
[5] . Brna, P., Ohlsson, S.and Pain, H. (1993) (Eds.). Proceeding of Artificial Intelligent in Education ‘93. Charlottesville, VA: Association for the Advancement of Computing in Education.
[6] . Challagulla, M & Leddo, J. (2025). Assessing and Remediating Knowledge Improves Problem- solving Performance in Middle School Math Compared to Remediating Problem-solving Steps. International Journal of Social Science and Economic Research, 10(8), 3618-3629.
[7] . Cynkin, C. and Leddo, J. (2023). Teaching Students to Self-Assess Using Cognitive Structure Analysis: Helping Students Determine What They Do and Do Not Know. International Journal of Social Science and Economic Research, 8(9), 3009-3020.
[8] . Dandemraju, A., Dandemraju, R. and Leddo, J. (2024). Teaching students to self-assess their own chemistry knowledge. International Journal of Social Science and Economic Research, 9(2), 541-549.
[9] . De Kleer, J. and Brown, J.S. (1981). Mental models of physical mechanisms and their acquisition. In J.R. Anderson (Ed.), Cognitive Skills and their acquisition. Hillsdale, NJ: Erlbaum
[10] . Greer, J. (1995) (Ed.) Proceeding of Artificial Intelligent in Education ‘95. Charlottesville, VA: Association for the Advancement of Computing in Education.
[11] . Hasanein, M., & Sobaih, A. (2023). Conversational AI chatbots for health-related support: User engagement, accuracy, and satisfaction. European Journal of Investigation in Health, Psychology and Education.
[12] . Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F. and Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274.
[13] . Koedinger, K. R., & Corbett, A. (2006). Cognitive Tutors: Technology Bringing Learning Sciences to the Classroom. In R. K. Sawyer (Ed.), The Cambridge handbook of: The learning sciences (pp. 61–77). Cambridge University Press.
[14] . Konadu, E., & Kusi, A. (2025). Effects of conversational mental-health chatbots on stress, confidence, and engagement among high-school and college students. American Journal of Education and Learning.
[15] . Leddo, J., Chen, T., Menachery, A., Agarwal, J. and Agarwal, T. (2021). Towards Improving Personal Assistants and Educational Software: How Questions Are Answered Affects Learning. International Journal of Social Science and Economic Research, 6(2), 696-705.
[16] . Leddo, J. Clark, A. and Clark, E. (2021). Self-Assessment of Understanding: We Don’t Always Know What We Know. International Journal of Social Science and Economic Research, 6(6), 1717-1725.
[17] . Leddo, J. Clark, E. and Clark, A. (2025). Using self-assessment and remediation to raise middle school student achievement in math. International Journal of Social Science and Economic Research, 10(3), 1083-1092.
[18] . Leddo, J., Cohen, M.S., O’Connor, M.F., Bresnick, T.A., and Marvin, F.F. (1990). Integrated knowledge elicitation and representation framework (Technical Report 90?3). Reston, VA: Decision Science Consortium, Inc..
[19] . Leddo, J., Li, S. and Zhang, Y. (2022). Cognitive Structure Analysis: A technique for assessing what students know, not just how they perform. International Journal of Social Science and Economic Research, 7(11), 3716-3726.
[20] . Leddo, J. and Sak, S. (1994). Knowledge Assessment: Diagnosing what students really know. Presented at Society for Technology and Teacher Education.
[21] . Leddo, J., Zhang, Z. and Pokorny, R. (1998).Automated Performance Assessment Tools. Proceedings of the Interservice/Industry Training Systems and Education Conference. Arlington, VA: National Training Systems Association.
[22] . Liang, I. and Leddo, J. (2020). An intelligent tutoring system-style assessment software that diagnoses the underlying causes of students’ mathematical mistakes. International Journal of Advanced Educational Research, 5(5), 26-30.
[23] . Létourneau, A., Deslandes Martineau, M., Charland, P., Karran, J. A., Boasen, J., and Léger, P. M. (2025). A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. Science of Learning, 10(1), 29.
[24] . Maviti, A. and Leddo, J. (2025). A Self-assessment Chatbot Greatly Outperforms Chat GPT in Teaching High School Students Calculus. International Journal of Social Science and Economic Research, 10(10), 5519-5531.
[25] . Maviti, A., Leddo, J. and Prakash, P. (2025). A Self-assessment Chatbot Greatly Outperforms Gemini in Teaching High School Students Calculus. International Journal of Social Science and Economic Research, 10(11), 6073-6085.
[26] . Nehra, P. and Leddo, J. (2024). The effects of Cognitive Structure Analysis in self-assessing and remediating knowledge gaps in introductory Spanish. International Journal of Social Science and Economic Research, 9(12), 5956-5964.
[27] . Newell, A. and Simon, H.A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Prakash, P. and Leddo, J. (2025a). Using Self-Assessment and Remediation to Raise Student Achievement in Reading Comprehension. International Journal of Social Science and Economic Research, 10(1), 277-286.
[28] . Prakash, P. and Leddo, J. (2025b). Using Self-Assessment and Remediation to Raise Student Achievement in Mathematics. International Journal of Social Science and Economic Research, 10(1), 447-456.
[29] . Prakash, P. and Leddo, J. (2025c). Using Self-Assessment and Remediation to Raise Student Achievement in History. International Journal of Social Science and Economic Research, 10(3), 650-659.
[30] . Prakash, P. and Leddo, J. (2025d). Using Self-Assessment and Remediation to Raise Middle School Student Achievement in Reading Comprehension. International Journal of Social Science and Economic Research, 10(3), 1130-1140.
[31] . Prakash, P. and Leddo, J. (2025e). Using Self-Assessment and Remediation to Raise Middle School Student Achievement in Science. International Journal of Social Science and Economic Research, 10(4), 1471-1482.
[32] . Quillian, M.R. (1966). Semantic memory. Cambridge, MA: Bolt, Beranek and Newman.
[33] . Rapolu, S. and Leddo, J. (2026). Impact of a Self-Assessment Chatbot on High School Students’ Learning of Cell Communication. International Journal of Social Science and Economic Research, 11(1), 320-333.
[34] . Sathiyamoorthy, S.S. and Leddo, J. (2025). Using Self-Assessment and Remediation to Raise College Student Achievement in Psychology. International Journal of Social Science and Economic Research, 10(5), 1859-1869.
[35] . Schank, R.C. (1982). Dynamic Memory: A theory of learning in computers and people. New York: Cambridge University Press.
[36] . Schank, R.C. and Abelson, R.P. (1977). Scripts, Plans, Goals, and Understanding. Hillsdale, NJ: Erlbaum.
[37] . Van Lehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational psychologist, 46(4), 197-221.
[38] . Villegas-Ch, W., Buenano-Fernandez, D., Navarro, A. M., and Mera-Navarrete, A. (2025). Adaptive intelligent tutoring systems for STEM education: analysis of the learning impact and effectiveness of personalized feedback. Smart Learning Environments, 12(1), 1-31.
[39] . Wang, Y. and Leddo, J. (2025). Improving the Effectiveness of Chatbots by Incorporating Self-Assessed User Knowledge into the Question-Answering Process. International Journal of Social Science and Economic Research, 10(8), 3574-3586.
[40] . Zawacki-Richter, O., Marín, V. I., Bond, M., and Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators?. International journal of educational technology in higher education, 16(1), 1-27.
[41] . Zhou, L.N. and Leddo, J. (2023). Cognitive Structure Analysis: Assessing Students’ Knowledge of Precalculus. International Journal of Social Science and Economic Research, 8(9), 2826-2836.

ABSTRACT:
Personal assistants, chatbots, and large language models (LLMs) increasingly pervade our society. Many people turn to chatbots and LLMs for medical advice and to answer medical questions. While chatbots and LLMs will inform users of their potential to make mistakes, since following bad medical advice can be harmful, they focus primarily on conveying information rather than making sure the user understands the information. This is an important issue as well, since incorrectly following good advice can also be harmful. To address this issue, we created a chatbot to teach users about neurological diseases. Our chatbot has users self-assess what they know about the subject so that when users ask it questions, it takes the user’s existing level of knowledge into account and focuses on filling knowledge gaps.
We tested the effectiveness of this self-assessment chatbot against Chat GPT. 20 Participants were taught about neurological diseases and then half used Chat GPT to help them understand what they learned and half used our self-assessment chatbot. Afterwards, both groups received a post-test to see how much they learned. Those using Chat GPT scored 73.75%, on average, on the post-test, while those using the self-assessment chatbot scored a statistically higher 83.5% on average.

IJSSER is Member of