ChatGPT nearly on par with clinicians in Nigerian cancer study

Can an AI like ChatGPT write usable information on how to prevent cancer? Researchers in Nigeria have investigated. The results have been published in the Journal of Clinical and Translational Science.

The authors describe it as the first study to assess AI-generated patient information on cancer prevention in Nigeria and in Africa. Studies of AI and cancer information from low- and middle-income countries are scarce.

How the study was conducted

The researchers compared cancer prevention materials created with ChatGPT (the GPT-4o model) with materials created by clinicians in Nigeria. A total of 409 adults were recruited during a World Cancer Day walk in Abuja on February 1, 2025. Of those, 371 completed the questionnaire and are included in the analysis.

Participants rated the materials on three points: usefulness, understandability and clarity.

Roughly on par overall

Overall, the ChatGPT materials were on par with the clinicians' materials, and there were no significant overall differences. There were exceptions, however: The clinicians' statements on "healthy diet" and "limit alcohol" were rated as clearer than the AI's corresponding statements.

Both types of materials had a reading level of around 8th grade. That is above the recommended 6th-grade level, so the texts are harder to read than they should be. The ChatGPT text was also 64 percent longer, at 253 words versus 154.

Trust in AI

Only 14 percent of participants used AI for health information, but 58 percent said they trusted it. According to a post on OncoDaily by lead author Runcie Chidebe, participants most often turn to doctors (73 percent) and Google (52 percent). Ninety percent were interested in cancer prevention apps.

Important caveats

Participants were highly educated and predominantly urban: 90 percent held at least a bachelor's degree, and only 20 percent lived in rural areas. According to the authors, this likely overstates how clear the text seems to the general population. The study measured only immediate impressions, not whether people remembered the advice or changed their behavior, and not whether the content was factually correct.

The authors conclude that GPT-4o can produce patient information comparable to that of clinicians. However, all AI output requires clinical review and adaptation to culture and health system before it is put into use. According to Chidebe, culturally relevant, evidence-based African information is also needed in AI systems to prevent misinformation.