{"id":66700,"date":"2023-12-11T11:25:05","date_gmt":"2023-12-11T15:25:05","guid":{"rendered":"https:\/\/coinscreed.com\/staging\/?p=66700"},"modified":"2023-12-11T11:25:07","modified_gmt":"2023-12-11T15:25:07","slug":"chatgpt-4-0-passes-clinical-neurology-exam","status":"publish","type":"post","link":"https:\/\/coinscreed.com\/staging\/chatgpt-4-0-passes-clinical-neurology-exam\/","title":{"rendered":"ChatGPT 4.0 Passes Clinical Neurology Exam"},"content":{"rendered":"\n

ChatGPT 4.0, the most recent iteration of OpenAI's large language model (LLM), scored 85% correctly on a clinical neurology exam of the American Board of Psychiatry and Neurology during a proof-of-concept<\/a> study.<\/p>\n\n\n\n

\"ChatGPT
ChatGPT 4.0 Passes Clinical Neurology Exam<\/figcaption><\/figure>\n\n\n\n

A group of researchers from the German Cancer Research Center in Heidelberg and University Hospital Heidelberg published the experiment results on December 7. Two LLMs were examined on May 31: ChatGPT 3.5 and its subsequent iteration, ChatGPT 4.0.<\/p>\n\n\n\n

The researchers supplemented a subset of the European Board of Neurology questions with those from the American Board of Psychiatry and Neurology\r\n \r\n \r\n \r\n <\/g>\r\n \r\n \r\n \r\n <\/clipPath>\r\n <\/defs><\/svg><\/span><\/a>‘s neurology exam question bank.<\/p>\n\n\n\n

The accuracy rate of the older ChatGPT model was 66.8%, or 1306 correct responses out of 1956, whereas the more recent ChatGPT 4.0 improved to 85% with 1662 correct answers. Humans achieved an average score of 73.8%. <\/p>\n\n\n\n

ChatGPT 4.0 demonstrated superior performance compared to human users on psychological, cognitive, and behavioral-related questions, effectively “passing” the neurology exam with a score of 70%. A passing grade is typically equivalent to 70% accurate responses in academic institutions.<\/p>\n\n\n\n

\n

\u201cThese findings suggest that with further refinements, large language models <\/a>could have significant applications in clinical neurology.\u201d<\/em><\/p>\n<\/blockquote>\n\n\n\n

As per the experiment's conducting group, the following modifications to the LLMs should be considered for clinical neurology application:<\/p>\n\n\n\n

The researchers note that several reservations remain. Although the documentation and decision-making support systems offer a distinct opportunity to implement LLMs, neurologists should exercise prudence in practice due to their continued limitations regarding high-order cognitive tasks. One of the authors of the study, Dr. Varun Venkataramani, stated:<\/p>\n\n\n\n

\n

We see our study more as a proof of concept for the capabilities of LLMs. There is still development needed and probably even specific fine-tuning of LLMs to make them properly applicable for clinical neurology.\u00a0<\/em><\/p>\n<\/blockquote>\n\n\n\n

More precisely, our research is a demonstration of the potential of LLMs. Further development and precise refinement are required to ensure that LLMs are applicable in clinical neurology.<\/p>\n\n\n\n

AI is already tackling significant healthcare challenges, including combating antibiotic overprescribing in Hong Kong and discovering a remedy for cancer on behalf of AstraZeneca<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

ChatGPT 4.0, the most recent iteration of OpenAI’s large language model (LLM), scored 85% correctly on a clinical neurology exam of the American Board of Psychiatry and Neurology during a proof-of-concept study. A group of researchers from the German Cancer Research Center in Heidelberg and University Hospital Heidelberg published the experiment results on December 7. […]<\/p>\n","protected":false},"author":12,"featured_media":66709,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9],"tags":[17547,17548,15168,14081],"class_list":["post-66700","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech","tag-chatgpt-4","tag-clinical-neurology-exam","tag-llm","tag-openai"],"jetpack_featured_media_url":"https:\/\/coinscreed.com\/staging\/wp-content\/uploads\/2023\/12\/image-35.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/posts\/66700","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/comments?post=66700"}],"version-history":[{"count":0,"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/posts\/66700\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/media\/66709"}],"wp:attachment":[{"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/media?parent=66700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/categories?post=66700"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coinscreed.com\/staging\/wp-json\/wp\/v2\/tags?post=66700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}