AI vs human abstraction of pain scores and analgesic trends in low-dose radiation therapy for osteoarthritis: A concordance study
Presenter: Camille Schwartz, No Degree Session: Large Language Models in the Clinic Time: 4/20/2026 2:00:00 PM → 4/20/2026 5:00:00 PM
Authors
Camille Schwartz 1 , Michael Anderson 2 , Kelsey Moakler 2 , Bradley Newby 2 , David Davenport 2 , Matthew Schwartz 3 1 University of Nevada, Las Vegas, Las Vegas, NV, 2 Comprehensive Cancer Centers of Nevada, Henderson, NV, 3 University of Nevada, Las Vegas (UNLV), Las Vegas, NV
Abstract
Background / Significance: Accurate clinical data abstraction underpins outcomes research but remains resource-intensive and variable. Artificial intelligence (AI) offers potential to streamline this process while maintaining accuracy, yet its validation in real-world oncology data is limited. This undergraduate-led study evaluated Microsoft Copilot in replicating human chart abstraction for pain and functional outcomes among osteoarthritis (OA) patients treated with low-dose radiation therapy (LDRT), emphasizing concordance, efficiency, and reproducibility. Methods: Clinical notes from 30 patients (55 joints) treated with LDRT (3 Gy in six 0.5 Gy fractions) between August 2024 and August 2025 were analyzed. Human reviewers manually extracted Numeric Rating Scale (NRS, 0-10) and von Pannewitz Score (VPS, 0-4) data at baseline, end-of-treatment (EOT), and one-month follow-up. Microsoft Copilot (HIPAA-compliant) independently extracted identical data. Discrepancies were classified as exact, minor (≤2-point difference), or missed. Concordance was calculated using intraclass correlation coefficient (ICC) for NRS and weighted kappa for VPS. Abstraction time per chart was recorded. Results: AI achieved 92 percent exact match for NRS (ICC = 0.96, 95 percent CI 0.93-0.98) and 94 percent for VPS (kappa = 0.91). No fabricated data were produced; one human omission was detected by AI. Mean abstraction time was about 2 minutes per chart versus 30 minutes for humans, a greater than 10-fold efficiency gain. Conclusions: AI-assisted abstraction showed near-perfect concordance with human reviewers while reducing time and labor by over 90 percent. This undergraduate-led investigation demonstrates that responsible AI implementation can enhance accuracy, reproducibility, and efficiency in radiation-oncology outcomes research and accelerate evidence generation in oncology. Disclosure: No external funding or conflicts of interest. Data abstraction performed using Microsoft Copilot (HIPAA-compliant); no generative AI tools were used for writing.
Disclosure
C. Schwartz, None.. M. Anderson, None.. K. Moakler, None.. B. Newby, None.. D. Davenport, None.. M. Schwartz, None.
Cited in
Control: 185 · Presentation Id: 2551 · Meeting 21436