Implementation and validation of an automated, longitudinal robotic surgical evaluation and feedback program at a high-volume center and impact on training Journal Article
| Authors: | Wilcox Vanden Berg, R. N.; Vertosick, E. A.; Sjoberg, D. D.; Cha, E. K.; Coleman, J. A.; Donahue, T. F.; Eastham, J. A.; Ehdaie, B.; Laudone, V. P.; Pietzak, E. J.; Smith, R. C.; Goh, A. C. |
| Article Title: | Implementation and validation of an automated, longitudinal robotic surgical evaluation and feedback program at a high-volume center and impact on training |
| Abstract: | Background: Surgical education lacks a standardized, proficiency-based approach to evaluation and feedback. Objective: To assess the implementation and reception (ie, feasibility) of an automated, standardized, longitudinal surgical skill assessment and feedback system, and identify baseline trainee (resident and fellow) characteristics associated with achieving proficiency in robotic surgery while learning robotic-assisted laparoscopic prostatectomy. Design, setting, and participants: A quality improvement study assessing a pilot of a surgical experience tracking program was conducted over 1 yr. Participants were six fellows, eight residents, and nine attending surgeons at a tertiary cancer center. Intervention: Trainees underwent baseline self-assessment. After each surgery, an evaluation was completed independently by the trainee and attending surgeons. Performance was rated on a five-point anchored Likert scale (trainees were considered “proficient” when attending surgeons’ rating was ≥4). Technical skills were assessed using the Global Evaluative Assessment of Robotic Skills (GEARS) and Prostatectomy Assessment and Competency Evaluation (PACE). Outcome measurements and statistical analysis: Program success and utility were assessed by evaluating completion rates, evaluation completion times, and concordance rates between attending and trainee surgeons, and exit surveys. Baseline characteristics were assessed to determine associations with achieving proficiency. Results and limitations: Completion rates for trainees and attending surgeons were 72% and 77%, respectively. Fellows performed more steps/cases than residents (median [interquartile range]: 5 [3–7] and 3 [2–4], respectively; p < 0.01). Prior completion of robotics or laparoscopic skill courses and surgical experience measures were associated with achieving proficiency in multiple surgical steps and GEARS domains. Interclass correlation coefficients on individual components were 0.27–0.47 on GEARS domains. Conclusions: An automated surgical experience tracker with structured, longitudinal evaluation and feedback can be implemented with good participation and minimal participant time commitment, and can guide curricular development in a proficiency-based education program by identifying modifiable factors associated with proficiency, individualizing education, and identifying improvement areas within the education program. Patient summary: An automated, standardized, longitudinal surgical skill assessment and feedback system can be implemented successfully in surgical education settings and used to inform education plans and predict trainee proficiency. © 2024 The Authors |
| Keywords: | adult; outcome assessment; cancer center; correlation coefficient; training; education; prostatectomy; surgeon; surgical training; surgery; feedback system; robotics; self evaluation; education program; longitudinal study; learning; learning curve; participation; human; male; article; likert scale; robot assisted surgery; quality improvement study |
| Journal Title: | European Urology Open Science |
| Volume: | 62 |
| ISSN: | 2666-1691 |
| Publisher: | Elsevier BV |
| Date Published: | 2024-04-01 |
| Start Page: | 81 |
| End Page: | 90 |
| Language: | English |
| DOI: | 10.1016/j.euros.2024.02.014 |
| PROVIDER: | scopus |
| PMCID: | PMC10926308 |
| PUBMED: | 38468865 |
| DOI/URL: | |
| Notes: | The MSK Cancer Center Support Grant (P30 CA008748) is acknowledged in the PDF -- Corresponding author is MSK authors: Alvin C. Goh -- Source: Scopus |
Altmetric
Citation Impact
BMJ Impact Analytics
MSK Authors
Related MSK Work