Artificial intelligence (AI) has numerous applications in surgical quality assurance. We assessed AI accuracy in evaluating the critical view of safety (CVS) and intraoperative events using 1,051 laparoscopic cholecystectomy videos.

The videos were annotated by AI for disease severity, CVS achievement, and intraoperative events.

Surgeons performed focused video review on procedures with more than one intraoperative event. AI versus surgeon annotation of CVS components and intraoperative events were compared. For all cases intraoperative-event association with CVS achievement and severity was examined using ordinal logistic regression.

Using AI annotation, surgeons reviewed 50 videos/hr. CVS was achieved in fewer than 10% of cases. Hepatocystic triangle and cystic plate visualization was achieved more often in low-severity cases.

AI-surgeon agreement for all CVS components exceeded 75%, with higher agreement in high-severity cases. Surgeons agreed with 99% of AI-annotated intraoperative events. AI-annotated intraoperative events were associated with both disease severity and number of CVS components not achieved.

AI annotation allows for efficient video review and is a promising quality assurance tool. Disease severity may limit its use and surgeon oversight is still required, especially in complex cases. Continued refinement may improve AI applicability and allow for automated assessment