Minimally invasive surgery is complex and prone to variation not routinely objectively measured. We established an association between skills and patient outcomes. The evolving application of artificial intelligence techniques could assist intraoperative analysis. In this study, we analysed 77 rectal cancer operations’ videos from a multicentre RCT that were recorded unedited and underwent blinded manual analysis using a validated, bespoke performance assessment tool (LapTMEpt) and the Objective Clinical Human Reliability Analysis (OCHRA). The OCHRA methodology involved segmentation of the 77 operations and manually annotating each case for the enacted errors and near misses. We provide a detailed description of the errors and near misses of over 380 hours of video analysis, containing 1377 errors. This dataset can inform machine learning to assist progress toward a fully automated, objective assessment of surgical skills.