The Virtual Operative Assistant

When looking at modern cancer treatment, one of the most important aspects is educating the doctors of the future. Artificial Intelligence can prove to be a very useful tool when large amounts of data are being used. A recent study on The Virtual Operative Assistant completed by Nykan Mirchi, Vincent Bissonnette, Recai Yilmaz, Nicole Ledwos, Alexander Winkler-Schwartz, and Ronaldo F. Del Maestro, shows the benefits of using AI to conduct training that tests cognitive skills and determines the level of psychomotor expertise that an operator possesses through the use of a surgical simulation. The study aimed to create a model that used AI and could be utilized to train surgeons and authenticate the model using the Virtual Operative Assistant. Artificial Intelligence technologies have often been labeled “black boxes” because of how difficult it is to understand how their algorithm makes a certain decision, however this newly designed model shows potential in multitudes of fields, including economics, finance, and healthcare.

During the experiment, 50 participants, all with differing levels of expertise were recruited and classified into two groups: skilled and novice. The classification was completed after all of the participants were asked to complete a complex virtual reality simulation where they had to remove a brain tumor located beneath the pia mater subpial tissue, a type of connective tissue, using two devices-one in each hand. The Virtual Operative Assistant that was created was able to generate large amounts of data from each individual's performance depending on how they interacted with the surgical simulation. This data ranged anywhere from the position of the tool to the forces applied on structures within the body.

In order for the Virtual Operative Assistant to perform the classification, many performance metrics were generated that were representative of differing levels of expertise in the surgical field. Out of the 250 metrics created, only 4 metrics with the highest level of accuracy were chosen after careful consideration and selection processes. The metrics included two for movement and two for safety, each with negative weights. The metric whose corresponding weight was the largest, played the greatest role in the decision making process of the algorithm. Once the metric set was finalized, the algorithm needed to be trained and tested. In order to train it, the algorithm would alter values of weights for each metric until it achieved the highest accuracy for prediction. The algorithm was then effectively tested using a completely new data set. 

The Virtual Operative Assistance was quite successful in classifying the skilled and novice subjects of the study using the four metrics. After the machine learning algorithm computed its classification between “skilled” or “novice”, it also gave users a breakdown of its assessment on both the safety and movement metrics. However, rather than assessing each metric on its own, the Virtual Operative Assistance included the relationship between metrics, allowing students to recognize that one strong metric may be making up for poor performance in another one. The model was also very specific, in the sense that the user must obtain full proficiency in all of the metrics in Step 1 to move on to Step 2, and if the user was unable to achieve this, they had to repeat the task until full competency was obtained.

Aside from its impressive ability to determine skill level, the feedback that the Virtual Operative Assistant is able to provide makes it extremely important in the world of science as it allows for a magnified understanding of the critical parts of surgery. The user receives three different types of feedback during and upon completion of the task: auditory, text, and video-based. A combination of automated audio and video-based feedback specific to cognitive metric issues that are identified is used to imitate a surgeon instructor in the operating room. Positive behaviors and components of the task that require improvement are broken down by text feedback, and the video-based feedback allows the trainee to compare themself to a skilled surgeon. The feedback supports positive behavior while also supplying detailed information on which habits of the operator need improvement.

Both aims of the study, introducing a new form of AI (The Virtual Operative Assistant) for teaching through simulation based training, and validation of the model with a complex task, were achieved. This new technology enables scientists to understand the expertise of an individual, identify cognitive expertise in tasks that are much too complex for human teachers to notice, and mimic real life training, all making it the perfect tool for simulation-based learning. With such a device, medical students will be able to practice surgery in a similar setting to the operating room, before they complete surgeries on actual patients. However, there are many hurdles that must be overcome before this educational model can be officially implemented into surgical education. Although expertise classification, unbiased feedback, and instructor input all help to shape this unique and technologically advanced tool into a promise for medical advancement in the future, downsides such as lack of surgeon involvement, relatively broad parameters for classification, and the ability to cheat the system prevent it from being established in surgical education. 

The upcoming blog post will address certain weaknesses as well as methods AI researchers can use in order to improve simulation training for surgeons.

References

Mirchi, N., Bissonnette, V., Yilmaz, R., Ledwos, N., Winkler-Schwartz, A., & Del Maestro, R. (2020, February 27). The Virtual Operative Assistant: An explainable artificial intelligence tool for simulation-based training in surgery and medicine. Retrieved August 10, 2020, from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7046231/