Longtime motion-capture company Vicon announced it is developing a markerless motion-capture system that uses machine learning and artificial intelligence, resulting in a high-quality, high-output system. The company kicked off a two-phase Pioneer Program whereby Vicon will work with industry groups and users across all segments to determine the possibilities of the technology. That will then be followed by a hands-on access period. Vicon teamed with Dreamscape at Siggraph to offer an interactive demonstration of the technology involving a VR experience.
What do we think? For years, mocap vendors have displayed their motion-capture technology on the exhibitor floor at Siggraph. Mostly, we have seen mainly women in mocap suits stretching, prancing around the volume, and making exaggerated motions, which were then featured on displays for all to see how the motion was applied to the character, whether a skeleton that looks like a primitive stick figure, or a monster, humanoid, or other type of model. Year after year, though, these demonstrations tend to draw eyeballs. In recent years, we have seen more interesting examples—basketball players, people riding miniature bikes, and more—as mocap tech advanced.
I tried out the Vicon/Dreamscape example and have to say that it went a long way to selling the technology. I did have a discussion with Vicon ahead of the demo about its technology, and by trying it myself, I was able to truly appreciate the ease of use, the low latency, and the accuracy. Moreover, I was able to experience firsthand the ability to move unencumbered. It’s one thing to hear those things but quite another to realize them yourself.
Vicon embarks on a new era of mocap with markerless tech that integrates ML and AI
Vicon has been in the motion-capture business for nearly four decades. Even though its systems are used for a wide range of applications, including engineering and life sciences, Vicon was able to turn up the spotlight on itself and motion capture as a whole with its use in the film industry during the late 1990s. Over the years, Vicon has provided various mocap options, from optical solutions to inertia systems. At Siggraph 2023, the company revealed that it will be offering a markerless option, too.
The fact that the new system is markerless is new for Vicon but not for motion capture. According to Vicon, its goal with markerless, as with all its product development, is not to race to be first, but rather to take the time so it can meticulously create the highest-quality, highest-output technology.
Thanks to advances in computer vision, markerless systems have been developed that negate the need for actors to wear special equipment. What is new, however, is the integration of machine learning and artificial intelligence, which Vicon has done. For the past three years, Vicon has been working on the markerless mocap system at its R&D facility in Oxford, UK, under the direction of a new team headed by CTO Mark Finch.
“Today marks the beginning of a new era for motion capture. The ability to capture motion without markers, while maintaining industry-leading accuracy and precision, is an incredibly complex feat. After an initial research phase, we have focused on developing the markerless capture algorithms, robust real-time tracking, labeling, and solving needed to make this innovation a reality,” said Imogen Moorhouse, Vicon CEO. “What we are demonstrating at Siggraph is not a one-off concept or simply a technology demonstrator. It is our first step towards future product launches, which will culminate in a first-of-its-kind platform for markerless motion capture.”
Moorhouse said the company wanted to enter the markerless space with unparalleled motion-capture precision and accuracy. “So even before we launched our markerless motion-capture platform, we wanted to demonstrate how we at Vicon have pushed the boundary of what was perceived as possible,” she said. “At Siggraph 2023, we showcased a six-person, real-time, full-body markerless solve to bring that to life.”
For that fun but compelling demonstration, Vicon teamed up with VR company Dreamscape. There was a constant line waiting to see the markerless solution in action, which was used to drive a six-person multi-modal real-time solve scenario, set against Dreamscape’s location-based VR adventure “The Clockwork Forest,” created in partnership with Audemars Piguet. In the experience, participants are miniaturized to ant size as they explore a mechanical wonderland, while working cohesively to repair the so-called Source of Time and restore nature to its natural rhythm.
Artanim, the research arm of Dreamscape, worked with Vicon to integrate the new technology within the experience. “Achieving best-in-class virtual body ownership and immersion in VR requires both accurate tracking and very low latency. We spent substantial R&D effort evaluating computational performance of machine learning-based tracking algorithms, implementing and fine-tuning the multi-modal tracking solution, as well as taking the best from the full-body markerless motion capture and VR headset tracking capabilities,” said Chris Hoskin, Vicon CMO.
Machine learning is at the core of what Vicon has been working on in its research and development phase. When most people think of machine learning and artificial intelligence, they cite the large language models that power solutions like ChatGPT, Hoskin noted. At Vicon, the focus is on computer vision. Rather than words and language, they are using image-based datasets to train machine learning and AI models that sit at the core of its markerless solution, she explained.
In the showcase demonstration, that markerless recognition of the human form—using prototype cameras, software, and algorithms—was used to track six people, with their full body solved in real time, in a VR experience. As Moorhouse explained, this completely removes the need for participants to wear heavy backpacks and gloves plastered with motion-capture markers. And as a result, the VR experience is more seamless and believable, as the motion-capture technology is largely invisible and non-invasive.
Aaron Grosky, president and COO of Dreamscape, agreed that markerless motion capture provides a more true-to-life adventure within an immersive VR experience by enabling more free-flowing movement and exploration with very little user gear required by the participants. The accuracy of Vicon’s markerless technology makes a big difference, too. He added that his company has been waiting for the point when markerless mocap can support applications requiring this level of precision.
Grosky noted the implications for both quality of experience and ease of operations across all its efforts, from location-based entertainment to transforming the educational journey with the company’s Dreamscape Learn, are huge, he added. Nevertheless, the potential use cases for markerless motion capture go far beyond VR and entertainment. Vicon sees this as having incredible potential in industries from engineering to life sciences to aerospace and defense, and beyond.
Vicon is still tweaking the end-to-end markerless solution and has not provided a time frame for release. The next step leading up to general availability is the start of the Pioneer Program, which also launched at the conference. The program rollout will occur in two phases. In the first one, the Pioneer Phase, Vicon will be collaborating with motion-capture end users, partners, influencers, evangelists, and distributors to identify all the use cases for markerless motion capture. This will involve special interest groups, along with webinars and live sessions, to discuss the technology and determine what is possible. The second phase, the Early Adopter Phase, will provide hands-on access to the technology prior to launch.
Because Vicon works across multiple market sectors, participation in the Pioneer Program will span entertainment, life science, engineering, as well as the VFX and VR communities.