Siemens and Nvidia aim to bring more virtual reality to digital twins

Siemens and Nvidia have agreed to work more closely together to drive the development of digital twins that can rely on more realistic simulations and in turn use those simulations to help improve the training of machine-learning algorithms that the systems they model will come to rely on.

The first step in the expanded partnership is to connect tools in the Siemens Xcelerator lineup to Nvidia’s virtual-world creation software Omniverse. This will see tools such as the process simulations generated by the Tecnomatix portfolio be used to deliver data on the movement of robots to the Omniverse rendering environment. At this stage, the focus is more on plant and buildings simulations though the linkups will likely expand into product design, where tools such as PAVE360 already provide digital twins of the electronics within a road vehicle.

At the launch event, Siemens CEO Roland Busch used the example of robots in a factory to describe how this approach would likely work. “Imagine your factory in China is slowing down: it produces fewer parts every day nothing bad but it’s all adding up. The team at the factory has no idea why it’s happening.”

Busch described bringing a range of engineers at various locations, including the manufacturer of the robots on the production line into a VR environment. “They immerse themselves in the digital twin of the plant. It mirrors exactly what is happening in the real plant and it is not a still photograph. It is in real time, down to the physical behavior of the robots. The team travels back in time in the metaverse to when the output was strong to see what has changed since then. They realise that one robot on the feeder line has missed its latest software update and it’s out of sync. The team updates the software in the digital twin and the robot immediately speeds up and works in sync. Now the team is confident enough to update the software on the real robot.”

Though a key change with the linkup will be the ability to create photorealistic real-time visualisations for digital twins that can be used to drive virtual reality and augmented reality displays, execs from both companies were keen to stress that the benefit would come more from fidelity to real-world processes rather than look and feel.

“The digital twin is physics-based. It doesn’t just look like the real thing, it behaves like the real thing,” claimed Siemens CEO Roland Busch. “This is not about animation but simulation. If you don’t mimic the real world accurately, you don’t get the benefit out of it. The digital twin has to be comprehensive. If you simulate something on a digital twin and you figure out you want to change something in the real world, you better get it right.”

Jensen Huang, Nvidia co-founder and CEO, pointed to the need for the digital twin and the physical systems it represents to match: “You need to believe they are the same. That’s why it’s so profoundly different to a video game.”

The decision comes amid a launch by the German giant of what it calls the Xcelerator marketplace, an extension of the original Xcelerator strategy launched in the autumn of 2019 and into which Siemens has gradually been pulling its range of software tools. This expansion will bring in software from partners that are built around the Siemens products, such as its Mendix “low code” software tools and APIs in an attempt to lower the entry cost to building digital twins and encourage digitalisation of processes in general. “Companies need help integrating information technology and operational technology,” said Dirk Didascalou, CTO of Siemens Digital Industries Software.

Cedrik Neike, CEO of Siemens Digital Industries, added that the approach will allow other tools that support the APIs used within Xcelerator: “With Xcelerator we want to be open so you’re not locked-in with anyone.”

Design for HaaS

Didascalou said the approach taken by Siemens will likely lead to further changes in the way that hardware architectures are designed in the various automation fields, pointing to machine-tool maker Heller as an example with the idea of “hardware as a service” in which production optimisations over time can be added based on analysis of flows to see where bottlenecks are forming. The company’s machine tools can communicate with a Siemens shopfloor computer to calculate how best to organise the flow of parts and so reduce delays.

“Now they are going even further with a new business model where a customers can buy or lease a machine at a much lower price that has the basic functionality and then when they need it or if they just want to try it, they can upgrade to the premium features at the press of a button,” Didascolou explained.

In principle, the use of more digital twins would allow the customer to see how the upgrade would perform without having to commit physical parts to the experiment.

Another key change will be in the flow of information between the digital twin and the design tools. BMW board member Milan Nedeljković said, “The digital twin is not the challenge. The challenge is to link the digital twin into the existing systems.”

As part of the partnership, the companies aim to link the environments so that changes made in the Omniverse realm will be rippled back to the design environment to ensure that the models do not go out of sync and to ensure that change requests are completed without having to re-enter each one by hand.

A final component of the tie-up will likely lie in the training of AI systems. In the field of automated driving, scenarios based on synthetic data rendered in virtual worlds are already being used to provide neural networks with more data than can be obtained even in millions of miles of real-world driving. Huang pointed to factory automation as another situation where synthetic data can improve the training of robots expected to work closely with humans. Though the PAVE360 environment that was introduced to aid the testing of complex electronic in-car systems does not have a link into Omniverse as yet, Tony Hemmelgarn, CEO of Siemens Digital Industries Software characterised both as important “shift left” technologies.

“It’s making those decisions early, so that you haven’t committed to the physical design until you’re pretty certain that you’ve got what you want,” Hemmelgarn added.