VentureBeat: Now, let’s talk about the industrial side of the metaverse. The metaverse is often showcased in VR in consumer experiences, but the industrial opportunity seems much more expansive. Virginie, can you kick us off?Maillard: We identify different application domains of the industrial metaverse in all phases of the product life cycle, from design engineering to testing and validation to production, operation, and even training. Let me give you an example in each domain.The first one, in the design and engineering phase, the industrial metaverse will allow more collaboration between stakeholders, with additional possibility to involve non-technical stakeholders in the process thanks to new user experiences. From this perspective, the industrial metaverse drives a democratization of simulations, allowing experts to communicate more effectively with non-experts. In that role, it will be possible to explore more design and manufacturing options more interactively, involving more viewpoints to create better products with shorter time to market as a benefit. This is the first concrete vision we have for the metaverse.The second one, in the testing and validation phase–the combination of photorealistic environments with multidisciplinary, multiphysics simulations gives us the possibility to virtually explore more operational scenarios. For example, we can generate synthetic data to train and test autonomous systems through machine learning, like AGVs in factories, robots, or autonomous cars. Since everything is virtual, we can investigate a large range of critical situations, like accidents, and perform massive testing that we wouldn’t be able to afford in real life. The intensive virtual training and testing of systems can ensure more safety, and again, shorter time to market.
The third example I have for you is in production. The major use case for the industrial metaverse is virtual reconditioning. That means you can virtually install and test new devices and software without disturbing an ongoing production line. For example, you can visualize new equipment in the factory with augmented reality, and you can also simulate interaction with existing manufacturing assets. This already exists, but let’s say the industrial metaverse will bring an augmented user experience into this domain.The last example I have for you is in operation. The major use case for the industrial metaverse here is already pretty well-known. It’s visualization of workflows and data around physical assets. For example, the operation data of a machine can be displayed in augmented reality on a tablet screen. The next step, we think, will be to bring information through new interfaces to the shop floor, to the operators, so they can make faster and better decisions. We call these executable digital twins. That’s a reduced model, self-contained, that can run on edge devices. This will be a compilation of the full scope of information we can have on the shop floor. As you can see, there’s a wide range of industrial applications for the metaverse.
Nvidia’s Earth-2 will be a digital twin of the planet.VentureBeat: Amy, what are opportunities that you’re focused on for your metaverse activities?Bunszel: First and foremost, there’s a lot of overlap with what Virginie talks about. In general, the themes of collaboration and innovation, having all this data together and all the stakeholders together, that’s going to be revolutionary for all the industries we work in. In particular, there’s one area I’ll double-click on a little bit, and that’s in AEC – architecture, engineering, and construction.We’re looking at design, build, and operate. One of the great things about looking across that whole life cycle is you can also start to look at sustainability, bringing decisions you would have been making in the build phase up into the design phase where you can leverage all the context you get in a metaverse or a digital twin to really understand the downstream implications of decision you might make about materials or construction methods. We’re excited about the opportunity, for both infrastructure and buildings, that the metaverse will give our customers to make high-impact decisions earlier in their design cycle, when they can have the most impact in the long term.VentureBeat: Lori, where are you focusing your development for industrial metaverses?Hufford: I agree with a lot of the same themes Virginie and Amy are talking about. For us, again, it’s the melding of the digital physical world to produce better outcomes for infrastructure. That’s where we’re focused. For example, during design, humans are interacting with digital twins to conduct interdisciplinary design reviews globally across the supply chain. In the world we live in today it’s important to be able to leverage talent anywhere, any place, to end up with the best design possible.With the digital twin, an infrastructure digital twin, we’re able to bring in models, reality context, maps. We can identify issues and visualize changes as they’re introduced into the design process, communicate with markups, and use digital tools in the metaverse to ensure a validation of design. We’re also looking, in construction, at using augmented reality to visualize and review construction sequencing, minimizing risk and leading to better outcomes. I can also echo some things Virginie talked about regarding operations, using augmented reality with infrastructure digital twins to improve safety, do simulations training, that sort of thing. There are so many opportunities to meld the worlds together that result in better outcomes.VentureBeat: How important is it for people to be able to take assets from one virtual world to another in the metaverse? How are you overcoming the universal interchange issues here, moving data from one industrial world to another?Lebaredian: If you step back again to 1993, when the first web browser came out, it’s hard to imagine that the web and everything we know about the internet would exist today if we didn’t have a common way of describing a web page. HTML–it wasn’t just nice to have a format that could be exchanged between browsers. It was essential. If every web browser that was created had a different way of describing content, then we wouldn’t be where we are today.If we look at the metaverse as this evolution and extension of the internet, of the web, a spatial overlay, then the content inside this metaverse has to be in a form that’s interchangeable, that everyone can contribute to and understand and participate in. We’re still very early, though. Just like in 1993, HTML 1.0, it couldn’t do much. You could put some text together with an image and hyperlinks. It took about 20 years for us to add all of the features that we enjoy today with video and interactive content and procedures and all these things. It’s going to take a long time for us to get to where we need to go with the metaverse, but there’s a lot we can do today.We settled on Universal Scene Description as the beginning of this. We didn’t see anything better out there, and quite frankly we think it’s quite good. But there’s still a lot of work to do. We need to layer more standards, how to describe everything we might want in a world, and how to do that in a very physically accurate way, one that matches the real world. Not just something that looks good. We’re contributing heavily to USD, and we invite all of our partners and everyone else to do so as well. Over the coming years we’ll sort out what the specific forms are, the specific patterns of how we describe different aspects of these worlds for different industries, layered on top of a common scheme.
Nvidia’s Earth 2 digital twin goals unveiled at its 2021 GTC conference.VentureBeat: Timoni, do you want to take a crack at that as well? How important is portability and interchange, and how are we going to do that?West: Rev gave a great background. USD is awesome. We’re also big fans. What I’m very curious about is–I like how Rev talked about the web in the early days. There’s always the question of what we actually need to have interoperability about. The reason why I’m not quite full-on “rah rah interoperability”–I’m big into web standards. I still am to this day. The only problem is that it is really, really tricky – and here I go back to thinking like a game designer – to be able to describe the behavior of an object in one space versus another.Maybe with industrial it’s a bit easier because they’ve already gone ahead and created amazing standards and levels of interoperability that make sense of the massive amounts of data that they deal with. But a classic example we see all the time is, I want to take a sword from one game and play with it in another game. Or another space, I should say, a virtual world. What does it do? What if this is a game with no swords? What if it’s a pacifist world? We may have the file format aspect that we can solve, but that later behavior of interoperability and interactivity that is a crucial aspect that differentiates the metaverse from the internet we know and love today, I really don’t have an answer to that. My guess is that it’s going to very much be a needs-must situation. As people demand this, as they find a need for it, we’ll see a home-brewed effort to enable the type of behavior that users are asking for.But as far as standards and file formats and that type of interoperability, I think that we’re making tremendous headway. We’re having great conversations and doing great work, actively discussing these things all the time. We talk to Nvidia and AutoDesk quite a bit along these lines.Lebaredian: I totally agree. It’s super hard. There are no easy answers to any of these things. That being said, I don’t think it means we shouldn’t try. A lot of smart people in the world, a lot of engineers are out there that can solve these problems. If there’s a will, I think we’ll figure it out.If you go back to the early days of the web, we had the same things being said about how to make applications that have behavior and interactive in there. And we had many false starts. We had things like Flash and ActiveX and all this other stuff. It sorted itself out. Strangely, we ended up with JavaScript as the way in which we describe everything.West: How did that happen? But it happened.Lebaredian: If you told me 20 years ago that not only would JavaScript be powering the internet and the web, but we’d actually compile C++ into JavaScript for web assembly, I’d say you were insane. That’s impossible. It will never work. But engineers do miraculous things when they need to. If there’s a need for this, I think we’ll figure it out. We just have to be willing to go and try. We’re going to have a lot of experiments, and many will fail, but some things will work.West: I don’t mean to sound negative. I was thinking more about–I think we’re saying the same things. I know people are going to want this. I’m very curious to see what gets pushed out. Like you said, we could not have predicted JavaScript as the de facto lingua franca of the dynamic internet. But it’s going to be very interesting to see. It’s a fun time to be here. I’m glad to be at this level of exploration right now.
A scene from an Ericsson Omniverse environment.VentureBeat: I’m curious, for our other panelists–can you think of a scenario for portability in why you might want to use something from somebody else in your applications for the metaverse? Gamers buy a lot of digital items and create avatars. They want to carry that stuff with them from one place to another. But what about in the industrial metaverse side? Can you think of a reason right now that you would want to have that kind of portability?Maillard: For sure. What we call openness or interoperability is part of the future of this industrial metaverse as an enabler, for the simple reason that it’s a way to connect the tools together, provide appropriate interfaces and consistent data management between the tools. The second aspect is to easily transfer virtual assets into different virtual environments. For example, a model of a car can flow from design and validation into production and test and operation, at each stage gaining more maturity to feed back and improve the next iteration of design as a full interconnected system. To do that we need interoperability because we need different tools to talk together.VentureBeat: I suppose that in a digital factory, you might have people making cars, but all the parts of those cars might be commonly used among different factories and different digital twins. In that kind of scenario, if there’s not necessarily a need to create the same part for two different digital twin factories, that’s where it becomes useful.I am curious about what’s here and now about the metaverse. What’s tangible for you? What are some concrete examples of metaverse work flows?West: It’s interesting. Again, it comes down to what you think the metaverse is. In terms of tangibility, the biggest examples of what I would see as solid, concrete uses of contextual computing would be a combination of devices. The metaverse originally was an idea about virtual reality. I’ve since–when I got into VR I was very much thinking, “This is it! It’ll replace all our screens with AR glasses! No hands!” And in the last seven years I’ve come to realize that screens are great, and so are keyboards. Every computing device we have comes with its advantages and disadvantages. VR is great for presence. It’s terrible for writing code. If you want to write the great American novel or do transcription, you need a keyboard. All these devices are just tools to help achieve your goal.And so a great example of what I would call a futuristic use case is when you have multiple different people who are able to use these different devices in the way that best suits the workflow they have. For example, somebody is wearing a HoloLens and they’re looking at a detailed report on an engine. Someone else has an iPad, and a third person is joining them from far away in VR. That type of interoperability between devices, that real time interactivity and ability to display information in the ways that best suit the needs of each person via their device, to me that’s the best and truest expression of what excites me about this new type of computing.Hufford: I couldn’t agree more with Timoni’s comment about–we have to be able to meet people where they are. We have to be able to incorporate this technology into their existing processes and workflows. That’s important to us. It has to be accessible and it has to be open, so that users can imagine their own applications and develop their own metaverse applications – in our case for infrastructure design, operation, and construction.I can give a specific example about how a leading organization is using infrastructure digital twins in the metaverse today. The ITER Project in France consists of 35 countries collaborating to build the world’s largest tokamak, a magnetic fusion device designed to prove the feasibility of nuclear fusion as a large-scale, carbon-free source of energy. It’s built on the same principle that powers the sun and stars – nuclear fusion, not fission. This is a huge design and construction project with a lot of complexity, complex engineering models, millions of parts. It has to be constructed off site and lifted into place.ITER wanted to allow humans to get into the digital twin and experience it. The container models were brought into Bentley Synchro, which is powered on the iTwin platform for 4D construction modeling and sequencing. Then, leveraging Unreal and Omniverse and using Cloud XR, all of this came together to create this amazing experience to immerse the team.The level of fidelity is important here. We have to be able to do this without reducing the complexity of the model in order for it to provide engineering value. It’s the first time that the client has been able to be inside the model. The photorealistic lighting enhances the experience, and the infrastructure digital twin, which contains all of the engineering data needed, allows the client and the users to fully make use of the experience.Another example being used today is remote bridge inspections. There’s so much existing infrastructure in the world. Performing bridge inspections to ensure safety and identify maintenance can be dangerous and time-consuming. The metaverse can help with that. Again, there has to be a high level of detail to be able to visualize cracks and rust and that sort of thing. Drones are used to capture reality models of a bridge, and then inspectors use HoloLens to inspect the resulting models. Inspectors can perform more inspections with less travel and work remotely, improving their safety as well.
Lockheed Martin is using Omniverse to simulate wildfires.VentureBeat: I’m curious about what the next evolution of the metaverse is going to be. What do you think is going to be real in, say, one year or five years? What might take as long as 10 years?Maillard: As I said at the beginning, for Siemens, the industrial metaverse is an evolution of what we already provide to enable digital twins and digital threads. In the future we see that realistic immersion, real time feedback, and openness will allow our customers to extend their experience of the industrial metaverse.We already discussed interoperability, so let me talk about immersion and real time. Regarding immersion, I mean the photorealistic quality of virtual 3D worlds. It will definitely allow for a better fit between human cognition and human interaction with the digital world. The second benefit is training autonomous systems, as I mentioned earlier, using synthetic data. That’s why immersion is important as an evolution. That won’t necessarily involve VR and AR devices, though. A laptop or tablet can be enough because they already provide immersion. That’s true in gaming today.The other important point is real-time collaboration. In the future, users should get instant feedback from their actions and interaction with others. They’ll feel a synchronous experience. The challenge here is not to sacrifice the accuracy of the simulation and prediction. In the end, the purpose of this real-time simulation is to test what-if scenarios quickly at the early stages of design and production and solve problems. That’s why, at Siemens, we already provide a set of multi-physics high-fidelity tools, and we continue to work on the next generation of technologies that will accelerate simulation while keeping the needed accuracy. Even if we already have a good foundation, the building blocks of the industrial metaverse, we still have an exciting time to come.Hufford: A lot of folks here have been talking about communication. Our view of communication is going to evolve and improve over the course of time. Of course people to people. We’ve been talking about that. But people to assets, assets to people, and assets to assets. A variety of modalities, including XR and others, will be involved there.To give some examples, assets that are able to alert that they’re experiencing an operation condition and require maintenance, or that would be able to guide a maintenance worker through their repair processes. Virtual lobbies where people and assets can interact. It’s going to allow us to more greatly leverage talent to design and construct and operate infrastructure. For example, using holograms to bring remote experts for consultation on site, expanding our ability to leverage talent across the globe.As I mentioned before, we need to manage our relationship to the natural world. As time goes on and the metaverse continues to evolve, we ourselves at Bentley–we want to be good stewards of the planet. The volume and complexity of this infrastructure data–it’s a lot of data that’s going to power the metaverse. We need to be looking for architectures and software designs that minimize compute and reduce carbon footprint along the journey.VentureBeat: What are some of the tech blockers that are getting in the way of these goals?Bunszel: First, I want to reiterate that–I think all of our customers are able to experiment today with the technology that exists out there. Digital twins have been happening for many years. They’re giving people a great context to jump off and do even more. But when I think about blockers, I have three that come to mind.First, in many cases, if we’re taking existing design data, we’re trying to use it for a new purpose. We’re going to need to automate how we get the right amount of information into the metaverse. To the point about being mindful of compute and the carbon footprint of everything we run in the metaverse, there are things we need to do to prep the data so it’s most effective for what we’re asking to be done there.Another topic we haven’t really discussed yet is trust. In many cases, our customers are putting their intellectual property into the metaverse. Figuring out how we leverage all of the technology we have in other areas around security, privacy rights, compliance, reputation, fraud–there’s a lot of potential to do amazing things, but we also need to make sure we’re securing all the IP that goes into these environments.The third part for me is, again, this notion of dynamic–I think Virginie mentioned this. The metaverse is not static. Things will change in the metaverse. They might need to be reflected back on the original data and vice versa. Having this real time, interactive, always up to date environment is going to unlock even more capability for collaboration when we figure out how to transfer all that data back and forth seamlessly.
Virtual Jensen Huang of Nvidia.VentureBeat: What are the next major technological leaps that need to occur to get to this next level of the metaverse?Lebaredian: Everything we’re talking about here today with the metaverse implicitly assumes that we have the capability of accurately simulating these worlds and having it match the physical world, the real one that we’re in. As of today, the world simulators that we have are largely designed for entertainment. They’re designed for games. We need to find ways of expanding these world simulators to be physically correct and take advantage of the computing power that’s available to us, using it to its maximum so we can attain these superpowers.Once you can simulate a world and do that accurately, you get a few things. If we simulate how light interacts with matter, we can see these worlds. We can see how they’re going to look. If you can take a world, ingest it from the real world and reconstruct it inside these virtual worlds, simulate it that way, we can teleport virtually to anywhere in the world. If you can run a physics simulation and see how the state of that world changes, and actually have confidence in how it evolves, you essentially get a time machine. We can go into the future and see what this world is going to look like, how it’s going to behave some time in the future. That allows us to explore many possible futures. We can change the initial state, explore all the possibilities, and choose the best future, the one we think is going to be the most efficient, the most sustainable, the safest, or whatever you’re optimizing for.Our challenge here is to get that physical accuracy, and that’s going to require a lot of computation, but it also requires a lot of cooperation between all of the tools and all of the simulators and everything that’s out there, so we can combine all of the elements of these worlds that we already have into a form that’s simulatable. If you think about it, every man-made thing around us exists in some 3D digital form somewhere. Somebody used a CAD tool for every product that you’re using, every building you’re in. Unfortunately, they diverged at some point from what was actually built, what’s in reality. The better we make these tools in the collaboration frameworks and allow this synchronization of the real world and the digital world versions of things, the sooner we’ll be able to attain those superpowers of teleportation and time travel.VentureBeat: Timoni, any closing thoughts?West: I’d just add to that. We do have the CAD files that provide references to a lot of things that exist today. But we don’t have that for things that are handmade, things that existed prior to computer-aided design. I do love that we’re leaning deeply into things like photogrammetry and object reconstruction and everything else we need to be able to bring the real world online, in a sense, and start to merge the digital and physical to a point where we can have these types of super-realistic simulations. That will help us all make better decisions, and more important, to help computers respond to us in a way that’s truly democratizing, to allow all people to use computers in a way that makes sense to them.
Quelle:
Foto:
The Omniverse Replicator for DriveImage Credit: Nvidia