Built with NVIDIA
Omniverse & Isaac Sim
Robotics simulation, OpenUSD scene development, synthetic data generation, digital twins, and sim-to-real validation with NVIDIA Omniverse and Isaac Sim.
By Doruk Sönmez, M.Sc.
AI Solutions Architect, CTai LABS/Connect Tech Inc.
Technical review: Rob Callaghan, P.Eng., Chief Product Officer, Connect Tech Inc.
Key Takeaways
- NVIDIA® Omniverse™ is a collection of accelerated libraries, microservices, SDKs, and APIs built on OpenUSD for developing physical ai simulation applications and agentic workflows (NVIDIA, 2026a)
- NVIDIA Isaac Sim™ 6.0.1 is an open-source robotics simulation framework built on Omniverse libraries and OpenUSD, with GPU-accelerated physics, RTX sensor simulation, ROS 2 connectivity, and synthetic data generation (NVIDIA, 2026b)
- Simulation can reduce the cost and risk of collecting physical data or testing every scenario on hardware, but the reality gap remains. Domain randomization, sensor modeling, calibration, and real-world verification are still required for reliable transfer (Muratore et al., 2022)
- CTai LABS builds simulation scenes, converts robot and CAD assets into OpenUSD, configures physics and sensors, develops Replicator data generation, connects ROS 2 software, and verifies virtual results against the physical system
- The service is designed for robotics, industrial automation, inspection, material handling, and other Physical ai programs where teams need a repeatable virtual test environment rather than a one-off visualization
CTai LABS provides NVIDIA® Omniverse™ and Isaac Sim™ engineering from asset preparation through virtual testing, synthetic dataset creation, ROS 2 connectivity, and physical validation. Across those stages, CTai LABS serves as Your Physical ai Integration Partner, connecting the virtual robot to the sensors, software, compute platform, and real machine the customer intends to deploy. The customer retains its robot design, software, datasets, product ownership, and IP.
NVIDIA Omniverse provides the accelerated 3D and simulation technologies used to build physical ai applications around OpenUSD. NVIDIA currently describes Omniverse as a collection of libraries and microservices for simulation, RTX rendering and sensor modeling, physics, OpenUSD data exchange, and agentic workflows (NVIDIA, 2026a).
NVIDIA Isaac Sim applies those capabilities to robotics. The current 6.0.1 release is open source and supports OpenUSD-native scenes, RTX and physics-based sensors, ROS 2 connectivity, PhysX and experimental Newton physics backends, synthetic data generation, and extensible Python or Kit-based development (NVIDIA, 2026b).
The value is not a photorealistic robot demo. A useful virtual testbed has to represent the robot, sensors, geometry, materials, timing, control interfaces, and operating conditions closely enough to answer an engineering question. Research on sim-to-real transfer consistently identifies differences between simulated and physical dynamics, sensing, contact, and environments as the core reality gap, which is why randomization and real-world verification remain part of the process (Muratore et al., 2022).
Simulation should answer a deployment question
A simulator becomes valuable when it replaces or accelerates a physical engineering activity. Teams may need to test navigation before a facility is available, generate labeled camera data for rare conditions, exercise a manipulation policy across thousands of variations, validate ROS 2 behavior without occupying the robot, or compare sensor placement before hardware is frozen.
Different goals require different levels of fidelity. A synthetic perception dataset depends heavily on camera and rendering assumptions. Contact-rich manipulation depends on collision geometry, friction, mass, actuator behavior, and control timing. A facility digital twin depends on geometry, coordinate systems, asset semantics, and a disciplined method for keeping the virtual representation aligned with reality. Reviews of robotics digital twins and synthetic-data systems emphasize the same point: useful virtual models depend on purpose-specific fidelity and traceable data, not visual similarity alone (Mazumder et al., 2023; Schieber et al., 2024).
CTai LABS defines the virtual test around the acceptance criteria first, then selects the Omniverse and Isaac Sim capabilities needed to produce evidence for that decision.
Figure 1. CTai LABS turns physical requirements and engineering assets into a repeatable Isaac Sim environment, then compares the virtual behavior with representative hardware before the results are used for deployment decisions.
What CTai LABS Omniverse & Isaac Sim services can cover
OpenUSD scene and asset development
OpenUSD is the native foundation for Omniverse and Isaac Sim. It supports composed 3D scenes, references, variants, metadata, transforms, materials, physics schemas, and non-destructive edits across complex asset hierarchies (NVIDIA, 2026a).
Engineering can include conversion or preparation of CAD, URDF, MJCF, and other assets for OpenUSD-based simulation; scene organization; scale and coordinate correction; collisions and physical properties; and reusable robot, sensor, and world assets. The goal is a scene representation that can be versioned and automated, not a manually assembled file that only works on one workstation.
Isaac Sim robot and environment configuration
Isaac Sim provides the robot-specific environment for assembling OpenUSD assets, applying physics, adding actuators and sensors, running scripts, and connecting the virtual robot to external software. Version 6.0 adds broader physics-backend support and updated RTX sensor scheduling, while 6.0.1 is NVIDIA’s current general-availability release as of June 2026 (NVIDIA, 2026b).
Engineering may include articulation setup, joint limits, mass and inertia, collision meshes, materials, friction, gravity, controller interfaces, scene lighting, dynamic objects, and repeatable reset conditions. Parameters are selected from available physical data and adjusted only with a documented reason.
ROS 2 and robotics-software connectivity
Isaac Sim can connect to ROS 2 through its bridge and simulation-control interfaces. Current documentation supports programmatic stepping, state control, entity management, ROS 2 topics, and launch workflows that allow existing robot software to interact with the virtual scene (NVIDIA, 2026d).
The customer’s ROS 2 graph can be connected to simulation time, message rates, frames, perception and control nodes, and automated tests using the same interfaces expected on the physical robot. The virtual testbed can therefore become part of continuous engineering rather than a separate demonstration tool.
Synthetic data generation with Replicator
Isaac Sim Replicator can generate labeled synthetic datasets by varying scene content and capturing sensor outputs together with annotations. Current workflows support configurable environments, randomized physics, cameras, render products, and writers for offline dataset generation (NVIDIA, 2026c).
Dataset engineering may define the schema, asset distribution, camera poses, lighting, materials, object placement, occlusion, motion, and randomization ranges needed for the training or evaluation objective. Synthetic data is most useful when it expands conditions that are expensive, rare, dangerous, or poorly labeled in the physical dataset.
A systematic review of indoor synthetic-data generation found that graphics-based approaches provide control over scene variation and ground-truth labels, while realism, domain mismatch, and dataset design remain significant concerns (Schieber et al., 2024). Those concerns are addressed through representative parameter ranges, held-out physical data, and task-level evaluation rather than by assuming photorealism guarantees transfer.
Domain randomization and sim-to-real transfer
No simulator reproduces every property of a physical robot. Domain randomization deliberately varies uncertain parameters so learned policies or perception models experience a family of virtual conditions instead of one idealized world. Parameters may include friction, mass, lighting, textures, camera pose, sensor noise, latency, object position, or dynamics.
Muratore et al. (2022) describe randomization as a central sim-to-real technique while also cautioning that simulator bias and parameter selection affect transfer. Uncertain physical quantities can be mapped to defensible distributions, automated experiments, and real-platform checks that show whether the resulting model or policy generalizes.
RTX sensor simulation and perception testing
Robotics perception may depend on RGB cameras, depth, LiDAR, radar, or other sensors whose output changes with geometry, materials, lighting, motion, and noise. Omniverse RTX technologies and Isaac Sim sensor models provide controllable virtual measurements for testing perception software before every physical sensor configuration is available (NVIDIA, 2026a; NVIDIA, 2026b).
Sensor work can cover placement, field of view, resolution, frame rate, timing, semantic labels, and scene variation, followed by comparison against representative physical captures. The comparison helps determine where additional calibration or model adjustment is needed.
Robot learning and Isaac Lab handoff
Isaac Sim can serve as the environment for robot learning workflows, including reinforcement learning through NVIDIA Isaac Lab. Large numbers of virtual testbeds can be used to train or evaluate policies before selected behaviors are transferred to hardware.
Robot-learning preparation may cover the robot and scene representation, customer-defined observations and actions, sensor models, randomization, and repeatable evaluation scenarios. The customer’s learning algorithm and task logic remain customer-owned.
Digital twins for industrial and robotics programs
A digital twin is more than a 3D model when it is used to represent a physical asset or process for engineering, monitoring, commissioning, or decision support. Robotics literature describes digital twins as lifecycle tools spanning design, simulation, operation, and maintenance, with data exchange between the physical and virtual representations central to their usefulness (Mazumder et al., 2023).
OpenUSD scenes can be structured around reusable assets, coordinate systems, robot cells, conveyors, workpieces, sensors, and facility geometry, with selected operational data connected where the use case requires it. For industrial automation, the same environment can support layout review, robot reach studies, software testing, sensor evaluation, or virtual commissioning activities. Broader Industry 5.0 research also highlights digital twins as a way to connect collaborative robotics, simulation, and human-centered manufacturing workflows (Zafar et al., 2024).
Automated scenario testing and regression
Repeatability is one of simulation’s strongest advantages. A physical test may be difficult to reproduce exactly, while a virtual scenario can be reset, seeded, varied, and executed hundreds of times with machine-readable results.
Scripted scenarios may cover navigation, manipulation, perception, sensor loss, obstruction, object variation, or other customer-defined cases. Results can feed a regression suite so changes to ROS 2 nodes, models, control parameters, or robot assets are tested against the same scenario set before hardware trials.
Compute and deployment correlation
Simulation hardware and robot hardware do not have to be identical, but the transfer assumptions must be explicit. Training or rendering may run on larger NVIDIA GPUs while the deployed application runs on Jetson, IGX, or another Edge target. Timing, model size, sensor rates, and runtime dependencies still have to be measured on the physical compute platform.
The CTI EdgeAI Stack becomes relevant at the handoff point, where the simulated software and models have to operate with the actual sensor interfaces, Edge compute, BSP, networking, power, and thermal limits. Virtual test evidence can then be tied to the hardware configuration intended to ship.
Figure 2. Sim-to-real confidence comes from correlation across geometry, physics, sensors, timing, software, and expected variation. Physical measurements remain the reference used to decide whether the virtual testbed is accurate enough for the intended engineering task.
Simulation, synthetic data, and digital twins solve different problems
| Need | Primary virtual capability | Engineering question |
|---|---|---|
| Robot software testing | Isaac Sim + ROS 2 | Can perception, planning, and control software execute representative scenarios before hardware time is available? |
| Synthetic perception data | Isaac Sim Replicator | Which scene variations and labels are needed to train or evaluate the model against physical data? |
| Robot learning | Isaac Sim + Isaac Lab | Can policies be trained across enough variation to transfer to the real robot with measured performance? |
| Industrial digital twin | Omniverse + OpenUSD + Isaac Sim | Which facility, robot, sensor, and operational data must stay aligned for the engineering use case? |
| Virtual validation | Automated scenarios and metrics | Which failures or acceptance cases can be exercised repeatedly before physical testing? |
A digital twin is not automatically a validation model
A richly detailed scene can still be unsuitable for engineering decisions if its geometry, physical parameters, sensors, or timing are wrong for the question being asked. Reviews of digital twins in robotics distinguish visual representation from models used for simulation, prediction, and lifecycle decision support (Mazumder et al., 2023).
CTai LABS defines the required fidelity around the use case and records which parameters come from CAD, device specifications, calibration, measurement, estimation, or deliberate randomization. Unknowns remain visible rather than being hidden behind visual realism.
Typical Omniverse & Isaac Sim engagements
- Converting CAD, URDF, MJCF, or existing robot assets into reusable OpenUSD scenes
- Building an Isaac Sim environment for a robot, workcell, warehouse, inspection station, or material-handling application
- Connecting an existing ROS 2 graph to virtual sensors, robot state, and simulation time
- Generating labeled synthetic camera or depth data with Replicator for perception development
- Randomizing lighting, materials, object placement, sensor noise, friction, mass, or other uncertain parameters for sim-to-real studies
- Creating RTX camera, depth, LiDAR, or other sensor configurations before hardware placement is finalized
- Preparing robot-learning environments and evaluation scenarios for Isaac Lab workflows
- Building an industrial digital twin around OpenUSD assets, robot cells, facility geometry, and selected operational data
- Automating regression tests for navigation, manipulation, perception, or fault scenarios
- Correlating virtual timing, sensing, or motion against the physical robot before using simulation results for deployment decisions
How a CTai LABS Omniverse & Isaac Sim engagement works
1.
Define the engineering question
Identify what the virtual testbed has to prove or accelerate: software testing, sensor evaluation, synthetic data, robot learning, layout, digital-twin use, or another measurable objective.
2.
Inventory physical and digital assets
Collect CAD, URDF or MJCF files, robot specifications, sensor data, coordinate systems, ROS 2 interfaces, calibration, facility geometry, models, and any measurements available from the real system.
3.
Build the OpenUSD scene
Prepare reusable assets, coordinate frames, physics properties, materials, collisions, scene geometry, and versioned scene composition.
4.
Connect robot software and sensors
Configure the virtual robot, sensor models, ROS 2 interfaces, timing, controllers, Replicator components, or learning interfaces required by the use case.
5.
Create scenarios and variation
Define scripted tests, randomization ranges, dataset distributions, or operating cases that exercise the conditions the customer needs to understand.
6.
Correlate with physical measurements
Compare selected virtual outputs with real sensor captures, timing, motion, contact, or application results. Adjust parameters where measured evidence supports the change.
7.
Handoff and extend
Document the OpenUSD assets, scripts, dependencies, scenario definitions, assumptions, metrics, and known gaps. The customer retains its simulation assets, robot software, data, product ownership, and IP.
A useful simulation environment becomes an engineering asset when its results can inform decisions about the physical system. CTai LABS connects OpenUSD assets, Isaac Sim, synthetic data, ROS 2 software, sensors, and real-world measurements so teams can test more scenarios virtually while maintaining a clear connection to the robot or machine they intend to deploy. The customer receives reusable simulation assets, documented assumptions, and a virtual test environment designed to evolve alongside the physical application.
Book a Demo
Bring the video analytics requirement, camera or sensor architecture, existing application or DeepStream setup if one exists, ai models, expected stream count, target NVIDIA compute platform, and the performance problem the team is trying to solve. CTai LABS will assess the full video flow, identify the Metropolis components and specialist work required, and scope the route to a validated release.
ABOUT THE AUTHOR
Doruk Sönmez, M.Sc.
AI Solutions Architect, CTai LABS
Doruk is an AI Solutions Architect at CTai LABS, the Physical AI and Edge AI services division of Connect Tech Inc., an NVIDIA Elite Partner. An NVIDIA DLI Certified Instructor, he specializes in deploying vision-language models, agentic AI workflows, and accelerated video pipelines on NVIDIA Jetson platforms.
Learn More
Resources and Frequently Asked Questions
Related
Sources
Mazumder, A., Sahed, M. F., Tasneem, Z., Das, P., Badal, F. R., Ali, M. F., Ahamed, M. H., Abhi, S. H., Sarker, S. K., Das, S. K., & Hasan, M. M. (2023). Towards next generation digital twin in robotics: Trends, scopes, challenges, and future. Heliyon, 9(2), e13359.
https://doi.org/10.1016/j.heliyon.2023.e13359Muratore, F., Ramos, F., Turk, G., Yu, W., Gienger, M., & Peters, J. (2022). Robot learning from randomized simulations: A review. Frontiers in Robotics and AI, 9, 799893.
https://doi.org/10.3389/frobt.2022.799893NVIDIA. (2026a). NVIDIA Omniverse.
https://docs.nvidia.com/omniverse/index.htmlNVIDIA. (2026b). NVIDIA Isaac Sim 6.0.1 documentation.
https://docs.isaacsim.omniverse.nvidia.com/latest/NVIDIA. (2026c). Scene-based synthetic dataset generation. NVIDIA Isaac Sim documentation.
https://docs.isaacsim.omniverse.nvidia.com/latest/replicator_tutorials/tutorial_replicator_scene_based_sdg.htmlNVIDIA. (2026d). ROS 2 simulation control. NVIDIA Isaac Sim documentation.
https://docs.isaacsim.omniverse.nvidia.com/latest/ros2_tutorials/tutorial_ros2_simulation_control.htmlSchieber, H., Demir, K. C., Kleinbeck, C., Yang, S. H., & Roth, D. (2024). Indoor synthetic data generation: A systematic review. Computer Vision and Image Understanding, 240, 103907.
https://doi.org/10.1016/j.cviu.2023.103907Zafar, M. H., Langås, E. F., & Sanfilippo, F. (2024). Exploring the synergies between collaborative robotics, digital twins, augmentation, and Industry 5.0 for smart manufacturing: A state-of-the-art review. Robotics and Computer-Integrated Manufacturing, 89, 102769.
https://doi.org/10.1016/j.rcim.2024.102769Frequently Asked Questions
What is NVIDIA Omniverse?
NVIDIA Omniverse is a collection of accelerated libraries, microservices, SDKs, and APIs built on OpenUSD for physical ai simulation applications, RTX rendering and sensor simulation, physics, data exchange, and agentic workflows (NVIDIA, 2026a).
What is NVIDIA Isaac Sim?
NVIDIA Isaac Sim is an open-source robotics simulation framework built on Omniverse libraries and OpenUSD. It provides physics, RTX and physics-based sensor simulation, robot models, ROS 2 connectivity, synthetic data generation, and extensible Python or Kit-based development (NVIDIA, 2026b).
Can CTai LABS import our robot CAD or URDF into Isaac Sim?
Yes. The work can include CAD, URDF, MJCF, and related asset preparation, coordinate and scale correction, collisions, mass and inertia, materials, joints, sensors, and reusable OpenUSD scene structure.
Can Isaac Sim connect to our existing ROS 2 application?
Yes. Isaac Sim includes ROS 2 bridges and simulation-control interfaces. Topics, transforms, clocks, sensors, robot state, services, and automated tests can be connected to the customer’s existing ROS 2 software (NVIDIA, 2026d).
Can CTai LABS generate synthetic training data?
Yes. Isaac Sim Replicator can generate sensor outputs and annotations while scenes are varied programmatically. Scene distribution, labels, randomization, cameras, lighting, objects, and the validation method can be designed around the model’s data requirement (NVIDIA, 2026c).
Does synthetic data eliminate the need for real data?
No. Synthetic data can extend coverage and provide labels for difficult conditions, but domain mismatch still has to be measured. Real-world data remains important for calibration, held-out evaluation, and sim-to-real verification (Schieber et al., 2024).
What is domain randomization?
Domain randomization varies uncertain simulation parameters such as lighting, textures, friction, mass, sensor noise, or object placement so models or policies are not trained against one idealized environment. It is a widely studied technique for improving sim-to-real robustness (Muratore et al., 2022).
Can CTai LABS build an industrial digital twin?
Yes, when the use case is defined. The scope can include OpenUSD asset structure, facility geometry, robot cells, sensors, operational data interfaces, simulation scenarios, and validation. The required fidelity depends on whether the twin supports design, testing, commissioning, monitoring, or another engineering objective.
Does the customer keep its simulation assets and IP?
Yes. CTai LABS works as an extension of the customer’s engineering team. The customer retains its robot design, OpenUSD assets created for the engagement, software, datasets, models, product ownership, and intellectual property.
Ready to Build Smarter?
Let’s create the intelligent Edge AI solution that moves your business forward.