CTai LABS Insights
Case Study: Physical ai Evaluation Kit | Real2Sim Digital Twin
Closing the Real2Sim-Sim2Real Gap at the Edge
By Doruk Sönmez, M.Sc.
AI Solutions Architect, CTai LABS, a department of Connect Tech Inc. NVIDIA DLI Certified Instructor
Review: Kara Price, Senior Marketing & Events Specialist, Connect Tech Inc, ConnectTech.com
Key Results
- Why Choose CTai LABS’ Physical ai Evaluation Kit: This kit gives robotics and autonomous systems teams a working environment for evaluating Real2Sim-Sim2Real workflows on real Edge hardware, connecting live sensor data, simulation, foundation models, and Physical ai deployment in one local pipeline
- Real-time Real2Sim projection: NVIDIA® Jetson T5000 runs Vision Transformers-based detection and 3D Body Pose Estimation via DeepStream 9.1, streaming low-latency data into a custom NVIDIA Isaac™ Sim Omniverse extension on NVIDIA DGX Spark to bridge the physical and digital worlds in real time
- Autonomous manipulation and scene understanding: Jetson T5000 serves Isaac GR00T N1.7 to a manipulator-based AMR, while DGX Spark uses Scene Analyzer Agent to enable environmental awareness using Cosmos 3 Edge and live gaussian splatting-based simulation data
- Local Processing: The entire pipeline runs locally with no cloud dependency, supporting fast, reliable, and secure operation
- Smarter robots: Enables more intelligent decision-making in dynamic environments and interaction with the world
- ROS2 compatible: Easily connects with hardware, sensors, and standard robotics development tools for faster deployment
What Is the Physical ai Evaluation Kit?
Physical ai development requires teams to move continuously between the physical world and simulation: bringing real-world sensor data into a digital environment for testing, then transferring validated models and policies back onto physical systems (Han et al., 2026). CTai LABS’ Physical ai Evaluation Kit demonstrates that Real2Sim-Sim2Real loop as a working, local pipeline across NVIDIA’s three-computer architecture: Edge capture on NVIDIA Jetson Thorâ„¢, simulation on NVIDIA DGX Spark, and open foundation models trained using NVIDIA DGX Systems (Connect Tech, 2026b).
The kit brings together digital twin environments built with Gaussian Splatting and NVIDIA Omniverse, Cosmos 3 Edge, and DeepStream 9.1-based Metropolis workflows across the three-computer architecture. Together, these technologies create a real-time projection layer connecting digital twins, the physical world, agentic ai, and robotics at the Edge. Rather than treating simulation and deployment as separate stages, the kit closes the loop: the physical world gets realized inside the simulation, and the simulation improves what runs back in the physical world.
How the Real2Sim-Sim2Real Pipeline Works
- Multi-View 3D Tracking at the Edge. Real-world data is captured by GMSL, MIPI, Network, or USB cameras connected to the Connect Tech Anvil-T5 Edge System, powered by NVIDIA Jetson T5000 (Connect Tech, 2026a). The Edge system tracks 3D body poses and Real-Time Location System (RTLS) data from multiple cameras in real time using NVIDIA Metropolis workflows, executing Vision Transformer-based detection and 3D body-pose estimation through DeepStream 9.1 (NVIDIA, 2026a).
- Real2Sim streaming to the digital twin. This spatial vision analytics data streams locally to an NVIDIA DGX Spark desktop supercomputer, where NVIDIA Isaac Sim renders a live digital twin of the physical scene through a custom NVIDIA Omniverse extension, populating it with digital human representations derived directly from the Edge sensor data.
- Scene understanding inside the twin. Inside the simulation, CTai LABS’ Scene Analyzer Agent, built on the NVIDIA AI Blueprint for Video Search and Summarization (VSS) (NVIDIA, 2026c), connects to a virtual camera that can be repositioned on demand within the digital twin. The agent uses NVIDIA Cosmos 3 Edge, a world model optimized for Edge deployment, to interpret the scene and identify events of interest (NVIDIA, 2026b). NVIDIA Nemotron LLM then summarizes those findings and stores them in a local database with spatial-temporal relationships, making the event history queryable.
- Autonomous manipulation. Jetson T5000 serves Isaac GR00T N1.7 to a manipulator-based autonomous mobile robot running on Jetson Orinâ„¢ NX, enabling real-world manipulation informed by the same reasoning pipeline validated in simulation.
- Sim2Real redeployment. When model policies are validated in simulation, they are redeployed to the Anvil-T5 with Jetson T5000 for real-world Physical ai deployment, closing the Real2Sim-Sim2Real loop end to end.
Figure 1 : Three compute layers architecture showing the data flow between Edge, Simulation/Digital Twin, and Foundation Models, with sim-to-real feedback and model updates linking the layers.
What This Kit Accomplishes
This demonstration inverts the usual simulation workflow. Instead of a digital twin built once from static assets, the twin here is continuously updated from real-world camera feeds, so the simulation reflects what is actually happening in the physical environment at low latency. Instead of requiring development teams to integrate vision analytics, simulation, model inference, and deployment workflows independently, CTai LABS brings those components together as a working local pipeline. The result is an evaluation environment where teams can explore how live physical data, digital twins, foundation models, and Edge deployment interact before building that architecture into their own production system.
The kit also demonstrates the CTI EdgeAI Stack in practice, bringing Edge compute, sensor data, accelerated ai software, simulation, models, and deployment workflows into one working evaluation environment.
The entire workflow runs locally, with no cloud dependency. Jetson T5000 on the Anvil-T5 System handles real-time Edge perception and processing, while NVIDIA DGX Spark supports simulation, scene understanding, and event logging. Together, the systems keep the Real2Sim-Sim2Real workflow local while distributing workloads according to their compute requirements.
Benefits for Customers
Real-time Real2Sim projection
Jetson T5000 executes Vision Transformers and 3D body pose estimation via DeepStream 9.1, streaming low-latency data directly into a custom Isaac Sim extension on DGX Spark to bridge the physical and digital worlds in real time.
Autonomous manipulation and scene understanding
Jetson T5000 serves Isaac GR00T N1.7 to a manipulator-based AMR for real-world manipulation, while DGX Spark generates scene descriptions via Cosmos 3 Edge and live simulation data.
Local Processing
The pipeline runs locally without cloud dependency, supporting fast, reliable, and secure operation.
Reduced Real2Sim-Sim2Real friction
Because the digital twin is continuously grounded in live sensor data, teams can validate model policies against a simulation that more closely reflects current real-world conditions before redeployment.
Integrated, hardware-ready pipeline
By combining Connect Tech’s Anvil-T5 System with NVIDIA’s Physical ai stack, the kit provides an integrated environment for evaluating the path between ai development, simulation, and real-world deployment.
Applications of the Physical ai Evaluation Kit
- Robotics development and validation. Teams building autonomous mobile robots, humanoids, drones, or manipulators can evaluate policies in simulation before transferring them to physical hardware, supporting comparisons between simulated and real-world performance (Kadian et al., 2020).
- Industrial and facility monitoring. The same Multi-View 3D Tracking and scene-understanding pipeline supports real-time monitoring of people and assets in an industrial setting or facility, with a queryable event history available for review.
- Autonomous systems in connectivity-limited environments. Because the full pipeline, from Edge capture through simulation to model redeployment, runs locally, it is suited to robotics and monitoring workloads in remote or bandwidth-constrained sites.
- ai research and foundation-model evaluation. Research teams can use the kit as a testbed for evaluating VLMs, VLAs, and WAMs such as Cosmos 3 Edge against live, spatio-temporally grounded scenes rather than static benchmarks.
Ready to Close the Real2Sim-Sim2Real Loop?
The Physical ai Evaluation Kit combines Connect Tech’s production-proven Anvil-T5 System with NVIDIA’s Physical ai stack, giving engineering teams an integrated, hardware-ready environment for evaluating robotics, industrial monitoring, and autonomous systems workflows. Bring your robotics or autonomous systems use case to CTai LABS, Your Physical ai Integration Partner. Our team can help you evaluate the hardware, sensor architecture, simulation environment, models, and Edge deployment requirements behind a Real2Sim-Sim2Real workflow, then integrate those layers into a system built around your application.
ABOUT THE AUTHOR
Doruk Sönmez, M.Sc.
AI Solutions Architect, CTai LABS
Doruk is an AI Solutions Architect at CTai LABS, the Physical AI and Edge AI services division of Connect Tech Inc., an NVIDIA Elite Partner. An NVIDIA DLI Certified Instructor, he specializes in deploying vision-language models, agentic AI workflows, and accelerated video pipelines on NVIDIA Jetson platforms.
Learn More
Sources and Frequently Asked Questions
Sources
Connect Tech. (2026a). Anvil-T5 Edge System with NVIDIA Jetson Thor.
https://connecttech.com/product/anvil-t5-edge-system-with-nvidia-jetson-thor/Connect Tech. (2026b, March 16). Connect Tech and CTai LABS demonstrate a deployable Physical AI workflow running live: From real-world vision analytics to Digital Twin and back.
https://connecttech.com/connect-tech-ctai-labs-demo-deployable-physical-ai-workflow/Han, X., Yu, J., Liu, M., Chen, Y., Lyu, X., Tian, Y., Wang, B., Zhang, W., & Pang, J. (2026). RE3SIM: Generating high-fidelity simulation data via 3D-photorealistic Real-to-Sim for robotic manipulation. 2026 IEEE International Conference on Robotics and Automation (ICRA).
Kadian, A., Truong, J., Gokaslan, A., Clegg, A., Wijmans, E., Lee, S., Savva, M., Chernova, S., & Batra, D. (2020). Sim2Real predictivity: Does evaluation in simulation predict real-world performance? IEEE Robotics and Automation Letters, 5(4), 6670–6677.
NVIDIA. (2026a). DeepStream SDK 9.1 for NVIDIA dGPU/X86 and Jetson.
https://docs.nvidia.com/metropolis/deepstream/9.1/text/DS_Release_notes.htmlNVIDIA. (2026b, July 15). Japan’s robotics and manufacturing leaders build on NVIDIA Cosmos to advance Physical AI frontier. NVIDIA Newsroom.
https://nvidianews.nvidia.com/news/japans-robotics-and-manufacturing-leaders-build-on-nvidia-cosmos-to-advance-physical-ai-frontierNVIDIA. (2026c). NVIDIA AI Blueprint for Video Search and Summarization.
https://build.nvidia.com/nvidia/video-search-and-summarization/blueprintcardReady to Build Smarter?
Let’s create the intelligent Edge AI solution that moves your business forward.