Case Study
Multi-Camera 3D Reconstruction for
Real-Time Spatial Perception
Parallel monocular metric-depth pipelines for application-ready 3D perception
By Ceri Nelmes
Tech Marketing Leader & Journalist | Building Brands at the Edge of What’s Next | Connect Tech & CTai Labs
Review: Senior Marketing & Events Specialist | Content Strategy | Campaign Execution | ConnectTech.com
Key Results
- CTai LABS turns synchronized GMSL camera streams into low-latency metric depth and colorized 3D point clouds for robots, vehicles, and intelligent machines
- The demonstrated architecture processes three 1080p/30 fps monocular streams in parallel on the Gauntlet Carrier Board for NVIDIA® Jetson Thor™
- NVIDIA® TensorRT™ optimization accelerates the depth model, while ROS 2 connects the 3D output to the application
- Multiple monocular cameras do not automatically imply stereo matching or one fused persistent map; the reconstruction architecture must match the required output
- Real-time performance is an end-to-end target measured from camera exposure through application-ready 3D output under representative system load
- CTai LABS can begin with an idea or join at any point to integrate cameras, optimize a model, repair a data path, validate performance, or carry the system through deployment on Connect Tech hardware
- The customer retains its cameras, data, models, application, calibration assets, and intellectual property
CTai LABS, a department of Connect Tech, helps robotics and autonomy teams convert live camera streams into spatial information that downstream software can use. This page explains the architecture demonstrated in CTai LABS’ real-time 3D spatial intelligence workflow: multiple monocular GMSL cameras, accelerated metric-depth inference, colorized 3D point-cloud generation, ROS 2 integration, and deployment on Connect Tech hardware powered by NVIDIA Jetson Thor. CTai LABS can start with only an application goal, or enter later with an existing camera rig, dataset, trained model, ROS 2 graph, performance trace, or deployment baseline.
What CTai LABS Actually Does for Spatial Perception in 3D
The demo runs three 1080p/30 fps GMSL camera streams through parallel monocular depth-estimation pipelines on the Connect Tech Gauntlet Carrier Board for NVIDIA® Jetson Thor™ with the NVIDIA Jetson T5000 module. Each camera provides ordinary 2D RGB frames. A trained depth model is optimized with NVIDIA® TensorRT™ and executed on the Edge platform. The resulting metric-depth image is projected into a colorized 3D point cloud that can be visualized, published into ROS 2, or connected to an approved application.
The distinction matters. This demonstration does not depend on a calibrated stereo pair, estimate platform pose, or claim to maintain one persistent fused world map across all cameras. It demonstrates a scalable multi-camera compute and data path in which several monocular streams can produce 3D spatial output concurrently. If a customer needs cross-camera fusion, surround-view alignment, temporal mapping, or fusion with LiDAR, radar, IMU, GNSS, or odometry, CTai LABS can design and validate those additional layers as a separate system requirement.
| Workstream | What is integrated | Program outcome |
|---|---|---|
| Perception and model pipeline | Detection, tracking, segmentation, classification, depth, and scene understanding | Mission-relevant outputs with measured accuracy, confidence, latency, and degraded-mode behaviour. |
| Sensor and camera integration | EO/IR cameras, GMSL cameras, LiDAR, radar, and other mission sensors | Synchronized and calibrated sensor inputs with controlled interfaces into the perception stack. |
| Navigation and timing | GNSS, IMU, time sources, positioning, and geolocation data | Perception outputs aligned with vehicle state, position, orientation, and mission timing. |
| Edge compute and deployment | NVIDIA Jetson, Connect Tech carrier boards, rugged systems, storage, networking, and I/O | A bounded deployment architecture matched to compute, bandwidth, SWaP, thermal, and environmental constraints. |
| Validation and flight test | Recorded data, telemetry, performance traces, test scenarios, and acceptance criteria | Evidence that the integrated perception system performs as required under representative operating conditions. |
Figure 1. The CTai LABS demo processes multiple monocular camera streams in parallel from GMSL ingest to application-ready 3D output.
“Real-time 3D perception is not a model running in isolation. It is the complete path from synchronized camera capture through metric-depth inference and application-ready point clouds, measured while the rest of the robot is operating.”
Doruk Sonmez, AI Solutions Architect, CTai LABS
Why Monocular Metric Depth Is Useful
A monocular camera does not measure range directly. A learned metric-depth model estimates a distance value for each image pixel from a single RGB frame. The result can be transformed into a 3D point cloud when the camera model and output convention are known. This approach can add dense spatial cues without requiring a stereo baseline or an active depth sensor at every viewpoint. It also introduces model-dependent limits, including domain shift, reflective or transparent surfaces, poor lighting, motion blur, uncertain boundaries, and depth ranges that differ from the training data.
Recent peer-reviewed research shows why this field is moving quickly while still requiring system-level validation. UniDepth demonstrated cross-domain monocular metric-depth estimation from a single image, Metric3D v2 advanced zero-shot metric depth and surface-normal prediction, and Depth Anything V2 improved depth-model efficiency and generalization while supporting metric fine-tuning (Piccinelli et al., 2024; Hu et al., 2024; Yang et al., 2024). A 2025 survey identifies generalization, boundary quality, metric consistency, and deployment efficiency as continuing challenges (Zhang et al., 2025). CTai LABS therefore validates the chosen model on representative customer data and the target Edge platform rather than treating a public benchmark as deployment evidence.
Multi-Camera Does Not Automatically Mean Stereo or Fusion
A multi-camera system can be built in several ways. Cameras may run independently to provide wider coverage, be paired for stereo depth, share overlapping views for cross-camera estimation, or contribute to a fused spatial model. The correct architecture depends on the application output. The CTai LABS demo uses multiple monocular cameras and parallel metric-depth inference. A customer system may add synchronization, cross-camera calibration, overlap management, confidence weighting, motion compensation, or fusion only where those functions are required.
This boundary prevents two common errors. First, multiple cameras should not be described as stereo unless calibrated image pairs and correspondence are actually used. Second, several point clouds should not be described as one coherent world model unless they have been transformed into a shared reference frame and validated for temporal and spatial consistency. Peer-reviewed multi-camera depth research similarly treats spatial and temporal relationships as explicit parts of the method rather than automatic properties of camera count (Chen et al., 2025).
From Camera Frames to 3D Spatial Output
Camera bring-up and deterministic ingest
CTai LABS integrates the camera, serializer, deserializer, driver, device tree or sensor framework, pixel format, resolution, frame rate, trigger path, clocks, and health reporting. Connect Tech’s GMSL3 Camera Platform supports GMSL3 and GMSL2 cameras, Power over Coax, software-controlled configuration, and shared triggering on supported carrier boards. On NVIDIA Jetson Thor, GMSL streams can be exposed as separate camera devices through the supported camera framework.
Image preparation and model inputs
The production pipeline must reproduce the model’s expected color space, resize or crop policy, normalization, camera parameters, tensor layout, and output scaling. CTai LABS traces these operations from the original frame to the inference tensor so an apparent model error can be separated from a camera, format, preprocessing, or metadata error.
TensorRT inference optimization
NVIDIA TensorRT compiles trained models into optimized GPU inference engines and supports precision choices, dynamic shapes, layer fusion, CUDA graphs, and multi-stream execution. CTai LABS establishes a reference output before optimization, then measures task-level equivalence, inference latency, throughput, memory, power, thermals, and sustained behavior on the selected Connect Tech platform. The objective is not the smallest isolated inference number. It is the best complete-system result that preserves acceptable depth behavior.
Metric depth to point cloud
Each depth pixel is projected through the camera model into 3D coordinates and associated with source-image color. CTai LABS validates depth units, invalid-pixel handling, clipping range, confidence thresholds, coordinate conventions, density, output rate, and the memory cost of point-cloud generation. If the application needs a different representation, the output can be transformed into approved obstacle, occupancy, mesh, measurement, or structured spatial data without implying that every representation is part of the base demo.
ROS 2 and application integration
The pipeline can publish images, depth maps, camera information, diagnostics, and point clouds into ROS 2, or connect directly to another application interface. CTai LABS defines message rates, Quality of Service, timestamps, coordinate frames, queue depth, backpressure, recording, replay, failure reporting, and recovery so the consumer receives spatial data with the context needed to use it.
Why Connect Tech Hardware Is Part of the Architecture
The carrier board, camera expansion, power, cooling, storage, networking, mechanical design, software support, and service plan determine whether streams arrive reliably and whether the inference workload can run continuously. The demo uses Gauntlet because it combines the NVIDIA Jetson T5000 with a 16-lane MIPI CSI-2 expansion path, high-speed networking, NVMe expansion, and Connect Tech camera-board compatibility.
| Connect Tech platform | Role in the solution | Relevant architecture value |
|---|---|---|
| Gauntlet Carrier Board for NVIDIA Jetson Thor | Demo and custom system foundation | Jetson T5000 support, 16-lane MIPI CSI-2 expansion, high-speed networking, NVMe, and broad I/O. |
| GMSL3 Camera Platform | Multi-camera acquisition | Up to eight supported GMSL3/GMSL2 inputs, Power over Coax, shared trigger routing, monitoring, and protected camera power. |
| Anvil-T5 Edge System | Integrated deployment path | Factory-integrated Jetson Thor Edge system with active cooling, storage and network expansion, and vision-ready camera options. |
| Rogue-T5 Carrier Board | Commercially deployable compact platform | Ruggedized connectivity, multi-lane MIPI CSI-2, NVMe, networking, and Jetson T5000 compute for product integration. |
Real-Time Is an End-to-End Engineering Target
A camera pipeline can report fast inference while still delivering stale or irregular 3D output. Exposure time, sensor buffering, serialization, deserialization, driver queues, memory copies, preprocessing, inference, point-cloud construction, ROS 2 transport, visualization, and the consuming application all contribute to elapsed time. Concurrent workloads can also change the result. CTai LABS measures the complete path with the camera count, resolution, frame rate, model, output format, and other ai tasks that represent the intended system.
Figure 2. End-to-end spatial-perception performance must be measured across the complete camera-to-application chain.
- Per-camera capture latency, timestamps, frame intervals, and dropped-frame behavior
- Queue depth, buffering, stream synchronization, and recovery after camera or process interruption
- CPU, GPU, accelerator, encoder, decoder, memory, and storage utilization under sustained load
- TensorRT inference latency, depth confidence, temporal stability, and output equivalence against the reference model
- Point-cloud construction time, output rate, memory footprint, coordinate correctness, and downstream consumption
- Glass-to-glass and camera-to-application latency distributions, including worst-case behavior rather than only averages
Where This Solution Fits
Multi-camera metric-depth perception is useful where broader visual coverage or several simultaneous viewpoints provide more value than one camera can deliver. The output must still be validated against the operating environment, required range, scene dynamics, and consequence of error.
- Autonomous mobile robots and mobile manipulators that need depth cues at several sides of the platform
- Autonomous vehicles, research vehicles, and off-road machines that need concurrent surround-view spatial output
- Industrial work cells that need multi-angle geometry around parts, tools, fixtures, or robot motion
- Inspection systems that use several viewpoints to reduce occlusion and expose surface geometry
- Teleoperation and remote-assistance systems that need real-time spatial context from distributed cameras
- Spatial ai applications that require metric depth or point clouds before higher-level scene understanding and reasoning
Example Multi-Camera 3D Reconstruction Projects
Three-camera AMR surround perception
An AMR team wants metric-depth cues at the front and sides of the vehicle without adding active depth hardware at every viewpoint. CTai LABS defines the camera placement, brings up three GMSL streams on Gauntlet, optimizes the selected monocular depth model with TensorRT, publishes depth and point clouds into ROS 2, and validates range error, temporal stability, latency, dropped frames, sustained load, and recovery in representative aisles and lighting.
Multi-angle robotic inspection
An automation program needs several synchronized views around a work cell so an inspection application can reason about surfaces that one camera cannot see. CTai LABS integrates the cameras and trigger path, establishes per-camera calibration, implements parallel depth inference, transforms approved outputs into the application’s coordinate convention, and supplies repeatable test recordings, performance traces, known limits, and a deployment baseline.
Existing depth model moved to Edge deployment
A team has a research model that runs on a workstation but cannot sustain the required camera count or latency on its target system. CTai LABS profiles preprocessing, model execution, memory movement, point-cloud generation, and ROS 2 transport, builds a TensorRT engine, selects the appropriate Connect Tech Jetson platform, and verifies the optimized output against the original model and customer-defined acceptance data.
Architecture Through Deployment
- Define the required 3D output, coordinate convention, operating range, field of view, motion, lighting, failure cases, latency and accuracy budgets, downstream consumer, and acceptance evidence.
- Select cameras, lenses, interface, trigger method, Connect Tech carrier or Edge system, NVIDIA Jetson module, networking, storage, power, thermal design, enclosure, and lifecycle path.
- Bring up the Board Support Package, camera drivers, SerDes devices, streams, metadata, timestamps, calibration assets, recording, replay, health monitoring, and recovery.
- Integrate preprocessing, the reference depth model, TensorRT runtime, concurrent inference, point-cloud generation, ROS 2 or application interfaces, diagnostics, and data governance controls.
- Validate representative scenes, depth behavior, temporal stability, end-to-end latency, throughput, dropped frames, sustained load, environmental conditions, restarts, and degraded modes.
- Deliver versioned software, model and configuration lineage, build inputs, interface documentation, calibration procedure, performance evidence, deployment instructions, known limits, and engineering handoff.
What CTai LABS Can Deliver
- Camera-to-application architecture, system requirements, latency and accuracy budgets, interface ownership, risk register, and acceptance matrix.
- Connect Tech hardware and NVIDIA Jetson platform selection with camera, network, storage, power, thermal, enclosure, expansion, service, and lifecycle requirements.
- Board Support Package and driver baseline, GMSL bring-up, timing and trigger configuration, camera calibration, health monitoring, recording, replay, and recovery.
- Reference and TensorRT-optimized metric-depth pipelines with preprocessing, concurrent execution, point-cloud generation, ROS 2 integration, and approved application interfaces.
- Representative evaluation dataset, depth and temporal-stability results, equivalence evidence, latency and throughput traces, sustained-load results, failure tests, and known limits.
- Versioned release, software bill of materials, model and configuration lineage, deployment image, test procedures, interface documentation, and knowledge transfer.
Your Physical ai Integration Partner
CTai LABS combines Connect Tech production hardware, in-house Board Support Package (BSP) engineering, multi-camera integration, NVIDIA Jetson and TensorRT experience, ROS 2 software, model optimization, and physical validation. The team can begin with an idea, solve one difficult interface, or carry the complete camera-to-3D system from architecture through deployment while the customer retains its data, models, application, calibration assets, device design, and intellectual property.
Book a Demo
- The required 3D output, operating environment, target field of view, reconstruction range, motion profile, latency target, accuracy target, and downstream application.
- Current or proposed cameras, resolution, frame rate, lens, interface, trigger method, cable requirements, mounting assumptions, and additional sensors.
- Connect Tech hardware or preferred NVIDIA Jetson platform, ROS 2 distribution, model files, source code, logs, recordings, datasets, and existing performance measurements.
- Power, thermal, storage, networking, enclosure, environmental, lifecycle, service, cybersecurity, and deployment requirements.
- The evidence required for the next decision, including depth accuracy, temporal stability, throughput, end-to-end latency, dropped frames, sustained load, recovery, and application behavior.
ABOUT THE AUTHOR
Ceri Nelmes
Tech Marketing Leader & Journalist | Building Brands at the Edge of What’s Next |
Connect Tech & CTai Labs
Ceri Nelmes is Head of Marketing at Connect Tech and an experienced technology journalist and digital strategist. She covers the technologies and market shifts shaping embedded computing, Edge AI, Physical ai, robotics, autonomous systems, and the NVIDIA® ecosystem. Working with Connect Tech and CTai LABS subject-matter experts, she turns engineering developments into accurate, useful reporting for developers, technical buyers, business leaders, and the media.
Learn More
Resources and Frequently Asked Questions
Related
Sources
Chen, Z., Zhao, H., Hao, X., Yuan, B., & Li, X. (2025). STViT+: Improving self-supervised multi-camera depth estimation with spatial-temporal context and adversarial geometry regularization. Applied Intelligence, 55, 328.
https://doi.org/10.1007/s10489-024-06191-6Connect Tech. (2025). Gauntlet Carrier Board for NVIDIA Jetson Thor. Retrieved August 26, 2026, from
https://connecttech.com/product/gauntlet-carrier-board-for-nvidia-jetson-thor/Connect Tech. (2025). GMSL3 Camera Platform for NVIDIA Jetson AGX Orin and Jetson Thor T5000. Retrieved August 26, 2026, from
https://connecttech.com/product/gmsl3-camera-platform-nvidia-jetson-thor/Connect Tech. (2025). GMSL Vision for Real-Time 3D Spatial Intelligence demo description. Internal demo poster.
Hu, M., Yin, W., Zhang, C., Cai, Z., Long, X., Chen, H., Wang, K., Yu, G., Shen, C., & Shen, S. (2024). Metric3D v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence.
https://ieeexplore.ieee.org/document/10638254/NVIDIA. (2026). GMSL Camera Development Guide.
https://docs.nvidia.com/jetson/archives/r39.2/DeveloperGuide/SD/CameraDevelopment/SIPLFramework/SIPL-for-L4T/GMSL/GMSLCameraDevelopment.htmlNVIDIA. (2026). NVIDIA TensorRT Documentation.
https://docs.nvidia.com/deeplearning/tensorrt/latest/Piccinelli, L., Yang, Y.-H., Sakaridis, C., Segu, M., Li, S., Van Gool, L., & Yu, F. (2024). UniDepth: Universal monocular metric depth estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10106-10116.
https://openaccess.thecvf.com/content/CVPR2024/html/Piccinelli_UniDepth_Universal_Monocular_Metric_Depth_Estimation_CVPR_2024_paper.htmlYang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., & Zhao, H. (2024). Depth Anything V2. Advances in Neural Information Processing Systems, 37.
https://doi.org/10.52202/079017-0688Zhang, J., Wu, Y., & Jiang, H. (2025). Survey on monocular metric depth estimation. Computers, 14(11), 502.
https://doi.org/10.3390/computers14110502Frequently Asked Questions
What is the difference between monocular depth, stereo depth, and multi-camera 3D reconstruction?
No. The current demo runs three monocular GMSL streams in parallel and produces metric depth and colorized point-cloud output. A shared persistent map, cross-camera fusion, pose estimation, or additional sensor fusion can be engineered when the customer application requires it.
Does the CTai LABS demo fuse all three cameras into one persistent map?
No. The current demo runs three monocular GMSL streams in parallel and produces metric depth and colorized point-cloud output. A shared persistent map, cross-camera fusion, pose estimation, or additional sensor fusion can be engineered when the customer application requires it.
Can CTai LABS build a system with more than three cameras?
Synchronization depends on how the outputs are used. Independent streams can tolerate different timing than a system that compares or fuses simultaneous observations. Moving robots, vehicles, and objects make timestamp error more consequential. CTai LABS defines and validates the timing requirement for the intended application.
Do the cameras need to be synchronized?
Synchronization depends on how the outputs are used. Independent streams can tolerate different timing than a system that compares or fuses simultaneous observations. Moving robots, vehicles, and objects make timestamp error more consequential. CTai LABS defines and validates the timing requirement for the intended application.
Does multi-camera 3D reconstruction require LiDAR?
No. Monocular, stereo, or multi-view camera methods can produce depth and 3D structure. LiDAR can be added when the application benefits from direct range measurements, longer-distance performance, a complementary failure mode, or a ground-truth reference for evaluation.
Is NVIDIA Jetson Thor required?
No. The correct NVIDIA Jetson module depends on camera count, resolution, frame rate, model, point-cloud workload, output rate, and the other ai processes running concurrently. Jetson Thor is relevant when the camera and spatial workloads require its compute, memory, and I/O envelope. CTai LABS also deploys on supported Jetson Orin platforms.
Can CTai LABS integrate GMSL2 and GMSL3 cameras?
CTai LABS measures the complete path from camera exposure through application-ready output. Validation can include depth error, temporal stability, frame timing, dropped frames, queue depth, inference latency, point-cloud construction, memory and compute utilization, sustained load, failure recovery, and downstream application behavior.
How is real-time 3D reconstruction validated?
CTai LABS measures the complete path from camera exposure through application-ready output. Validation can include depth error, temporal stability, frame timing, dropped frames, queue depth, inference latency, point-cloud construction, memory and compute utilization, sustained load, failure recovery, and downstream application behavior.
Can CTai LABS start before cameras or a depth model are selected?
Yes. CTai LABS can begin from the intended 3D output, environment, range, field of view, motion, latency and accuracy targets, deployment constraints, and program milestone, then define the camera, model, Connect Tech hardware, software, integration, and validation path.
Ready to Build Smarter?
Let’s create the intelligent Edge AI solution that moves your business forward.