We’re at AutoSens Europe! Stop by booth #105 or see our events page to schedule a meeting,
When we talk with customers about SPAD sensors, they’re often most familiar with time-of-flight imaging. It’s one of the most established uses of the technology, and many of us already carry SPAD-based depth sensors in our smartphones. More broadly, time-of-flight is one example of active imaging, where image acquisition is coordinated with a controlled light source rather than relying only on light already present in the scene.
At Ubicept, our main focus is passive imaging. Ubicept Photon Fusion (UPF) and our Flexible Light Acquisition and Representation Engine (FLARE) use the sparse, extremely high-speed output of SPAD sensors to produce images in challenging conditions involving low light, fast motion, and high dynamic range. You can learn more about these technologies here.
But adding powerful passive imaging capabilities doesn’t take away what SPAD sensors already do well. On supported cameras, FLARE can also coordinate acquisition with external illumination, allowing the same imaging system to use techniques based on laser pulses, projected patterns, directional lighting, and other controlled sources. Let’s look at a few examples.
Time-of-flight uses the precise timing capabilities of SPAD sensors to measure distance. A light source emits a short pulse, the sensor detects returning photons, and the elapsed time provides information about how far the light traveled. Repeating those measurements across the scene produces a depth image.
FLARE can support time-of-flight operation on compatible SPAD cameras, allowing the same imaging sensor used for passive capture to produce depth measurements when needed. Because the intensity and depth measurements come from the same sensor and optical path, this can also simplify alignment compared with systems that use separate cameras, particularly when the camera or scene is moving.
Gated imaging uses the same precise timing capabilities in a different way. Rather than estimating distance from the arrival time of returning photons, the sensor captures photons only during a selected interval after an illumination pulse. Since reflections from nearby objects arrive before reflections from farther away, the timing of that interval determines which range of distances the camera captures.
This can be useful when imaging through fog, rain, smoke, or snow. Light scattered from particles close to the camera can reduce the contrast of objects farther away. Delaying the capture window can reject some of that nearby backscatter while preserving light returning from the distance of interest.
FLARE can configure these timing windows on supported SPAD cameras and synchronize them with the illumination source.
Not every active imaging technique requires the nanosecond-level timing precision that SPAD sensors can offer. Some work perfectly well with conventional CMOS cameras. In those cases, the advantage of SPADs is less about precisely measuring when a photon arrived and more about how quickly the sensor can acquire data.
Photometric stereo, for example, estimates surface shape by illuminating an object from different directions and comparing the resulting images. With a conventional camera, those lighting conditions are typically cycled in sync with image capture, so a 120 fps or 240 fps camera limits how quickly the full sequence can be acquired. If the object moves between frames, that motion can introduce errors or artifacts into the reconstruction.
Structured light has a similar constraint. A known pattern, or sequence of patterns, is projected onto the scene and captured by the camera to recover 3D shape. The faster that sequence can be captured, the more closely the individual measurements correspond to the same scene.
This is where SPAD sensors become especially interesting. The sensors we work with can produce raw binary frames at rates approaching 100,000 frames per second, and Ubicept Photon Fusion (UPF) can reconstruct those measurements into useful intensity images. That can compress a photometric-stereo or structured-light sequence that might otherwise span tens of milliseconds into a much shorter interval, reducing motion between illumination states and the artifacts that motion can introduce.
For many applications, the real value isn’t choosing between active and passive imaging. It’s being able to get both types of information from the same sensor with very little time between them.
Active imaging can provide precise scene structure, such as the exact shape and distance of a road sign. Passive imaging provides the intensity information needed to answer a different question: what does the sign actually say? With UPF and FLARE, supported SPAD cameras can capture both kinds of information using the same sensor and optical path, with the active and passive measurements closely aligned in time.
That can be especially important in dynamic scenes, where separate depth and intensity sensors may see slightly different versions of the world and then require synchronization, calibration, and sensor fusion to bring their outputs together. Using a common SPAD sensor can simplify that problem while providing both geometric and visual information when the application needs them.
One of the more advanced techniques we’ve demonstrated is indirect perception, or what’s known in the research literature as non-line-of-sight (NLOS) imaging.
The idea is to use a visible surface, such as a wall, as an intermediate reflector. A pulsed source illuminates that surface, some of the light reaches the hidden scene, and a fraction eventually returns along the reverse path. Those additional path lengths appear as tiny differences in photon arrival time.
By measuring those delays with a SPAD camera, FLARE can recover information about geometry that would otherwise be occluded. It’s a specialized application, but a strong example of how precise photon timing can extend perception beyond the camera’s direct line of sight.
