FMCW lidar (offline)¶
An FMCW (frequency-modulated continuous-wave) lidar sweeps the frequency of its laser linearly, a chirp, and mixes the returning light with its own laser. Light that traveled longer left with an older frequency, so the beat frequency grows with the path length; a moving surface also shifts it by its Doppler shift. A triangular chirp, up and then down, separates the two (Kim et al. 2025, Eq. 9):
with \(B\) the chirp bandwidth, \(T\) its duration, \(l\) the optical path length and \(u\) the path velocity. This tutorial
renders the up- and down-chirp spectra of the Cornell box with DopplerHistogramPathTracerInline, with the moving
boxes of the Doppler spectrum tutorial, and recovers the distance and the velocity of
every pixel, from the mean spectra and from a single measurement with speckle.
It uses the scene described in Doppler rendering.
1. Load the scene¶
As in the Doppler spectrum tutorial, the scene is loaded with DontMergeMaterials, so
that the boxes keep the names the velocities refer to. The camera position gives the true distances later.
import falcorcomp as falcor
import numpy as np
import matplotlib
matplotlib.use("Agg") # no window
import matplotlib.pyplot as plt
testbed = falcor.Testbed(create_window=False)
# Keep every material separate, so that each object keeps its name (identical materials are merged by default).
testbed.load_scene("cornell-box/scene-v4-nolight.pbrt", falcor.SceneBuilderFlags.DontMergeMaterials)
testbed.resize_frame_buffer(256, 256)
testbed.scene.camera.aspectRatio = 1.0
testbed.clock.pause()
position = testbed.scene.camera.position
camera_position = np.array([position.x, position.y, position.z])
2. Build the render graph¶
The graph is that of the Doppler spectrum tutorial with a chirp:
chirpBandwidth,chirpDuration: the laser sweeps 1 GHz in 1 µs, up and then down. The range term is then \(B/(Tc) = 3.34\) MHz per meter of path length, and with the light at the camera, 6.67 MHz per meter of distance: the box, 6 to 8 m away, gives 40 to 55 MHz, the same order as the ±26 MHz Doppler shift of 20 m/s.frequencyMin,frequencyMax,frequencyBin: 512 bins from 0 to 100 MHz, now of beat frequency. A bin, 0.195 MHz, is 2.9 cm of distance or 0.15 m/s of velocity.useSingleChannel: one value per bin, for the two spectra of 512 bins at 256 x 256.
GBufferRT and VelocityGroundTruthInline give the true distance and velocity of each pixel’s primary hit.
WAVELENGTH = 1550.0 # nm
BANDWIDTH, DURATION = 1.0, 1.0 # the chirp: 1 GHz in 1 us, up and then down
F_MIN, F_MAX, BINS = 0.0, 100.0, 512 # beat frequencies, MHz
VELOCITIES = {"TallBox": {"linear": [0.0, 0.0, 20.0]}, "ShortBox": {"linear": [0.0, 0.0, -20.0]}} # m/s
# The range term of the beat frequency per meter of path length, B / (T c), in MHz per meter.
RANGE_FREQUENCY_PER_LENGTH = BANDWIDTH * 1000.0 / (DURATION * 299.792458)
graph = testbed.create_render_graph("FMCW")
graph.create_pass("VBuffer", "VBufferRT", {"samplePattern": "Center", "sampleCount": 1})
# A point light at the camera, as in a lidar.
graph.create_pass("Light", "LaserLight", {
"isLightSourceLaser": False, "laserCollocated": True, "laserPower": [10.0, 10.0, 10.0],
})
graph.create_pass("Tracer", "DopplerHistogramPathTracerInline", {
"samplesPerPixel": 64, "maxBounces": 3, "computeDirect": True,
"wavelength": WAVELENGTH,
"chirpBandwidth": BANDWIDTH, "chirpDuration": DURATION, # GHz, us: a triangular chirp
"frequencyMin": F_MIN, "frequencyMax": F_MAX, "frequencyBin": BINS,
"velocities": VELOCITIES,
"useSingleChannel": True, "singleChannel": "luminance", # one value per bin: a quarter of the memory
"accumulate": True,
})
# The true distance (from the world positions) and velocity, for comparison.
graph.create_pass("GBuffer", "GBufferRT", {"samplePattern": "Center", "sampleCount": 1})
graph.create_pass("Truth", "VelocityGroundTruthInline", {"mode": "doppler", "velocities": VELOCITIES})
graph.add_edge("VBuffer.vbuffer", "Tracer.vbuffer")
graph.add_edge("VBuffer.viewW", "Tracer.viewW")
graph.add_edge("Light", "Tracer")
graph.add_edge("Light", "Truth")
graph.mark_output("Tracer.spectrum") # up-chirp
graph.mark_output("Tracer.spectrumDown") # down-chirp
graph.mark_output("GBuffer.posW")
graph.mark_output("Truth.velocity")
testbed.render_graph = graph
3. Render¶
FRAMES = 64 # 64 frames x 64 spp
for _ in range(FRAMES):
testbed.frame()
4. Read the spectra¶
With a chirp, the tracer has two outputs on the same bins: spectrum, the up-chirp, and spectrumDown, the
down-chirp. With useSingleChannel, to_numpy() returns bins x height x width.
# With accumulate, a spectrum is the sum of the frames, per MHz: divide by the frame count for the mean.
up = graph.get_output("Tracer.spectrum").to_numpy() / FRAMES # (bins, height, width)
down = graph.get_output("Tracer.spectrumDown").to_numpy() / FRAMES
frequencies = F_MIN + (np.arange(BINS) + 0.5) * (F_MAX - F_MIN) / BINS # bin centers, MHz
# The true distance to the primary hit and velocity towards the camera; NaN where the camera ray misses (a few
# pixels along a crack in the bottom-right corner).
position = graph.get_output("GBuffer.posW").to_numpy()
hit = position[..., 3] > 0
true_distance = np.where(hit, np.linalg.norm(position[..., :3] - camera_position, axis=-1), np.nan)
true_velocity = np.where(hit, graph.get_output("Truth.velocity").to_numpy().reshape(hit.shape), np.nan)
5. Distance and velocity from the two peaks¶
The strongest light of a pixel is its single reflection, so each spectrum peaks at that reflection’s beat frequency. Their mean is the range term and half their difference the Doppler shift. With the light at the camera, a surface at distance \(d\) moving towards the camera at \(v\) has \(l = 2d\) and \(u = 2v\):
def reconstruct(up, down):
"""The beat frequency is f_R - f_D on the up-chirp and f_R + f_D on the down-chirp."""
f_up = frequencies[np.argmax(up, axis=0)]
f_down = frequencies[np.argmax(down, axis=0)]
f_range, f_doppler = (f_up + f_down) / 2, (f_down - f_up) / 2
# The light is at the camera: the path length is twice the distance.
distance = f_range / RANGE_FREQUENCY_PER_LENGTH / 2
# f_D = u / wavelength, and the path velocity u is twice the velocity towards the camera.
velocity = f_doppler * WAVELENGTH / 1000.0 / 2
return distance, velocity
distance, velocity = reconstruct(up, down)
distance[~hit] = velocity[~hit] = np.nan
6. A single measurement¶
The rendered spectra are means. A real measurement sees speckle: the light comes from many microscopic reflectors with random phases, and the power of each frequency bin of one measurement is exponentially distributed around the mean, independently from bin to bin (Kim et al. 2025, Algorithm 1). One draw per bin gives a single measurement.
# Speckle makes the power of each bin exponentially distributed around the mean spectrum (Kim et al. 2025,
# Algorithm 1).
rng = np.random.default_rng(0)
up_single = rng.exponential(up)
down_single = rng.exponential(down)
distance_single, velocity_single = reconstruct(up_single, down_single)
distance_single[~hit] = velocity_single[~hit] = np.nan
7. Show the spectra of three pixels¶
The figure at the top. The approaching tall box (A, 6.78 m) peaks at 19.4 MHz on the up-chirp and 71.0 MHz on the down-chirp, 25.8 MHz on either side of its range term (the dashed line); the receding short box (B) the other way around, at 66.5 and 15.1 MHz; the static back wall (C) at 52.4 MHz on both. The weaker humps next to the peaks are light that also bounced off a wall: its path is longer, and only some of its segments move with the box, so its Doppler shift is smaller. The top axis reads a beat frequency as the distance of a static reflector; only C’s peaks are at its distance. The thin lines are one measurement: speckle changes the power of every bin by several dB.
image = up.sum(0) * (F_MAX - F_MIN) / BINS # the steady image: the spectrum summed over frequency
pixels = {"A: tall box": (140, 90), "B: short box": (185, 175), "C: back wall": (60, 200)}
figure, axes = plt.subplots(1, 4, figsize=(17, 4.4), gridspec_kw={"width_ratios": [1, 1.4, 1.4, 1.4]},
constrained_layout=True)
axes[0].imshow(np.clip(image / np.percentile(image, 99), 0, 1) ** (1 / 2.2), cmap="gray")
axes[0].set_title("steady image")
axes[0].set_xticks([])
axes[0].set_yticks([])
for axis, (name, (y, x)) in zip(axes[1:], pixels.items()):
axes[0].plot(x, y, "o", mfc="none", mec="C1", mew=2)
axes[0].annotate(name[0], (x, y), xytext=(7, -7), textcoords="offset points", color="C1", weight="bold")
peak = max(up[:, y, x].max(), down[:, y, x].max())
for mean, single, chirp, color in [(up, up_single, "up", "C0"), (down, down_single, "down", "C3")]:
axis.plot(frequencies, 10 * np.log10(single[:, y, x] / peak + 1e-9), color=color, lw=0.8, alpha=0.35,
label=f"{chirp}, one measurement")
axis.plot(frequencies, 10 * np.log10(mean[:, y, x] / peak + 1e-9), color=color, lw=1.5,
label=f"{chirp}, mean")
# The range term of the true distance: the up- and down-chirp peaks sit at f_R -+ f_D around it.
axis.axvline(2 * true_distance[y, x] * RANGE_FREQUENCY_PER_LENGTH, color="gray", ls="--", lw=1,
label="$f_R$ (true distance)")
axis.set_title(f"{name}: {true_distance[y, x]:.2f} m, {true_velocity[y, x]:+.1f} m/s")
axis.set_xlabel("beat frequency (MHz)")
axis.set_xlim(F_MIN, F_MAX)
axis.set_ylim(-45, 12)
# The distance at which a static reflector would give this beat frequency.
axis.secondary_xaxis("top", functions=(lambda f: f / (2 * RANGE_FREQUENCY_PER_LENGTH),
lambda d: d * 2 * RANGE_FREQUENCY_PER_LENGTH)).set_xlabel(
"distance if static (m)")
axes[1].set_ylabel("power (dB, relative to the peak)")
axes[3].legend(fontsize=8, loc="upper right")
figure.savefig("fmcw_spectra.png", dpi=90)
8. Show the distance and velocity¶
From the mean spectra, the distance is within 7 mm of the truth at half of the pixels (a bin is 2.9 cm of distance), and the velocity is within 0.08 m/s everywhere. From one measurement, most pixels keep their values, but where light bounced between surfaces competes with the single reflection, along the edges and in the corners of the box, speckle can lift a bin of that light above the peak on one of the chirps: 8 % of the pixels are off by more than 10 cm, and 5.5 % by more than 1 m/s. The few white pixels along the bottom-right corner are a crack in the scene’s meshes, where the camera rays miss.
figure, axes = plt.subplots(2, 3, figsize=(12, 7.6), constrained_layout=True)
for row, (name, truth, mean, single, colormap, limits) in enumerate([
("distance (m)", true_distance, distance, distance_single, "viridis", (6.0, 8.6)),
("velocity (m/s)", true_velocity, velocity, velocity_single, "bwr", (-20, 20))]):
for axis, (title, values) in zip(axes[row], [("true", truth), ("from the mean spectra", mean),
("from one measurement", single)]):
shown = axis.imshow(values, cmap=colormap, vmin=limits[0], vmax=limits[1], interpolation="nearest")
axis.set_title(f"{name}, {title}")
axis.set_xticks([])
axis.set_yticks([])
figure.colorbar(shown, ax=axes[row].tolist(), fraction=0.03)
figure.savefig("fmcw_reconstruction.png", dpi=90)
The full script: fmcw_lidar_offline.py. It takes about 20 seconds.