Corvus ISR Publishes Synthetic Benchmark Results for Tracker Accuracy

Corvus ISR, known for its wide-area motion imagery (WAMI) exploitation tools, has released a detailed public tracker benchmark that compares two prominent tracker models on a fixed synthetic scene. This benchmark employs a perfect ground truth dataset, generated entirely synthetically, allowing for precise measurement of each model’s performance under controlled conditions. The synthetic environment ensures that every pixel and object movement is known exactly, eliminating uncertainties typical in real-world data.

Age 18–24?Offer from Amazon

Prime made for students and young adults

• Fast, free delivery for dorm and study essentials
• Prime Video and Amazon Music included
• Member-only deals

Try Prime for Young Adults →Free trial for eligible 18–24 year olds

As an affiliate, we earn on qualifying purchases.Hide

The two models under evaluation are the v1 “greedy nearest-neighbour” baseline and the v2 “confirmed-track auction”. The baseline model features a simple, two-pass greedy association with constant-velocity prediction and fixed 2-second coasting, representing the published floor in tracking complexity. In contrast, the v2 model incorporates advanced features such as three-tier auction association, velocity-consistency gating, and noise-scaled reservation pricing, reflecting the latest innovations in tracking algorithms. The benchmark’s consistency in metric definitions and sensor models ensures a fair comparison, counting every identity switch with strictness beyond standard MOT challenge metrics.

Results from the benchmark reveal significant improvements with v2. For example, under a baseline scenario of 150 movers at 2fps, the number of ID switches per minute decreased from 2,042 to 1,183, a reduction of 42.1%. Similar reductions are observed in dense scenes with 400 movers, dropping from 14,032 to 8,040 (−42.7%). These metrics are solely dependent on the models’ tracking logic, as detection performance was held constant due to the identical sensor and detection generation setup. The results highlight how sophisticated association strategies can markedly improve tracking stability.

To emphasize transparency, Corvus ISR publishes the failure numbers openly. Both models still produce thousands of identity errors per minute in challenging scenarios, and these results serve as a scientific baseline rather than marketing. Synthetic scenes with perfect ground truth allow for measurement, not marketing. As the company states, “Vendors who show only successes ask for faith; a published failure matrix asks for measurement.” This approach encourages honest progress in the field and provides a clear target for future improvements.

From an engineering perspective, the v2 tracker demonstrates impressive speed, averaging approximately 1.2 milliseconds per sensor tick at a density of 400 objects. Even in worst-case scenarios, it remains well within the 10-millisecond real-time budget, making it practical for deployment. The entire benchmark process is accessible through a live demo where anyone can reproduce the results. Simply press “Run benchmark” to observe the models’ performance in real time—no sign-up or NDA required.

The methodology behind this synthetic benchmark underscores the importance of perfect ground truth data for rigorous evaluation. By removing real-world complexities, it provides a controlled environment where algorithmic improvements can be objectively assessed. Publishing these failure metrics, rather than just successes, fosters transparency and accelerates innovation in the field of motion tracking technology.

For science-minded readers interested in exploring these models further, Corvus ISR’s benchmark is a valuable resource. It demonstrates the significance of fixed-seed synthetic testing and invites everyone to reproduce it live. Why not try running the benchmark yourself and see how your tracker stacks up against these results?

Powered by Thorsten Meyer AI