Track Length and Exposure Weighting in Driving Evaluation

Authors

  • Nicholas Fisher Author
  • Oliver Wallace Author
  • Bruce Phillips Author

Abstract

Equal-track and equal-annotation summaries give different visibility pictures in KITTI tracking data. The mean occlusion fraction is 0.612 when each eligible track receives equal weight, compared with 0.492 across annotations. Among 574 tracks, longer observed histories also have lower adjusted occlusion fractions. This secondary analysis makes exposure weighting explicit; it does not interpret observed length as a randomly assigned duration or measure tracking accuracy. The result motivates preserving trajectory structure when designing evaluations of generated driving scenes.

References

  1. Meng, Zijie, et al. "OmniDrive: An LLM-Choreographed Multi-Agent World Model with Unified Latent Co-Compression for Multi-View Driving Video Generation." arXiv preprint arXiv:2606.17536 (2026).
  2. White, Halbert. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica, vol. 48, no. 4, 1980, pp. 817–838. https://doi.org/10.2307/1912934.
  3. Cameron, A. Colin, and Douglas L. Miller. “A Practitioner’s Guide to Cluster-Robust Inference.” Journal of Human Resources, vol. 50, no. 2, 2015, pp. 317–372. https://doi.org/10.3368/jhr.50.2.317.
  4. Greenland, Sander, Judea Pearl, and James M. Robins. “Causal Diagrams for Epidemiologic Research.” Epidemiology, vol. 10, no. 1, 1999, pp. 37-48. https://doi.org/10.1097/00001648-199901000-00008.
  5. Benjamini, Yoav and Yosef Hochberg. “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing.” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 57, no. 1, 1995, pp. 289-300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.
  6. Roberts, David R., et al. “Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure.” Ecography, vol. 40, no. 8, 2017, pp. 913-929. https://doi.org/10.1111/ecog.02881.
  7. Varma, Sudhir and Richard Simon. “Bias in error estimation when using cross-validation for model selection.” BMC Bioinformatics, vol. 7, no. 1, 2006. https://doi.org/10.1186/1471-2105-7-91.
  8. Wilkinson, Mark D., et al. "The FAIR Guiding Principles for Scientific Data Management and Stewardship." Scientific Data, vol. 3, 2016, article 160018. https://doi.org/10.1038/sdata.2016.18.
  9. Sculley, D., et al. "Hidden Technical Debt in Machine Learning Systems." Advances in Neural Information Processing Systems, vol. 28, 2015. https://papers.nips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html
  10. Bland, J. Martin, and Douglas G. Altman. "Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement." The Lancet, vol. 327, no. 8476, 1986, pp. 307-310. https://doi.org/10.1016/S0140-6736(86)90837-8.
  11. Guo, Chuan, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. “On Calibration of Modern Neural Networks.” Proceedings of the 34th International Conference on Machine Learning, PMLR, vol. 70, 2017, pp. 1321–1330. https://proceedings.mlr.press/v70/guo17a.html.
  12. Geifman, Yonatan, and Ran El-Yaniv. “Selective Classification for Deep Neural Networks.” Advances in Neural Information Processing Systems, vol. 30, 2017. https://arxiv.org/abs/1705.08500.
  13. Ovadia, Yaniv, et al. “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.” Advances in Neural Information Processing Systems, vol. 32, 2019. https://arxiv.org/abs/1906.02530.
  14. Geiger, Andreas, Philip Lenz, and Raquel Urtasun. "Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite." 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 3354-3361. https://doi.org/10.1109/CVPR.2012.6248074.
  15. Caesar, Holger, et al.. "nuScenes: A multimodal dataset for autonomous driving" arXiv preprint arXiv:1903.11027, 2019. https://arxiv.org/abs/1903.11027.
  16. Yu, Fisher, et al.. "BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning" arXiv preprint arXiv:1805.04687, 2018. https://arxiv.org/abs/1805.04687.
  17. Sun, Pei, et al.. "Scalability in Perception for Autonomous Driving: Waymo Open Dataset" arXiv preprint arXiv:1912.04838, 2019. https://arxiv.org/abs/1912.04838.
  18. Luiten, Jonathon, et al.. "HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking" arXiv preprint arXiv:2009.07736, 2020. https://arxiv.org/abs/2009.07736.
  19. Zhang, Yifu, et al.. "ByteTrack: Multi-Object Tracking by Associating Every Detection Box" arXiv preprint arXiv:2110.06864, 2021. https://arxiv.org/abs/2110.06864.
  20. Weng, Xinshuo, et al.. "AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics" arXiv preprint arXiv:2008.08063, 2020. https://arxiv.org/abs/2008.08063.
  21. Unterthiner, Thomas, et al.. "Towards Accurate Generative Models of Video: A New Metric & Challenges" arXiv preprint arXiv:1812.01717, 2018. https://arxiv.org/abs/1812.01717.
  22. Wang, Zhou, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. "Image Quality Assessment: From Error Visibility to Structural Similarity." IEEE Transactions on Image Processing, vol. 13, no. 4, 2004, pp. 600-612. https://doi.org/10.1109/TIP.2003.819861.
  23. Zhang, Richard, et al.. "The Unreasonable Effectiveness of Deep Features as a Perceptual Metric" arXiv preprint arXiv:1801.03924, 2018. https://arxiv.org/abs/1801.03924.
  24. Hafner, Danijar, et al.. "Dream to Control: Learning Behaviors by Latent Imagination" arXiv preprint arXiv:1912.01603, 2019. https://arxiv.org/abs/1912.01603.
  25. Ha, David and Jürgen Schmidhuber. "World Models" arXiv preprint arXiv:1803.10122, 2018. https://arxiv.org/abs/1803.10122.
  26. Ye, Hu, et al.. "IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models" arXiv preprint arXiv:2308.06721, 2023. https://arxiv.org/abs/2308.06721.
  27. Huang, Ziqi, et al.. "VBench: Comprehensive Benchmark Suite for Video Generative Models" arXiv preprint arXiv:2311.17982, 2023. https://arxiv.org/abs/2311.17982.
  28. Radford, Alec, et al.. "Learning Transferable Visual Models From Natural Language Supervision" arXiv preprint arXiv:2103.00020, 2021. https://arxiv.org/abs/2103.00020.
  29. Gebru, Timnit, et al.. "Datasheets for datasets" Communications of the ACM, 2021. https://doi.org/10.1145/3458723.
  30. Levine, Sergey, Aviral Kumar, George Tucker, and Justin Fu. “Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems.” arXiv, 2020, arXiv:2005.01643. https://arxiv.org/abs/2005.01643.

Downloads

Published

2026-08-12

Issue

Vol. 2 (2026): Archive issue of 2026-08-12

Section

Articles

License

Copyright (c) 2026 Nicholas Fisher, Oliver Wallace, Bruce Phillips

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.