Home  ›  Continuous Energy Minimization for Multi Target Tracking Pami

Continuous Energy Minimization for Multi Target Tracking Pami

Written By Monday 24 October 2022 Add Comment Edit

Abstract

Most (3D) multi-object tracking methods rely on appearance-based cues for data association. By contrast, we investigate how far we can get by only encoding geometric relationships between objects in 3D space as cues for data-driven data association. We encode 3D detections as nodes in a graph, where spatial and temporal pairwise relations among objects are encoded via localized polar coordinates on graph edges. This representation makes our geometric relations invariant to global transformations and smooth trajectory changes, especially under non-holonomic motion. This allows our graph neural network to learn to effectively encode temporal and spatial interactions and fully leverage contextual and motion cues to obtain final scene interpretation by posing data association as edge classification. We establish a new state-of-the-art on nuScenes dataset and, more importantly, show that our method, PolarMOT, generalizes remarkably well across different locations (Boston, Singapore, Karlsruhe) and datasets (nuScenes and KITTI).

Keywords

  • 3D multi-object tracking
  • Graph neural networks
  • Lidar scene understanding

References

  1. Aygün, M., et al.: 4D panoptic lidar segmentation. In: CVPR (2021)

    Google Scholar

  2. Bergmann, P., Meinhardt, T., Leal-Taixé, L.: Tracking without bells and whistles. In: ICCV (2019)

    Google Scholar

  3. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. JIVP 2008, 1–10 (2008)

    Google Scholar

  4. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: ICIP (2016)

    Google Scholar

  5. Braso, G., Leal-Taixé, L.: Learning a neural solver for multiple object tracking. In: CVPR (2020)

    Google Scholar

  6. Brendel, W., Amer, M.R., Todorovic, S.: Multi object tracking as maximum weight independent set. In: CVPR (2011)

    Google Scholar

  7. Butt, A.A., Collins, R.T.: Multi-target tracking by Lagrangian relaxation to min-cost network flow. In: CVPR, June 2013

    Google Scholar

  8. Caesar, H., et al.: nuScenes: a multimodal dataset for autonomous driving. In: CVPR (2020)

    Google Scholar

  9. Chiu, H.K., Prioletti, A., Li, J., Bohg, J.: Probabilistic 3D multi-object tracking for autonomous driving. In: ICRA (2021)

    Google Scholar

  10. Choi, W.: Near-online multi-target tracking with aggregated local flow descriptor. In: ICCV (2015)

    Google Scholar

  11. Dellaert, F., Thorpe, C.: Robust car tracking using Kalman filtering and Bayesian templates. In: Conference on Intelligent Transportation Systems (1997)

    Google Scholar

  12. Frossard, D., Urtasun, R.: End-to-end learning of multi-sensor 3D tracking by detection. ICRA (2018)

    Google Scholar

  13. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR (2012)

    Google Scholar

  14. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: International Conference on Machine Learning, pp. 1263–1272. PMLR (2017)

    Google Scholar

  15. Gori, M., Monfardini, G., Scarselli, F.: A new model for learning in graph domains. In: IJCNN (2005)

    Google Scholar

  16. Held, D., Levinson, J., Thrun, S., Savarese, S.: Combining 3D shape, color, and motion for robust anytime tracking. In: RSS (2014)

    Google Scholar

  17. Kim, A., Ošep, A., Leal-Taixé, L.: EagerMOT: 3D multi-object tracking via sensor fusion. In: ICRA (2021)

    Google Scholar

  18. Kim, D., Woo, S., Lee, J.Y., Kweon, I.S.: Video panoptic segmentation. In: CVPR (2020)

    Google Scholar

  19. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

    Google Scholar

  20. Kuhn, H.W., Yaw, B.: The Hungarian method for the assignment problem. Naval Res. Logist. Q., 83–97 (1955)

    Google Scholar

  21. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: fast encoders for object detection from point clouds. In: CVPR (2019)

    Google Scholar

  22. Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese CNN for robust target association. In: CVPR Workshops (2016)

    Google Scholar

  23. Leal-Taixé, L., Fenzi, M., Kuznetsova, A., Rosenhahn, B., Savarese, S.: Learning an image-based motion context for multiple people tracking. In: CVPR (2014)

    Google Scholar

  24. Leibe, B., Leonardis, A., Schiele, B.: Robust object detection with interleaved categorization and segmentation. IJCV 77(1–3), 259–289 (2008)

    CrossRef  Google Scholar

  25. Leibe, B., Schindler, K., Cornelis, N., Gool, L.V.: Coupled object detection and tracking from static cameras and moving vehicles. PAMI 30(10), 1683–1698 (2008)

    CrossRef  Google Scholar

  26. Li, J., Gao, X., Jiang, T.: Graph networks for multiple object tracking. In: WACV (2020)

    Google Scholar

  27. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: ICCV (2017)

    Google Scholar

  28. Liu, L., et al.: On the variance of the adaptive learning rate and beyond. In: ICLR, April 2020

    Google Scholar

  29. Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. In: ICLR (2017)

    Google Scholar

  30. Luiten, J., Fischer, T., Leibe, B.: Track to reconstruct and reconstruct to track. RAL 5(2), 1803–1810 (2020)

    Google Scholar

  31. Luiten, J., et al.: HOTA: a higher order metric for evaluating multi-object tracking. IJCV 129, 548–578 (2020)

    Google Scholar

  32. Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. PAMI 36(1), 58–72 (2014)

    CrossRef  Google Scholar

  33. Moosmann, F., Stiller, C.: Joint self-localization and tracking of generic objects in 3D range data. In: ICRA (2013)

    Google Scholar

  34. Mykheievskyi, D., Borysenko, D., Porokhonskyy, V.: Learning local feature descriptors for multiple object tracking. In: ACCV (2020)

    Google Scholar

  35. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML (2010)

    Google Scholar

  36. Nguyen, U., Heipke, C.: 3d pedestrian tracking using local structure constraints. ISPRS J. Photogrammetry Remote Sens. 166, 347–358 (2020)

    CrossRef  Google Scholar

  37. Ošep, A., Mehner, W., Mathias, M., Leibe, B.: Combined image- and world-space tracking in traffic scenes. In: ICRA (2017)

    Google Scholar

  38. Ošep, A., Mehner, W., Voigtlaender, P., Leibe, B.: Track, then decide: category-agnostic vision-based multi-object tracking. In: ICRA (2018)

    Google Scholar

  39. Petrovskaya, A., Thrun, S.: Model based vehicle detection and tracking for autonomous urban driving. AR 26, 123–139 (2009)

    Google Scholar

  40. Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR (2011)

    Google Scholar

  41. Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum PointNets for 3D object detection from RGB-D data. In: CVPR (2017)

    Google Scholar

  42. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: CVPR (2017)

    Google Scholar

  43. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: NeurIPS (2017)

    Google Scholar

  44. Reich, A., Wuensche, H.J.: Monocular 3D multi-object tracking with an EKF approach for long-term stable tracks. In: FUSION (2021)

    Google Scholar

  45. Schulter, S., Vernaza, P., Choi, W., Chandraker, M.K.: Deep network flow for multi-object tracking. In: CVPR (2017)

    Google Scholar

  46. Sharma, S., Ansari, J.A., Krishna Murthy, J., Madhava Krishna, K.: Beyond pixels: leveraging geometry and shape cues for online multi-object tracking. In: ICRA (2018)

    Google Scholar

  47. Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: CVPR (2019)

    Google Scholar

  48. Shi, W., Rajkumar, R.: Point-GNN: graph neural network for 3d object detection in a point cloud. In: CVPR (2020)

    Google Scholar

  49. Son, J., Baek, M., Cho, M., Han, B.: Multi-object tracking with quadruplet convolutional neural networks. In: CVPR (2017)

    Google Scholar

  50. Tang, S., Andres, B., Andriluka, M., Schiele, B.: Subgraph decomposition for multi-target tracking. In: CVPR (2015)

    Google Scholar

  51. Teichman, A., Levinson, J., Thrun, S.: Towards 3D object recognition via classification of arbitrary object tracks. In: ICRA (2011)

    Google Scholar

  52. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: KPConv: flexible and deformable convolution for point clouds. In: ICCV (2019)

    Google Scholar

  53. Tokmakov, P., Li, J., Burgard, W., Gaidon, A.: Learning to track with object permanence. In: ICCV (2021)

    Google Scholar

  54. Voigtlaender, P., et al.: MOTS: multi-object tracking and segmentation. In: CVPR (2019)

    Google Scholar

  55. Weng, X., Wang, J., Held, D., Kitani, K.: 3D multi-object tracking: a baseline and new evaluation metrics. In: IROS (2020)

    Google Scholar

  56. Weng, X., Wang, Y., Man, Y., Kitani, K.: GNN3DMOT: graph neural network for 3D multi-object tracking with multi-feature learning. In: CVPR (2020)

    Google Scholar

  57. Wu, H., Han, W., Wen, C., Li, X., Wang, C.: 3D multi-object tracking in point clouds based on prediction confidence-guided data association. IEEE TITS 23, 5668–5677 (2021)

    Google Scholar

  58. Wu, H., Li, Q., Wen, C., Li, X., Fan, X., Wang, C.: Tracklet proposal network for multi-object tracking on point clouds. In: IJCAI (2021)

    Google Scholar

  59. Xu, Y., Ošep, A., Ban, Y., Horaud, R., Leal-Taixé, L., Alameda-Pineda, X.: How to train your deep multi-object tracker. In: CVPR (2020)

    Google Scholar

  60. Yan, Y., Mao, Y., Li, B.: Second: sparsely embedded convolutional detection. Sensors 18(10), 3337 (2018)

    CrossRef  Google Scholar

  61. Yin, T., Zhou, X., Krähenbühl, P.: Center-based 3D object detection and tracking. In: CVPR (2021)

    Google Scholar

  62. Zaech, J.N., Liniger, A., Dai, D., Danelljan, M., Van Gool, L.: Learnable online graph representations for 3D multi-object tracking. IEEE R-AL , 5103–5110 (2022)

    Google Scholar

  63. Zeng, Y., Ma, C., Zhu, M., Fan, Z., Yang, X.: Cross-modal 3D object detection and tracking for auto-driving. In: IROS (2021)

    Google Scholar

  64. Zhang, L., Yuan, L., Nevatia, R.: Global data association for multi-object tracking using network flows. In: CVPR (2008)

    Google Scholar

  65. Zhou, X., Koltun, V., Krähenbühl, P.: Tracking objects as points. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 474–490. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_28

    CrossRef  Google Scholar

  66. Zhou, Y., Tuzel, O.: VoxelNet: end-to-end learning for point cloud based 3D object detection. In: CVPR (2018)

    Google Scholar

Download references

Acknowledgement

This research was partially funded by the Humboldt Foundation through the Sofja Kovalevskaja Award.

Author information

Authors and Affiliations

  1. Technical University of Munich, Munich, Germany

    Aleksandr Kim, Guillem Brasó, Aljoša Ošep & Laura Leal-Taixé

Corresponding author

Correspondence to Aleksandr Kim .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Kim, A., Brasó, G., Ošep, A., Leal-Taixé, L. (2022). PolarMOT: How Far Can Geometric Relations Take us in 3D Multi-object Tracking?. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13682. Springer, Cham. https://doi.org/10.1007/978-3-031-20047-2_3

Download citation

  • .RIS
  • .ENW
  • .BIB

  • DOI : https://doi.org/10.1007/978-3-031-20047-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20046-5

  • Online ISBN: 978-3-031-20047-2

  • eBook Packages: Computer Science Computer Science (R0)

brownthatted1993.blogspot.com

Source: https://link.springer.com/chapter/10.1007/978-3-031-20047-2_3

0 Response to "Continuous Energy Minimization for Multi Target Tracking Pami"

Post a Comment

Subscribe to: Post Comments (Atom)

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel