Autonomous Viewpoint-Dependent Semantic Perception

In this research project we investigate approaches for autonomous semantic perception. In particular, we proposed an algorithm for robust visual classification of an object of interest observed from multiple views using a black-box Bayesian classifier which provides a measure of uncertainty, in the presence of significant ambiguity and classifier noise, and of localization error. The fusion of classifier outputs takes into account viewpoint dependency and spatial correlation among observations, as well as pose uncertainty when these observations are taken and a measure of confidence provided by the classifier itself. Furthermore, we developed a novel approach that infers a distribution over posterior class probabilities within a Bayesian framework, while accounting for model uncertainty. This distribution enables reasoning about uncertainty in the posterior classification, and thus is of prime importance for robust classification and object-level perception in uncertain and ambiguous scenarios, and for safe autonomy in general. Additional related works are provided below.

Related Publications:

Journal Articles

  1. V. Tchuiev and V. Indelman, “Epistemic Uncertainty Aware Semantic Localization and Mapping for Inference and Belief Space Planning,” Artificial Intelligence, Special Issue on Risk-Aware Autonomous Systems, 2023.
    Tchuiev23ai.pdf DOI: 10.1016/j.artint.2023.103903
  2. V. Tchuiev and V. Indelman, “Distributed Consistent Multi-Robot Semantic Localization and Mapping,” IEEE Robotics and Automation Letters (RA-L), no. 3, Jul. 2020.
    Tchuiev20ral.pdf DOI: 10.1109/LRA.2020.3003275 Tchuiev20ral.supplementary Tchuiev20ral.video
  3. Y. Feldman and V. Indelman, “Spatially-Dependent Bayesian Semantic Perception under Model and Localization Uncertainty,” Autonomous Robots, 2020.
    Feldman20arj.pdf DOI: 10.1007/s10514-020-09921-0
  4. V. Tchuiev and V. Indelman, “Inference over Distribution of Posterior Class Probabilities for Reliable Bayesian Classification and Object-Level Perception,” IEEE Robotics and Automation Letters (RA-L), no. 4, 2018.
    Tchuiev18ral.pdf Tchuiev18ral.slides DOI: 10.1109/LRA.2018.2852844

Conference Articles

  1. T. Lemberg and V. Indelman, “Hybrid Belief Pruning with Guarantees for Viewpoint-Dependent Semantic SLAM,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2022.
    Lemberg22iros.pdf Lemberg22iros.video
  2. Y. Feldman and V. Indelman, “Towards Self-Supervised Semantic Representation with a Viewpoint-Dependent Observation Model,” in International Conference on Computational Photography (ICCP), May 2021.
    Feldman21iccp.poster
  3. Y. Feldman and V. Indelman, “Autonomous Semantic Perception in Uncertain Environments,” in RSS Pioneers Workshop, Jul. 2021.
    Feldman21rss_ws.pdf Feldman21rss_ws.poster
  4. Y. Feldman and V. Indelman, “Towards Self-Supervised Semantic Representation with a Viewpoint-Dependent Observation Model,” in Workshop on Self-Supervised Robot Learning, in conjunction with Robotics: Science and Systems (RSS), Jul. 2020.
    Feldman20rss_ws.pdf Feldman20rss_ws.supplementary Feldman20rss_ws.video
  5. V. Tchuiev, Y. Feldman, and V. Indelman, “Data Association Aware Semantic Mapping and Localization via a Viewpoint Dependent Classifier Model,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Nov. 2019.
    Tchuiev19iros.pdf Tchuiev19iros.slides
  6. Y. Feldman and V. Indelman, “Bayesian Viewpoint-Dependent Robust Classification under Model and Localization Uncertainty,” in IEEE International Conference on Robotics and Automation (ICRA), May 2018.
    Feldman18icra.pdf Feldman18icra.poster
  7. Y. Feldman and V. Indelman, “Towards Robust Autonomous Semantic Perception,” in Workshop on Representing a Complex World: Perception, Inference, and Learning for Joint Semantic, Geometric, and Physical Understanding, in conjunction with IEEE International Conference on Robotics and Automation (ICRA), May 2018.
    Feldman18icra_ws.pdf

Theses

  1. Y. Feldman, “Semantic Perception under Uncertainty with Viewpoint-Dependent Models,” PhD thesis, Technion - Israel Institute of Technology, 2022.
    Feldman22thesis.pdf Feldman22thesis.slides Feldman22thesis.video
  2. V. Tchuiev, “Autonomous Classification Under Uncertainty,” PhD thesis, Technion - Israel Institute of Technology, 2021.
    Tchuiev21thesis.pdf Tchuiev21thesis.slides Tchuiev21thesis.video