page visits since 2017
Free Web Counters unique visits since 2017

Google Scholar  /  DBLP  /  Semantic Scholar  /  GitHub  /  LinkedIn  /  Twitter  /  ResearchGate

Xinshuo Weng is a research scientist at NVIDIA Research working with Marco Pavone. She received a Ph.D. in Robotics (2018-2022) and a Master in Computer Vision (2016-17) working with Kris Kitani at Carnegie Mellon University. She's also worked with Yaser Sheikh at Facebook Reality Lab as a research engineer to help build “Photorealistic Telepresence”. Her bachelor was received from Wuhan University. Her research interest lies in generative models and 3D computer vision for autonomous systems. She has developed 3D multi-object tracking systems such as AB3DMOT that received >1,300 stars on GitHub. Also, she is leading a few autonomous driving workshops at top conferences such as NeurIPS, IJCAI, ICCV, ICML and IROS. She was awarded a number of fellowships and nominations such as the Qualcomm Innovation Fellowship for 2020 and Facebook Fellowship Finalist for 2021.

Specifically, Xinshuo's research spans the tasks of object detection, multi-object tracking, re-identification, trajectory prediction, and motion planning, with an ultimate goal of building an autonomous system such as self-driving cars that can safely interact with others in multi-agent dynamic environments. Towards this goal, she develops computational models to solve each individual task using machine learning techniques such as graph neural networks, transformers, generative adversarial networks and variational auto-encoders. To make the entire robot system robust and safe, Xinshuo's research also aims to seamlessly integrate models across tasks by building differentiable pipelines, propagating uncertainties from the upstream to downstream models, and exploring the most effective structure of the differentiable pipelines.

Fields: Computer Vision, Robotics, Machine Learning
Topics: 3D Computer Vision, Autonomous Driving, Generative Models, Perception, Imitation Learning
  • 06/2022 - Gave my PhD thesis defense at CMU [Slides]
  • 05/2022 - Keynote speaker at ICRA 2022 Workshop on Fresh Perspectives on the Future of Autonomous Driving [Slides] [Video]
  • 04/2022 - Gave a guest lecture at CMU 16-831 (Statistical Techniques in Robotics) [Slides] [Video]
  • 04/2022 - One paper accepted at Intelligent Vehicle Symposium 2022
  • 03/2022 - Organizing ICML 2022 Workshop on Safe Learning for Autonomous Driving
  • 03/2022 - Serving as an associate editor for AI and Autonomous Systems
  • 03/2022 - One paper accepted at CVPR 2022
  • 02/2022 - Organizing IJCAI 2022 Workshop on Artificial Intelligence for Autonomous Driving
  • 02/2022 - Invited speaker at Toyota Research Institute Machine Learning Reading Group [Slides]
  • 01/2022 - Gave research talks at NVIDIA Research, Borealis AI, Waabi, Samsung AI Center Toronto
  • 12/2021 - Gave a guest lecture at CMU 16-720B (Computer Vision) [Slides] [Video]
  • 11/2021 - Gave my PhD thesis proposal at CMU [Slides]
  • 10/2021 - Invited speaker at AiBee R&D Seminar Series [Slides]
  • 10/2021 - One paper accepted at BMVC 2021
  • 10/2021 - Keynote speaker and panelist at ICCV 2021 Workshop on SSLL: Share Stories and Lessons Learned [Slides] [Video]
  • 07/2021 - Three papers accepted at ICCV 2021
  • 07/2021 - Keynote speaker at IV 2021 Workshop on 3D Deep Learning for Automated Driving [Slides] [Video]
  • 07/2021 - Organizing NeurIPS 2021 Workshop on Machine Learning for Autonomous Driving
  • See More
  • 06/2021 - Keynote speaker at CVPR 2021 Workshop on Robust Video Scene Understanding [Slides]
  • 06/2021 - Announced the AIODrive trajectory forecasting challenge on the Precognition workshop, CVPR 2021 [Slides]
  • 06/2021 - Keynote speaker at CVPR 2021 Workshop on Autonomous Navigation [Slides]
  • 05/2021 - Honored to receive the Best Reviewer Award at ICML 2021
  • 05/2021 - Honored to receive the Outstanding Reviewer Award at CVPR 2021
  • 05/2021 - Organizing IROS 2021 Workshop on Multi-Agent Interaction and Relational Reasoning
  • 04/2021 - Honored to be selected into Facebook Fellowship Finalist 2021
  • 04/2021 - Invited speaker at MIT Vision and Graphics Seminar [Slides] [Video]
  • 04/2021 - Organizing ICCV 2021 Workshop on Multi-Agent Interaction and Relational Reasoning
  • 03/2021 - Organizing IJCAI 2021 Workshop on Artificial Intelligence for Autonomous Driving
  • 03/2021 - One paper accepted at CVPR 2021
  • 02/2021 - Two papers accepted at ICRA 2021 (Best Student Paper Candidate)
  • 12/2020 - Invited speaker at Wayve and Computer Vision Talks [Slides] [Video]
  • 12/2020 - Honored to receive the Outstanding Reviewer Award at ACCV 2020
  • 12/2021 - Gave a guest lecture at CMU 16-720B (Computer Vision) [Slides]
  • 10/2020 - Papers accepted at CoRL 2020, WACV 2021, ISARC 2020
  • 08/2020 - Honored to receive the Qualcomm Innovation Fellowship 2020
  • 08/2020 - Keynote speaker at ECCV 2020 Workshop on Benchmarking Trajectory Forecasting Models [Slides]
  • 08/2020 - Organizing NeurIPS 2020 Workshop on Machine Learning for Autonomous Driving
  • 08/2020 - Four (one oral, three spotlight) papers accepted at ECCV 2020 Workshops
  • 06/2020 - Two papers accepted at IROS 2020
  • 06/2020 - Keynote Speaker at CVPR 2020 Workshop on Scalibility in Autonomous Driving [Slides] [Video]
  • 04/2020 - One paper accepted at TPAMI 2020
  • 03/2020 - One paper accepted at CVPR 2020
  • 08/2019 - One paper accepted at ICCV Workshops 2019
  • 06/2019 - We release the code for our 3D MOT paper here
  • 06/2019 - Three papers accepted at BMVC 2019, ACMMM 2019, IROS 2019
  • 01/2018 - One paper accepted at CVPR 2018
  • 10/2017 - One paper accepted at WACV 2018
  • Whose Track Is It Anyway? Improving Robustness to Tracking Errors with Affinity-based Trajectory Prediction (Affinipred)
    Xinshuo Weng, Boris Ivanovic, Kris Kitani, Marco Pavone
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022
    PDF | BibTex
    The first trajectory prediction framework that removes the need for input past trajectories and also the error-prone data association step


    PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
    Xinshuo Weng*, Ye Yuan*, Kris Kitani
    Robotics and Automation Letters (RA-L), 2021
    with presentation at IEEE International Conference on Robotics and Automation (ICRA), 2021
    PDF | Demo | Website | Slides | BibTex
    The first parallelized 3D MOT and trajectory forecasting method with object interaction modeling and diverse trajectory samples

    Inverting the Pose Forecasting Pipeline with SPF2: Sequential Pointcloud Forecasting for Sequential Pose Forecasting
    Xinshuo Weng, Jianren Wang, Sergey Levine, Kris Kitani, Nick Rhinehart
    Conference on Robot Learning (CoRL), 2020
    PDF | Demo | Website | Slides | BibTex | Supp
    By learning to forecast future LiDAR point clouds, we build a forecast-then-detect pipeline for pose forecasting by reversing the steps of detection and forecasting


    GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with 2D-3D Multi-Feature Learning
    Xinshuo Weng, Yongxin Wang, Yunze Man, Kris Kitani
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020
    PDF | Demo | Website | Slides | BibTex
    The first multi-object tracking method that leverages Graph Neural Network for object interaction modeling

    3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
    Xinshuo Weng, Jianren Wang, David Held, Kris Kitani
    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
    PDF | Code | Demo | Website | Slides | BibTex
    A 3D multi-object tracker that achieves state-of-the-art performance with the fastest speed

    Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
    Xinshuo Weng, Kris Kitani
    IEEE International Conference on Computer Vision (ICCV) Workshops, 2019.
    PDF | Poster | BibTex
    By projecting the 2D image to a pseudo-LiDAR point cloud representation, our monocular 3D detection pipeline quadruples the performance over prior art