Lu Jiang

Research Scientist (Google) &
Adjunct Professor (CMU)
lujiang [at]
Greeting! I am a staff research scientist at Google and an adjunct faculty member at Carnegie Mellon University. My CV can be found here. My research goal is to solve the real problem on big multimodal data. My research area is in the interdisciplinary area of Machine Learning and Multimedia, specifically including robust deep learning, content creation, and video understanding.

news What's New:
  • [2020/06] Our dataset, The Garden of Forking Paths, is now available on GitHub. It is the first dataset that allows us to compare models in a quantitative way in terms of their ability to predict multiple plausible futures.
  • [2020/05] Co-organized two workshops in CVPR 2020: (1) AI for Content Creation (2) Language and Vision with applications to Video Understanding.
  • [2019/09] Congrats to our former intern for receiving Baidu Scholarship 2019 (10 recipients globally).
  • [2019/09] Served as the NSF SBIR/STTR review panelist.
  • [2019/07] Our work on robust learning for machine translation has been named as the best paper candidate in ACL 2019 (1% of all submitted long papers).
  • [2019/05] Our code for Composing Text and Image for Image Retrieval (CVPR 2019) is now on GitHub. It shows a new task of using vision and language research for retrieval.
  • [2019/05] Our code for future activity prediction (CVPR 2019) is now available. It is the first and currently the best model for joint path and activity prediction. Check out excellent cool demo video.
  • [2019/05] Our code for Eidetic-3D LSTM (ICLR 2019) is now on GitHub.
  • [2019/03] Two guest lectures (LTI-11-775) on vision + language at Carnegie Mellon University.
  • [2019/03] Served as an area chair for ACM Multimedia 2019.
  • [2019/01] The code for our graph distillation (ECCV 2018) is now on GitHub.
  • [2018/12] Check out our MemexQA dataset published in TPAMI 2019.
  • [2018/09] Dealing with noisy data in deep learning? Check out our code for ICML 2018.
  • [2018/07] Check out our code for visual question answering over sequence data (CVPR 2018)
  • [2018/03] Congrats to our intern Zelun (Alan) Luo, who was co-hosted with Juan Carlos, for receiving Ph.D. offers from top universities (MIT/Stanford/UC Berkeley/CMU).