I am currently an undergraduate at Shanghai Jiao Tong University (SJTU), majoring in Artificial Intelligence.

I am also a visiting student/visitor at Massachusetts Institute of Technology and MIT-IBM Watson AI Lab, where I will be working under the guidance of Prof. Josh Tenenbaum and Prof. Chuang Gan.

In addition, as a research assistant at the Machine Vision and Intelligence Group (MVIG), I am under the supervision of Prof. Cewu Lu and Dr. Lixin Yang. Previously, I worked with Prof. Junchi Yan at the SJTU-ReThinkLab.

🔥 News

  • 2023.09:  Two papers were accepted by NeurIPS 2023. One spotlight and one poster.
  • 2023.08:  Color-NeuS was released on GitHub and arXiv.
  • 2023.07:  3D-LLM was released on GitHub and arXiv.
  • 2023.07:  One paper CHORD was accepted by ICCV 2023.
  • 2023.04:  One paper IOT-CL was accepted by ICML 2023.
  • 2022.04:  Build the website: AISJTU.ICU.

📖 Publications

NeurIPS 2023

3D-LLM: Injecting the 3D World into Large Language Models

Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan


NeurIPS 2023 (Spotlight)

NeurIPS 2023

Relative Entropic Optimal Transport: a (Prior-aware) Matching Perspective to (Unbalanced) Classification.

Liangliang Shi, Haoyu Zhen, Gu Zhang, Junchi Yan

NeurIPS 2023


Color-NeuS: Reconstructing Neural Implicit Surfaces with Color

Licheng Zhong, Lixin Yang, Kailin Li, Haoyu Zhen, Mei Han, Cewu Lu


ICCV 2023

CHORD: Category-level in-Hand Object Reconstruction via Shape Deformation

Kailin Li, Lixin Yang, Haoyu Zhen, Zenan Lin, Xinyu Zhan, Licheng Zhong, Jian Xu, Kejian Wu, Cewu Lu


ICCV 2023

ICML 2023

Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective

Liangliang Shi, Gu Zhang, Haoyu Zhen, Jintao Fan, Junchi Yan


ICML 2023

👨‍💻 Undergraduate Works


PyBlend: A Python Library for Blender

pip install pyblendPyPIGithub

PyBlend, a project I’ve passionately contributed to, is a Python library that provides a high-level interface for Blender scripting. It empowers artists and developers like me to effortlessly enhance Blender workflows with an array of dynamic functions.


A Website For Courses of AI Pilot Class, SJTU

Founder and Primary Contributor: Haoyu Zhen

Project Source CodeHomepage

I built a community and developed a website for the AI pilot class at Shanghai Jiao Tong University. The website enables students to evaluate the quality of different courses and share learning materials.

🎖 Honors and Awards

  • 2021.11 Huawei “Smart Base” Scholarship.
  • 2021.11 Academic Excellence Scholarship of SJTU (top 10%).
  • 2019.10 The first prize of the 36th CPHO semi-finals, Beijing Division (Rank: 17, top 1%).

📖 Educations

  • 2020.09 - (now), Undergraduate, Artificial Intelligence, Shanghai Jiao Tong University. GPA: 92.80/100, 4.03/4.3 (Rank: 4/88).

    Detailed Academic Performance [Implementation and Lecture Notes]
    • 4th semester: GPA:95.24/100, 4.20/4.3 (Rank 1/89).
    • Sophomore year: GPA:94.04/100, 4.12/4.3 (Rank 2/89).
  • 2014.09 - 2020.06, Class of Physics Olympiad, Beijing National Day School.

🖥️ Experiences

Machine Vision and Intelligence Group

Research undergraduate, 2021.12 - (now)

Advisor: Cewu Lu


Research undergraduate, 2022.09 - 2023.05

Advisor: Junchi Yan

🍫 Life

  • My best friend: Yilin Sun. In our sophomore year, we teamed up to finish our first project in AI2611 Machine Learning:
    GEFF sym
    We proposed a novel method for extracting gaze features from the face and eyes using the PIXIE model and contrastive learning (SimCLR) framework
    - Gaze Estimation with Fused Features.
  • I am an active athlete who plays for the
    Badminton Team of SEIEE sym
    Nov. 2021, School of Electronic Information and Electrical Engineering vs. School of Mechanical Engineering Badminton Match.
    and the
    SJTU frisbee team. sym
    Oct. 11, 2022, taken at the frisbee pickup game, Zhiyuan Football Field, SJTU.
    In addition to these sports, I also enjoy playing baseball, table tennis, football, and basketball.
  • I am a fan of DC movies, as well as the TV series Rick & Morty, Breaking Bad, and Better Call Saul.
  • Build LEGO Paris city sym
    Oct. 17, 2022, We built the LEGO Paris city together.
    We will always have Paris (2022.10) 🇫🇷