I am an incoming PhD student at UMass Amherst, supervised by Prof. Chuang Gan. Currently, I’m a senior undergraduate at Shanghai Jiao Tong University (SJTU), where I work closely with Prof. Cewu Lu and Dr. Lixin Yang from the Machine Vision and Intelligence Group (MVIG).
Previously, I am a visiting student at the Massachusetts Institute of Technology and the MIT-IBM Watson AI Lab, guided by Prof. Josh Tenenbaum. In addition, I have worked with Prof. Junchi Yan at the SJTU-ReThinkLab.
🔥 News
- 2024.04: I will give a talk at OpenDriveLab, Shanghai AI Lab.
- 2024.03: Introduce 3D-VLA.
- 2023.10: One paper Color-NeuS was accepted by 3DV 2024.
- 2023.09: Two papers were accepted by NeurIPS 2023. One spotlight and one poster.
- 2023.07: Introduce 3D-LLM.
- 2023.07: One paper CHORD was accepted by ICCV 2023.
- 2023.04: One paper IOT-CL was accepted by ICML 2023.
- 2022.04: Build the website: AISJTU.ICU.
📖 Publications
3D-VLA: 3D Vision-Language-Action Generative World Model
Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, Chuang Gan
3D-LLM: Injecting the 3D World into Large Language Models
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan
Relative Entropic Optimal Transport: a (Prior-aware) Matching Perspective to (Unbalanced) Classification.
Liangliang Shi, Haoyu Zhen, Gu Zhang, Junchi Yan
NeurIPS 2023
Color-NeuS: Reconstructing Neural Implicit Surfaces with Color
Licheng Zhong, Lixin Yang, Kailin Li, Haoyu Zhen, Mei Han, Cewu Lu
3DV 2024
CHORD: Category-level in-Hand Object Reconstruction via Shape Deformation
Kailin Li, Lixin Yang, Haoyu Zhen, Zenan Lin, Xinyu Zhan, Licheng Zhong, Jian Xu, Kejian Wu, Cewu Lu
ICCV 2023
Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective
Liangliang Shi, Gu Zhang, Haoyu Zhen, Jintao Fan, Junchi Yan
ICML 2023
👨💻 Undergraduate Works
PyBlend: A Python Library for Blender
pip install pyblend｜PyPI｜Github｜Gallery｜
PyBlend, a project I’ve passionately contributed to, is a Python library that provides a high-level interface for Blender scripting. It empowers artists and developers like me to effortlessly enhance Blender workflows with an array of dynamic functions.
A Website For Courses of AI Pilot Class, SJTU
Founder and Primary Contributor: Haoyu Zhen
I built a community and developed a website for the AI pilot class at Shanghai Jiao Tong University. The website enables students to evaluate the quality of different courses and share learning materials.
🎖 Honors and Awards
- 2023.12 Rong Chang Scholarship of SJTU (10 recipients, 30000¥).
- 2021/2022/2023.11 Academic Excellence Scholarship of SJTU (top 10%).
- 2021.11 Huawei “Smart Base” Scholarship.
- 2019.10 The first prize of the 36th CPHO semi-finals, Beijing Division (Rank: 17, top 1%).
📖 Education
-
2023.06 - 2023.12, Visiting Student, Massachusetts Institute of Technology.
-
2020.09 - (now), Undergraduate, Artificial Intelligence, Shanghai Jiao Tong University. GPA: 92.80/100, 4.03/4.3 (Rank: 4/88).
Detailed Academic Performance [Implementation and Lecture Notes]
- 4th semester: GPA:95.24/100, 4.20/4.3 (Rank 1/89).
- Sophomore year: GPA:94.04/100, 4.12/4.3 (Rank 2/89).
🖥️ Experience
CoCoSci Lab, Massachusetts Institute of Technology and MIT-IBM Watson AI Lab
Visiting Student, 2023.03 - 2023.12
Advisor: Josh Tenenbaum, Chuang Gan
🍫 Life
- I spent a wonderful time at the Beijing National Day School from 2014 to 2020. I was a member of the Class of Physics Olympiad.
-
My best friend: Yilin Sun.
In our sophomore year, we teamed up to finish our first project in AI2611 Machine Learning:
We proposed a novel method for extracting gaze features from the face and eyes using the PIXIE model and contrastive learning (SimCLR) framework -
I am an active athlete who plays for the
Nov. 2021, School of Electronic Information and Electrical Engineering vs. School of Mechanical Engineering Badminton Match.
Oct. 11, 2022, taken at the frisbee pickup game, Zhiyuan Football Field, SJTU. - I am a fan of DC movies, as well as the TV series Rick & Morty, Breaking Bad, and Better Call Saul.