PI Renjie

PhD student at CSE department of HKUST

rpi(at)connect(dot)ust(dot)hk

About Me

I am currently a 3rd year PhD student in the CSE department of Hong Kong University of Science and Technology (HKUST), co-supervised by Professor Tong Zhang and Professor Xiaofang Zhou. Before that, I was an undergraduate student at the University of Hong Kong (HKU) major in Computer Engineering.

My current research interest mainly lies in Data-centric AI, Multi-modal Large Language Model (MLLM) and optimization algorithms. I am fortunate and honored to have received Apple Scholarship for AIML, 2024. I am open to cooporation, if you are interested, please email me ~

During my leisure time, I play lots of sports such as basketball, jumping rope and jogging. I am a music lover as well, here's a clip of me performing on stage.

Education & Experiences

• the University of Hong Kong
  Sept 2016 - 2021, department of computer engineering, first class honors

• the Hong Kong University of Science and Technology
  Sept 2021 - ongoing, department of computer science and engineering

• the University of Maryland (College Park)
  Sept 2018 - January 2019, exchange student

• Peking University (北京大学)
  June 2017 - July 2017, visiting student

Publications

→ Full list

PerceptionGPT: Effectively Fusing Visual Perception into LLM
Renjie Pi*, Lewei Yao, Jiahui Gao, Jipeng Zhang, Tong Zhang
2024 Conference on Computer Vision and Pattern Recognition (CVPR 2024)
[PDF] [Code]

DetGPT: Detect What You Need Via Reasoning
Renjie Pi*, Jiahui Gao*, Shizhe Diao*, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, Tong Zhang
The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)
[PDF] [Code]

Speciality vs generality: An empirical study on catastrophic forgetting in fine-tuning foundation models
Yong Lin, Lu Tan, Hangyu Lin, Zeming Zheng, Renjie Pi, Jipeng Zhang, Shizhe Diao, Haoxiang Wang, Han Zhao, Yuan Yao, Tong Zhang
Preprint
[PDF] [Code]

Effective Bilevel Optimization via Minimax Reformulation
Xiaoyu Wang, Rui Pan, Renjie Pi, Tong Zhang
Preprint
[PDF] [Code]

DynaFed: Tackling Client Data Heterogeneity with Global Dynamics
Renjie Pi*, Weizhong Zhang*, Yueqi Xie*, Jiahui Gao, Xiaoyu Wang, Sunghun Kim, Qifeng Chen
2023 Conference on Computer Vision and Pattern Recognition (CVPR 2023)
[PDF] [Code]

Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization
Yueqi Xie*, Weizhong Zhang*, Renjie Pi*, Fangzhao Wu, Qifeng Chen, Xing Xie, Sunghun Kim
Preprint
[PDF]

A Holistic View of Noise Transition Matrix in Deep Learning and Beyond
Yong Lin*, Renjie Pi*, Weizhong Zhang, Xiaobo Xia, Jiahui Gao, Xiao Zhou, Tongliang Liu, Bo Han
2023 International Conference on Learning Representations (ICLR 2023, Spotlight)
[PDF] [Code]

Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
Jiahui Gao*, Renjie Pi*, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Xiaodan Liang, Zhenguo Li, Lingpeng Kong
2023 International Conference on Learning Representations (ICLR 2023, Spotlight)
[PDF] [Code]

Measure the Predictive Heterogeneity
Jiashuo Liu, Jiayun Wu, Renjie Pi, Renzhe Xu, Xingxuan Zhang, Bo Li, Peng Cui
2023 International Conference on Learning Representations 2023 (ICLR 2023)
[PDF]

Probabilistic Bilevel Coreset Selection
Xiao Zhou*, Renjie Pi*, Weizhong Zhang*, Yong Lin, Tong Zhang
2022 International Conference on Machine Learning (ICML 2022)
[PDF] [Code]

Model Agnostic Sample Reweighting for Out-of-Distribution Learning
Xiao Zhou, Yong Lin, Renjie Pi, Weizhong Zhang, Renzhe Xu, Peng Cui, Tong Zhang
2022 International Conference on Machine Learning 2022 (ICML 2022)
[PDF] [Code]

G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation
Lewei Yao*, Renjie Pi*, Hang Xu, Wei Zhang, Zhenguo Li, Tong Zhang
2021 International Conference on Computer Vision (ICCV 2021)
[PDF]

Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
Lewei Yao*, Renjie Pi*, Hang Xu, Wei Zhang, Zhenguo Li, Tong Zhang
2021 Conference on Computer Vision and Pattern Recognition (CVPR 2021)
[PDF]

Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
Han Shi*, Renjie Pi*, Hang Xu, Zhenguo Li, James T. Kwok, Tong Zhang.
2020 Neural Information Processing Systems (NeurIPS 2020)
[PDF] [Code]

Awards & Honors

• Apple Scholarship for AIML (2024)
• HKU Foundation Entrance Scholarship
• Entrance Scholarships in Electrical and Electronic Engineering
• S Y King Memorial Scholarship in Electrical and Electronic Engineering
• Postgraduate Studentship, HKUST