profile photo

Yu Pan (潘宇)

I am a research scientist at Huawei Noah’s Ark Lab. Prior to that, I received my Ph.D. degree from the School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen (HITSZ), supervised by Prof. Zenglin Xu.

I major in investigating combinations of tensor decomposition technique and deep neural networks on a variety of tasks, including model compression, efficient training, etc.

Feel free to contact me!

Intersts: Tensor Learning, Model Compression, Model Initialization, Training Efficiency.

Email  /  CV  /  Google Scholar  /  Github

News
12/2023: One paper is accepted in AAAI 2024.

09/2023: One paper is accepted in NeurIPS 2023.

02/2023: Publish a preprint about tensorial neural networks with a collection on the web.

05/2022: One paper is accepted in ICML 2022.

02/2022: Publish a Latex template paperlighter.sty for writing papers in a simple way.
Service
Reviewer, NeurIPS 2020-2024

Reviewer, ICML 2021-2024

Reviewer, ICLR 2022-2025
Selected Publications (* denotes equal contribution)
Preparing Lessons for Progressive Training on Language Models
Yu Pan*, Ye Yuan*, Yichun Yin, Jiaxin Shi, Zenglin Xu,
Ming Zhang, Lifeng Shang, Xin Jiang, Qun Liu
AAAI, 2024 (Oral, Top 10%)
arXiv

Accelerating the pretraining of language models by employing LVPS to prelearn the functionalities of deeper layers at a reduced resource cost.

Reusing Pretrained Models by Multi-linear Operators for Efficient Training
Yu Pan, Ye Yuan, Yichun Yin, Zenglin Xu, Lifeng Shang, Xin Jiang, Qun Liu
NeurIPS, 2023
arXiv

Utilizing tensor ring matrix product operator (TR-MPO) to grow a small pretrained model to a large counterpart for efficient training.

Tensor Networks Meet Neural Networks: A Survey and Future Perspectives
Yu Pan*, Maolin Wang*, Zenglin Xu, Xiangli Yang, Guangxi Li, Andrzej Cichocki
Preprint, 2023
arXiv / code

A thoroughly investigated survey for tensorial neural networks (TNNs) on network compression, information fusion and quantum circuit simulation.

A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks
Yu Pan, Zeyong Su, Ao Liu, Jingquan Wang, Nannan Li, Zenglin Xu
ICML, 2022
abs / slide / arXiv

Calculating suitable variances of weights for arbitrary Tensorial Convolutional Neural Networks (TCNNs).

RegNet: Self-Regulated Network for Image Classification
Jing Xu, Yu Pan, Xinglin Pan, Kun Bai, Steven Hoi, Zhang Yi, Zenglin Xu
TNNLS, 2022
abs / arXiv

Applying recurrent neural networks (RNNs) to regulate convolutional neural networks (CNNs) for performance improvement.

TedNet: A Pytorch Toolkit for Tensor Decomposition Networks
Yu Pan, Maolin Wang, Zenglin Xu
Neurocomputing, 2022
abs / arXiv / code

A toolkit named TedNet for giving a flexible way to construct Tensor Decomposition Networks (TDNs).

Heuristic Rank Selection with Progressively Searching Tensor Ring Network
Yu Pan*, Nannan Li*, Yaran Chen, Zixiang Ding, Dongbin Zhao, Zenglin Xu
Complex & Intelligent Systems, 2021
abs / arXiv

Applying Genetic Algorithm (GA) to search tensor ring based deep models.

Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition
Yu Pan, Jing Xu, Maolin Wang, Jinmian Ye, Fei Wang, Kun Bai, Zenglin Xu
AAAI, 2019
abs / arXiv / code

Utilizing tensor ring decomposition for compressing recurrent neural networks (RNNs) by factorizing the input-to-hidden layer.




Source code from this website.