|
Yu Pan (潘宇)
I am a research scientist at Huawei Noah’s Ark Lab. Prior to that, I received my Ph.D. degree from the School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen (HITSZ), supervised by Prof. Zenglin Xu.
I major in investigating combinations of tensor decomposition technique and deep neural networks on a variety of tasks, including model compression, efficient training, etc.
Feel free to contact me!
Intersts: Tensor Learning, Model Compression, Model Initialization, Training Efficiency.
Email  / 
CV  / 
Google Scholar  / 
Github
|
12/2023: One paper is accepted in AAAI 2024.
09/2023: One paper is accepted in NeurIPS 2023.
02/2023: Publish a preprint about tensorial neural networks with a collection on the web.
05/2022: One paper is accepted in ICML 2022.
02/2022: Publish a Latex template paperlighter.sty for writing papers in a simple way.
|
Reviewer, NeurIPS 2020-2024
Reviewer, ICML 2021-2024
Reviewer, ICLR 2022-2025
|
Selected Publications (* denotes equal contribution)
|
|
Preparing Lessons for Progressive Training on Language Models
Yu Pan*,
Ye Yuan*,
Yichun Yin,
Jiaxin Shi,
Zenglin Xu,
Ming Zhang,
Lifeng Shang,
Xin Jiang,
Qun Liu
AAAI, 2024 (Oral, Top 10%)
arXiv
Accelerating the pretraining of language models by employing LVPS to prelearn the functionalities of deeper layers at a reduced resource cost.
|
|
Reusing Pretrained Models by Multi-linear Operators for Efficient Training
Yu Pan,
Ye Yuan,
Yichun Yin,
Zenglin Xu,
Lifeng Shang,
Xin Jiang,
Qun Liu
NeurIPS, 2023
arXiv
Utilizing tensor ring matrix product operator (TR-MPO) to grow a small pretrained model to a large counterpart for efficient training.
|
|
Tensor Networks Meet Neural Networks: A Survey and Future Perspectives
Yu Pan*,
Maolin Wang*,
Zenglin Xu,
Xiangli Yang,
Guangxi Li,
Andrzej Cichocki
Preprint, 2023
arXiv /
code
A thoroughly investigated survey for tensorial neural networks (TNNs) on network compression, information fusion and quantum circuit simulation.
|
|
A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks
Yu Pan,
Zeyong Su,
Ao Liu,
Jingquan Wang,
Nannan Li,
Zenglin Xu
ICML, 2022
abs /
slide /
arXiv
Calculating suitable variances of weights for arbitrary Tensorial Convolutional Neural Networks (TCNNs).
|
|
RegNet: Self-Regulated Network for Image Classification
Jing Xu,
Yu Pan,
Xinglin Pan,
Kun Bai,
Steven Hoi,
Zhang Yi,
Zenglin Xu
TNNLS, 2022
abs /
arXiv
Applying recurrent neural networks (RNNs) to regulate convolutional neural networks (CNNs) for performance improvement.
|
|
TedNet: A Pytorch Toolkit for Tensor Decomposition Networks
Yu Pan,
Maolin Wang,
Zenglin Xu
Neurocomputing, 2022
abs /
arXiv /
code
A toolkit named TedNet for giving a flexible way to construct Tensor Decomposition Networks (TDNs).
|
|
Heuristic Rank Selection with Progressively Searching Tensor Ring Network
Yu Pan*,
Nannan Li*,
Yaran Chen,
Zixiang Ding,
Dongbin Zhao,
Zenglin Xu
Complex & Intelligent Systems, 2021
abs /
arXiv
Applying Genetic Algorithm (GA) to search tensor ring based deep models.
|
|
Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition
Yu Pan,
Jing Xu,
Maolin Wang,
Jinmian Ye,
Fei Wang,
Kun Bai,
Zenglin Xu
AAAI, 2019
abs /
arXiv /
code
Utilizing tensor ring decomposition for compressing recurrent neural networks (RNNs) by factorizing the input-to-hidden layer.
|
|