Yu Pan (潘宇)

I am a third-year Ph.D. candidate at School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen (HITSZ), supervised by Prof. Zenglin Xu.

I major in investigating combinations of tensor decomposition technique and deep neural networks on a variety of tasks, including model compression, efficient training, etc.

Feel free to contact me!

Intersts: Tensor Learning, Model Compression, Model Initialization, Training Efficiency.

Email / CV / Google Scholar / Github

News

12/2023: One paper is accepted in AAAI 2024.

09/2023: One paper is accepted in NeurIPS 2023.

02/2023: Publish a preprint about tensorial neural networks with a collection on the web.

05/2022: One paper is accepted in ICML 2022.

02/2022: Publish a Latex template paperlighter.sty for writing papers in a simple way.

Service

Reviewer, NeurIPS 2020-2023

Reviewer, ICML 2021-2024

Reviewer, ICLR 2022-2024

Selected Publications (* denotes equal contribution)

	Preparing Lessons for Progressive Training on Language Models Yu Pan, Ye Yuan, Yichun Yin, Jiaxin Shi, Zenglin Xu, Ming Zhang, Lifeng Shang, Xin Jiang, Qun Liu AAAI, 2024 (Oral, Top 10%) arXiv Accelerating the pretraining of language models by employing LVPS to prelearn the functionalities of deeper layers at a reduced resource cost.
	Reusing Pretrained Models by Multi-linear Operators for Efficient Training Yu Pan, Ye Yuan, Yichun Yin, Zenglin Xu, Lifeng Shang, Xin Jiang, Qun Liu NeurIPS, 2023 arXiv Utilizing tensor ring matrix product operator (TR-MPO) to grow a small pretrained model to a large counterpart for efficient training.
	Tensor Networks Meet Neural Networks: A Survey and Future Perspectives Yu Pan, Maolin Wang, Zenglin Xu, Xiangli Yang, Guangxi Li, Andrzej Cichocki Preprint, 2023 arXiv / code A thoroughly investigated survey for tensorial neural networks (TNNs) on network compression, information fusion and quantum circuit simulation.
	A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks Yu Pan, Zeyong Su, Ao Liu, Jingquan Wang, Nannan Li, Zenglin Xu ICML, 2022 abs / slide / arXiv Calculating suitable variances of weights for arbitrary Tensorial Convolutional Neural Networks (TCNNs).
	RegNet: Self-Regulated Network for Image Classification Jing Xu, Yu Pan, Xinglin Pan, Kun Bai, Steven Hoi, Zhang Yi, Zenglin Xu TNNLS, 2022 abs / arXiv Applying recurrent neural networks (RNNs) to regulate convolutional neural networks (CNNs) for performance improvement.
	TedNet: A Pytorch Toolkit for Tensor Decomposition Networks Yu Pan, Maolin Wang, Zenglin Xu Neurocomputing, 2022 abs / arXiv / code A toolkit named TedNet for giving a flexible way to construct Tensor Decomposition Networks (TDNs).
	Heuristic Rank Selection with Progressively Searching Tensor Ring Network Yu Pan, Nannan Li, Yaran Chen, Zixiang Ding, Dongbin Zhao, Zenglin Xu Complex & Intelligent Systems, 2021 abs / arXiv Applying Genetic Algorithm (GA) to search tensor ring based deep models.
	Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition Yu Pan, Jing Xu, Maolin Wang, Jinmian Ye, Fei Wang, Kun Bai, Zenglin Xu AAAI, 2019 abs / arXiv / code Utilizing tensor ring decomposition for compressing recurrent neural networks (RNNs) by factorizing the input-to-hidden layer.

Source code from this website.