Zhenheng Tang

Zhenheng Tang

PostDoc Researcher in Computer Science, The Hong Kong University of Science and Technology

Full Publication List

The * represents equal contribution, 📧 corresponding author.

  • K. Lai*, Z. Tang*, X. Pan, P. Dong, X. Liu, H. Chen, L. Shen, B. Li, X. Chu. Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing. Arxiv 2025.

  • X. Liu*, Z. Tang*, P. Dong, Z. Li, B. Li, X. Hu, X. Chu. ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference. Arxiv 2025.

  • X. Liu, Z. Tang, H. Chen, P. Dong, Z. Li, X. Zhou, B. Li, X. Hu, X. Chu. Can LLMs Maintain Fundamental Abilities under KV Cache Compression? Arxiv 2025.

  • L. Shen*, Z. Tang*, L. Wu, Y. Zhang, X. Chu, T. Qin, B. Han. Hot-pluggable Federated Learning: briding General and Personalized FL via Dynamic Selection. In ICLR 2025.

  • Z. Tang, X. Liu, Q. Wang, P. Dong, B. He, X. Chu, B. Li. The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve? ICLR Blogpost Track 2025.

  • Q. Wang, Z. Tang, B. He. Can LLM Simulations Truly Reflect Humanity? A Deep Dive. ICLR Blogpost Track 2025.

  • P. Dong, L. Li, Y. Zhong, D. Du, R. Fan, Y. Chen, Z. Tang, Q. Wang, W. Xue, Y. Guo, X. Chu. STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs. In ICLR 2025.

  • P. Dong, L. Li, Z. Tang, X. Liu, Z. Wei, Q. Wang, X. Chu. ParZC: Parametric Zero-Cost Proxies for Efficient NAS. In AAAI 2025.

  • X. Pan, W. Lin, L. Zhang, S. Shi, Z. Tang, R. Wang, B. Li, X. Chu. FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models. In ASPLOS 2025.

  • Z. Tang, Y. Zhang, P. Dong, Y. Cheung, A. C. Zhou, B. Han, X. Chu. FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion.” In NeurIPS 2024 (spotlight).

  • L. Shen*, Z. Tang*💡 , L. Wu, Y. Zhang, X. Chu, T. Qin, B. Han. “Hot Pluggable Federated Learning.” In Workshop of Federated Foundation Models@NeurIPS 2024 (Oral, Outstanding Student Paper Award).

  • L. Li, P. Dong, Z. Tang, X. Liu, Q. Wang, W. Luo, W. Xue, Q. Liu, X. Chu, Y. Guo. Discovering Sparsity Allocation for Layer-wise Pruning of Large Language Models. In NeurIPS 2024.

  • Q. Li, X. Liu, Z. Tang, P. Dong, Z. Li, X. Pan, X. Chu. Should We Really Edit Language Models? On the Evaluation of Edited Language Models. In NeurIPS 2024.

  • P. Dong, L. Li, X. Liu, Z. Tang, X. Liu, Q. Wang, X. Chu. LPZero: Language Model Zero-cost Proxy Search from Zero. In EMNLP 2024 Findings.

  • Z. Tang, J. Huang, R. Yan, Y. Wang, Z. Tang💡📧, S. Shi, A. C. Zhou, X. Chu📧. Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning. In ICPP 2024.

  • P. Dong, L. Li, Z. Tang, X. Liu, X. Pan, Q. Wang, X. Chu📧. Evolving Symbolic Pruning Metric From Scratch for Large Language Models. In ICML 2024.

  • Y. Tang, P. Dong, Z. Tang, X. Chu, J. Liang📧. VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting. In CVPR Workshop 2024.

  • Z. Tang, Y. Zhang, S. Shi, X. Tian, T. Liu, B. Han, X.📧 Chu. FedImpro: Measuring and Improving Client Update in Federated Learning. In ICLR 2024.

  • Y. Wang, Y. Chen, Z. Li, Z. Tang, R. Guo, X. Wang, Q. Wang, AC Zhou, X. Chu📧. BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems. arXiv preprint arXiv:2401.17644.

  • Y. Wang, S. Shi, X. He, Z. Tang, X. Pan, Y. Zheng, X. Wu, AC Zhou, B. He, X. Chu📧. Reliable and Efficient In-Memory Fault Tolerance of Large Language Model Pretraining. arXiv preprint arXiv:2310.12670.

  • Z. Tang, Y. Wang, X. He, L. Zhang, X. Pan, Q. Wang, R. Zeng, K. Zhao, S. Shi📧, B. He, X. Chu📧. FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs. In IJCAI-LLM Workshop 2023.09.,

  • Z. Tang, S. Shi, B. Li, X. Chu📧. GossipFL: A Decentralized Federated Learning Framework with Sparsified and Adaptive Communication. In IEEE Transactions on Parallel and Distributed Systems, 2022.

  • X. He, J. Yao, Y. Wang, Z. Tang, C. K. Chun, S. Simon, B. Han, and X. Chu📧. NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension. In AAAI 2023.

  • Z. Tang, Y. Zhang*, S. Shi, X. He, B. Han, X. Chu📧. Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning. In Proceedings of the 39th International Conference on Machine Learning, 2022.

  • C. He, A. D. Shah, Zhenheng Tang, D. Fan, A. N. Sivashunmugam, K. Bhogaraju, M. Shimpi, L. Shen, X. Chu, M. Soltanolkotabi and S. Avestimehr. FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks. In FL-AAAI-22 workshop, 2022. [Best Paper Award]

  • Z. Liao, H. Yan, Z. Tang, X. Chu, T. Tao📧. Deep learning identifies leak in water pipeline system using transient frequency response. In Process Safety and Environmental Protection 2021.

  • Z. Tang, Zhikai Hu, Shaohuai Shi, Yiu-ming Cheung, Yilun Jin, Zhenghang Ren, Xiaowen Chu📧. Data Resampling for Federated Learning with Non-IID Labels. In FTL-IJCAI workshop, 2021.

  • S. Shi, Z. Tang, X. Chu, C. Liu, W. Wang, and B. Li. A quantitative surveyof communication optimizations in distributed deep learning. IEEE Network,35(3):230–237, 2021.

  • S. Shi, Z. Tang, Q. Wang, K. Zhao, and X. Chu. Layer-wise adaptive gradientsparsification for distributed deep learning with convergence guarantees. In ECAI 2020 * 24th European Conference on Artificial Intelligence, pages 1467–1474. IOS Press, 2020.

  • Z. Tang, S. Shi, and X. Chu📧. Communication-efficient decentralized learning withsparsification and adaptive peer selection. In ICDCS 2020.

  • Y. Wang, Q. Wang, S. Shi, X. He, Z. Tang, K. Zhao, X. Chu. Benchmarking the Performance and Energy Efficiency of AI Accelerators for AI Training. In CCGRID 2020.

  • Z. Tang, Y. Wang, Q. Wang, and X. Chu. The impact of gpu dvfs on the energy andperformance of deep learning: An empirical study. In Proceedings of the Tenth ACM International Conference on Future Energy Systems, e-Energy ‘19.

  • S. Shi, K. Zhao, Q. Wang, Z. Tang, and X. Chu. A convergence analysis ofdistributed sgd with communication-efficient gradient sparsification. IJCAI-19.

  • S. Shi, Q. Wang, K. Zhao, Z. Tang, Y. Wang, X. Huang, and X. Chu. A distributedsynchronous sgd algorithm with global top-k sparsification for low bandwidthnetworks. In ICDCS 2019.

  • X. Zhou, Z. Tang, W. Xu, F. Meng, X. Chu, K. Xin, and G. Fu📧. Deep learningidentifies accurate burst locations in water distribution networks. Water Research,166:115058, 2019.

  • X. He, S. Wang, S. Shi, Z. Tang, Y. Wang, Z. Zhao, J. Dai, R. Ni, X. Zhang, X. Liu,Z. Wu, W. Yu, and X. Chu. Computer-aided clinical skin disease diagnosis usingcnn and object detection models. In 2019 IEEE International Conference on BigData (Big Data), pages 4839–4844, 2019.

Preprint

  • Z. Tang, X. Chu, R. Ran, S. Lee, S. Shi, Y. Zhang, Y. Wang, A. Liang, S. Avestimehr, C. He📧. FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training. arXiv preprint arXiv:2303.01778.

  • Z. Tang, S. Shi, W. Wang, B. Li and X. Chu📧. Communication-efficient distributeddeep learning: A comprehensive survey. CoRR, abs/2003.06307, 2020.