Zhen Dong*, Dequan Wang*, Qijing Huang*, Yizhao Gao, Yaohui Cai, Tian Li, Bichen Wu, Kurt Keutzer, John Wawrzynek. “ CoDeNet: Algorithm-hardware Co-design for Deformable Convolution ,” Oral, FPGA 2021.

[16] Zhen Dong, Z. Zhou, Z.F. Li, P. Huang, L.F. Liu, X.Y. Liu, J.F. Kang. “RRAM-based Convolutional Neural Networks for High Accuracy Pattern Recognition Tasks,” [VLSI-SNW 2018], Oral Presentation.

[2] Z. Dong, Z. Yao, D. Arfeen, A. Gholami, M. Mahoney, K. Keutzer, HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks, NeurIPS 2020. Precisions for all layers are 100% automatically selected.

Yaohui Cai1, Zhewei Yao 2, Zhen Dong , Amir Gholami 2, Michael W. Mahoney , Kurt Keutzer2 1Peking University, 2University of California, Berkeley. Motivation

Zhen Dong, Yaohui Cai, Amir Gholami, Tianjun Zhang, Kurt Keutzer University of California at Berkeley, Peking University {zhendong, amirgh, tianjunz, keutzer}@berkeley.edu [email protected] 1. Contact Email ID. ZhenDong<[email protected]> 2. Description of model architecture. Model architecture OurmodelisbasedonShuffleNetv2[1] 0:5 version.

In (Dong et al. 2019), a Hessian AWare Quantization (HAWQ) is developed for mixed-bits assignments. The main idea is that the parameters in NN layers with higher Hessian spectrum (i.e., larger top eigenvalues) are more sensitive to quantization and require higher precision as compared to layers with small Hessian spectrum. However, there exist 7M

Shen Sheng, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael Mahoney, Kurt Keutzer Q-BERT: Hessian-based Quantization for BERT Hessian-based ultra-low precision quantization (down to 2-bit); Group-wise Quantization for multi-head attention model (BERT); 13x smaller model with at most 2% accuracy loss. 4th Layer 10th Layer

7. Disadvantages for OBS • Computation of Hessian Inverse • Per-parameter sensitivity measurement • Unstructured – hardware unfriendly Methodology

Nov 26, 2024 · Back To Top © Zhen Dong 2024Zhen Dong 2024

Dec 12, 2017 · Back To Top © Zhen Dong 2024Zhen Dong 2024