[Bug] LongCite-glm4-9b awq quantization error

[maxin9966]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

transformer.encoder.layers.38 smooth weight done.
Traceback (most recent call last):
File "/home/ma/miniconda3/envs/lmdeploy/bin/lmdeploy", line 8, in
sys.exit(run())
File "/home/ma/miniconda3/envs/lmdeploy/lib/python3.9/site-packages/lmdeploy/cli/entrypoint.py", line 36, in run
args.run(args)
File "/home/ma/miniconda3/envs/lmdeploy/lib/python3.9/site-packages/lmdeploy/cli/lite.py", line 126, in auto_awq
auto_awq(**kwargs)
File "/home/ma/miniconda3/envs/lmdeploy/lib/python3.9/site-packages/lmdeploy/lite/apis/auto_awq.py", line 116, in auto_awq
smooth_layers(layers, fc2fcs, norm2fcs, act_scales, w_group_size,
File "/home/ma/miniconda3/envs/lmdeploy/lib/python3.9/site-packages/lmdeploy/lite/quantization/awq.py", line 277, in smooth_layers
smooth_ln_fcs(ln, fcs, a_scales[a_name], group_size)
File "/home/ma/miniconda3/envs/lmdeploy/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/ma/miniconda3/envs/lmdeploy/lib/python3.9/site-packages/lmdeploy/lite/quantization/awq.py", line 134, in smooth_ln_fcs
assert torch.isnan(p).sum() == 0
AssertionError

Reproduction

CUDA_VISIBLE_DEVICES=3 lmdeploy lite auto_awq /home/ma/work/modelscope/LongCite-glm4-9b --work-dir /home/ma/work/modelscope/LongCite-glm4-9b-4bit

Environment

0.6.0

Error traceback

No response

[AllentDan]

We did not test the model LongCite-glm4-9b. May try #1844