Qwen2.5运行无法成功

[liuchuan01]

标准示例：

环境：

soc环境
transformers：4.45.2
torch： 2.5.1
LLM-TPU：commit db27775 (HEAD -> main, origin/main, origin/HEAD)
Author: yi.chu <[email protected]>
Date: Mon Jan 13 20:50:07 2025 +0800

[language_model] run bmodel successfully in pcie

tpu-mlir：? 1684x默认，项目12月14日pull
driver版本：0.4.9 LTS
libsophon：#1 SMP Wed May 22 10:11:21 CST 2024

路径：

/workspace/LLM-TPU/models/Qwen2_5/python_demo

操作：

#!/bin/bash
set -ex

if ls ./python_demo/*cpython*.so 1> /dev/null 2>&1; then
  echo "cpython.so exists!"
else
  pushd python_demo
  rm -rf build && mkdir build && cd build
  cmake .. && make
  cp *cpython* ..
  popd
fi

# run demo
echo $PWD
python  python_demo/pipeline.py --model /data/bmodels/qwen2.5-7b_int4_seq8192_1dev.bmodel --tokenizer ./support/token_config --devid 0

问题：

模型使用的是官方的，下载到另一个目录中

+ ls './python_demo/*cpython*.so'
+ pushd python_demo
~/torchv/LLM-TPU/models/Qwen2_5/python_demo ~/torchv/LLM-TPU/models/Qwen2_5
+ rm -rf build
+ mkdir build
+ cd build
+ cmake ..
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
SoC mode, starting......
-- Found PythonInterp: /root/anaconda3/bin/python (found suitable version "3.12.7", minimum required is "3.7")
-- Found PythonLibs: /root/anaconda3/lib/libpython3.12.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /home/linaro/.local/include (found version "2.13.6")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/linaro/torchv/LLM-TPU/models/Qwen2_5/python_demo/build
+ make
Scanning dependencies of target chat
[ 50%] Building CXX object CMakeFiles/chat.dir/chat.cpp.o
In file included from /home/linaro/torchv/LLM-TPU/models/Qwen2_5/python_demo/chat.cpp:26:
/home/linaro/torchv/LLM-TPU/models/Qwen2_5/python_demo/../support/include/utils.h:16:10: fatal error: cnpy.h: No such file or directory
   16 | #include "cnpy.h"
      |          ^~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/chat.dir/build.make:63: CMakeFiles/chat.dir/chat.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/chat.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

[liuchuan01]

外部运行./run.sh --model qwen2.5-7b也一样的错误
./run.sh --model llama2-7b就没问题

[liuchuan01]

前面的问题是项目缺少了cnpy库，解决方法：

克隆 cnpy 仓库

git clone https://github.com/rogersce/cnpy.git
cd cnpy

编译安装

mkdir build
cd build
cmake ..
make
sudo make install
sudo ldconfig # 更新动态链接库缓存
cd ../.. # 回到 python_demo 目录

清理之前的构建

rm -rf build
mkdir build
cd build

重新构建

cmake ..
make
cp cpython ..
cd ..

解决完后又出现：
[BMRT][get_bdc_cmd_len:249] FATAL:BMRT_ASSERT: 0
python: /home/linaro/LLM-TPU/models/Qwen2_5/python_demo/chat.cpp:125: void Qwen::init(const std::vector&, std::string): Assertion `true == ret' failed.
./run_demo2.sh: line 30: 2045 Aborted /root/anaconda3/bin/python python_demo/pipeline.py --model ../../bmodels/qwen2.5-7b_int4_seq8192_1dev.bmodel --tokenizer ./support/token_config --devid 0

[chuxiaoyi2023]

[BMRT][get_bdc_cmd_len:249] FATAL:BMRT_ASSERT: 0

这个报错挺奇怪的，没遇到过

看了一下的bm-smi版本，是0.4.9 LTS

有点太古老了，换成0.5.0