【ollama】（2）：在linux搭建情况，编译ollama代码，测试qwen大模子，当地运行速率飞快，本质上是对llama.cpp 项目封装

关于 ollama 项目

https://github.com/ollama/ollama/tree/main/docs
https://www.bilibili.com/video/BV1oS421w7aM/

   【ollama】（2）：在linux搭建情况，编译ollama代码，测试qwen大模子，当地运行速率飞快，本质上是对llama.cpp 项目封装

1，下载代码和子模块

1. git clone --recurse-submodules https://github.com/ollama/ollama.git
2. 正克隆到 'ollama'...
3. remote: Enumerating objects: 11260, done.
4. remote: Counting objects: 100% (494/494), done.
5. remote: Compressing objects: 100% (263/263), done.
6. remote: Total 11260 (delta 265), reused 357 (delta 190), pack-reused 10766
7. 接收对象中: 100% (11260/11260), 6.92 MiB | 57.00 KiB/s, 完成.
8. 处理 delta 中: 100% (6984/6984), 完成.
9. 子模组 'llama.cpp'（https://github.com/ggerganov/llama.cpp.git）已对路径 'llm/llama.cpp' 注册
10. 正克隆到 '/data/home/test/go/src/ollama/llm/llama.cpp'...
11. fatal: 无法访问 'https://github.com/ggerganov/llama.cpp.git/'：GnuTLS recv error (-110): The TLS connection was non-properly terminated.

复制代码

要是失败实行，子模块更新：

1. $ git submodule update
2. 正克隆到 '/data/home/test/go/src/ollama/llm/llama.cpp'...
3. remote: Enumerating objects: 12802, done.
4. remote: Counting objects: 100% (12802/12802), done.
5. remote: Compressing objects: 100% (3561/3561), done.
6. remote: Total 12483 (delta 9258), reused 12045 (delta 8826), pack-reused 0
7. 接收对象中: 100% (12483/12483), 10.19 MiB | 679.00 KiB/s, 完成.
8. 处理 delta 中: 100% (9258/9258), 完成 260 个本地对象.
9. 来自 https://github.com/ggerganov/llama.cpp
10. * branch            c2101a2e909ac7c08976d414e64e96c90ee5fa9e -> FETCH_HEAD
11. 子模组路径 'llm/llama.cpp'：检出 'c2101a2e909ac7c08976d414e64e96c90ee5fa9e'

复制代码

2，然后就可以编译了

1. go build
2. llm/payload_linux.go:7:12: pattern llama.cpp/build/linux/*/*/lib/*: no matching files found

复制代码

必要编译 llama.cpp 的代码，

1. test@thinkPadE15:~/go/src/ollama$ cd llm/generate/
2. test@thinkPadE15:~/go/src/ollama/llm/generate$ ls
3. gen_common.sh  gen_darwin.sh  generate_darwin.go  generate_linux.go  generate_windows.go  gen_linux.sh  gen_windows.ps1
5. test@thinkPadE15:~/go/src/ollama/llm/generate$ bash gen_linux.sh
7. - Build files have been written to: /data/home/test/go/src/ollama/llm/llama.cpp/build/linux/x86_64/cpu
8. + cmake --build ../llama.cpp/build/linux/x86_64/cpu --target ext_server -j8
9. [  6%] Generating build details from Git
10. [ 20%] Building C object CMakeFiles/ggml.dir/ggml-backend.c.o
11. [ 20%] Building C object CMakeFiles/ggml.dir/ggml.c.o
12. [ 20%] Building C object CMakeFiles/ggml.dir/ggml-quants.c.o
13. [ 26%] Building C object CMakeFiles/ggml.dir/ggml-alloc.c.o
14. -- Found Git: /usr/bin/git (found version "2.34.1")
15. [ 33%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
16. [ 33%] Built target build_info

复制代码

然后颠末漫长的编译，就而可以天生了在 build 文件夹。内里有相干的liib库了。

3，再实行 go build 就可以了

1. go build
3. ls -lha ollama
4. -rwxrwxr-x 1 test test 34M  3月  9 22:13 ollama

复制代码

会有几个告诫，题目不大：

1. gpu_info.h:33:23: note: in definition of macro ‘LOG’
2.    33 |       fprintf(stderr, __VA_ARGS__); \
3.       |                       ^~~~~~~~~~~
4. gpu_info_cuda.c:159:41: note: format string is defined here
5.   159 |     LOG(h.verbose, "[%d] CUDA usedMem %ld\n", i, memInfo.used);
6.       |                                       ~~^
7.       |                                         |
8.       |                                         long int
9.       |                                       %lld

复制代码

运行 qwen 模子：
https://ollama.com/library/qwen

1. ./ollama run qwen:0.5b
2. pulling manifest
3. pulling fad2a06e4cc7...  25% ▕█████████████                                           ▏  96 MB/394 MB  368 KB/s  13m28s

复制代码

框架本质上照旧利用了 llama.cpp 项目的。
测试速率飞快：

1. >>> 你好
2. 你好！有什么我可以帮您的吗？
4. >>> 北京景点推荐
5. 在北京，有很多值得一去的地方。以下是一些推荐的北京景点：
6. 1. 故宫：世界文化遗产，拥有丰富的历史文化和艺术价值。
7. 2. 长城：世界自然遗产，拥有壮丽的山峦和河流景观。
8. 3. 北京天安门广场：中国国家象征，拥有壮观的建筑景观。
9. 以上是一些推荐的北京景点。希望对您有所帮助！
11. >>> 测 你是谁
12. 我是阿里云研发的大规模语言模型“通义千问”。如果您有任何问题或需要帮助，请随时告诉我，我会尽力提供支持和解答。

复制代码

4，检察文件存储位置

2. - macOS: `~/.ollama/models`
3. - Linux: `/usr/share/ollama/.ollama/models`
4. - Windows: `C:\Users\<username>\.ollama\models`

复制代码

5，可以利用 openai api 标准接口测试

1. curl http://localhost:11434/api/chat -d '{
2.   "model": "qwen:7b",
3.   "messages": [
4.     {
5.       "role": "user",
6.       "content": "你好"
7.     }
8.   ]
9. }'
11. curl http://localhost:11434/v1/chat/completions \
12.     -H "Content-Type: application/json" \
13.     -d '{
14.         "model": "qwen:7b","stream":true,
15.         "messages": [
16.             {
17.                 "role": "user",
18.                 "content": "你好"
19.             }
20.         ]
21.     }'

复制代码

支持流返回：

1. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001611,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"},"finish_reason":null}]}
3. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001611,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"！"},"finish_reason":null}]}
5. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001612,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"有什么"},"finish_reason":null}]}
7. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001612,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"问题"},"finish_reason":null}]}
9. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001612,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"或者"},"finish_reason":null}]}
11. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001612,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"需要"},"finish_reason":null}]}
13. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001612,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"帮助"},"finish_reason":null}]}
15. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001613,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"吗"},"finish_reason":null}]}
17. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001613,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"？"},"finish_reason":null}]}
19. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001613,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"我"},"finish_reason":null}]}
21. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001613,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"在这里"},"finish_reason":null}]}
23. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001613,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}
25. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001614,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\n"},"finish_reason":null}]}
27. data: {"id":"chatcmpl-263","object":"chat.completion.chunk","created":1710001614,"model":"qwen:7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}
29. data: [DONE]

复制代码