Skip to content

Commit

Permalink
deploy: 5876a47
Browse files Browse the repository at this point in the history
  • Loading branch information
@c121914yu
c121914yu committed Jan 9, 2024
1 parent 42c261e commit 35969a7
Show file tree
Hide file tree
Showing 6 changed files with 47 additions and 47 deletions.
2 changes: 1 addition & 1 deletion docs/development/custom-models/reranker/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
<span class="material-icons size-20 menu-icon align-middle">menu</span></button>
<span><div id=docsearch></div></span></div><div class="d-flex align-items-center"><ul class="list-unstyled mb-0"><li class="list-inline-item mb-0"><a href=https://github.com/labring/FastGPT alt=github rel="noopener noreferrer" target=_blank><div class="btn btn-icon btn-default border-0"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentcolor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>GitHub</title><path d="M9 19c-5 1.5-5-2.5-7-3m14 6v-3.87a3.37 3.37.0 00-.94-2.61c3.14-.35 6.44-1.54 6.44-7A5.44 5.44.0 0020 4.77 5.07 5.07.0 0019.91 1S18.73.65 16 2.48a13.38 13.38.0 00-7 0C6.27.65 5.09 1 5.09 1A5.07 5.07.0 005 4.77 5.44 5.44.0 003.5 8.55c0 5.42 3.3 6.61 6.44 7A3.37 3.37.0 009 18.13V22"/></svg></div></a></li><li class="list-inline-item mb-0"><a href=https://oss.laf.run/htr4n1-images/fastgpt-qr-code.jpg alt=wechat rel="noopener noreferrer" target=_blank><div class="btn btn-icon btn-default border-0"><svg xmlns="http://www.w3.org/2000/svg" width="28" height="28" viewBox="0 0 28 28" fill="none" stroke="currentcolor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>微信群</title><path d="M21.6696 15.9962C21.9464 15.9925 22.2107 15.881 22.4064 15.6853 22.6021 15.4896 22.7137 15.2252 22.7173 14.9485c0-.581-.465500000000002-1.0477-1.0477-1.0477C21.0875 13.9008 20.622 14.3675 20.622 14.9485c0 .583300000000001.4666 1.0477 1.0476 1.0477zm-5.1625.0C16.7839 15.9925 17.0482 15.881 17.2439 15.6853S17.5512 15.2252 17.5548 14.9485c0-.581-.466699999999999-1.0477-1.0477-1.0477C15.9238 13.9008 15.4595 14.3675 15.4595 14.9485c0 .583300000000001.4655 1.0477 1.0476 1.0477zm7.6615 5.88C24.0939 21.9177 24.0342 21.9818 23.9981 22.0594 23.9621 22.1369 23.9515 22.2239 23.968 22.3078 23.968 22.3638 23.968 22.421 23.9971 22.4793c.1144.486500000000003.343 1.2612.343 1.2904C24.3401 23.8548 24.3693 23.912 24.3693 23.9703 24.3693 24.0041 24.3626 24.0376 24.3496 24.0688 24.3367 24.1 24.3177 24.1284 24.2937 24.1522 24.2697 24.1761 24.2413 24.195 24.21 24.2078 24.1788 24.2206 24.1453 24.2272 24.1115 24.227 24.0531 24.227 24.0251 24.199 23.968 24.171L22.2775 23.1957C22.154 23.1277 22.0168 23.0886 21.8761 23.0813 21.791 23.0813 21.7046 23.0813 21.6475 23.1093 20.8448 23.3392 20.0141 23.4535 19.1263 23.4535c-4.27.0-7.7082-2.8665-7.7082-6.4225.0-3.5548 3.4382-6.4225 7.7082-6.4225 4.2688.0 7.707 2.8677 7.707 6.4225.0 1.9215-1.0313 3.6703-2.6647 4.8463V21.8762zM19.4518 9.44883C19.3429 9.44521 19.234 9.44327 19.1251 9.443c-4.8696.0-8.8736 3.3367-8.8736 7.5892.0.6463.0932999999999993 1.2716.266 1.8666H10.4136C9.39426 18.8885 8.38067 18.7442 7.39896 18.4695 7.31263 18.4403 7.22629 18.4403 7.13996 18.4403 6.96754 18.444 6.79921 18.4935 6.65229 18.5838L4.61296 19.7563C4.55463 19.7855 4.49746 19.8147 4.44029 19.8147 4.35703 19.8138 4.27744 19.7803 4.21856 19.7214 4.15968 19.6625 4.1262 19.5829 4.12529 19.4997 4.12529 19.4133 4.15329 19.3562 4.18246 19.2698 4.21046 19.2418 4.46946 18.2968 4.61296 17.7252 4.61296 17.6668 4.64096 17.5817 4.64096 17.5245 4.64065 17.4245 4.61721 17.3259 4.57248 17.2364 4.52775 17.147 4.46295 17.0691 4.38313 17.0088c-1.98217-1.4035-3.2165-3.4918-3.2165-5.8111.0-4.26187 4.165-7.6977 9.27497-7.6977 4.3925.0 8.085 2.5305 9.0102 5.94767V9.44883zM13.4388 9.9365c.6685.0 1.197-.5565 1.197-1.197.0-.6685-.528499999999999-1.197-1.197-1.197S12.2418 8.071 12.2418 8.7395 12.7703 9.9365 13.4388 9.9365zm-6.13667.0c.6685.0 1.19816-.5565 1.19816-1.197.0-.6685-.52966-1.197-1.19816-1.197-.66734.0-1.197.528499999999999-1.197 1.197s.52966 1.197 1.197 1.197z"/></svg></div></a></li></ul><button id=mode class="btn btn-icon btn-default ms-2" type=button aria-label="Toggle user interface mode">
<span class=toggle-dark><svg xmlns="http://www.w3.org/2000/svg" height="30" width="30" viewBox="0 0 48 48" fill="currentcolor"><path d="M24 42q-7.5.0-12.75-5.25T6 24t5.25-12.75T24 6q.4.0.85.025.45.025 1.15.075-1.8 1.6-2.8 3.95t-1 4.95q0 4.5 3.15 7.65Q28.5 25.8 33 25.8q2.6.0 4.95-.925T41.9 22.3q.05.6.075.975Q42 23.65 42 24q0 7.5-5.25 12.75T24 42zm0-3q5.45.0 9.5-3.375t5.05-7.925q-1.25.55-2.675.825Q34.45 28.8 33 28.8q-5.75.0-9.775-4.025T19.2 15q0-1.2.25-2.575t.9-3.125q-4.9 1.35-8.125 5.475Q9 18.9 9 24q0 6.25 4.375 10.625T24 39zm-.2-14.85z"/></svg></span><span class=toggle-light><svg xmlns="http://www.w3.org/2000/svg" height="30" width="30" viewBox="0 0 48 48" fill="currentcolor"><path d="M24 31q2.9.0 4.95-2.05T31 24t-2.05-4.95T24 17t-4.95 2.05T17 24t2.05 4.95T24 31zm0 3q-4.15.0-7.075-2.925T14 24t2.925-7.075T24 14t7.075 2.925T34 24t-2.925 7.075T24 34zM3.5 25.5q-.65.0-1.075-.425Q2 24.65 2 24t.425-1.075Q2.85 22.5 3.5 22.5h5q.65.0 1.075.425Q10 23.35 10 24t-.425 1.075T8.5 25.5zm36 0q-.65.0-1.075-.425Q38 24.65 38 24t.425-1.075T39.5 22.5h5q.65.0 1.075.425Q46 23.35 46 24t-.425 1.075-1.075.425zM24 10q-.65.0-1.075-.425Q22.5 9.15 22.5 8.5v-5q0-.65.425-1.075Q23.35 2 24 2t1.075.425T25.5 3.5v5q0 .65-.425 1.075Q24.65 10 24 10zm0 36q-.65.0-1.075-.425T22.5 44.5v-5q0-.65.425-1.075Q23.35 38 24 38t1.075.425.425 1.075v5q0 .65-.425 1.075Q24.65 46 24 46zM12 14.1l-2.85-2.8q-.45-.45-.425-1.075.025-.625.425-1.075.45-.45 1.075-.45t1.075.45L14.1 12q.4.45.4 1.05.0.6-.4 1-.4.45-1.025.45T12 14.1zm24.7 24.75L33.9 36q-.4-.45-.4-1.075t.45-1.025q.4-.45 1-.45t1.05.45l2.85 2.8q.45.45.425 1.075-.025.625-.425 1.075-.45.45-1.075.45t-1.075-.45zM33.9 14.1q-.45-.45-.45-1.05.0-.6.45-1.05l2.8-2.85q.45-.45 1.075-.425.625.025 1.075.425.45.45.45 1.075t-.45 1.075L36 14.1q-.4.4-1.025.4t-1.075-.4zM9.15 38.85q-.45-.45-.45-1.075t.45-1.075L12 33.9q.45-.45 1.05-.45.6.0 1.05.45.45.45.45 1.05.0.6-.45 1.05l-2.8 2.85q-.45.45-1.075.425-.625-.025-1.075-.425zM24 24z"/></svg></span></button></div></div></div><div class=container-fluid><div class=layout-spacing><div class="d-md-flex justify-content-between align-items-center"><nav aria-label=breadcrumb class="d-inline-block pb-2 mt-1 mt-sm-0"><ul id=breadcrumbs class="breadcrumb bg-transparent mb-0" itemscope itemtype=https://schema.org/BreadcrumbList><li class="breadcrumb-item text-capitalize active" aria-current=page itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><a itemprop=item href=/docs/><i class="material-icons size-20 align-text-bottom" itemprop=name>Home</i></a><meta itemprop=position content="1"></li><li class="breadcrumb-item text-capitalize" itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><a itemprop=item href=/docs/development/><span itemprop=name>开发与部署指南</span></a><meta itemprop=position content="2"></li><li class="breadcrumb-item text-capitalize" itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><a itemprop=item href=/docs/development/custom-models/><span itemprop=name>本地模型使用</span></a><meta itemprop=position content="3"></li><li class="breadcrumb-item text-capitalize active" itemprop=itemListElement itemscope itemtype=https://schema.org/ListItem><span itemprop=name>接入 ReRank 重排模型</span><meta itemprop=position content="4"></li></ul></nav></div><div class="row flex-xl-nowrap"><div class="docs-toc col-xl-3 d-xl-block"><toc><div class="fw-bold text-uppercase mb-2">接入 ReRank 重排模型</div><nav id=toc><ul><li><ul><li><a href=#推荐配置>推荐配置</a></li><li><a href=#部署>部署</a><ul><li><a href=#环境要求>环境要求</a></li><li><a href=#源码部署>源码部署</a></li><li><a href=#docker-部署>docker 部署</a></li></ul></li><li><a href=#接入-fastgpt>接入 FastGPT</a></li></ul></li></ul></nav></toc></div><div class="docs-toc-mobile d-print-none d-xl-none"><button id=toc-dropdown-btn class="btn-secondary dropdown-toggle" type=button data-bs-toggle=dropdown data-bs-offset=0,0 aria-expanded=false>
Table of Contents</button><nav id=toc-mobile><ul class=dropdown-menu><li><ul><li><a href=#推荐配置>推荐配置</a></li><li><a href=#部署>部署</a><ul><li><a href=#环境要求>环境要求</a></li><li><a href=#源码部署>源码部署</a></li><li><a href=#docker-部署>docker 部署</a></li></ul></li><li><a href=#接入-fastgpt>接入 FastGPT</a></li></ul></li></ul></nav></div><div class="docs-content col-12 col-xl-9 mt-0"><div class="mb-0 d-flex"><i class="material-icons title-icon me-2">sort</i><h1 class="content-title mb-0">接入 ReRank 重排模型</h1></div><p class="lead mb-3">接入 ReRank 重排模型</p><div id=content class=main-content data-bs-spy=scroll data-bs-root-margin="0px 0px -65%" data-bs-target=#toc-mobile><div data-prismjs-copy data-prismjs-copy-success data-prismjs-copy-error><h2 id=推荐配置>推荐配置 <a href=#%e6%8e%a8%e8%8d%90%e9%85%8d%e7%bd%ae class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><p>推荐配置如下：</p><table class="table table-hover table-striped-columns"><thead><tr><th>类型</th><th>内存</th><th>显存</th><th>硬盘空间</th><th>启动命令</th></tr></thead><tbody><tr><td>base</td><td>>=4GB</td><td>>=3GB</td><td>>=8GB</td><td>python app.py</td></tr></tbody></table><h2 id=部署>部署 <a href=#%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><h3 id=环境要求>环境要求 <a href=#%e7%8e%af%e5%a2%83%e8%a6%81%e6%b1%82 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ul><li>Python 3.10.11</li><li>CUDA 11.7</li><li>科学上网环境</li></ul><h3 id=源码部署>源码部署 <a href=#%e6%ba%90%e7%a0%81%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ol><li>根据上面的环境配置配置好环境，具体教程自行 GPT；</li><li>下载 <a href=app.py>python 文件</a></li><li>在命令行输入命令 <code>pip install -r requirments.txt</code>；</li><li>按照<a href=https://huggingface.co/BAAI/bge-reranker-base rel=external target=_blank>https://huggingface.co/BAAI/bge-reranker-base<svg width="16" height="16" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="currentcolor" d="M14 5c-.552.0-1-.448-1-1s.448-1 1-1h6c.552.0 1 .448 1 1v6c0 .552-.448 1-1 1s-1-.448-1-1V6.414l-7.293 7.293c-.391.39-1.024.39-1.414.0-.391-.391-.391-1.024.0-1.414L17.586 5H14zM5 7c-.552.0-1 .448-1 1v11c0 .552.448 1 1 1h11c.552.0 1-.448 1-1v-4.563c0-.552.448-1 1-1s1 .448 1 1V19c0 1.657-1.343 3-3 3H5c-1.657.0-3-1.343-3-3V8c0-1.657 1.343-3 3-3h4.563c.552.0 1 .448 1 1s-.448 1-1 1H5z"/></svg></a>下载模型仓库到app.py同级目录</li><li>添加环境变量 <code>export ACCESS_TOKEN=XXXXXX</code> 配置 token，这里的 token 只是加一层验证，防止接口被人盗用，默认值为 <code>ACCESS_TOKEN</code> ；</li><li>执行命令 <code>python app.py</code>。</li></ol><p>然后等待模型下载，直到模型加载完毕为止。如果出现报错先问 GPT。</p><p>启动成功后应该会显示如下地址：</p><p><img src=https://cdn.jsdelivr.us/gh/yangchuansheng/fastgpt-imgs@main/imgs/chatglm2.png alt loading=lazy class=medium-zoom-image></p><blockquote><p>这里的 <code>http://0.0.0.0:6006</code> 就是连接地址。</p></blockquote><h3 id=docker-部署>docker 部署 <a href=#docker-%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ul><li>镜像名: <code>luanshaotong/reranker:v0.1</code></li><li>端口号: 6006</li><li>大小：约8GB</li></ul><p><strong>设置安全凭证（即oneapi中的渠道密钥）</strong></p><div class=prism-codeblock><pre id=e45f452 class=language->
Table of Contents</button><nav id=toc-mobile><ul class=dropdown-menu><li><ul><li><a href=#推荐配置>推荐配置</a></li><li><a href=#部署>部署</a><ul><li><a href=#环境要求>环境要求</a></li><li><a href=#源码部署>源码部署</a></li><li><a href=#docker-部署>docker 部署</a></li></ul></li><li><a href=#接入-fastgpt>接入 FastGPT</a></li></ul></li></ul></nav></div><div class="docs-content col-12 col-xl-9 mt-0"><div class="mb-0 d-flex"><i class="material-icons title-icon me-2">sort</i><h1 class="content-title mb-0">接入 ReRank 重排模型</h1></div><p class="lead mb-3">接入 ReRank 重排模型</p><div id=content class=main-content data-bs-spy=scroll data-bs-root-margin="0px 0px -65%" data-bs-target=#toc-mobile><div data-prismjs-copy data-prismjs-copy-success data-prismjs-copy-error><h2 id=推荐配置>推荐配置 <a href=#%e6%8e%a8%e8%8d%90%e9%85%8d%e7%bd%ae class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><p>推荐配置如下：</p><table class="table table-hover table-striped-columns"><thead><tr><th>类型</th><th>内存</th><th>显存</th><th>硬盘空间</th><th>启动命令</th></tr></thead><tbody><tr><td>base</td><td>>=4GB</td><td>>=3GB</td><td>>=8GB</td><td>python app.py</td></tr></tbody></table><h2 id=部署>部署 <a href=#%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h2><h3 id=环境要求>环境要求 <a href=#%e7%8e%af%e5%a2%83%e8%a6%81%e6%b1%82 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ul><li>Python 3.10.11</li><li>CUDA 11.7</li><li>科学上网环境</li></ul><h3 id=源码部署>源码部署 <a href=#%e6%ba%90%e7%a0%81%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ol><li>根据上面的环境配置配置好环境，具体教程自行 GPT；</li><li>下载 <a href=https://github.com/labring/FastGPT/tree/main/python/reranker/bge-reranker-base rel=external target=_blank>python 文件<svg width="16" height="16" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="currentcolor" d="M14 5c-.552.0-1-.448-1-1s.448-1 1-1h6c.552.0 1 .448 1 1v6c0 .552-.448 1-1 1s-1-.448-1-1V6.414l-7.293 7.293c-.391.39-1.024.39-1.414.0-.391-.391-.391-1.024.0-1.414L17.586 5H14zM5 7c-.552.0-1 .448-1 1v11c0 .552.448 1 1 1h11c.552.0 1-.448 1-1v-4.563c0-.552.448-1 1-1s1 .448 1 1V19c0 1.657-1.343 3-3 3H5c-1.657.0-3-1.343-3-3V8c0-1.657 1.343-3 3-3h4.563c.552.0 1 .448 1 1s-.448 1-1 1H5z"/></svg></a></li><li>在命令行输入命令 <code>pip install -r requirments.txt</code>；</li><li>按照<a href=https://huggingface.co/BAAI/bge-reranker-base rel=external target=_blank>https://huggingface.co/BAAI/bge-reranker-base<svg width="16" height="16" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="currentcolor" d="M14 5c-.552.0-1-.448-1-1s.448-1 1-1h6c.552.0 1 .448 1 1v6c0 .552-.448 1-1 1s-1-.448-1-1V6.414l-7.293 7.293c-.391.39-1.024.39-1.414.0-.391-.391-.391-1.024.0-1.414L17.586 5H14zM5 7c-.552.0-1 .448-1 1v11c0 .552.448 1 1 1h11c.552.0 1-.448 1-1v-4.563c0-.552.448-1 1-1s1 .448 1 1V19c0 1.657-1.343 3-3 3H5c-1.657.0-3-1.343-3-3V8c0-1.657 1.343-3 3-3h4.563c.552.0 1 .448 1 1s-.448 1-1 1H5z"/></svg></a>下载模型仓库到app.py同级目录</li><li>添加环境变量 <code>export ACCESS_TOKEN=XXXXXX</code> 配置 token，这里的 token 只是加一层验证，防止接口被人盗用，默认值为 <code>ACCESS_TOKEN</code> ；</li><li>执行命令 <code>python app.py</code>。</li></ol><p>然后等待模型下载，直到模型加载完毕为止。如果出现报错先问 GPT。</p><p>启动成功后应该会显示如下地址：</p><p><img src=https://cdn.jsdelivr.us/gh/yangchuansheng/fastgpt-imgs@main/imgs/chatglm2.png alt loading=lazy class=medium-zoom-image></p><blockquote><p>这里的 <code>http://0.0.0.0:6006</code> 就是连接地址。</p></blockquote><h3 id=docker-部署>docker 部署 <a href=#docker-%e9%83%a8%e7%bd%b2 class=anchor aria-hidden=true><i class="material-icons align-middle">link</i></a></h3><ul><li>镜像名: <code>luanshaotong/reranker:v0.1</code></li><li>端口号: 6006</li><li>大小：约8GB</li></ul><p><strong>设置安全凭证（即oneapi中的渠道密钥）</strong></p><div class=prism-codeblock><pre id=e45f452 class=language->
<code>ACCESS_TOKEN=mytoken</code>
</pre></div><p><strong>运行命令示例</strong></p><div class=prism-codeblock><pre id=621e3ae class=language-sh>
<code>docker run -d --name reranker -p 6006:6006 -e ACCESS_TOKEN=mytoken luanshaotong/reranker:v0.1</code>
Expand Down
Loading

0 comments on commit 35969a7

Please sign in to comment.