Skip to content

Commit

Permalink
chatbox guide
Browse files Browse the repository at this point in the history
  • Loading branch information
hahuyhoang411 committed Nov 22, 2023
1 parent 6c6ec99 commit f72cc2a
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 4 deletions.
48 changes: 44 additions & 4 deletions docs/docs/examples/chatbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,50 @@
title: Nitro with Chatbox
---

:::info COMING SOON
:::
This guide demonstrates how to integrate Nitro with Chatbox, showcasing the compatibility of Nitro with various platforms.

<!--
## What is Chatbox?
Chatbox is a versatile desktop client that supports multiple cutting-edge Large Language Models (LLMs). It is available for Windows, Mac, and Linux operating systems.

## How to use Nitro as backend -->
For more information, please visit the [Chatbox official GitHub page](https://github.com/Bin-Huang/chatbox).


## Downloading and Installing Chatbox

To download and install Chatbox, follow the instructions available at this [link](https://github.com/Bin-Huang/chatbox#download).

## Using Nitro as a Backend

1. Start Nitro server

Open your command line tool and enter:
```
nitro
```

> Ensure you are using the latest version of [Nitro](new/install.md)
2. Run the Model

To load the model, use the following command:

```
curl http://localhost:3928/inferences/llamacpp/loadmodel \
-H 'Content-Type: application/json' \
-d '{
"llama_model_path": "model/llama-2-7b-chat.Q5_K_M.gguf",
"ctx_len": 512,
"ngl": 100,
}'
```

3. Config chatbox
Adjust the `settings` in Chatbox to connect with Nitro. Change your settings to match the configuration shown in the image below:

![Settings](img/chatbox.PNG)

4. Chat with the Model

Once the setup is complete, you can start chatting with the model using Chatbox. All functions of Chatbox are now enabled with Nitro as the backend.

## Video demo
Binary file added docs/docs/examples/img/chatbox.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f72cc2a

Please sign in to comment.