Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
hahuyhoang411 committed Nov 22, 2023
1 parent 763b3a3 commit eab61b2
Show file tree
Hide file tree
Showing 4 changed files with 23 additions and 9,876 deletions.
6 changes: 2 additions & 4 deletions docs/docs/new/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ curl https://api.openai.com/v1/chat/completions \

</div>

> [Why is there a `gpt-3.5-turbo` model on Nitro?](docs#faq)
- **Extends OpenAI's API with helpful model methods:**
- [Unload model](features/load-unload#unload-model)
- [Checking model status](features/load-unload/#status)
Expand Down Expand Up @@ -119,7 +121,3 @@ Nitro welcomes contributions in various forms, not just coding. Here are some wa

- [drogon](https://github.com/drogonframework/drogon): The fast C++ web framework
- [llama.cpp](https://github.com/ggerganov/llama.cpp): Inference of LLaMA model in pure C/C++

## FAQ
:::info COMING SOON
:::
20 changes: 20 additions & 0 deletions docs/docs/new/faq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: FAQs
slugs: /faq
---

### 1. Is Nitro the same as Llama.cpp with an API server?

Yes, that's correct. However, Nitro isn't limited to just Llama.cpp; it will soon integrate multiple other models like Whisper, Bark, and Stable Diffusion, all in a single binary. This eliminates the need for you to develop a separate API server on top of AI models. Nitro is a comprehensive solution, designed for ease of use and efficiency.

### 2. Is Nitro simply Llama-cpp-python?

Indeed, Nitro isn't bound to Python, which allows you to leverage high-performance software that fully utilizes your system's capabilities. With Nitro, learning how to deploy a Python web server or use FastAPI isn't necessary. The Nitro web server is already fully optimized.

### 3. Why should I switch to Nitro over Ollama?

While Ollama does provide similar functionalities, its design serves a different purpose. Ollama has a larger size (around 200MB) compared to Nitro's 3MB distribution. Nitro's compact size allows for easy embedding into subprocesses, ensuring minimal concerns about package size for your application. This makes Nitro a more suitable choice for applications where efficiency and minimal resource usage are key.

### 4. Why is the model named "chat-gpt-3.5"?

Many applications implement the OpenAI ChatGPT API, and we want Nitro to be versatile for any AI client. While you can use any model name, we've ensured that if you're already using the chatgpt API, switching to Nitro is seamless. Just replace api.openai.com with localhost:3928 in your client settings (like Chatbox, Sillytavern, Oobaboga, etc.), and it will work smoothly with Nitro.
Loading

0 comments on commit eab61b2

Please sign in to comment.