Non-None values for the URL parameter of download_model(...) method always leads to HTTP/404 error ...

[nmvega]

Hello Friends:

Using gpt4all, I wrote a script to download GGUF files from gpt4all.io and huggingface.co. It works, but not completely.

According to the SDK reference documentation, and in particular lines 402 and 403 of the download_model() method, I should be able to provide:

• model_name: (Example: Meta-Llama-3-8B-Instruct.Q4_0), and
• url: (Example: https://gpt4all.io/models/gguf/Meta-Llama-3-8B-Instruct.Q4_0.gguf

But when I specify any value for url that is not None, as in the example above, I get a HTTP/404 error response.

I've tried everything I can think of but cannot overcome this problem. I need the url so the source repository for the GGUF file can be specified - but it's not working.

You can easily test this by commenting / uncommenting exactly one line below. The following line, when commented out (i.e., url is not None), allows the error to emerge. When not commented out (i.e., url not None), the script succeeds, but without being able to specify a URL - which is the reason for this script.

• repo_base_url = None # COMMENT / UNCOMMENT to see HTTP/404 issue. <======== COMMENT / UNCOMMENT.

Under a separate cover:

• Is there any way to instantiate GPT4All("model.guff") and not trigger an immediate download?

Any ideas? Thank you in advance! =:)

SCRIPT:

from gpt4all import GPT4All
from pathlib import Path
import pprint

def manage_model_download():
    """
    Handles the process of setting up the download cache-path, selecting a
    GGUF repository and downloading a user-specified model in GGUF format.
    """

    # =================================================================
    # Define repository list.
    # =================================================================
    REPO_LIST = [("gpt4all", "https://gpt4all.io/models/gguf"),
                 ("HuggingFace", "https://huggingface.co"),]

    # =================================================================
    # Setup cache directory.
    # =================================================================
    cache_dir = Path.home() / ".cache" / "gpt4all"
    cache_dir.mkdir(parents=True, exist_ok=True)

    # =================================================================
    # Get model selection from user.
    # =================================================================
    model_name = input("""Enter name of model to download"""
                       """[Default: 'Meta-Llama-3-8B-Instruct.Q4_0.gguf']: """).strip()
    if not model_name:
        model_name = "Meta-Llama-3-8B-Instruct.Q4_0.gguf"
    elif not model_name.endswith(".gguf"):
        model_name += ".gguf"

    # =================================================================
    # Select model repository.
    # =================================================================
    while True:
        print("Select the number for the model's repository:")
        for idx, (name, _) in enumerate(REPO_LIST, start=1):
            print(f"  {idx}. {name}")
        choice = input("Enter the number: ")
        if choice.isdigit() and 1 <= int(choice) <= len(REPO_LIST):
            repo_base_url = REPO_LIST[int(choice) - 1][1]
            break
        print("Invalid choice. Please try again.")

    # =================================================================
    # Attempt to download the model.
    # =================================================================
    print(f"\nDownloading model: {model_name}\nFrom URL: {repo_base_url}/{model_name}\n")
    try:
        repo_base_url = None # COMMENT / UNCOMMENT to see HTTP/404 issue. <======== COMMENT / UNCOMMENT.
        gpt4all = GPT4All(model_name, allow_download=True, device="cuda")
        result = gpt4all.download_model(model_name, url=repo_base_url, verbose=True, model_path=cache_dir)
        pprint.pprint(result)
        print(f"Model '{model_name}' downloaded successfully to {cache_dir}")
    except Exception as e:
        print(f"An error occurred during download: {e}")

if __name__ == "__main__":
    manage_model_download()