docs/integrations/chat/huggingface/

[giscus[bot]]

docs/integrations/chat/huggingface/

This will help you getting started with langchainhuggingface chat models. For detailed documentation of all ChatHuggingFace features and configurations head to the API reference. For a list of models supported by Hugging Face check out this page.

https://python.langchain.com/docs/integrations/chat/huggingface/

[sorooshsorkhani]

I have an issue with the output of the chat_model. I see a lot of tokens like <|eot_id|>``<|start_header_id|>``user<|end_header_id|>.
I use the code provided on this page. I had to modify it a little bit to avoid some errors.

Below is my code:

import transformers, torch
from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline

model_id = "meta-llama/Llama-3.2-1B-Instruct"
pipe = transformers.pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    max_new_tokens=2048,
    do_sample=False,
    repetition_penalty=1.03
)
pipe.tokenizer.pad_token_id = pipe.model.config.eos_token_id

model_kwargs = {'temperature':0.3}
llm = HuggingFacePipeline(pipeline = pipe, model_kwargs = model_kwargs)

chat_model = ChatHuggingFace(llm=llm)

from langchain_core.messages import (
    HumanMessage,
    SystemMessage,
)

messages = [
    SystemMessage(content="You're a helpful assistant"),
    HumanMessage(
        content="What happens when an unstoppable force meets an immovable object?"
    ),
]

ai_msg = chat_model.invoke(messages)
print(ai_msg.content)

And this is the output:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 14 Nov 2024

You're a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>

What happens when an unstoppable force meets an immovable object?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

In the context of physics, when an unstoppable force meets an immovable object, the outcome is often referred to as a "static frictional lock" or a "static frictional resistance." This occurs when two objects are in contact and cannot move relative to each other due to their immense weight or inertia.

In this scenario, the force of static friction (also known as "inertial force") acts to resist the motion of one object against the other. The force of static friction is proportional to the normal force (the force exerted by the surface on the objects) and the coefficient of static friction between them.

When an unstoppable force meets an immovable object, the following events can occur:

1. **The force of static friction increases**: As the unstoppable force tries to overcome the static friction, it generates a greater force to resist the motion. This increased force causes the object to slow down or even come to a complete stop.
2. **The object may not move**: If the force of static friction is strong enough, it can overcome the unstoppable force and prevent the object from moving. In this case, the object will remain stationary.
3. **The unstoppable force may cause the object to deform**: If the force of static friction is significant, it can cause the object to deform or change shape, such as bending or crushing, due to the immense pressure exerted by the unstoppable force.

Examples of unstoppable forces include:

* Gravity (the force of attraction between masses)
* Friction (the force that opposes motion between two surfaces in contact)
* Electromagnetic forces (such as repulsion or attraction between charged particles)

Examples of immovable objects include:

* A heavy, solid object on a frictionless surface
* A massive, rigid structure (like a mountain or a skyscraper)
* A planet or moon orbiting a star

Keep in mind that the concept of unstoppable force and immovable object is a bit of a simplification, as in reality, there are many exceptions and nuances to consider. However, this basic principle provides a useful starting point for understanding the behavior of these fundamental forces and objects.

Is this what I'm supposed to have?
I know I can have a better output with post processing the message. But, is there anything I should've done differently to prevent this?

[shan-max]

buy a car

[shan-max]

build a home.

[A96M]

add in the pipeline_kwargs { "return_full_text": False}. By the way, I would take into consideration buying a car