Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Export base Message class in OpenAI chat SDK #616

Conversation

kevinmingtarja
Copy link
Member

@kevinmingtarja kevinmingtarja commented Nov 26, 2024

Description

This PR would allow users to create their own message with custom role. I encountered this use case when trying to use Llama 3.1 (served using an OpenAI compatible server), which has a different set of supported roles: [system, assistant, user, ipython].

Basically, i want to be able to do:

package main

import (
	"github.com/hypermodeinc/modus/sdk/go/pkg/models"
	"github.com/hypermodeinc/modus/sdk/go/pkg/models/openai"
)

type customMessage struct {
	openai.MessageBase
}

func generate(prompt string) (string, error) {
        ...
        input, err := model.CreateInput(
		...
		&customMessage{
			MessageBase: openai.MessageBase{
				Role:    "custom",
				Content: "...",
			},
		},
	)
        ...
}

Given the increasing number of inference providers implementing the OpenAI API (to some degree), I think we might need this for other models too eventually.

Checklist

  • Code compiles correctly and linting passes locally
  • For all code changes, an entry added to the CHANGELOG.md file describing and linking to this PR

@kevinmingtarja kevinmingtarja requested a review from a team as a code owner November 26, 2024 01:52
@kevinmingtarja kevinmingtarja enabled auto-merge (squash) November 26, 2024 01:58
@kevinmingtarja kevinmingtarja merged commit e5abb35 into main Nov 26, 2024
86 checks passed
@kevinmingtarja kevinmingtarja deleted the kevinm/hyp-2650-support-messages-with-custom-role-in-openai-model-interface branch November 26, 2024 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants