Specifying best_of argument for chat completion

[Artein]

Hi,

Is it possible to specify best_of argument to generate several results on server? If it's possible — how to do that?

[Frogley]

It seems like what you're after is the 'N' parameter, which allows you to specify how many chat completion choices to generate for each input message. All you need to do is include this parameter in your request like this:

var completionResult = openAiService.ChatCompletion.CreateCompletionAsStream(
    new ChatCompletionCreateRequest()
    {
        Messages = message.Messages,
        Model = OpenAI.ObjectModels.Models.Gpt_4,
        Temperature = 0.2f,
        MaxTokens = 500,
        N = 3,  //The N parameter!
    }, cancellationToken: CancellationToken.None);

For more information on API parameters, you can check out the documentation at api-reference

[Artein]

Thank you @Frogley for your reply.

By saying "on server", I meant OpenAI servers. Yes, I know for 'n' argument existence. But my intent is to utilize 'best_of' (say out of 3) and receive the single top result.

Occasionally, GPT replies in the wrong format (based on my requirements in the prompt). By utilizing 'best_of' I want to try to omit such cases by automatically selecting the best one. This is just an experiment, but worth trying.

[Artein]

@Frogley , FYI

best_of
Generates best_of completions server-side and returns the "best" (the one with the highest log probability per token). Results cannot be streamed. When used with n, best_of controls the number of candidate completions and n specifies how many to return – best_of must be greater than n.
Note: Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for max_tokens and stop.

[Frogley]

I apologize for the misunderstanding. OpenAI does indeed offer the 'best_of' parameter, but it's only available in the completions endpoint. You can use this parameter like this:

var completionResult = openAiService.Completions.CreateCompletion(
    new CompletionCreateRequest()
    {
        Prompt = "prompt",
        Model = OpenAI.ObjectModels.Models.Gpt_4,
        Temperature = 0.2f,
        MaxTokens = 500,
        BestOf = 3 // 'best_of' parameter here!
    }, cancellationToken: CancellationToken.None);

However, this parameter is not available in other endpoints, such as ChatCompletions.

[Artein]

Oh, it makes sense now. Will check just Completion. Thank you, @Frogley