Bug: SK runs his self-made function by ignoring the auto-invoke function config

[zzzhy]

Describe the bug
I set the toolcallbehavior of allowOnlyKernelFunctions as true, and passed the functions to the kernel
but the kernel make a functiontoolcall by himself and just invoke it.

To Reproduce
Steps to reproduce the behavior:

         List<KernelFunction<?>> functions = kernel.getPlugins().stream()
                .map(KernelPlugin::getFunctions)
                .map(Map::values)
                .flatMap(Collection::stream)
                .toList();
        FunctionResult<String> preResult = kernel
                .invokeAsync(function)
                .withToolCallBehavior(ToolCallBehavior.allowOnlyKernelFunctions(true, functions))
                .withArguments(
                        KernelFunctionArguments.builder()
                                .withInput(userInput)
                                .withVariable("metadata", metadata)
                                .build())
                .withResultType(ContextVariableTypes.getGlobalVariableTypeForClass(String.class))
                .block();    // occur exception here, for invoke a self-make function

Expected behavior
Only invoke the passed functions.

Screenshots

Maven

• Version: 3.3.9
• Dependencies: 1.4.3

Platform

• IDE: IDEA
• JDK version: 17

Additional context
Excption: com.azure.core.exception.HttpResponseException: Status code 400, "{"error":{"message":"Invalid 'messages[1].tool_calls[1].function.name': string does not match pattern. Expected a string that matches the pattern '^[a-zA-Z0-9_-]+$'.","type":"invalid_request_error","code":"invalid_value","param":"messages[1].tool_calls[1].function.name"}}"

[dsgrieve]

Which model are you using?

[zzzhy]

Which model are you using?

I'm using GPT-4o as the model.
My requirement is to implement a dynamic method invoker, which uses the llm to automatically select the method and extract the required parameters from the conversation with the user.
When all things get done ,then inform the background service of this information for invocation.
The problem is SK use this info and made a function-call.😂

[zzzhy]

All method metadata and user intents are assembled in the prompt(in userInput and metadata variables).

[johnoliver]

I have run a sample based on the original code, and it appears to execute as expected:

• AutoInvokeFunctionTest

Are you able to share more context/a failing sample of the code that is failing?

[zzzhy]

@johnoliver Sorry for some mistake description before.
When using gpt-4o-mini, it tried to call the backend service method such as aiotDeviceGatewayImpl$scrollDeviceInfoByParam, and then exception occurred.
When using gpt-4o, the model just return the expercted json.
I am using 4o for now.

My prompt is:

You are a dataset matching expert. You will identify the user's intent based on the dataset definition (Init) and matching rules (Rules) from the user's input (User Input) and then return the matching result in JSON format.

# Matching Rules - Rules:
1. Match based on the dataset - Init definition, referring to the example (few-shots).
2. If fully matched (set status = 0), the returned data must be in JSON format, structured as follows, where `args` are the parameters used to query the dataset (the parameter array may contain heterogeneous elements, matching based on the dataset definition):
    ```json
    {
      "status": 0,
      "desc": "Your intent is xxxx, all parameters have been matched",
      "args": [
        {"argName": "yyy1", "argValue": "yyy2"},
        {"argName": "zzz", "argValue": 100}
      ],
      "identity": "xxx3"
    }
    ```
3. If not fully matched (set status = 1), output the following JSON structure, where `desc` is the prompt message, `identity` is the dataset identifier (could be matched, may be an empty string), and `args` are the parameters that may be matched:
    ```json
    {
      "status": 1,
      "desc": "Your input is incomplete, please describe xxxx/Your intent is xxxx, you still need to provide the xxxx parameter",
      "args": [
        {"argName": "argName1", "argValue": "argValue1"},
        {"argName": "argName2", "argValue": "argValue2"}
      ],
      "identity": "xxx3"
    }
    ```
4. **Special Note**: If the data the user requires has been retrieved, return the `records` field (where `headers` is the table header, and `rows` are the actual data corresponding to the header):
    ```json
    {
      "status": 0,
      "desc": "Your intent is xxxx, all parameters have been matched",
      "records": {
        "headers": ["colName1", "colName2", "colName3"],
        "rows": [["obj1", "obj2", "obj3"]]
      },
      "args": [
        {"argName": "argName1", "argValue": "argValue1"},
        {"argName": "argName2", "argValue": "argValue2"}
      ],
      "identity": "xxx3"
    }
    ```
5. Only return JSON, no other unrelated content, and no unnecessary statements.

# Dataset - Init:
Note:
1. Only return the most matching dataset intent. Each identity represents a unique dataset, and each dataset can implement one or more intents (Intent).
2. The intent is considered fully matched only if all required parameters are matched; otherwise, it is considered incomplete. Non-required parameters should also be returned.
3. The content of the dataset cannot be used as a tool call.
Dataset Definition: 
{{$metadata}}

# Example:
Input: I want to query the weather in Shenzhen
Output:
```json
{
  "status": 0,
  "desc": "Your intent is to query the weather, you matched the weather query plugin, and the city you entered is Shenzhen",
  "args": [{"argName": "cityName", "argValue": "Shenzhen"}],
  "identity": "weatherSearchPlugin$queryCityWeatherByCityName"
}
```
Input: I want to query the weather
Output:
```json
{
  "status": 1,
  "desc": "Your intent is to query the weather, you matched the weather query plugin, please enter the city name you want to query",
  "args": [],
  "identity": "weatherSearchPlugin$queryCityWeatherByCityName"
}
```
Input: I want to query
Output:
```json
{
  "status": 1,
  "desc": "Your input does not match any intent, please re-describe your query",
  "args": [],
  "identity": ""
}
```
Input: I want to know the city national standard code in Shenzhen in MT bizCode
Output:
```json
{
  "status": 0,
  "desc": "Your intent is to query the city national standard code, the city you entered is Shenzhen",
  "args": [{"argName": "cityName", "argValue": "Shenzhen"}, {"argName": "bizName", "argValue": "MT"}],
  "records": {"headers": ["cityCode"], "rows": [[440300]]},
  "identity": ""
}
```
Input: I want to query the data of device 003C84FF's identifier=temperature from yesterday
Output:
```json
{
  "status": 0,
  "desc": "Your intent is to query the data for a specific identifier of a device for a certain period",
  "args": [
    {"argName": "deviceId", "argValue": "003C84FF"},
    {"argName": "identifier", "argValue": "temperature"},
    {"argName": "startTime", "argValue": "2025-01-01 00:00:00"},
    {"argName": "endTime", "argValue": "2025-01-02 00:00:00"}
  ],
  "identity": "TWQYJEEGNN"
}
```
Input: My emp number is xxx, I want to query the device information of MT device 003C84FF
Output:
```json
{
  "status": 0,
  "desc": "Your intent is to query the device information of MT device 003C84FF",
  "args": [
    {"argName": "emp", "argValue": "xxx"},
    {"argName": "bizName", "argValue": "MT"},
    {"argName": "deviceIdOrName", "argValue": "003C84FF"}
  ],
  "identity": "aiotDeviceGatewayImpl$scrollDeviceInfoByParam"
}
```

# User Input
Input: My emp number is {{$emp}}, {{$input}}
Output:

My metadata looks like:

Dataset Definition:
Dataset Identity: aiotDeviceGatewayImpl$scrollDeviceInfoByParam, Description: Query device information based on the business name, device ID or device name (deviceIdOrName, supports fuzzy query), and user's emp number.
Intent Input Parameter Combinations and Sorting Parameters (Intent):
Intent Description: Query device information based on the business name (bizName, such as XS, MT, etc.), device ID or device name (deviceIdOrName, supports fuzzy query), and user's emp number.
Required Parameters List:
- Field Name: deviceIdOrName, Field Description: Device ID or device name, Field Type: java.lang.String
- Field Name: bizName, Field Description: Business name, such as MT, XS, etc., must be an exact match, Field Type: java.lang.String
- Field Name: emp, Field Description: User's emp number, Field Type: java.lang.String
Optional Parameters List:
- Field Name: deviceOnlineStatus, Field Description: Device online status, 1 for online, 0 for offline, Field Type: java.lang.Integer

Dataset Identity: weatherSearchPlugin$queryCityWeatherByCityName, Description: Query city weather based on the city name.
Intent Input Parameter Combinations and Sorting Parameters (Intent):
Intent Description: Query city weather based on the city name (cityName).
Required Parameters List:
- Field Name: cityName, Field Description: City name, Field Type: java.lang.String