function_call APIのパラメータタイプについて、もっと調べてみた

Sun, 18 Jun 2023 11:37:07 +0900

OpenAIのAI、function_callのパラメータについて、もうちょっと調べてみました。
試してみた結果、type以外の細かいことは、descriptionに書けということです。

JSONスキーマっての見たら、もうちょっと細かい指定ができるんではないか？と思って、 https://json-schema.org/

まずは、minLengthとmaxLengthを試してみた。 [https://json-schema.org/understanding-json-schema/reference/string.html#id5]

import openai
import json

def submit():
    messages = [{"role": "user", "content": "test the sample_function"}]
    functions = [
        {
            "name": "sample_function",
            "description": "test for parameters",
            "parameters": {
                "type": "object",
                "properties": {
                    "arg1": {
                        "type": "string",
                        "minLength": 1,
                        "maxLength": 2,
                        "description": "string data",
                    }
                },
                "required": ["arg1"],
            },
        }
    ]
    try:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo-0613",
            messages=messages,
            functions=functions,
            function_call="auto",
        )
        print( response.choices[0] )
    except Exception as ex:
        return ex.args

submit()

実行結果

$ python3 test.py 
{
  "finish_reason": "function_call",
  "index": 0,
  "message": {
    "content": null,
    "function_call": {
      "arguments": "{\n  \"arg1\": \"Hello World\"\n}",
      "name": "sample_function"
    },
    "role": "assistant"
  }
}

minLengthとmaxLengthはあっさり無視されるらしい。

次は、[https://json-schema.org/understanding-json-schema/reference/string.html#id6](Regular Expressions)
これが使えたら、結構うれしいんじゃないか？

                "properties": {
                    "arg1": {
                        "type": "string",
                        "pattern": "^(\\([0-9]{3}\\))?[0-9]{3}-[0-9]{4}$",
                        "description": "string data",
                    }
                },

実行結果

$ python3 test.py 
{
  "finish_reason": "function_call",
  "index": 0,
  "message": {
    "content": null,
    "function_call": {
      "arguments": "{\n  \"arg1\": \"Hello World\"\n}",
      "name": "sample_function"
    },
    "role": "assistant"
  }
}

やっぱり無視されました。ですよね〜〜〜〜。

もしかして type に numberを指定しても無視するんじゃないか？　　と思って

import openai
import json

def submit():
    messages = [{"role": "user", "content": "generate test data for sample_function. and test it."}]
    functions = [
        {
            "name": "sample_function",
            "description": "test for parameters",
            "parameters": {
                "type": "object",
                "properties": {
                    "arg1": {
                        "type": "number",
                        "description": "",
                    }
                },
                "required": ["arg1"],
            },
        }
    ]
    try:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo-0613",
            messages=messages,
            functions=functions,
            function_call="auto",
        )
        print( response.choices[0] )
    except Exception as ex:
        return ex.args

submit()

実行結果

{
  "finish_reason": "stop",
  "index": 0,
  "message": {
    "content": "To generate test data for `sample_function`, I need to know the expected output or the desired behavior of the function. Please provide more information about the expected behavior, any constraints on the input, and the desired output format.",
    "role": "assistant"
  }
}

うーむ。functoinの仕様を明確にしろってか？？？なら、これでどうだ？

            "name": "sample_function",
            "description": "return tha input data, just sample",
            "parameters": {
                "type": "object",
                "properties": {
                    "arg1": {
                        "type": "number",
                        "description": "",
                    }
                },
                "required": ["arg1"],
            },

実行結果

{
  "finish_reason": "function_call",
  "index": 0,
  "message": {
    "content": null,
    "function_call": {
      "arguments": "{\n  \"arg1\": 5\n}",
      "name": "sample_function"
    },
    "role": "assistant"
  }
}

おお！数字ってのは守ってくれるらしい。
じゃあ、範囲指定はどうか？

                    "arg1": {
                        "type": "number",
                        "minimum": 100,
                        "maximum": 200,
                        "description": "",
                    }

実行結果

{
  "finish_reason": "function_call",
  "index": 0,
  "message": {
    "content": null,
    "function_call": {
      "arguments": "{\n  \"arg1\": 10\n}",
      "name": "sample_function"
    },
    "role": "assistant"
  }
}

うむ、やっぱり範囲指定は無視するらしい。
つまり以下のようにdescriptionに書けば・・・・

                    "arg1": {
                        "type": "number",
                        "description": "range 100 to 200",
                    }

実行結果

{
  "finish_reason": "function_call",
  "index": 0,
  "message": {
    "content": null,
    "function_call": {
      "arguments": "{\n  \"arg1\": 150\n}",
      "name": "sample_function"
    },
    "role": "assistant"
  }
}

ちゃんと守ってくれるらしい。

function_call APIのパラメータタイプについて調べてみた

Sun, 18 Jun 2023 10:55:44 +0900

OpenAIのAPIで、fucntion_callが発表されました。英語が苦手なのでドキュメントを読んでも手探り状態です。一番悩んだのがfunctionの引数の指定の仕方です。 JSON Schemaで指定するらしいんですけど？ https://json-schema.org/

実際にどうなるか、テストコードで調べて見ました。結果から推測するには、typeに指定できるのは下記っぽい

string
number
integer
boolean
object

うーん？スキーマとしては配列もいけそうな気がしたのだが・・・・

テスト用コードです

import openai
import json

def submit(data):
    messages = [{"role": "user", "content": data["mesg"]}]
    functions = [
        {
            "name": data["func_name"],
            "description": data["func_description"],
            "parameters": {
                "type": "object",
                "properties": {
                    data["prop_name"]: {
                        "type": data["prop_type"],
                        "description": data["prop_descriptoin"],
                    }
                },
                "required": [data["required"]],
            },
        }
    ]
    try:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo-0613",
            messages=messages,
            functions=functions,
            function_call="auto",
        )
        #print( response )
        return response.choices[0].message.content
    except Exception as ex:
        return ex.args

data={}
data["mesg"] = 'hello'
data["func_name"] = 'b'
data["func_description"] = "c"
data["prop_name"] = 'd'
data["prop_type"] = 'string'
data["prop_descriptoin"] = 'e'
data["required"] = 'f'

print( "{:16s} {}".format("type","output"))
for t in ['askdfja','string','String','integer','number','number[]','float','float{}','{}','[]','double','bool','boolean','dictionary','array','array of string','object','オブジェクト','文字列','文字','{"aaa":"bbb"}','[string]','string,number,boolean']:
    data["prop_type"] = t
    print( "{:16s} {}".format(t,submit(data)))

実行結果、こうなりましたよ

type             output
askdfja          ("Invalid schema for function 'b': 'askdfja' is not valid under any of the given schemas",)
string           Hello! How can I assist you today?
String           ("Invalid schema for function 'b': 'String' is not valid under any of the given schemas",)
integer          Hi, how can I assist you today?
number           Hello! How can I assist you today?
number[]         ("Invalid schema for function 'b': 'number[]' is not valid under any of the given schemas",)
float            ("Invalid schema for function 'b': 'float' is not valid under any of the given schemas",)
float{}          ("Invalid schema for function 'b': 'float{}' is not valid under any of the given schemas",)
{}               ("Invalid schema for function 'b': '{}' is not valid under any of the given schemas",)
[]               ("Invalid schema for function 'b': '[]' is not valid under any of the given schemas",)
double           ("Invalid schema for function 'b': 'double' is not valid under any of the given schemas",)
bool             ("Invalid schema for function 'b': 'bool' is not valid under any of the given schemas",)
boolean          Hi there! How can I assist you today?
dictionary       ("Invalid schema for function 'b': 'dictionary' is not valid under any of the given schemas",)
array            (["Invalid schema for function 'b': In context=('properties', 'd'), array schema missing items"],)
array of string  ("Invalid schema for function 'b': 'array of string' is not valid under any of the given schemas",)
object           Hello! How can I assist you today?
オブジェクト           ("Invalid schema for function 'b': 'オブジェクト' is not valid under any of the given schemas",)
文字列              ("Invalid schema for function 'b': '文字列' is not valid under any of the given schemas",)
文字               ("Invalid schema for function 'b': '文字' is not valid under any of the given schemas",)
{"aaa":"bbb"}    ('Invalid schema for function \'b\': \'{"aaa":"bbb"}\' is not valid under any of the given schemas',)
[string]         ("Invalid schema for function 'b': '[string]' is not valid under any of the given schemas",)
string,number,boolean ("Invalid schema for function 'b': 'string,number,boolean' is not valid under any of the given schemas",)

chatGPT on ストックバック

function_call APIのパラメータタイプについて、もっと調べてみた

function_call APIのパラメータタイプについて調べてみた