Very nice writeup that also mirrors my experience when optimizing function calling. Thanks for putting the post together!
I think all the recommendations can also be used 1:1 with ChatGPT custom GPTs, but for those I also have 2 additional recommendations:
- There, providing example responses (in contrast to example requests shown in this post) really seems to help the model choose which function to call based on where it can see the desired information in the output. Haven't tried that technique yet in direct GPT4 function calling
- If you are starting out with an existing OpenAPI schema and/or have one where the model may retrieve identical information via different endpoints, I've found that it helps to trim down the whole schema to the endpoints it should see (can be done with tags on endpoints + a simple script to strip them). This can really help in making the model focus on the functionality you need from the API.
Improving GPT 4 Function Calling Accuracy · HackerTrans
I think all the recommendations can also be used 1:1 with ChatGPT custom GPTs, but for those I also have 2 additional recommendations:
- There, providing example responses (in contrast to example requests shown in this post) really seems to help the model choose which function to call based on where it can see the desired information in the output. Haven't tried that technique yet in direct GPT4 function calling
- If you are starting out with an existing OpenAPI schema and/or have one where the model may retrieve identical information via different endpoints, I've found that it helps to trim down the whole schema to the endpoints it should see (can be done with tags on endpoints + a simple script to strip them). This can really help in making the model focus on the functionality you need from the API.