OpenAI has released its chat completion API with a new feature - the "system" role in prompts.
Unlike the text completion API supporting only text prompts, the new API provides a clear separation between instructions for the assistant and the messages in the conversation, enabling more precise control over behavior and responses.
This article will focus on crafting useful system role instructions for the ChatGPT model by applying some prompt engineering recommendations. If you need to get up to speed on API integration quickly, you can do so here.
OpenAI API Prompt example
In the text completion API, to provide instructions for the response or behaviors you want, you can write the following AI prompt:
"You will be asked for travel recommendations by a tourist. Answer as if you were a travel guide, and give no more than three recommendation options per answer. Just answer with the options and don't give any introduction. Use markdown to format your response.
Tourist: Where can I eat sushi in New York?"
In the new chat completion API, the request is formatted as JSON:
CODE: https://gist.github.com/diego-suarez/c1a6532621e770f3deeb9b01196317d1.js
Separating system instructions and user messages provides more precise control over the model's responses. This is a key feature of the ChatGPT prompt engineering.
For example, the text completion API might generate a response like the following.
"Guide: - Sushi Yasuda: Located in Midtown East, this restaurant is highly acclaimed for its traditional sushi and omakase.
- Ushiwakamaru: Located in Greenwich Village, this restaurant is known for its modern take on sushi.
- Sushi Zo: Located in the East Village, this restaurant offers a unique omakase experience."
As shown, the above answer starts with Guide: as the model thought that it should add the actor of the message since the prompt included Tourist: on the previous message.
Avoid that when using APIs since our application needs different response parsing. Such issues can be avoided with the new OpenAI ChatGPT API, as the instructions are separated from the message content.
Writing System Role Instructions
Writing good system prompts can be tricky and require multiple iterations. Start writing a simple prompt with what you already know to expect, and then add new instructions after some testing.
Try to write short instructions, but keep them explicit on what you expect from the system or how it should behave. It helps to write each instruction in a new line so it's easier to read the list of rules.
Focus on what you expect from the system and what you don't. If you don't know at first, you don't have to worry; you will after testing different inputs.
Use different "levels" of importance, like the following:
- The answer must be in HTML format.
- The assistant must not give an introduction.
- The assistant can use emojis on the responses to be more friendly, if necessary.
Let's look at some instruction types you may want to include.
Describing The ChatGPT Assistant’s Role
You can describe the assistant's role in detail so it knows how you want it to act. You can reference the profession, personality traits, or behaviors you expect. Also, if you know it in advance, explain what the request text will look like.
Assistant examples:
- The assistant is a financial advisor that gives investment recommendations.
- The assistant is a chef specializing in seafood who provides recipes.
- The system is a geocoding API that answers with the nearest city for a given latitude-longitude coordinates.
- The assistant is a Ruby on Rails software developer who explains what code snippets do.
- The assistant is a funny clown who jokes about the requested topic.
Providing Context In Your OpenAI Prompts
A key aspect of prompt engineering is to give helpful context information to the assistant.
Since you can change the prompt on each request to the API, it's good to fulfill with dynamic data that can be helpful but changes based on the request context.
For example, if you have information about the request or the user that is meaningful to the answer, customize the prompt to include such information:
- The user language is #{language}. The answer should be written in that language.
- The user interested in the following product is #{product.title}. The product description is #{product.description}. The technical product specifications are Weight: #{product.weight}, Length: #{product.length}, Depth: #{product.depth}.
- The following are the available ingredients in the kitchen: #{ingredients_list}.
- The user has the following food preferences: #{user.food_preferences}
- The following cities should be blacklisted on your answers: #{blacklisted_cities}
Specifying A Response Format
The format you should expect in the response should be like the following:
- The response should be a JSON object with the following keys: city (string), countryCode (ISO 3166 format string), latitude (float), and longitude (float). If you don't know the value for a key, the value should be completed with null.
- The response should be in Markdown format. You can highlight important words in bold. Divide the response into sections, each one with a title.
- The response should be a blog post.
- The response should be an email with a greeting, body, and closure.
Give Your Response Length Expectations
Another great technique of prompt engineering is to tell the system the length of the response you expect, for example:
- The assistant response should be brief and concise.
- The response should include detailed information.
- The answer should never be longer than two paragraphs.
- The blog post should contain exactly three sections.
Handling Empty/Unknown Responses
Since there can be cases where the model does not know the answer, the conversation input is unclear, or doesn't apply to the system, the OpenAI ChatGPT model will not provide a useful response, and you want to identify those cases on your end.
One approach you can use for those cases is instructing the model to answer with a particular message when that happens. Then on your backend, you can look for that message and flag that request as "No response."
Some example instructions can be:
- If you don't know the answer, you should answer with the exact text "I don't know the answer."
- If you think your response may be inaccurate or vague, do not write it and answer with the exact text "I don't have a response."
- If the system can't find any city near the input coordinates, the answer should be an empty JSON object: {}.
Additional Instructions
You can also include other instructions that may be helpful for the system, like the following.
The tone of the response:
- The assistant should be friendly and polite.
- The assistant should be funny and entertaining.
- The assistant should be formal and professional.
Request source citations (links).
Encourage (or discourage) creativity (This can help, but you may want to learn about temperature and top_p parameters).
Test And Iterate
Once you have written your system prompt, test it using the OpenAI playground with different inputs that may occur in real scenarios. Also, try using unexpected prompts and border cases. Pay attention to the responses and find unexpected behaviors, formats, lengths, or anything you want to fine-tune in your responses.
Once you have identified what you want to improve, edit your system prompt again, add additional instructions that may help fix it, remove instructions that could be causing the unexpected response, modify the existing ones, and iterate until you get the desired outcome.
Refining your system prompt is an iterative process, and you may need to undergo several rounds of testing to adjust your instructions. Keep track of the inputs and outputs during testing to observe patterns in the model's behavior. Use this information to refine your instructions further and make them more effective.
As you refine your instructions and gain more experience with the GPT-4 Chat API, you'll better anticipate the model's behavior and craft instructions that generate the desired responses.
Take Me To Sydney
You may have heard about New Bing, a chat assistant with network connectivity released by Microsoft and powered by OpenAI models. When the beta was released, some users could disclose the system instructions with an internal codename - Sydney.
The New Bing assistant probably uses the Chat Completion API (or a similar one customized for Bing) to interact with users. If you are curious about its instructions, it's an interesting example to explore and take inspiration from when learning how to write system instructions using the Chat Completion API.
You can find the disclosed instructions (confirmed by Microsoft) in this article from The Verge.
What To Take Away
Writing effective system instructions for the GPT-4 Chat API is key to ensuring that the responses you receive meet your expectations. It requires a deep understanding of the model's behavior and the ability to iterate and improve your instructions.
By following these tips, guidelines, and prompt engineering best practices, you'll be well on your way to crafting system instructions that generate accurate, clear, and useful responses for your use cases. Remember that it is a process that requires practice and iteration, so don't be discouraged if your initial attempts don't yield perfect results.
With time and experience, you'll become proficient in writing system instructions that cater to your specific use case. The new GPT-4 Chat API is a powerful tool that can be used to build various products.
By combining an effective system prompt with message inputs (and history) and other API parameters not covered in this article, you can utilize its full potential for your business needs.
A note from OpenAI
One final thing to consider is the following warning from OpenA's official documentation:
"gpt-3.5-turbo-0301 does not always pay strong attention to system messages. Future models will be trained to pay strong attention to system messages."
If you are using GPT-3.5-turbo, you can already utilize the system role input; however, be aware that it will not pay strong attention to it. On the other hand, if you have access to the GPT-4 preview, you can take full advantage of this powerful feature.
