Pricing
Company
Free Trial
We Use Cookies

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with this.

See our cookie policy.

Automation Action: ChatGPT

Send a prompt to ChatGPT and assign the response to a variable. Prompts can be assigned a Conversation Id if previous prompts/responses should be included when part of a conversation.

Built-In Action

Send a prompt to OpenAI ChatGPT and assign the response to a variable. You can send single one-off prompts or prompts that are part of a conversation. The ThinkAutomation ChatGPT action enables you to automate requests to ChatGPT and then use the response further in your Automation.

Before you can use this action you must create an account with OpenAI. Go to OpenAI and click the Get Started link to create an account.

On your account page select API Keys and generate a new secret key. Make a note of this key as it is only displayed once. This is your OpenAI API Key.

Specify your OpenAI API Key. You can specify a different OpenAI API Key on each ChatGPT action. If you will only be using a single OpenAI Account then you can enter your API key in the ThinkAutomation Server Settings - Integrations - ChatGPT section. This key will be used by default if you do not specify one on the ChatGPT action itself.

Specify the Operation:

  • Ask ChatGPT To Respond To A Prompt
  • Add Context To A Conversation
  • Clear Conversation Context
  • Ask ChatGPT To Respond To A Prompt With An Image
Ask ChatGPT To Respond To A Prompt

The System Message is optional. This can help to set the behavior of the assistant. For example: 'You are a helpful assistant'.

The Prompt is the text you want a response to.

Examples:


What category is the email below? Is it sales, marketing, support or spam?
Respond with 'sales', 'marketing', 'support' or 'spam' only.
If it is a support email and it appears to be urgent, respond with 'support:urgent'.
Subject: %Msg_Subject%
%Msg_Digest%
Response: sales              

Extract the name and mailing address from this email:
Dear Kelly,
It was great to talk to you at the seminar. I thought Jane's talk was quite good.
Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
Best,
Maya
Response:
Name: Maya
Mailing Address: 2111 Ash Lane, Crestview CA 92002              

I am flying from Manchester (UK) to Orlando.
What are the airport codes? Respond with just the codes separated by comma.
Response:
MAN,MCO
Prompts you send to ChatGPT have a limit of approximately 2000 words.

Tip: When using ChatGPT to analyze incoming emails, you can use the %Msg_Digest% built-in variable instead of %Msg_Body%. The %Msg_Digest% variable contains the last reply text only, with all blank lines and extra whitespace removed. It is also trimmed to the first 750 characters. This is usually enough to categorize the text and will ensure the prompt does not exceed the token limit.

Tip: If you want to use ChatGPT to automatically respond to incoming emails you should use the %Msg_LastReplyBody% built-in variable instead of the %Msg_Body%. The %Msg_LastReplyBody% variable contains only the current reply, without any quoted text & previous replies. This ensures only the current message and not the entire email thread is sent.

ChatGPT Model

The Open AI Model entry allows you to select the OpenAI Model to use. You can select from:

  • gpt-3.5-turbo-16k (max tokens 16384)
  • gpt-3.5-turbo (max tokens 4096)
  • gpt-4 (max tokens 8192)
  • gpt-4o (max tokens 128000)
  • text-davinci-003 (max tokens 4097)
  • text-davinci-002 (max tokens 4097)
  • code-davinci-002 (max tokens 8001)
  • Or your own fine-tuned model name

See the OpenAI documentation for details about the different models. GPT-3.5-turbo-16k is the default and works for most scenarios.

Response

Specify the variable to receive the response from the Assign Response To list.

You can also optionally assign the number of tokens used for the prompt/response. Select the variable to receive the tokens used from the Assign Used Token Count To list. OpenAI charges are based on tokens used. For example, the current pricing for gpt-3.5-turbo is $0.002 per 1000 tokens (roughly 4k of text).

Moderation

If the Requires Moderation option is enabled then the prompt text will be validated against the OpenAI moderation endpoint before being sent. This ensures that the prompt text complies with OpenAI's usage policies. This no-cost check takes some additional time. You should enable this option if you will be sending any public-generated content (for example, from a public facing chat bot or public receiving email responder). It ensures that no prompt text can be sent that violates OpenAI usage policies.

Conversations

You can optionally specify a Conversation Id. This is useful if multiple ChatGPT requests will be made and you want to include previous prompts/responses for context, or if you want to add your own context prior to asking ChatGPT for a response.

The Conversation Id can be any text. For example, setting it to %Msg_FromEmail% will link any requests for the same incoming email address. The built-in variable %Msg_ConversationId% can be used for the Conversation Id. This is a hash of the from/to addresses and subject.

The Max Conversation Lines entry controls the maximum number of previous prompts/response pairs that are included with each request. For example, if the Max Conversation Lines is set to 25 then the last (most recent) 25 prompt/response pairs will be sent prior to the current prompt. As the conversation grows, the oldest items will be removed to prevent the total prompt text going over the Open AI token limit.

Conversations are shared by all Automations within a Solution and conversation lines older than 48 hours are removed.

For example:

Suppose you send 'What is the capital city of France?' in one prompt and receive a response. If you then send another separate prompt of 'What is the population?' with the same conversation id then you will receive a correct response about the population of Paris because ChatGPT already knows the context. This would work across multiple Automation executions for up to 48 hours, as long as the conversation id is the same.

Add Context To A Conversation

You can add context to a conversation. Context is used to help ChatGPT give the correct answer to a question based on the context you have provided. This can be static text, or you can search articles related to the incoming message from the Embedded Knowledge Store and add the most relevant articles. Using the ChatGPT action along with the Add Context operations enables you to create bots that can answer business specific questions - even when ChatGPT itself has no knowledge of the subject matter.

Lookup Context From Knowledge Store

Using the Embedded Knowledge Store enables you to add text and documents (PDF, Word, HTML etc) as articles to a Knowledge Store collection. You can then search the knowledge store using the incoming message as the Search Text. This will return the Top x most relevant articles from the specified Knowledge Store Collection.

The Relevancy Threshold setting controls the relevancy level. Articles below the relevancy % will not be included. This value defaults to 75%.

The Return Max Tokens entry should be set to a value lower than the maximum tokens that the OpenAI model supports (see above). It is recommended to keep this at approximately 50% of the model maximum. For example, if you will be using the gpt-3.5-turbo-16k model, then this allows 16384 tokens, so the Return Max Tokens entry should be no higher than 8192. This will ensure that there is enough tokens remaining for the prompt text and the ChatGPT response itself.

The returned articles will each be added as context, allowing ChatGPT to use this information when responding to the incoming message.

If you want to add a specific article from a Knowledge Store collection, rather than relying on a search (for example, based on specific keywords found in the incoming message), use the Embedded Knowledge Store action to lookup a specific article (using the Get operation). Set the Return As to Json, assign the returned article to a %variable%, then add this variable to the Add Static Context tab.

Top Title/Tag

You can optionally assign the Top Title and/or Top Tag to variables. The first (most relevant) article Title and Tag will be assigned. This can be used further in your Automation. For example: Suppose we have a number of articles relating to 'pricing' or 'quotes' or 'sales'. We could tag all of these with a 'sales' tag. At the end of the Automation we could check the Top Tag %variable%. If this is equal to 'sales' we could send an email to the sales team informing them that someone has asked something sales related - we could send the question and ChatGPT response with this email.

Add Static Context

You can also add Static Context. This can be any text (which can contain %variable% replacements). This can be used to provide default context, for example:


You are a very enthusiastic representative working at {your company}.
Given the following sections from our documentation, answer the user's question using only that information, outputted in markdown format.
If you are unsure and the answer is not explicitly written in the documentation, say "Sorry, I don't know how to help with that."

You should always add some default context, this should be used to tell ChatGPT who and what it is and how it should respond. You can also provide some general information about your business.

For email responder bots you can use default context such as:


Your name is '{bot name}' and you are a very enthusiastic representative working at {your company} answering emails. Given the provided sections from our documentation, answer the question using only that information, outputted in markdown format.
If an answer cannot be found in the information provided, respond with 'I cannot help with that' only.
Do not try and answer the question if the information is not provided.
Add a friendly greeting and sign off message to your response. Your email address is '{bot email address}'.
My email address is %Msg_FromEmail%

This tells ChatGPT that it's responding to emails and to include a greeting and sign off message with its response.

If the Required option is enabled then the context text will remain in the conversation. This option can be used to ensure that the default or important context is always part of the conversation.

You could also lookup static context via a database or web lookup. For example: If the customer provides an email at the start of the chat, or you are responding to incoming emails, you could lookup customer & accounting/order information and add this to the context in case the customer asks about outstanding orders. Or you could lookup current service status if the user wants live status information.

Regardless of how the context is added, the same context wont be added to a conversation if the conversation already has it. So you can add standard context (for example, general information about your business) along with searched for context within your Automation prior to asking ChatGPT to a response.

You can add multiple ChatGPT - Add Context To A Conversation actions in your Automation prior to the ChatGPT - Ask ChatGPT To Respond To A Prompt action.

For example: Suppose you have a company chat bot on your website using the Web Chat message source. A user asks 'What are the benefits of widgets, and can you tell me the current price?'. You first add default context, you then do a knowledge base search with the Search Text set to the incoming question. This adds the most relevant articles relating to widgets to the conversation as context. If the incoming message contains 'widgets', you could then do a database lookup to get the current price for Widgets and add 'The current price for Widgets is %Price%' as static context. ChatGPT will then be able to answer the user's questions from the context you provided.

The context itself does not appear in the chat and it is not saved by OpenAI. It is only added to the prompt sent to ChatGPT to provide context to help ChatGPT answer that specific question. The benefit of this is that you can use the standard ChatGPT models without training - and you can provide up to date context by keeping your local knowledge base updated or looking up context from your own database. It allows you to quickly create working bots, maintain up-to-date information, and harness the full potential of ChatGPT while maintaining control over their data and data privacy.

Note: ChatGPT has a limit of 4096 tokens per request (or 16384 tokens for the newer gpt-3.5-turbo-16k model and 128,000 for the gpt-4o model). Typically a token corresponds to about 4 characters of text. Prompt text includes any context added and the response from ChatGPT itself. ThinkAutomation automatically removes older context to ensure the token limit is not exceeded. If you add too much context, some of it may not be included.
Adding Tabular Context

You can add tabular context to a conversation. A user can then ask questions relating to the data.

For example, you could lookup invoices for a customer (based on the email address provided) from a database and return the data in CSV format:


invoice_number,invoice_date,product,amount_due
INV-2023351,2023-01-05,Plain Widgets,1500.00
INV-2023387,2023-01-10,Orange Niblets,2500.00
INV-2023421,2023-01-15,Flat Widgets,1800.00
INV-2023479,2023-01-20,Flat Widgets,3500.00
INV-2023521,2023-01-25,Round Niblets,1200.00

You would assign the CSV data to a %variable% and then add Static Context:


Given the following list of invoices in CSV format for the user, answer questions about this data. The 'amount_due' column gives the outstanding balance for the invoice in dollars.
%CSVData%

The chat user could then ask questions such as:

What is the date of my most recent invoice? or Can you show me a list of my invoices?

When adding context as tabular data, you need to proceed the data with a clear instruction of what the data is. You would need to experiment with the prompt text to ensure ChatGPT responds correctly.

Clear Conversation Context

This operation will clear any Context added to a conversation. Specify the Conversation Id.

Ask ChatGPT To Response To A Prompt With An Image

This operation can be used to ask a question about an image. The image can be a local file or a URL. The System Message is optional. This can help to set the behavior of the assistant. For example: 'You are a helpful assistant'.

The Prompt is the text is the question you want to ask about the provided image.

In the Image Path Or URL entry, specify a local file path or URL for the image. You can use %variable% replacements. The following image types are supported: PNG, JPEG, WEBP & GIF.

Select the variable to receive the response from the Assign ChatGPT Response To list.

Examples:

You can ask general questions about the image, such as 'What is in this image?' or 'Is this image an animal?'. You can also perform OCR, for example: 'Convert the image to text.' or 'The image is a receipt. What is the total paid and the tax?'.

Note: The OpenAI Vision API is currently in preview.
ChatGPT Rate Limits

Your OpenAI account will set a rate limit for the maximum requests per minute. The Open AI API Key - Rate Limit Retries setting determines how many times ThinkAutomation will retry the request if a rate limit error is returned. It will automatically increase the wait time for each retry. The default wait period is 30 seconds. If the request still fails after the retries then an error will be raised.

If your ChatGPT action is timing out then your OpenAI account may be rate limited. You can increase the rate limit by adding pre-payment funds to your OpenAI account. This will move your account to then next usage tier. For example, adding a $50 pre-payment will move your account to usage tier 2 - which will have a higher rate limit and faster responses. See: OpenAI Rate Limits for more information.

ChatGPT Uses

ChatGPT has many uses within ThinkAutomation. Other than being a regular chat bot that has knowledge of many subjects, you can use it to:

  • Create a Chat Bot using the Web Chat Message Source type that can answer specific questions about your business by adding private context using the Embedded Knowledge Store.
  • Provide automated responses to incoming emails or SMS messages, utilizing the Embedded Knowledge Store.
  • Create a Teams Message Source type so that Microsoft Teams users can ask your bot questions.
  • Create a Chat Bot Automation that uses the API Message Source type. This can be the same type of Automation that you would use for the Web Chat Message Source type - but called via the API. This would allow you to use ThinkAutomation from your own chat web app or external chat apps/products.
  • Parse unstructured text and extract key information.
  • Summarize text.
  • Anonymize text.
  • Classify text.
  • Translate text.
  • Sentiment analysis.
  • Correct grammar/spelling.
  • Convert natural language into code (SQL, PowerShell etc).
  • Ask questions about images or extract text from images.

and much more. See: Examples - OpenAI