Can the SDK tell me the number of input tokens a message will consume before sending?

[danelliottster]

I want to include as much information as possible to the LLM without exceeding the input token limit. Can I tell, exactly, how many tokens an input message will be prior to sending the request?

Thank you.

[rm2631]

Yes you can using tiktoken. This will estimate the number of tokens your request will send.
Here's a function to do so:

def _num_tokens_from_string(string: str, model_name="gpt-3.5-turbo") -> int:
    """Returns the number of tokens in a text string.
    :param string: text string
    :return: number of tokens
    """
    encoding = tiktoken.encoding_for_model(model_name)
    num_tokens = len(encoding.encode(string))
    return num_tokens

Here's the doc: https://pypi.org/project/tiktoken/

Simply call this function in a while loop and reduce the length of your prompt in the loop until you fit within your token limit. I also usually allow for some room between the limit and what I send (95% of 4096 for instance)

[strangeleo02]

Handling tiktoken Encodings Safely with Fallbacks

When estimating token counts using tiktoken, relying strictly on a dynamic or hardcoded model name can cause your application to throw a KeyError. This happens if the library is not updated frequently enough to recognize a newly released model name, even though the underlying tokenization encoding schema remains identical to previous versions.

To make your token-counting logic resilient against future model releases, you should implement a fallback mechanism. If fetching the encoding by the model name fails, the function catches the error and falls back to a standard, widely used base encoding identifier.

Robust Token Counting & Budget Trimming

Here is the updated implementation featuring an encoding fallback strategy and a safe token-budget trimming loop:

import tiktoken

def num_tokens_from_string(string: str, model_name: str) -> int:
    """Returns the number of tokens in a text string. 
    Falls back to a standard base encoding if the specific model name is unrecognized.
    """
    try:
        encoding = tiktoken.encoding_for_model(model_name)
    except KeyError:
        # Fallback to a standard base encoding if the specific model string 
        # isn't recognized by the currently installed tiktoken version.
        encoding = tiktoken.get_encoding("cl100k_base")
        
    return len(encoding.encode(string))

[tamish-max]

Not exactly from the SDK alone.

For a preflight estimate, the usual tool is tiktoken, but it is still only an estimate because the server can add overhead for message wrappers, tools, JSON schema, etc.

For the exact number actually used, inspect response.usage.prompt_tokens after the request.

The practical workflow is:

• estimate locally with tiktoken;
• leave headroom, especially if you use tools / structured outputs;
• calibrate against usage.prompt_tokens from real calls.

So: exact before send = generally no; good estimate before send = yes.