Log LLM calls

When you call an LLM directly, outside of LangChain or a supported integration, you need to provide specific metadata so that LangSmith can display token counts, calculate costs, and let you open the run in the Playground with the correct provider and model.

There are four requirements for a fully functional LLM trace:

Requirement	What to do	Enables
1. Set run_type="llm"	Pass run_type="llm" to @traceable	LLM-specific rendering, token/cost display
2. Format inputs/outputs	Use OpenAI, Anthropic, or LangChain message format	Structured message rendering, Playground support
3. Set ls_provider and ls_model_name	Pass both in metadata	Cost tracking, Playground model selection
4. Provide token counts	Set usage_metadata on the run	Token counts and cost calculation

If you are using LangChain OSS, the OpenAI wrapper, or the Anthropic wrapper, these details are handled automatically.

The examples on this page use the traceable decorator/wrapper (the recommended approach for Python and JS/TS). The same requirements apply if you use the RunTree or API directly.

Messages format

When tracing a custom model or a custom input/output format, it must either follow the LangChain format, OpenAI completions format or Anthropic messages format. For more details, refer to the OpenAI Chat Completions or Anthropic Messages documentation. The LangChain format is:

Show LangChain format

messages · array · required

A list of messages containing the content of the conversation.

role · string · required

Identifies the message type. One of: system | reasoning | user | assistant | tool

content · array · required

Content of the message. List of typed dictionaries.

Show Content options

type · string · required

One of: text | image | file | audio | video | tool_call | server_tool_call | server_tool_result.

Show text

type · literal('text') · required

text · string · required

Text content.

annotations · object[]

List of annotations for the text

extras · object

Additional provider-specific data.

Show reasoning

type · literal('reasoning') · required

text · string · required

Text content.

extras · object

Additional provider-specific data.

Show image

type · literal('image') · required

url · string

URL pointing to the image location.

base64 · string · required

Base64-encoded image data.

id · string

Reference ID to an externally stored image (e.g., in a provider’s file system or in a bucket).

mime_type · string

Image MIME type (e.g., image/jpeg, image/png).

Show file (e.g., PDFs)

type · literal('file') · required

url · string

URL pointing to the file.

base64 · string · required

Base64-encoded file data.

id · string

Reference ID to an externally stored file (e.g., in a provider’s file system or in a bucket).

mime_type · string

File MIME type (e.g., application/pdf).

Show audio

type · literal('audio') · required

url · string

URL pointing to the audio file.

base64 · string · required

Base64-encoded audio data.

id · string

Reference ID to an externally stored audio file (e.g., in a provider’s file system or in a bucket).

mime_type · string

Audio MIME type (e.g., audio/mpeg, audio/wav).

Show video

type · literal('video') · required

url · string

URL pointing to the video file.

base64 · string · required

Base64-encoded video data.

id · string

Reference ID to an externally stored video file (e.g., in a provider’s file system or in a bucket).

mime_type · string

Video MIME type (e.g., video/mp4, video/webm).

Show tool_call

type · literal('tool_call') · required

name · string

args · object · required

Arguments to pass to the tool.

id · string

Unique identifier for this tool call.

Show server_tool_call

type · literal('server_tool_call') · required

id · string · required

Unique identifier for this tool call.

name · string · required

The name of the tool to be called.

args · object · required

Arguments to pass to the tool.

Show server_tool_result

type · literal('server_tool_result') · required

tool_call_id · string · required

Identifier of the corresponding server tool call.

id · string

Unique identifier for this tool call.

status · string · required

Execution status of the server-side tool. One of: success | error.

output

Output of the executed tool.

tool_call_id · string

Must match the id of a prior assistant message’s tool_calls[i] entry. Only valid when role is tool.

usage_metadata · object

Use this field to send token counts and/or costs with your model’s output. See Provide token and cost information for more details.

Text and reasoning

 inputs = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Hi, can you tell me the capital of France?"
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The capital of France is Paris."
        },
        {
          "type": "reasoning",
          "text": "The user is asking about..."
        }
      ]
    }
  ]
}

input = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What's the weather in San Francisco?"
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [{"type": "tool_call", "name": "get_weather", "args": {"city": "San Francisco"}, "id": "call_1"}],
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "content": [
        {
          "type": "text",
          "text": "{\"temperature\": \"18°C\", \"condition\": \"Sunny\"}"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The weather in San Francisco is 18°C and sunny."
        }
      ]
    }
  ]
}

inputs = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What breed is this dog?"
        },
        {
          "type": "image",
          "url": "https://fastly.picsum.photos/id/237/200/300.jpg?hmac=TmmQSbShHz9CdQm0NkEjx1Dyh_Y984R9LpNrpvH2D_U",
          # alternative to a url, you can provide a base64 encoded image
          # "base64": "<base64 encoded image>",
          "mime_type": "image/jpeg",
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "This looks like a Black Labrador."
        }
      ]
    }
  ]
}

input = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What is the price of AAPL?"
        }
      ]
    }
  ]
}

output = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "server_tool_call",
          "name": "web_search",
          "args": {
            "query": "price of AAPL",
            "type": "search"
          },
          "id": "call_1"
        },
        {
          "type": "server_tool_result",
          "tool_call_id": "call_1",
          "status": "success"
        },
        {
          "type": "text",
          "text": "The price of AAPL is $150.00"
        }
      ]
    }
  ]
}

Convert custom I/O formats into LangSmith compatible formats

If you’re using a custom input or output format, you can convert it to a LangSmith compatible format using process_inputs/processInputs and process_outputs/processOutputs functions on the @traceable decorator (Python) or traceable function (TS).

process_inputs/processInputs and process_outputs/processOutputs accept functions that allow you to transform the inputs and outputs of a specific trace before they are logged to LangSmith. They have access to the trace’s inputs and outputs, and can return a new dictionary with the processed data.

Here’s a boilerplate example of how to use process_inputs and process_outputs to convert a custom I/O format into a LangSmith compatible format:

Show the code

Python

class OriginalInputs(BaseModel):
    """Your app's custom request shape"""

class OriginalOutputs(BaseModel):
    """Your app's custom response shape."""

class LangSmithInputs(BaseModel):
    """The input format LangSmith expects."""

class LangSmithOutputs(BaseModel):
    """The output format LangSmith expects."""

def process_inputs(inputs: dict) -> dict:
    """Dict -> OriginalInputs -> LangSmithInputs -> dict"""

def process_outputs(output: Any) -> dict:
    """OriginalOutputs -> LangSmithOutputs -> dict"""


@traceable(run_type="llm", process_inputs=process_inputs, process_outputs=process_outputs)
def chat_model(inputs: dict) -> dict:
    """
    Your app's model call. Keeps your custom I/O shape.
    The decorators call process_* to log LangSmith-compatible format.
    """

Identify a custom model in traces

When using a custom model, it is recommended to also provide the following metadata fields to identify the model when viewing traces and when filtering.

• ls_provider: The provider of the model, e.g. “openai”, “anthropic”, etc.
• ls_model_name: The name of the model, e.g. “gpt-4.1-mini”, “claude-3-opus-20240229”, etc.

Python

from langsmith import traceable

inputs = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "I'd like to book a table for two."},
]
output = {
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "Sure, what time would you like to book the table for?"
            }
        }
    ]
}

@traceable(
    run_type="llm",
    metadata={"ls_provider": "my_provider", "ls_model_name": "my_model"}
)
def chat_model(messages: list):
    return output

chat_model(inputs)

import { traceable } from "langsmith/traceable";

const messages = [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "I'd like to book a table for two." }
];
const output = {
    choices: [
        {
            message: {
                role: "assistant",
                content: "Sure, what time would you like to book the table for?",
            },
        },
    ],
    usage_metadata: {
        input_tokens: 27,
        output_tokens: 13,
        total_tokens: 40,
    },
};

// Can also use one of:
// const output = {
//     message: {
//         role: "assistant",
//         content: "Sure, what time would you like to book the table for?"
//     }
// };
//
// const output = {
//     role: "assistant",
//     content: "Sure, what time would you like to book the table for?"
// };
//
// const output = ["assistant", "Sure, what time would you like to book the table for?"];

const chatModel = traceable(
    async ({ messages }: { messages: { role: string; content: string }[] }) => {
        return output;
    },
    {
        run_type: "llm",
        name: "chat_model",
        metadata: {
            ls_provider: "my_provider",
            ls_model_name: "my_model"
        }
    }
);

await chatModel({ messages });

This code will log the following trace:

If you implement a custom streaming chat_model, you can “reduce” the outputs into the same format as the non-streaming version. This is currently only supported in Python.

def _reduce_chunks(chunks: list):
    all_text = "".join([chunk["choices"][0]["message"]["content"] for chunk in chunks])
    return {"choices": [{"message": {"content": all_text, "role": "assistant"}}]}

@traceable(
    run_type="llm",
    reduce_fn=_reduce_chunks,
    metadata={"ls_provider": "my_provider", "ls_model_name": "my_model"}
)
def my_streaming_chat_model(messages: list):
    for chunk in ["Hello, " + messages[1]["content"]]:
        yield {
            "choices": [
                {
                    "message": {
                        "content": chunk,
                        "role": "assistant",
                    }
                }
            ]
        }

list(
    my_streaming_chat_model(
        [
            {"role": "system", "content": "You are a helpful assistant. Please greet the user."},
            {"role": "user", "content": "polly the parrot"},
        ],
    )
)

If ls_model_name is not present in extra.metadata, other fields might be used from the extra.metadata for estimating token counts. The following fields are used in the order of precedence:

1. metadata.ls_model_name
2. inputs.model
3. inputs.model_name

To learn more about how to use the metadata fields, refer to the Add metadata and tags guide.

Provide token and cost information

Token counts enable cost calculation and are displayed in the trace UI. There are two ways to provide them:

• Set usage_metadata on the run tree: call get_current_run_tree() / getCurrentRunTree() inside your @traceable function and set the usage_metadata field. This does not change your function’s return value.
• Return usage_metadata in the output: include usage_metadata as a top-level key in the dictionary your function returns.

Supported usage_metadata fields

Field	Type	Description
input_tokens	int	Total input/prompt tokens
output_tokens	int	Total output/completion tokens
total_tokens	int	Sum of input + output (optional, can be inferred)
input_token_details	object	Breakdown: cache_read, cache_creation, audio, text, image
output_token_details	object	Breakdown: reasoning, audio, text, image

To send costs directly (for non-linear pricing), you can also include input_cost, output_cost, and total_cost fields. See Cost tracking for details on configuring model pricing and viewing costs in the UI.

Time-to-first-token

If you are using traceable or one of our SDK wrappers, LangSmith will automatically populate time-to-first-token for streaming LLM runs. However, if you are using the RunTree API directly, you will need to add a new_token event to the run tree in order to properly populate time-to-first-token.

Here’s an example:

Python

from langsmith.run_trees import RunTree
run_tree = RunTree(
    name="CustomChatModel",
    run_type="llm",
    inputs={ ... }
)
run_tree.post()
llm_stream = ...
first_token = None
for token in llm_stream:
    if first_token is None:
      first_token = token
      run_tree.add_event({
        "name": "new_token"
      })
run_tree.end(outputs={ ... })
run_tree.patch()

import { RunTree } from "langsmith";
const runTree = new RunTree({
    name: "CustomChatModel",
    run_type: "llm",
    inputs: { ... },
});
await runTree.postRun();
const llmStream = ...;
let firstToken;
for (const token of llmStream) {
    if (firstToken == null) {
        firstToken = token;
        runTree.addEvent({ name: "new_token" });
    }
}
await runTree.end({
    outputs: { ... },
});
await runTree.patchRun();

---

Edit this page on GitHub or file an issue.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.