Skip to content

Python: fix(python): include cache tokens in input_token_count for Bedrock and Claude - #10003

Open
123XieC wants to merge 2 commits into
microsoft:mainfrom
123XieC:fix/include-cache-tokens-in-input-count
Open

123XieC wants to merge 2 commits into
microsoft:mainfrom
123XieC:fix/include-cache-tokens-in-input-count

Conversation

@123XieC

@123XieC 123XieC commented Nov 3, 2026

Copy link
Copy Markdown

Motivation & Context

Same root cause as #9808 (AnthropicClient, fixed by #9859), in two more connectors that #9859 did not touch.

  • BedrockChatClient._parse_usage maps Converse inputTokens to input_token_count. With prompt caching, Bedrock reports cache reads/writes separately in cacheReadInputTokens / cacheWriteInputTokens, and inputTokens covers only the uncached part. So on main the usage is internally inconsistent (input 12 + output 40 but total 9352).
  • ClaudeAgent builds input_token_count from the Claude Agent SDK ResultMessage.usage["input_tokens"], which is Anthropic's uncached-only field, and computes total_token_count = input + output, so both undercount.

input_token_count is total input for the OpenAI and Gemini clients and is exported as gen_ai.usage.input_tokens, which the OTel GenAI semconv says should include cached tokens.

Description & Review Guide

  • Bedrock (_parse_usage): sum cacheReadInputTokens + cacheWriteInputTokens into input_token_count. Cache-specific fields are unchanged.
  • Claude (ClaudeAgent): sum cache_creation_input_tokens + cache_read_input_tokens into input_token_count, and derive total_token_count from the inclusive input total. Cache-specific fields are unchanged.
  • Updated the two existing unit tests whose expected values now reflect the inclusive input count:
    • test_parse_usage_surfaces_cache_tokens: input_token_count 10 → 21.
    • test_run_captures_result_message_usage_and_finish_reason: input_token_count 42 → 50, total_token_count 60 → 68.

No new tests were added because the existing tests already cover the cache-token paths and their updated assertions verify the fix.

Verification

  • python -m pytest python/packages/bedrock/tests/test_bedrock_client.py -q → all pass, including the updated test_parse_usage_surfaces_cache_tokens.
  • python -m pytest python/packages/claude/tests/test_claude_agent.py -q → all pass, including the updated test_run_captures_result_message_usage_and_finish_reason.

Related Issue

Fixes #9988

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: locating the two connectors' usage parsing, and drafting the minimal change following the already-merged #9859 pattern. I reproduced the undercount locally and ran the updated tests myself.

Copilot AI balanced review requested due to automatic review settings November 3, 2026 03:15
@123XieC
123XieC temporarily deployed to github-app-auth November 3, 2026 03:15 — with GitHub Actions Inactive
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Nov 3, 2026
@github-actions github-actions Bot changed the title fix(python): include cache tokens in input_token_count for Bedrock and Claude Python: fix(python): include cache tokens in input_token_count for Bedrock and Claude Nov 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the Bedrock and Claude cache-token accounting changes.

No open issues 🎉

What changed in this PR

Extends the cache-token accounting fix from #9859 to the Bedrock and Claude connectors, addressing #9988 and aligning input usage with other providers.

Changes:

  • Include cache read and write/creation tokens in both connectors’ input_token_count, preserving separate cache counters.
  • Compute Claude’s total_token_count from the inclusive input count plus output tokens.
  • Update existing cache-token tests to assert the corrected counts.
File Description
python/​packages/​claude/​tests/​test_claude_agent.py Updates expected input and total counts to include cached tokens.
python/​packages/​claude/​agent_framework_claude/​_agent.py Includes cache creation and read tokens in input and total usage.
python/​packages/​bedrock/​tests/​test_bedrock_client.py Updates the expected input count to include cached tokens.
python/​packages/​bedrock/​agent_framework_bedrock/​_chat_client.py Includes cache read and write tokens in input usage.

Review effort: Balanced

@123XieC
123XieC deployed to github-app-auth November 7, 2026 02:30 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
github-app-auth — c4d37300 Deployed Nov 7, 2026 by 123XieC via add_label #27338
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: BedrockChatClient and ClaudeAgent report input_token_count without cached prompt tokens

2 participants