Research-backed calculator guide

Customer-Level Token Usage Tracking: What to Log for Accurate AI Margins

Learn what to log for customer-level token usage tracking, how to allocate billable AI cost, and how to correct estimates without overstating SaaS margins.

What to log for customer-level token usage tracking

1. Identity and ownership

Use a durable internal tenant ID rather than an email address as the ledger key. A customer can change its domain, contact, plan, or billing account while remaining the same economic unit. If your product has workspaces, seats, or reseller accounts, capture each level explicitly: account_id, workspace_id, user_id, and, when relevant, contract_id. Keep a separate mapping table for customer mergers, deletions, and plan changes.

Propagate identity before the first model span, not after the response. Langfuse’s user-tracking documentation describes propagating a userId across observations so usage cost can be aggregated by user, and it cautions that the value should be a short string and set early enough to cover child observations.^[2] The same design applies to a home-grown ledger: attach identity to the root request and inherit it through retrieval, tool, retry, and model-call spans.

Treat identity as sensitive operational data. Hash or tokenise identifiers in analytics if analysts do not need the customer’s literal name. Never put prompt contents, secrets, or full tool payloads in a high-cardinality dashboard label by default. Retain a privacy-safe join key and a controlled audit location for detailed debugging.

2. Usage and request context

Record the provider-returned usage object verbatim in a raw column, then normalize it into typed fields. At minimum, normalize input tokens, output tokens, and total tokens. Add cache-read, cache-write, audio, image, and reasoning fields when they exist. The normalized fields should preserve the provider’s unit and semantics; tokens are not interchangeable with seconds, characters, images, or tool invocations.

Record request_id, trace_id, span_id, parent_request_id, started_at, completed_at, status, finish_reason, streaming, and retry_number. A parent trace lets you answer two different questions: “What did this customer’s workflow consume?” and “What did the successful answer cost?” The first includes charged retries and parallel calls. The second can be a product metric, but it is not the COGS total.

OpenTelemetry’s GenAI attributes include the provider, requested and response model, operation, request settings, response ID, token type, input and output usage, cache usage, and tool metadata.^[3] You do not need to adopt every attribute, but using a stable subset prevents each provider adapter from inventing a different schema.

3. Pricing and classification

Store price_card_id or pricing_version on each costed row. A model name alone does not prove the rate: rates can vary by input versus output, cache status, context tier, service tier, region, batch mode, or contract. Store the numerator and denominator used in the calculation, such as USD per 1,000,000 tokens, rather than only the rounded dollar result.

Then classify the row. Useful statuses include customer_billable, internal_nonbillable, trial_included, provider_credit, failed_no_usage, pending_reconciliation, and corrected. Classification should be a decision with a reason and actor or rule version. If a row moves from estimate to provider-confirmed actual, append a correction event that references the original row.

Langfuse describes two legitimate cost paths: ingest provider usage and cost, or infer cost from a model definition and usage. It gives ingested values priority and supports custom model definitions when a model or private rate is not in its defaults.^[1] That distinction matters for customer margins. An inferred amount is an estimate until matched to your provider’s usage or invoice evidence.

A ledger design that avoids common errors

A practical event row can look like this:

usage_event_id, account_id, workspace_id, user_id, feature,
provider, model, request_id, trace_id, parent_request_id,
input_tokens, output_tokens, cache_read_tokens, reasoning_tokens,
request_status, retry_number, environment, billable_class,
price_card_id, estimated_cost_usd, provider_cost_usd, occurred_at

Keep estimated_cost_usd and provider_cost_usd separate. If the response gives token counts but not cost, calculate an estimate using the rate card active at the request time. If a provider invoice later says the applicable rate or unit was different, populate the actual field and create a delta event. This preserves the original forecast and makes the margin change explainable.

Use mutually exclusive buckets. If a provider reports cached input inside its input total, do not add the cache-read count again. Langfuse explicitly warns that overlapping usage buckets double count usage and inferred cost, and shows how an inclusive prompt count can be converted into ordinary input plus cached-input buckets.^[1] Your own warehouse should enforce the same invariant: the sum of exclusive buckets equals the provider total for the event, within a documented provider-specific rule.

Separate attempts from customer-visible requests. A retry may share a parent_request_id with the original attempt while retaining its own provider request ID and usage. A parallel agent fan-out should have one parent workflow and multiple child calls. This lets finance sum all charged calls, while product teams can also inspect cost per completed workflow.

Mark traffic by environment. Production customer traffic, internal QA, sandbox usage, and offline evaluation should not share the same billable default. A missing environment value should be an exception queue, not an automatic customer charge. Likewise, a missing customer ID should be classified as unattributed_pending, then resolved through a join to the authenticated request—not guessed from a prompt.

Worked cost example

Assume a SaaS team wants an illustrative estimate for one customer, Acme, during a one-day pilot. The following rates are assumptions for the example, not a vendor quote: $2.00 per 1 million input tokens and $8.00 per 1 million output tokens, with no cache discount. The ledger has 120 successful production requests and 6 provider-charged retries. Ten internal evaluation requests are excluded from Acme’s billable COGS but remain visible in total platform spend.

The customer-level COGS estimate is therefore $0.912, not $0.974. The latter would combine Acme’s billable usage with internal evaluation traffic. If your pricing policy allocates internal evaluation to all active customers, document that allocation separately rather than quietly attaching it to Acme’s token row. For a month, multiply the daily result only if traffic shape is stable; otherwise sum daily events because retries, model mix, caching, and plan limits vary.

A provider may later confirm that 18,000 of the 228,000 input tokens were cache reads at a different rate. Replace or supplement the estimate with the provider’s actual unit treatment, post a correction event, and recompute the customer-period total. Never edit the original event so that the audit trail loses the reason for the change.

From usage rows to AI SaaS margin

A customer margin report needs more than token totals. Join the customer-period rollup to recognized revenue, included usage, credits, refunds, and any shared infrastructure allocation. Report at least: gross revenue, customer-billable model COGS, allocated shared AI COGS, gross profit, gross margin percentage, included-usage consumption, and confidence status.

A simple margin formula is:

customer gross margin %
= (recognized revenue - customer AI COGS - allocated shared AI COGS)
  / recognized revenue × 100

Label the result as estimated when any material cost is inferred, usage is missing, or invoice reconciliation is pending. The site’s AI API invoice reconciliation calculator is useful when you need to compare request-level estimates with provider totals. For a broader operating model, use the AI Agent Cost Calculator to include non-token operating costs, and compare instrumentation spend or incident savings with the AI observability ROI calculator.

Do not force every platform cost into a customer row. Gateway subscription fees, observability retention, dedicated infrastructure, and human support may be shared costs. Choose a documented allocation key such as active customers, successful AI tasks, or attributed token spend. The key is less important than consistency, versioning, and showing the unallocated remainder.

Operational checks before you trust the dashboard

First, reconcile daily provider usage totals to the sum of event rows by provider, model, and billing period. Investigate missing, duplicated, and late events. Second, compare raw provider totals with normalized exclusive buckets. Third, sample a handful of traces and verify that the customer ID survives retrieval, tool calls, streaming, and retries. Fourth, compare estimated dollars with invoice dollars and report variance by model and price-card version.

Watch for five failure patterns. Counting visible answer text misses hidden reasoning or provider-reported output. Counting requests instead of tokens hides prompt growth and long completions. Treating a retry as free understates COGS. Adding cached tokens to an inclusive input total double counts. Finally, using a preflight estimate as an actual charge can misstate cost: Anthropic says its token-counting endpoint is an estimate, may include system-added tokens that are not billed, and recommends using response usage for requests with unsupported server tools or MCP connectors.^[4]

A customer token usage dashboard should therefore show both “estimated cost” and “provider-confirmed cost,” with filters for account, plan, feature, model, environment, and billable class. Add a queue for unattributed or pending rows. The dashboard is a control surface, not the ledger of record; the append-only event table and correction history are what make a finance review defensible.

FAQ

What is the best identifier for customer-level token usage tracking?

Use an immutable internal account or tenant ID as the economic owner, then carry workspace and user IDs as additional dimensions. Do not make an email address the primary key. Keep an auditable mapping when accounts merge, split, or change billing ownership.

Should failed LLM requests count toward a customer’s token cost?

Count them in AI COGS when the provider reports billable usage. Whether the customer is charged is a separate product-policy decision. Record both facts: provider usage and customer billing class. A failed application response can still have consumed and incurred input or output tokens.

How do I handle retries without double counting?

Give each attempt its own provider request ID and usage row, then link attempts to one parent workflow or customer-visible request. Sum every charged attempt for COGS. Use the parent relationship to explain why the customer saw one answer even though the provider billed multiple calls.

Is total token count enough for a customer token usage dashboard?

No. Split input, output, cache, and reasoning usage where available, and retain provider-specific raw fields. The same total can reflect prompt bloat, long answers, cache reads, or agent retries. Those causes need different margin and engineering actions.

How often should customer usage be reconciled to invoices?

Use daily checks for missing or duplicated events and a formal period-end reconciliation after the provider’s billing data is available. Keep the date range, provider export or API response, rate-card version, currency, and unresolved variance with the reconciliation record. Do not represent an estimate as a final invoice match.