What the generations CSV export contains, what the scores file adds, and how they map to structured evaluation calls

When your application is instrumented to record them, Langfuse generations contain prompts, completions, token counts, latency, and scores. Exporting those generations lets you use recorded calls as evaluation inputs.

This guide covers the exact columns that matter, how scores attach to individual calls, and what the importer does when columns are missing.

Exporting from Langfuse

The import expects a generations CSV, not a traces CSV. In Langfuse:

  1. •Open the generations table under Tracing. If your version uses an observations table, select generation observations
  2. •Apply the date range and model filters you need
  3. •Use the export menu to choose As CSV
  4. •Download the completed export from Project Settings → Exports, then check that its header contains the required column names below

Langfuse documents the table export flow. Its UI and export schemas can change; this importer requires the camel-case columns listed below, so an export using names such as trace_id needs converting before upload.

Optionally, export a scores CSV to include human review notes. Scores join to generations by observation ID or trace ID. Only scores with source: HUMAN_ANNOTATION are saved as feedback notes on the imported call; automated scores are not saved as evaluation results.

A trace-level export with id instead of the required traceId column fails column validation. Check the file header rather than relying on the export filename.

Required columns

Three columns must be present for the import to proceed:

ColumnPurpose
idObservation ID, used to join observation-level scores
traceIdSource trace ID; also used for deduplication
inputPrompt messages (JSON-encoded)

If any of these are missing, the import returns a column validation error that names the missing field.

How the input column is parsed

Langfuse stores input as a JSON-encoded messages array:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Summarise this invoice: ..."}
]

The importer joins system and developer messages in order as system_prompt, and takes the last user message as test_input (the input that varies per call). A longer conversation is also kept as conversation history.

If input is a plain string rather than a JSON array, it's treated as the user message and system prompt is left blank.

Field mapping

Langfuse columnEvaluation field
idExternal observation ID
traceIdTrace ID and deduplication key
modelModel name
inputPrompt (full messages) + system_prompt + test_input (extracted)
outputResponse
usage.inputInput token count
usage.outputOutput token count
usage.totalTotal token count
latencyCopied as milliseconds without unit conversion; check the export unit
calculatedTotalCostCost in USD
levelCall status (DEFAULT or ERROR)
statusMessageError message if level is ERROR
modelParametersTemperature, top_p, and other parameters (JSON)
tagsCopied as-is to the evaluation call

Adding scores

The parser reads score names, numeric values, comments, and sources. On the saved call, HUMAN_ANNOTATION scores become text notes such as accuracy=1.0 (correct total). It does not save the score objects as structured evaluation results or copy automated judge scores into the call feedback.

Scores are joined in this order:

  1. •Scores whose observationId matches the generation id
  2. •Additional trace-level scores whose traceId matches and whose observationId is empty

Multiple matching human scores are included in the feedback notes, even if they share a name.

Deduplication

Deduplication runs when you explicitly name an existing experiment under the same agent. It compares the incoming traceId with that experiment's trace_id:<traceId> tags. Without an existing experiment name, each import creates a new experiment.

This check operates at trace level, not generation level. A later import can skip a new generation if another generation from the same trace is already present. Rows with an empty traceId cannot be matched this way. Check the import summary before assuming every generation was added.

Missing and optional fields

The importer is permissive about optional columns:

  • •Missing output: Response field is left blank. The call is still valid; you can evaluate token counts and latency.
  • •Missing model: The model is left empty.
  • •Missing cost columns: Cost is not recomputed for this CSV format; the saved call uses zero. Treat that as missing cost data, not a free call.
  • •Malformed JSON in input: The raw string is used as the prompt; parsing errors don't block the row.

Import limits

A single import accepts up to 5,000 generations. For larger exports, split by date range and import in batches. To append to an existing experiment, specify its name and review the trace-level deduplication behavior above.

FAQ

Can I import scores without a matching generations file?

No. The scores file is an optional supplement to the generations file, not a standalone import.

What happens if Langfuse changed its CSV column names?

The importer checks for the three required columns (id, traceId, input) and fails fast with a clear error if they're missing. For optional columns, missing ones are silently skipped.

Can I import generations from multiple Langfuse projects in one file?

The importer treats the CSV as a flat list and does not preserve project boundaries unless you include them as tags. It creates one experiment, or appends to an explicitly named existing experiment under the chosen agent. Keep source IDs distinguishable and review the imported rows.

Does the importer use calculatedTotalCost from Langfuse, or recalculate?

It uses the numeric calculatedTotalCost value, including zero. It does not recalculate a missing cost; the saved call then uses zero. Preserve source costs when you need reliable cost comparisons.

Will a re-import update existing calls with new scores?

No. Calls whose traceId matches an existing imported trace are skipped. A re-import is not an update operation. Use a separate experiment to compare a refreshed export without changing the earlier import.