Skip to content

Add Langfuse feedback support #147

@loukratz-bv

Description

@loukratz-bv

Currently, the feedback API in qtype/interpreter/feedback_api.py and telemetry.py only support Phoenix

Proposed Enhancement

Implement Langfuse feedback submission in the create_feedback_endpoint function to support users who use Langfuse as their telemetry backend.

Current State

elif telemetry.provider == "Langfuse":
    # TODO: Implement Langfuse feedback submission
    raise NotImplementedError(
        "Langfuse feedback not yet implemented"
    )

Requirements

  • Research Langfuse's feedback/annotation API
  • Implement feedback submission for all three feedback types (thumbs, rating, category)
  • Ensure it follows the same pattern as Phoenix implementation
  • Handle score conversion appropriately for each feedback type
  • Add appropriate error handling and logging

Context

The Phoenix implementation uses client.annotations.add_span_annotation() with parameters: span_id, annotation_name, label, score, explanation, and annotator_kind="HUMAN".

Langfuse will need a similar implementation adapted to its SDK/API.

Related Files

  • qtype/interpreter/feedback_api.py - Feedback endpoint implementation
  • qtype/interpreter/telemetry.py - Langfuse telemetry setup (already implemented)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions