Skip to content

Conversation

@reakaleek
Copy link
Member

@reakaleek reakaleek commented Jan 27, 2026

Summary

Fixes a bug where the Ask AI chatbot could not remember previous messages in a conversation.

Problem

When a user asked a follow-up question like "What did I ask before?", the AI responded that it had no conversation history. This happened only on the first follow-up message after starting a new conversation.

Root cause

The bug was a conversation ID mismatch between two parts of the system:

User asks question (no conversation ID)
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│  LLM Gateway generates ID "abc-123" for its ThreadId   │
└─────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│  Stream Transformer generates DIFFERENT ID "xyz-789"   │
│  and sends it to the frontend                          │
└─────────────────────────────────────────────────────────┘
        │
        ▼
User sends follow-up with ID "xyz-789"
        │
        ▼
LLM Gateway looks for conversation "xyz-789" → NOT FOUND

The LLM Gateway used one ID internally, but the Stream Transformer created a different ID for the frontend. When the user continued the conversation, the IDs did not match.

Solution

We now ensure one ID is used everywhere:

  1. When a new conversation starts, the gateway generates the ID
  2. The gateway returns this ID in a new GeneratedConversationId field
  3. The stream transformer uses this same ID when sending events to the frontend

New API design

// Simple response with one self-documenting field
public record AskAiGatewayResponse(
    Stream Stream, 
    Guid? GeneratedConversationId  // Non-null only when gateway created a new ID
);
Scenario GeneratedConversationId
New conversation (LLM Gateway) <new GUID>
Continuing conversation null
Agent Builder (any) null (it handles IDs in the stream)

Test plan

  • Start a new Ask AI conversation
  • Ask a follow-up question like "What did I just ask?"
  • Verify the AI remembers the previous question
  • Start another new conversation and verify it works again

@reakaleek reakaleek requested a review from a team as a code owner January 27, 2026 10:46
@reakaleek reakaleek requested a review from Mpdreamz January 27, 2026 10:46
@reakaleek reakaleek added the fix label Jan 27, 2026
@reakaleek reakaleek enabled auto-merge (squash) January 27, 2026 10:50
reakaleek and others added 2 commits January 27, 2026 11:54
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Comment on lines +85 to +100
catch (Exception ex)
{
Logger.LogError(ex, "Error transforming stream for transformer {TransformerType}. Stream processing will be terminated.", GetType().Name);
_ = parentActivity?.SetTag("error.type", ex.GetType().Name);
try
{
await ProcessStreamAsync(reader, writer, conversationId, parentActivity, cancellationToken);
// Complete writer first, then reader - but don't try to complete reader
// if the exception came from reading (would cause "read operation pending" error)
await writer.CompleteAsync(ex);
}
catch (OperationCanceledException ex)
catch (Exception completeEx)
{
Logger.LogDebug(ex, "Stream processing was cancelled for transformer {TransformerType}", GetType().Name);
}
catch (Exception ex)
{
Logger.LogError(ex, "Error transforming stream for transformer {TransformerType}. Stream processing will be terminated.", GetType().Name);
_ = parentActivity?.SetTag("error.type", ex.GetType().Name);
try
{
// Complete writer first, then reader - but don't try to complete reader
// if the exception came from reading (would cause "read operation pending" error)
await writer.CompleteAsync(ex);
}
catch (Exception completeEx)
{
Logger.LogError(completeEx, "Error completing pipe after transformation error for transformer {TransformerType}", GetType().Name);
}
return;
Logger.LogError(completeEx, "Error completing pipe after transformation error for transformer {TransformerType}", GetType().Name);
}
return;
}
@reakaleek reakaleek merged commit a38a8a4 into main Jan 27, 2026
31 checks passed
@reakaleek reakaleek deleted the feature/fix-conversation-id-handling branch January 27, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants