Integrate Voice Agents into Agents SDK #542
Conversation
|
commit: |
|
@renandincer do you mind fixing all the build issues please? |
|
I'll take over this PR. thanks @renandincer / @itzmanish! |
|
cool! when is this gonna be released guys? |
|
@itzmanish can you give me access to push to this branch |
craigsdennis
left a comment
craigsdennis
left a comment
There was a problem hiding this comment.
Love seeing this progress, I was just perusing and dropped some thoughts
| JSON.stringify({ | ||
| type: "media", | ||
| version: 1, | ||
| identifier: randomUUID(), |
There was a problem hiding this comment.
Identifier is random in every loop..that intentional?
| } from "./realtime"; | ||
|
|
||
| // export const REALTIME_AGENTS_SERVICE = "https://agents.realtime.cloudflare.com"; | ||
| export const REALTIME_AGENTS_SERVICE = "https://kind-waves-give.loca.lt"; |
| return split.length > 2; | ||
| } | ||
|
|
||
| export async function* processNDJSONStream( |
There was a problem hiding this comment.
This looks like SSE parsing, do we have a library/utility here already?
| { | ||
| private _state = DEFAULT_STATE as State; | ||
| private _disposables = new DisposableStore(); | ||
| private realtime?: Realtime; |
There was a problem hiding this comment.
Probably want an _realtime here to follow existing pattern
| Props extends Record<string, unknown> = Record<string, unknown> | ||
| > | ||
| extends Server<Env, Props> | ||
| implements RealtimePipelineComponent |
There was a problem hiding this comment.
This feels like we want a wrapper around the Agent that turns it into a pipeline component. Exposes all the stuff in L#327 - L#344. Could wrap any primitive that way. Feels odd to "install" by default
There was a problem hiding this comment.
But I might not be understanding what a "component" is in this case.
| await this.mcp.restoreConnectionsFromStorage(this.name); | ||
| await this.broadcastMcpServers(); | ||
| if (this.realtimePipelineComponents) { | ||
| const { CF_ACCOUNT_ID, CF_API_TOKEN } = this.env as { |
There was a problem hiding this comment.
This would do well in the wrapper. @threepointone can you get at these values without forcing them to be added to env?
| }; | ||
| }; | ||
|
|
||
| export class Realtime { |
There was a problem hiding this comment.
This really feels like it should be a binding to me. Like I want env.REALTIME and all this stuff is configured in wrangler I never see it.
| // Different STT provider | ||
| const stt = new CustomSTT(this.env.CUSTOM_API_KEY); | ||
|
|
||
| // Multiple TTS voices |
There was a problem hiding this comment.
What is this use case?
There was a problem hiding this comment.
This probably deserves it's own folder if it stays in the library. I think realtime is where I would look.
There was a problem hiding this comment.
Is there a front-end for this?
6 participants
No description provided.