Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions docs/core-concepts/tools-function-calling.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,89 @@ use Prism\Prism\Facades\Tool;
$tool = Tool::make(SearchTool::class);
```

## Concurrent Tool Execution

When the AI calls multiple tools in a single step, Prism normally executes them sequentially. For I/O-bound operations like API calls or database queries, you can enable concurrent execution to run tools in parallel, reducing total wait time.

### Marking Tools as Concurrent

Use the `concurrent()` method to mark a tool as safe for parallel execution:

```php
use Prism\Prism\Facades\Tool;

$weatherTool = Tool::as('weather')
->for('Get current weather conditions')
->withStringParameter('city', 'The city to get weather for')
->using(function (string $city): string {
// API call that takes ~500ms
return Http::get("https://api.weather.com/{$city}")->json('conditions');
})
->concurrent();

$stockTool = Tool::as('stock_price')
->for('Get current stock price')
->withStringParameter('symbol', 'The stock ticker symbol')
->using(function (string $symbol): string {
// Another API call that takes ~500ms
return Http::get("https://api.stocks.com/{$symbol}")->json('price');
})
->concurrent();
```

When the AI calls both tools in a single step, they'll execute in parallel instead of sequentially - taking ~500ms total instead of ~1000ms.

### How It Works

Prism uses [Laravel's Concurrency facade](https://laravel.com/docs/12.x/concurrency) to execute concurrent tools. Under the hood, tools marked as concurrent are grouped and run in parallel, while sequential tools run one at a time.

The execution flow:
1. Prism groups tool calls by their concurrency setting
2. Concurrent tools execute in parallel via `Concurrency::run()`
3. Sequential tools execute one at a time
4. Results are returned in the original order, regardless of execution order

### When to Use Concurrent Tools

**Good candidates for concurrent execution:**
- External API calls (weather, stocks, search)
- Database queries that don't depend on each other
- File reads from different sources
- Any I/O-bound operation

**Keep sequential (don't mark as concurrent):**
- Tools that modify shared state
- Tools where execution order matters
- Tools with side effects that could conflict
- CPU-bound operations (concurrency won't help)

### Mixed Execution

You can mix concurrent and sequential tools in the same request:

```php
$searchTool = Tool::as('search')
->for('Search the web')
->withStringParameter('query', 'Search query')
->using(fn (string $query): string => $this->search($query))
->concurrent(); // Safe to run in parallel

$saveResultTool = Tool::as('save_result')
->for('Save a result to the database')
->withStringParameter('data', 'Data to save')
->using(fn (string $data): string => $this->save($data));
// Sequential - modifies database state
```

Prism handles the grouping automatically. Concurrent tools run in parallel, then sequential tools run in order.

### Error Handling

Errors in concurrent tools are handled the same way as sequential tools. If one concurrent tool fails, other concurrent tools still complete, and all results (including errors) are returned in the original order.

> [!NOTE]
> Concurrent execution requires Laravel's Concurrency feature, available in Laravel 11+. Make sure you have the appropriate concurrency driver configured. See [Laravel's Concurrency documentation](https://laravel.com/docs/12.x/concurrency) for setup details.

## Using Laravel MCP Tools
You can use existing [Laravel MCP](https://github.com/laravel/mcp) Tools in Prism directly, without using the Laravel MCP Server:

Expand Down
3 changes: 3 additions & 0 deletions phpunit.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,7 @@
<directory suffix=".php">./src</directory>
</include>
</source>
<php>
<env name="CONCURRENCY_DRIVER" value="sync"/>
</php>
</phpunit>
178 changes: 132 additions & 46 deletions src/Concerns/CallsTools.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
namespace Prism\Prism\Concerns;

use Generator;
use Illuminate\Support\Facades\Concurrency;
use Illuminate\Support\ItemNotFoundException;
use Illuminate\Support\MultipleItemsFoundException;
use Prism\Prism\Exceptions\PrismException;
Expand Down Expand Up @@ -47,67 +48,152 @@ protected function callTools(array $tools, array $toolCalls): array
*/
protected function callToolsAndYieldEvents(array $tools, array $toolCalls, string $messageId, array &$toolResults): Generator
{
foreach ($toolCalls as $toolCall) {
$groupedToolCalls = $this->groupToolCallsByConcurrency($tools, $toolCalls);

$executionResults = $this->executeToolsWithConcurrency($tools, $groupedToolCalls, $messageId);

foreach (array_keys($toolCalls) as $index) {
$result = $executionResults[$index];

$toolResults[] = $result['toolResult'];

foreach ($result['events'] as $event) {
yield $event;
}
}
}

/**
* @param Tool[] $tools
* @param ToolCall[] $toolCalls
* @return array{concurrent: array<int, ToolCall>, sequential: array<int, ToolCall>}
*/
protected function groupToolCallsByConcurrency(array $tools, array $toolCalls): array
{
$concurrent = [];
$sequential = [];

foreach ($toolCalls as $index => $toolCall) {
try {
$tool = $this->resolveTool($toolCall->name, $tools);
$output = call_user_func_array(
$tool->handle(...),
$toolCall->arguments()
);

if (is_string($output)) {
$output = new ToolOutput(result: $output);
if ($tool->isConcurrent()) {
$concurrent[$index] = $toolCall;
} else {
$sequential[$index] = $toolCall;
}
} catch (PrismException) {
$sequential[$index] = $toolCall;
}
}

$toolResult = new ToolResult(
toolCallId: $toolCall->id,
toolName: $toolCall->name,
args: $toolCall->arguments(),
result: $output->result,
toolCallResultId: $toolCall->resultId,
artifacts: $output->artifacts,
);
return [
'concurrent' => $concurrent,
'sequential' => $sequential,
];
}

$toolResults[] = $toolResult;
/**
* @param Tool[] $tools
* @param array{concurrent: array<int, ToolCall>, sequential: array<int, ToolCall>} $groupedToolCalls
* @return array<int, array{toolResult: ToolResult, events: array<int, ToolResultEvent|ArtifactEvent>}>
*/
protected function executeToolsWithConcurrency(array $tools, array $groupedToolCalls, string $messageId): array
{
$results = [];

yield new ToolResultEvent(
id: EventID::generate(),
timestamp: time(),
toolResult: $toolResult,
messageId: $messageId,
success: true
);
$concurrentClosures = [];

foreach ($toolResult->artifacts as $artifact) {
yield new ArtifactEvent(
id: EventID::generate(),
timestamp: time(),
artifact: $artifact,
toolCallId: $toolCall->id,
toolName: $toolCall->name,
messageId: $messageId,
);
}
} catch (PrismException $e) {
$toolResult = new ToolResult(
toolCallId: $toolCall->id,
toolName: $toolCall->name,
args: $toolCall->arguments(),
result: $e->getMessage(),
toolCallResultId: $toolCall->resultId,
);
foreach ($groupedToolCalls['concurrent'] as $index => $toolCall) {
$concurrentClosures[$index] = fn () => $this->executeToolCall($tools, $toolCall, $messageId);
}

if ($concurrentClosures !== []) {
foreach (Concurrency::run($concurrentClosures) as $index => $result) {
$results[$index] = $result;
}
}

foreach ($groupedToolCalls['sequential'] as $index => $toolCall) {
$results[$index] = $this->executeToolCall($tools, $toolCall, $messageId);
}

return $results;
}

/**
* @param Tool[] $tools
* @return array{toolResult: ToolResult, events: array<int, ToolResultEvent|ArtifactEvent>}
*/
protected function executeToolCall(array $tools, ToolCall $toolCall, string $messageId): array
{
$events = [];

try {
$tool = $this->resolveTool($toolCall->name, $tools);
$output = call_user_func_array(
$tool->handle(...),
$toolCall->arguments()
);

if (is_string($output)) {
$output = new ToolOutput(result: $output);
}

$toolResult = new ToolResult(
toolCallId: $toolCall->id,
toolName: $toolCall->name,
args: $toolCall->arguments(),
result: $output->result,
toolCallResultId: $toolCall->resultId,
artifacts: $output->artifacts,
);

$toolResults[] = $toolResult;
$events[] = new ToolResultEvent(
id: EventID::generate(),
timestamp: time(),
toolResult: $toolResult,
messageId: $messageId,
success: true
);

yield new ToolResultEvent(
foreach ($toolResult->artifacts as $artifact) {
$events[] = new ArtifactEvent(
id: EventID::generate(),
timestamp: time(),
toolResult: $toolResult,
artifact: $artifact,
toolCallId: $toolCall->id,
toolName: $toolCall->name,
messageId: $messageId,
success: false,
error: $e->getMessage()
);
}

return [
'toolResult' => $toolResult,
'events' => $events,
];
} catch (PrismException $e) {
$toolResult = new ToolResult(
toolCallId: $toolCall->id,
toolName: $toolCall->name,
args: $toolCall->arguments(),
result: $e->getMessage(),
toolCallResultId: $toolCall->resultId,
);

$events[] = new ToolResultEvent(
id: EventID::generate(),
timestamp: time(),
toolResult: $toolResult,
messageId: $messageId,
success: false,
error: $e->getMessage()
);

return [
'toolResult' => $toolResult,
'events' => $events,
];
}
}

Expand Down
1 change: 1 addition & 0 deletions src/Facades/Tool.php
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
* @method static BaseTool for(string $description)
* @method static BaseTool using(Closure|callable $fn)
* @method static BaseTool make(string|Tool|\Laravel\Mcp\Server\Tool $tool)
* @method static BaseTool concurrent(bool $concurrent = true)
* @method static BaseTool withParameter(Schema $parameter, bool $required = true)
* @method static BaseTool withStringParameter(string $name, string $description, bool $required = true)
* @method static BaseTool withNumberParameter(string $name, string $description, bool $required = true)
Expand Down
14 changes: 14 additions & 0 deletions src/Tool.php
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ class Tool
/** @var null|false|Closure(Throwable,array<int|string,mixed>):string */
protected null|false|Closure $failedHandler = null;

protected bool $concurrent = false;

public function __construct()
{
//
Expand Down Expand Up @@ -111,6 +113,18 @@ public function withErrorHandling(?Closure $handler = null): self
return $this;
}

public function concurrent(bool $concurrent = true): self
{
$this->concurrent = $concurrent;

return $this;
}

public function isConcurrent(): bool
{
return $this->concurrent;
}

public function withParameter(Schema $parameter, bool $required = true): self
{
$this->parameters[$parameter->name()] = $parameter;
Expand Down
Loading