Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
name: Documentation
on:
push:
branches: [main]
permissions:
contents: write
jobs:
deploy:
uses: innmind/github-workflows/.github/workflows/documentation.yml@main
secrets: inherit
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
/composer.lock
/vendor
/.cache
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# This command is intended to be run on your computer
serve-doc:
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material

build-doc:
docker run --rm -it -v ${PWD}:/docs squidfunk/mkdocs-material build
121 changes: 14 additions & 107 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Abstraction on top of `Fiber`s to coordinate multiple tasks asynchronously.

The goal is to easily move the execution of any code built using [`innmind/operating-system`](https://packagist.org/packages/innmind/operating-system) from a synchronous context to an async one. This means that it's easier to experiment running a piece of code asynchronously and then move back if the experiment is not successful. This also means that you can test each part of an asynchronous system synchronously.
The goal is to easily move the execution of any code built using [`innmind/operating-system`](https://innmind.org/OperatingSystem/) from a synchronous context to an async one. This means that it's easier to experiment running a piece of code asynchronously and then move back if the experiment is not successful. This also means that you can test each part of an asynchronous system synchronously.

## Installation

Expand All @@ -22,116 +22,23 @@ use Innmind\Async\{
Scope\Continuation,
};
use Innmind\OperatingSystem\{
Factory,
OperatingSystem,
};
use Innmind\Filesystem\Name;
use Innmind\HttpTransport\Success;
use Innmind\Http\{
Request,
Method,
ProtocolVersion,
};
use Innmind\Url\{
Url,
Path,
Factory,
};
use Innmind\Immutable\Sequence;

[$users] = Scheduler::of(Factory::build());
->sink([0, 0, false])
Scheduler::of(Factory::build())
->sink(null)
->with(
static function(array $carry, OperatingSystem $os, Continuation $continuation, Sequence $results): Continuation {
[$users, $finished, $launched] = $carry;

if (!$launched) {
return $continuation
->carryWith([$users, $finished, true])
->schedule(Sequence::of(
static fn(OperatingSystem $os): int => $os
->remote()
->http()(Request::of(
Url::of('http://some-service.tld/users/count'),
Method::get,
ProtocolVersion::v11,
))
->map(static fn(Success $success): string => $success->response()->body()->toString())
->match(
static fn(string $response): int => (int) $response,
static fn() => throw new \RuntimeException('Failed to count the users'),
),
static fn(OperatingSystem $os): int => $os
->filesystem()
->mount(Path::of('some/directory/'))
->unwrap()
->get(Name::of('users.csv'))
->map(static fn($file) => $file->content()->lines())
->match(
static fn(Sequence $lines) => $lines->reduce(
0,
static fn(int $total): int => $total + 1,
),
static fn() => throw new \RuntimeException('Users file not found'),
),
));
}

$finished += $results->size();
$users = $results->reduce(
$users,
static fn(int $total, int $result): int => $total + $result,
);
$continuation = $continuation->carryWith([$users, $finished, $launched]);

if ($finished === 2) {
$continuation = $continuation->finish();
}

return $continuation->wakeOnResult();
},
static fn(
$_,
OperatingSystem $os,
Continuation $continuation,
) => $continuation
->schedule(Sequence::of(
static fn(OperatingSystem $os) => importUsers($os),
static fn(OperatingSystem $os) => importProducts($os),
))
->finish(),
);
```

This example counts a number of `$users` coming from 2 sources.

The `Scheduler` object behaves as a _reduce_ operation, that's why it has 2 arguments: a carried value and a reducer (called a scope in this package).

The carried value here is an array that holds the number of fetched users, the number of finished tasks and whether it already launched the tasks or not.

The scope will launch 2 tasks if not already done; the first one does an HTTP call and the second one counts the number of lines in a file. The scope will be called again once a task finishes and their results will be available inside the fourth argument `$results`, it will add the number of finished tasks and the number of users to the carried value array. If both tasks are finished then the scope calls `$continuation->terminate()` to instruct the loop to stop.

When the scope calls `->terminate()` and that all tasks are finished then `->with()` returns the carried value. Here it will assign the aggregation of both tasks results to the value `$users`.

> [!NOTE]
> As long as you use the `$os` abstraction passed as arguments the system will automatically suspend your code when necessary. This means that you don't even need to think about it.

> [!NOTE]
> The scope `callable` is also run asynchronously. This means that you can use it to build a socket server and wait indefinitely for new connections without impacting the execution of already started tasks.

> [!WARNING]
> Do NOT return the `$os` variable outside of the tasks or the scope as it may break your code.

> [!NOTE]
> Since this package has been designed by only passing arguments (no global state) it means that you can compose the use of `Scheduler`, this means that you can run a new instance of `Scheduler` inside a task and it will behave transparently. (Although this feature as not been tested yet!)

## Limitations

### HTTP calls

Currently HTTP calls are done via `curl` but it can't be integrated in the same loop as other streams. To allow the coordination of multiple tasks when doing HTTP calls the system use a timeout of `10ms` and switches between tasks at this max rate.

To fix this limitation a new implementation entirely based on PHP streams needs to be created.

Meanwhile if your goal is to make multiple concurrent HTTP calls you don't need this package. [`innmind/http-transport`](https://innmind.org/documentation/getting-started/concurrency/http/) already support concurrent calls on it's own (without the limitation mentionned above).

### SQL queries

SQL queries executed via `$os->remote()->sql()` are still executed synchronously.

To fix this limitation a new implementation entirely based on PHP streams needs to be created.

### Number of tasks

It seems that the current implementation of this package has a [limit of around 100K concurrent tasks](https://bsky.app/profile/baptouuuu.bsky.social/post/3lwr7pei2ek2f) before it starts slowing down drastically.

A simple script scheduling 100k tasks that each halts the process for 10 second will take ~13s.
Binary file added docs/assets/favicon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/fonts/MonaspaceNeon-Regular.woff
Binary file not shown.
24 changes: 24 additions & 0 deletions docs/assets/logo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
113 changes: 113 additions & 0 deletions docs/assets/stylesheets/extra.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
@font-face {
font-family: "Monaspace Neon";
font-weight: normal;
font-style: normal;
src: url("../fonts/MonaspaceNeon-Regular.woff");
}

:root {
--md-code-font: "Monaspace Neon";
}

:root {
--light-md-code-hl-number-color: #f76d47;
--light-md-code-hl-function-color: #6384b9;
--light-md-code-hl-operator-color: #39adb5;
--light-md-code-hl-constant-color: #7c4dff;
--light-md-code-hl-string-color: #9fc06f;
--light-md-code-hl-punctuation-color: #39adb5;
--light-md-code-hl-keyword-color: #7c4dff;
--light-md-code-hl-variable-color: #80cbc4;
--light-md-code-hl-comment-color: #ccd7da;
--light-md-code-bg-color: #fafafa;
--light-md-code-fg-color: #ffb62c;
--light-md-code-hl-variable-color: #6384b9;
--dark-md-code-hl-number-color: #f78c6c;
--dark-md-code-hl-function-color: #82aaff;
--dark-md-code-hl-operator-color: #89ddff;
--dark-md-code-hl-constant-color: #c792ea;
--dark-md-code-hl-string-color: #c3e88d;
--dark-md-code-hl-punctuation-color: #89ddff;
--dark-md-code-hl-keyword-color: #c792ea;
--dark-md-code-hl-variable-color: #e8f9f9;
--dark-md-code-hl-comment-color: #546e7a;
--dark-md-code-bg-color: #263238;
--dark-md-code-fg-color: #ffcb6b;
--dark-md-code-hl-variable-color: #82aaff;
}

@media (prefers-color-scheme: light) {
.language-php > * {
--md-code-hl-number-color: var(--light-md-code-hl-number-color);
--md-code-hl-function-color: var(--light-md-code-hl-function-color);
--md-code-hl-operator-color: var(--light-md-code-hl-operator-color);
--md-code-hl-constant-color: var(--light-md-code-hl-constant-color);
--md-code-hl-string-color: var(--light-md-code-hl-string-color);
--md-code-hl-punctuation-color: var(--light-md-code-hl-punctuation-color);
--md-code-hl-keyword-color: var(--light-md-code-hl-keyword-color);
--md-code-hl-variable-color: var(--light-md-code-hl-variable-color);
--md-code-hl-comment-color: var(--light-md-code-hl-comment-color);
--md-code-bg-color: var(--light-md-code-bg-color);
--md-code-fg-color: var(--light-md-code-fg-color);
}

.language-php .na {
--md-code-hl-variable-color: var(--light-md-code-hl-variable-color);
}
}

[data-md-color-media="(prefers-color-scheme: light)"] .language-php > * {
--md-code-hl-number-color: var(--light-md-code-hl-number-color);
--md-code-hl-function-color: var(--light-md-code-hl-function-color);
--md-code-hl-operator-color: var(--light-md-code-hl-operator-color);
--md-code-hl-constant-color: var(--light-md-code-hl-constant-color);
--md-code-hl-string-color: var(--light-md-code-hl-string-color);
--md-code-hl-punctuation-color: var(--light-md-code-hl-punctuation-color);
--md-code-hl-keyword-color: var(--light-md-code-hl-keyword-color);
--md-code-hl-variable-color: var(--light-md-code-hl-variable-color);
--md-code-hl-comment-color: var(--light-md-code-hl-comment-color);
--md-code-bg-color: var(--light-md-code-bg-color);
--md-code-fg-color: var(--light-md-code-fg-color);
}

[data-md-color-media="(prefers-color-scheme: light)"] .language-php .na {
--md-code-hl-variable-color: var(--light-md-code-hl-variable-color);
}

@media (prefers-color-scheme: dark) {
.language-php > * {
--md-code-hl-number-color: var(--dark-md-code-hl-number-color);
--md-code-hl-function-color: var(--dark-md-code-hl-function-color);
--md-code-hl-operator-color: var(--dark-md-code-hl-operator-color);
--md-code-hl-constant-color: var(--dark-md-code-hl-constant-color);
--md-code-hl-string-color: var(--dark-md-code-hl-string-color);
--md-code-hl-punctuation-color: var(--dark-md-code-hl-punctuation-color);
--md-code-hl-keyword-color: var(--dark-md-code-hl-keyword-color);
--md-code-hl-variable-color: var(--dark-md-code-hl-variable-color);
--md-code-hl-comment-color: var(--dark-md-code-hl-comment-color);
--md-code-bg-color: var(--dark-md-code-bg-color);
--md-code-fg-color: var(--dark-md-code-fg-color);
}

.language-php .na {
--md-code-hl-variable-color: var(--dark-md-code-hl-variable-color);
}
}

[data-md-color-media="(prefers-color-scheme: dark)"] .language-php > * {
--md-code-hl-number-color: var(--dark-md-code-hl-number-color);
--md-code-hl-function-color: var(--dark-md-code-hl-function-color);
--md-code-hl-operator-color: var(--dark-md-code-hl-operator-color);
--md-code-hl-constant-color: var(--dark-md-code-hl-constant-color);
--md-code-hl-string-color: var(--dark-md-code-hl-string-color);
--md-code-hl-punctuation-color: var(--dark-md-code-hl-punctuation-color);
--md-code-hl-keyword-color: var(--dark-md-code-hl-keyword-color);
--md-code-hl-variable-color: var(--dark-md-code-hl-variable-color);
--md-code-hl-comment-color: var(--dark-md-code-hl-comment-color);
--md-code-bg-color: var(--dark-md-code-bg-color);
--md-code-fg-color: var(--dark-md-code-fg-color);
}

[data-md-color-media="(prefers-color-scheme: dark)"] .language-php .na {
--md-code-hl-variable-color: var(--dark-md-code-hl-variable-color);
}
45 changes: 45 additions & 0 deletions docs/getting-started/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Getting started

## Installation

```sh
composer require innmind/async
```

## Setup

```php title="async.php"
<?php
declare(strict_types = 1);

require 'path/to/vendor/autoload.php';

use Innmind\Async\{
Scheduler,
Scope\Continuation,
};
use Innmind\OperatingSystem\{
Factory,
OperatingSystem,
};

$os = Factory::build();
Scheduler::of($os)
->sink(null) #(1)
->with(function(
$_, #(2)
OperatingSystem $os,
Continuation $continuation,
): Continuation {
// this is the scope that will schedule tasks

return $continuation;
});
```

1. You'll learn in a [later chapter](../scopes/lifecycle.md#carry-a-value) what this value is. For now leave it like this.
2. You'll learn in a [later chapter](../scopes/lifecycle.md#carry-a-value) what this value is. For now leave it like this.

You can run this script via `php async.php` in your terminal. For now it executes an infinite loop that does nothing.

You'll see in the next chapter what you can inside the loop.
Loading
Loading