[data](deps): Bump dask[complete] from 2022.10.1 to 2023.5.0 in /python/requirements/data_processing#201
Open
dependabot[bot] wants to merge 424 commits intomasterfrom
Conversation
Author
|
Dependabot tried to add |
dependabot
bot
added
the
dependencies
replaced the Ray core collab example with Ray AIR Signed-off-by: Simran Mhatre <simran@anyscale.com>
Some preparation for upcoming deploy refactor PRs. - Rename fields of `ApplicationState` as private fields - Add route prefix to deployment info (it will be used by application state to deploy deployments in a reconciler loop) - Stop setting num replicas from autoscaling in `deployment_config.num_replicas`, instead put it in a separate field of deployment info. - determine initial autoscaled num replicas in deployment state manager .deploy()
…ray-project#34813) [core] update ActorPool map/map_unordered to submit functions eagerly Signed-off-by: Matthew Deng <matt@anyscale.com>
…ject#34467)" (ray-project#34914) This reverts commit af018f6.
…-project#34903) Signed-off-by: Justin Yu <justinvyu@anyscale.com>
And recompile requirements. Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
…gce run with debug wheels (ray-project#34897) Closes ray-project#34591 The current stacktrace when sigabort doesn't yield any useful
…ct#34147)" (ray-project#34933) This reverts commit aeed2b3.
Preview: https://anyscale-ray--34567.com.readthedocs.build/en/34567/data/batch_inference.html Signed-off-by: Max Pumperla <max.pumperla@googlemail.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
…-project#34840) We've introduced a new execution path for Ray Tune behind a feature flag. The unit tests have been stable for some time. To get better dogfooding results, we should enable it per default, so that all unit and release tests are run using the new engine. If we run into problems, we can easily revert this PR before we cut the 2.5 release branch. Signed-off-by: Kai Fricke <kai@anyscale.com>
…4942) Remove some legacy code related to multiple core workers in one process since we now no longer support it. Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
…ct#34926) Signed-off-by: Cuong Nguyen <can@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com>
…prove performance. (ray-project#34877)
…vars (ray-project#34856) Add usage of expansion of certain PATH variables in env vars to documentation Signed-off-by: LarryLian <554538252@qq.com>
…t#34788) This PR makes a number of improvements to tackle issues uncovered in dogfooding: 1. Instead of just a print, we render a table for configs and results at the start of training (closes ray-project#34784) 2. We round float results to 5 significant numbers after the decimal point (closes ray-project#34785) 3. We track the last printed result and only print the result at the end of training if it hasn't been printed before (closes ray-project#34786) 4. We divide the results by "automatic" results and trainer-specific results (closes ray-project#34787) Signed-off-by: Kai Fricke <kai@anyscale.com>
The team flag is wrong for these tests. Update it to data.
ray-project#34804 broke python/ray/tests:test_autoscaler_aws because that unit-test does not allow to specify a key-value pair in the aws provider. Release tests need to specify that key though. Fix release tests to use the same file to we don't have to specify separated keys. Signed-off-by: Cuong Nguyen <can@anyscale.com>
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Signed-off-by: Max Pumperla <max.pumperla@googlemail.com> Signed-off-by: amogkam <amogkamsetty@yahoo.com> Co-authored-by: amogkam <amogkamsetty@yahoo.com>
This PR gives a detailed doc and a step-by-step guide on how to configure the TLS Auth for the Ray cluster. Related issue number Closes ray-project/kuberay#889
Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
So that failure will not be missed. Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
And also apply buildifer formatting to the `release/BUILD` file.
…ect types) (ray-project#35359) There were a couple bugs in our handling of complex ndarrays: We weren't consistently falling back to PandasBlock for object dtypes. This was due to raising different exception types, some of which were not caught at the upper layer. This PR simplifies our exception handling path, removing legacy code. We weren't calling create_ragged_ndarray for certain return types due to a bug in the shape mismatch detection code.
Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>
This opens as default in the Grafana page but for the Dashboard UI, the default is still scoped to the latest session.
Add task detail page with logs Update actor logs to use state-api actor_id filter instead of filename to fetch logs Hide log tabs if there is only 1 tab available. Update the driver logs in the jobs page to show explanation why logs are not available.
We changed the logging content with per task magic token. Updating test logic.
…e API (ray-project#35124) This PR adds better support for listing jobs (ray list jobs) and getting submission job logs (ray logs job --id) The state API client mirrors the implementation of job endpoint and depends on job's implementations for retrieving job related info.
Signed-off-by: Matthew Deng <matt@anyscale.com>
…35376) When master is not merged, a file not found error can come up. This update to the script will prevent that from happening by catching errors and setting default values for the variables. Signed-off-by: Kai Fricke <kai@anyscale.com>
…_name> (ray-project#35371) We updated the dashboard to show actor's name with actor repr, and we should do the same for actor tasks. Caveat: Actor.__init__ is currently not showing up as repr_name.__init__ because during creation task initialization, we haven't initialized the actor states yet, so the repr func should not yet be called. There's workaround (i.e we could modify the init task name later at rendering, but chose not implement in this PR), but would be rather hacky. The major issue is that repr info for an actor is only available on the executor, and on the submitter (or encoded in actor handle). Given the feature is only for dashboard, I feel this the intrusive change in this PR is better.
…ct#34833) For full test coverage, we should migrate all tests in `test_ray_trial_executor` and `test_trial_runner_*` to use the new execution backend as well. This is a WIP PR to migrate the first batch of these unittests. Signed-off-by: Kai Fricke <kai@anyscale.com>
…ure docs, prepare default on (ray-project#35129) This prepare to enable the new output engine per default. When activated, a hint is displayed how to disable the new output engine. This PR also adds documentation around experimental features in Ray AIR. Signed-off-by: Kai Fricke <kai@anyscale.com> Signed-off-by: Kai Fricke <krfricke@users.noreply.github.com> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
…project#35355) All release tests are using remote execution via anyscale at this point. Remove code path for local environment setup. In particular, the driver_setup is used to install packages on buildkite host, which is no longer neccessary/used. Signed-off-by: Cuong Nguyen <can@anyscale.com>
This PR adds telemetry for the storage / syncing configuration by adding 1 usage tag: `AIR_STORAGE_CONFIGURATION`
The storage configuration is set by `RunConfig(storage_path, sync_config)`. The possible configurations are:
- 'driver' = Default head node syncing if no remote path is specified
- 'local' = No synchronization at all.
- 'nfs' = Using a mounted shared network filesystem.
- ('s3', 'gs', 'hdfs', 'custom_remote_storage'): Various remote storage schemes.
- ('local_uri', 'memory'): Mostly used by internal testing by setting `storage_path`
to `file://` or `memory://`.
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Revert some debug logs added in ray-project#35062 Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
Be extra careful with fetching task references in the controller update loop. - The `deploy_obj_ref` task reference should only be fetched once (for both application state and http state). - Also, `RayTaskError` and `RuntimeEnvSetupError` may not be the only two types of possible exceptions thrown by `ray.get()` (e.g. saw a case of `RaySystemError`) so we should catch all exceptions.
…oject#34904) This PR adds telemetry for built-in experiment tracking integrations by adding 3 usage tags: 1. `AIR_SETUP_WANDB_INTEGRATION_USED` ("1" if used) 2. `AIR_SETUP_MLFLOW_INTEGRATION_USED` ("1" if used) 3. `AIR_CALLBACKS` (a JSON string representing a dict of callback name -> count) - The key `CustomCallback` gets a tally if the user passed in just a subclass of `Callback` - The key `CustomLoggerCallback` gets a tally if the user passed in a custom `LoggerCallback` (not including above) The need for 1 and 2 is because wandb and mlflow allow the `setup_x` path, where the user calls this in their training function and logs whatever they want themselves. These 3 can be used together to extract the total wandb/mlflow integration usage. (Ex: `setup_wandb` usage + `WandbLoggerCallback` usage. There may be some overlap, as it's technically possible to use both.) Signed-off-by: Justin Yu <justinvyu@anyscale.com>
) Previously, we have been following the xgboost/lightgbm conventions of fitting for num_boost_round, regardless of how many iterations the model already has been fitted on. This, however, causes issues when resuming from checkpoints, especially during training/tuning, as you may end up with more trees than desired: Trial has num_boost_round=100 Trial fits for 50 and dies Trial is restored from checkpoint, model starts with 50 iterations already complete Because num_boost_round=100, model is fitted for 100 iterations, giving a total of 150 iterations instead of desired 100 Now, we will subtract the already completed iterations when resuming. num_boost_round was already a part of **train_kwargs, we just promote it here for docstring purposes. --------- Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
) Docs follow-up for: ray-project#34584
Signed-off-by: Kai Fricke <kai@anyscale.com>
…oject#35403) This reverts commit 496024d.
- Multiplexed metrics. - Pass multiplexed information into controller and make it available at `RunningReplicaInfo`.
This is part of effort to make state API no longer experimental: Move everything under ray/experimental/state into ray/util/state Declare state API's python SDK to be DeveloperAPI, CLIs commands to be Stable. Make all imports from ray.experimental.state.api to ray.util.state: from ray.util.state import list_tasks # works ... Forward importing from ray.experimental.state to ray.utils.state such that existing users will work. from ray.experimental.state.api import list_tasks # old way works, with warning Add warning and telemetry for the deprecating import path (ray.experimental.state)
… in RLlib release tests (ray-project#35354) Signed-off-by: Artur Niederfahrenhorst <attaismyname@googlemail.com>
Bumps [dask[complete]](https://github.com/dask/dask) from 2022.10.1 to 2023.5.0. - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](dask/dask@2022.10.1...2023.5.0) --- updated-dependencies: - dependency-name: dask[complete] dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
dependabot
bot
force-pushed
the
dependabot/pip/python/requirements/data_processing/dask-complete--2023.5.0
branch
from
e074d09 to
29204fd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps dask[complete] from 2022.10.1 to 2023.5.0.
Commits
12c7c10bump version to 2023.5.0193ae7bFixclone+from_arrayfailure (#10211)e9845aaImplementnumeric_only=FalseforGroupBy.corrandGroupBy.cov(#1...980fd92Clean up futures docs (#10266)1dcc856Add support fornumeric_only=FalseinDataFrame.var(#10250)a6ed1b2Warn when meta is passed toapply(#10256)5064339Fix dataframe reductions for ea dtypes (#10150)3f6ff72Addnumeric_onlysupport toDataFrame.mode(#10257)62f4f1fAddIndexAPI reference (#10263)2a2273dRemoveimageioversion restriction in CI (#10260)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)