Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions contributing/BACKENDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ base class. See its docstrings for descriptions of the methods that your class s

Refer to examples:
- Offline providers:
[datacrunch.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/datacrunch.py),
[verda.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/verda.py),
[aws.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/aws.py),
[azure.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/azure.py),
[lambdalabs.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/lambdalabs.py).
Expand All @@ -64,7 +64,7 @@ Add your provider in the following places:
For offline providers, you can add data quality tests under `src/integrity_tests/`.
Data quality tests are run after collecting offline catalogs to ensure their integrity.

Refer to examples: [test_datacrunch.py](https://github.com/dstackai/gpuhunt/blob/main/src/integrity_tests/test_datacrunch.py),
Refer to examples: [test_verda.py](https://github.com/dstackai/gpuhunt/blob/main/src/integrity_tests/test_verda.py),
[test_gcp.py](https://github.com/dstackai/gpuhunt/blob/main/src/integrity_tests/test_gcp.py).

### 1.6. Submit a pull request
Expand Down Expand Up @@ -125,7 +125,7 @@ Then add these models to `AnyBackendConfig*` unions in [`src/dstack/_internal/co
The script also generates `*BackendStoredConfig` that extends `*BackendConfig` to be able to store extra parameters in the DB. By the same logic, it generates `*Config` that extends `*BackendStoredConfig` with creds and uses it as the main `Backend` and `Compute` config instead of using `*BackendConfigWithCreds` directly.

Refer to examples:
[datacrunch](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/datacrunch/models.py),
[verda](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/verda/models.py),
[aws](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/aws/models.py),
[gcp](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/gcp/models.py),
[azure](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/models.py), etc.
Expand All @@ -136,7 +136,7 @@ Go to `compute.py` and implement `Compute` methods.
Optionally, extend and implement `ComputeWith*` classes to support additional features such as fleets, volumes, gateways, placement groups, etc. For example, extend `ComputeWithCreateInstanceSupport` to support fleets.

Refer to examples:
[datacrunch](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/datacrunch/compute.py),
[verda](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/verda/compute.py),
[aws](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/aws/compute.py),
[gcp](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/gcp/compute.py),
[azure](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/azure/compute.py), etc.
Expand All @@ -146,7 +146,7 @@ Refer to examples:
Go to `configurator.py` and implement custom `Configurator` logic. At minimum, you should implement creds validation.
You may also need to validate other config parameters if there are any.

Refer to examples: [datacrunch](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/datacrunch/configurator.py),
Refer to examples: [verda](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/verda/configurator.py),
[aws](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/aws/configurator.py),
[gcp](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/gcp/configurator.py),
[azure](https://github.com/dstackai/dstack/blob/master/src/dstack/_internal/core/backends/azure/configurator.py), etc.
Expand Down Expand Up @@ -195,7 +195,7 @@ For some VM-based backends, the `dstack` team also maintains
[custom VM images](../scripts/packer/README.md) with the required dependencies
and `dstack`-specific optimizations.

Examples of VM-based backends include: `aws`, `azure`, `gcp`, `lambda`, `datacrunch`, `tensordock`, etc.
Examples of VM-based backends include: `aws`, `azure`, `gcp`, `lambda`, `verda`, etc.

#### 3.1.2. Container-based backend compute type

Expand Down
2 changes: 1 addition & 1 deletion contributing/GPUHUNT.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Some providers offer extreme flexibility in possible configurations, but not all
- Filters out if: outdated family, not supported family
- Queries configuration details to fill CPU, RAM, and GPU information

### DataCrunch
### Verda

- Just queries all offers via API

Expand Down
13 changes: 13 additions & 0 deletions docs/assets/images/verda-logo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 4 additions & 2 deletions docs/docs/concepts/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -854,9 +854,11 @@ There are two ways to configure OCI: using client credentials or using the defau
compartment_id: ocid1.compartment.oc1..aaaaaaaa
```

### DataCrunch
<span id="datacrunch"></span>

Log into your [DataCrunch](https://cloud.datacrunch.io/) account, click Keys in the sidebar, find `REST API Credentials` area and then click the `Generate Credentials` button.
### Verda (formerly DataCrunch) { #verda }

Log into your [Verda](https://console.verda.com/signin) account, click Keys in the sidebar, find `REST API Credentials` area and then click the `Generate Credentials` button.

Then, go ahead and configure the backend:

Expand Down
10 changes: 5 additions & 5 deletions docs/docs/reference/server/config.yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,18 +258,18 @@ to configure [backends](../../concepts/backends.md) and other [server-level sett
type:
required: true

##### `projects[n].backends[type=datacrunch]` { #datacrunch data-toc-label="datacrunch" }
##### `projects[n].backends[type=verda]` { #verda data-toc-label="verda" }

#SCHEMA# dstack._internal.core.backends.datacrunch.models.DataCrunchBackendConfigWithCreds
#SCHEMA# dstack._internal.core.backends.verda.models.VerdaBackendConfigWithCreds
overrides:
show_root_heading: false
type:
required: true
item_id_prefix: datacrunch-
item_id_prefix: verda-

###### `projects[n].backends[type=datacrunch].creds` { #datacrunch-creds data-toc-label="creds" }
###### `projects[n].backends[type=verda].creds` { #verda-creds data-toc-label="creds" }

#SCHEMA# dstack._internal.core.backends.datacrunch.models.DataCrunchAPIKeyCreds
#SCHEMA# dstack._internal.core.backends.verda.models.VerdaAPIKeyCreds
overrides:
show_root_heading: false
type:
Expand Down
4 changes: 2 additions & 2 deletions docs/partners.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,9 @@ hide:
</span> -->

<span class="feature-cell" target="_blank">
<img width="52" style="margin: 10px -4px;" src="/assets/images/datacrunch-logo.svg">
<img width="42" src="/assets/images/verda-logo.svg">
<h3>
DataCrunch
Verda
</h3>
</span>
</div>
Expand Down
10 changes: 7 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ dependencies = [
"python-multipart>=0.0.16",
"filelock",
"psutil",
"gpuhunt==0.1.14",
"gpuhunt==0.1.15",
"argcomplete>=3.5.0",
"ignore-python>=0.2.0",
"orjson",
Expand Down Expand Up @@ -186,7 +186,11 @@ gcp = [
"dstack[server]",
]
datacrunch = [
"datacrunch",
"verda; python_version >= '3.10'",
"dstack[server]",
]
verda = [
"verda; python_version >= '3.10'",
"dstack[server]",
]
kubernetes = [
Expand All @@ -211,5 +215,5 @@ nebius = [
"dstack[server]",
]
all = [
"dstack[gateway,server,aws,azure,gcp,datacrunch,kubernetes,lambda,nebius,oci]",
"dstack[gateway,server,aws,azure,gcp,verda,kubernetes,lambda,nebius,oci]",
]
2 changes: 2 additions & 0 deletions src/dstack/_internal/core/backends/base/offers.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ def get_catalog_offers(
catalog: Optional[gpuhunt.Catalog] = None,
) -> List[InstanceOffer]:
provider = backend.value
if backend == BackendType.DATACRUNCH:
provider = BackendType.VERDA.value # Backward compatibility
if backend == BackendType.LAMBDA:
provider = "lambdalabs"
if backend == BackendType.AMDDEVCLOUD:
Expand Down
9 changes: 9 additions & 0 deletions src/dstack/_internal/core/backends/configurators.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,15 @@
except ImportError:
pass

try:
from dstack._internal.core.backends.verda.configurator import (
VerdaConfigurator,
)

_CONFIGURATOR_CLASSES.append(VerdaConfigurator)
except ImportError:
pass

try:
from dstack._internal.core.backends.vultr.configurator import VultrConfigurator

Expand Down
1 change: 1 addition & 0 deletions src/dstack/_internal/core/backends/datacrunch/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# DataCrunch backend for backward compatibility
12 changes: 7 additions & 5 deletions src/dstack/_internal/core/backends/datacrunch/backend.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
from dstack._internal.core.backends.base.backend import Backend
from dstack._internal.core.backends.datacrunch.compute import DataCrunchCompute
from dstack._internal.core.backends.datacrunch.models import DataCrunchConfig
from dstack._internal.core.backends.verda.backend import VerdaBackend
from dstack._internal.core.backends.verda.models import VerdaConfig
from dstack._internal.core.models.backends.base import BackendType


class DataCrunchBackend(Backend):
# Deprecated
# TODO: Remove in 0.21
class DataCrunchBackend(VerdaBackend):
TYPE = BackendType.DATACRUNCH
COMPUTE_CLASS = DataCrunchCompute

def __init__(self, config: DataCrunchConfig):
def __init__(self, config: VerdaConfig):
self.config = config
self._compute = DataCrunchCompute(self.config)
self._compute = DataCrunchCompute(self.config, self.TYPE)

def compute(self) -> DataCrunchCompute:
return self._compute
Loading