How to make it work with Azure Databricks - MLFlow

[jannavarro]

In Azure Databricks, MLFlow is supported and can be accessed via API. The MS docs says:

You can invoke the MLflow REST API using URLs of the form https://<databricks-instance>/api/2.0/mlflow/<api-endpoint>, replacing <databricks-instance> with the <region>.azuredatabricks.net domain name of your Azure Databricks deployment.

So in my appsettings.json I have:

{
  "MLFlowConfiguration": {
    "MLFlowServerBaseUrl": "https://southeastasia.azuredatabricks.net",
    "APIBase": "api/2.0/mlflow/"
  }
}

But trying that with the lines of code below returns an exception:

var experimentName = Guid.NewGuid().ToString();
var experimentRequest = await _mlFlowService.GetOrCreateExperiment(experimentName);

Error:
'Object reference not set to an instance of an object.'

   at MLFlow.NET.Lib.Services.MLFlowService.<GetOrCreateExperiment>d__4.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at MLNETMLFlowSampleConsole.Program.<RunExperiment>d__3.MoveNext() in C:\Samples\2019\MLNETMLFlowSample\MLNETMLFlowSampleConsole\Program.cs:line 55
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at MLNETMLFlowSampleConsole.Program.<Main>d__2.MoveNext() in C:\Samples\2019\MLNETMLFlowSample\MLNETMLFlowSampleConsole\Program.cs:line 34

[xentropic-dev]

I'm also interested in using this to monitor a remote MLFlow server. I'm using Azure ML Studio, so I have an azure:// based uri provided for my workspace. Can one of the maintainers of this repo chirp in here to let us know if it's possible? I don't mind chipping in a little of my time to make it happen if it is currently not possible.

[jakkaj]

Hi @xentropy-git I was one of the original developers on this project. Unfortunately we've not looked at this project in quite some time and we really should be marking this repo as read only. I don't have current knowledge of the product to say yes or no directly to your answer sorry!