mlflow Community-edition

Source: vignettes/mlflow-community-edition.Rmd

This article describes how to communicate with mlflow hosted on Databricks Community Edition website.

Setup mlflow experiment on Databricks Community Edition

  1. Sign-in or register Databricks Community Edition

  2. In the left side-bar, create an experiment by clicking on Workspace -> Create -> MLflow Experiment.

Setup mlflow in R

  1. Install mlflow 
remotes::install_cran("mlflow")
library(mlflow)
mlflow::install_mlflow(python_version = "3.9")
  1. Define the following environment variables in .Renviron:
MLFLOW_TRACKING_URI=databricks
MLFLOW_EXPERIMENT_ID=3504022373425916
DATABRICKS_USERNAME=bilbo.baggins@gmail.com
DATABRICKS_PASSWORD=9nPS5ocV69n2@
DATABRICKS_HOST=https://community.cloud.databricks.com

Modify the following three variables to fit your Databricks Community Edition details:

  • DATABRICKS_USERNAME: Your login credentials;
  • DATABRICKS_PASSWORD: Your password; and
  • MLFLOW_EXPERIMENT_ID: The experiment id (16 digits at the end of the workspace URL).

Tip: If you don’t have the .Renviron file, you can create one with usethis::edit_r_environ('project').

  1. Load .Renviron by calling readRenviron(usethis::proj_path('.Renviron')).

Tip: If you are using a package file structure for your project, then you could read ‘.Renviron’ every time the package is loaded, i.e. pkgload::load_all() by calling readRenviron in .onAttach. See code snippet.

# ./R/zzz.R
.onAttach <- function(...) {
    file <- system.file(".Renviron", package = pkgload::pkg_name())
    suppressWarnings(try(readRenviron(file), silent = TRUE))
}