Skip to content

allow me to specify host address for ollama and maybe other vendors too #22

@keneo

Description

@keneo

I host my ollama on a separate host with strong GPU. I connect to it like this:

create_chat_ollama_custom<-function (host="127.0.0.1", port=11434) 
{
  engine <- httr2::request(base_url = glue::glue("http://{host}:{port}/api/chat"))
  chat <- list(vendor_name = "ollama", engine = engine, messages = list(), 
               params = list(stream = FALSE))
  class(chat) <- "chat"
  return(chat)
}

However I think it would be usefull also for others if you allow host parameter in your create_chat, so that we could use it like this:

create_chat('ollama', host='...my.gpu.host...')

I would implement like this: add host parameter with default value NA, and for each vendor, use vendor default address if host is NA or use specified host if provided. I.e. with dplyr::coalesce

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions