Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm

[ishaan-jaff]

This PR adds support for models from all the above mentioned providers using litellm https://github.com/BerriAI/litellm

TLDR: developer gets:

• more models out of the box, everything litellm integrates with: https://litellm.readthedocs.io/en/latest/supported/
• instant integration of new models by modifying the model param (Easy to add models in the future, just change the model param)

Here's a sample of how it's used:

from litellm import completion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion("command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

[ishaan-jaff]

@swyxio @jakubno can you please take a look at this PR when you get the chance ? Happy to add more docs/tests if this initial PR looks good😊

[krrishdholakia]

any updates on this? @swyxio @jakubno

[iamjameswalters]

Very interested in seeing this merged, would be a partial answer to #90

[krrishdholakia]

hey @iamjameswalters any specific providers /models you're trying to use? want to make sure we have it supported - https://docs.litellm.ai/docs/providers

[iamjameswalters]

hey @iamjameswalters any specific providers /models you're trying to use? want to make sure we have it supported - https://docs.litellm.ai/docs/providers

Specifically I'm looking to use Starcoder, which it looks like I can do with litellm through the HuggingFace API. I'm taking this PR for a spin right now to see how it works. 🙂️

[ishaan-jaff]

Hi @swyxio just wanted to follow up on this PR

What was litellm missing to be useful to you? Any feedback here would be helpful