diff --git a/docs-website/docs/overview/get-started.mdx b/docs-website/docs/overview/get-started.mdx index 2dc3be4d43..ea4c0637a8 100644 --- a/docs-website/docs/overview/get-started.mdx +++ b/docs-website/docs/overview/get-started.mdx @@ -2,7 +2,7 @@ title: "Get Started" id: get-started slug: "/get-started" -description: "Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources." +description: "Learn how to quickly get up and running with Haystack. Build your first RAG pipeline and tool-calling Agent with step-by-step examples for multiple LLM providers." --- import Tabs from '@theme/Tabs'; @@ -10,7 +10,7 @@ import TabItem from '@theme/TabItem'; # Get Started -Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources. +Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing Haystack, building your first RAG pipeline, and creating a tool-calling Agent. ## Build your first RAG application @@ -22,48 +22,81 @@ First, install the minimal form of Haystack: pip install haystack-ai ``` -
+In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). Choose your preferred LLM provider from the tabs below. For easier use, you can also set the API key as an environment variable. -Were you already using Haystack 1.x? + + -:::warning +[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package. -Installing `farm-haystack` and `haystack-ai` in the same Python environment (virtualenv, Colab, or system) causes problems. +```python +from haystack import Pipeline, Document +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.utils import Secret +from haystack.dataclasses import ChatMessage -Installing both packages in the same environment can somehow work or fail in obscure ways. We suggest installing only one of these packages per Python environment. Make sure that you remove both packages if they are installed in the same environment, followed by installing only one of them: +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) -```bash -pip uninstall -y farm-haystack haystack-ai -pip install haystack-ai -``` +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ + ), + ChatMessage.from_user("{{question}}") +] -If you have any questions, please reach out to us on the [GitHub Discussion](https://github.com/deepset-ai/haystack/discussions) or [Discord](https://discord.com/invite/VBpFzsgRVF). -::: +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = OpenAIChatGenerator( + api_key=Secret.from_env_var("OPENAI_API_KEY"), + model="gpt-4o-mini" +) + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") -
+question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) -In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). You can choose between OpenAI or Hugging Face as your LLM provider. For easier use, you can also set the API key as an environment variable (`OPENAI_API_KEY` or `HF_API_TOKEN`). +print(results["llm"]["replies"]) +``` - - + + -:::note -**Using OpenAIChatGenerator requires an OpenAI API key with sufficient quota.** -New users on the free tier may immediately encounter a `429` ("insufficient_quota") error when running -this example. If you do not have enough OpenAI credits, try the Hugging Face tab instead. -::: +[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. ```python -# import necessary dependencies from haystack import Pipeline, Document -from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.components.generators.chat import HuggingFaceAPIChatGenerator from haystack.components.retrievers import InMemoryBM25Retriever from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.components.builders import ChatPromptBuilder from haystack.utils import Secret from haystack.dataclasses import ChatMessage -# create a document store and write documents to it document_store = InMemoryDocumentStore() document_store.write_documents([ Document(content="My name is Jean and I live in Paris."), @@ -71,7 +104,6 @@ document_store.write_documents([ Document(content="My name is Giorgio and I live in Rome.") ]) -# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer. prompt_template = [ ChatMessage.from_system( """ @@ -80,32 +112,157 @@ prompt_template = [ {% for doc in documents %} {{ doc.content }} {% endfor %} - Question: """ ), - ChatMessage.from_user( - "{{question}}" + ChatMessage.from_user("{{question}}") +] + +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = HuggingFaceAPIChatGenerator( + api_type="serverless_inference_api", + api_params={"model": "Qwen/Qwen2.5-72B-Instruct"}, + token=Secret.from_env_var("HF_API_TOKEN") +) + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") + +question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) + +print(results["llm"]["replies"]) +``` + + + + +Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic): + +```bash +pip install anthropic-haystack +``` + +See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details. + +```python +from haystack import Pipeline, Document +from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.utils import Secret +from haystack.dataclasses import ChatMessage + +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) + +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ ), - ChatMessage.from_system("Answer:") + ChatMessage.from_user("{{question}}") ] -# create the components adding the necessary parameters retriever = InMemoryBM25Retriever(document_store=document_store) prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") -llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini") +llm = AnthropicChatGenerator( + api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), + model="claude-sonnet-4-5-20250929" +) -# Create the pipeline and add the components to it. The order doesn't matter. -# At this stage, the Pipeline validates the components without running them yet. rag_pipeline = Pipeline() rag_pipeline.add_component("retriever", retriever) rag_pipeline.add_component("prompt_builder", prompt_builder) rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") + +question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) + +print(results["llm"]["replies"]) +``` + + + + +Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock): + +```bash +pip install amazon-bedrock-haystack +``` + +See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details. + +```python +import os +from haystack import Pipeline, Document +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.dataclasses import ChatMessage + +os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID" +os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY" +os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION" + +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) + +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ + ), + ChatMessage.from_user("{{question}}") +] -# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name"). +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0") + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) rag_pipeline.connect("retriever", "prompt_builder.documents") rag_pipeline.connect("prompt_builder", "llm") -# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components. question = "Who lives in Paris?" results = rag_pipeline.run( { @@ -118,25 +275,25 @@ print(results["llm"]["replies"]) ``` - + + +Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai): -:::note -**Using HuggingFaceAPIChatGenerator requires a Hugging Face API token.** -You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. -This API is rate-limited but perfect for experimentation. -::: +```bash +pip install google-genai-haystack +``` + +See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details. ```python -# import necessary dependencies from haystack import Pipeline, Document -from haystack.components.generators.chat import HuggingFaceAPIChatGenerator +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator from haystack.components.retrievers import InMemoryBM25Retriever from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.components.builders import ChatPromptBuilder from haystack.utils import Secret from haystack.dataclasses import ChatMessage -# create a document store and write documents to it document_store = InMemoryDocumentStore() document_store.write_documents([ Document(content="My name is Jean and I live in Paris."), @@ -144,7 +301,6 @@ document_store.write_documents([ Document(content="My name is Giorgio and I live in Rome.") ]) -# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer. prompt_template = [ ChatMessage.from_system( """ @@ -153,36 +309,25 @@ prompt_template = [ {% for doc in documents %} {{ doc.content }} {% endfor %} - Question: """ ), - ChatMessage.from_user( - "{{question}}" - ), - ChatMessage.from_system("Answer:") + ChatMessage.from_user("{{question}}") ] -# create the components adding the necessary parameters retriever = InMemoryBM25Retriever(document_store=document_store) prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") -llm = HuggingFaceAPIChatGenerator( - api_type="serverless_inference_api", - api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"}, - token=Secret.from_env_var("HF_API_TOKEN") +llm = GoogleGenAIChatGenerator( + api_key=Secret.from_env_var("GOOGLE_API_KEY"), + model="gemini-2.5-flash" ) -# Create the pipeline and add the components to it. The order doesn't matter. -# At this stage, the Pipeline validates the components without running them yet. rag_pipeline = Pipeline() rag_pipeline.add_component("retriever", retriever) rag_pipeline.add_component("prompt_builder", prompt_builder) rag_pipeline.add_component("llm", llm) - -# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name"). rag_pipeline.connect("retriever", "prompt_builder.documents") rag_pipeline.connect("prompt_builder", "llm") -# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components. question = "Who lives in Paris?" results = rag_pipeline.run( { @@ -194,9 +339,213 @@ results = rag_pipeline.run( print(results["llm"]["replies"]) ``` + + + +
+ +Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options. + +Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx). + +You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. + +
+ + + + +### Next Steps + +Ready to dive deeper? Check out the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial for a step-by-step guide on building a complete RAG pipeline with your own data. + +## Build your first Agent + +Agents are AI systems that can use tools to gather information, perform actions, and interact with external systems. Let's build an agent that can search the web to answer questions. + +This example requires a [SerperDev API key](https://serper.dev/) for web search. Set it as the `SERPERDEV_API_KEY` environment variable. + + + + +[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=OpenAIChatGenerator( + api_key=Secret.from_env_var("OPENAI_API_KEY"), + model="gpt-4o-mini" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import HuggingFaceAPIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=HuggingFaceAPIChatGenerator( + api_type="serverless_inference_api", + api_params={"model": "Qwen/Qwen2.5-72B-Instruct"}, + token=Secret.from_env_var("HF_API_TOKEN") + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic): + +```bash +pip install anthropic-haystack +``` + +See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details. + +```python +from haystack.components.agents import Agent +from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=AnthropicChatGenerator( + api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), + model="claude-sonnet-4-5-20250929" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock): + +```bash +pip install amazon-bedrock-haystack +``` + +See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details. + +```python +import os +from haystack.components.agents import Agent +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch + +os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID" +os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY" +os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION" + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0"), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai): + +```bash +pip install google-genai-haystack +``` + +See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details. + +```python +from haystack.components.agents import Agent +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=GoogleGenAIChatGenerator( + api_key=Secret.from_env_var("GOOGLE_API_KEY"), + model="gemini-2.5-flash" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +
+ +Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options. + +Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx). + +You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. + +
+ -### Adding Your Data +### Next Steps -Instead of running the RAG pipeline on example data, learn how you can add your own custom data using [Document Stores](../concepts/document-store.mdx). +For a hands-on guide on creating a tool-calling agent that can use both components and pipelines as tools, check out the [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent) tutorial. diff --git a/docs-website/docusaurus.config.js b/docs-website/docusaurus.config.js index d0b2abb013..c6ee35896b 100644 --- a/docs-website/docusaurus.config.js +++ b/docs-website/docusaurus.config.js @@ -122,6 +122,54 @@ const config = { from: '/docs', to: '/docs/intro', }, + { + from: '/docs/get_started', + to: '/docs/get-started', + }, + { + from: '/docs/document_store', + to: '/docs/document-store', + }, + { + from: '/docs/components_overview', + to: '/docs/components-overview', + }, + { + from: '/docs/nodes_overview', + to: '/docs/components-overview', + }, + { + from: '/docs/retriever', + to: '/docs/retrievers', + }, + { + from: '/docs/ranker', + to: '/docs/rankers', + }, + { + from: '/docs/pipeline', + to: '/docs/pipelines', + }, + { + from: '/docs/prompt_node', + to: '/docs/promptbuilder', + }, + { + from: '/docs/ready_made_pipelines', + to: '/docs/pipeline-templates', + }, + { + from: '/docs/join_documents', + to: '/docs/documentjoiner', + }, + { + from: '/docs/dynamicchatpromptbuilder', + to: '/docs/chatpromptbuilder', + }, + { + from: '/docs/dynamicpromptbuilder', + to: '/docs/promptbuilder', + }, ], }, ], diff --git a/docs-website/versioned_docs/version-2.22/overview/get-started.mdx b/docs-website/versioned_docs/version-2.22/overview/get-started.mdx index 2dc3be4d43..ea4c0637a8 100644 --- a/docs-website/versioned_docs/version-2.22/overview/get-started.mdx +++ b/docs-website/versioned_docs/version-2.22/overview/get-started.mdx @@ -2,7 +2,7 @@ title: "Get Started" id: get-started slug: "/get-started" -description: "Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources." +description: "Learn how to quickly get up and running with Haystack. Build your first RAG pipeline and tool-calling Agent with step-by-step examples for multiple LLM providers." --- import Tabs from '@theme/Tabs'; @@ -10,7 +10,7 @@ import TabItem from '@theme/TabItem'; # Get Started -Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources. +Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing Haystack, building your first RAG pipeline, and creating a tool-calling Agent. ## Build your first RAG application @@ -22,48 +22,81 @@ First, install the minimal form of Haystack: pip install haystack-ai ``` -
+In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). Choose your preferred LLM provider from the tabs below. For easier use, you can also set the API key as an environment variable. -Were you already using Haystack 1.x? + + -:::warning +[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package. -Installing `farm-haystack` and `haystack-ai` in the same Python environment (virtualenv, Colab, or system) causes problems. +```python +from haystack import Pipeline, Document +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.utils import Secret +from haystack.dataclasses import ChatMessage -Installing both packages in the same environment can somehow work or fail in obscure ways. We suggest installing only one of these packages per Python environment. Make sure that you remove both packages if they are installed in the same environment, followed by installing only one of them: +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) -```bash -pip uninstall -y farm-haystack haystack-ai -pip install haystack-ai -``` +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ + ), + ChatMessage.from_user("{{question}}") +] -If you have any questions, please reach out to us on the [GitHub Discussion](https://github.com/deepset-ai/haystack/discussions) or [Discord](https://discord.com/invite/VBpFzsgRVF). -::: +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = OpenAIChatGenerator( + api_key=Secret.from_env_var("OPENAI_API_KEY"), + model="gpt-4o-mini" +) + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") -
+question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) -In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). You can choose between OpenAI or Hugging Face as your LLM provider. For easier use, you can also set the API key as an environment variable (`OPENAI_API_KEY` or `HF_API_TOKEN`). +print(results["llm"]["replies"]) +``` - - + + -:::note -**Using OpenAIChatGenerator requires an OpenAI API key with sufficient quota.** -New users on the free tier may immediately encounter a `429` ("insufficient_quota") error when running -this example. If you do not have enough OpenAI credits, try the Hugging Face tab instead. -::: +[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. ```python -# import necessary dependencies from haystack import Pipeline, Document -from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.components.generators.chat import HuggingFaceAPIChatGenerator from haystack.components.retrievers import InMemoryBM25Retriever from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.components.builders import ChatPromptBuilder from haystack.utils import Secret from haystack.dataclasses import ChatMessage -# create a document store and write documents to it document_store = InMemoryDocumentStore() document_store.write_documents([ Document(content="My name is Jean and I live in Paris."), @@ -71,7 +104,6 @@ document_store.write_documents([ Document(content="My name is Giorgio and I live in Rome.") ]) -# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer. prompt_template = [ ChatMessage.from_system( """ @@ -80,32 +112,157 @@ prompt_template = [ {% for doc in documents %} {{ doc.content }} {% endfor %} - Question: """ ), - ChatMessage.from_user( - "{{question}}" + ChatMessage.from_user("{{question}}") +] + +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = HuggingFaceAPIChatGenerator( + api_type="serverless_inference_api", + api_params={"model": "Qwen/Qwen2.5-72B-Instruct"}, + token=Secret.from_env_var("HF_API_TOKEN") +) + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") + +question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) + +print(results["llm"]["replies"]) +``` + + + + +Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic): + +```bash +pip install anthropic-haystack +``` + +See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details. + +```python +from haystack import Pipeline, Document +from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.utils import Secret +from haystack.dataclasses import ChatMessage + +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) + +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ ), - ChatMessage.from_system("Answer:") + ChatMessage.from_user("{{question}}") ] -# create the components adding the necessary parameters retriever = InMemoryBM25Retriever(document_store=document_store) prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") -llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini") +llm = AnthropicChatGenerator( + api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), + model="claude-sonnet-4-5-20250929" +) -# Create the pipeline and add the components to it. The order doesn't matter. -# At this stage, the Pipeline validates the components without running them yet. rag_pipeline = Pipeline() rag_pipeline.add_component("retriever", retriever) rag_pipeline.add_component("prompt_builder", prompt_builder) rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") + +question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) + +print(results["llm"]["replies"]) +``` + + + + +Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock): + +```bash +pip install amazon-bedrock-haystack +``` + +See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details. + +```python +import os +from haystack import Pipeline, Document +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.dataclasses import ChatMessage + +os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID" +os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY" +os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION" + +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) + +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ + ), + ChatMessage.from_user("{{question}}") +] -# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name"). +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0") + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) rag_pipeline.connect("retriever", "prompt_builder.documents") rag_pipeline.connect("prompt_builder", "llm") -# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components. question = "Who lives in Paris?" results = rag_pipeline.run( { @@ -118,25 +275,25 @@ print(results["llm"]["replies"]) ``` - + + +Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai): -:::note -**Using HuggingFaceAPIChatGenerator requires a Hugging Face API token.** -You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. -This API is rate-limited but perfect for experimentation. -::: +```bash +pip install google-genai-haystack +``` + +See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details. ```python -# import necessary dependencies from haystack import Pipeline, Document -from haystack.components.generators.chat import HuggingFaceAPIChatGenerator +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator from haystack.components.retrievers import InMemoryBM25Retriever from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.components.builders import ChatPromptBuilder from haystack.utils import Secret from haystack.dataclasses import ChatMessage -# create a document store and write documents to it document_store = InMemoryDocumentStore() document_store.write_documents([ Document(content="My name is Jean and I live in Paris."), @@ -144,7 +301,6 @@ document_store.write_documents([ Document(content="My name is Giorgio and I live in Rome.") ]) -# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer. prompt_template = [ ChatMessage.from_system( """ @@ -153,36 +309,25 @@ prompt_template = [ {% for doc in documents %} {{ doc.content }} {% endfor %} - Question: """ ), - ChatMessage.from_user( - "{{question}}" - ), - ChatMessage.from_system("Answer:") + ChatMessage.from_user("{{question}}") ] -# create the components adding the necessary parameters retriever = InMemoryBM25Retriever(document_store=document_store) prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") -llm = HuggingFaceAPIChatGenerator( - api_type="serverless_inference_api", - api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"}, - token=Secret.from_env_var("HF_API_TOKEN") +llm = GoogleGenAIChatGenerator( + api_key=Secret.from_env_var("GOOGLE_API_KEY"), + model="gemini-2.5-flash" ) -# Create the pipeline and add the components to it. The order doesn't matter. -# At this stage, the Pipeline validates the components without running them yet. rag_pipeline = Pipeline() rag_pipeline.add_component("retriever", retriever) rag_pipeline.add_component("prompt_builder", prompt_builder) rag_pipeline.add_component("llm", llm) - -# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name"). rag_pipeline.connect("retriever", "prompt_builder.documents") rag_pipeline.connect("prompt_builder", "llm") -# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components. question = "Who lives in Paris?" results = rag_pipeline.run( { @@ -194,9 +339,213 @@ results = rag_pipeline.run( print(results["llm"]["replies"]) ``` + + + +
+ +Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options. + +Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx). + +You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. + +
+ + + + +### Next Steps + +Ready to dive deeper? Check out the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial for a step-by-step guide on building a complete RAG pipeline with your own data. + +## Build your first Agent + +Agents are AI systems that can use tools to gather information, perform actions, and interact with external systems. Let's build an agent that can search the web to answer questions. + +This example requires a [SerperDev API key](https://serper.dev/) for web search. Set it as the `SERPERDEV_API_KEY` environment variable. + + + + +[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=OpenAIChatGenerator( + api_key=Secret.from_env_var("OPENAI_API_KEY"), + model="gpt-4o-mini" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import HuggingFaceAPIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=HuggingFaceAPIChatGenerator( + api_type="serverless_inference_api", + api_params={"model": "Qwen/Qwen2.5-72B-Instruct"}, + token=Secret.from_env_var("HF_API_TOKEN") + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic): + +```bash +pip install anthropic-haystack +``` + +See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details. + +```python +from haystack.components.agents import Agent +from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=AnthropicChatGenerator( + api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), + model="claude-sonnet-4-5-20250929" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock): + +```bash +pip install amazon-bedrock-haystack +``` + +See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details. + +```python +import os +from haystack.components.agents import Agent +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch + +os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID" +os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY" +os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION" + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0"), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai): + +```bash +pip install google-genai-haystack +``` + +See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details. + +```python +from haystack.components.agents import Agent +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=GoogleGenAIChatGenerator( + api_key=Secret.from_env_var("GOOGLE_API_KEY"), + model="gemini-2.5-flash" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +
+ +Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options. + +Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx). + +You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. + +
+ -### Adding Your Data +### Next Steps -Instead of running the RAG pipeline on example data, learn how you can add your own custom data using [Document Stores](../concepts/document-store.mdx). +For a hands-on guide on creating a tool-calling agent that can use both components and pipelines as tools, check out the [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent) tutorial. diff --git a/docs-website/versioned_docs/version-2.23-unstable/overview/get-started.mdx b/docs-website/versioned_docs/version-2.23-unstable/overview/get-started.mdx index 2dc3be4d43..ea4c0637a8 100644 --- a/docs-website/versioned_docs/version-2.23-unstable/overview/get-started.mdx +++ b/docs-website/versioned_docs/version-2.23-unstable/overview/get-started.mdx @@ -2,7 +2,7 @@ title: "Get Started" id: get-started slug: "/get-started" -description: "Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources." +description: "Learn how to quickly get up and running with Haystack. Build your first RAG pipeline and tool-calling Agent with step-by-step examples for multiple LLM providers." --- import Tabs from '@theme/Tabs'; @@ -10,7 +10,7 @@ import TabItem from '@theme/TabItem'; # Get Started -Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources. +Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing Haystack, building your first RAG pipeline, and creating a tool-calling Agent. ## Build your first RAG application @@ -22,48 +22,81 @@ First, install the minimal form of Haystack: pip install haystack-ai ``` -
+In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). Choose your preferred LLM provider from the tabs below. For easier use, you can also set the API key as an environment variable. -Were you already using Haystack 1.x? + + -:::warning +[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package. -Installing `farm-haystack` and `haystack-ai` in the same Python environment (virtualenv, Colab, or system) causes problems. +```python +from haystack import Pipeline, Document +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.utils import Secret +from haystack.dataclasses import ChatMessage -Installing both packages in the same environment can somehow work or fail in obscure ways. We suggest installing only one of these packages per Python environment. Make sure that you remove both packages if they are installed in the same environment, followed by installing only one of them: +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) -```bash -pip uninstall -y farm-haystack haystack-ai -pip install haystack-ai -``` +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ + ), + ChatMessage.from_user("{{question}}") +] -If you have any questions, please reach out to us on the [GitHub Discussion](https://github.com/deepset-ai/haystack/discussions) or [Discord](https://discord.com/invite/VBpFzsgRVF). -::: +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = OpenAIChatGenerator( + api_key=Secret.from_env_var("OPENAI_API_KEY"), + model="gpt-4o-mini" +) + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") -
+question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) -In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). You can choose between OpenAI or Hugging Face as your LLM provider. For easier use, you can also set the API key as an environment variable (`OPENAI_API_KEY` or `HF_API_TOKEN`). +print(results["llm"]["replies"]) +``` - - + + -:::note -**Using OpenAIChatGenerator requires an OpenAI API key with sufficient quota.** -New users on the free tier may immediately encounter a `429` ("insufficient_quota") error when running -this example. If you do not have enough OpenAI credits, try the Hugging Face tab instead. -::: +[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. ```python -# import necessary dependencies from haystack import Pipeline, Document -from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.components.generators.chat import HuggingFaceAPIChatGenerator from haystack.components.retrievers import InMemoryBM25Retriever from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.components.builders import ChatPromptBuilder from haystack.utils import Secret from haystack.dataclasses import ChatMessage -# create a document store and write documents to it document_store = InMemoryDocumentStore() document_store.write_documents([ Document(content="My name is Jean and I live in Paris."), @@ -71,7 +104,6 @@ document_store.write_documents([ Document(content="My name is Giorgio and I live in Rome.") ]) -# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer. prompt_template = [ ChatMessage.from_system( """ @@ -80,32 +112,157 @@ prompt_template = [ {% for doc in documents %} {{ doc.content }} {% endfor %} - Question: """ ), - ChatMessage.from_user( - "{{question}}" + ChatMessage.from_user("{{question}}") +] + +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = HuggingFaceAPIChatGenerator( + api_type="serverless_inference_api", + api_params={"model": "Qwen/Qwen2.5-72B-Instruct"}, + token=Secret.from_env_var("HF_API_TOKEN") +) + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") + +question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) + +print(results["llm"]["replies"]) +``` + + + + +Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic): + +```bash +pip install anthropic-haystack +``` + +See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details. + +```python +from haystack import Pipeline, Document +from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.utils import Secret +from haystack.dataclasses import ChatMessage + +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) + +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ ), - ChatMessage.from_system("Answer:") + ChatMessage.from_user("{{question}}") ] -# create the components adding the necessary parameters retriever = InMemoryBM25Retriever(document_store=document_store) prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") -llm = OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY"), model="gpt-4o-mini") +llm = AnthropicChatGenerator( + api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), + model="claude-sonnet-4-5-20250929" +) -# Create the pipeline and add the components to it. The order doesn't matter. -# At this stage, the Pipeline validates the components without running them yet. rag_pipeline = Pipeline() rag_pipeline.add_component("retriever", retriever) rag_pipeline.add_component("prompt_builder", prompt_builder) rag_pipeline.add_component("llm", llm) +rag_pipeline.connect("retriever", "prompt_builder.documents") +rag_pipeline.connect("prompt_builder", "llm") + +question = "Who lives in Paris?" +results = rag_pipeline.run( + { + "retriever": {"query": question}, + "prompt_builder": {"question": question}, + } +) + +print(results["llm"]["replies"]) +``` + + + + +Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock): + +```bash +pip install amazon-bedrock-haystack +``` + +See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details. + +```python +import os +from haystack import Pipeline, Document +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator +from haystack.components.retrievers import InMemoryBM25Retriever +from haystack.document_stores.in_memory import InMemoryDocumentStore +from haystack.components.builders import ChatPromptBuilder +from haystack.dataclasses import ChatMessage + +os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID" +os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY" +os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION" + +document_store = InMemoryDocumentStore() +document_store.write_documents([ + Document(content="My name is Jean and I live in Paris."), + Document(content="My name is Mark and I live in Berlin."), + Document(content="My name is Giorgio and I live in Rome.") +]) + +prompt_template = [ + ChatMessage.from_system( + """ + Given these documents, answer the question. + Documents: + {% for doc in documents %} + {{ doc.content }} + {% endfor %} + """ + ), + ChatMessage.from_user("{{question}}") +] -# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name"). +retriever = InMemoryBM25Retriever(document_store=document_store) +prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") +llm = AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0") + +rag_pipeline = Pipeline() +rag_pipeline.add_component("retriever", retriever) +rag_pipeline.add_component("prompt_builder", prompt_builder) +rag_pipeline.add_component("llm", llm) rag_pipeline.connect("retriever", "prompt_builder.documents") rag_pipeline.connect("prompt_builder", "llm") -# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components. question = "Who lives in Paris?" results = rag_pipeline.run( { @@ -118,25 +275,25 @@ print(results["llm"]["replies"]) ``` - + + +Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai): -:::note -**Using HuggingFaceAPIChatGenerator requires a Hugging Face API token.** -You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. -This API is rate-limited but perfect for experimentation. -::: +```bash +pip install google-genai-haystack +``` + +See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details. ```python -# import necessary dependencies from haystack import Pipeline, Document -from haystack.components.generators.chat import HuggingFaceAPIChatGenerator +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator from haystack.components.retrievers import InMemoryBM25Retriever from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.components.builders import ChatPromptBuilder from haystack.utils import Secret from haystack.dataclasses import ChatMessage -# create a document store and write documents to it document_store = InMemoryDocumentStore() document_store.write_documents([ Document(content="My name is Jean and I live in Paris."), @@ -144,7 +301,6 @@ document_store.write_documents([ Document(content="My name is Giorgio and I live in Rome.") ]) -# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer. prompt_template = [ ChatMessage.from_system( """ @@ -153,36 +309,25 @@ prompt_template = [ {% for doc in documents %} {{ doc.content }} {% endfor %} - Question: """ ), - ChatMessage.from_user( - "{{question}}" - ), - ChatMessage.from_system("Answer:") + ChatMessage.from_user("{{question}}") ] -# create the components adding the necessary parameters retriever = InMemoryBM25Retriever(document_store=document_store) prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*") -llm = HuggingFaceAPIChatGenerator( - api_type="serverless_inference_api", - api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"}, - token=Secret.from_env_var("HF_API_TOKEN") +llm = GoogleGenAIChatGenerator( + api_key=Secret.from_env_var("GOOGLE_API_KEY"), + model="gemini-2.5-flash" ) -# Create the pipeline and add the components to it. The order doesn't matter. -# At this stage, the Pipeline validates the components without running them yet. rag_pipeline = Pipeline() rag_pipeline.add_component("retriever", retriever) rag_pipeline.add_component("prompt_builder", prompt_builder) rag_pipeline.add_component("llm", llm) - -# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name"). rag_pipeline.connect("retriever", "prompt_builder.documents") rag_pipeline.connect("prompt_builder", "llm") -# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components. question = "Who lives in Paris?" results = rag_pipeline.run( { @@ -194,9 +339,213 @@ results = rag_pipeline.run( print(results["llm"]["replies"]) ``` + + + +
+ +Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options. + +Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx). + +You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. + +
+ + + + +### Next Steps + +Ready to dive deeper? Check out the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial for a step-by-step guide on building a complete RAG pipeline with your own data. + +## Build your first Agent + +Agents are AI systems that can use tools to gather information, perform actions, and interact with external systems. Let's build an agent that can search the web to answer questions. + +This example requires a [SerperDev API key](https://serper.dev/) for web search. Set it as the `SERPERDEV_API_KEY` environment variable. + + + + +[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=OpenAIChatGenerator( + api_key=Secret.from_env_var("OPENAI_API_KEY"), + model="gpt-4o-mini" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import HuggingFaceAPIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=HuggingFaceAPIChatGenerator( + api_type="serverless_inference_api", + api_params={"model": "Qwen/Qwen2.5-72B-Instruct"}, + token=Secret.from_env_var("HF_API_TOKEN") + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic): + +```bash +pip install anthropic-haystack +``` + +See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details. + +```python +from haystack.components.agents import Agent +from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=AnthropicChatGenerator( + api_key=Secret.from_env_var("ANTHROPIC_API_KEY"), + model="claude-sonnet-4-5-20250929" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock): + +```bash +pip install amazon-bedrock-haystack +``` + +See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details. + +```python +import os +from haystack.components.agents import Agent +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch + +os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID" +os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY" +os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION" + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0"), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai): + +```bash +pip install google-genai-haystack +``` + +See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details. + +```python +from haystack.components.agents import Agent +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.tools import ComponentTool +from haystack.components.websearch import SerperDevWebSearch +from haystack.utils import Secret + +search_tool = ComponentTool(component=SerperDevWebSearch()) + +agent = Agent( + chat_generator=GoogleGenAIChatGenerator( + api_key=Secret.from_env_var("GOOGLE_API_KEY"), + model="gemini-2.5-flash" + ), + tools=[search_tool], + system_prompt="You are a helpful assistant that can search the web for information." +) + +result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")]) + +print(result["last_message"].text) +``` + + + + +
+ +Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options. + +Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx). + +You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. + +
+ -### Adding Your Data +### Next Steps -Instead of running the RAG pipeline on example data, learn how you can add your own custom data using [Document Stores](../concepts/document-store.mdx). +For a hands-on guide on creating a tool-calling agent that can use both components and pipelines as tools, check out the [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent) tutorial.