AI Cooking Guide

A chatbot with cooking knowledge, build specifically for cooking.

Description

This is a simple cooking specific agent that can retrieve recipies, give clarifications, and call fuctions, such as starting a timer. It support both voice and text input, and responses are read out loud.

Usage

1. Host the client on a website, then set up the server (See Program Logic)

2. Repackage it as a electron app, then set up the server (See Program Logic)

3. Repackage it on mobile, then set up the server (See Program Logic)

Program Logic

This app uses deepseek V3 as the llm powering the assistant (though this can be easily configured). Using TTS and STT, the user can talk to deepseek, and it can respond. The LLM has cooking specific knowlege. The flow of the app is like this:

1. Assistant asks the user what they want to eat, and/or what ingredients are available.
2. Assistant calls a function and pass in several key words to search an external database of existing recipes. Each recipe is broken into steps.
3. If any comes up, assistant asks the user if they want the top five results in order. If none of them is desired, assistant discusses with the user. Coversation mode is used until assistant calls another function, where it passes in all the steps, as an alternative to the recipies in the database.
4. At this point, there is a recipe. Assistant reads the first step of the recipe, and the user is allowed to chat and ask for claification. This goes on until assistant decides they should move on (e.g. after the user says ok, next step or something equivilant). Assistant decides this by calling a function nextstep(). (previous step can also be called).
5. This continues until the recipy is done, after which the app return to (1).

Issues and Possible Improvements

1. There is currently very limited sercurity and no login system. This means the client side has no rate limits and is vulnerable for exploits. The server might get DDOSed as well.

The simple fix is a login system and a rate limit on the serverside, as well as possibly adding some extra encyption and using cloudfare.

2. Menu is only added manually. With so many exisiting communities and resources online that is regularly updated, it is hard for people to trust an AI over their favorite tutorials online.

Therefore, some sort of scaping mechanism for new menus as well as links to original sources is highly neccessary as a improvement.

3. The current STT doen't work in non-chrome environment like electron. The voice is also very robotic.

The fix is to move most of the speech handeling from client-side to the server-side. The client side essentially only uploads a recording of the user, and the server returns a voiceline. Also more voices could be added.

4. Finetune the LLM for cooking.

Ending Notes

This project is build almost complemetly using Grok 4. This README on the other hand is completely hand written. CSS based on tailwind (https://tailwindcss.com/)

This project is under the MIT Lisense.