Generative AI: An Introductory Script

This script serves as an introduction to Generative AI and was developed for the elective module "Generative AI," offered to master's students of the "Data Science" program at the University of Applied Sciences Kiel. Built using quarto, this resource is designed to provide an accessible overview of key topics and applications in this rapidly evolving field.

While not an exhaustive guide to Generative AI, the script highlights foundational concepts, modern applications, and practical techniques that empower students to engage with and explore the possibilities of these transformative technologies.

You can find a rendered html version of the script by following this link

---

📚 Contents and Learning Objectives

Topics Covered

Based on the module database entry, this script covers the following key areas:

Open Source Language Models

• Overview of available models
• Tools like Ollama
• Synthetic text generation for training sets

Agent Systems

• Frameworks: LlamaIndex, LangChain, and Haystack
• Function calling and data analysis

Embeddings and Vector Stores

• Semantic search
• Embedding based recommendations
• Retrieval-augmented generation

AI Image Generators

• Generative Adversarial Networks (GANs)
• Variational Autoencoders (VAEs) and Diffusion Models
• Generative techniques for image dataset augmentation

Fine-Tuning of LLMs and Diffusion Models

• Techniques such as LoRA, QLoRA, and MoRA

---

Learning Objectives

Students engaging with this module will:

Understand:

• Fundamentals of Generative AI systems.
• Theoretical and practical foundations of Generative AI.
• Applications and modern use cases of Generative AI.

Develop Skills to:

• Explain and apply open-source language models.
• Implement and utilize agent systems.
• Leverage embeddings and vector stores for tasks like semantic search.
• Generate and apply AI-created images using advanced techniques.
• Fine-tune large language models and diffusion models for specific tasks.

Collaborate Effectively to:

• Organize teamwork for Generative AI projects.
• Report, present, and explain team solutions.
• Interpret and communicate technical and functional aspects of Generative AI projects.

---

🗓 Schedule

Number:	Date:	Title:	Topics:
1	10.11.	Getting started with (L)LMs	Language Model Basics
			Choosing open source models
			Basics of using open source models (Huggingface, Ollama, LLM-Studio, Llama.cpp, ...)
2	12.11.	Prompting	Prompting strategies
			Generation of synthetic texts
3	17.11.	Function Calling	Code generation and function calling
			MCP
4	19.11.	Embedding-based retrieval systems	Semantic embeddings and vector stores
			Retrieval augmented and interleaved generation
5	24.11.	Agent basics	Fundamentals of agents
			Examples of agent-frameworks (Llamaindex, LangChain & smolagents)
6	26.11.	LLM-pipelines
7	1.12.	LLM-based automated Data Analysis
8	3.12.	AI image generation I	AI image generator basics
			Diffusion Models and Variational Autoencoders
			Multimodal models
9	8.12.	AI image generation II	Using Open Source AI image generation models
			AI image generators in agent systems
10	10.12.	Finetuning Basics	Basics of Finetuning strategies
		Rank adaptation	Fundamentals of High and Low-Rank Adaptation of Language and Diffusion Models
			(Q)LoRA fine-tuning using Unsloth
11	15.12.	Alignment	Central principles of Model-Alignment
			Reinforcement Learning from Human Feedback (RLHF)
12	17.12.	Project presentations
	16.01.	Project submission on moodle

---

⚙️ Built With

• Quarto: A modern tool for publishing and sharing technical content.

---

💡 Notes

This script is continuously evolving and is intended to inspire curiosity and provide actionable insights into Generative AI. Contributions and feedback are always welcome.