Module 6: Generative AI
Weeks 29 to 31. Explore the foundations behind generative AI: Transformers and Attention, LLMs (GPT, BERT), text generation and fine-tuning with the OpenAI API. First build key ideas with NumPy and small Python examples, then use the API to create text-generation applications.
| Chapter | Week | Topics |
|---|---|---|
| 29 | Week 29 | Transformers and Attention Mechanism |
| (Self-Attention, Positional Encoding) | ||
| 30 | Week 30 | Large Language Models (GPT, BERT) and |
| LLM Applications | ||
| 31 | Week 31 | Text Generation and Fine-tuning using |
| OpenAI API |
<mark>NOTE</mark>
-
Two kinds of code: (1) NumPy/Python code (attention, BPE, sampling, RAG, expense tracking, dataset, agent skeleton): run it
-
anywhere, the OUTPUT is real. (2) OpenAI API, Agents SDK, Hugging Face and Keras code: run it in Colab; these need an API key or internet, and their outputs are samples (the model's answers may differ a little each time).
-
The OpenAI API is paid: you are charged by tokens. Set a spending limit on the dashboard and never put your API key in code
-
or on GitHub.
-
Model names and features change quickly: the model name is kept in a variable in the code. Before running, check today's
-
model name in the OpenAI docs.
Chapters
- Chapter 29: Transformers and Attention Mechanism
- Chapter 30: Large Language Models (GPT, BERT) and LLM Applications
- Chapter 31: Text Generation and Fine-tuning using OpenAI API