Skip to main content

Module 6: Generative AI

Weeks 29 to 31. Explore the foundations behind generative AI: Transformers and Attention, LLMs (GPT, BERT), text generation and fine-tuning with the OpenAI API. First build key ideas with NumPy and small Python examples, then use the API to create text-generation applications.

ChapterWeekTopics
29Week 29Transformers and Attention Mechanism
(Self-Attention, Positional Encoding)
30Week 30Large Language Models (GPT, BERT) and
LLM Applications
31Week 31Text Generation and Fine-tuning using
OpenAI API
<mark>NOTE</mark>​
  • Two kinds of code: (1) NumPy/Python code (attention, BPE, sampling, RAG, expense tracking, dataset, agent skeleton): run it

  • anywhere, the OUTPUT is real. (2) OpenAI API, Agents SDK, Hugging Face and Keras code: run it in Colab; these need an API key or internet, and their outputs are samples (the model's answers may differ a little each time).

  • The OpenAI API is paid: you are charged by tokens. Set a spending limit on the dashboard and never put your API key in code

  • or on GitHub.

  • Model names and features change quickly: the model name is kept in a variable in the code. Before running, check today's

  • model name in the OpenAI docs.

Chapters​

Module Test​

Module 6 Test.