Chapter 35: FastAPI for ML Deployment
MODULE 6 | MLOps
Chapter 35: FastAPI for ML Deployment
Until now your model lived inside a Python file or notebook . No website, mobile app or other program can run it directly. In this chapter we will wrap the model in an API so that anyone can send data to it over the internet (or on your own computer) and get an answer back. For this job we will use FastAPI .
TOPIC CHECKLIST (tick them off as you go)
-
☐ What an API is (GET, POST, JSON, status codes)
-
☐ Installing FastAPI and the first endpoint
-
☐ Serving a trained model through the
/predictendpoint -
Input validation with Pydantic
-
☐
-
Testing with Swagger docs (
/docs) -
☐
-
☐ Automatic testing with
TestClientandpytest
35.1 What is an API?
Example of JSON
{"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}
| HTTP method | Purpose | Example in ML |
|---|---|---|
| GET | Gettinginformation | /health (is the server fine?), /symptoms (get the list) |
| POST | Sendingdata to get work done | /predict (send features, get a prediction) |
| Status code | Meaning | When it appears |
|---|---|---|
| 200 | OK, all good | The request succeeded |
| 400 | Bad Request | Error in the request (from our own check) |
| 404 | Not Found | No such URL exists |
| 422 | Unprocessable Entity | Wrong data format or value (Pydantic's check) |
| 500 | Server Error | A problem in the code or model inside the server |
35.2 Installing FastAPI and the First Endpoint
FastAPI is a fast and easy Python framework made for building APIs. It runs together with Uvicorn , which is the server (the program that runs the API). We installed both in Chapter 34. Let's first write the smallest possible API. Put this code in hello.py:
hello.py
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def home():
return {"message": "Hello! FastAPI is running"}
Now run the server in the terminal (hello is the file name, and app is the object inside it):
TERMINAL
uvicorn hello:app --reload TERMINAL OUTPUT (the folder name and process numbers will differ on your machine) INFO: Will watch for changes in these directories: ['/tmp/snip'] INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit) INFO: Started reloader process [377] using StatReload INFO: Started server process [380] INFO: Waiting for application startup. INFO: Application startup complete.
Now open http://127.0.0.1:8000 in the browser, or run curl in another terminal:
TERMINAL
curl http://127.0.0.1:8000/ OUTPUT {"message":"Hello! FastAPI is running"}
| Part of the code | Meaning |
|---|---|
app = FastAPI() | Theactual objectof our API; all endpoints connect to it |
@app.get("/") | Decorator: tells us that the function below will run on aGETrequest, at the**/**address |
return {...} | A Pythondictionaryis automatically turned intoJSON |
--reload | When you change the code, the serverrestarts by itself(in development only) |
35.3 Serving a Trained Model Through the /predict Endpoint
Now the real work: we plug the iris_model.joblib saved in Chapter 34 into the API. Look at two things carefully : the model is loaded once, at the start (not on every request), and /predict is POST because we are sending data. The file name is iris_api.py:
iris_api.py
import joblib
from fastapi import FastAPI
from pydantic import BaseModel, Field
app = FastAPI(title="Iris Predictor")
model = joblib.load("iris_model.joblib")
CLASS_NAMES = ["setosa", "versicolor", "virginica"]
class IrisInput(BaseModel):
sepal_length:
float = Field(gt=0, lt=15) sepal_width:
float = Field(gt=0, lt=15) petal_length:
float = Field(gt=0, lt=15) petal_width:
float = Field(gt=0, lt=15) @app.get("/health")
def health():
return {"status": "ok"} @app.post("/predict")
def predict(data: IrisInput):
features = [[data.sepal_length, data.sepal_width,
data.petal_length, data.petal_width]]
probs = model.predict_proba(features)[0]
idx = int(probs.argmax())
return {"species": CLASS_NAMES[idx], "confidence": round(float(probs[idx]), 3)}
Run the server (uvicorn iris_api:app --reload) and send the measurements of two different flowers to see the result:
TERMINAL
curl http://127.0.0.1:8000/health
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4,"petal_width":0.2}'
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":6.7,"sepal_width":3.0,"petal_length":5.2,"petal_width":2.3}'
OUTPUT
{"status":"ok"}
{"species":"setosa","confidence":1.0}
{"species":"virginica","confidence":1.0}
35.4 Input Validation with Pydantic
If someone sends -1 cm or "abc" , the model will get bad data and the API will give a strange answer without telling you. Pydantic solves this problem: in the IrisInput class you state what each field should look like, and FastAPI checks it before it reaches the model .
| Part of Pydantic | What it does |
|---|---|
class IrisInput(BaseModel): | Defines the**structure (schema)**of the input |
sepal_length: float | The type of this field must befloat("abc" will not work) |
Field(gt=0, lt=15) | The value must begreater than 0andless than 15(gt = greater than, lt = lessthan) |
| Field is missing | Automatically gives a422error and a clear explanation |
Now send wrong input and see (a negative value in the first, and a missing field in the second):
TERMINAL
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":-1,"sepal_width":3.5,"petal_length":1.4,"petal_width":0.2}'
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4}'
OUTPUT
{"detail":[{"type":"greater_than","loc":["body","sepal_length"],
"msg":"Input should be greater than 0","input":-1,"ctx":{"gt":0.0}}]}
{"detail":[{"type":"missing","loc":["body","petal_width"],"msg":"Field required",
"input":{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4}}]}
(The real response comes on a single line; it is broken into two lines here to make it readable.) Both times the status code was 422 (the server log shows 422 Unprocessable Entity), and loc tells you which field has the error. This data never reached the model. This is the benefit of validation : cheap, fast and safe .
35.5 Testing with Swagger Docs (/docs)
FastAPI's loveliest feature: a testing page is made automatically for your API. While the server is running, open this in the browser: http://127.0.0.1:8000/docs. This page is called Swagger UI .
-
Open
http://127.0.0.1:8000/docsin the browser. You will see the list of all your endpoints (/health,/predict). -
Click on
POST /predict, then press Try it out . -
Write your own JSON in the request body box (or change the values in the given example).
-
Press Execute . Below, in the Response body , you will see the answer and the Code (200 or 422) .
-
In the Schema section your
IrisInputis also shown, which means the documentation is ready for others.
| Address | What it is |
|---|---|
/docs | Swagger UI: interactive testing page |
/redoc | Another documentation page that is easy to read |
/openapi.json | The complete map of the API in a machine-readable form |
35.6 Automatic Testing: TestClient and pytest
Clicking by hand in /docs every time is tiring, and in CI/CD (Chapter 38) this work must happen automatically. This is why we write tests in code . FastAPI's TestClient calls the API without running a server. The file name is test_iris_api.py:
test_iris_api.py
from fastapi.testclient import TestClient
from iris_api import app
client = TestClient(app)
GOOD = {"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}
def test_health():
res = client.get("/health") assert res.status_code == 200 assert res.json() == {"status": "ok"}
def test_predict_ok():
res = client.post("/predict", json=GOOD) assert res.status_code == 200 assert res.json()["species"] == "setosa"
def test_predict_negative_value():
res = client.post("/predict", json={**GOOD, "sepal_length": -1}) assert res.status_code == 422
def test_predict_missing_field():
bad = {k: v for k, v in GOOD.items() if k != "petal_width"}
res = client.post("/predict", json=bad) assert res.status_code == 422
TERMINAL
pytest -v
OUTPUT
test_iris_api.py::test_health PASSED [ 25%]
test_iris_api.py::test_predict_ok PASSED [ 50%]
test_iris_api.py::test_predict_negative_value PASSED [ 75%]
test_iris_api.py::test_predict_missing_field PASSED [100%]
========================= 4 passed, 1 warning in 1.56s =========================
35.7 Small Good Habits for Production
| Habit | Why |
|---|---|
Keep a /health endpoint | Docker, load balancers and monitoring use it to learn whether the API is alive |
| Also define the format of theresponsewith i | The response will always stay the same shape and documented |
response_model / Pydantic | |
| Load the modelonce | Loading on every request makes the API slow |
Setlimitson values (gt, lt) | So that useless input never reaches the model |
| Write tests | The fear of the API breaking after every change goes away |
35.8 Common Mistakes and Hands-on
Hands-on
-
Save the model of one of your old projects (for example Titanic or Iris) with
jobliband build its/predictAPI . -
Change the limits of
Field(gt=..., lt=...)and see which values give 422 . -
Open
/docsand make 3 different predictions through Swagger. -
Write one more test in
test_iris_api.py: make the test fail when the response of/healthis wrong (change the response on purpose and see).
KEY TAKEAWAYS
-
An API is the model's waiter : it takes the request to the model and brings back a JSON response .
-
GET is for getting information, POST is for sending data and getting a prediction.
-
In FastAPI, an endpoint is made with the decorators
@app.get(...)/@app.post(...), and Uvicorn runs the server. -
Load the model once , and serve it through
/predict; Pydantic stops wrong input with a 422 . -
Testing is easy in
/docs(Swagger), andTestClient+pytestmake testing automatic.