Skip to main content

Chapter 35: FastAPI for ML Deployment

MODULE 6 | MLOps​

Chapter 35: FastAPI for ML Deployment​

Until now your model lived inside a Python file or notebook . No website, mobile app or other program can run it directly. In this chapter we will wrap the model in an API so that anyone can send data to it over the internet (or on your own computer) and get an answer back. For this job we will use FastAPI .

TOPIC CHECKLIST (tick them off as you go)​

  • ☐ What an API is (GET, POST, JSON, status codes)

  • ☐ Installing FastAPI and the first endpoint

  • ☐ Serving a trained model through the /predict endpoint

  • Input validation with Pydantic

  • ☐

  • Testing with Swagger docs (/docs)

  • ☐

  • ☐ Automatic testing with TestClient and pytest

35.1 What is an API?​

Example of JSON​
{"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}
HTTP methodPurposeExample in ML
GETGettinginformation/health (is the server fine?), /symptoms (get the list)
POSTSendingdata to get work done/predict (send features, get a prediction)
Status codeMeaningWhen it appears
200OK, all goodThe request succeeded
400Bad RequestError in the request (from our own check)
404Not FoundNo such URL exists
422Unprocessable EntityWrong data format or value (Pydantic's check)
500Server ErrorA problem in the code or model inside the server

35.2 Installing FastAPI and the First Endpoint​

FastAPI is a fast and easy Python framework made for building APIs. It runs together with Uvicorn , which is the server (the program that runs the API). We installed both in Chapter 34. Let's first write the smallest possible API. Put this code in hello.py:

hello.py​
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def home():
return {"message": "Hello! FastAPI is running"}

Now run the server in the terminal (hello is the file name, and app is the object inside it):

TERMINAL​

uvicorn hello:app --reload TERMINAL OUTPUT (the folder name and process numbers will differ on your machine) INFO: Will watch for changes in these directories: ['/tmp/snip'] INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit) INFO: Started reloader process [377] using StatReload INFO: Started server process [380] INFO: Waiting for application startup. INFO: Application startup complete.

Now open http://127.0.0.1:8000 in the browser, or run curl in another terminal:

TERMINAL​

curl http://127.0.0.1:8000/ OUTPUT {"message":"Hello! FastAPI is running"}

Part of the codeMeaning
app = FastAPI()Theactual objectof our API; all endpoints connect to it
@app.get("/")Decorator: tells us that the function below will run on aGETrequest, at the**/**address
return {...}A Pythondictionaryis automatically turned intoJSON
--reloadWhen you change the code, the serverrestarts by itself(in development only)

35.3 Serving a Trained Model Through the /predict Endpoint​

Now the real work: we plug the iris_model.joblib saved in Chapter 34 into the API. Look at two things carefully : the model is loaded once, at the start (not on every request), and /predict is POST because we are sending data. The file name is iris_api.py:

iris_api.py

import joblib
from fastapi import FastAPI
from pydantic import BaseModel, Field
app = FastAPI(title="Iris Predictor")
model = joblib.load("iris_model.joblib")
CLASS_NAMES = ["setosa", "versicolor", "virginica"]
class IrisInput(BaseModel):
sepal_length:
float = Field(gt=0, lt=15) sepal_width:
float = Field(gt=0, lt=15) petal_length:
float = Field(gt=0, lt=15) petal_width:
float = Field(gt=0, lt=15) @app.get("/health")
def health():
return {"status": "ok"} @app.post("/predict")
def predict(data: IrisInput):
features = [[data.sepal_length, data.sepal_width,
data.petal_length, data.petal_width]]
probs = model.predict_proba(features)[0]
idx = int(probs.argmax())
return {"species": CLASS_NAMES[idx], "confidence": round(float(probs[idx]), 3)}

Run the server (uvicorn iris_api:app --reload) and send the measurements of two different flowers to see the result:

TERMINAL​
curl http://127.0.0.1:8000/health
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4,"petal_width":0.2}'
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":6.7,"sepal_width":3.0,"petal_length":5.2,"petal_width":2.3}'
OUTPUT​
{"status":"ok"}
{"species":"setosa","confidence":1.0}
{"species":"virginica","confidence":1.0}

35.4 Input Validation with Pydantic​

If someone sends -1 cm or "abc" , the model will get bad data and the API will give a strange answer without telling you. Pydantic solves this problem: in the IrisInput class you state what each field should look like, and FastAPI checks it before it reaches the model .

Part of PydanticWhat it does
class IrisInput(BaseModel):Defines the**structure (schema)**of the input
sepal_length: floatThe type of this field must befloat("abc" will not work)
Field(gt=0, lt=15)The value must begreater than 0andless than 15(gt = greater than, lt = less
than)
Field is missingAutomatically gives a422error and a clear explanation

Now send wrong input and see (a negative value in the first, and a missing field in the second):

TERMINAL​
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":-1,"sepal_width":3.5,"petal_length":1.4,"petal_width":0.2}'
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" \
-d '{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4}'
OUTPUT​
{"detail":[{"type":"greater_than","loc":["body","sepal_length"],
"msg":"Input should be greater than 0","input":-1,"ctx":{"gt":0.0}}]}
{"detail":[{"type":"missing","loc":["body","petal_width"],"msg":"Field required",
"input":{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4}}]}

(The real response comes on a single line; it is broken into two lines here to make it readable.) Both times the status code was 422 (the server log shows 422 Unprocessable Entity), and loc tells you which field has the error. This data never reached the model. This is the benefit of validation : cheap, fast and safe .

35.5 Testing with Swagger Docs (/docs)​

FastAPI's loveliest feature: a testing page is made automatically for your API. While the server is running, open this in the browser: http://127.0.0.1:8000/docs. This page is called Swagger UI .

  1. Open http://127.0.0.1:8000/docs in the browser. You will see the list of all your endpoints (/health, /predict).

  2. Click on POST /predict , then press Try it out .

  3. Write your own JSON in the request body box (or change the values in the given example).

  4. Press Execute . Below, in the Response body , you will see the answer and the Code (200 or 422) .

  5. In the Schema section your IrisInput is also shown, which means the documentation is ready for others.

AddressWhat it is
/docsSwagger UI: interactive testing page
/redocAnother documentation page that is easy to read
/openapi.jsonThe complete map of the API in a machine-readable form

35.6 Automatic Testing: TestClient and pytest​

Clicking by hand in /docs every time is tiring, and in CI/CD (Chapter 38) this work must happen automatically. This is why we write tests in code . FastAPI's TestClient calls the API without running a server. The file name is test_iris_api.py:

test_iris_api.py

from fastapi.testclient import TestClient
from iris_api import app
client = TestClient(app)
GOOD = {"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}
def test_health():
res = client.get("/health") assert res.status_code == 200 assert res.json() == {"status": "ok"}
def test_predict_ok():
res = client.post("/predict", json=GOOD) assert res.status_code == 200 assert res.json()["species"] == "setosa"
def test_predict_negative_value():
res = client.post("/predict", json={**GOOD, "sepal_length": -1}) assert res.status_code == 422
def test_predict_missing_field():
bad = {k: v for k, v in GOOD.items() if k != "petal_width"}
res = client.post("/predict", json=bad) assert res.status_code == 422
TERMINAL​
pytest -v
OUTPUT​
test_iris_api.py::test_health PASSED [ 25%]
test_iris_api.py::test_predict_ok PASSED [ 50%]
test_iris_api.py::test_predict_negative_value PASSED [ 75%]
test_iris_api.py::test_predict_missing_field PASSED [100%]
========================= 4 passed, 1 warning in 1.56s =========================

35.7 Small Good Habits for Production​

HabitWhy
Keep a /health endpointDocker, load balancers and monitoring use it to learn whether the API is alive
Also define the format of theresponsewith
i
The response will always stay the same shape and documented
response_model / Pydantic
Load the modelonceLoading on every request makes the API slow
Setlimitson values (gt, lt)So that useless input never reaches the model
Write testsThe fear of the API breaking after every change goes away

35.8 Common Mistakes and Hands-on​

Hands-on​

  • Save the model of one of your old projects (for example Titanic or Iris) with joblib and build its /predict API .

  • Change the limits of Field(gt=..., lt=...) and see which values give 422 .

  • Open /docs and make 3 different predictions through Swagger.

  • Write one more test in test_iris_api.py: make the test fail when the response of /health is wrong (change the response on purpose and see).

KEY TAKEAWAYS​
  • An API is the model's waiter : it takes the request to the model and brings back a JSON response .

  • GET is for getting information, POST is for sending data and getting a prediction.

  • In FastAPI, an endpoint is made with the decorators @app.get(...) / @app.post(...), and Uvicorn runs the server.

  • Load the model once , and serve it through /predict; Pydantic stops wrong input with a 422 .

  • Testing is easy in /docs (Swagger), and TestClient + pytest make testing automatic.