Chapter 39: Logging & Monitoring
MODULE 6 | MLOps
Chapter 39: Logging & Monitoring
Model deploy ho gaya, API chal rahi hai, CI hara hai. Kaam khatam? Nahi. Asal imtihaan ab shuru hota hai: live model par logon ka asli data aata hai, aur wo kabhi bhi dheere dheere kharab ho sakta hai. Is chapter me hum seekhte hain ke app ke andar ki khabar (logging) kaise rakhein, aur model ki sehat (monitoring) kaise dekhein.
39.1 Logging kyun zaroori hai?
39.2 Python logging module
logging Python ke sath pehle se aata hai, kuch install nahi karna. Pehle ye chhota sa code chalate hain:
logging_demo.py
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
datefmt="%H:%M:%S",
)
logger = logging.getLogger("demo")
logger.debug("Yeh debug message nazar nahi aayega (level INFO hai)")
logger.info("Model load ho gaya")
logger.warning("Input me ek value missing thi, default use ki")
logger.error("Prediction fail ho gayi")
try:
1 / 0
except ZeroDivisionError:
logger.exception("Calculation me masla")
OUTPUT (waqt aur file ka rasta tumhare yahan alag honge) 07:12:40 | INFO | demo | Model load ho gaya 07:12:40 | WARNING | demo | Input me ek value missing thi, default use ki 07:12:40 | ERROR | demo | Prediction fail ho gayi 07:12:40 | ERROR | demo | Calculation me masla Traceback (most recent call last): File "logging_demo.py", line 16, in <module> 1 / 0 ~~^~~ ZeroDivisionError: division by zero
Dhyan do: DEBUG wali line nazar nahi aayi , kyunke humne level=logging.INFO rakha. Level ka matlab hai: "is darje ya us se oopar ke messages dikhao". logger.exception(...) except ke andar chalao: ye error ke sath poori Traceback bhi likhta hai.
| Level | Kab use karein | Misaal |
|---|---|---|
| DEBUG | Development metafseelimaloomat | "Input vector ki shape (1, 18)" |
| INFO | Normal,achhikhabar | "Model load ho gaya", "Prediction: Dengue" |
| WARNING | Kuch ajeeb hai, lekin appchal rahihai | "Confidence bohat kam", "Unknown symptom" |
| ERROR | Koi kaamfailhua | "Prediction fail ho gayi" |
| CRITICAL | App ya system kegirneka khatra | "Model file mili hi nahi" |
| Format ka tukra | Kya likhta hai |
|---|---|
%(asctime)s | Waqt(date aur time) |
%(levelname)s | Level ka naam (INFO, ERROR...) |
%(name)s | Logger kanaam(getLogger("demo") wala) |
%(message)s | Tumhara likha huamessage |
39.3 Logs ko file me likhna
Screen ki logs band hote hi ghayab ho jati hain. Isliye production me unhen file me likhte hain. Sirf filename= dene se kaam ho jata hai:
file_logging.py
import logging
logging.basicConfig(
filename="app.log",
level=logging.INFO,
format="%(asctime)s | %(levelname)s | %(message)s",
)
logging.info("Server shuru hua")
logging.info("Prediction: Dengue | 11.4 ms")
logging.warning("Confidence bohat kam: 0.31")
print("Log file me likh diya")
TERMINAL type app.log # Windows cat app.log # Mac / Linux OUTPUT (date aur waqt tumhare yahan alag honge) 2026-10-22 07:21:23,960 | INFO | Server shuru hua 2026-10-22 07:21:23,960 | INFO | Prediction: Dengue | 11.4 ms 2026-10-22 07:21:23,960 | WARNING | Confidence bohat kam: 0.31
39.4 FastAPI app me logging
Hamari Medical Assistant API (Chapter 40) me har prediction par ye logging hoti hai (app/main.py ka hissa):
app/main.py (hissa)
import logging
import time logging.basicConfig( level=logging.INFO, format="%(asctime)s | %(levelname)s | %(name)s | %(message)s", )
logger = logging.getLogger("medical-assistant") @app.post("/predict", response_model=PredictResponse)
def predict(req: PredictRequest):
start = time.perf_counter()
unknown = [s for s in req.symptoms if s not in SYMPTOMS]
if unknown:
logger.warning("Unknown symptoms: %s", unknown) raise HTTPException(status_code=400, detail=f"Ye symptoms nahi milay: {unknown}") ...
ms = (time.perf_counter() - start) * 1000 logger.info("predict | symptoms=%d | top=%s | %.1f ms", len(req.symptoms), top3[0][0], ms)
Ab server chalate hue API ko ek sahi aur ek galat request bhejo. Terminal me ye asli log dikhti hai:
OUTPUT (Uvicorn ki apni INFO: lines ke sath mili hui)
2026-10-22 07:11:24,743 | INFO | medical-assistant | predict | symptoms=4 | top=Dengue |
11.4 ms
INFO: 127.0.0.1:42682 - "POST /predict HTTP/1.1" 200 OK
2026-10-22 07:11:24,751 | WARNING | medical-assistant | Unknown symptoms: ['xyz']
INFO: 127.0.0.1:42686 - "POST /predict HTTP/1.1" 400 Bad Request
Ek hi nazar me pata chal gaya: kitni symptoms aayin, model ne kya bataya, kitna waqt laga , aur galat input par WARNING . Ye hi monitoring ka kaccha maal hai.
ETHICS / ZIMMEDARI
Medical app me log likhte waqt user ki nijji maloomat ka khayal rakho. Humne log me sirf symptoms ki ginti aur top prediction likhi, na ke user ka naam, number ya poori symptoms ki list. Health data bohat hassas hota hai. Logs me kabhi password, API key, naam, phone number ya ID mat likho, aur sirf utna likho jitna masla dhoondne ke liye zaroori ho.
39.5 Model Performance Monitoring
Monitoring ka matlab hai live model ki sehat par nazar rakhna . Ye teen tarah ki hoti hai:
| Qism | Kya dekhte hain | Misaal | Kitni aasani se |
|---|---|---|---|
| System | API zinda hai? Jawab ka waqt (latency), error ki ginti | Latency 11.4 ms, 400/500 errors | Asaan, abhi se |
| Data | Input waisa hi hai jaisa training me tha? | Umar ka average 35 se 48 hua | Darmiyani(drift check) |
| Qism | Kya dekhte hain | Misaal | Kitni aasani se |
|---|---|---|---|
| Model | Prediction sahi hain? (accuracy, precision...) | Aakhri 5 me se 3 sahi | Mushkil: asal jawab (ground truth) der se milta hai |
Model ki accuracy naapne ke liye sahi jawab (ground truth) chahiye. Maslan loan model me asal nateeja mahino baad pata chalta hai. Jab asal jawab milta rahe , to ek chalti hui accuracy (sirf aakhri N predictions par) ginte hain. Ye chhota sa monitor dekho:
accuracy_monitor.py
from collections import deque
class AccuracyMonitor:
"""Aakhri N predictions me se kitni sahi thin, wo track karta hai."""
def __init__(self, window=5, threshold=0.8):
self.results = deque(maxlen=window)
self.threshold = threshold
def add(self, predicted, actual):
self.results.append(predicted == actual)
def accuracy(self):
return sum(self.results) / len(self.results)
def check(self):
acc = self.accuracy()
if len(self.results) == self.results.maxlen and acc < self.threshold:
return f"ALERT: accuracy {acc:.0%} threshold {self.threshold:.0%} se neeche hai"
return f"OK: accuracy {acc:.0%}"
monitor = AccuracyMonitor(window=5, threshold=0.8)
pairs = [("A", "A"), ("B", "B"), ("A", "A"), ("C", "C"), ("B", "B"),
("A", "B"), ("C", "A"), ("B", "B")]
for pred, actual in pairs:
monitor.add(pred, actual)
print(f"pred={pred} actual={actual} -> {monitor.check()}")
OUTPUT
pred=A
actual=A -> OK:
accuracy 100%
pred=B
actual=B -> OK:
accuracy 100%
pred=A
actual=A -> OK:
accuracy 100%
pred=C
actual=C -> OK:
accuracy 100%
pred=B
actual=B -> OK:
accuracy 100%
pred=A
actual=B -> OK:
accuracy 80%
pred=C
actual=A -> ALERT:
accuracy 60% threshold 80% se neeche hai
pred=B
actual=B -> ALERT:
accuracy 60% threshold 80% se neeche hai
deque(maxlen=5) sirf aakhri 5 nateeje yaad rakhta hai; naya aaye to sab se purana khud nikal jata hai. Jaise hi accuracy threshold (80%) se neeche gayi, ALERT mil gaya. Asli system me ye alert email ya message bhejta hai.
39.6 Data Drift ka basic idea
drift_check.py
import numpy as np
from scipy.stats import ks_2samp
rng = np.random.default_rng(42)
# Training ke waqt umar ka data
train_age = rng.normal(loc=35, scale=8, size=1000)
# Live data 1: waisa hi
live_ok = rng.normal(loc=35, scale=8, size=500)
# Live data 2:
log ab zyada boorhay (drift)
live_drift = rng.normal(loc=48, scale=8, size=500)
for name, live in [("live_ok", live_ok), ("live_drift", live_drift)]:
stat, p = ks_2samp(train_age, live)
status = "DRIFT!"
if p < 0.05
else "theek"
print(f"{name:11s} | train mean={train_age.mean():.1f} live mean={live.mean():.1f} "
f"| KS={stat:.3f} p={p:.4f} -> {status}")
OUTPUT
live_ok | train mean=34.8 live mean=35.0 | KS=0.038 p=0.7169 -> theek
live_drift | train mean=34.8 live mean=46.7 | KS=0.571 p=0.0000 -> DRIFT!
| Nateeja | Matlab |
|---|---|
KS | Dono taqseemon mekitna farqhai (0 = bilkul ek jaisi, 1 = bilkul alag) |
p | Agarkoi asal farq na hotato itna farqdekhne ka imkaan.p chhota (< 0.05)= farq sach me hai |
live_ok | p = 0.72,farq ka koi saboot nahi, sab theek |
live_drift | p = 0.0000,bohat wazeh farq: drift mil gaya |
SAB SE ZAROORI
p < 0.05 ek rehnuma hai, hukm nahi. Bahut bare data par chhota sa farq bhi "significant" nikal aata hai, aur chhote data par bara farq bhi chhup sakta hai. Isliye KS ki qeemat aur asal averages (mean 35 vs 47) dono dekho , aur har feature par alag check chalao. Asli projects me Evidently jese open-source tools ye kaam ek sath kar dete hain.
39.7 Drift ya girti performance milne par kya karein?
-
Pehle tasdeeq karo: kya data me bug hai (galat column, missing values) ya sach me farq aaya hai?
-
Naya data jama karo jo ab ki haqeeqat dikhata ho.
-
Model dobara train karo aur MLflow (Chapter 37) me naya run log karo; purane se compare karo.
-
Behtar ho to registry me naya version banao aur
championalias us par lagao. -
CI/CD (Chapter 38) se naya model test aur deploy karo.
-
Monitoring dobara dekhte raho. Chakkar jari rehta hai (Chapter 34 ka lifecycle).
Dekha? Is module ke saare chapter ek chakkar me jur gaye: FastAPI (serve), Docker (package), MLflow (tracking), CI/CD (automation), Logging & Monitoring (nigrani), aur phir dobara training.
39.8 Monitoring ke aage ke tools
| Tool | Kaam |
|---|---|
| Prometheus + Grafana | API ke numbers (requests, latency, errors) jama karna aurdashboardpar graph dikhana |
| Evidently | Data drift aur model quality kitayyar reports(Python library) |
| Cloud monitoring(AWS CloudWatch, Azure Monitor) | Cloud par chalti apps ki logs aur alerts (Big Data & Cloud wale module me) |
39.9 Aam Galtiyan aur Hands-on
AAM GALTIYAN
- Production me **
print()
** par bharosa karna.
-
Har cheez DEBUG me likh kar logs ka dher laga dena (kaam ki baat dhoondna mushkil).
-
Logs me nijji maloomat ya passwords likhna.
-
Deploy kar ke monitoring bhool jana ; model chupke se kharab hota rehta hai.
-
Sirf accuracy dekhna aur latency, errors aur input ka data na dekhna.
-
Drift test ki p-value par andha bharosa karna.
Hands-on
-
Chapter 35 ki
iris_api.pymelogginglagao: har/predictpar species aur latency likho. -
Logs ko ek file (
app.log) me likhwao aur kholo. -
AccuracyMonitorkiwindowaurthresholdbadal kar ALERT ka waqt badlo. -
drift_check.pyme live data kalocko 35 se 38 karo aur dekho kis qeemat par DRIFT nazar aana shuru hota hai. -
Likho: tumhare kisi purane project me kaun sa feature drift ho sakta hai aur kyun?
KEY TAKEAWAYS
-
Logging app ka black box hai: waqt, level aur message ke sath kya hua likhti hai.
-
Levels: DEBUG < INFO < WARNING < ERROR < CRITICAL ;
basicConfigse level, format aur file tay hota hai. -
Monitoring teen jagah dekhti hai: system (latency, errors), data (drift), model (accuracy).
-
Data drift = live data training data se alag ho gaya; KS test se ek feature par check kar sakte hain.
-
Drift milne par: tasdeeq, naya data, dobara training, MLflow, CI/CD , aur monitoring jari.
-
Medical jese masalon me logs me nijji data kam se kam likho.