AI Deployment aur MLOps interview questions & answers

Socho aap apni ghar ki chaabi ek ajnabi ko de rahe ho aur keh rahe ho "mere liye ghar chalao, main wahaan nahi rahunga, aur mujhe andar dikhega bhi nahi." Aap seedhe chaabi nahi de doge -- pehle aap khud se us ajnabi ke ghar me rehne ke baare me chaar promises loge: lights aur paani wahi tarike se kaam karenge jaise tab karte the jab aap wahaan rehte the, ajnabi thermostat badal sakega bina deewaar todkar rebuild kiye, aap ghar ka haal check kar sakoge bina wahaan khade hue, aur agar kuch galat ho jaaye toh aap apna ghar wapas us hi tarah paa sakoge. Deployment bilkul yahi handover hai, bas ghar aapka code hai aur ajnabi ek machine hai jise aap shaayad kabhi naam se login karke nahi dekhoge.

Ch10 ne app ko aapke machine pe chalaya, aur us architecture ka sketch diya jismein ye badhta hai. Ye topic "ye mere machine pe chalta hai" sentence ke baad ki har cheez hai -- wo moment jab wahi code kahin chalna shuru karta hai jahaan aap dekh nahi rahe, un logon ke liye jinse aap kabhi mile hi nahi.

Production me, "ye kaam kar raha hai" ka matlab "maine click kiya aur ek baar chal gaya" hona band ho jaata hai, aur uska matlab ban jaata hai ki aap chaar alag sawaalon ka jawaab de sako, us code ke baare me jo aapko ab dikhta nahi, us user ke liye jisse aap kabhi milenge nahi, us machine pe jo aapki desk ke neeche wali nahi hai:

1. Har jagah same chalta hai -- image aur uski pinned dependencies aapke machine, CI, aur production me identical hain. Koi "mere machine pe chalta hai" wala gap nahi, kyunki koi alag machine-specific setup step bacha hi nahi jise skip kiya ja sake.
2. Bina rebuild ke configure hota hai -- database, provider key, ya environment badalna ek environment-variable change hai, naya image nahi. Agar ek setting badalne ke liye code edit karke naya build ship karna pade, toh wo configuration nahi hai, wo ek chhupa hua redeploy hai.
3. Chalte waqt visible hai -- logs, metrics aur cost aapko batate hain ki wo abhi kya kar raha hai, kyunki aap jaake screen pe dekh nahi sakte. Ek deployed process jise aap observe nahi kar sakte, wo "chal" nahi raha, wo bas "kahin bahar" hai.
4. Reversible hai -- aap exact pichhla version naam le sakte ho aur minutes me wahaan wapas ja sakte ho. Agar aap ye nahi bata sakte ki live pe konsa version hai, toh aap use rollback nahi kar sakte, aur rollback wo ek fix hai jo tab bhi kaam karta hai jab baaki sab kuch jal raha ho.

🌍 Real-world example: ek FastAPI chat endpoint apna provider key os.environ["PROVIDER_API_KEY"] se request handler ke andar padhta hai. Aapke laptop pe ye hamesha kaam karta hai, kyunki aapke laptop ke shell profile ne hamesha ye variable export kiya hai -- aapne ise mahino pehle ek baar set kiya tha aur bhool gaye. Naye machine pe bhi health check pass ho jaata hai, kyunki health check kabhi us line ko chhuta hi nahi. Pehle real user ka chat message hi wo pehla moment hai jab naye machine pe koi cheez us variable ko genuinely padhti hai, aur wo wahaan hai hi nahi. Container dikhta deployed hai, healthy hai, aur toota hua hai -- ek saath -- aur ye pata lagane ke liye ek paying user lag gaya.

Ek AI app ek paanchvi promise add karta hai jo kisi CRUD app ko kabhi karni hi nahi padi: model khud ek remote dependency hai jise aap own nahi karte. Uski latency, uski availability aur uska exact version kisi aur ke decisions hain, unke apne schedule pe liye gaye, aur aapka deployment in teeno ke bina warning badalne ke baad bhi kaam karta rehna chahiye -- provider ka ek slow din, ek maintenance window jiske baare me aapko bataya hi nahi gaya, ek model version jo chupke se naye se badal diya gaya. Upar wale chaaron promises me se koi bhi isse aapko protect nahi karta; yahi wajah hai ki ye chapter baar-baar "provider down / slow / different hai" ko ek edge case nahi, ek first-class failure maankar chalta hai.

💡 The four promises (chaar promises) = har jagah same chalta hai, bina rebuild ke configure hota hai, chalte waqt visible hai, reversible hai. Is chapter ka baad ka har topic in chaaron me se ek hai -- jab koi topic kahe "ye promise 2 hai", uska yehi matlab hai.

💡 Remote dependency jise aap own nahi karte = ek service (yahaan, model provider) jiski latency, uptime aur version aap control, patch ya guarantee nahi kar sakte -- sirf monitor kar sakte ho aur ek fallback rakh sakte ho.

💡 Deploy artefact = wo frozen cheez jo actually ship hoti hai -- ek image, pinned dependencies ka ek set -- "repo" ke ulat, jo source hai jise aap edit karte ho, wo cheez nahi jo chal rahi hai.

Jab code aapke laptop se nikalta hai toh actually kya tootta hai, concrete taur pe:

• Ek machine jo aapki nahi hai -- alag OS, alag CPU, ek library jo aapke laptop ke exact Python ke against compile hui thi aur kisi aur ke against nahi.
• Ek environment jo aapka nahi hai -- koi shell profile nahi, koi .env file nahi jo aap bhool gaye the ki wahaan hai, koi test files ka folder nahi jo aapne haath se banaya tha aur kabhi commit hi nahi kiya.
• Users jinhe aap dekh nahi sakte -- aap "abhi aapne kya click kiya" nahi pooch sakte jaise ek colleague ke kandhe se jhaank kar poochte ho; events ka sirf wahi version aapko milta hai jise aapne pehle se, record karne ka faisla kiya tha.
• Koi print() nahi -- wo debugging reflex jo laptop pe har jagah kaam karta hai, uske paas print karne ke liye koi jagah nahi hai. Agar aapne deploy se pehle logs wire nahi kiye, toh wo information request khatam hote hi gayab ho jaati hai, aisa nahi ki "kahin hai jahaan main baad me jaake dekh loonga".

Kab use karein ye framing: kisi bhi cheez ko doosre insaan tak pahunchne se pehle mentally chaar-promises checklist chala lo -- ek hiring manager ke liye demo, ek dost jo aapka project test kar raha hai, ek teammate jo aapki branch pull kar raha hai. Jis moment koi aapke alawa process start karta hai, chaaron apply hote hain, chaahe aapne use "deploy karna" bola ho ya nahi.

Kab NAHI use karein / Trade-off: ek script ke liye jise sirf aap khud, apne hi machine pe chalate ho, chaaron promises se guzarna pure overhead hai -- koi doosra observer nahi jiske liye configure karna ho, koi unseen user nahi jiske liye log karna ho, aapke alawa kisi aur ke liye rollback karne ko kuch nahi. Ye frame apni cost tabhi kamaata hai jab koi aapke alawa bhi is process ke chalte rehne me stakeholder ban jaata hai.

Standard definition (interview me bolo): Deployment is the set of guarantees that let code you can no longer directly observe keep working for users you will never meet: it must run identically across environments (pinned dependencies, a frozen artefact), be reconfigurable without a rebuild (externalised config and secrets), remain observable while running (logs, metrics, cost), and be reversible to a known-good prior version. An AI application adds a fifth: the model it calls is a remote dependency it does not own, whose latency, availability and version can change on the provider's schedule, not the deployer's.

"""
Same tiny "app" written two ways: the naive way reads an env var lazily,
inside the request handler -- the way almost every first FastAPI+AI app
is written. The validated way reads it once, at startup, through
pydantic-settings. Run on a machine where PROVIDER_API_KEY is NOT set
(simulating "the new machine" -- the laptop had it in a shell profile
nobody copied over).
"""
import os
from pydantic import Field
from pydantic_settings import BaseSettings

# make sure the var that only ever existed on "the laptop" is absent here
os.environ.pop("PROVIDER_API_KEY", None)


def naive_handle_request():
    """This is what most first FastAPI+AI apps look like: read the key
    right where it is used, inside the request handler."""
    api_key = os.environ["PROVIDER_API_KEY"]  # ambient laptop state
    return f"called provider with key starting {api_key[:3]}..."


class Settings(BaseSettings):
    provider_api_key: str = Field(min_length=8)


def run_naive():
    print("== naive: env var read lazily, inside the handler ==")
    print("  app import / startup : OK  (nothing touched the var yet)")
    try:
        print("  health check          : 200 OK (health check never reads it either)")
        result = naive_handle_request()
        print(f"  first request         : {result}")
    except KeyError as e:
        print(f"  FIRST REAL REQUEST     : 500 KeyError: {e}   <-- found in production, by a user")


def run_validated():
    print("== validated: typed settings, loaded once at startup ==")
    try:
        Settings()
        print("  startup                : OK")
    except Exception as e:
        first_line = str(e).splitlines()[0]
        print(f"  startup                : CRASH  {first_line}")
        print("  -> the process never becomes healthy, so the deploy never")
        print("     shifts traffic to it; the previous, working version keeps serving")


if __name__ == "__main__":
    run_naive()
    print()
    run_validated()

Socho ek poora apartment pack ho raha hai -- har furniture, already-tested wiring, har light bulb ka exact brand -- ek sealed shipping container ke andar, aur wahi same container har destination pe truck se bheja jaata hai. Koi bhi apartment ko pahunchne ke baad re-decorate nahi karta. Jo pahunchta hai wo exactly wahi hai jo pack kiya gaya tha, screw-by-screw, har ek stop pe.

In production, ek container image wahi packed apartment hai: ek frozen filesystem plus aapki app ke har pinned dependency, ek baar build hokar har machine pe unchanged ship hota hai jo use run karti hai. Aapka repo furniture ka catalogue hai aur use assemble karne ki instructions -- container banane ke liye useful, lekin production me koi catalogue run nahi karta; wo sealed container run karte hain. Yahi mental flip ek beginner ko karna padta hai: jo cheez ship hoti hai wo aapka code nahi hai, wo aapke code ne jo image banaayi hai wo hai.

🌍 Real-world example: ek team image build karti hai, use myapp:latest tag karti hai, aur deploy kar deti hai. Do hafte baad production me ek bug aata hai. Koi usi Dockerfile se rebuild karta hai use "reproduce" karne ke liye, us build ko bhi myapp:latest tag kar deta hai, aur ab koi nahi bata sakta in dono me se konsi actually chal rahi hai -- dono hi same naam ka jawab dete hain. latest kisi bhi specific, frozen bytes ke set ko point nahi karta; ye bas ek label hai jise kisi ka bhi agla docker build chupke se overwrite kar sakta hai. Outage me jo fix hamesha kaam karta hai -- "jo version kaam kar raha tha wapas laga do" -- use exactly jaanna padta hai konsi image thi, aur latest iska jawab nahi de sakta.

💡 Image = ek frozen filesystem snapshot: aapki app ka code plus har pinned dependency, ek baar build hoti hai, har jagah identically run hoti hai.

💡 Layer = image me ek instruction ka filesystem change (ek RUN, ek COPY); Docker har layer ko alag se cache karta hai aur sirf us instruction ko re-execute karta hai jiske inputs badle hon.

💡 .dockerignore = ek file jo batati hai ki build context me kya kabhi nahi jaayega -- .git, .env, __pycache__, local venvs -- .gitignore jaisa hi idea hai, bas isse ye decide hota hai ki image me kya bake hota hai.

💡 Non-root user = container ke andar process root ki jagah ek unprivileged user ke roop me chalti hai, taaki compromise hui process wahaan kuch bhi likh na sake jahaan use nahi likhna chahiye.

THE MEASUREMENT (naapa hua) -- layer order cosmetic kyun nahi hai

Is machine pe Docker install nahi hai (verified), toh is topic ke liye koi image build nahi hui. Jo real aur measured hai, isi course ke us venv me jahaan FastAPI backend chalta hai:

import torch     : 3.5 s warm / 10.2 s cold      torch (__pycache__ ke bina) : 462.4 MB
import fastapi   : 1 second se kaafi kam         fastapi                     :   0.8-1.4 MB
                                                  poora venv                  : ek gigabyte se zyada

Ek typical Python AI backend ki requirements.txt torch (ya transformers, jo isi pe depend karta hai) ko fastapi ke saath khinch laati hai. fastapi khud 1.5 MB se kam ka hai aur 1 second se kaafi kam me import ho jaata hai; akela torch disk pe fastapi se roughly 500x bada hai (~462 MB as shipped, ~514 MB __pycache__ bante hi) aur import hone me seconds lagte hain, milliseconds nahi -- measured 3.5 s warm filesystem cache ke saath aur 10.2 s cold, aur ek freshly-started container hamesha cold hota hai. Ye dependency set install karna image build karne ka sabse mehenga step hai -- aapke apne source ko copy karne se kaafi zyada mehenga, jo typically kuch sau KB ka hota hai.

Yahi wajah hai ki Dockerfile ki do lines ka order almost kisi bhi cheez se zyada matter karta hai: COPY requirements.txt . aur RUN pip install -r requirements.txt ko COPY . . (baaki source) se pehle rakhna. Docker image ko layer-by-layer banata hai, aur har layer ko uske inputs ke hisaab se cache karta hai. Agar requirements.txt nahi badla, toh pip install wali layer -- jisne wo ~462 MB, multi-second wala kaam kiya tha -- cache se hi reuse ho jaati hai, chaahe aapne aaj app.py ko dus baar edit kiya ho. Source ko requirements se pehle copy karo, aur har single source edit pip-install layer ko bhi invalidate kar dega, toh multi-second wali torch install ek line ke code change ke liye bhi har rebuild pe dobara chalegi. Same Dockerfile content, opposite build speed, sirf line order ki wajah se.

Dockerfile

Pinned base image, .dockerignore uske saath rakhi hui, dependencies source se pehle copy aur install ki hui, ek non-root user, aur wo port jisse process actually sunta hai EXPOSE se declare kiya hua. CMD wahi uvicorn command hai jo aap haath se chalate, bas ab container ka ek persistent process ban gayi hai.

Image me kabhi secret bake mat karo

Ek copy ki hui .env file, ya build ARG ke roop me pass kiya gaya secret, ek build layer ka hissa ban jaata hai -- aur ek layer tab delete nahi hoti jab baad ka koi instruction file ko delete karta hai. docker history (ya bas ek purani layer ko dobara extract karke) use recover kiya ja sakta hai, kyunki jitni bhi layers kabhi build hui thi wo sab image ke andar stored rehti hain, wo bhi jinhe baad ke RUN rm ne mitane ki koshish ki thi. Koi bhi jo image ko docker pull kar sakta hai -- ek teammate, ek compromise hua CI runner, registry read access wala koi bhi -- wo kabhi bake hue secret ko recover kar sakta hai, permanently, chaahe aapne use kitne bhi steps baad "delete" kyun na kiya ho. Secrets deploy time ke environment me rehne chahiye (ek secrets manager, CI-injected env vars), kabhi kisi COPY me nahi, kabhi kisi ARG me nahi, image me bilkul nahi. config-and-secrets (is chapter ka ek locked topic) poora pattern owns karta hai; yahaan ka rule zyada narrow aur absolute hai: jo cheez secret rehni chahiye wo kabhi kisi Dockerfile instruction me nahi aani chahiye.

Kab use karein: apni khud ki Dockerfile likho jab aapko layer order pe control chahiye (upar wale mehenge install step ko protect karne ke liye), ek non-root user, ek exact pinned base image, aur ek specific CMD -- ye har us real AI backend ke liye zaroori hai jisme torch ya transformers jaisa heavy dependency ho.

Kab NAHI use karein: ek chhoti, dependency-light service ke liye, platform ka auto-build path (ek buildpack jaisa Cloud Native Buildpacks ya Nixpacks) source se seedhe ek working image bana sakta hai, bina kisi Dockerfile ke. Trade-off control ka hai: aap iski layers ko reorder nahi kar sakte, iska base image choose nahi kar sakte, aur non-root user ko utni tarah guarantee nahi kar sakte jitna ek hand-written Dockerfile deta hai. Jis service ka build time ek sau-se-zyada-MB, das-second-import wale dependency se dominate hota hai, wahaan wo khoya hua control exactly wo control hai jiski aapko zaroorat hai -- Dockerfile likho.

Standard definition (interview me bolo): A container image is a frozen, immutable filesystem snapshot -- application code plus every pinned dependency -- built once from a Dockerfile and run unchanged on any host with a container runtime; it, not the repository, is the artefact that ships. A Dockerfile builds that image as an ordered sequence of cached layers, so instructions that change rarely (installing requirements.txt) belong before instructions that change often (copying application source), letting an unrelated code edit reuse an expensive dependency-install layer instead of re-running it. latest is a mutable tag, not a version, and cannot identify which bytes are actually running in production -- images must be tagged by an immutable identifier, typically the commit SHA, so that a rollback means redeploying a known-good tag rather than guessing.

Ek plain sentence isme kya chala aur kya nahi: neeche wali Dockerfile illustrative hai aur kabhi build nahi hui (is machine pe Docker install nahi hai); upar diye size aur import-time wale numbers real measurements hain jo isi course ke actual Python venv se liye gaye hain.

# ---- ILLUSTRATIVE (package not installed here; not executed) ----
FROM python:3.13.7-slim

WORKDIR /app

# system packages needed only to build wheels; the apt cache is removed in the
# SAME layer so it never inflates the image
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
    && rm -rf /var/lib/apt/lists/*

# requirements.txt copied BEFORE the source: this layer only re-runs pip install
# when requirements.txt itself changes, not on every source edit
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# source enters the image last -- editing app.py invalidates only this layer
# and everything below it, never the (expensive) install layer above
COPY . .

# never run the process as root inside the container
RUN useradd --create-home --shell /bin/bash appuser
USER appuser

EXPOSE 8000

CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Kya bana rahe hain: aapne Ch4 mein ek chhota sa RAG service banaya tha -- kuch pages ko chunk karna, embed karna, question ke liye sabse close chunks retrieve karna, aur unhe model ko grounded context ke roop mein dena. Ye project usi idea ko deployable banata hai: kisi bhi ajnabi ka laptop ise chala sake, ek galat config user tak pahunchne se pehle crash ho jaaye, ek orchestrator ye bata sake ki ye sach mein kaam kar raha hai ya nahi, aur har request ka ek trail bache jise aap measure kar sakein. Ye project FREE hai lekin is chapter ke topics 3-16 LOCKED hain -- maan lijiye aapne unmein se koi nahi dekha. Neeche har ## Step N apna idea khud se dobara derive karta hai; yahan kuch bhi kisi aise topic par depend nahi karta jo aapne nahi padha.

🌍 Real-world example: us farak ko socho jo ek script ke beech hota hai jo tab kaam karta hai jab AAP use APNE machine par chalate ho, aur ek API ke beech jise koi hiring panel kal subah bina aapke paas khade hue browser se hit kar sake. In dono ke beech ka gap yahi poora project hai: pinned versions taaki ye do baar same tarah install ho, ek config jo aadha-toota shuru hone se mana kar de, "kya ye sach mein ready hai" poochne ka tareeka, logs jo baad mein batayein kya hua, ek number jo bataye ye kitna slow hai (feel nahi, real), aur ek container description jise koi bhi platform chala sake.

💡 deployable = koi ajnabi, jo aapka machine nahi hai us par, isse chala sake aur wahi behaviour paaye jo aapne paaya -- bina aapki terminal history padhe ki aapke paas kya install tha. 💡 compose = ek YAML file jo kai containers (yahan: API aur ek cache) describe karti hai jo ek command se saath mein khade hote hain, aapas mein baat karne ke liye wired.

Standard definition (interview me bolo): deploying a RAG service means pinning what it depends on, validating its configuration before it accepts traffic, proving it is ready rather than merely running, emitting logs and a metrics summary that describe what actually happened, and packaging it as a container artefact plus a deploy checklist any platform can follow -- the same four promises (runs the same everywhere, configured without a rebuild, visible while running, reversible) applied to one concrete app.

⚠ Ek seedha sentence ki kya chala aur kya nahi: neeche ke har step, Step 7 (Dockerfile aur compose file) ko chhodkar, is machine par ek real uvicorn server ke against sach mein chala -- ek deterministic fake retriever aur fake model ke saath, taaki kisi API key ki zaroorat na pade. Docker khud is machine par install nahi hai, isliye Dockerfile aur compose file illustrative artefacts hain -- lekin compose file ko phir bhi yaml.safe_load se sach mein parse kiya gaya taaki ye prove ho ki ye valid YAML hai, aur uski parsed service list ko genuine output ke roop mein print kiya gaya.

Step 1 -- Dependencies pin karna (ek floating version promise 1 todta hai)

fastapi==0.141.1
uvicorn==0.52.0
pydantic==2.13.4
pydantic-settings==2.14.2
httpx==0.28.1

Kyun requirements.txt mein plain fastapi likhna, app code ki ek line likhne se pehle hi, ek bug hai: deployment ka promise 1 hai "har jagah same chalta hai." Ek unpinned fastapi us din ka LATEST release install karta hai jis din image build hoti hai -- aaj, agle hafte, aur jis din koi teammate rebuild kare, teeno alag versions ke teen alag installs hain, aur file mein kahin nahi likha kaunsa aapne test kiya tha. Ek minor bump kisi parameter ka naam badal sakta hai, koi default badal sakta hai, ya koi deprecated code path aapke neeche se hata sakta hai, aur ye chupke se hota hai -- build fail nahi hoti, bas alag code install karti hai aur usi requirements file ko wahi bata deti hai. Har direct dependency ko us exact version par pin karo jispar aapne test kiya (==, >= nahi), aur pin file ko jaan-boojhkar regenerate karo (ek naya PR, review kiya hua) na ki har rebuild par drift hone do. Upar ke versions wahi hain jinke against ye project is machine par sach mein chalaya gaya.

Step 2 -- Typed settings, ek SecretStr provider key ke saath, startup par validate

from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    provider_api_key: SecretStr = Field(min_length=8)
    model_name: str = "small-chat-model"
    request_timeout_s: float = 12.5

settings = Settings()   # env vars padhta hai; kuch bhi missing ya malformed ho toh turant raise karta hai

Ye IMPORT time par kyun chalna chahiye, na ki kisi request handler ke andar: agar aap os.environ["PROVIDER_API_KEY"] ko lazily, us endpoint ke andar padhte ho jise iski zaroorat hai, toh container theek se shuru hota hai, uska health check theek se pass hota hai, aur missing variable aapke pehle real user ko 500 ke roop mein milta hai. Isse pydantic-settings class mein module load time par padhna palat deta hai: ek missing ya bahut chhoti key turant raise karti hai isse pehle ki app ek bhi request accept kar sake, toh ek toote hue config ka rollout kabhi "successful" bana hi nahi -- ye crash hota hai, platform pichla working version live rakhta hai, aur kisi bhi user ki request us gap ko kabhi dekhti nahi. Startup par crash hona ek bug nahi, ek feature hai.

Is machine par verified (real run, key ka crash prove karne ke liye API key ki zaroorat nahi):

== missing key entirely ==
CONTAINER START -> CRASH: ('provider_api_key',) - Field required

== key too short (fails min_length=8) ==
CONTAINER START -> CRASH: ('provider_api_key',) - Value should have at least 8 items after validation, not 5

Dono exactly wahi outcome hain jo aap chahte ho: process traffic serve karne ke bajaye upar aane se mana kar deta hai jab config kisi ne check hi nahi kiya.

SecretStr key ko un jagahon se leak hone se bhi rokta hai jinhe log bhool jaate hain -- settings object print karna, use kisi error report mein dump kar dena, galti se log kar dena:

print(settings)      -> provider_api_key=SecretStr('**********') model_name='small-chat-model' request_timeout_s=12.5
f-string of field     -> **********
model_dump()          -> {'provider_api_key': SecretStr('**********'), ...}
model_dump_json()     -> {"provider_api_key":"**********", ...}
explicit read         -> sk-not...     (sirf .get_secret_value() se)

Ek plain str field in sabhi lines mein asli key print kar deta. SecretStr SAFE cheez ko default banata hai aur aapko get_secret_value() likhne par majboor karta hai sirf us ek jagah jahan sach mein raw value ki zaroorat hai -- aur yahi sabse badi wajah hai ki ek real key kabhi kisi deploy command mein literal ke roop mein nahi honi chahiye: ek command shell history aur CI logs mein chala jaata hai, SecretStr kisi aisi cheez ko redact nahi kar sakta jo kabhi usse hokar gayi hi nahi.

Step 3 -- lifespan startup jo index ek baar load karta hai, aur load hone tak NOT READY report karta hai

STATE = {"ready": False, "chunks": None}

async def load_index_in_background():
    # ek real embedding-model / vector-index load sach mein seconds le sakta hai
    await asyncio.sleep(0.5)
    STATE["chunks"] = build_chunks()
    STATE["ready"] = True

@asynccontextmanager
async def lifespan(app: FastAPI):
    asyncio.create_task(load_index_in_background())   # isse await MAT karo -- neeche dekho
    yield
    STATE["ready"] = False

app = FastAPI(lifespan=lifespan)

lifespan kyun, aur purana @app.on_event("startup") kyun nahi: on_event deprecated hai -- lifespan process start aur process stop par ek baar code chalane ka current, supported tareeka hai, aur yahi wo jagah hai jahan aap cleanup bhi kar sakte ho (clients close karna, background work cancel karna) usi jagah jahan aapne setup kiya tha. Load ek background task kyun hai, na ki koi cheez jiske liye startup step wait kare: agar load_index_in_background() ko lifespan ke andar seedha await kiya jaata, toh poora ASGI server KOI bhi connection accept nahi karta -- /health bhi nahi -- jab tak load khatam na ho. Ek chhote demo ke liye ye invisible hai; ek real embedding model ke liye wo load sach mein seconds le sakta hai, aur us poori window mein container ka port jawaab hi nahi deta, jo "not ready" bolne se bhi bura hai -- ek orchestrator "abhi shuru ho raha hai" aur "mar gaya" mein farak nahi bata sakta agar kuch sun hi nahi raha. Load ko background task ke roop mein fire karne se server turant upar aa jaata hai aur imaandari se jawaab deta hai jabki asli kaam uske peeche ho raha hota hai.

Step 4 -- /health (liveness) aur /ready (readiness) -- do alag sawaal

@app.get("/health")
def health():
    return {"status": "alive"}          # "kya ye process fasa hua hai? agar haan toh restart karo."

@app.get("/ready")
def ready():
    if not STATE["ready"]:
        return JSONResponse(status_code=503, content={"status": "not ready", "reason": "index still loading"})
    return {"status": "ready"}          # "kya is instance ko abhi traffic bhejna chahiye?"

Ek dependency check liveness ke andar KABHI kyun nahi jaana chahiye: liveness ka jawaab hai "agar main fail karun toh restart karo," aur restart us problem ko fix nahi kar sakta jo kisi doosre machine par rehti hai. Agar /health khud vector DB ya model provider ko check karta aur jab bhi koi bhi down ho 503 return karta, orchestrator ek bilkul healthy process ko baar-baar maar kar restart karta -- har instance restart-loop karta hai jabki asli fault kahin aur baitha hota hai. Readiness isi check ke liye sahi jagah hai, kyunki uska jawaab ("abhi traffic mat bhejo") exactly wahi sahi response hai jab koi remote dependency down ho, ya index abhi load ho raha ho.

Is machine par verified (real uvicorn, start karne se pehle port free hone ka assert):

== liveness vs readiness, right after startup (index still loading) ==
/health -> 200 {'status': 'alive'}
/ready  -> 503 {'status': 'not ready', 'reason': 'index still loading'}

== /ready once the index finished loading ==
/ready  -> 200 {'status': 'ready'}

Process poore waqt alive tha (/health kabhi neeche nahi gaya); ye traffic ke laayak sirf tab bana jab index exist karne laga. Yahi gap hai jiske liye ek startup probe hoti hai.

Step 5 -- Structured JSON logs, ek request id, model, tokens, latency aur cache status ke saath

request_id = str(uuid.uuid4())
log_entry = {
    "request_id": request_id, "model": settings.model_name,
    "tokens_in": tokens_in, "tokens_out": tokens_out,
    "latency_ms": round(latency_ms, 2), "cache_status": cache_status, "status": "ok",
}
print("LOG", json.dumps(log_entry))

Ek dict kyun, ek sentence kyun nahi: f"user ne kuch poocha, thoda time laga" ek baar padhne mein readable hai aur hamesha ke liye unqueryable -- aap "kal MISS requests ka mera p95 latency kya tha" ye ek dher saare prose se nahi puch sakte. Ek structured entry wo hai jise ek real log aggregator (ye project bas print karta hai; ek production wala kahin bhej deta hai) filter, group aur aggregate kar sakta hai. request_id wo hai jo aapko ek hi request ke sabhi lines dhundhne deta hai, ek service mein jo ek saath saikdon request ka jawaab de rahi ho -- iske bina, do concurrent requests ki log lines mein farak nahi kiya ja sakta. Is run se verified real output, har request ki ek line:

LOG {"request_id": "74fa2c73-...", "model": "small-chat-model", "tokens_in": 7, "tokens_out": 17, "latency_ms": 0.18, "cache_status": "MISS", "status": "ok"}
LOG {"request_id": "322f42e8-...", "model": "small-chat-model", "tokens_in": 7, "tokens_out": 17, "latency_ms": 0.03, "cache_status": "HIT",  "status": "ok"}
LOG {"request_id": "9faed588-...", "model": "small-chat-model", "tokens_in": 0, "tokens_out": 0,  "latency_ms": 0.01, "cache_status": "N/A",  "status": "error"}

Raw key kabhi log mat karo -- notice settings.model_name log hota hai, settings.provider_api_key kabhi nahi; Step 2 ka SecretStr ek accidental logging.info("config=%s", settings) ko bhi khud redact kar deta hai.

Step 6 -- Ek summary/metrics endpoint: p50/p95/p99, error rate, cache hit rate

def percentile(values, p):
    s = sorted(values)
    return s[min(int(len(s) * p), len(s) - 1)] if s else 0.0

@app.get("/metrics")
def metrics():
    latencies = [e["latency_ms"] for e in REQUEST_LOG]
    errors = sum(1 for e in REQUEST_LOG if e["status"] == "error")
    hits = sum(1 for e in REQUEST_LOG if e["cache_status"] == "HIT")
    served = sum(1 for e in REQUEST_LOG if e["cache_status"] in ("HIT", "MISS"))
    return {
        "total_requests": len(REQUEST_LOG),
        "p50_ms": round(percentile(latencies, 0.50), 2),
        "p95_ms": round(percentile(latencies, 0.95), 2),
        "p99_ms": round(percentile(latencies, 0.99), 2),
        "error_rate": round(errors / len(REQUEST_LOG), 3),
        "cache_hit_rate": round(hits / served, 3) if served else 0.0,
    }

Percentiles kyun, average kyun nahi: ek LLM endpoint ki latency naturally long-tailed hoti hai -- zyaadatar requests fast hoti hain, koi slow generation ya cache miss chand ko bahut door khinch deta hai, aur unhe average karna kisi ko bhi describe nahi karta. p50 wo hai jo ek typical user ne mehsoos kiya; p95/p99 wo hai jo aapke sabse unlucky users ne mehsoos kiya, aur alert usi par lagana chahiye. Verified real output, is server se saat real requests ke baad (chaar MISS/HIT pairs aur ek jaan-boojhkar error):

{
  "total_requests": 7, "p50_ms": 0.03, "p95_ms": 0.18, "p99_ms": 0.18,
  "error_rate": 0.143, "cache_hit_rate": 0.5
}

error_rate (7 mein se 1, us request se jismein query nahi thi) aur cache_hit_rate (4 served requests mein se 2) exactly wo do aur numbers hain jo latency ke saath ek dashboard mein chahiye -- ye kitni baar fail hoti hai, aur ye kitni baar mehenga kaam karne se bach gayi.

Step 7 -- Ek Dockerfile + compose stack (ILLUSTRATIVE -- is machine par Docker install nahi hai)

# ---- ILLUSTRATIVE (package not installed here; not executed) ----
FROM python:3.13-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
RUN useradd --create-home appuser && chown -R appuser /app
USER appuser

EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

requirements.txt ko baaki source se PEHLE copy aur install kiya jaata hai: Docker har layer ko cache karta hai, toh aapka app code badalna (jo hardam hota hai) har dependency ko dobara install karne par majboor nahi karta (jo mushkil se badalti hai) -- sirf requirements.txt ko chhuna us layer ko invalidate karta hai. USER appuser ka matlab hai ek container escape attacker ko box ke andar root nahi de deta. Kuch bhi secret copy nahi hota; ek real key deploy time par environment variable ke roop mein inject ki jaati hai, kabhi ek layer mein bake nahi ki jaati -- ek COPY .env . key ko image mein hamesha ke liye readable chhod deta, chaahe file baad ki kisi layer se hata bhi di jaaye.

# ---- ILLUSTRATIVE (package not installed here; not executed) ----
services:
  api:
    build: .
    image: rag-api:${GIT_SHA:-dev}
    ports: ["8000:8000"]
    env_file: [".env"]
    depends_on:
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('http://localhost:8000/health')"]
      interval: 10s
      timeout: 3s
      retries: 3
  redis:
    image: redis:7-alpine
    volumes: ["redis_data:/data"]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
volumes:
  redis_data:

Notice kya missing hai: ek version: key. Ye ek purane Compose file format ka hissa thi; jis Compose Specification ko ye file follow karti hai wo isse use nahi karta, aur ek current Compose install ise wapas daalne par warning deta hai -- ise omit karo. depends_on ka shape notice karo: ek plain depends_on: [redis] sirf redis CONTAINER ke start hone ka wait karta hai, redis ke connections accept karne ke liye ready hone ka nahi -- api container upar aa sakta hai aur ek aise redis se connect karne ki koshish kar sakta hai jo abhi bhi boot ho raha ho. condition: service_healthy isse redis ke apne healthcheck ke pass hone tak wait karwata hai, jo asli guarantee thi jo aap chahte the. compose yahan exactly yahi matlab rakhta hai: ek file, do services, poore local stack ko saath khade karne ka ek command -- ye ek local/CI convenience hai, production orchestrator nahi.

Docker ise banane ke liye nahi chalaya gaya -- koi build log nahi hai, koi image size nahi hai, aur koi container sach mein start nahi hua. Jo sach mein chalaya gaya: isi exact compose file ko yaml.safe_load se parse karna, jo prove karta hai ki ye syntactically valid hai aur uska real structure print karne deta hai:

parsed OK, top-level keys: ['services', 'volumes']
services: ['api', 'redis']
api depends_on: {'redis': {'condition': 'service_healthy'}}
has version key: False

Step 8 -- Ek deploy checklist jo aap KISI bhi container platform tak le ja sakte ho

Jo bhi platform container chalaaye (koi managed container service, ek Kubernetes cluster, ek chhota VM), yahi checklist lagti hai -- ye deployment ke properties hain, kisi ek vendor ke product ke nahi:

1. Pinned dependencies -- requirements.txt mein har package ka ek exact version hai (Step 1).
2. Config startup par validate hoti hai -- koi missing ya malformed setting container ke healthy banne se pehle usse crash kar deti hai, kabhi kisi user se discover nahi hoti (Step 2).
3. Koi secret kahin bhi literal nahi hai -- na image mein, na repo mein, na deploy command ke argument list mein; ye deploy time par environment variable ke roop mein inject hota hai ya secret manager se padha jaata hai, aur SecretStr (ya iske jaisa kuch) use logs se bahar rakhta hai (Step 2).
4. Liveness aur readiness alag endpoints hain, aur sirf liveness process restart karta hai -- ek remote dependency ka down hona NOT READY dikhna chahiye, kabhi NOT ALIVE nahi (Step 4).
5. Har request traceable hai -- ek request id us request ki har log line se guzarta hai, aur log structured hai, prose nahi (Step 5).
6. Ek metrics summary exist karta hai jo percentiles report karta hai, sirf average nahi -- kam se kam p95/p99, error rate, cache hit rate (Step 6).
7. Image ko kisi aisi cheez se tag kiya jaata hai jispar aap wapas point kar sako (ek commit SHA), sirf latest se kabhi nahi -- agar aap ye nahi bata sakte ki kaunsi image chal rahi hai, toh aap rollback nahi kar sakte, aur rollback wo outage fix hai jo hamesha kaam karta hai.
8. Container ko support karne wala stack declare kiya jaata hai, yaad nahi rakha jaata -- ye kis par depend karta hai (ek cache, ek database), aur kaun kiske truly ready hone ka wait karta hai, sirf start hone ka nahi.

Inmein se koi bhi item kisi platform ka naam nahi leta. Checklist ka yahi point hai: ye same aath cheezein hain chahe container ek managed service par jaaye, ek cluster par, ya ek akela VM par -- platforms ke beech jo badalta hai wo hai aap deploy command ko unpar kaise point karte ho, ye nahi ki deployed cheez ka pehle se kya sach hona chahiye.

Ye project sach mein kya prove karta hai, aur kya nahi

Prove karta hai: ek deployable AI service ko chahiye har piece is machine par ek live server ke against sach mein bana aur chala -- pinned dependencies, ek config jo galat value par serve karne ke bajaye crash hoti hai, ek background-loaded index jiski ek imaandaar not-ready window hai, sahi tarike se jawaab diye gaye do alag health sawaal, ek request id aur token counts ke saath structured logs, aur real requests se real percentiles report karta ek metrics endpoint.

Nahi karta (interviewer ke dhoondhne se pehle khud bata do): model call ek deterministic fake hai, koi live provider nahi -- ise badalna upar ke wiring mein kuch nahi badalta. Is machine par koi real Redis, database, ya container runtime nahi hai, toh compose file aur Dockerfile illustrative shapes hain, sirf valid YAML ke roop mein verified, kabhi sach mein build ya run nahi hui. Aur yahan sab kuch ek single process hai -- Step 6 ka chapter sibling shared state par (workers-and-shared-state, locked) agla problem hai jo ye exact app worker ek se zyaada hote hi turant face karta hai.

import asyncio
import hashlib
import json
import time
import uuid
from collections import Counter
from contextlib import asynccontextmanager

from fastapi import FastAPI
from fastapi.responses import JSONResponse
from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings


# ---- Step 1: pinned deps live in requirements.txt (fastapi==0.141.1, uvicorn==0.52.0, ...) ----

# ---- Step 2: typed settings, SecretStr key, validated at import time ----
class Settings(BaseSettings):
    provider_api_key: SecretStr = Field(min_length=8)
    model_name: str = "small-chat-model"
    request_timeout_s: float = 12.5

settings = Settings()   # crashes here if PROVIDER_API_KEY is missing or too short


# ---- Step 3: lifespan loads the index in the background; not-ready until loaded ----
STATE = {"ready": False, "chunks": None}
PAGES = {
    "https://hirenix.in/docs/pricing": "Hirenix offers a free plan and a Pro plan...",
    "https://hirenix.in/docs/refunds": "Refunds are available within 7 days...",
}

def fake_embed(text):
    import re
    return Counter(re.findall(r"[a-z]+", text.lower()))

def build_chunks():
    return [{"text": t, "source": u, "vector": fake_embed(t)} for u, t in PAGES.items()]

async def load_index_in_background():
    await asyncio.sleep(0.5)          # stand-in for a real embedding-model / index load
    STATE["chunks"] = build_chunks()
    STATE["ready"] = True

@asynccontextmanager
async def lifespan(app: FastAPI):
    asyncio.create_task(load_index_in_background())   # NOT awaited -- server stays responsive
    yield
    STATE["ready"] = False

app = FastAPI(lifespan=lifespan)


# ---- Step 4: liveness vs readiness ----
@app.get("/health")
def health():
    return {"status": "alive"}

@app.get("/ready")
def ready():
    if not STATE["ready"]:
        return JSONResponse(status_code=503, content={"status": "not ready", "reason": "index still loading"})
    return {"status": "ready"}


REQUEST_LOG = []
CACHE = {}

def count_tokens(text):
    return len(text.split())

def cosine_sim(v1, v2):
    import math
    common = set(v1) & set(v2)
    dot = sum(v1[w] * v2[w] for w in common)
    n1 = math.sqrt(sum(v * v for v in v1.values()))
    n2 = math.sqrt(sum(v * v for v in v2.values()))
    return dot / (n1 * n2) if n1 and n2 else 0.0

def retrieve(query, k=2):
    qvec = fake_embed(query)
    scored = [(cosine_sim(qvec, c["vector"]), c) for c in STATE["chunks"]]
    scored.sort(key=lambda x: x[0], reverse=True)
    return scored[:k]


# ---- Step 5 + 6: structured logs (request id, tokens, latency, cache) + /metrics ----
@app.post("/ask")
def ask(payload: dict):
    request_id = str(uuid.uuid4())
    start = time.perf_counter()
    query = payload.get("query", "")

    key = hashlib.sha256(f"{settings.model_name}:{query}".encode()).hexdigest()
    cached = CACHE.get(key)
    if cached is not None:
        cache_status, answer = "HIT", cached["answer"]
        tokens_in, tokens_out = count_tokens(query), cached["tokens_out"]
    else:
        cache_status = "MISS"
        top = retrieve(query)
        answer = f"Based on {', '.join(c['source'] for _, c in top)}: grounded answer to '{query}'."
        tokens_in, tokens_out = count_tokens(query), count_tokens(answer)
        CACHE[key] = {"answer": answer, "tokens_out": tokens_out}

    latency_ms = (time.perf_counter() - start) * 1000
    entry = {"request_id": request_id, "model": settings.model_name, "tokens_in": tokens_in,
              "tokens_out": tokens_out, "latency_ms": round(latency_ms, 2),
              "cache_status": cache_status, "status": "ok"}
    REQUEST_LOG.append(entry)
    print("LOG", json.dumps(entry))
    return {"request_id": request_id, "answer": answer, "cache_status": cache_status}


def percentile(values, p):
    s = sorted(values)
    return s[min(int(len(s) * p), len(s) - 1)] if s else 0.0

@app.get("/metrics")
def metrics():
    latencies = [e["latency_ms"] for e in REQUEST_LOG]
    errors = sum(1 for e in REQUEST_LOG if e["status"] == "error")
    hits = sum(1 for e in REQUEST_LOG if e["cache_status"] == "HIT")
    served = sum(1 for e in REQUEST_LOG if e["cache_status"] in ("HIT", "MISS"))
    return {"total_requests": len(REQUEST_LOG),
            "p50_ms": round(percentile(latencies, 0.50), 2),
            "p95_ms": round(percentile(latencies, 0.95), 2),
            "p99_ms": round(percentile(latencies, 0.99), 2),
            "error_rate": round(errors / len(REQUEST_LOG), 3) if REQUEST_LOG else 0.0,
            "cache_hit_rate": round(hits / served, 3) if served else 0.0}