Infrastructure Build Sheet

How to read this build sheet

The Infrastructure module explains why your licence decides your architecture. This page is the parts list: what to run where, what each option costs per unit, and the three builds.

Fintech infrastructure is three planes that fail differently and should be decided separately.

Optimise for reversibility first

The module's phrase is optimise for reversibility first. Here is what that means as an engineering rule, because it is the single decision that determines how much the next five years cost.

Almost every infrastructure choice in a fintech is made once, under time pressure, on incomplete information. You will be wrong about some of them. The question is not how to be right — it is how much a wrong answer costs to undo.

The practical consequence is that the cheap reversals should be behind interfaces and the expensive ones should be boring. An adapter in front of your model provider costs a day and buys you the ability to switch when a price moves 80%. An adapter in front of your core banking platform is a fantasy — nobody has ever successfully abstracted a core.

Where your data is allowed to be

Before any tooling decision, settle where data is allowed to be. In India this is not a preference.

The RBI's payment data storage direction requires the full end-to-end transaction details for payment systems to be stored only in India. Data may be processed abroad, but it must be brought back and the foreign copy deleted within the specified window. The DPDP Act adds consent, purpose limitation and breach notification on top, for personal data generally.

This has an architectural consequence that surprises people: the cheapest model is often not available to you for the workloads that matter most. That is not a reason to give up on cost. It is a reason to design a router that knows which data may leave and which may not, and to keep an in-country path for the second category.

Raw materials — serving, compute and data

How to use each one — model serving

The market has three price tiers, and they are an order of magnitude apart

As of September 2026 the shape is stable even though individual numbers are not. Budget and open-weight models sit near $0.03–$0.45 per million input tokens. The production tier clusters hard at $2 input — several major models landed on exactly that figure, which is not a coincidence but the most contested price point in the market. Frontier models run $4–$10 input and up to $50 output.

# NUMBER 1: your BLENDED rate. Comparing "input price" across providers is the
# single most common costing error. Output is routinely 5-6x input, so the
# ranking flips depending on your own input:output ratio.

def blended(in_price, out_price, in_tokens, out_tokens):
    """$ per 1M tokens at YOUR mix, not at the vendor's headline."""
    total = in_tokens + out_tokens
    return (in_price * in_tokens + out_price * out_tokens) / total

# A summarisation workload: long input, short output (90/10).
print(blended(2.00, 10.00, 90, 10))     # mid-tier  -> $2.80
print(blended(0.14,  0.28, 90, 10))     # budget    -> $0.154
# A generation workload: short input, long output (20/80). Same models.
print(blended(2.00, 10.00, 20, 80))     # mid-tier  -> $8.40   <- 3x the above
print(blended(0.14,  0.28, 20, 80))     # budget    -> $0.252

# NUMBER 2: the self-host CROSSOVER. Below it, the API wins on cost and on
# every operational dimension. Above it, self-hosting starts to pay.

def crossover(cluster_inr_per_hour, tokens_per_sec, api_usd_per_1m, usd_inr=88.0):
    """Monthly token volume at which self-hosting becomes cheaper."""
    self_inr_per_1m = cluster_inr_per_hour / (tokens_per_sec * 3600 / 1_000_000)
    api_inr_per_1m  = api_usd_per_1m * usd_inr
    if self_inr_per_1m >= api_inr_per_1m:
        return None                      # self-hosting never wins at this rate
    return {"self_inr_per_1m": round(self_inr_per_1m, 2),
            "api_inr_per_1m":  round(api_inr_per_1m, 2),
            "saving_per_1m":   round(api_inr_per_1m - self_inr_per_1m, 2)}

# One L40S at an Indian provider, serving a small open model.
print(crossover(cluster_inr_per_hour=61, tokens_per_sec=900, api_usd_per_1m=0.20))

# WHAT TO CHECK
# [ ] compute the blended rate on YOUR OWN traffic sample, not on an assumed
#     ratio. Pull a week of real prompts and count the tokens
# [ ] tokens_per_sec must come from a benchmark on YOUR model at YOUR batch
#     size and sequence length. Vendor throughput figures are peak, not typical
# [ ] the GPU is billed 24x7; your traffic is not. Divide by REAL utilisation.
#     A cluster busy 15% of the day costs ~6.7x its headline per token
# [ ] add to the self-host side: an engineer, monitoring, model upgrades,
#     capacity headroom for spikes, and a second node so a reboot is not an
#     outage. None of it is on the GPU price list
# [ ] add to the API side: egress, and the retry traffic from rate limits
# [ ] re-run this quarterly. An 80% price cut on one side moves the crossover
#     by 5x, and that has happened

The gotcha nobody documents: benchmark parity is not price parity, and the gap is enormous. On one widely-cited coding benchmark, five models score within 0.4 percentage points of each other — and their output prices run from $1.20 to $12 per million tokens. A tenfold spread for a rounding error in capability. Higher up, the best-scoring model costs roughly 42 times the cheapest in that band.

Which means the only question worth asking is whether your tasks live in the gap. Build an evaluation set of a hundred real examples from your own product, run the tiers against it, and look at where they actually differ. For classification, extraction, routing and first-pass drafting the answer is usually that they do not, and you should be on the cheap tier. For work where a weak answer creates cleanup downstream, pay.

Self-hosting in India

The economics are genuinely different here, and better than most teams assume.

Indian GPU providers run roughly 60–70% below hyperscaler Mumbai pricing. An H100 lists around ₹219–362 per hour domestically against ₹600–740 on a hyperscaler's India-facing infrastructure. Reserved pricing brings the effective rate toward ₹130–150. Spot capacity runs as low as ₹70–88 for interruptible work.

Buying outright is almost never right for an AI-serving workload: an H100 lands at ₹27–34 lakh per unit with import duties, a server at ₹2–5 crore, weeks to procure and months to deploy, against the same compute available in sixty seconds on an hourly rate.

Two India-specific mechanics worth knowing: GPU cloud spend abroad sits under the Liberalised Remittance Scheme and generally needs no RBI approval below $250,000 a year, and IGST on foreign cloud services applies via reverse charge but is claimable as input credit — so the headline dollar figure overstates the real cost to an Indian business.

How to use each one — residency and vectors

Residency as code

Residency is usually written as a policy and enforced nowhere. The version that survives an inspection is a function on the inference path.

from enum import Enum

class Residency(Enum):
    INDIA_ONLY = "india_only"     # RBI payment data. Cannot leave. Full stop.
    PERSONAL   = "personal"       # DPDP. May leave with consent + safeguards.
    OPEN       = "open"           # Product docs, policy text, public content.

# The classification is per FIELD, not per request. One request routinely
# carries fields in all three categories, and the strictest one wins.
FIELD_CLASS = {
    "txn_narration":   Residency.INDIA_ONLY,
    "beneficiary_vpa": Residency.INDIA_ONLY,
    "card_last4":      Residency.INDIA_ONLY,
    "customer_name":   Residency.PERSONAL,
    "customer_email":  Residency.PERSONAL,
    "policy_text":     Residency.OPEN,
    "product_faq":     Residency.OPEN,
}

ROUTES = {
    Residency.INDIA_ONLY: {"model": "self-hosted-in-mumbai", "egress": False},
    Residency.PERSONAL:   {"model": "vendor-with-india-region", "egress": False},
    Residency.OPEN:       {"model": "cheapest-global-api", "egress": True},
}

def route(payload: dict):
    present = [FIELD_CLASS[k] for k in payload if k in FIELD_CLASS]
    unknown = [k for k in payload if k not in FIELD_CLASS]
    if unknown:
        # An unclassified field is treated as the STRICTEST class, never the
        # loosest. New fields appear constantly and the default must be safe.
        present.append(Residency.INDIA_ONLY)
    strictest = (Residency.INDIA_ONLY if Residency.INDIA_ONLY in present
                 else Residency.PERSONAL if Residency.PERSONAL in present
                 else Residency.OPEN)
    r = dict(ROUTES[strictest])
    r.update({"residency": strictest.value, "unclassified_fields": unknown})
    return r

# WHAT TO CHECK
# [ ] the default for an UNKNOWN field is the strictest class. A field added by
#     a product team next sprint must not silently start crossing a border
# [ ] unclassified_fields is logged and alerted on. It is a backlog, not noise
# [ ] redaction happens BEFORE the call, not by asking the model to ignore
#     things. A prompt instruction is not a residency control
# [ ] the route taken is recorded with each inference, alongside the model and
#     version. "Which model saw this customer's data" must be answerable
# [ ] the in-country path is LOAD TESTED. It is the one you cannot fail over
#     away from, so it needs the headroom
# [ ] "transient" and "not used for training" do not change the analysis. If
#     payment data left India, it left India
# [ ] embeddings derived from restricted data inherit the restriction. A vector
#     is not anonymisation

The gotcha nobody documents: embeddings. Teams treat a vector as a safe derived artefact because it is not human-readable, and store it wherever is convenient. It is not anonymisation — embedding inversion is a real and demonstrated technique, and the vector is derived from the restricted data. If the source field was India-only, the embedding is too, and so is the vector store holding it. This catches teams who carefully route their inference and then ship their whole index to a hosted vector database in another region.

Vector storage: use the database you already have

For almost every fintech workload, pgvector inside your existing Postgres is the right answer, and the reasons are mostly not about performance.

• One fewer system to place, secure and localise. Under a residency regime that is a real saving, not an aesthetic one.
• Transactional consistency with the rows the vectors describe. A separate vector store drifts from its source and nobody notices until retrieval starts returning deleted records.
• Your existing backup, access control and audit already cover it.

Move to a dedicated store when you have a specific measured reason: index size beyond what your database can hold comfortably, or filtering requirements it cannot express. Not because a benchmark on a public dataset showed a latency difference that is invisible next to your model's own response time.

How to use each one — observability and cost

Everything above is a choice you make once. This is the part you live with.

import time, json

# The AI cost failure mode is not a big number. It is a number nobody saw
# growing. Record cost per call, at the call, with the thing that caused it.

def traced_call(client, model, messages, *, purpose, customer_id, route_info,
                prices):                       # prices: {"in": usd/1M, "out": usd/1M}
    t0 = time.perf_counter()
    resp = client.create(model=model, messages=messages)
    ms = (time.perf_counter() - t0) * 1000

    u = resp.usage
    cost = (u.input_tokens * prices["in"] + u.output_tokens * prices["out"]) / 1_000_000

    emit({
        "ts": time.time(),
        "purpose": purpose,              # "fraud_explain", "kyc_extract", ...
        "model": model,
        "model_version": resp.model,     # the RESOLVED version, not the alias
        "residency": route_info["residency"],
        "input_tokens": u.input_tokens,
        "output_tokens": u.output_tokens,
        "cached_tokens": getattr(u, "cached_input_tokens", 0),
        "usd": round(cost, 6),
        "latency_ms": round(ms, 1),
        "customer_id": customer_id,      # for per-customer unit economics
    })
    return resp

# WHAT TO CHECK
# [ ] "purpose" is mandatory and comes from a fixed enum. Cost per FEATURE is
#     the number that lets you kill an expensive feature nobody uses. Cost per
#     model tells you nothing actionable
# [ ] log the RESOLVED model version, not the alias you requested. A provider
#     moving an alias to a new version changes your quality and your bill with
#     no deploy on your side, and this field is how you find out
# [ ] track cached_tokens separately. Prompt caching can be a 10x+ discount on
#     repeated context and it is invisible unless you measure it
# [ ] alert on daily spend RATE, not monthly total. A monthly budget alert
#     fires on the 28th, which is 27 days late
# [ ] cost per customer, so unit economics are a fact rather than an estimate
# [ ] emit through OpenTelemetry, not a vendor SDK. The observability vendor is
#     a reversible decision only if you never coupled to them
# [ ] sample traces, but NEVER sample the cost counter. Sampled cost is wrong

The gotcha nobody documents: log the resolved model version, not the alias you asked for. Calling a provider's stable alias is convenient and it means the model underneath can change without a deploy on your side — different quality, different token consumption, different bill. Teams investigate a quality regression for a week before someone thinks to check whether the model changed. One field in your log makes that a five-minute question.

Cost per unit

Two more corrections worth making before anyone signs anything. Add an engineer, monitoring, model upgrades and a second node to the self-hosted side — a single-node deployment means a reboot is an outage. And re-run the comparison quarterly, because an 80% price cut on the API side moves the crossover point by a factor of five, and that has already happened once this year.

Best combinations

Combinations that conflict

• Payment data in a prompt to an offshore API. A cross-border transfer, regardless of transience or training assurances.
• Embeddings of restricted data in a hosted vector store abroad. A vector is not anonymisation and inherits the restriction of its source.
• H100s for inference serving. A training card doing a serving job, at three to five times the right price.
• Self-hosting at low utilisation. The GPU bills continuously; your traffic does not.
• A vendor observability SDK at a thousand call sites. An afternoon's decision turned into a quarter's migration.
• Trying to abstract the core banking platform. Nobody has done it. Choose slowly instead.
• Calling a model alias and not logging the resolved version. Your quality and your bill can change with no deploy on your side.
• Monthly budget alerts. They fire on the 28th, which is 27 days late. Alert on daily rate.

Three recommended builds

Strong and expensive

Build: commercial core banking platform → hyperscaler in an Indian region for the operational plane → reserved GPU capacity with a domestic provider for in-country inference → frontier API for open-data workloads → commercial observability on top of OpenTelemetry → a platform team.

Use when: you hold a licence with supervisory expectations, transaction volume is large, and an outage is a regulatory conversation rather than a bad afternoon.

Cost shape: core platform licence dominates everything else, by an order of magnitude.

Trade: the expensive decisions are also the slow ones. A threshold change can take a release cycle, and the core you picked is the core you have for years.

Strong and reasonable — the default

Build: Postgres as the ledger, modelled carefully and append-only → a BaaS or sponsor-bank relationship for the licence layer → Kafka or Redpanda for the operational plane → an adapter in front of every model provider → a residency router with an in-country path for restricted data and the cheap tier for everything else → pgvector in the same Postgres → OpenTelemetry and Langfuse → deterministic fallbacks behind every model.

Use when: you have engineers and you want to be able to change your mind about the things that are cheap to change.

Cost shape: model API spend on the cheap tier is small; the operational plane is modest; the ledger is Postgres. The dominant cost is the team.

Trade: you own the ledger correctness properties. That is the right trade, because they are also the thing you can never outsource responsibility for.

Strong and lean

Build: one Postgres → one budget-tier model API behind an adapter → a single residency rule that keeps payment data out of prompts entirely → pgvector → structured logs with cost per call → rules-only fallback.

Use when: pre-launch or early, and every hour on infrastructure is an hour not on the product.

Cost shape: tens of dollars a month for inference, plus a small database.

Trade: no redundancy and manual scaling. Acceptable. What is not acceptable at any size is the ledger shortcut — floats for money, mutable rows, a single timestamp. Those cost nothing on day one and are effectively unfixable once a year of data sits on top of them.

What next

One number to watch that is not on any dashboard by default: the share of AI calls that fell back to rules, and the trend. Zero means the fallback has never been exercised and you do not know whether it works. A rising line means something upstream is degrading and nobody has noticed. It is the single most informative metric about the AI plane, and almost nobody plots it.

What this feeds

Sources

Every figure, rule and date on this page, and where to check it. Entries are typed so you can see which numbers are primary-sourced and which are industry reporting — they are not equivalent, and treating them as if they were is how a confident wrong number gets repeated.