Notes on how production grade cloud architectures look

Learning how a production grade Wise application cloud architecture looks like.

We have a queue like structure creating events for consumers, a cluster holding groups of pods serving different purposes, Grafana that monitors all these pods and CI/CD pipelines that handle deployments to these pods after testing and reviewing logs.

Microservices

A pod is the smallest subset of an app that can be deployed. Pods are deployed via kubernetes and are iterations of the same app — however they're never the entire app but versions of microservices. For instance, the most common microservice distribution for a enterprise grade fintech app looks like this:

Containers and pods

Kubernetes pods run containers, so everything Helm or YAML deploys is ultimately a container. Each microservice is packaged as a Docker image.

A pod is typically one container. Think of those terms as interchangeable although they might have differences. Containers are easy, deployed through docker, like one virtual machine that is isolated from your computer and holds all the info that's needed.

Each pod holds a container and kubernetes allows you to create multiple pods and can also increase and decrease their numbers as need arises. This is useful because we don't want to run a huge app that's always running because requests might decrease or increase. So we want to be able to adjust as the need arises to save costs.

Multiple replicas of the same app

Even within a single microservice, companies run multiple pods (replicas):

• For high availability → if one pod crashes, others serve traffic
• For load balancing → handle more requests concurrently
• For rolling updates → deploy new versions without downtime

Service and Ingress

Water tower analogy: A water pump is necessary to pump water to a society. The average water requirement for a town might be 400 gallons. But we can't design a water pump based on average needs, rather we need to account for the ceiling, the highest water requirement. Say it's morning and everyone needs water right now and water needs hit 600 gallons. So the total water need is 600 gallons.

Creating a water tower allows you to draw from a set resource instead of having to create one big pump. That's equivalent to a kubernetes cluster. The cluster reduces the number of pods and increases them to serve the upper limit of needs as the need arises.

Service

• Purpose: load balances traffic to pods and gives them a stable network identity
• Types:
  • ClusterIP → internal only
  • NodePort → expose on cluster nodes
  • LoadBalancer → external cloud load balancer (AWS ELB, GCP, etc.)

Ingress

• Purpose: route external HTTP/HTTPS traffic into your cluster
• Ingress controller is required (Nginx, Traefik, AWS ALB)
• You define Ingress rules that map hostnames/paths to Services

Helm Charts

Helm is a package manager for Kubernetes. Helm is useful in simplifying complex deployments into reusable "charts," enabling versioning, rollbacks, and consistent management across environments.

charts/
├── ledger/
│   ├── templates/
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   └── configmap.yaml
│   └── values.yaml

You write YAML templates describing:

• Deployments (what pods to run, how many replicas)
• Services (how to route traffic to pods)
• Ingress (how HTTP traffic enters)
• ConfigMaps / Secrets
• StatefulSets (for DBs)
• CronJobs / Jobs
• Monitoring stacks (Grafana, Prometheus)

You can deploy all services at once (helm install ledger ./charts/ledger).

• You can update services individually or as a group (helm upgrade ledger ./charts/ledger)
• All configuration lives in values.yaml (environment-specific settings)

CI/CD pipeline for a service like Wise

[Code push]
    ↓
[CI Stage] → Build → Lint → Unit Tests → Integration Tests
    ↓
[Docker Stage] → Build Docker images → Push to registry
    ↓
[CD Stage] → Deploy to cluster via Helm → Run smoke tests / integration tests
    ↓
[Monitoring Stage] → Alert if deployment fails, metrics report

Kafka Queues

Kafka is a distributed event streaming platform:

• It allows services to publish and subscribe to events asynchronously.
• Acts as a buffer / message bus between microservices.
• Decouples services so they don't need to call each other directly.

How Kafka works

1. Producer publishes an event to a topic

   Producer → Topic: transactions Event: {id: 123, from: A, to: B, amount: 100}

2. Kafka stores the event in partitions of the topic

   • Partitions are ordered logs (append-only)
   • Each partition is replicated for durability

3. Consumers subscribe to the topic

   • Each consumer tracks its offset: "I have read up to event #45 in partition 1"
   • Consumers read asynchronously at their own pace

4. Kafka does NOT delete events immediately

   • The event remains in the log until it expires (retention policy)
   • Multiple consumers can independently read the same event

Async consumption model

• Producers push messages asynchronously
• Consumers pull messages on their own schedule
• Each consumer group has its own offset pointer

Diagram

Topic: transactions Partition 0: [e1, e2, e3, e4, e5, ...]  <-- ordered log

Consumer Group 1 (Ledger Service) offset = 3 → reads e4 next

Consumer Group 2 (Notification Service) offset = 1 → reads e2 next

Why Kafka is preferred in large systems like Wise

1. Decoupling of services

   • Transaction service doesn't need to call all downstream services directly

2. Durable, persistent events

   • If notification service is down, it can catch up later

3. Multiple consumers

   • Ledger service, notification service, analytics service can all read the same events

4. Audit & replay

   • Financial services need to replay transactions in case of errors

5. Scalability under load

   • Webhooks might fail under spikes; Kafka buffers and distributes events