MODULE 07 · SYSTEM ARCHITECTURE

Horizontal Scaling

Horizontal scaling adds more instances or nodes instead of making one machine bigger, so work can be shared across multiple system units.

01 · What is it?

Horizontal scaling, also called scaling out, increases capacity by adding more instances, machines, or nodes that can share the workload.

A request may be routed to any healthy application instance, while a cluster or service coordinates the collection of machines. The system grows by adding units rather than by upgrading only one unit.

Core idea: use more cooperating machines to handle more work.

02 · Think of it like...

Think of a busy shop opening more locations instead of making one shop endlessly larger.

A shared brand and operating process can serve more customers across several locations. The benefit is more total capacity and a smaller blast radius when one location has trouble, but the locations must coordinate inventory, staffing, and customer records.

Incoming Traffic
Load Balancer
Instance A + Instance B
Shared Capacity

03 · What does it do?

Horizontal scaling lets a system add parallel capacity when one instance is no longer enough.

Add Instances

Run more copies of a stateless application so requests can be processed in parallel.

Spread Traffic

Use a load balancer or routing layer to send work to healthy instances.

Limit Blast Radius

A failure can remove one instance while other instances continue serving traffic.

Scale in Steps

Add or remove units as demand changes instead of resizing one large machine.

More instances help only when the application, data, network, and dependencies can work with a distributed shape.

04 · Where is it used?

Horizontal scaling is common when demand varies, availability matters, or a workload can be split across independent workers.

Web ApplicationsMany parallel requests

Run multiple frontend or backend instances behind a load balancer during normal traffic and peaks.

Background JobsMore workers

Add queue consumers so image processing, notifications, or exports finish faster.

Large ServicesIndependent capacity

Scale the busiest service without giving every service a larger machine.

Cloud PlatformsElastic capacity

Use automated instance groups or containers to match capacity to a changing workload.

05 · Example

A meal-delivery app expects a large dinner-time spike. One application server is reaching its request and CPU limits, even though the database still has headroom.

  1. The team checks request rate, p95 latency, error rate, CPU, memory, database connections, and queue depth.
  2. They run three identical application instances behind a load balancer instead of relying on one larger server.
  3. They move session state to a shared store or signed cookie so any healthy instance can handle the next request.
  4. They test scale-out and scale-in behavior, then watch the database and payment provider for new bottlenecks.

PM Perspective

Ask what work can safely be shared, whether users can land on any instance, and which state must be shared or reconstructed. Track p95 latency, error rate, successful orders, instance count, database saturation, and infrastructure cost. More instances improve capacity only if the next dependency is not already the limit.

The product gains parallel request capacity, but it also takes on routing, shared-state, deployment, and observability responsibilities.

06 · How it connects

Traffic grows
Load Balancer
Instance A / B / C
Shared State + Database
Observe and Adjust

Horizontal scaling changes the number of workers in the serving layer; it does not automatically scale the database, cache, queue, or external provider.

07 · Do Not Confuse

Horizontal Scaling

Add more instances or nodes. It can increase parallel capacity and reduce dependence on one machine, but it requires routing and coordination.

Vertical Scaling

Make one machine bigger. It is often simpler for a single component, but it has a hardware ceiling and can remain a single point of failure.

Load Balancing

Distribute requests across available instances. It helps horizontal scaling work, but it is a routing mechanism—not the whole scaling strategy.

Auto Scaling

Automatically add or remove capacity from rules or signals. Auto scaling can use horizontal or vertical changes; it is the policy, not the direction.

High Availability

Keep service usable when components fail. Multiple instances can support availability, but only if health checks, dependencies, and recovery paths are designed too.

Performance Optimization

Make code, queries, or architecture more efficient. Optimization can delay the need to scale, but it is different from adding capacity.

Horizontal scaling answers “how many cooperating units?” It does not answer “how should state, failure, or data consistency work across them?”

08 · Topic Glossary

ENGLISH TERM中文SIMPLE ENGLISH中文小白理解
Horizontal Scaling水平扩展Adding more machines or instances to share work.增加更多机器或实例,一起分担工作。
Scaling Out向外扩展Another name for horizontal scaling.水平扩展的另一种说法。
Instance实例One running copy of an application or service.正在运行的一份应用或服务副本。
Node节点A machine or worker unit in a system.系统里的一台机器或一个工作单元。
Load Balancer负载均衡器A component that routes traffic across healthy instances.把请求分配给健康实例的组件。
Stateless无状态An instance does not need local memory of a user's previous request.实例不必把用户之前的请求记在自己本地。
Shared State共享状态Information that multiple instances can read and update.多个实例都能读取和更新的信息。
Health Check健康检查A test used to decide whether an instance can receive work.用来判断实例是否还能接收工作的测试。
Blast Radius影响范围The amount of a system affected by one failure.一次故障会影响到系统多大范围。
Auto Scaling自动扩缩容Automatically changing capacity from rules or signals.根据规则或指标自动调整资源数量或大小。

09 · What to Remember

Horizontal scaling adds more instances or nodes instead of making one machine bigger.

It is useful for parallel web requests, background workers, and workloads that can be divided across independent units.

It can improve capacity and reduce reliance on one machine, but it introduces routing, shared-state, consistency, deployment, and dependency trade-offs.

↑