The first thing people usually do while designing systems is jump directly into databases, microservices, Kubernetes, Redis, queues, and scaling.
That is the wrong approach.
When designing systems like LeetCode, start with a simple question:
“What exactly are we trying to build?”
System design is not about throwing fancy technologies into a diagram.
It is about solving a problem step-by-step.
So let us build this like a story.
Step 1 — Understanding the Product
Before touching architecture, define the features.
For a LeetCode-like platform, the core requirements are:
Authentication
Problems / Problem List
Submissions
Leaderboard
Discussions
Now let us expand what “Submissions” actually means.
A submission is not just code.
It contains:
Problem statement reference
User code
Programming language
Test cases
Submission status
Execution result
Memory usage
Runtime
Timestamps
This becomes the heart of the entire system.
Everything else is secondary.
Step 2 — The Most Important Rule
Whenever a user submits code:
Save it immediately to the database.
Do not first execute the code.
Do not first validate it.
Do not first send it somewhere else.
Persist it first.
Why?
Because systems fail.
Workers crash. Containers die. Servers restart. Queues get delayed.
But if the submission is already stored safely in the database, you never lose user data.
That is why in the architecture:
Client sends submission
Primary server receives it
Entry is created in PostgreSQL immediately
Only after that does further processing begin.
This is an extremely important system design principle.
Step 3 — Thinking About Scale
Now let us talk about the “fancy words”.
People hear terms like:
CAP Theorem, Horizontal Scaling, Throughput, Latency, Fault Tolerance, Availability
And immediately try to force them everywhere.
Do not do that.
Use concepts only where they actually matter.
For LeetCode, the biggest challenge is not authentication.
It is code execution.
Because executing user code is:
CPU intensive
Dangerous
Slow
Unpredictable
Resource heavy
One user might submit:
while True:
pass
Another user may try:
rm -rf /
Another might try memory bombs.
So the entire architecture revolves around handling code execution safely and at scale.
Step 4 — High-Level Flow
Here is the overall flow of the system:
Client submits code
Primary server receives submission
Submission is stored in PostgreSQL
Job is pushed into AWS SQS
Worker picks job from queue
Worker sends code to execution engine
Code runs inside isolated runtime containers
Result gets stored back in database
Client polls for status updates
Simple.
Do not overcomplicate it.
Step 5 - Why We Use a Queue
The queue is one of the most important parts of the architecture.
In this design, AWS SQS acts as a buffer between:
API servers
Code execution system
Without queues, your primary server becomes overloaded very quickly.
Imagine:
10,000 users submitting code simultaneously
Each execution taking several seconds
Some submissions hanging forever
If the API server directly executes code:
Requests block
Server threads get exhausted
Latency increases
Entire application slows down
Instead:
Save submission
Push job into queue
Return response immediately
This makes the system responsive.
The user gets:
{
"submissionId": "123"
}
And later polls for results.
Step 6 - Polling vs WebSockets
In the architecture diagram, the client continuously asks:
GET /submission/:id
This is polling.
Why polling first?
Because it is simpler.
Many beginners immediately jump to WebSockets.
But system design is about choosing the simplest thing that works.
Polling works perfectly fine for:
Hackathons
MVPs
Early-stage systems
Later, if needed, you can move to:
WebSockets
Server-Sent Events (SSE)
Realtime updates
Start simple.
Step 7 — The Worker / Orchestrator
Now comes the interesting part.
The worker continuously reads jobs from SQS.
Its responsibilities are:
Fetch submission data
Determine programming language
Send code to correct runtime
Track execution status
Handle retries
Store final results
Think of the orchestrator as a traffic controller.
It does not execute code itself.
It delegates execution to isolated runtimes.
Step 8 — Code Execution Engine
This is the core of the system.
The execution engine contains isolated runtimes for:
Python
JavaScript
Java
Each runtime runs inside containers.
Why containers?
Because user code cannot be trusted.
Containers provide isolation.
This prevents:
File system access
Network abuse
Infinite resource consumption
Attacks on host machine
Step 9 - Security Layer
This is where most online judge systems become difficult.
Running user code is dangerous.
That is why the architecture includes:
Docker isolation
No network calls
Restricted system access
CPU limits Memory limits
For example:
Limit execution time to 2 seconds
Limit memory to 256 MB
Disable internet access
Prevent root access
If a submission exceeds limits:
Kill the container
Mark submission as failed
Never trust user code.
Step 10 — Why Separate Runtimes?
Different languages behave differently.
Python execution is different from Java.
Java needs:
Compilation step
JVM startup
JavaScript behaves differently from compiled languages.
Separating runtimes gives:
Better isolation
Easier debugging
Independent scaling
Cleaner deployments
You can scale Python workers independently if Python traffic becomes high.
Step 11 — AWS Fargate for Scaling
The execution engine is deployed on AWS Fargate.
Why?
Because execution load is unpredictable.
Some days:
100 submissions
Some days:
1 million submissions
Fargate helps automatically:
Add containers
Remove containers
Scale based on load
This prevents idle infrastructure costs.
You only pay for what you use.
Step 12 — Database Design Thoughts
PostgreSQL is enough initially.
You do not need Cassandra, DynamoDB, or distributed databases on day one.
Start simple.
Possible tables:
Users:
users
- id
- username
- email
Problems:
problems
- id
- title
- difficulty
- statement
Submissions:
submissions
- id
- user_id
- problem_id
- language
- code
- status
- runtime
- memory
This is enough to start.
Do not prematurely optimize.
Step 13 — Where CAP Theorem Actually Matters
Now we can finally mention CAP theorem properly.
In distributed systems, you usually balance between:
Consistency
Availability
Partition Tolerance
For LeetCode:
Availability matters
Consistency matters for submissions
Partition tolerance matters because systems are distributed
But honestly:
You do not need deep CAP discussions for an MVP.
Most beginners force these terms unnecessarily.
Mention them only when scaling becomes genuinely complex.
The biggest mistake beginners make in system design is trying to build Google on day one.
Good system design is iterative.
Start with:
Simple API
Simple database
Queue
Worker
Isolated execution
Then improve step-by-step.
The architecture shown here already teaches several real-world concepts:
Async processing
Queue-based systems
Distributed execution
Container isolation
Fault tolerance
Horizontal scaling
And most importantly:
Always store the submission before processing it.
That single decision saves systems in production more than people realise.
You can find the complete diagram here:
