flowchart TD
A["You want to build a weather app"] --> B["Problem: you don't own the data"]
B --> C["The idea of an API: a controlled gate"]
C --> D["How one request works, end to end"]
D --> E["Anatomy: endpoints, requests, responses"]
E --> F["Zooming out: types of APIs"]
F --> G["The rules of the gate: protocols"]
G --> H["Guarding the gate: auth and authz"]
H --> I["The producer's view: API lifecycle"]
I --> J["Payoff: ready to build with FastAPI"]
style A fill:#10a37f,color:#fff
style J fill:#f59e0b,color:#fff
Introduction to APIs
Suppose you want to build a small app that shows people the weather. Nothing fancy: a user opens it, and it tells them whether to grab an umbrella before leaving the house. Simple idea. But almost immediately you hit a wall.
To show the weather, you need weather data. Where does it come from?
One option is to gather it yourself. You would need temperature, humidity, wind, and precipitation readings for every city your users might care about, refreshed constantly, forever. That means sensors, satellites, weather stations, and a small army of people maintaining them. For an app you wanted to build over a weekend, this is absurd.
So you look for a shortcut, and there is an obvious one. Companies already do this. They run the sensors, they maintain the databases, and they would happily let you use their data, perhaps for a fee. The natural next thought is: “Great, just give me access to your database and I will pull what I need.”
And here is where you hit the real problem. No company is going to hand you direct access to their database. Letting outsiders run arbitrary queries against your production data is a security nightmare and an operational one too. They have no idea what you might do, intentionally or by accident, once you are inside.
So we are stuck between two bad options: build everything ourselves (impossible), or get raw database access (which nobody will grant). We need a middle path. We need a way for the company to share exactly the data they choose, through a controlled, well-guarded opening, without ever exposing what is behind it.
That controlled opening is an API. This entire post is about understanding what it is, how it works, and everything that surrounds it.
What is an API?
An Application Programming Interface (API) is a set of rules and protocols that lets different software applications talk to each other. Think of it as a bridge between two systems that lets them share data and functionality without either one reaching into the other’s internal code.
Let us make this concrete with the weather problem. The company will not open its database to you. Instead, it builds a specialized gate: a small, locked-down entryway that the outside world is allowed to approach. You walk up to this gate, ask a specific, pre-approved question (“what is the weather in New York right now?”), and the gate hands back exactly the answer, nothing more. You never see the database. You never see their code. You just use the gate.
That gate is the API. It is the only thing standing between your weather app and the company’s data, and that is precisely the point. The company controls what the gate allows, so it can share weather readings while keeping everything else locked away.
This single idea, a controlled gate between two systems, unlocks a long list of benefits. APIs let different platforms communicate and share data without friction. They let you build apps faster by borrowing functionality instead of rebuilding it. They let systems scale and adapt as needs change. They let organizations open their services to outside developers and partners in a controlled, secure way, while enforcing security policies and keeping costs down. Every one of these benefits flows from that same root: a guarded doorway you can offer to the outside world without giving away the house.
The journey ahead
We now have the core idea. The rest of this post follows the natural path a builder takes once they accept that an API is the answer. Figure 1 shows the whole trail before we start walking.
We start by watching a single request travel through the gate, because once you have seen that happen once, every later idea has something solid to attach to. Then we name the parts we saw, zoom out to the different kinds of APIs and the languages they speak, look at how the gate is guarded, and finally step around to the other side to see how an API is built and maintained over its lifetime. By the end, you will be ready for the next post in this series, where we actually build one with FastAPI.
How a single request works, end to end
Before we catalog every kind of API and protocol, let us trace one concrete request all the way through. We will use the exact question from before: our weather app wants to know the weather in New York right now. Everything abstract later in the post is just a generalization of what happens in this one trip.
Here is the round trip, stage by stage.
- The request is initiated:
- The process starts when a client (a web browser, a mobile app, anything that consumes an API) sends a request to the API. In our case, the weather app is the client. Because both the app and the company live on the web, they communicate over HTTP, the protocol that the web runs on Fielding et al. (2022). The app forms a request that essentially says, “please give me the current weather for New York,” and sends it off.
- The request arrives at an endpoint:
- The request does not go to “the company” in some vague sense. It goes to a specific address, a URL known as the API endpoint. This is the gate from our metaphor: the exact spot the API has opened for incoming requests. For the weather app, the endpoint is the precise web address where the company listens for weather queries.
- The server processes the request:
- The company’s server receives the request at that endpoint and gets to work. It checks that the request is well-formed, confirms the caller is allowed in (more on that when we discuss authentication), and then goes and looks up the data, which may mean querying the very database we were never allowed to touch directly. The gate touches the database so we do not have to.
- The server generates a response:
- Once the server has the data, it packages up a response. For a weather query, that response holds the current conditions in New York, formatted in a structured way (usually JSON) that the app can read.
- The response is delivered:
- The server sends that response back through the gate to the app. The app receives it and does whatever it likes with the data: shows it to the user, runs further calculations on it, or both.
Figure 2 shows this whole round trip at a glance. Reading it top to bottom traces the same five stages, and it is worth keeping this picture in mind for the rest of the post, because everything else is a detail hanging off one of these arrows.
sequenceDiagram
participant App as Weather App (Client)
participant API as API Endpoint (Gate)
participant Srv as Company Server
participant DB as Weather Database
App->>API: GET request: weather in New York?
API->>Srv: Validate and forward request
Srv->>DB: Look up the data
DB-->>Srv: Return weather records
Srv-->>API: Build JSON response
API-->>App: 200 OK plus weather data
One thing to notice: the app and the database never speak directly. Every interaction passes through the gate, which is exactly the controlled, secure middle path we wanted. We have now seen the gate work. The natural next question is: what are the actual parts that made that trip possible? Let us name them.
The anatomy of an API
In the round trip above, a few specific things did the heavy lifting: the address we sent to, the message we sent, and the message we got back. Now that you have seen them in action, let us give them their proper names and look at each one closely.
The endpoint
The endpoint is a specific URL where a client can reach the API to perform some action. It is the communication touchpoint between client and server, and it is the gate itself. An endpoint usually carries two kinds of extra information beyond its base address: path parameters and query parameters.
A path parameter is part of the URL path that identifies a specific resource. A query parameter (everything after the ?) refines or filters the request. The difference is easiest to see by example. Figure 3 breaks down a typical weather endpoint into its pieces.
So in https://api.example.com/cities/nyc?units=metric, the base address points at the gate, /cities/nyc is a path parameter naming the resource we want (the city), and ?units=metric is a query parameter telling the server we would like the temperature in Celsius. To take a more familiar shopping example: https://www.amazon.com/bestsellers uses bestsellers as a path parameter, while https://www.amazon.com/bestsellers/s?k=samsung adds ?k=samsung as a query parameter to search within that category.
The request
The request is the message a client sends to the API to ask for information or to perform an operation. Every request is built from a few standard pieces:
- Method: the type of operation. The most common HTTP methods are
GET(retrieve data),POST(create new data),PUT(update existing data), andDELETE(remove data). In our weather example the app usedGET, because it only wants to read. - Endpoint: the URL the request is sent to, as described above.
- Headers: metadata about the request, such as the content type or authentication credentials.
- Body: the actual data being sent to the server (relevant for
POSTandPUT, where you are sending something).
Here is what a request that creates a new user might look like. Notice it carries a body, because it is sending data, unlike our read-only weather GET:
POST /users HTTP/1.1
Host: api.example.com
Content-Type: application/json
Authorization: Bearer token123
Body:
{
"name": "John Doe",
"email": "john.doe@example.com"
}The choice of method is what tells the server what kind of action you want. The same endpoint can often handle a GET, a POST, a PUT, and a DELETE, each doing something different.
The response
The response is the message the API sends back after processing the request. It mirrors the request’s structure and has its own standard pieces:
- Status code: a number signaling the outcome.
200 OKmeans success;404means the resource was not found; and as we will see shortly,401and403signal authentication and authorization problems. - Headers: metadata about the response, such as content type or caching instructions.
- Body: the actual data being returned, typically as JSON or XML.
Here is the response our weather server might send back, with the data the app asked for:
HTTP/1.1 200 OK
Content-Type: application/json
Body:
{
"id": 123,
"name": "John Doe",
"email": "john.doe@example.com"
}The status code is the first thing a well-behaved client checks. A 200 says “here is your data, all good.” Anything else tells the client something went wrong and, often, what.
Rate limiting and quotas
There is one more part of the anatomy worth knowing, and it exists to protect the gate from being overwhelmed. Rate limiting and quotas control how many requests a client may make within a given time window. They prevent abuse (whether malicious or accidental) and keep usage fair across all the clients sharing the API.
Servers usually communicate these limits through response headers:
X-RateLimit-Limit: the maximum number of requests allowed in the window.X-RateLimit-Remaining: how many requests you have left in the current window.Retry-After: how long to wait before trying again once you have hit the limit.
If your weather app fires off thousands of requests per second, these mechanisms are what stop it from taking down the company’s service, and what politely tell your app to slow down.
We have now followed a request through the gate (Figure 2) and named all the moving parts: the endpoint, the request, the response, and the rate limits that keep things orderly. Looking back at Figure 1, we have completed the “how one request works” and “anatomy” stops on the trail. Everything so far has been about a single API and a single trip through it. Next we zoom out: our weather API is just one kind of API, and it speaks just one of several possible languages. Let us survey the landscape.
Zooming out: the types of APIs
Our weather API is what is called a web API, but that is only one category. APIs come in several types, distinguished mostly by what they connect to and where they live. Figure 4 lays them out, and it is worth pausing on one relationship in particular: web APIs are a special case of remote APIs, which the figure shows by nesting one inside the other.
flowchart LR
Root["Types of APIs"] --> Lib["Library API"]
Root --> DB["Database API"]
Root --> HW["Hardware API"]
Root --> GUI["GUI API"]
Root --> Rem["Remote API"]
subgraph RemoteGroup["Remote APIs (over network)"]
Rem --> Web["Web API (a subset, over HTTP/HTTPS)"]
Rem --> Intra["Intranet API (private network only)"]
end
Let us walk through each type.
- Web API:
- These are accessible over the web using HTTP or HTTPS, which means they let applications communicate across the internet, usually exchanging data as JSON or XML. Our weather app talking to the OpenWeather API is a perfect example, as is embedding a Google Map on a website.
- Library API:
- These are provided by libraries or frameworks to expose specific functionality to developers, so you can call predefined functions instead of writing them from scratch. If you have used data or machine-learning tools, you have used these: the NumPy API for numerical computation, the TensorFlow API for building and training models, and the Matplotlib API for creating visualizations programmatically.
- Remote API:
- These interact with systems on a different network or server, typically over the internet or an intranet. This is the broader family that web APIs belong to: every web API is a remote API, but not every remote API is a web API. Some remote APIs are intranet APIs, reachable only inside a private network. Deploying virtual machines through the AWS EC2 API or retrieving files via the Google Drive API are remote-API examples.
- Database API:
- These let applications interact with databases in a structured way, performing the CRUD operations (Create, Read, Update, Delete). Examples include the MySQL Connector API, the MongoDB API for NoSQL data, and the Firebase Realtime Database API.
- Hardware API:
- These let software talk to physical devices, providing an abstraction layer so you can control hardware without low-level programming. GPU APIs like CUDA or OpenCL for parallel computation, APIs for IoT sensors and smart appliances, and APIs for controlling drones or robots all fall here.
- GUI API:
- These are for building and managing graphical user interfaces programmatically. The Java Swing API for desktop GUIs, Python’s Tkinter, and the Android SDK for mobile interfaces are all GUI APIs.
So our weather API is a web API, which makes it a remote API, and it happens to speak over HTTP. But how it structures its conversation, the precise rules it follows, is a separate question. That is what protocols are about, and it is where we go next.
The rules of the gate: API protocols
We know the gate works and we know what kind of gate ours is. But a gate needs an agreed-upon language so that both sides understand each other. An API protocol is exactly that: the set of rules an API follows when structuring its requests and responses.
Here is why this section is worth your attention. There is not one universal language; there are several, and each was designed for a different situation. A protocol built for a public web app makes different trade-offs than one built for high-speed communication between internal services. Knowing the main options means that when you build your own API, you can pick the right tool instead of defaulting to whatever you saw first. We will look at five: REST, SOAP, GraphQL, gRPC, and WebSocket.
REST
REST, short for Representational State Transfer, is a lightweight architectural style that uses HTTP for communication, which is why it is the workhorse of web APIs. It was introduced by Roy Fielding in his doctoral dissertation Fielding (2000), and it rests on a few core principles, two of which matter most here: statelessness and representing resources through URLs.
“Statelessness” means each request is self-contained and independent of every other; the server does not remember anything about previous requests. This makes REST APIs simple to scale, because any server can handle any request without needing shared memory of what came before. The common HTTP methods (GET, POST, PUT, DELETE) map naturally onto operations on resources. REST is everywhere: web apps, mobile apps, and large public APIs like those from Twitter and GitHub. For a deeper, practical treatment of designing REST APIs well, Richardson & Ruby (2007) is a classic reference.
A REST request and its JSON response look like this:
GET /users/123 HTTP/1.1
Host: example.com{
"id": 123,
"name": "John Doe",
"email": "john.doe@example.com"
}SOAP
SOAP, short for Simple Object Access Protocol, relies on XML messaging and is defined by a W3C recommendation World Wide Web Consortium (W3C) (2007). It is built for more structured and secure interactions, with built-in error handling and security extensions (such as WS-Security). It is heavier than REST, but that weight buys rigor, which is why it persists in enterprise settings like banking and payments where strict contracts matter.
A SOAP request is wrapped in an XML envelope:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GetUserDetails>
<UserId>123</UserId>
</GetUserDetails>
</soap:Body>
</soap:Envelope>GraphQL
GraphQL flips the usual arrangement: instead of the server deciding what each endpoint returns, the client specifies exactly the data it wants (GraphQL Foundation, 2021). This solves two common REST annoyances, over-fetching (getting more data than you need) and under-fetching (needing several requests to assemble what you want). All queries go to a single endpoint, and the client describes the shape of the response it expects.
A GraphQL query asks for precisely the fields it needs, including nested ones:
{
user(id: "123") {
name
email
posts {
title
comments {
text
}
}
}
}gRPC
gRPC, short for Google Remote Procedure Call, is a high-performance framework from Google that uses Protocol Buffers (Protobuf) to serialize messages (gRPC Authors, n.d.). It runs over HTTP/2, supports bidirectional streaming, works across many programming languages, and keeps payloads small. These traits make it a strong choice for fast communication between internal microservices, where efficiency matters more than human-readability.
A gRPC service is defined in a .proto file:
service UserService {
rpc GetUserDetails (UserRequest) returns (UserResponse);
}
message UserRequest {
int32 user_id = 1;
}
message UserResponse {
int32 user_id = 1;
string name = 2;
string email = 3;
}WebSocket
The WebSocket protocol provides full-duplex communication over a single, persistent TCP connection (Fette & Melnikov, 2011). Unlike the request-and-response pattern of REST, where the client must ask every time, WebSocket keeps the connection open so both sides can send data freely without re-establishing the link. Its low latency makes it ideal for real-time use cases: chat applications, online gaming, and live data feeds (think live stock tickers).
Opening a WebSocket connection and listening for updates looks like this:
const socket = new WebSocket("ws://example.com/stocks");
socket.onmessage = (event) => {
console.log("Stock update:", event.data);
};Comparing the most common protocols
REST, SOAP, and GraphQL are the three you will meet most often when building web APIs, so it helps to see them side by side. Table 1 summarizes the trade-offs.
| Feature | REST | SOAP | GraphQL |
|---|---|---|---|
| Data Format | JSON, XML, etc. | XML only | JSON only |
| Flexibility | High | Low | Very High |
| Performance | Fast | Slower | Efficient |
| Use Case | Modern web APIs | Enterprise applications | Dynamic client needs |
We now understand the gate, its parts, the family it belongs to, and the languages it can speak. But we have quietly skipped over something the server did back in stage 3 of our round trip: it checked that the caller was allowed in. A gate is not much of a gate if anyone can walk through it. So let us turn to how the gate is guarded.
The producer’s view: the API lifecycle
So far we have stood on the consumer’s side, using an API someone else built. Building and operating an API is a journey of its own, with distinct stages from first idea to eventual shutdown. This end-to-end process is the API lifecycle: a structured framework for designing, developing, deploying, managing, and eventually retiring an API so that it stays effective, scalable, and secure. Figure 6 shows the stages and, importantly, that the process loops: a new version sends you back to the drawing board.
flowchart TD
P["Planning and Design"] --> D["Development"]
D --> Dep["Deployment"]
Dep --> M["Monitoring and Management"]
M --> U["Updates and Versioning"]
U --> Ret["Retirement"]
U -.->|"new version"| P
Let us walk through each stage.
- Planning and design:
- You start by identifying the business goals, user needs, and the problem the API will solve, and by deciding who the audience is (external developers, partners, or internal teams). Then you define the endpoints, the request and response formats, and the data models, choosing a methodology like REST, GraphQL, or gRPC. Good design here means intuitive endpoints, consistent naming, and proper versioning from the start. Tools like Postman, SwaggerHub, and Stoplight help with design and documentation.
- Development:
- Now you implement the backend logic and infrastructure. You write the server-side code using a framework such as Flask, FastAPI, or Express.js; integrate authentication and security; and perform unit testing to validate individual functions. Tools like Git, Jenkins, Docker, and Kubernetes support the build-and-test pipeline (CI/CD).
- Deployment:
- You release the API into a production environment where real users can reach it. This typically means deploying through staging, testing, and production environments, using cloud services like AWS, Azure, or GCP for scalability and reliability, and setting up an API gateway to handle routing, caching, and load balancing. Popular gateways include AWS API Gateway, Kong, and Apigee.
- Monitoring and management:
- Once live, you watch the API closely: tracking performance metrics like response times, uptime, and error rates; gathering usage data to understand traffic patterns and popular endpoints; spotting areas to optimize; and enforcing the rate limiting we discussed earlier to prevent abuse. Tools like Datadog, New Relic, and Grafana help here.
- Updates and versioning:
- As needs evolve, you update the API, and the cardinal rule is to avoid breaking existing clients. When breaking changes are unavoidable, you provide clear migration paths, notify users well in advance of deprecated features, and give realistic timelines for the transition. This is the stage that loops back to planning when a genuinely new version is warranted.
- Retirement:
- Eventually an API outlives its usefulness and is retired or deprecated. Doing this responsibly means informing stakeholders well in advance, offering tools or resources to help users migrate to newer APIs, and archiving or securely disposing of any associated stored data.
Where we have been, and what comes next
Let us close the loop. We started with a frustrating problem: you wanted to build a weather app, but you did not own any weather data, and nobody would hand you their database. The resolution was the API, a controlled gate that lets two systems share exactly what they choose without exposing anything else.
From that one idea, we followed a single request through the gate and back, then named every part of the machinery it touched: endpoints, requests, responses, and the rate limits that keep traffic orderly. We zoomed out to see that our weather API was one type among many (a web API, which is a kind of remote API), and that the gate can speak several languages, with REST, SOAP, GraphQL, gRPC, and WebSocket each suited to different needs. We saw how the gate is guarded by authentication (who are you?) and authorization (what can you do?). And finally we stepped around to the builder’s side to walk the full API lifecycle, from planning to retirement.
That builder’s side is exactly where this series goes next. You now understand what an API is and why every piece exists. In the next post, we stop describing gates and start building one, using FastAPI to turn all of this theory into a working API of our own. The weather app has been our guide the whole way; soon you will be able to build the gate it needs.