From CGI to PHP and application servers

CGI is a way for a web server to start a new program for every request. PHP put code right inside HTML, and an application server keeps a program running.

How the server learned to think

We left off with the fact that a dynamic page needs code that builds it at the moment of the request. But code on a server can be run in different ways, and almost everything depends on the choice: speed, the price of hosting, what can be built. Three ways appeared one after another, and all three are alive today.

CGI: a new program for every request

The first sites were static. To let a page depend on the request, CGI (the Common Gateway Interface) was invented in 1993. The idea is simple: when a web server receives a request for a certain address, it doesn’t look for a file but starts a program, hands it the request (the address, the parameters, the headers) and sends the visitor whatever the program prints. The program can be written in anything: Perl (most often), C, shell. The rules of the conversation are written down in the standard RFC 3875.

It was a brilliant idea: the server no longer knows anything about your task, it just starts a program. But the drawbacks showed at once. Starting a program is expensive: the system creates a process, loads the interpreter and reads the code, and then it all gets destroyed, and for the next request everything is repeated. With ten visitors a minute that is bearable. With thousands, the server spends more time starting than answering.

PHP: code inside the page

In 1995 Rasmus Lerdorf, a Danish-Canadian programmer, wrote a set of small programs for his own pages, and PHP grew out of it. Its main idea is that code goes right into the HTML:

<h1>Hello, <?php echo $name; ?></h1>

The server runs the insertions and sends the finished page. For a beginner it was paradise: you write a page as usual and, where you need to, you put in some code. PHP quickly became the language of “sites built from a database”, and its hosting cost pennies and was available at every provider. WordPress, Drupal and Joomla are built on it, and so are 1C-Bitrix in Russia and Magento for shops, which means an enormous share of the sites in the world.

PHP’s model is very simple: every request starts on a clean sheet. The script runs from start to finish, the page is sent, and all the variables are forgotten. The next request knows nothing about the previous one. It’s a limitation, but also a protection: there’s nothing to pile up, mistakes don’t accumulate in memory, and one request can’t spoil another. Whatever has to be remembered between requests (who is logged in, what is in the cart) is kept outside: in the database, in a session file, in a cache.

To avoid starting PHP afresh for every request, as CGI did, FastCGI was invented, and for PHP its implementation is PHP-FPM (the FastCGI Process Manager). It’s a pool of ready processes: nginx hands the request to a free one, which runs the script, sends the answer, forgets the variables and waits for the next. The interpreter no longer has to be started, and the “clean sheet” stays.

Application servers: a program that doesn’t die

For many tasks the clean sheet gets in the way. A chat holds an open connection with every visitor and has to send messages to all of them at once. An online game remembers where the players are. A dashboard with live graphs follows a stream of events. These need a program that runs all the time, keeps shared state in memory and takes requests itself.

That’s how application servers are built: Node.js, which runs JavaScript (since 2009), Python applications behind the WSGI and ASGI interfaces (Django stands on them), Java and C#, Go, Elixir with the Phoenix framework, Ruby on Rails. The application is started once and takes requests by the thousand, and in front of it stands the nginx you already know, as a reverse proxy.

The price of a living program is a set of cares that PHP doesn’t have: memory leaks (forgotten state piles up), the need to restart the application carefully when it’s updated, and race conditions (when two requests change the same data at once). That’s why, for ordinary sites “with a form and a database”, PHP’s model is still reasonable, and an application server is chosen when permanent connections or heavy processing are needed.

Look at the three designs side by side. Send a few requests and watch the counter:

Send a request Send five at once Start over Requests: 0

CGI

Each request: a new program is started.

After the answer: the program ends and its memory is freed.

Counter in the code ($visits++): 0

Time for 0 requests: ≈ 0 ms

PHP-FPM

Each request: a ready worker is taken from the pool.

After the answer: the script’s variables are forgotten after the answer.

Counter in the code ($visits++): 0

Time for 0 requests: ≈ 0 ms

Application server

Each request: the program is already running.

After the answer: variables live on between requests.

Counter in the code ($visits++): 0

Time for 0 requests: ≈ 0 ms

Send a few requests and watch the counter.

The times are illustrative and show the order of magnitude: starting a program is expensive, a pool of ready workers is cheaper, and a running program is cheapest of all. Real figures depend on the language, the code and the hardware.

Check yourself: why does the counter in PHP start from zero on every request?

The script runs afresh for each request, and its variables don’t survive its end. To remember something between requests, it’s written to the database, a session or a cache.

Why it turned out this way

A language is rarely chosen on its merits. In the 2000s cheap hosting came with PHP, so WordPress appeared, so thousands of specialists and plugins appeared, so it pays to write the next site in PHP. This chain is called the ecosystem effect, and a statement like “Python is better” can’t undo it.

The opposite is true too: a language with a living application is chosen when the task needs it. That’s the subject of the next lesson.

Practice

You’re on a site from 2010, in the age of content management systems. Take the page of a section of your project and describe, step by step, what happens to a request for it: who accepts it, who builds the page, where the data comes from and how the response goes back.

Then answer three questions. Which model do you choose (a process per request or a living application), and why. Where would you put a cache, and what would it store. And does cheap PHP hosting suit your project. For a business card, a shop or a blog the answer is more often yes, and for a chat or a game no.

What was wrong with 2010

CGI scripts were written straight into the cgi-bin folder, and any mistake in one turned into a hole in the server: the request’s parameters went straight into commands, and the scripts ran with the server’s rights. The first PHP suffered from the same: code and markup mixed in one file, the database password right in the page, and register_globals let anyone set any variable through the address. A generation of sites in the 2000s was written like that, which is how PHP got its reputation as an unsafe language.

Today’s PHP is quite different: it has types, frameworks (Laravel, Symfony) and strict rules. And the “one request, one process” model, which then looked like a flaw, is now considered a virtue: it’s simple, reliable and scales well just by adding servers.