A Beginner’s Guide to HTTP and REST

Hypertext Transfer Protocol (HTTP) is the life of the web. It's used every time you transfer a document or make an AJAX request. But HTTP is surprisingly a relative unknown among some web developers.

This introduction will demonstrate how the set of design principles known as REST underpin HTTP. You'll learn how to embrace its fullest power by building interfaces, which can be used from nearly any device or operating system.

Envato Market also has thousands of useful code scripts, plugins and apps to help you with web development, such as Premium URL Shortener, a PHP script that you can install on your server to create custom shortened URLs.

Why REST?

REST is a simple way to organize interactions between independent systems. It's been growing in popularity since 2005, and it inspires the design of services such as the Twitter API. This is due to the fact that REST allows you to interact with minimal overhead with clients as diverse as mobile phones and other websites. In theory, REST is not tied to the web, but it's almost always implemented as such, and was inspired by HTTP. As a result, REST can be used wherever HTTP can.

The alternative is building relatively complex conventions on top of HTTP. Often, this takes the shape of entire new languages. The most illustrious examples are SOAP and GraphQL. You have to learn a completely new set of conventions, but you never use HTTP to its fullest power. Because REST has been inspired by HTTP and plays to its strengths, it is the best way to learn how HTTP works.

After an initial overview, we'll examine each of the HTTP building blocks: URLs, HTTP verbs, and response codes. We'll also review how to use them in a RESTful way. Along the way, we'll illustrate the theory with an example application, which simulates the process of keeping track of data related to a company's clients through a web interface.

HTTP

HTTP is the protocol that allows for sending documents back and forth on the web. A protocol is a set of rules that determines which messages can be exchanged, and which messages are appropriate replies to others. Another common protocol is POP3, which you might use to fetch email on your hard disk.

In HTTP, there are two different roles: server and client. In general, the client always initiates the conversation; the server replies. HTTP is text based; that is, messages are essentially bits of text, although the message body can also contain other media. Text usage makes it easy to monitor an HTTP exchange.

HTTP messages are made of a header and a body. The body can often remain empty; it contains data that you want to transmit over the network, in order to use it according to the instructions in the header. The header contains metadata, such as encoding information; but, in the case of a request, it also contains the important HTTP methods. In the REST style, you will find that header data is often more significant than the body.

Spying HTTP at Work

If you use Chrome or Firefox Developer Tools, click on Network on the top bar to view HTTP requests in the website you are currently at. You might have to refresh the page with the Network developer tools open to see the logs. For example:

Another helpful way to familiarize yourself with HTTP is to use a dedicated client, such as cURL. cURL is a command line tool that is available on all major operating systems.

Once you have cURL installed, type:

1	curl -v google.com

This will display the complete HTTP conversation. Requests are preceded by >, while responses are preceded by <.

URLs

URLs are how you identify the things that you want to operate on. We say that each URL identifies a resource. These are exactly the same URLs which are assigned to webpages. In fact, a webpage is a type of resource.

Let's take a more exotic example and consider our sample application, which manages the list of a company's clients. /clients will identify all clients, while /clients/jim will identify the client named "Jim", assuming that he is the only one with that name.

In these examples, we do not generally include the hostname in the URL, as it is irrelevant from the standpoint of how the interface is organized. Nevertheless, the hostname is important to ensure that the resource identifier is unique all over the web. We often say you send the request for a resource to a host. The host is included in the header separately from the resource path, which comes right on top of the request header:

1	GET /clients/jim HTTP/1.1
2
3	Host: example.com

Resources are best thought of as nouns. For example, the following is not RESTful:

1	/clients/add

This is because it uses a URL to describe an action. This is a fairly fundamental point in distinguishing RESTful from non-RESTful systems.

Finally, URLs should be as precise as needed; everything needed to uniquely identify a resource should be in the URL. You should not need to include data identifying the resource in the request. This way, URLs act as a complete map of all the data your application handles.

But how do you specify an action? For example, how do you say that you want a new client record created instead of retrieved? That is where HTTP verbs come into play.

HTTP Verbs

Each request specifies a certain HTTP verb, or method, in the request header. This is the first all-caps word in the request header. For instance, GET / HTTP/1.1 means the GET method is being used, while DELETE /clients/anne HTTP/1.1 means the DELETE method is being used.

HTTP verbs tell the server what to do with the data identified by the URL. The request can optionally contain additional information in its body, which might be required to perform the operation—for instance, data you want to store with the resource. You can supply this data in cURL with the -d option.

If you've ever created HTML forms, you'll be familiar with two of the most important HTTP verbs: GET and POST. But there are far more HTTP verbs available. The most important ones for building RESTful API are GET, POST, PUT, and DELETE. Other methods are available, such as HEAD and OPTIONS, but they are rarer. If you want to know about all other HTTP methods, the official source is IETF.

GET

GET is the simplest type of HTTP request method—the one that browsers use each time you click a link or type a URL into the address bar. It instructs the server to transmit the data identified by the URL to the client. Data should never be modified on the server side as a result of a GET request. In this sense, a GET request is read-only, but of course, once the client receives the data, it is free to do any operation with it on its own side—for instance, format it for display.

PUT

A PUT request is used when you wish to create or update the resource identified by the URL. For example, PUT /clients/robin might create a client called Robin on the server. You will notice that REST is completely back-end agnostic; there is nothing in the request that informs the server how the data should be created—just that it should. This allows you to easily swap the back-end technology if the need should arise. PUT requests contain the data to use in updating or creating the resource in the body. In cURL, you can add data to the request with the -d switch:

1	curl -v -X PUT -d "some text"

DELETE

DELETE should perform the contrary of PUT; it should be used when you want to delete the resource identified by the URL of the request.

1	curl -v -X DELETE /clients/anne

This will delete all data associated with the resource, identified by /clients/anne.

POST

POST is used when the processing you wish to happen on the server should be repeated, if the POST request is repeated (that is, they are not idempotent; more on that below). In addition, POST requests should cause processing of the request body as a subordinate of the URL you are posting to.

In plain words, POST /clients/ should not cause the resource at /clients/ itself to be modified, but a resource whose URL starts with /clients/. For instance, it could append a new client to the list, with an id generated by the server:

1	/clients/some-unique-id

PUT requests are used easily instead of POST requests, and vice versa. Some systems use only one, some use POST for create operations and PUT for update operations (since with a PUT request you always supply the complete URL), and some even use POST for updates and PUT for creates.

Often, POST requests are used to trigger operations on the server which do not fit into the Create/Update/Delete paradigm, but this is beyond the scope of REST. In our example, we'll stick with PUT all the way.

Classifying HTTP Methods

Safe and Unsafe Methods

Safe methods are those that never modify resources. The only safe method, from the four listed above, is GET. The others are unsafe because they may result in a modification of the resources.

Idempotent Methods

These methods achieve the same result, no matter how many times the request is repeated: they are GET, PUT, and DELETE. The only non-idempotent method is POST.

PUT and DELETE being considered idempotent might be surprising, but it's quite easy to explain. Repeating a PUT method with the same body should modify a resource in a way that it remains identical to the one described in the previous PUT request: nothing will change! Similarly, it makes no sense to delete a resource twice. It follows that no matter how many times a PUT or DELETE request is repeated, the result should be the same as if it had been done only once.

Remember: it's you, the programmer, who ultimately decides what happens when a certain HTTP method is used. There's nothing inherent to HTTP implementations that will automatically cause resources to be created, listed, deleted, or updated. You must be careful to apply the HTTP protocol correctly and enforce these semantics yourself.

Representations

We can sum up what we have learned so far in the following way: the HTTP client and HTTP server exchange information about resources identified by URLs.

We say that the request and response contain a representation of the resource. By representation, we mean information, in a certain format, about the state of the resource or how that state should be in the future. Both the header and the body are pieces of the representation.

The HTTP headers, which contain metadata, are tightly defined by the HTTP spec; they can only contain plain text and must be formatted in a certain manner.

The body can contain data in any format, and this is where the power of HTTP truly shines. You know that you can send plain text, pictures, HTML, and XML in any human language. Through request metadata or different URLs, you can choose between different representations for the same resource. For example, you might send a webpage to browsers and JSON to applications.

The HTTP response should specify the content type of the body. This is done in the header, in the Content-Type field. For instance:

1	Content-Type: application/json

For simplicity, our example application only sends JSON back and forth, but the application should be designed in such a way that you can easily change the format of the data to tailor it for different clients or user preferences.

HTTP Client Libraries

To experiment with the different request methods, you need a client, which allows you to specify which method to use. Unfortunately, HTML forms do not fit the bill, as they only allow you to make GET and POST requests. In real life, APIs are accessed programmatically through a separate client application, or through JavaScript in the browser.

That's why, in addition to the server, it is essential to have good HTTP client capabilities available in your programming language of choice.

A very popular HTTP client library is, again, cURL. You've already been familiarized with the cURL command from earlier in this tutorial. cURL includes both a standalone command-line program and a library that can be used by various programming languages. In particular, cURL is, more often than not, the HTTP client solution of choice for PHP developers. Other languages, such as Python, offer more native HTTP client libraries.

Setting Up the Example Application

Now we will build a barebones example application. You can build the example application in either Node.js or PHP by following the respective section and using the respective folder in the code attachment. Both applications work identically. If you are not sure which to pick, Node.js might be a better choice since it is more commonly used now.

Node.js & Express

In order to run the example application, you need Node.js installed. Once you have, open the node.js directory in the source code attachment and run npm install.

PHP

To run the example application, you will need to install PHP 5 and a web server with some mechanism to run PHP. The current version must be at least version 5.2 to have access to the json_encode() and json_decode() functions.

As for servers, the most common choice is still Apache with mod_php, but you're free to use any alternatives that you're comfortable with. There is a sample Apache configuration, which contains rewrite rules to help you set up the application quickly. All requests to any URL starting with /clients/ must be routed to our server.php file.

In Apache, you need to enable mod_rewrite and put the supplied mod_rewrite configuration somewhere in your Apache configuration or your .htacess file. This way, server.php will respond to all requests coming from the server. The same must be achieved with Nginx, or whichever alternative server you decide to use.

How the Example Application Works

Node.js and Express

If you look through the code, you will see a few different methods like app.get or app.put. These are different routes. Each route matches a certain URL and HTTP method.

app.get("/clients", (_, res) => {
    ...
});
app.get("/clients/:client", (req, res) => {
	...
});
app.put("/clients/:client", (req, res) => {
	...
});
app.delete("/clients/:client", (req, res) => {
	...
});

You might have noticed :client in the URL. That is a parameter, which means that anything in that part of the URL will match that route, and that part of the URL will be passed as a parameter. Inside the route handler functions, you can see a comment describing the logic. Finally, there is app.listen.

1	app.listen(port, () => {
2	console.log(`Example app listening on port ${port}`);
3	});

This starts the server at the port specified by port. The callback function is executed after the server starts.

PHP

There are two keys to processing requests the REST way. The first key is to initiate different processing depending on the HTTP method—even when the URLs are the same. In PHP, there is a variable in the $_SERVER global array that determines which method has been used to make the request:

1	$_SERVER['REQUEST_METHOD']

This variable contains the method name as a string—for instance 'GET', 'PUT', and so on.

The other key is to know which URL has been requested. To do this, we use another standard PHP variable:

1	$_SERVER['REQUEST_URI']

This variable contains the URL starting from the first forward slash. For instance, if the host name is example.com, 'https://example.com/' would return '/', while 'http://example.com/test/' would return '/test/'.

Let's first attempt to determine which URL has been called. We only consider URLs starting with 'clients'. All others are invalid.

$resource = array_shift($paths);

if ($resource == 'clients') {
    $name = array_shift($paths);

    if (empty($name)) {
        $this->handle_base($method);
    } else {
        $this->handle_name($method, $name);
    }

} else {
    // We only handle resources under 'clients'
    header('HTTP/1.1 404 Not Found');
}

We have two possible outcomes:

The resource is the clients, in which case, we return a complete listing.
There is a further identifier.

If there is a further identifier, we assume it is the client's name and again forward it to a different function, depending on the method. We use a switch statement, which should be avoided in a real application:

switch($method) {
  case 'PUT':
      $this->create_contact($name);
      break;

  case 'DELETE':
      $this->delete_contact($name);
      break;

  case 'GET':
      $this->display_contact($name);
      break;

  default:
      header('HTTP/1.1 405 Method Not Allowed');
      header('Allow: GET, PUT, DELETE');
      break;
  }

Response Codes

You might have noticed that the example application uses the PHP header(), passing some strange-looking strings as arguments. The header() function prints the HTTP headers and ensures that they are formatted appropriately. Headers should be the first thing in the response, so you shouldn't output anything else before you are done with the headers. Sometimes, your HTTP server may be configured to add other headers, in addition to those you specify in your code.

Headers contain all sorts of meta information—for example, the text encoding used in the message body or the MIME type of the body's content. In this case, we are explicitly specifying the HTTP response codes. HTTP response codes standardize a way of informing the client about the result of its request. By default, PHP returns a 200 response code, which means that the response is successful.

The server should return the most appropriate HTTP response code; this way, the client can attempt to repair its errors, assuming there are any. Most people are familiar with the common 404 Not Found response code, but there are a lot more available to fit a wide variety of situations.

Keep in mind that the meaning of an HTTP response code is not extremely precise; this is a consequence of HTTP itself being rather generic. You should attempt to use the response code that most closely matches the situation at hand. That being said, don't worry too much if you can't find an exact fit.

Here are some HTTP response codes which are often used with REST:

200 OK

This response code indicates that the request was successful.

201 Created

This indicates the request was successful and a resource was created. It is used to confirm the success of a PUT or POST request.

400 Bad Request

The request was malformed. This happens especially with POST and PUT requests, when the data does not pass validation or is in the wrong format.

404 Not Found

This response indicates that the required resource could not be found. This is generally returned to all requests which point to a URL with no corresponding resource.

401 Unauthorized

This error indicates that you need to perform authentication before accessing the resource.

405 Method Not Allowed

The HTTP method used is not supported for this resource.

409 Conflict

This indicates a conflict. For instance, you are using a PUT request to create the same resource twice.

500 Internal Server Error

When all else fails; generally, a 500 response is used when processing fails due to unanticipated circumstances on the server side, which cause the server to error out.

Exercising the Example Application

Let's begin by simply fetching information from the application. We want the details of the client, 'jim', so let's send a simple GET request to the URL for this resource:

1	curl -v http://localhost:80/clients/jim

This will display the complete message headers. The last line in the response will be the message body; in this case, it will be JSON containing Jim's address (remember that omitting a method name will result in a GET request; also replace localhost:80 with the server name and port you are using).

Next, we can obtain the information for all clients at once:

1	curl -v http://localhost:80/clients/

Then we create a new client, named Paul:

1	curl -v -X "PUT" http://localhost:80/clients/paul -d '{"address":"Sunset Boulevard" }' -H 'content-type: application/json'

Now you will receive a list of all clients containing Paul as a confirmation.

Finally, to delete a client:

1	curl -v -X "DELETE" http://localhost:80/clients/anne

You will find that the returned JSON no longer contains any data about Anne.

If you try to retrieve a non-existent client, for example:

1	curl -v http://localhost:80/clients/jerry

You will obtain a 404 error, while, if you attempt to create a client which already exists:

1	curl -v -X "PUT" http://localhost:80/clients/anne

You will instead receive a 409 error.

Conclusion

It's important to remember that HTTP was conceived to communicate between systems that share nothing but an understanding of the protocol. In general, the fewer assumptions beyond HTTP you make, the better: this allows the widest range of programs and devices to access your API.

I used PHP in this tutorial because it is most likely the language most familiar to Envato Tuts+ readers. That said, PHP, although designed for the web, is probably not the best language to use when working in a REST way, as it handles PUT requests in a completely different fashion than GET and POST.

Beyond PHP and Node.js, you might consider the following:

Go for higher-performance web servers.
The various Ruby frameworks (Rails and Sinatra).
Python, as it has good REST support. Plain Django and WebOb or Werkzeug should work.

Among the applications which attempt to adhere to REST principles, the classic example is the Atom Publishing Protocol, though it's honestly not used too often in practice. For a modern application, which is built on the philosophy of using HTTP to the fullest, refer to Apache CouchDB.

Have fun!

This post has been updated with contributions from Jacob Jackson. Jacob is a web developer, a technical writer, and a frequent open-source contributor.