Image with a tablet and a big API heading

RESTful API Authentication

While working on the public API for our product Phazer (unfortunately, the API is not released yet, so can not check it out at the moment), we came across a topic probably everybody has to deal with who is creating a REST(ful) API that delivers user related content or perform user related actions: the user authentication and, to be more specific, the security of this process.

Simply said the authentication process is responsible for determining who is making an API request. Often this process is followed by an authorization that evaluates if a user has the permissions to perform a specific action. The authorization for sure completely depends on your product respectively the API and what content it offers and will not be further discussed in this post. We will focus on the topic how to identify the user who is performing an API request and the security behind this process.

The authentication of an API request requires the same data as you know from common login scenarios. While you there need a user name/email and password the credentials for the API authentication are often called something like id and secret and they are provided to you by the organization that offers the API . So for example the API of PayPal uses client_id and secret, Amazon uses AWSAccessKeyId and AWSSecretAccessKey and Facebook uses app-id and app-secret. This data pair is send to the API, there it will be evaluated and, if it is correct, the user requesting the API can be authenticated. The question now is what is the best way to send this credentials to the API?

Basic Authentication

The most simple way you can imagine is to send your id and secret to the API as plain or encoded text. A common way for this scenario is to use the HTTP header in the so called HTTP Basic Authentication. There you simply deliver your credentials by creating a string that contains your id and secret separated by a colon ("id:secret"), Base64 encode this string and append it to the HTTP header of your API request. So for example let's assume that your id is "exampleId" and your secret is "exampleSecret", than you will encode the string "exampleId:exampleSecret" in Base64, which will result in "ZXhhbXBsZUlkOmV4YW1wbGVTZWNyZXQ=". Then an example request could look like this:

GET /example HTTP/1.1
Host: api.example.com
Authorization: Basic ZXhhbXBsZUlkOmV4YW1wbGVTZWNyZXQ=

Here we are performing a GET request to the "/example" resource of the host "api.example.com". The encoded string is added to the Authorization field of the HTTP header after the keyword "Basic" (which is not strictly required , but it is common to use it to indicate that you are using basic authentication). Don't be confused by the word "Authorization", it is the standard name of this header part even it carries authentication information and no authorization information. When the API server receives this request it simply can decode the string, extract the id and secret from it, compare it to its database and authenticate the user.

As you can see, this approach is quite simple. But its simplicity results in a disadvantage from a security point of view. Because your credentials are only encrypted, your request is vulnerable by man-in-the-middle-attacks, as every eavesdropper can retrieve your credentials when he is listening your requests. That is why you have to protect this authentication type by using a secure connection (SSL/TLS) under all circumstances! Besides the method of providing your credentials in the HTTP header, there is also the possibility to send them as data in the request body. But as they are sent in plain you will face the same security issue too.

This basic authentication is used in OAuth 2.0, as you can read in the specification. OAuth 2.0 itself is implemented in the APIs of some famous companies like Facebook or PayPal. The described security issue is one of the main topics developers complain about when talking about OAuth 2.0 (also check this post for more information).

Hashed Authentication

A way of preventing the above mentioned security problem is to use hashing for your API authentication requests. Commonly this is called HMAC (Hash-based message authentication code). Such a request could have the following basic structure:

GET /example HTTP/1.1
Host: api.example.com
Authorization: hmac <id>:<signature>

Compared to the above shown basic authentication request we have changed the "Basic" keyword to "hmac" (you can choose whatever you want, even nothing is okay). After that we provide our id followed by a colon and the signature, which is a hashed string. So how is that signature created?

At first we take the request method we are using and the API resource we are requesting and concatenate those by using a plus sign. In our example this would be "GET" and "/example" resulting in "GET+/example". Then we perform the hashing method on this string. The appropriate PHP function for example can be found on php.net and looks like this:

string hash_hmac ( string $algo , string $data , string $key [, bool $raw_output = false ] )

For our example we could use the following implementation:

$hashedString =  hash_hmac("sha256" , utf8_encode("GET+/example") , "exampleSecret" );
$hashedString = "0243bba9b2b63472c304add00e0105c607f8a9172f321ec6704986ed6972da90";

As you see, we are using the SHA-256 hashing algorithm, which is a common use case, UTF-8 encode our above created string and use our secret as the hashing key. This creates a 64 char long string consisting of numbers and lower case letters. With that we are almost done creating our request signature, in a last step we simply Base64 encode the hashed string:

$signature =  base64_encode($hashedString);
$signature =  base64_encode("0243bba9b2b63472c304add00e0105c607f8a9172f321ec6704986ed6972da90");
$signature =  "MDI0M2JiYTliMmI2MzQ3MmMzMDRhZGQwMGUwMTA1YzYwN2Y4YTkxNzJmMzIxZWM2NzA0OTg2ZWQ2OTcyZGE5MA==";

With that our final request would look like:

GET /example HTTP/1.1
Host: api.example.com
Authorization: hmac exampleId:MDI0M2JiYTliMmI2MzQ3MmMzMDRhZGQwMGUwMTA1YzYwN2Y4YTkxNzJmMzIxZWM2NzA0OTg2ZWQ2OTcyZGE5MA==

So what is the advantage of this authentication method? Obviously we are sending our id in plain text as we did in the basic authentication example above. But our secret itself it not sent, but is used to generate the signature, which holds a hashed string. The crucial point is that a hash can not be reversed. That means even if someone eavesdrops your request, he will just get the hashed string, but can not retrieve your secret from that. That means that this authentication method is not vulnerable to man-in-the-middle-attacks like the basic authentication was and will work safely even with plain HTTP requests. Nevertheless the usage of SSL is recommended in this case too.

An open question is how the API server now can use the passed information to authenticate the user. As the API creates the credentials for its users, it also has stored those credentials in its own database. When a request is reaching out to the API, it knows what user is trying to perform the request, because the id is sent as plain text in the HTTP header. So the server can get the appropriate secret from its database and, as it for sure also knows the requested resource and the request method, can create the signature itself. Then it will compare this self created signature with the signature passed with the request. If they match, then the user has successfully authenticated, otherwise the authentication failed.

If you think a little further, the so far discussed solution of hashed authentication still has a security issue. For sure, an eavesdropper can not create the secret from a captured request. But he can simply perform the request again and again. For example if he would capture your request to the Facebook API (and Facebook would use such an authentication approach) where you were asking for your news feed, he could perform this request again and again as often as he wants and could get the data from your personal news feed. To prevent this scenario, the information sent to the API for an hashed authentication is often extended by some additional data.

A first step to prevent the usage of an captured API request again and again can be the addition of the current time to the HTTP header in the Date field. Let's assume that the current time would be "24 Dez 2017 16:00:00", then the HTTP header could look like this:

GET /example HTTP/1.1
Host: api.example.com
Date: 24 Dez 2017 16:00:00
Authorization: hmac exampleId:<signature>

Furthermore this time string has to be integrated in the creation process of our signature. Otherwise it would have no effect on the security of our request, because an eavesdropper could simply change it. This is commonly done by simple adding the time string to the string that is hashed. In our example we previously used the string "GET+/example" which now is extended to "GET+/example+24 Dez 2017 16:00:00" and we will receive a new hashed string and a new signature:

$hashedString =  hash_hmac("sha256" , utf8_encode("GET+/example+24 Dez 2017 16:00:00") , "exampleSecret" );
$hashedString = "b74af3b360566176b21a23fa37cd2c97ada44b8fb3d95c126ac7918be42b076e";
$signature =  base64_encode($hashedString);
$signature =  base64_encode("b74af3b360566176b21a23fa37cd2c97ada44b8fb3d95c126ac7918be42b076e");
$signature =  "Yjc0YWYzYjM2MDU2NjE3NmIyMWEyM2ZhMzdjZDJjOTdhZGE0NGI4ZmIzZDk1YzEyNmFjNzkxOGJlNDJiMDc2ZQ==";

With that we now can create our API request:

GET /example HTTP/1.1
Host: api.example.com
Date: 24 Dez 2017 16:00:00
Authorization: hmac exampleId:Yjc0YWYzYjM2MDU2NjE3NmIyMWEyM2ZhMzdjZDJjOTdhZGE0NGI4ZmIzZDk1YzEyNmFjNzkxOGJlNDJiMDc2ZQ==

The API server now at first checks the time string and rejects every request (even if the signature is correct) that is outside a specific time interval (for example 10 minutes) from the current time. If the passed time is okay, than the API will again create the signature itself by using the date string in the Date field of the HTTP header and compare it to the passed signature and either the authentication will succeed or fail. With that implementation for example the above mentioned eavesdropper could only get your Facebook news feed in a 10 minute interval (or whatever is specified by the API) after your own request to the API. Because if he would change the time string in the Date field of the HTTP header to fit the 10 minutes interval, the created signature would not be correct anymore. And he is not able to change the signature, because he does not know the secret.

To improve this further, the request can contain an unique string that guarantees that every API request can performed only once. The basic structure of such an request could be:

GET /example HTTP/1.1
Host: api.example.com
Date: 24 Dez 2017 16:00:00
Authorization: hmac <id>:<uniqueString>:<signature>

This unique string is created by the user who performs the API request and it can be any arbitrary combination of letters and numbers, but, as the name says, it has to be unique. For that purpose it is reasonable to choose an adequate length for that string. Let's assume we are using a 32 char string with the value of "fa0bb3e3ac827d997b198adfcc0a1538". As every char can be one of the 26 letters in the alphabet or one number from 0 to 9, there are 3632 = 6.33 * 1049 possible combinations for this string, which means that the chance is not too bad that you created an unique one. If the specific API implements the above discussed time stamp checking method, than your unique string furthermore has to be unique only in the used time interval. Adding our example unique string to our request header will result in:

GET /example HTTP/1.1
Host: api.example.com
Date: 24 Dez 2017 16:00:00
Authorization: hmac exampleId:fa0bb3e3ac827d997b198adfcc0a1538:<signature>

Simply adding this unique string to the Authorization field in plain text will not give us any benefit, because the eavesdropper simply could create his own unique string and use all other parts of the captured request to perform it again. So we also have to integrate this unique string in the creation process of our signature. Like we did with the time string before, we simply add the unique string to the string we are going to hash, which will result in new signature:

$hashedString =  hash_hmac("sha256" , utf8_encode("GET+/example+24 Dez 2017 16:00:00+fa0bb3e3ac827d997b198adfcc0a1538") , "exampleSecret" );
$hashedString = "c982aa6becd757351fb8e6f02c5087e8c6ff8af30403bcedca64607719538488";
$signature =  base64_encode($hashedString);
$signature =  base64_encode("c982aa6becd757351fb8e6f02c5087e8c6ff8af30403bcedca64607719538488");
$signature =  "Yzk4MmFhNmJlY2Q3NTczNTFmYjhlNmYwMmM1MDg3ZThjNmZmOGFmMzA0MDNiY2VkY2E2NDYwNzcxOTUzODQ4OA==";

With that now we have our complete API request:

GET /example HTTP/1.1
Host: api.example.com
Date: 24 Dez 2017 16:00:00
Authorization: hmac exampleId:fa0bb3e3ac827d997b198adfcc0a1538:Yzk4MmFhNmJlY2Q3NTczNTFmYjhlNmYwMmM1MDg3ZThjNmZmOGFmMzA0MDNiY2VkY2E2NDYwNzcxOTUzODQ4OA==

The API server will now at first check the time string like explained above. If that is okay, then it will check the unique string. For that purpose, the API has to store all incoming unique keys (at least for the used time interval) in its database. When a new request is reached out to the API, it will check if the passed unique string has been used before. If this is the case, then it will reject the request. If the unique string was not used before, then again the API will use it together with all the other information to create the signature itself and to verify if the user authentication is successful or not. With this approach is guaranteed, that every request can only be performed once. So even if someone eavesdrops your API request, he can neither get all necessary credentials from it nor perform the request again.

Hashed authentication is used in OAuth 1.0 (see the specs), which for example is used by Amazon. Even it is more complex, from a security point of view, it is for sure a better approach then the basic authentication used in OAuth 2.0, because it does not rely on a secure connection while performing authentication requests.

One thought on “RESTful API Authentication

Leave a Reply

Your email address will not be published. Required fields are marked *