Azure Active Directory: Verify issued JWT in node.js

Microsoft Azure Active Directory is a steady growing identity- and access-management platform which can be used from developers to swap out user management, authentication and authorisation. Azure Active Directory offers several end points and authentication protocols e.g. SAML2, WS-FED or oAuth2. A widely adopted protocol is oAuth2 which ends up with an issued JWT token. This article describes how the JWT token issued by Azure Active Directory can be verified in a node.js application.

Anatomy of a JWT
A JWT token is a non-encrypted digitally signed JSON payload which contains different attributes (claims) to identify the user.

jwt

The header is very static and should be used to identify which algorithm was used for the digital signing. This signing algorithm needs to be used to verify the digital signature in the node.js application later on. The payload contains the JSON object with all the claims and information which can be used to verify the user. Trusting this content is only possible when the digital signature of the token is valid and some standard claims, e.g. the issuer or the audience are verified. Otherwise it could be that someone else generated a JWT (man in the middle attack) to get unauthorised access to your application. The signature is the last part of the JWT and needs to be used for verification of the payload. This signature was generated with the algorithm described in the header to prevent unauthorised access.

How AAD issues a token
Azure Active Directory offers every developer the possibility to create applications. If this application is a multi tenant application, other active directory administrators are able to install this application into their directory. At the end of the day an Azure Active Directory application can live in many tenants. Every tenant in the AAD ecosystem has an own set of keys and certificates which are used to sign cryptographic messages. This means that when a directory with the Id “DIRAAA” issues a token for an application the issuer would be

https://sts.windows.net/DIRAAA/

If a directory with the Id “DIRBBBB” issues a token for the same application the issuer would be

https://sts.windows.net/DIRBBB/

So the node.js application needs to verify if the token was issued from the directory we expect. Another side effect of this is, that Azure Active Directory uses different keys for every tenant to issue tokens. This means that the validation code needs to get the right verification key for the token. Microsoft uses RS256 for JWTs issued via oAuth2, so the right certificate needs to be downloaded from somewhere.

Download the right certificates
Microsoft publishes the certificates (public portion of the signing keys) as part of the well known OpenId configuration. It can be downloaded here:

https://login.windows.net/<<tenantid>>/.well-known/openid-configuration

The result is a JSON payload which contains the jwks_uri that should be used to download the certificates. Behind the URI several certificates are available and we currently don’t know which the right one is. The simplest way would be to do a little brute force and verify the JWT against every certificate.

Verification Strategy
Inventing code which is able to verify any AAD issued JWT, without knowing if the application is a multi-tenant or single tenant application is the goal. The following process describes a possible algorithm which can be implemented with existing JWT libraries very easily:

  1. Decode the token to extract the tenant-id because the tenant-id is part of the payload, stored as tid-claim. (!!! Currently we don’t know if we can trust this information !!!)
  2. Download the signing certificates from the well known openid configuration endpoint Microsoft provides. The end point url can be generated with the help of the tenant-id.
  3. Verify the JWT with RS256 against the downloaded certificates. For this, every existing JWT module can be used.
  4. After the token is validated check if the iss-claim contains the same value we expect from the tenantid.

After this process the system verifies the token and we know that this token was issued by Azure Active Directory for the described tenant. This means we are now able to rely on this information.

Node.js integration
All described steps are implemented in a small node package which allows to verify a given token as long as the node application has internet access and can download the certificates. The component can be installed via:

npm install azure-ad-jwt –save

A basic example to verify a given token could look like this:

var aad = require('azure-ad-jwt');
var jwtToken = '<<yourtoken>>';
aad.verify(jwtToken, null, function(err, result) {
if (result) {
console.log("JWT is valid");
} else {
console.log("JWT is invalid: " + err);
}
});

view raw
verify-aad-token.js
hosted with ❤ by GitHub

The component is currently not intended to be an express middleware but it’s easy to extend it that way. A good starting point is the express-jwt middleware which should be used as starting point. The current implementation does not work with certificate caching, so when your system has a huge amount of verification requests it makes no sense to download the certificates during every request. This can be done once when the application starts or in a small cache implementation which invalidates the certificate when it was expired as well.

I hope this helps everybody in the node.js space to integrate Azure Active Directory very fast and easily. The described component is used from the Azure Cost Monitor in the production environment so feel free to integrate the package also in your real world applications.

So when you have any questions, feel free and leave a message on this blog.

Azure App Services: Restart your node-WebJobs during GitDeploy

With Azure App Services (aka. Azure WebSites), the Microsoft Azure cloud offers a great, highly scalable and simple way to host cloud and SaaS services. Besides ASP.NET, several other platforms and languages are supported, e.g. node.js, Python or Java. I personally prefer hosting services written in node.js on this nice managed service of Microsoft.

A common problem for web-services are background jobs like e.g. sending out e-mails or calculating some sales numbers once a day. This use-case can be addressed with Azure WebJobs which are running on the same instance as the web service itself. Jamie Espinosa described the behaviour of WebJobs on an Azure Friday very well. Azure Friday is BTW hosting a whole series about Azure WebJobs, so check it out to get more information.

Normally when deploying a web service into the Azure WebSite the associated WebJobs will be restarted out of the box. A special thing of node.js based Azure WebJobs is that only when the run.js file is changed the WebJob will be restarted. This means when the system just changes an other module or updates the npm dependencies no restart will be enforced.

The whole deployment is based on the Kudu-Project and this project offers so called Post-Deployment-Action-Hooks to trigger a simple script right after the successful deployment of the sources. When ever the run.js file becomes touched the system just restarts the web service, so the solution for this deployment issue was to write a short batch which touches all run.js files:

@echo off

echo Restarting all WebJobs
for /R ..\wwwroot\App_Data\jobs %%G IN (*run.js) DO echo Touching %%G
for /R ..\wwwroot\App_Data\jobs %%G IN (*run.js) DO touch %%G

exit 0

This script can be registered as Post-Deployment-Action-Hook via FTP at every Azure WebSite. Just copy the file to the following location:

deployment-hooks

This works fine but after all there is still one piece missing: How to get the deployment hooks deployed with git themselves? There are several options to reconfigure the deployment hook directory but I was not able to figure this out. So when you have an idea, feel free and leave a message to discuss any options.

Build your own Twitter – Part 3 – Azure Timeline Service for Node.js

The last part of this article series described the principles of Twitter-like services based on Azure Storage Tables. This part now describes the structure of a new node module which acts as a timeline service. This service can be used very easily in existing node projects.

To integrate this node module just install the azure-timeline-service via node package manager. This integrates everything that is required automatically:

npm install azure-timeline –save

The module allows to post events to a specific user timeline and the timeline of all followers. The following snip-let illustrates it:

var user = azureTimelineService.createSubject(“<>”, “<>”);

user.postEvent(‘login’, { timestamp: new Date() }).then(function() {
console.log(“DONE”);
});

Every method works asynchronous based on promises. Following another user is as simple as posting an event to a timeline

user.follow(user01).then(function() {
console.log(“DONE”);
})

Following a user means all events this user posts to a timeline will be posted to the followers timeline as well. Last but not least loading a timeline is important. The system returns currently all events from a timeline which is a point of change in the future:

user.loadTimeline().then(function(events) {
console.log(events);
})

All samples are implemented in the sample file of the Azure Timeline project here. Any questions? Feel free to open an issue at GitHub or just stay in touch via this block.

Build your own Twitter – Part 2 – Azure Table Structures

Azure Table Store offers a very important scalability feature which should be used when working with timelines. The partitionkey in every table allows Microsoft to put entities on different servers. Let’s recheck the limits of Azure Table Store to make a right decision (http://azure.microsoft.com/en-us/documentation/articles/azure-subscription-service-limits):

  1. Azure Table Store returns 1000 entities per page. If the result contains more entities the client needs to query several times -> A timeline service should never page to render the first timeline
  2. Azure Table Store returns 2000 entities with 1KB of size per second as a guaranteed SLA –> A timeline service should never request more data per page to stay performant
  3. Azure Table Store allows to store up to 500TB per storage account, it could be stored in one table or different –> A timeline service should be able to handle several storage accounts, at least theoretically.

With all this limitations in mind it’s possible to build a table structure for the timeline service as follows:

  1. timelines
    The timlines table contains all timelines the system has registered. The partitionkey of this table is timeline identifier so every subjects timeline can be stored on different nodes. The partitionkey should be a key generated from the subjects  identification, e.g. liveid{{UID of LiveId-Token}}. This prevents the system to lookup an other table to get the timeline identifier when the subject tries to render them.In addition the event which should be stored on several timelines can be identified by his event identifier as the row key. This allows the system to also implement removal jobs because a multiple stored event can be identified as the single one.
  2. subjectFollowers
    the subjectFollowers are a list of subjects following a specific subject. The partitionkey of this table is also the subject identifier so it’s easy to get all followers of a subject. In addition the row key becomes important because it identifies the subject who is following someone else. This gives a system the option to find all followers of a specific subject and all subjects a specific subject is following very fast. Works well in both directions.

This simple data structure allows the service to handle hundreds of different timelines and relations. Especially the background worker can now identify on which timeline the event needs to be stored.

followers

Last but not least, when it comes to requesting the timeline content it will only be returned in pages of 250 elements to stay healthy with the performance. An other page can be requested at any time when the user starts paging.

Build your own Twitter – Part 1 – A timeline service with Azure Table Store

The Azure Cost Monitor and many other cloud services I’m working on, needed a timeline service to present aggregated events and actions similar to an audit trail. When I was thinking about this, it came to my mind that the requirements I had were very similar to a Twitter feed but with less features.

Let’s recap what is needed to build a timeline service in general:

  1. The Timeline
    The Timeline is associated with a subject and contains tons of events happened and triggered by other subjects the timeline owner follows.
  2. The Event
    The Event is the incident that happens in the real world and is stored in different timelines. This means every follower of a subject will get the event in his own timeline. So a single event can be stored in many different timelines.
  3. The Subject
    The Subject is someone or something which triggers an event. Normally it is a natural person with an own timeline but it could also be a piece of hard/software.
  4. The Target(s)
    The Targets are subjects with a timeline who are following other subjects. This means every subject becomes a follower or target as soon as it is following an other subject. Posting an event then means sending the message to the timeline of every subject following the sender.

The following picture should give a good overview about the entities in the timeline service:

Screen Shot 2015-03-21 at 10.40.15

Starting with this definitions in mind it’s possible to identify components needed for building a timeline service:

  • Storage for the timeline 
    The storage for the timeline needs to be able to offer a very fast read performance and an OK write performance. The performance should especially not be dependent from the amount of data in the system. I chose Azure Table Store for this because I can store the information in different partitions and get a clear read performance SLA for every partition. In addition to this it’s totally cheap and payable – also for startups.
  • Access broker to the storage 
    Normally NoSQL storage which can be used for timeline access needs a little helper to manage access. In general a RESTful web-service acts as an access broker and hands out pre-signed links to the timeline content. This ensures that no timeline data needs to go through the timeline service at all. Only the pre-authorised access links will be generated for the system. This also means that at the end the client SDK needs to handle the raw data for the timeline. This makes it a bit more complicated but it lets the performance stay in a good range. An other important operation needs to be implemented in the Access broker as well: Posting events to the subjects followers is a slow and complex operation the system needs to implement with a direct server call normally executed asynchronous with the help of a worker job.
  • Metadata storage for subject and targets
    Last but not least the combination of subject & targets needs to be stored somewhere in the backend. Azure Table Store can be used for this as well. All the operations to create a follow-ship can be implemented in the timeline service as well but not the rights & permission checks to post or create relations because the timeline service should be used machine-2-machine with API tokens.

The following graphic shows the technical architecture of a good scalable timeline service based on Azure services:

arch-timeline

The next part of this tutorial will focus on the correct and scalable table structure based on Microsoft Azure Table Storage.

Update: Deploy AngularJS-Apps to Azure WebSites with Codeship

A couple of weeks ago I wrote a tutorial on how to deploy an Angular.js application into Azure WebSites. I explained why the simple GitHub deployment that comes with Azure is not usable and how Codeship as shipping service can do the magic for every team.

Since this article I started using Codeship more extensive and since it’s a part of my daily business I don’t want to miss it at all. Here @Matrix42 HackWeek, Codeship was the most chosen solution to implement AdHoc continuous deployment. I would not be suprised when we replace our traditional TFS build agents in the next months 🙂

To make it more intuitive and easy for everybody who works with node and a javascript task runner, I decided to transform the illustrated deployment script into a node module. The Azure-Deploy module is super simple to integrate in existing NPM driven projects and can be added to existing javascript tasks as well. At the end the system offers a simplified usage like this:

grunt deploy:production

This result is more simple to integrate into the deployment scripts of Codeship than everything else. The shell script of course works but this component gives you the freedom to stay with the current task runner of your choice. To get more information visit the github page for this project: https://github.com/dei79/node-azure-deploy

azure-queue-client: build azure queue based workers in node.js

Microsoft Azure offers a very powerful and cheap queueing system based on Azure Storage. As a node developer the challenge is to build a simple to use system which is able to consume messages from the azure queue. The Azure Cost Monitor is for instance using this module to process all costs analytic tasks in the backend.

The module azure-queue-client is able to implement this in a couple of simple steps. It supports multiple workers in different processes and on different machines. The following example illustrates the usage:

// config with your settings
var qName = ‘<<YOURQUEUENAME>>’;
var qStorageAccount = ‘<<YOURACCOUNTNAME>>’;
var qStorageSecret = ‘<<YOURACCOUNTSECRET>>’;
var qPolling = 2;

// load the modules
var queueListener = require(‘azure-queue-client’).AzureQueueListener;
var q = require(“Q”);

// establish a message handler
queueListener.onMessage(function(message) {
  var defer = q.defer();

console
.log(‘Message received: ‘ + JSON.stringify(message));
defer.resolve();

return
defer.promise;
});

// start the listening
queueListener.listen(qName, qStorageAccount, qStorageSecret, qPolling, null);

Developers who are using the Azure Scheduler might recognize that the payload of the scheduler is encapsulated in a XML wrapper. This XML wrapper can be handled by the module as well so that it doesn’t matter if the message comes from an other queue client or the Azure Scheduler.

This module makes writing job workers in node, hosted on Azure or any other cloud provider a breeze.

SEO – Is your Azure WebSites hosted AngularJS App ready for Google :-)

Every AngularJS application is just a website which is generated by executing javascript. The google crawler and also other crawlers are not able to collect information from these sites. To handle this problem a couple of services like AjaxSnapshots or Prerender.io are trying to fill the gap. Basically these services are generating a snapshot of a website without any javascript in it. Whenever the search engine visits the page, the system delivers a plain html page without any javascript. Many technical details on how search engines crawl a page can be found here. Users who are hosting on Azure WebSites and trying to use this kind of pre-rendering tools may stuck with some configuration issues.

The required rewrite rules for the web.conf are written very fast or can be found in the AjaxSnapshots documentation:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <rewrite>
            <rules>
                <rule name="AjaxSnapshotsProxy" stopProcessing="true">
                    <!-- test all requests -->
                    <match url="(.*)" />
                    <conditions trackAllCaptures="true">
                        <!-- only proxy requests with an _escaped_fragment_ query parameter -->
                        <add input="{QUERY_STRING}" pattern="(.*_escaped_fragment_=.*)" />
                        <!-- used to capture the scheme/protocol for use in the rewrite below --> 
                        <add input="{CACHE_URL}" pattern="^(https?://)" />
                    </conditions>
                    <!-- send the request to the AjaxSnapshots service -->
                    <action type="Rewrite" 
                    url="http://api.ajaxsnapshots.com/makeSnapshot?url={UrlEncode:{C:2}{HTTP_HOST}:{SERVER_PORT}{UNENCODED_URL}}&amp;apikey=<YOUR API KEY>" 
                    logRewrittenUrl="true" appendQueryString="false" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>

An example for Prerender.io can be found here.

After applying the web.conf to the website, the browser returns directly a 404 which means that the website behind the target URI is wrong. This is because Azure Websites enabled the redirect module but rewriting URLs to external target requires a revers proxy module additionally. This module which is part of the application request routing in IIS is not activated in Azure WebSites by default. Thanks to the following nice trick it’s possible to use the reverse proxy module also in Azure WebSites:

http://ruslany.net/2014/05/using-azure-web-site-as-a-reverse-proxy/

As usual all technical frameworks, components or services described in this blog are used in production for several weeks in different applications, e.g. the Azure Cost Monitor. These steps should help to get every AngularJS application ready for google and other bots when the site is hosted on Azure WebSites.

Cache Fighters Part 3 – A real life example combined with grunt-ng-constant

In the last two parts of this article series, the caching behaviour of AngularJS applications was described and a solution for all issues in a typical app was delivered.
The last part now is focussed on developing a real life example which demonstrates the authors favourite solution.

When following this tutorial Yeoman should be installed on the machine. In addition the angular generator of Yeoman is required.

Generate a simple application
Yeoman initially generates a skeleton of a AngularJS application. Firing the following command starts the generation process:

yo angular testapp

All this should be done in a directory called testapp because Yeoman is not generating a subdirectory automatically. During the process Yeoman asks some details which need to be answered as follows:

yeoman-answers

Integrate ngHelperDynamicTemplateLoader
The dynamic template loader is the base component in AngularJS applications to handle the caching issues for views and partials. It can be installed with the following command line:

bower install ng-helper-dynamic-template-loader –save

After that the http request interceptors need to be registered which can be done by adding the following lines in the configuration section of the application, typically implemented in the app.js file:

'use strict';

angular
  .module('testappApp', [
    'ngAnimate',
    'ngCookies',
    'ngResource',
    'ngRoute',
    'ngSanitize',
    'ngTouch',
    'ngHelperDynamicTemplateLoader'
  ])
  .config(function ($routeProvider, $dynamicTemplateLoaderProvider) {

    $dynamicTemplateLoaderProvider.registerInterceptors();

    $routeProvider
      .when('/', {
        templateUrl: 'views/main.html',
        controller: 'MainCtrl'
      })
      .when('/about', {
        templateUrl: 'views/about.html',
        controller: 'AboutCtrl'
      })
      .otherwise({
        redirectTo: '/'
      });
  });

Generate a deployment UUID
The custom caching strategy will be based on a deployment UUID. Whenever the deployment UUID is changed, the system will change the URI for the views & partials. There are two additional modules which are required. grunt-ng-constant is responsible for generating an individual build-related configuration file. The node-uuid module is able to generate a new deployment UUID whenever it is required. The modules can be installed via npm as follows:

npm install grunt-ng-constant –save-dev
npm install node-uuid –save-dev

After that a new grunt task is available and the following grunt configuration needs to be generated in the configuration file. The config section is building an app-env.js file during every build which contains the needed deployment UUID:

ngconstant: {
  options: {
    dest: '<%= yeoman.app %>/scripts/app-env.js',
    wrap: '"use strict";\n\n {%= __ngModule %}',
    name: 'app.env'
  },
  dist: {
    constants: {
      envDeployment: {
        deploymentUUID: uuid.v4()
      }
    }
  },
  serve: {
    constants: {
      envDeployment: {
        deploymentUUID: uuid.v4()
      }
    }
  }

}

Don’t forget to require the uuid module somewhere at the start of the existing grunt file:

var uuid = require('node-uuid');

Last but not least the grunt-ng-constant task needs to be added in the build-, test- and serve-tasks to ensure that a config file is generated:

grunt.registerTask('build', [
  'clean:dist',
  'ngconstant:dist',
grunt.task.run([
  'clean:server',
  'ngconstant:serve',

grunt.registerTask('test', [
  'clean:server',
  'ngconstant:serve',

When executing the configuration a module which is called “app.env” is generated and needs to be added in the dependency list of the AngularJS application:

angular
  .module('testappApp', [
    'app.env',

Additionally the “app-env.js” needs to be registered in the index.html so that it will be loaded during application start:

<!-- build:js({.tmp,app}) scripts/scripts.js -->
<script src="scripts/app-env.js"></script>

Build a custom caching strategy
Everything is settled for the deploymentUUID based caching strategy. In the ngHelperDynamicTemplateLoader every caching strategy is implemented as a specific service.

yo angular:service CustomTemplateCaching

The following service should be added to the application to ensure that the generated deploymentUUID is used:

‘use strict’;

angular.module(‘testappApp’).service(‘CustomTemplateCaching’, [‘envDeployment’, function(envDeployment) {
var self = this;
self.processRequest = function(requestConfig) {
if (requestConfig.url.indexOf(‘?’) === -1) {
requestConfig.url = requestConfig.url + ‘?v=’ + envDeployment.deploymentUUID;
} else {
requestConfig.url = requestConfig.url + ‘&v=’ + envDeployment.deploymentUUID;
}
return requestConfig;
}
}]);

The service can be assigned to the template loader as a caching strategy in the run region of the application

.run([ '$dynamicTemplateLoader', 'CustomTemplateCaching', function($dynamicTemplateLoader, CustomTemplateCaching) {
  $dynamicTemplateLoader.setCachingStrategyService(CustomTemplateCaching);
}]);

Finally after starting the application with the command:

grunt serve

the browser will load all of the templates with a special version parameter in the url. The version parameter contains the deploymentUUID which means the system is using the cache as long as nothing has changed but after an update the system is using a new uncached version of the views.

caching

The illustrated caching provider can be improved when using a grunt task which is building hash keys for every HTML file. The hash keys can then be used in the request so that HTML files would be reloaded from the server when the specific view has changed which is a more granularly approach.

The solution stays with the eager loading approach of views & partials which works good in bigger applications. For smaller and midsize applications compiling everything in a single javascript file feels ok as well.

This article should help to solve the views and partial caching issues in all AngularJS applications. The described approach is successfully used in production applications, e.g. Azure Cost Monitor over the last weeks. The sample application developed in this article can be found here.

Cache Fighters Part 2 – How to handle eager loaded partials/views

The previous part of this article series described the general behaviour of an AngularJS app when it comes to web caching. Using Yeoman and the generated grunt tasks solved many topics on the existing caching list. This article is focused on the different strategies to handle views & partials.

Views & partials are HTML files which are loaded from the AngularJS application during the first use. Whenever a user navigates to a new sub site or is using a new widget which was never loaded, AngularJS triggers an AJAX call to load the corresponding html file. Inline templates which are part of the javascript files are an exception of course.

Like every HTTP request the AJAX call can be cached from a web cache like the browser as well. Yeoman has out of the box no grunt task onboard to rename this files which would prevent caching.
But there a two options to solve this problem:

The first solution ensures that all external templates are combined to one single file as inline templates. During this process a grunt task generates one single javascript file, which contains all the views & partials. The grunt-angular-templates module allows to implement this behaviour which solves the caching issue in combination with the javascript minification as described in the last article at full length.

The second option is to implement a similar behaviour to what Yeomans grunt tasks ensure for javascript files. Whenever the view or partial is updated it should get a different name and would be reloaded from the server and not from the cache. This lets the application use as much cached files as possible and only reloads over the network when something is changed.

For the Azure Cost Monitor, I wrote a component called ngHelperDynamicTemplateLoader which totally implements this behaviour. The component is implementing a standard interceptor to add a new version parameter to every view loading AJAX call. The default caching strategie just adds a timestamp every time the view is loaded which means no caching at all. Voila, that is what we wanted to achieve.

In the next part of this article series I’m going to describe a real life example. This example will use grunt-ng-constant to generate context which can be used in a custom caching strategy.

Do you know any other options to deal with this topic? Then please leave a comment to kick off a short discussion about it.