Git Deployment – Shallow Clone Support in Azure App Services – The missing piece

Azure App Services and the open source project KuduSync behind this great Azure Service is a huge time saver for agile teams. Especially DevOps teams will like the continuous deployment features.  Personally I focus a lot on the Git based deployment which enables you to roll back and forward in seconds whenever it is required. Beside that, it is possible to work with standard tools available on market to implement continuous deployment or integration.

Deployments - Microsoft Azure 2017-07-18 06-48-11

When I started working with Azure App Services building Node.js apps, I wrote a little node package called Azure Deploy. It allowed me to push changes as part of a build process directly into the Azure App Service. Originally, CodeShip was the service of choice for the build process but since I need to support Git Repositories beside GitHub, BitBucket and GitLabs, I migrated to Visual Studio Team Services (VSTS) and the integrated build platform.

vso-build-tasks

After several months and hundreds of deploys, which means hundreds of commits to the local git repository, it became a fairly complex and fat thing. This is normally not a problem but my Azure Deploy package clones the local git repository from Azure App Service to a temp directory and copies the build output over it. Last but not least it commits and pushes the changes back to Azure. The big repository took more than 4 minutes to clone so I was wondering if I can use Shallow Clone to get only the latest state of the repository.

This idea works well on Unix based git servers, on GitHub or even in Visual Studio Team Services as well. But when you try to clone a local Git Repository of Azure App Services via Shallow Clone option

git clone --depth 1 https://github.com/jquery/jquery.git jquery

it ends up with an error. The error and its background is also documented in the GitHub project of KuduSync here. So what to do now?

Another nice option of Azure App Services is the option to pull changes from a Git Repository instantly after a commit. This works well in VSTS, based on GitHooks but also with GitHub and a couple other platforms. It’s also possible to clone via shallow clone flag from these repositories which closes the loop. The final solution is to commit into a VSTS or GitHub hosted publishing repository which triggers a pull deployment in Azure App Services.

At the end this change reduced the whole deployment time from 5 up to 9 minutes, down to approx. 90 seconds. You can find the updated Azure Deploy component in the NPM registry here.

Update: Deploy AngularJS-Apps to Azure WebSites with Codeship

A couple of weeks ago I wrote a tutorial on how to deploy an Angular.js application into Azure WebSites. I explained why the simple GitHub deployment that comes with Azure is not usable and how Codeship as shipping service can do the magic for every team.

Since this article I started using Codeship more extensive and since it’s a part of my daily business I don’t want to miss it at all. Here @Matrix42 HackWeek, Codeship was the most chosen solution to implement AdHoc continuous deployment. I would not be suprised when we replace our traditional TFS build agents in the next months 🙂

To make it more intuitive and easy for everybody who works with node and a javascript task runner, I decided to transform the illustrated deployment script into a node module. The Azure-Deploy module is super simple to integrate in existing NPM driven projects and can be added to existing javascript tasks as well. At the end the system offers a simplified usage like this:

grunt deploy:production

This result is more simple to integrate into the deployment scripts of Codeship than everything else. The shell script of course works but this component gives you the freedom to stay with the current task runner of your choice. To get more information visit the github page for this project: https://github.com/dei79/node-azure-deploy

Deploy AngularJS-Apps to Azure WebSites with Codeship

The Azure Cost Monitor is a SaaS application which is extended and improved frequently. Updates take place several times a week without interrupting end users. This agile approach of software development requires continuous integration and a structured deployment process to keep quality and development on a high level. Written in javascript – nodejs for the backend and angularjs for the frontend – the application is deployed to Azure WebSites, the fully managed web hosting solution of Microsoft.

Azure supports deployment from GitHub which works very well for nodejs applications, but angularjs applications need to be compiled to minify the code and rename the images, what prevents caching issues in the browser. The guys from Codeship are offering one of the best cloud platforms to enable continuous deployment for Azure WebSites and many other services.

This tutorial describes a continuous deployment process for an angularjs application to Azure WebSites, based on Codeship. The source code is hosted in a public GitHub repository and the angularjs app is scaffolded with yeoman.

Step 1: Create your AngularJS application

First of all a new Angular application needs to be created with the following yeoman command:

yo angular

This example uses Sass with Compass and Bootstrap, so this options need to be selected in the creation wizard of yeoman. After creating the application it could make sense to test if everything is generated correctly, by starting the development server with this command:

grunt serve

grunt-wiredep is a grunt task which is responsible for updating all the HTML files and minify the JS/CSS files. This task has an issue in version 1.7.0 so it needs to be updated to version 1.9.0 via package.json. The following line needs to be replaced in the package.json:

“grunt-wiredep”: “^1.7.0”,

to

“grunt-wiredep”: “^1.9.0”,

Last but not least the updated package needs to be installed with

npm install

Bower is the component which is used to install all needed web components. The Codeship sandbox has no bower installed out of the box, so the dependency needs to be added with the following command to the package.json as well:

npm install bower –save-dev

Besides that, the commandline interface for grunt needs to be part of the package.json, so that it becomes installed into the Codeship sandbox:

npm install grunt-cli –save-dev

Step 2: Beam Compass & Sass into the Codship sandbox

YeoMan is using Sass and Compass for modern CSS compilation, so it’s required in the Codeship sandbox as well. All used tools are ruby gems so a simple Gemfile with the following content defines the dependencies:

source “https://rubygems.org”

gem ‘sass’, “3.2.9”
gem ‘sass-globbing’, “>= 1.1.0”
gem ‘compass’, “0.12.2”
gem ‘breakpoint’, “2.0.5”
gem ‘singularitygs’, “< 2.0.0”
gem ‘chunky_png’, “1.3.3”

Alle components need to be installed with the command

bundle install

which generates the Gemfile.lock.

Step 3: Activate the CI build in Codeship

All changes described in the steps above should be commited in a GitHub repository. After that everything is prepared to create a new project in Codeship which is connected to the GitHub repository:

Screen Shot 2014-12-20 at 23.11.09

The following setup commands of the Codeship projects should be entered to install all dependencies and kick off the angular build:

rvm use 1.9.3
bundle install
npm install
bower install
grunt build

All the build output is stored in the dist directory. After all preparation, a simple push to the projects GitHub repository lets Codeship start building your application instantly. The first build takes a little bit longer because Codeship prepares the dependency caches. The second one should be done in about 1:30 minutes.

Step 4: Prepare your Azure WebSite

A new Azure Website needs to be created in the Azure Management Portal. Every Azure WebSite supports different ways to publish content, besides traditional FTP Azure WebSites supports deployment from source control. In the specific case of this tutorial the deployment from source control feature should be set to the option “Local Git Repository”:

Screen Shot 2014-12-20 at 23.13.10

Screen Shot 2014-12-20 at 23.14.12

This means Azure hosts a git repository and everything that is committed into this repository will become active in the Azure Website. After that, the next goal is to let Codeship commit all changes into this repository after every successful build.

Step 5: Bring deployment online

Codeship allows to add a deployment script under the project settings for specific branches. In this sample every committed change from the master branch is deployed. In a production environment the system should only be triggered from a specific deployment branch, e.g. deploy/azure.

The following script deploys the compiled angular app to the Azure Website:

# Config git
git config –global user.email “$GITMAIL”
git config –global user.name “$GITUSER”
# Clone the whole azure repository
cd
git clone $GITREPO azure
# Add the compiled app
cd azure
rm -R -f *
cp -R -f ~/clone/dist/* .
git add -A
git commit -m “Code shipped”
# push to azure
git push origin

Last but not least, all used variables need to be stored as environmental settings in Codeship. This prevents that Codeship prints out sensitive data to the logs and decouples configuration settings from deployment script:

Screen Shot 2014-12-20 at 23.15.09

The GITREPO variable contains the https based git URL from the local git repository of the Azure WebSite. It includes username and password in the URL, e.g.

https://{{user}}:{{password}}@{{website}}.scm.azurewebsites.net:443/{{website}}.git

The deployment credentials can be found in the Azure portal for the Azure website.

Finally a simple push into the Git repository of the project triggers a build via Codeship and a deployment into the right Azure WebSite. Users do not loose access to the SaaS application except for a couple seconds after redeploying when the IIS load the new application.

This fully automated process supports every agile software development process and helps you to focus on making features and not updating servers.

Happy Deploying 🙂