Git Deployment – Shallow Clone Support in Azure App Services – The missing piece

Azure App Services and the open source project KuduSync behind this great Azure Service is a huge time saver for agile teams. Especially DevOps teams will like the continuous deployment features.  Personally I focus a lot on the Git based deployment which enables you to roll back and forward in seconds whenever it is required. Beside that, it is possible to work with standard tools available on market to implement continuous deployment or integration.

Deployments - Microsoft Azure 2017-07-18 06-48-11

When I started working with Azure App Services building Node.js apps, I wrote a little node package called Azure Deploy. It allowed me to push changes as part of a build process directly into the Azure App Service. Originally, CodeShip was the service of choice for the build process but since I need to support Git Repositories beside GitHub, BitBucket and GitLabs, I migrated to Visual Studio Team Services (VSTS) and the integrated build platform.

vso-build-tasks

After several months and hundreds of deploys, which means hundreds of commits to the local git repository, it became a fairly complex and fat thing. This is normally not a problem but my Azure Deploy package clones the local git repository from Azure App Service to a temp directory and copies the build output over it. Last but not least it commits and pushes the changes back to Azure. The big repository took more than 4 minutes to clone so I was wondering if I can use Shallow Clone to get only the latest state of the repository.

This idea works well on Unix based git servers, on GitHub or even in Visual Studio Team Services as well. But when you try to clone a local Git Repository of Azure App Services via Shallow Clone option

git clone --depth 1 https://github.com/jquery/jquery.git jquery

it ends up with an error. The error and its background is also documented in the GitHub project of KuduSync here. So what to do now?

Another nice option of Azure App Services is the option to pull changes from a Git Repository instantly after a commit. This works well in VSTS, based on GitHooks but also with GitHub and a couple other platforms. It’s also possible to clone via shallow clone flag from these repositories which closes the loop. The final solution is to commit into a VSTS or GitHub hosted publishing repository which triggers a pull deployment in Azure App Services.

At the end this change reduced the whole deployment time from 5 up to 9 minutes, down to approx. 90 seconds. You can find the updated Azure Deploy component in the NPM registry here.

Advertisements

Azure Costs: Exporting data via API

Azure Costs now offers direct API access to integrate the processed and aggregated data directly in 3rd party applications like Qlik or Tableau. This option allows customers on an enterprise plan or higher to render or analyse the Azure Costs data in existing BI solutions.

Exporting data via API is very simple. Just follow the steps below:

  1. Register an application as trustee to control which data can be access from the 3rd party application in the team management view.

    ApiCredentials

  2. The application contains a client_id and a client_secret which can be used for the oAuth2 client credentials flow. Just issue a new security token at least every hour from the Azure Costs STS. We recommend to issue a token whenever you start a new transaction to ensure that permission changes are affective.

    curl -X POST -d “client_id={{YOUR CLIENT ID}}&client_secret={{YOUR CLIENT SECERT}}&grant_type=client_credentials” https://azure-costs.com/sts/issue/oauth/token

  3. Export the required report from the export api which returns the data in JSON format. Every single object describes one service including the quantity, costs and other meta information. The required teamId, contract number can be extracted from the URL of the Azure Costs dashboard

    curl -H “Authorization: Bearer {{ISSUED TOKEN}}” https:/api.azure-costs.com/api/v1/teams/{{teamId}}/contracts/{{contractId}}/reports/{{Year}}-{{Month}}/export?cache={{DATE-OF-THE-DAY}}

Interested in the new feature?
Try the new feature today by simply logging into your Azure Costs account. The feature is part of any enterprise plan.

Any questions, wishes or ideas? Try our feedback portal or drop a mail to help@azure-costs.com.

Azure Costs: Service Types and Categories

Today we’re very excited, to announce the release of our revised service types and service categories for azure costs. In the early days azure costs just supported the Service Type attribute. Together with our customers we identified the need of having a more efficient way to understand cost drives and spending spikes.
azure-costs-azure-cloud-cost-optimization-made-easy-2017-02-26-19-22-52
The Service Type property now shows a more detailed type of the Azure Service. Virtual Machines report the selected pricing tier or size in that property . SQL databases are also shown now with the size of the chosen Azure SQL instance.
azure-costs-azure-cloud-cost-optimization-made-easy-2017-02-26-19-24-50
In contrast to the Service Type, the Service Category property describes the service class in Azure, e.g. Azure App Services, Data Management which includes BigData and SQL Services or the classical Cloud Services.
The clear differentiation between this two properties should help to identify cost spikes faster and more reliable.

Interested in the new feature?
Try the new feature today by simply logging into your azure costs account. The feature is part of any plan.

Any questions, wishes or ideas? Try our feedback portal or drop a mail to help@azure-costs.com.

 

Azure Costs: Available as white-label solution

Today we are very excited to announce that Azure Costs can now be offered as a white-label solution for Cloud Solution Providers.

white-label-image

Using Azure Costs as a white-label solution includes the following features:

Service Provider Specific Top-Level-Domain incl. SSL Certificate:
Offering Azure Costs as white-label solution is delivered under your own service provider specific domain, e.g. costs.acme.com. It’s possible to bind a service provider specific SSL certificate to this domain as well.

Dedicated Storage Backend incl. support for the Azure Cloud Germany:
Every service provider is able to connect a dedicated Azure Storage Account. This ensures that all the customer data are stored and processed only in a service which is under control and management of the service provider.

Custom Identity Provider:
Servicer Provider often offer their customers account and identity infrastructure. Azure Costs as a white-label solution allows to connect custom identity providers based on the oAuth2 protocol. This allows your customers to stay with their existing accounts for the server providers infrastructure.

Interested in offering Azure Costs as a white-label solution?
Try Azure Costs as white-label solution, just contact as via a support request or drop a mail to help@azure-costs.com. We will guide you through the signup procedure.

Any questions, wishes or ideas? Try our feedback portal or drop a mail to help@azure-costs.com.

 

Azure Status Page Server Side Meters

In the last article we described the details about client side meters and how to use them with Azure Status Pages. But there are situation where you would like to observe your application from the outside view. This article describes the Service Side Meters Azure Status Page supports as well.

What is a Meter?

In the Azure Status Page project a Meter is something that describes a specific metric you would like to measure and observe. Meters can be defined in the Azure Status Page Site Extension as Service Side Meters. Another convenient way is to define and send meters in your own application. This kind of meters are called Client Side Meters and can be defined and updated via an SDK delivered from us.

Server Site Meters

meters-2017-02-26-18-56-32

Server Site Meters in Azure Status Page are checking if a given HTTP or HTTPs endpoint is available. As soon as the meter is defined, Azure Status Page checks the availability of the service on a recurring basis. Similar to the Client Side Meters a failed meter will influence the status page and will communicate it to the operations staff via PushOver notifications.

Which Server Side Checks are important for you? We are planning to offer more needed Server Side Meters soon. Please let us know via GitHub Issue or via blog comment what’s needed.

 

 

Azure Costs: Support for Azure Tag based grouping

Azure Tags are an efficient way for employees with access to the Azure Management Portal to organize and categorize Azure Services. Besides the Resource Groups, Azure Tags are the second important organisation feature in the Azure Management Portal.

azure-costs-azure-cloud-cost-optimization-made-easy-2017-02-26-19-42-45

Azure Costs now imports the Azure Tag definitions and the associated values automatically during the nightly sync. Every Azure Tag is offered as an additional group option for the given spending data. The import works for all contract types including Pay-as-you-Go, Cloud Solution Provider and Enterprise Agreements.

Interested in the new feature?
Try the new feature today by simply logging into your azure costs account. The feature is part of any plan.

Any questions, wishes or ideas? Try our feedback portal or drop a mail to help@azure-costs.com.

Azure Status Page Client Side Meters

In the last weeks we were working on the Azure Status Page Generator to give everybody the chance to generate Status Pages within minutes. Start right now with it by visiting our project page at GitHub.
This article digs deeper into the Client Side Meters feature of the Azure Status Page Site Generator.

metr1

What is a Meter?

In the Azure Status Page project, a Meter is something that describes a specific metric you would like to measure and observe. Meters can be defined in the Azure Status Page Site Extension as Service Side Meters. Another convenient way is, to define and send meters to your own application. This kind of meters are called Client Side Meters and can be defined and updated via an SDK, delivered from us.

Client Side Meter SDK

Azure Status Page currently offers SDKs for two major platforms. The nodejs SDK is available as NPM component:

npm install azure-status-page-client –save

Azure Status Page also supports C# application e.g. ASP.NET WebServices or WebWorker for background processing. Just install the NuGet package as follows:

Install-Package statuspageclient

Meter Examples

Good examples for Client Side Meters are the following, that we are also using in production:

  1. Queue Triggered WebJob HeartBeat
    A worker implemented based on Azure WebJobs and triggered via QueueMessage can be observed with a client side heartbeat meter. Just send a message during every polling to the Azure Status Page. As soon as your worker is blocking or hanging, a filed meter will be detected and you will get notified via PushOver.
  2. Free Disk-Space for Caches or Max Size of a file Cache
    Sometimes using file based caches on the application server can help to speed up your application dramatically. During the maintenance  process for the cache the client side meter can send the size of the cache or the free disk space to Azure Status Page. You will get an alert when the cache grows abnormally or your disk space exceeds.

Do you have other ideas for great client side meters? We are planning to offer pre-developed meters as part of our Client SDKs that you can plug into your application directly. Let us know via GitHub Issue or via blog comment what is needed.