News from the company...

AzureWatch retirement announcement and transition to CloudMonix

clock December 7, 2016 07:03 by author Igor Papirov

It has been more than 6 years since I originally launched AzureWatch to help companies monitor and auto-scale their Azure Cloud Services.  Over the years, as the Azure platform grew and matured, my team and I tried to adapt AzureWatch to help provide more value in the ever-changing Azure space. We implemented monitoring for SQL Azure, WebApps, Virtual Machines, and Service Bus. However, as Azure platform and our customers' needs started to rapidly evolve, it became increasingly obvious that AzureWatch's internal architecture and somewhat dated user interface would simply not be able to keep up.

Having learnt many hard lessons and with deeper understanding of what monitoring/automation needs Azure users have, we started to work on a new product called CloudMonix, to replace AzureWatch.  CloudMonix was designed to provide deep insight, /automated issue recovery/ and advanced auto-scaling to Azure (and in the future AWS) platforms.  In early 2015, CloudMonix, was launched with great success.  Since then, we've experienced a sustained 5-10% month-to-month growth rate.

Today, CloudMonix supports monitoring of over 30 different types of cloud resources, integrates with a dozen of third party products, offers beautiful UI and has tons of bells and whistles that are useful for Service Providers, DevOps teams, and Enterprises.  Almost every AzureWatch customer who saw our newsletters and tried CloudMonix, migrated to it and is very happy with the outcome.  The product is that good!  And what's most important, we update and improve CloudMonix on a weekly basis. Literally.

So, I am both saddened and happy to announce that we intend to retire AzureWatch in April of 2017.  We currently have remaining 16 customers that are still using it.  Unfortunately, it is becoming less and less profitable for us to maintain the infrastructure and support of AzureWatch as time progresses

I would like to invite every current and previous AzureWatch user to migrate to CloudMonix and my team and I are ready to every assistance and incentive to help migrate your team to the new platform.

To learn more about CloudMonix and to sign up for a free trial, please visit our website at http://cloudmonix.com

To understand the benefits of CloudMonix and see it in action please schedule a time here: https://paraleap.acuityscheduling.com

 

If I can assist in any other way, just email igor@paraleap.com

Short FAQ:

Q: What are the benefits of CloudMonix vs AzureWatch?

A: CloudMonix far exceeds both AzureWatch and native Azure monitoring in a number of areas:

 

  • Amount and depth of diagnostic data it can capture, visualize and provide ability to alert on.
  • Powerful automation capabilities that can automatically recover your resources from outages and issues. 
  • Sophisticated auto-scaling capabilities not only for Cloud Services, but also for resources such as SQL Azure and Data Warehouses, WebApps and Virtual Machines, etc. 
  • Native integrations for many 3rd party products like Slack, PageryDuty, Zendesk and Autotask. 
  • Amazing UI with ability to scroll your dashboards back in time to easily find root cause of previously occurred production issues. 

 

    Overall, the full list of benefits is too lengthy to outline in an email.  Happy to discuss the benefits via online meeting. Click here to schedule a discussion: https://paraleap.acuityscheduling.com/

Q: How hard is it to onboard with CloudMonix?

A: Onboarding is incredibly simple.  Setup Wizard takes 4-5 minutes to complete.  CloudMonix comes pre-defined with a number of useful metrics and alerts, so that it brings value right out of the gate.  Configuration Templates offer ability to easily propagate similar monitoring setups across many resources in 2 clicks.

Q: Is there an automatic migration from AzureWatch to CloudMonix?

A: Unfortunately there is not.  However, default monitoring profiles are pre-packaged with a number of useful metrics and alerts.  And everyone who has migrated so far, appreciated the chance to revisit and improve their monitoring setups

Q: What Azure resources can CloudMonix monitor/automate?

A: CloudMonix supports monitoring the following resources (ARM and Classic mode is supported)

 

  • Windows and Linux VMs
  • Cloud Services
  • Web Apps and Webjobs
  • Service Bus Topics and Queues
  • SQL Azure and SQL Data Warehouse
  • Azure Redis Cache
  • DocumentDb Collections
  • Azure Storage (Blob, Queue, Table and File storage)
  • Stream Analytics and Event Hubs
  • Media Services
  • Azure Automation Runbooks
  • Backup Vaults
  • Virtual Networking
  • Scheduler
  • And more coming (Data Factories, Azure Batch, Scale Sets, Azure Search, Service Fabric, etc.)
  • CloudMonix can also monitor various non-Azure resources, such as Oracle, MySql, Sql Server (non-Azure), Windows Servers (non-Azure), Sockets, URLs, and JSON/XML API endpoints

 



AzureWatch retirement announcement and transition to CloudMonix

clock December 7, 2016 07:03 by author Igor Papirov

It has been more than 6 years since I originally launched AzureWatch to help companies monitor and auto-scale their Azure Cloud Services.  Over the years, as the Azure platform grew and matured, my team and I tried to adapt AzureWatch to help provide more value in the ever-changing Azure space. We implemented monitoring for SQL Azure, WebApps, Virtual Machines, and Service Bus. However, as Azure platform and our customers' needs started to rapidly evolve, it became increasingly obvious that AzureWatch's internal architecture and somewhat dated user interface would simply not be able to keep up.

Having learnt many hard lessons and with deeper understanding of what monitoring/automation needs Azure users have, we started to work on a new product called CloudMonix, to replace AzureWatch.  CloudMonix was designed to provide deep insight, /automated issue recovery/ and advanced auto-scaling to Azure (and in the future AWS) platforms.  In early 2015, CloudMonix, was launched with great success.  Since then, we've experienced a sustained 5-10% month-to-month growth rate.

Today, CloudMonix supports monitoring of over 30 different types of cloud resources, integrates with a dozen of third party products, offers beautiful UI and has tons of bells and whistles that are useful for Service Providers, DevOps teams, and Enterprises.  Almost every AzureWatch customer who saw our newsletters and tried CloudMonix, migrated to it and is very happy with the outcome.  The product is that good!  And what's most important, we update and improve CloudMonix on a weekly basis. Literally.

So, I am both saddened and happy to announce that we intend to retire AzureWatch in April of 2017.  We currently have remaining 16 customers that are still using it.  Unfortunately, it is becoming less and less profitable for us to maintain the infrastructure and support of AzureWatch as time progresses

I would like to invite every current and previous AzureWatch user to migrate to CloudMonix and my team and I are ready to every assistance and incentive to help migrate your team to the new platform.

To learn more about CloudMonix and to sign up for a free trial, please visit our website at http://cloudmonix.com

To understand the benefits of CloudMonix and see it in action please schedule a time here: https://paraleap.acuityscheduling.com

 

If I can assist in any other way, just email igor@paraleap.com

Short FAQ:

Q: What are the benefits of CloudMonix vs AzureWatch?

A: CloudMonix far exceeds both AzureWatch and native Azure monitoring in a number of areas:

 

  • Amount and depth of diagnostic data it can capture, visualize and provide ability to alert on.
  • Powerful automation capabilities that can automatically recover your resources from outages and issues. 
  • Sophisticated auto-scaling capabilities not only for Cloud Services, but also for resources such as SQL Azure and Data Warehouses, WebApps and Virtual Machines, etc. 
  • Native integrations for many 3rd party products like Slack, PageryDuty, Zendesk and Autotask. 
  • Amazing UI with ability to scroll your dashboards back in time to easily find root cause of previously occurred production issues. 

 

 

 

 

 

    Overall, the full list of benefits is too lengthy to outline in an email.  Happy to discuss the benefits via online meeting. Click here to schedule a discussion: https://paraleap.acuityscheduling.com/

Q: How hard is it to onboard with CloudMonix?

A: Onboarding is incredibly simple.  Setup Wizard takes 4-5 minutes to complete.  CloudMonix comes pre-defined with a number of useful metrics and alerts, so that it brings value right out of the gate.  Configuration Templates offer ability to easily propagate similar monitoring setups across many resources in 2 clicks.

Q: Is there an automatic migration from AzureWatch to CloudMonix?

A: Unfortunately there is not.  However, default monitoring profiles are pre-packaged with a number of useful metrics and alerts.  And everyone who has migrated so far, appreciated the chance to revisit and improve their monitoring setups

Q: What Azure resources can CloudMonix monitor/automate?

A: CloudMonix supports monitoring the following resources (ARM and Classic mode is supported)

 

  • Windows and Linux VMs
  • Cloud Services
  • Web Apps and Webjobs
  • Service Bus Topics and Queues
  • SQL Azure and SQL Data Warehouse
  • Azure Redis Cache
  • DocumentDb Collections
  • Azure Storage (Blob, Queue, Table and File storage)
  • Stream Analytics and Event Hubs
  • Media Services
  • Azure Automation Runbooks
  • Backup Vaults
  • Virtual Networking
  • Scheduler
  • And more coming (Data Factories, Azure Batch, Scale Sets, Azure Search, Service Fabric, etc.)
  • CloudMonix can also monitor various non-Azure resources, such as Oracle, MySql, Sql Server (non-Azure), Windows Servers (non-Azure), Sockets, URLs, and JSON/XML API endpoints

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 



INTRODUCING CLOUDMONIX

clock May 9, 2015 13:55 by author Igor Papirov

Short version

AzureWatch is now over four years old and is a little... dated.  In order to keep up with our customers' needs and with innovations in Azure, we recently introduced a brand new monitoring product designed to replace AzureWatch.  The product is called CloudMonix and it is available at http://cloudmonix.com.  CloudMonix enhances Microsoft Azure by providing deep monitoring of most of Azure's infrastructure via live dashboards, ability to self-heal from many different production issues, on-demand historical performance and uptime reports, customizable alerts & notifications, sophisticated auto-scaling engine, integration to third party systems, and a lot more.  CloudMonix is not yet available in Azure Marketplace.

 

Long version

Five years ago, I set out on a journey to help Azure developers auto-scale their Azure systems.  AzureWatch, a product that I built for that purpose, was originally a Windows desktop application that allowed users to dynamically scale their Web and Worker roles with demand - a feature that Windows Azure lacked back in the day.  

Over the years, AzureWatch was continually enhanced to meet ever-growing customer needs.  As my team grew with the customer base, so did AzureWatch's feature set.  We migrated AzureWatch from a desktop app to a fully hosted cloud solution.  We added monitoring capabilities and live dashboards.  AzureWatch was enhanced to monitor SQL Azure, Storage, Websites and Virtual Machines.  Unfortunately, with each iteration of changes and features, we twisted and adopted AzureWatch to do something it was never meant to.  Ongoing maintenance became hard and time to innovate long.

As market conditions changed and need for auto-scaling decreased, due to basic auto-scaling features being introduced in Azure core platform, it was obvious that we needed to offer customers something more substantial and special.  After looking thru the backlog of customer requests, a few patterns clearly emerged, where we saw that we could add a ton of value

  • Very fast onboarding and intuitive UI.  No one has time to learn complex systems that don't work from the get-go
  • Deep insight into all levels of technical stack, not just servers. Cloud platforms offer many useful services and they all need to be considered in the overall health of a production system
  • Automatic self-healing.  While healing procedures are usually simple to script out (reboot server, restart service, clear cache, recycle app pool, truncate table, etc.), knowing WHEN to execute these procedures can be really challenging.
  • Ability to compare & contrast performance, uptime, and other metrics over time.  
  • Provide value not just for Azure but other cloud systems for shops that manage or utilize different cloud platforms

Taking into account that AzureWatch's back-end needed to be re-architected to become much more adaptive at monitoring new and different systems and AzureWatch's front-end UI needed a lot of work to become user-friendly, the decision to re-architect the whole product became a no-brainer.  So, in summer of 2014, we set out on a journey to re-invent ourselves and provide a new kind of SaaS service: "Stability-as-a-Service" to cloud production environments with a delightful user experience and backed by our amazing support.

Fast-forward to today.  We delivered and in some ways over-delivered on all of our goals with the release of CloudMonix in March of 2015.  All lessons learnt in the years of maintaining AzureWatch came in handy.  CloudMonix has automatic self-healing and enhanced auto-scaling engines; amazing and responsive UI; ability to integrate with other 3rd party systems; future support for public API, data-region affinity; in-depth monitoring of a number of popular Azure services and ability to develop others very quickly.  There is more, much more.  With a few known exceptions, CloudMonix far exceeds AzureWatch in features and functionality from the get-go and continues to evolve and expand very quickly.

I invite you experience CloudMonix for yourself at http://cloudmonix.com and tell us what you think.

Igor Papirov
CEO
Microsoft Azure Insider
Paraleap Technologies



AzureWatch now supports Azure Diagnostics Extension (SDK 2.5+)

clock March 31, 2015 03:12 by author Igor Papirov

We are happy to introduce support for monitoring of Cloud Services deployed with Azure SDK 2.5+ with Azure Diagnostics Extension to AzureWatch and CloudMonix.  Action is required for users who wish to monitor Azure Cloud Services using the Diagnostic Extension with AzureWatch.

In order to properly enable monitoring, kindly specify a valid Azure Storage account for capture of diagnostic data, in the Role configuration screen (see image below).  As with previous Azure Diagnostics Module, spreading diagnostic data from different roles across different Azure storage accounts is recommended.

 

 

Please be aware that any change to configuration of performance counters within a specified role, will result in AzureWatch making two calls to UpdateDeployment in order to uninstall and reinstall Azure Diagnostic extension.  While these calls are generally harmless, they may cause a short outage of web traffic if you only have 1 instance deployed within a Webrole.  



AzureWatch compatibility with Azure SDK v2.5 and above

clock January 15, 2015 03:18 by author Igor Papirov

At the end of 2014, Microsoft released a new Windows Azure SDK v2.5 that has significant impact on monitoring of Azure Cloud Services (web and worker roles) with Azure Diagnostics.  In a nutshell, Azure SDK 2.5 has obsoleted the way Azure Diagnostics was instrumented and collected via their Diagnostics module and switched over to a new Extension-based model utilized for monitoring of Virtual Machines.

 

From AzureWatch perspective, this means that users who depend on AzureWatch to monitor and auto-scale their Cloud Web and Worker roles should wait before upgrading to Azure SDK v2.5.  AzureWatch is currently not compatible with Azure SDK 2.5 when monitoring Cloud Services.  Monitoring of SQL Azure, Storage, VMs, and other assets is not impacted.  We are working on handling this situation and hope to have more news in the coming weeks ahead.



Major enhancements to AzureWatch Rules engine

clock March 11, 2014 11:47 by author Igor Papirov

A number of important changes are being introduced to AzureWatch this weekend (March 16th):

  • Ability to execute rules only after a sustained period of time
    • This feature allows for much better control of the scaling process.  For example, in certain situations it can be far more effective to scale when sustained load is over a certain threshold rather than try to predict what a moving average looks like.  It is important to know that if a rule is configured with a sustained time delay, it will only be executed after continuously being evaluated to TRUE for the specified period of time.
  • Ability to send ON and OFF alerts (a single ON email when alert evaluates to TRUE, and a single OFF email when it evaluates to FALSE)
    • This feature reduces spam when a certain rule's condition is continuously TRUE.  It also simplifies configuration since ON/OFF alerts no longer require throttling.  Unless modified, existing Alerts will work as they currently do.
  • Separation of Alerts from Management Actions (rules that notify will be separated from rules that execute scale actions, shutdowns, restarts, etc.)
    • This feature is relatively important as it may impact existing rule sets.  Going forward, rules that are Alerts will be evaluated separately from rules that are Management Actions.  When evaluating rules, all Alerts that qualify for execution will be evaluated and acted upon, not just the first one. Management Actions will continue to be evaluated until the first rule that qualifies for execution. Users who currently rely on Alerts to control execution of their Management Actions will want to revisit their scaling configurations.  We do expect percentage of such users to be either very small or non-existent.
    • After the upgrade, we plan to monitor AzureWatch's email queues and switch highly spamming Alerts to have ON/OFF logic.  Impacted customers will be notified.

While we expect minimum impact during or after the upgrade, we want to be transparent with our users: this is probably the most significant change to the Rules engine since the inception of AzureWatch. If you have any concerns, please contact Paraleap support team



Upcoming changes to the way alerts work in AzureWatch

clock January 28, 2014 12:08 by author Igor Papirov

We are currently working on changing the way alerts work in AzureWatch.  At this time, Rules for a monitored resource, that trigger either Alerts or Management Actions are ALL evaluated together in a single loop.  When a single Rule is evaluated to TRUE, all further evaluation of rules for that resource stops.  This will be changing going forward.  ALL rules that trigger alerts will be evaluated, regardless of their success or failure.  Only rules that contain Management Actions will be evaluated to the first TRUE rule.

Furthermore, we will be adding the following two options to the Rule engine:

1) Ability to send ON/OFF notifications for Alerts.  This means that when a condition for a Rule is TRUE, AzureWatch will send out an "Alert ON" alert and subsequently when the condition for a Rule no longer is true, AzureWatch will send out an "Alert OFF" alert.

2) Ability to trigger Rules (Alerts or Management Actions) only after a sustained period of time.  This means that when a condition is TRUE, a particular Rule will not be immediately acted upon.  Only if Rule's condition has been evaluated as TRUE for the specified period of time, will it trigger it's Alert or Management Action.



Monitor Windows Azure Service Dashboard!

clock January 22, 2014 23:28 by author Igor Papirov

AzureWatch users can now receive notifications when changes are published to the Windows Azure Service Dashboard.  Upon logging into AzureWatch portal, users can choose to subscribe to any of the Azure Service Dashboard feeds as shown in the screenshot below.  This feature is available free of charge to active AzureWatch users.

 

AzureWatch Dashboard notifications



Windows Event Log monitoring with AzureWatch

clock January 6, 2014 06:38 by author Igor Papirov

This week we are excited to release a long-requested feature in AzureWatch: support for active monitoring of Windows Event Logs.  We are starting things off with support for Application, System and Security logs in Windows Azure Cloud Services (Web or Worker Roles) and Windows Azure Virtual Machines.

Users can now create rule-based alerts that match against entries in a particular Event Log, against a specific event log level, name of the publishing application or text of the entry itself.  We've enhanced our Rules engine to support string comparison and search capabilities.  Even though the extra workload to accommodate monitoring of Event Logs is significant, this feature will be available at no additional charge to users who are already monitoring their servers with AzureWatch.

In order to enable Event Log monitoring, users need to specify which logs should AzureWatch be monitoring and at what level.  Take care to not overwhelm monitoring cycles by asking AzureWatch to inspect too much data per cycle.  Checkboxes and Minimum severity level that users can specify on the Role configuration screen will control how much data is being sent to AzureWatch. Hundreds of log entries per minute should be OK, but once the number of log events gets into thousands per minute, users with a lot of servers may notice slowdowns of their monitoring cycles.

Turn on Event Log monitoring

 

After instructing AzureWatch (and via AzureWatch, Windows Azure itself) as to what Event Log data needs to be transferred to Windows Azure Diagnostics Table Storage or captured through Powershell Remoting, users can create custom alert or management rules based on the Event Log data.  We've enabled a number of Event Log specific variables that can be used in formulas of rules as shown in the screenshot below.  Every event log entry captured during a particular monitoring cycle will be evaluated against user-specific formula until a first matching log entry is found.  Users can combine aggregate metrics together with Event Log based search criteria to create sophisticated rules.  For example, it is possible to create a monitoring rule that looks for .NET errors that occur only when average CPU utilization is over 70%.  In order to support text-search capabilities in our engine, we've added ability to search text-based variables via three new functions: Contains, EndsWith and StartsWith, all of which return TRUE or FALSE when called from the rule's formula

 

Going forward we intend to further enhance the engine by providing numeric counts of events that match particular criteria to the rules engine, so that users can create rules based not only on individual log entries but also based on quantity of specific events found.

 

Do you have any feedback as to what you'd like to see from AzureWatch?  Please do not hesitate to contact us!



New Dashboard is here!

clock November 11, 2013 10:35 by author Igor Papirov

Users logging into AzureWatch Management Portal last week found themselves pleasantly surprised.  We've completely overhauled our dashboard.  It is now a robust, configurable user experience designed to provide users with an ability to quickly and intuitively see all of the relevant and important monitored data for their Azure applications.  Check out a screenshot cutout or simply login with your account to experience the new interface


Dashboard



Better support for schedule-based scaling

clock August 12, 2013 03:54 by author Igor Papirov

Last major AzureWatch release contained support for monitoring and auto-scaling of Windows Azure Infrastructure as a Service platform - Virtual Machines.  This was a major release for us and all other feature requests took a back seat while we implemented this functionality.  Now, that Virtual Machine support is behind us, we're happy to start implementing various features that have been requested by our users. This week we are releasing the following three enhancements that our users may find useful: 

  • Support for custom scale limits based on time of day.  This feature allows AzureWatch to enforce different scaling ranges based on time of day and without the need to create scaling rules.  This is handy feature for many business applications and b2c websites where usage patterns are known upfront and vary by time.   Scaling rules will still kick-in and work, however the instance limits will adjust based on time
  • Ability to import configuration settings from one Role to another.  This is handy for users who have need to configure monitoring for a number of similar Roles.  We often get this request from customers who same Role configurations deployed in different Windows Azure data centers.
  • Support for linked accounts.  Customers who have multiple linked AzureWatch accounts can now easily switch between them by simply selecting a different linked account from the AzureWatch menu

 



Windows Azure Auto-scaling Options Side-by-Side Comparison

clock July 29, 2013 23:07 by author Igor Papirov

Gone are the days when AzureWatch was the only option for auto-scaling of Windows Azure cloud services.  Today, basic auto-scaling support is also offered in Windows Azure itself (free while in beta) and is available as a WASABi Enterprise Library for the do-it-yourself and host-it-yourself types.

This article examines the Pro's and Con's of each approach and explains when using AzureWatch's advanced features makes sense.  A quick reference guide is provided toward the end that documents the differences between three offerings in a side-by-side comparison table.

When it comes to ability to auto-scale, manage, heal and monitor various Windows Azure resources, Enterprise Library WASABi, native Windows Azure monitoring and autoscaling and AzureWatch all offer different features to satisfy different needs.

  • WASABi is great for folks who like to have complete control over their scaling implementation, do not mind tinkering with settings on lower level, and do not really need monitoring of such Azure services as SQL Azure or Storage.  WASABi allows users to deploy their own scaling engines, provide sophisticated XML-based configuration sets and basically tinker with things in depth.  Users running WASABi are also responsible for hosting and monitoring it.
  • Windows Azure's native support for auto-scaling was recently added into the platform itself.  This is a great option for users who prefer simplicity over flexibility and probably targets smaller or non-mission critical applications.  It is also great for users who are just starting out with Windows Azure and are not yet sure what features from an auto-scaling platform they need.
  • AzureWatch's strengths lie somewhere in the middle of the other two options.  It offers endless abilities to tinker with settings of its auto-scaling engine, albeit not at the same level of depth as WASABi, but at the same time it provides a nice configuration UI and a fully managed environment.   Furthermore, it supports monitoring of SQL Azure, Storage, and has a ton of other features that make it compelling for businesses running serious applications on the Azure platform.
    On top of supporting all of the customary auto-scaling options like upper/lower boundaries, throttling, etc. AzureWatch also allows scheduled-based scaling, ability to restart/re-image misbehaving VMs, scheduled-based shutdown-startup of servers, ability to scale based on rate of change in demand, ability to evaluate complex boolean rules, and tons of other features that its users have requested over the years to support their sophisticated monitoring and scaling strategies.
    AzureWatch is a commercial product not owned by Microsoft and while not expensive, it is not free and may not be the best choice for users who are just playing around with Windows Azure platform or who run small personal websites.

 

 

Windows Azure

WASABi

AzureWatch

Platform Coverage

     

Cloud Services (Web/Worker Roles)

Yes Yes Yes

Virtual Machines (in Availability Sets)

Yes No Yes

Virtual Machines (stand-alone)

No No Stop/Start/Reboot

Azure Websites

Yes No Yes

Azure Mobile Services

Yes No Not Yet

SQL Azure & Federations

No No Monitoring

Azure Storage

No No Monitoring

URL Endpoints

Monitoring (up to 2) No Monitoring (no limit)
       

Scaling Features

     

Frequency of Monitoring

5 minutes custom 1 minute

Performance Counter Evaluation

CPU Utilization Only Any Any

Queue Depths Evaluation

Storage & Service Bus Storage & Service Bus Storage & Service Bus

Scaling Cooldowns

Yes Yes Yes

Scheduled Scaling

No Yes Yes

Monitoring of Running Averages

60 minute (CPU only) custom up to 24hrs

Scale down to 0 servers
with Virtual Machines

No No Yes

Advanced scaling formulas with
running averages

No Yes Yes
       
       

Other

     

Simplicity of use and ease of
integration with Windows Azure

Simple Advanced Medium

Pricing

Free, while in beta Self-hosting
costs only
<$10 server/month

User Interface

Built into Windows Azure None AzureWatch
Management 
Studio

 

 

 



Introducing preview of auto-scaling, healing, and monitoring of Azure VM's (IaaS)

clock July 23, 2013 12:55 by author Igor Papirov

We are excited to introduce the long-awaited AzureWatch support for monitoring, manage, auto-scaling and healing of Azure Virtual Machines.  We are soft launching this functionality in preview mode and are looking for your feedback!

 

Functionality Overview:

Every minute of every hour, AzureWatch will connect through Powershell Remoting to any Azure VM's that is either stand-alone or a part of an availability set.  Once connected, it will capture any number of standard or custom Windows performance counters that have been configured by you and execute its usual suite of monitoring or scaling Rules.  After the Rules execute, AzureWatch can execute scaling, re-imaging, stop/start and alerting actions. Please refer to the following table to see what actions are supported under what conditions:

 

 Performance CountersQueues & Other MetricsAuto-ScalingAuto-RebootingAuto-Stop/StartAlerts
Stand-alone Windows VMs Supported Supported   Supported Supported Supported
Stand-alone Linux VMs   Supported   Supported Supported Supported
Windows-based Availability Sets Supported Supported Supported Supported Supported Supported
Linux-based Availability Sets   Supported Supported Supported Supported Supported

 

It is important to know that stand-alone Virtual Machines cannot be auto-scaled because they are not load-balanced by Windows Azure.  However, when a VM is part of an Azure Availability Set, it can be scaled up or down, provided that there are enough shutdown VM's that exist within the Availability Set.  During the scale-up event of an availability set, AzureWatch finds the next available VM that has been turned off and simply starts it up.  Conversely, when an Availability Set is scaled-down, AzureWatch simply finds the last active VM and shuts it down so that charges are not incurred.

 

The following must be done in order to have AzureWatch properly monitor Azure VM's:

AzureWatch currently supports capture of performance counters only from Windows-based VMs that have Powershell Remoting enabled.  Powershell Remoting port 5986 must be open within the Windows Firewall on the server itself.  In addition, a public endpoint mapping to local port 5986 must be defined in the Azure configuration of each monitored Virtual Machine.  Azure's public port can be any number.

Users who wish to auto-scale or auto-shutdown/start-up their VM's based on a schedule or queue counts, do not need Powershell Remoting enabled and are not limited to running Windows Server.



AzureWatch brings SaaS/HaaS App Monitoring and Autoscaling Service to the Windows Azure Store

clock June 26, 2013 12:00 by author Igor Papirov

Arlington Heights, IL

As a featured add-on in the Windows Azure Store, AzureWatch provides on-demand elasticity, healing, monitoring and just-in-time provisioning of resources – saving time and money


Paraleap Technologies
, a developer of high performance tools and services for cloud computing technology, today announced that their flagship product, AzureWatch, is now offered as an add-on option in the Windows Azure Store, part of the Windows Azure management portal. Developers and IT professionals can now easily implement AzureWatch’s dynamic scalability, healing and monitoring to their applications running on Microsoft’s Windows Azure cloud platform.

As one of the first autoscaling and monitoring services available, AzureWatch has proven itself to be a responsive and affordable service continuously improving and expanding its features. Earlier this year AzureWatch released its newest version which simplified the configuration process and in April expanded services again to offer healing-as-a-service (HaaS) for Windows Azure. HaaS now allows AzureWatch to automatically reboot or re-image servers saving clients time and money on servers that leak memory, disk space or other resources.

As an add-on available through the Windows Azure Store, IT pros can now quickly deploy AzureWatch from inside the Windows Azure management portal. The availability of AzureWatch through the Windows Azure Store helps deliver fast and cost-saving Software-as-a-Service (SaaS) and HaaS-based monitoring and autoscaling of applications running in Windows Azure. Designed with a focus on simplicity and ease of use, AzureWatch can be quickly configured to address each customer’s unique monitoring needs. By automatically scaling Windows Azure services to match real-time demand, AzureWatch saves customers time and money and eliminates the need for manual monitoring of Windows Azure-based resources.

“The Windows Azure Store is an integral part of the developer experience and makes it easy for developers to find and purchase add-ons to help create great applications,” says Scott Guthrie, Corporate Vice President, Windows Azure, Microsoft. “We are pleased to welcome Paraleap to the Windows Azure Store and look forward to giving developers access to Paraleap’s monitoring and autoscaling technologies for their applications.”

“The future of information technology rests in the cloud; Microsoft has long been a global leader in providing a secure open cloud platform. With more and more companies moving services and solutions to the cloud, it is crucial that they have access to the comprehensive monitoring services that AzureWatch provides. Offering AzureWatch through the Windows Azure Store enables us to educate developers and IT pros on the important work we do and extend our services as developers build, deploy and maintain their applications.” says Igor Papirov, founder and CEO of Paraleap Technologies.

“We have been using AzureWatch for close to a year to monitor our Windows Azure app. It is an essential part of our tool belt,” says Hector Obregon, CEO of Formiik.com. “The daily performance reports and metrics history have helped us solve many problems and the e-mail notifications work great, we have those feed directly into our support desk.”

Pricing and Availability 

AzureWatch pricing is simple, affordable and based solely upon what is monitored. AzureWatch is offered in five packages to Windows Azure customers: 

  • Free Package: monitors up to 500 unit-hours per month - free 
  • Basic Package: monitors up to 5 units per month - $39.90 per month 
  • Plus Package: monitors up to 10 units per month - $74.90 per month 
  • Super Package: monitors up to 25 units per month - $159.90 per month 
  • Ultra Package: monitors up to 100 units per month - $549.90 per month

 

About Paraleap Technologies 

Paraleap Technologies is a Chicago-based software company focused on providing tools and services for cloud computing technologies. As the Paraleap’s flagship product, AzureWatch is designed to add dynamic scalability and monitoring to applications running on Microsoft’s Windows Azure cloud platform.



Healing-as-a-Service for Windows Azure

clock May 21, 2013 22:00 by author Igor Papirov

We are pleased to introduce a number of exciting new features to AzureWatch!

Support for multiple Azure subscriptions

Users can now instrument a single AzureWatch account to monitor and auto-scale any number of Azure Subscriptions (previously the limit was one).  Furthermore, users no longer need to transfer all of their diagnostic data into a single storage account.  AzureWatch will automatically detect which storage account a particular Role is transferring its diagnostic data to.  This should simplify account management, billing and configuration steps for AzureWatch users.

 

Support for alerts based on conditions of individual servers

In addition to receiving alerts based on metrics aggregated across all servers within a compute Role, AzureWatch users can now receive alerts based on conditions of individual servers.  This can help support personnel quickly pin-point problematic servers and take appropriate action.

 

Support for self-healing!

In addition to generating alerts for individual servers, AzureWatch can now automatically reboot or re-image servers!  This is particularly useful for servers that are leaking memory, disk space, or other resources.

 

Enhanced support for multiple mailboxes

AzureWatch can now send alerts to multiple mailboxes at an account level.  Furthermore, each individual alert can be further customized to be sent to more mailboxes if needed. This lets AzureWatch users send alerts of different priority levels or areas of responsibility to different contacts.

 

In Detail

Each rule now has a flag (1) that allows it to inspect metric data server-by-server OR aggregated across all servers within a Role.  If a Rule is inspecting metric data across all servers within a Role (the default scenario) it can auto-scale servers within that Role and send out alerts based on general conditions of all servers within a Role.  However, if users choose to evaluate a particular rule on an individual server basis, it  can reboot or re-image servers instead (2).  This is obviously very useful for conditions related to memory leaks, fragmented or leaking disk space, accumulating "stuck" IIS requests, etc.  While AzureWatch uses Azure Service Management API in order to request rebooting and re-imaging of servers, users should exercise caution and consider putting appropriate thresholds or throttling in place to avoid inappropriate reboots.  

When performing any sort of an action based on Rule (scaling, rebooting, or simply alerting), AzureWatch will send notification to every mailbox specified at the account level.  However, more mailboxes can be configured at an individual Rule level (3).

 

Remarks

When Azure reboots a server (role instance), it will take it offline, restart the underlying operating system for that server, and brings the role instance back online. Any data that is written to the local disk is persisted across reboots. Any data that is in-memory is lost.

When Azure reimages a server (role instance), it will take it offline and write a fresh installation of the Windows Azure guest operating system to the virtual machine. The instance is then brought back online. Windows Azure attempts to maintain data in any local storage resources when the server is reimaged; however, in case of a transient hardware failure, the local storage resource may be lost.  Any data that is written to a local directory other than that defined by the local storage resource will be lost when the instance is reimaged.



Global Azure Bootcamp participants - Welcome!

clock April 26, 2013 07:03 by author Igor Papirov

Paraleap Technologies are proud to sponsor this year's Windows Azure Global Bootcamp.  It is exciting to see a community-based event of such large proportions organized around Windows Azure.  Our commitment to provide a free unlimited 1-month license to every participant has not wavered even as the size of potential giveaways grew from hundreds to nearly 10,000!  We are hopeful to welcome many new happy users to our every-growing user community.

As we get ready for the rush of signups, we would like to encourage new users to read the following instructions carefully in order to make their on-boarding as smooth as possible.  Our support personnel will be working non-stop throughout this time to make sure everyone's signup and monitoring experience is impeccable but it is unlikely that we would be able to reply to every question in minutes or hours, like we usually do.

  • In order to receive your free subscription, you will need a promo code from Bootcamp's organizers.  Please contact event organizers if you have not received your code.  We will not be able to provide any additional promo codes at this time.
  • Please read setup instructions to ensure your configuration stage goes smoothly
  • Please note that every screen within AzureWatch Management Portal contains an "i" button in the top right corner that provides context-sensitive help and detailed instructions
  • Please note that AzureWatch Management Portal explorer contains a Help branch with detailed instructions on various subjects of monitoring and auto-scaling 
  • Unless your question contains sensitive information, please post it on our support forums - we monitor those forums as closely as we monitor our support ticket queue, however, every answer on those forums can help hundreds of other users who may have a similar question.
  • If you have checked out AzureWatch and no longer need its monitoring capabilities we ask you to kindly Stop Monitoring and help us conserve our compute resources

 

Some frequently asked questions:

  • While we are firmly committed to supporting the full spectrum of Windows Azure platform, at this time AzureWatch does *not* yet support monitoring of Azure Virtual Machines (IAAS), Media Services, and HDInsight (Hadoop).  We *do* support Windows Azure Cloud Services (Web/Worker Roles), Azure Websites, SQL Azure, Federations, Storage, and portions of Service Bus 
  • If you would like to monitor multiple Azure subscriptions, you will need to open multiple AzureWatch accounts.  This will change soon, but as of right now, we limit one account to one subscription


Support for monitoring and auto-scaling of Azure Websites

clock April 9, 2013 15:01 by author Igor Papirov

Last week was very exciting for us at Paraleap:

  • First, we revamped our website with a crisp and modern look.  If you have a few minutes, please share with us your thoughts on the new design.
  • Second, we soft-launched a major new feature to AzureWatch: support for monitoring and auto-scaling of Azure Websites.  Users can now receive alerts or automatically scale the number of instances dedicated to Windows Azure websites.  There are no agents to install and no code to redeploy.  We simply hook into the Azure Service Management API, grab the running totals for standard metrics, extrapolate their rates of change and take this data through our rule-based monitoring and scaling engines.  Users can configure alerts and scale rules using the already familiar interface of AzureWatch Management Portal.


AzureWatch is Overhauled to Simplify and Expand on Monitoring and Autoscaling for Windows Azure Apps

clock March 6, 2013 02:09 by author Igor Papirov

Expanding on its popular line of monitoring products, Paraleap Technologies today announced public availability of the newest edition of AzureWatch, a monitoring and autoscaling service for applications running on top of Microsoft’s Windows Azure. The newest version introduces AzureWatch Management Portal that simplifies the configuration process allowing companies to quickly establish monitoring and autoscaling rules that save time and money. Updates in this version include:

• Revamped online interface that no longer needs desktop installation or agent software 
• Support for monitoring of SQL Azure, SQL Federations, Service Bus, and Azure Storage 
• Support for monitoring of websites and SSL certificates 
• Live performance dashboards 
• Streamlined configuration setup featuring revamped Setup Wizard

Since 2010 AzureWatch has provided monitoring and autoscaling solutions to hundreds of companies that rely on the Windows Azure platform. “With more and more companies moving services and solutions to the cloud, it is crucial that they have access to comprehensive monitoring services,” said Igor Papirov, founder and CEO of Paraleap Technologies. “Any minute when the health of your applications is compromised can impact not only your company’s bottom line but its reputation as well.”

“We have been using AzureWatch for close to a year to monitor our Azure app. It is an essential part of our tool belt,” says Hector Obregon, CEO of Formiik.com. “The daily performance reports and metrics history have helped us solve many problems and the e-mail notifications work great, we have those feed directly into our support desk.” Designed with a focus on simplicity and ease of use, AzureWatch can be quickly configured to address each customer’s unique monitoring needs. By automatically scaling Azure services to match real-time demand, AzureWatch saves customer’s time and money and eliminates the need for manual monitoring of Azure based resources. More information and a free preview are available at http://www.paraleap.com

About Paraleap Technologies 
Paraleap Technologies is a Chicago-based software company focused on providing tools and services for cloud computing technologies. As the Paraleap’s flagship product, AzureWatch is designed to add dynamic scalability and monitoring to applications running on Microsoft Windows Azure cloud platform.



Impact of the Azure Storage outage caused by expired SSL Certificates

clock February 26, 2013 00:08 by author Igor Papirov

As everyone is probably aware, Windows Azure suffered a world-wide Azure Storage outage on Friday 02/22/2013.  This outage was caused by an expired Microsoft SSL certificate.  The outage impacted Azure Storage, Azure Websites, Service Bus, Media Services, ACS, and Azure Management Portal and lasted for approximately ten hours.

AzureWatch monitors remained active during the outage.  Customers who monitored their storage accounts with "Alert on Failure" option turned on, began receiving alerts at approximately 8:32PM UTC (well in advance of any outage notice on Microsoft's Azure Dashboard).  

It is important to point out that AzureWatch's monitoring was impacted because key metrics located within our customers' Azure deployments were inaccessible.  Furthermore, AzureWatch Management Portal was unavailable because it currently relies on Azure Storage.  To help mitigate our outage, our engineers were available via recently implemented online chat interface to provide extra support to customers logging into our portal.

Overall, this outage yet again underscores the importance of monitoring.  No cloud provider will have a perfect up-time record.  Thus, the faster you know that something is wrong, the faster you can react with a contingency plan.  Even if you do not require all the sophisticated features of AzureWatch, we suggest that you at the minimum use our simple but effective free monitoring utility AzurePing that can send you alerts when it is unable to access your Azure resources.



AzureWatch Monitoring - Azure-related Outage

clock February 23, 2013 06:18 by author Igor Papirov

We are seeing world-wide connectivity errors connecting to Azure Storage.  Problems appear to be caused by an expired Azure SSL certificate

This Azure-related outage is impacting AzureWatch's ability to monitor customers subscriptions.  We apologize for the service degradation

 



AzureWatch Management Portal - Customer Preview Invitation

clock October 25, 2012 20:55 by author Igor Papirov

Things have been very busy at Paraleap Technologies.  Since the start of summer, we have been working on creating a new version of the AzureWatch configuration and monitoring interface.  The result of this effort is a great-looking AzureWatch management portal, preview of which I am proud to invite you to try.  In the forthcoming months, we will be decommissioning our legacy Windows-based Control Panel application and migrating all of our customers onto our new AzureWatch management portal.

 
The online portal can be found here
Instructions for usage can be found here
Sign-up link for free trial account can be found here
 
 
We would appreciate your candid feedback about our new portal.  Your feedback has been the main driver behind majority of the AzureWatch features in the last two years and we plan to continue to innovate and deliver additional value to you and your organization!

Important release notes about the preview of our new online AzureWatch management portal
  • Please keep in mind that the functionality offered during the customer preview phase may contain minor bugs and is subject to change. Let us know if you encounter anything strange!
  • For existing customers, all of your current configuration and monitoring data in the new portal is live and is configurable via the new interface - simply use your existing credentials to login
  • New setup Wizard contains a lot more default rules for monitoring of SQL Azure, Storage and URLs.  The Wizard can also be invoked at any time, not just when new deployments have been detected
  • The dashboard screen now self-updates once per minute (no more clicking on the Refresh button)
  • Creation of monitoring rules has been streamlined: no need to define Raw metrics upfront.  Simply create your aggregate metrics and use them in Rules
  • Greater visibility into your AzureWatch billing history and consumption
  • Once migration is complete, client-side agent-based monitoring option will be sunset along with the desktop Control Panel.  We will only continue to support our cloud-based monitoring.  We will work with the handful of you who are utilizing client-side monitoring so that we can migrate you over to our cloud-based monitoring.
Our desktop Control Panel has been around for nearly two years and it's eminent deprecation will enable resolution to certain nagging issues that have persisted throughout the lifetime of the service
  • No more installation issues related to Windows 8 incompatibilities, firewalls, or anti-virus warnings
  • No more login issues related to inaccurate time-clock on local Windows computers
  • No more intermittent resolution-related issues that caused our UI to look misaligned
  • No more account lockouts due to password-reset related issues
  • Full accessibility of AzureWatch monitoring tools on non-Windows machines and tablets
Please feel free to visit the new AzureWatch management portal at https://my.paraleap.com and don't forget to click on the "Beta Feedback" to provide us with your ideas and comments.

Thank you for your continued support and interest in our service!

Igor Papirov
President & CEO
Paraleap Technologies
igor@paraleap.com

 



Introducing Automated Daily Performance Reports

clock July 18, 2012 09:04 by author Igor Papirov

We are excited to introduce a new AzureWatch feature: daily performance charts delivered via email to our users.  This feature was requested rather frequently in the recent months, and we are happy to oblige.  No action is required to enable this feature.  Every active account (whether trial or paid) will automatically begin receiving daily reports free of charge.

We know that this new capability will provide AzureWatch users with greater insight into the performance of their applications running on top of the Windows Azure platform.



SQL Azure Federations - active monitoring setup and instructions

clock April 8, 2012 21:25 by author Igor Papirov

We are excited to introduce support for active monitoring of SQL Azure Federations!  Federations is one of the newer members to SQL Azure family.  Capable of mega-scale and unlimited size, SQL Azure Federations allow customers to distribute their data and processing power across multiple federated members (shards). While each federated member is a unique database, connecting to and keeping an active monitoring eye on all of these databases is challenging.  By using AzureWatch, database administrators can now actively monitor all federated members and be immediately notified when any one of them is getting too large, too slow, or needs other special attention.  

Experienced AzureWatch users and newcomers alike will find the setup of SQL Azure Federations to be relatively simple.  A new option on the navigation explorer enables entry of federations that need to be monitored.  

Enter the required credentials and make sure that your desktop PC has firewall access to your SQL Azure federated databases, so that the locally running AzureWatch Control Panel can connect and verify the information

Once the monitoring is running, AzureWatch will query the root database and all federated members once per minute in order to store, aggregate and evaluate vital statistics.  At the time of this writing AzureWatch will gather the following:

  • Database Size
  • Number of open transactions
  • Number of open connections
  • Number of blocking queries
  • Number of federated members (root database only)

 

In order to send out alerts when important events happen, AzureWatch will need to know how information needs to be aggregated, for how long and what are the rules that govern the generation of each alert. Users will want to decide if metrics need to be aggregated across all federated members together or by individual federated member.  For example, it may be important to know when any one of federated members has exceeded a certain size as this may be an indicator to consider a SPLIT of that federated member.  Alternatively, users may want to know when the total number of blocking queries against all federated members exceeds a certain threshold as this may indicate a problem with the application itself and would require further investigation.

This article will walk through the configuration of an alert that would trigger when any one of the federated members has exceeded 10GB in size.

Alerts generated by AzureWatch will trigger when Rules evaluate to TRUE.  Each Rule consists of a custom boolean formula that compares aggregate metrics.  For the example at hand, an aggregate metric defined by Federation Member must be setup to aggregate the Database Size raw metric.  To do so, create a new aggregate under the By Federation Member\Aggregate Metrics section as shown in the example below.

Final step, the configuration of the actual Rule that would send out an alert

The alert Rule is represented by a formula that uses aggregate metrics as variables and sends out an Email notification when its formula evaluates to TRUE.  In order to prevent a barrage of emails, setting a threshold for how frequently such emails would be sent might be prudent.  (In our example no more than once per 20 minutes)



Active monitoring of Azure Storage, websites, and SQL Azure

clock March 7, 2012 00:46 by author Igor Papirov

In the last few weeks, we released a colossal amount of new features to AzureWatch! Active monitoring of Websites, Azure Storage accounts, and SQL Azure databases just to name a few. We've also drastically improved capabilities of our Rules engine to allow for more complex rules.  In addition, we improved our Android and Mobile applications to allow for more streamlined access at seeing Azure application's health from smartphones.

For over a year now, AzureWatch has been helping customers with monitoring and auto-scaling of their systems in Windows Azure.  We're very excited about the new capabilities of our service and are looking forward to working on our next major featureset: implementation of monitoring of SQL Azure Federations.



Monitor your Azure Storage accounts with new version of AzureWatch!

clock March 5, 2012 22:17 by author Igor Papirov

Version 2.0.2 of AzureWatch introduces new capability to the product: monitoring and alerts of key performance and connectivity characteristics for Azure Storage accounts.

Once a minute, AzureWatch will connect to queue, blob and table storage endpoints of monitored Azure Storage accounts, and attempt to create and then purge a test queue message, a test blob, and a test table eneity object.  AzureWatch will capture key performance indicators as well as any storage errors that occur so that you can be notified immediately of any problems.

 

Setup instructions:

Find Azure Storage section in the Control Panel explorer

 

Add new Storage Account by right-clicking in the Azure Storage listing area

 

 

These simple steps will make sure that AzureWatch emails you if it was unable to perform all of its monitoring actions within the Timeout specified.  In addition, if you'd like to know when Azure Storage is simply slowing down, create an aggregate and a rule around the ResponseTime_msec metric:

 

To create an aggregate, simply visit Aggregate Metrics screen under Azure Storage section of Explorer:

 

And finally, get notified by a Rule when this aggregate is outside a safety threshold:



Monitoring Websites - setup instructions

clock February 28, 2012 00:01 by author Igor Papirov

v2.0.1 of AzureWatch enables true Website monitoring.

If configured, AzureWatch will ping your webpages once per minute.  If connection is refused or non-OK response is received, AzureWatch will immediately send out an email.  AzureWatch will also gather vital performance statistics in order to enable delivery of custom alerts via its rule engine and provide visualization of performance related data on its desktop, browser and smartphone UIs.

 

Setup and configuration of this feature is very straightforward.  

Define what websites need to be monitored and instrument them under Monitored Systems\Website section of the AzureWatch Control Panel tool.  Right-click on the listings screen in order to define new monitored website:

AzureWatch monitored website definition

 

Creating of an aggregate metric definition for monitoring of websites follows the same approach as creating aggregates for other types of metrics:

 

And lastly, preparing delivery of alerts when Average response time for the last 5 minutes exceeds 5 seconds can be done with definition of a sampe rule below:

 



AzureWatch now supports SQL Azure Monitoring!

clock February 23, 2012 01:07 by author Igor Papirov

We are happy to announce that AzureWatch now supports SQL Azure monitoring!  Please view the release notes below for detailed information on new features rolled out in AzureWatch v2.0.  


We have spent a lot of time working on this release as it lays down the foundation for us being able to monitor not only SQL Azure databases, but also other resources such as AppFabric Cache namespaces, Azure Storage accounts, and even availability and performance characteristics of public web pages.  We are planning to release support for these features in the next weeks as our development and quality assurance processes complete.  

You can download the latest version of AzureWatch Control Panel configuration tool here and view sample instructions regarding setup of monitoring for SQL Azure here
 
AzureWatch v2.0 Release Notes
  • Ability to monitor SQL Azure databases (extra charges may apply)
    • Get notified on connection failures
    • Monitor blocking queries
    • Monitor response times
    • Monitor database size
    • Monitor open connections
    • Monitor open transactions
  • Enhancements to the Rules engine include
    • Access current date & time during rule evaluation.  This can help with scheduling of events
    • Interrogate quantity of raw metrics aggregated during the rule evaluation.  This can help with sophisticated running-averages
  • Metric Reports have been enhanced
    • Metric data and charts are now shown in local time zone rather than UTC
    • Reports can be quickly viewed by using a predefined range dropdown
    • Charts have been redesigned to show more data
  • A number of smaller bugs fixed throughout the AzureWatch Android app, our reporting mobile site and the Control Panel configuration utility

 



Monitoring SQL Azure - setup instructions

clock February 22, 2012 23:50 by author Igor Papirov

SQL Azure Monitoring

The actual setup of SQL Azure monitoring is fairly straightforward.  Upon successful upgrade to the latest version of Control Panel (v2.0 at the time of this post), simply visit the newly available SQL Azure option in your Control Panel explorer.
 
Add databases to be monitored by right clicking anywhere in the SQL Azure Instances window.
 
SQL Azure monitoring setup
 
Fill out connection information for your SQL Azure database
 
SQL Azure connection information
 
Upon successfull setup of a SQL Azure database, AzureWatch will automatically instrument the capture of five key metrics: Database size, # of Open connections, # of Blocking Queries, # of Open Transactions and general Response Time.
 
It is time to aggregate the metrics over some periods of time, so that AzureWatch can properly alert you in case your Rules trigger.
 
Setting up of aggregations is performed via the next option in explorer: Aggregate Metrics.
 
SQL Azure rule evaluation
 
Color ranges will help AzureWatch highlight key metrics on the dashboard
 
And finally, the setup of alerts can be performed from the Rules screen located underneath Aggregate Metrics
 
SQL Azure alert
 
Congratulations! We just instructed AzureWatch to notify us if in the last 5 minutes our SQL Azure database was running any blocking queries.

 



Free Azure resource-monitoring utility: AzurePing

clock October 25, 2011 07:49 by author Igor Papirov

We are pleased to announce the release of AzurePing: a free Azure resource-monitoring utility.  AzurePing is a simple Windows Service that pings any number of Azure Storage resources, SQL (Azure) databases, and web URL's on a continuous basis.  Any errors are logged through log4net framework via a variety of appenders, such as email, SQL, flat files, Trace, etc.  For those not familiar with log4net, it is a popular open-source logging framework that can store logging entries to a variety of extendable appenders.

To find more information about AzurePing, visit our website at http://www.paraleap.com/azureping

And please help us spread the word about AzurePing!



Managing environments in a distributed, Azure, or other cloud based Visual Studio solution

clock September 13, 2011 05:00 by author Igor Papirov
During development, testing and support stages of the project, there is usually a need to test or debug code against multiple environments: Local with and without Azure emulation, Dev environment, QA environment, UAT environment, perhaps even Prod environment, etc.  Configuration changes can get very complicated when servers, URL’s, connection strings, etc. need to switch in unison when switching between different environments.  In general, handling more than one environment from Visual Studio can be very laborious when the overall project structure is complex.  Throwing Windows Azure in the mix only adds to debugging and deployment vows.

This article will describe techniques that we use to manage configuration changes among various projects in a large distributed Azure-based solution called AzureWatch and keep settings in sync as various environments are targeted during development, testing, and debugging sessions.  AzureWatch is an auto-scaling and monitoring system for Windows Azure applications.  By its nature it utilizes multiple WCF services, a few Windows-based clients and services, a number of web-based projects.  As readers can imagine, keeping all configuration settings in sync and "shifting" them together on demand without making a mistake can be challenging.

Visual Studio technologies utilized

 

Approach at a high level

For every debugging environment, create Visual Studio Configuration.  Break out sections from every .Config file that vary by environment and include them back via ConfigSource settings.  Include these sections as unique files into a separate Config project that has folders for every Configuration.  Also, copy *.csdef and *.csccfg files from Azure-related projects into these folders as well. In each folder, customize the broken off partial configuration files (that are included via ConfigSource) and azure configuration files. Create Post or Pre-Build event of the Config project to xcopy the files from a folder that matches current Configuration environment to root folder of Config project.  Include files from root folder of Config project into other projects as needed via “Add as Link” command.

Utilize Config transformations to instrument Production (and other environments if needed) specific configuration changes that deal with security, debugging, tracing, etc.

 

Now, let's look at these steps in detail:

 

Step 1

Create as many Visual Studio Configurations as necessary to support number of different environments that developers must be able to debug from Visual Studio

 

Step 2

Create a separate empty project that is a part of the overall solution called Config.  Include into the project folders, one per matching Visual Studio Configuration environment that were created in Step 1

 

Step 3

Break-off Connection strings, end-point configuration, and other environment-specific sections from existing .config files into separate “include-only” config sections.

Source example web.config:

  <connectionStrings configSource="config\connectionStrings.config"/>

Broken off example connectionStrings.config:

  <connectionStrings>

    <add connectionString="..." name="db_connection"></add>

  </connectionStrings>

 

May want to make sure that web-projects have "bin" folder as a part of the relative path: "\bin\config\partialConfigFileName.config"

Step 4

Include separate customized partial .config files and Azure-specific .cscfg files under every sub-folder within Config project

 

 

 

 

 

Step 5

Customize Pre-Build event for Config project to copy config files for current configuration to root folder:

xcopy /Y /R "$(ProjectDir)$(ConfigurationName)\*.config" "$(ProjectDir)"

xcopy /Y /R "$(ProjectDir)$(ConfigurationName)\*.cscfg" "$(ProjectDir)"

 

Step 6

Add-as-Link partial .config files into main projects from the ROOT of the Config project.  To keep things organized, feel free to include partial .config files under Config sub-folders.  Do the same with Azure-specific .cscfg files

Files added as links have a shortcut icon in Solution Explorer

Step 7

Manually add project dependencies to Config project from the other projects in the solution, that need partial config files to function.  This will ensure that Config project is built before other projects.  Project Dependencies can be found inside Solution Properties window, under "Project Dependencies" tab.

Do not forget to specify that partial config files set "Copy to Output Directory" as "Copy if Newer"

 

Step 8

You can still use Config Transformations to strip out debug information from Release configuration setting that will be ultimately used for publishing.

 

Conclusion

Now, every time you debug and run, the pre-compile event from Config project will copy all the files from its particular environment folder into its root folder.  And all other projects that link to this root folder for their partial .config files will automatically get updated .configs switched to proper environment in unison.

 

Revisions

This blog was revised on 06/10/2012 to include the extra XCOPY step that keeps Azure cscfg/csdef files in sync with the Config project

 



Auto scaling strategies for Windows Azure, Amazon's EC2 and other cloud platforms

clock May 23, 2011 17:05 by author Igor Papirov

The strategies discussed in this article can be applied to any cloud platform that has an ability to dynamically provision compute resources, even though I rely on examples from AzureWatch auto scaling and monitoring service for Windows Azure

The topic of auto scaling is an extremely important one when it comes to architecting cloud-based systems.  The major premise of cloud computing is its utility based approach to on-demand provisioning and de-provisioning of resources while paying only for what has been consumed.  It only makes sense to give the matter of dynamic provisioning and auto scaling a great deal of thought when designing your system to live in the cloud.  Implementing a cloud-based application without auto scaling is like installing an air-conditioner without a thermostat: one either needs to constantly monitor and manually adjust the needed temperature or pray that outside temperature never changes.

Many cloud platforms such as Amazon's EC2 or Windows Azure do not automatically adjust compute power dedicated to applications running on their platforms.  Instead, they rely upon various tools and services to provide dynamic auto scaling.  For applications running in Amazon cloud, auto-scaling is offered by Amazon itself via a service CloudWatch as well as third party vendors such as RightScale.  Windows Azure does not have its own auto scaling engine but third party vendors such as AzureWatch can provide auto scaling and monitoring.

Before deciding on when to scale up and down, it is important to understand when and why changes in demand occur.  In general, demand on your application can vary due to planned or unplanned events.  Thus it is important to initially divide your scaling strategies into these two broad categories: Predictable and Unpredictable demand.  

The goal of this article is to describe scaling strategies that gracefully handle unplanned and planned spikes in demand.  I'll use AzureWatch to demonstrate specific examples of how these strategies can be implemented in Windows Azure environment.  Important note: even though this article will mostly talk about scale up techniques, do not forget to think about matching scale down techniques. In some cases, it may help to think about building an auto scaling strategy in a way similar to building a thermostat.

 

Unpredictable demand

Conceptualizing use-cases of rarely occurring unplanned spikes in demand is rather straight forward.  Demand on your app may suddenly increase due to a number of various causes, such as:

  • an article about your website was published on a popular website (the Slashdot effect)
  • CEO of your company just ordered a number of complex reports before a big meeting with shareholders
  • your marketing department just ran a successful ad campaign and forgot to tell you about the possible influx of new users
  • a large overseas customer signed up overnight and started consuming a lot of resources

Whatever the case may be, having an insurance policy that deals with such unplanned spikes in demand is not just smart.  It may help save your reputation and reputation of your company.  However, gracefully handling unplanned spikes in demand can be difficult.  This is because you are reacting to events that have already happened.  There are two recommended ways of handling unplanned spikes:

Strategy 1: React to unpredictable demand

When utilization metrics are indicating high load, simply react by scaling up.  Such utilization metrics can usually include CPU utilization, amount of requests per second, number of concurrent users, amounts of bytes transferred, or amount of memory used by your application.  In AzureWatch you can configure scaling rules that aggregate such metrics over some amount of time and across all servers in the application role and then issue a scale up command when some set of averaged metrics is above a certain threshold.  In cases when multiple metrics indicate change in demand, it may also be a good idea to find a "common scaling unit", that would unify all relevant metrics together into one number.

 

Strategy 2: React to rate of change in unpredictable demand

Since scale-up and scale-down events take some time to execute, it may be better to interrogate the rate of increase or decrease of demand and start scaling ahead of time: when moving averages indicate acceleration or deceleration of demand.  As an example, in AzureWatch's rule-based scaling engine, such event can be represented by a rule that interrogates Average CPU utilization over a short period of time in contrast to CPU utilization over a longer period of time

 

(Fig: scale up when Average CPU utilization for the last 20 minutes is 20% higher than Average CPU utilization over the last hour and Average CPU utilization is already significant by being over 50%)

Also, it is important to keep in mind that scaling events with this approach may trigger at times when it is not really needed: high rate of increase will not always manifest itself in the actual demand that justifies scaling up.  However, in many instances it may be worth it to be on the safe side rather than on the cheap side.

 

Predictable demand

While reacting to changes in demand may be a decent insurance policy for websites with potential for unpredictable bursts in traffic, actually knowing when demand is going to be needed before it is really needed is the best way to handle auto scaling.  There are two very different ways to predict an increase or decrease in load on your application.  One way follows a pattern of demand based on historical performance and is usually schedule-based, while another is based on some sort of a "processing queue".

 

Strategy 3: Predictable demand based on time of day

There are frequently situations when load on the application is known ahead of time.  Perhaps it is between 7am and 7pm when a line-of-business (LOB) application is accessed by employees of a company, or perhaps it is during lunch and dinner times for an application that processes restaurant orders.  Whichever it may be, the more you know at what times during the day the demand will spike, the better off your scaling strategy will be.  AzureWatch handles this by allowing to specify scheduling aspects into execution of scaling rules.

 

 

Strategy 4: Predictable demand based on amount of work left to do

While schedule-based demand predictions are great if they exist, not all applications have consistent times of day when demand changes.  If your application utilizes some sort of a job-scheduling approach where the load on the application can be determined by the amount of jobs waiting to be processed, setting up scaling rules based on such metric may work best.  Benefits of asynchronous or batch job execution where heavy-duty processing is off-loaded to back-end servers can not only provide responsiveness and scalability to your application but also the amount of waiting-to-be-processed jobs can serve as an important leading metric in the ability to scale with better precision.  In Windows Azure, the preferred supported job-scheduling mechanism is via Queues based on Azure Storage.  AzureWatch provides an ability to create scaling rules based on the amount of messages waiting to be processed in such a queue.  For those not using Azure Queues, AzureWatch can also read custom metrics through a special XML-based interface.

 

Combining strategies

In the real world, implementing a combination of more than one of the above scaling strategies may be prudent.  Application administrators likely have some known patterns for their applications's behaviour that would define predictable bursting scenarios, but having an insurance policy that would handle unplanned bursts of demand may be important as well.  Understanding your demand and aligning scaling rules to work together is key to successful auto scaling implementation.

 



Squeeze out the every paid-for minute from Windows Azure!

clock May 20, 2011 17:26 by author Igor Papirov

Did you know that Windows Azure billing is done by the clock hour?  This means that if you scaled up an extra instance at 3:30pm and scaled down that instance at 4:30pm, you will incur charges for two hours: for 2-3pm clock hour, and for 3-4pm clock hour.  Thus, it may make sense to scale down only when the clock hour nears its completion.  To take advantage of this, latest version of AzureWatch introduces a new flag to allow for rules to be evaluated only during certain times in a clock hour.  Why scale-down in the beginning or middle of the hour, if the full hour was already paid for?



While Microsoft does not guarantee a specific time span within which instances will actually be deallocated after a scale-down request, Windows Azure is pretty good about deallocating instances only within a few minutes time.  In light of this, it might be prudent to not let the scale-down window to be too near the end of the clock hour, this can help avoid overruns into a new clock hour.



Stay informed about health of your Azure apps with an RSS feed!

clock May 8, 2011 10:16 by author Igor Papirov

We just rolled out a new feature for AzureWatch: an RSS feed containing information about the health of your Windows Azure application.  Simply login to our website and visit the Monitoring tab. From there on, you will be able to subscribe to an RSS feed specific to a particular Windows Azure Role or to all Roles at the same time.  

With the proliferation of RSS readers out there, you'll be able to check on the health of your Azure application at a moment's notice and without any effort.

 

Azure Monitoring RSS preview



New support forums and new version of AzureWatch

clock May 5, 2011 10:50 by author Igor Papirov

There are a few developments with AzureWatch, an autoscaling and performance monitoring service for Windows Azure applications, that you may be interested in:

  • We have a new Q&A support forum, with friendly design inspired by Stackoverflow.com.  I invite you all to participate, not only if you have AzureWatch specific questions, but also if your questions are about Windows Azure in general.  Our goal is to have this forum serve as another place to brainstorm autoscaling, monitoring, or other important Windows Azure-related architectural strategies.
  • Work continues on the mobile version of AzureWatch that is geared toward showing statistics and reports on smartphones and tablets running HTML5!
  • I will be demonstrating AzureWatch at the Azure Users Group on June 8th, 2011.  Join me!
  • Latest version of AzureWatch released into production over this weekend contains a small but handy new dashboard snippet that shows color-coded Ready statuses of all your monitored Azure instances.

 

Instance Statuses on Dashboard



AzureWatch v1.1.0 released

clock April 15, 2011 17:00 by author Igor Papirov

We are pleased to announce that v1.1.0 version of AzureWatch has been pushed to production.

In this release we have added support for gathering of custom metrics from web-points provided by you.  This is a fairly important and powerful feature.  In order to be able to monitor even greater variety of metrics, we can now poll web-pages located within your application.  We expect that these web-pages would provide AzureWatch with a simple XML containing metric values.  Certain metrics are just best suited to be polled.  Examples would be: size of SQL Azure database, number of registered users, number of placed orders in the last 24hrs or any other metric that is more associated with your application rather than a particular Azure instance.  As you launch AzureWatch Control Panel tool, you will notice new "External Links" screen that will allow you to enter URL's for these web-pages.  Expected XML format is provided right on the instructions screen (it will also be published on our website).  This feature is currently considered to be in beta.

Another, perhaps minor, but very useful new feature, allows you to throttle scaling events or email notifications at a finer level of detail.  In addition to a global cool-down timer for scaling events, each rule now has its own "delay" timer that would prevent its execution for a period of time.  This is very useful to prevent notification rules from firing too frequently and generating a flood of emails, or to allow Scale-Down rules to have different cool-down timers vs. Scale-Up rules.

We are also working on a version of a mobile application geared toward smartphones and tablets running HTML5!

As a reminder, if you did not get a chance to read our last newsletters:

  • We have relaxed our Free Trial conditions.  All previously expired trial accounts that have not used 500 monitored hours are active again!  Login to our website and see if your account is again active! 
  • AzureWatch can now monitor your Azure deployments directly from its cloud-based servers.
  • Latest version of our configuration tool can be found here.

 



Data Storage in Azure: SQL Azure or Azure Table Services?

clock April 5, 2011 14:16 by author Igor Papirov

A question frequently asked during architectural stages of a new Azure-based project: should we choose SQL Azure or Azure Table Storage to store our data?

Answer is typically: YES. Both of these storage technologies compliment rather than compete with each other.  If you are looking to gain maximum scalability, performance, and flexibility while at the same time paying the least amount of money, the trick is to understand the strong points of each technology and utilize both effectively.

Azure Table Services (ATS) is a new storage technology from Microsoft and is specifically designed to handle mega-scalability for applications and websites of Twitter/Facebook/Amazon's capacity.  Storage space is super cheap with ATS.  To offset the low cost and mega-scalability there are a few negative impacts that one must be aware of.  You are not only paying for storing the data, but also for accessing the data that lives in ATS.  There is also only limited support for transactions.  This is key to understand and design around.  ATS also is a new paradigm for developers to grasp.  Lastly, ATS forces your compute nodes to become mini-relational servers.  It simply does not do any of the relational processing we are all used to.  JOINs, GROUP BY's, ORDER BY's all have to be designed around or performed manually. 

I would say that ATS is best suited for large amounts of data that only rarely needs to be accessed or massaged.

On the other hand, SQL Azure is a fast, lighter-weight cloud-based relational storage.  Your developers will know how to code against it right away, because (barring a few small gotchas) coding for it simply like coding for any other SQL database.  On the negative side: SQL Azure is not meant to support huge applications like Facebook, Ebay, Twitter, etc.  Azure, stores your SQL Azure databases along with others in a multi-tenant environment on same servers and thus it has to throttle access should your application become too hot and impacts other databases on the same SQL Azure node.  SQL Azure is also somewhat pricey, at $10/gigabyte/month.  Having said, there is no cost to access the data stored in SQL Azure and there is plenty of CPU power dedicated to perform relational functionality on your data.  

I would recommend that SQL Azure for smaller amounts of frequently accessed data.

 

A few typical use-cases to illustrate the points:

Data in a banking system that stores customer account information along side a large amount of financial transactions would best divided across SQL Azure and ATS in the following way: Customer account information in SQL Azure, Financial Transactions in ATS.

Content for a large blog site should probably live in SQL Azure until it reaches a certain age and can be archived to ATS.



Custom Performance Counters support

clock March 20, 2011 09:00 by author Igor Papirov

We continue to innovate at a rapid pace.  Latest (v1.0.56) version of AzureWatch has been released.  Two key new features are:

  • Support for custom performance counters.  This feature allows AzureWatch users creation of scaling rules, notifications and alerts based upon highly customizable developer-created performance metrics.  For more information on creation of custom performance counters, visit this helpful blog.
  • Copy & Paste functionality for metric and rule setup.  This feature makes it easier for setting up similar metrics and rules across deployments with many roles.


If you did not get a chance to read our last newsletters:

  • We have relaxed our Free Trial conditions.  All previously expired trial accounts that have not used 500 monitored hours are active again!  Login to our website and see if your account is again active! 
  • AzureWatch can now monitor your Azure deployments directly from its cloud-based servers.
  • Latest version of our configuration tool can be found here.


As always, thank you for your continued interest in AzureWatch.  We appreciate you spreading the word about our service and welcome you to contact us with any questions, issues, or suggestions.

 



Server-side AzureWatch is released!

clock March 15, 2011 09:00 by author Igor Papirov

Thanks to your feedback, we are relaxing our Free Trial conditions.  We are pleased to announce that all trial accounts are now eligible for 500 free instance-hours or 14 days of unlimited usage - whichever is less restrictive.  Therefore we are activating all previously expired trial accounts that have not exhausted the 500 hour limit.  Login to our website and see if your account is now active!  Latest configuration tool is available for download here.

More news: you are no longer required to host on-premise monitoring agent.  AzureWatch can now monitor your Azure deployments directly from its cloud-based servers.

We have done a significant amount of work on handling certificates.  If you were having issues before with setting up certificates, we encourage you to try again.  For your convenience we have also published a small article on how to setup certificates with AzureWatch here.

Users running server-side beta-client, please update your clients to latest version found at here
And finally, AzureWatch system is backup from tonight's planned downtime.  The outage lasted approximately one hour.  We apologize for the inconvenience.

As always, thank you for your continued interest in AzureWatch.  We appreciate you spreading the word about our service and welcome you to contact us with any questions, issues, or suggestions.



Get informed about the health of your Azure applications "on the go"

clock February 21, 2011 17:09 by author Igor Papirov

Here is a tip that most administrators of Azure websites and applications will find invaluable.  Thanks to the scheduling features of AzureWatch, application administrators can setup up to receive regular email updates with all the key metrics that AzureWatch is monitoring for their Azure applications.  All it takes is a subscription to AzureWatch and a minute or two.

If you would like to receive email with latest peformance indicators values for your Azure application at certain intervals, say at 9am and 12pm and 3pm, simply configure three rules that always evaluate to true (ie: '1=1') to run between the hours of 8:59-9:01am, 11:59am-12:01pm and 2:59-3:01pm (assuming your rule-checking frequency is every 60 seconds).  Mark the rules as "Notification only" instead of "Scaling based", move them to the top of the sequential list and publish your changes. Take care to not make the time-ranges too wide, to avoid getting flooded with emails. AzureWatch will now execute new rules between the very narrow time ranges and email you values of all the variables it is tracking.



Microsoft Windows Azure Cloud's Elasticity is Enhanced by Chicago-based Paraleap Technologies

clock January 26, 2011 12:28 by author Igor Papirov

Paraleap Technologies created a unique Elasticity-as-a-Service offering designed to help applications running in Azure capitalize on the pay-for-use model of Microsoft's cloud technology.

Arlington Heights, IL January 26, 2011 -- Paraleap Technologies, a Chicago-based software company specializing in cloud-computing solutions, is proud to announce the long anticipated release of its flagship product AzureWatch, designed to provide dynamic scaling to applications running on Microsoft Windows Azure cloud platform.

After nearly a year of development and testing, AzureWatch is released with a purpose to deliver on true value of cloud computing: provide on-demand elasticity and just-in-time provisioning of compute resources for applications running in the cloud. By automatically scaling Azure compute nodes up or down to match real-time demand, AzureWatch gives customers confidence that their applications will automatically handle spikes and valleys in usage.

"Elasticity of compute resources is one of the primary driving forces behind the cloud computing revolution," says Igor Papirov, Founder and CEO of Paraleap Technologies. "Not only can our customers take advantage of this awesome benefit of the cloud by using our technology, but they can do so in under 10 minutes and without writing a single line of code."

Built with focus on simplicity and ease of use, AzureWatch can be setup in just minutes. While architected with a powerful and scalable engine to enable elasticity, AzureWatch also provides monitoring, alerts and comprehensive reporting services. Offered as a hybrid software-as-a-service, AzureWatch is available at http://www.paraleap.com and comes with a simple and affordable pricing model that can be used free with a 14-day trial period.

About Company
Founded in 2010, Paraleap Technologies is a Microsoft Bizspark startup specializing in software services for enabling cloud computing technologies. AzureWatch is Paraleap’s flagship product, designed to add dynamic scalability and monitoring to applications running on Microsoft Windows Azure cloud platform.

Contact Information
Igor Papirov, Founder & CEO
Paraleap Technologies
312-554-5327
igor@paraleap.com
http://www.paralep.com



Handling Unresponsive Windows Azure nodes

clock December 13, 2010 18:02 by author Igor Papirov

System downtimes can be more than just embarrassing . They can have negative impact on your revenue, customers, and reputation!  By using AzureWatch you can get notified in real-time in the event your Azure nodes become Unresponsive.  You can even configure AzureWatch to scale your application up when this happens!

Setting AzureWatch to monitor your node counts is fairly simple and straightforward.  AzureWatch already tracks how many nodes are Ready, Busy, Unresponsive, etc.  Simply setup an Aggregate Metric that calculates an average or last value of Unresponsive count for the last, say 5 minutes.  Then create a Rule that scales your Role UP when the newly created aggregated metric drops below a certain value.  It might be also good idea to place a safety check into the rule, to make sure that the system does not keep scaling up indefinitely .



Five tips for creating cost effective Windows Azure applications

clock December 7, 2010 02:05 by author Igor Papirov

Cloud-computing providers in general and Windows Azure in particular offer nearly infinite scalability, virtually unlimited capacity, blazing performance and extremely quick provision times.  However, to properly take advantage of these great benefits, teams need to plan ahead and understand all potential pitfalls and challenges.  One of the more significant differences between development of on-premise applications and cloud applications is a rather direct correlation between choices made during construction of an application and its support costs after deployment.  Because of the Windows Azure pricing model, every inefficient architectural decision and every inefficient line of code will show up as an extra line item on your Azure invoice.

This article will focus on a few actionable items that you can do today to minimize the cost of your Windows Azure application tomorrow.  The list of items is by no means exhaustive, but it will get you and your team thinking about the impact of your choices in a new dimension.

First, let's analyze the popular moving blocks when it comes to Windows Azure pricing. While seemingly straightforward individually, it is their combination together - and with obvious influence of already existing non-functional requirements for scalability, performance, security, availability, etc. - that make architecture in the cloud be a complex jigsaw puzzle.

Compute hours - quantity and size of nodes (servers) and charged by hour when online
Transfer costs - data that crosses Microsoft's data center boundaries is subject to transfer charges.
Azure Table Storage (ATS) costs - charge by gigabyte per month for the amount of space used
ATS transaction costs - charges for the amount of requests to ATS your application will make
Size of the SQL Azure databases - every database that you host in SQL Azure is charged by size

There are costs for other less frequently used services like Azure AppFabric or Content Delivery Network (CDN) that are not covered in this article.

Tip 1 - Avoid crossing data center boundaries

This is fairly straightforward.  Data that does not leave Microsoft data center is not subject to Transfer charges.  Keep your communication between compute nodes, SQL Azure, and Table Storage within the same data center as much as possible.  This is especially important for applications distributed among multiple geo-locations.  If you must communicate between different geo-locations, limit communication to non-transactional, batch calls that occur less frequently while utilizing compression where it makes sense to cut down on the amount of data transferred.  Employ caching technologies where possible.

Tip 2 - Minimize the number of compute hours by using auto scaling

Compute hours will likely make up the largest part of your Azure bill and thus need to receive the greatest amount of attention.  It is important to remember that Windows Azure does not automatically scale down the number of compute nodes, even if there is little or no demand on your application.  Architect for and plan to have an automatic scaling strategy in place, where the amount of nodes increases when demand spikes up and decreases when demand tapers off.  This can easily cut your bill for compute hours in half.  Implementing a comprehensive auto-scaling engine can be more complex than it sounds.  While there are a number of open-source examples that show the basics of how this can be done, it is also a perfect opportunity to outsource the auto-scaling to third party services such as AzureWatch.

In order for auto-scaling to be most effective, group your system components by their scaling strategies into Azure Roles.  It is important to keep in mind that if you need high availability of your components and want to take advantage of Azure SLA, you will need to maintain at least two online nodes for each Azure Role you have deployed.

Tip 3 - Use both Azure Table Storage (ATS) and SQL Azure

Try to not limit yourself to a choice between ATS or SQL Azure.  Instead, it would be best to figure out when to use both together to your advantage.  This is likely to be one of the tougher decisions that architects will need to make, as there are many compromises between relational storage of SQL Azure and highly scalable storage of ATS.  Neither technology is perfect for every situation.

On one hand accessing SQL Azure from within the boundaries of a data center is free and SQL Azure offers a familiar relational model which most developers will be comfortable with, transactions that assure data integrity, integration with popular ORM frameworks such as Entity Framework or NHibernate, and compatibility with numerous tools that work with relational databases.  On the other hand, ATS offers vastly greater scalability than SQL Azure and can hold a nearly infinite amount of data at a fraction of SQL Azure's cost.  You are charged, however, for every request made to ATS, even within the boundaries of a data center.

From a cost perspective, SQL Azure makes sense when access to data is not required to be highly scalable and when the amount of data is limited.  ATS makes sense for large amounts of data or when serious scalability is needed.

Tip 4 - ATS table modeling

If you have made the choice to use Azure Table Storage, you have essentially committed to converting parts of your data access components into mini database servers.  Setting Blob storage aside which is primarily used for media files or documents, ATS provides three levels of data hierarchy (Table, PartitionKey, and RowKey) that can be accessed and navigated extremely efficiently.  However, anything beyond that will require custom code and CPU cycles of your compute nodes.  This is the key difference to work around.  It would be prudent to spend a significant amount of time modeling table storage with appropriate Blobs, Tables, PartitionKeys and RowKeys to accommodate for efficient data storage and retrieval strategies.  This will not only speed up your transactions and minimize the amount of data transferred in and out of ATS, but also reduce the burden on you compute nodes that will be required to manipulate data and directly translate into cost savings across the board.

Tip 5 - Data purging in ATS

Because you are charged for every gigabyte stored in ATS, it may be prudent to have a data purging strategy.  However, while it may seem like a straightforward problem in a world of relational databases, this is not the case with ATS.  Since ATS is not relational, deletion of each and every row from an ATS table requires two transactions.  In certain cases it may be possible to delete a single row using only one transaction.  Either way, this is extremely slow, inefficient and expensive.  A better way would be to partition a single table into multiple versions (e.g. Sales2010, Sales2011, Sales2012, etc.) and purge obsolete data by deleting a version of a table at a time.

Conclusion

Shift to cloud computing represents a major leap forward and enables almost everyone, from small companies to large enterprise, reduce their capital expenses, minimize time to market and significantly decrease support costs.  By investing even small effort into planning ahead, cloud computing can result in meaningful savings and benefits.



Our first newsletter!

clock November 29, 2010 19:35 by author Igor Papirov

We are extremely excited to publish our very first newsletter.  It has been a month since the release of AzureWatch into public beta and the response has been overwhelmingly positive and constructive. Although the first day was somewhat rough since we managed to make a typo in the download URL(!), the month overall has been very productive and exciting. 

Since the initial announcement on November 1st, AzureWatch monitored over 3,000 compute instance hours and captured and analyzed over a million of performance metrics!  Furthermore, we have deployed over ten patches and one major release with brand new functionality which I would like to inform you about.

As of Friday, November 26th, AzureWatch was updated to support two important new features:

  • Historical reports.  Access to historical graphs and reports of your performance counters and other metrics.  With this feature, you are able to see historical values of your metrics for up to a month visually on a chart or in a table format.  You can drill down to actual raw values or simply look at the hourly or daily averages - all with just a few clicks of a button.  You can export or print the data too!
  • Tracking of instance statuses.  In addition to performance counters and queue lengths, AzureWatch now tracks instance counts by status.  It can differentiate between "Ready", "Busy", "Unresponsive", and other Azure Instance statuses.  You can now create rules using those metrics.  Want to get notified if any of your instances have become Unresponsive?  No problem, make a simple rule for that.  Want to scale UP if the number of Ready instances is below a certain threshold?  No problem, make a rule for that too.

We remain on target with releasing AzureWatch in production early in 2011.  We would like to thank you for your continued support and ask you to keep those bug reports and feature requests coming!



Setup auto-scaling for your Windows Azure applications in under 10 minutes!

clock November 10, 2010 15:26 by author Igor Papirov

So you are deploying your application on Windows Azure cloud platform.  One of the key features of a cloud platform like Azure is the ability to consume your compute resources with a utility model.  Pay for only what is used, whether storage space, compute power, or amount of data transferred. Dynamic allocation of more space or bandwidth is built-in to Azure, but with respect to compute power, Windows Azure allows you to issue scale up or scale-down commands relatively easily.  However, deciding when to do so can be a challenging task.  This blog entry describes a service AzureWatch and how it can dynamically scale Windows Azure applications.

Part One - Introduction

At its core, AzureWatch aggregates and analyzes performance counters, queue lengths, and other metrics and matches that data against user-defined rules. When a rule produces a "hit", a scaling action or a notification occurs.  You will need an account to install and use AzureWatch.  Follow this link to fill out a simple registration form.  After registration, download link for Windows-based configuration utility will be provided.

AzureWatch currently ships in two flavors: desktop edition and server-side edition.  Server-side AzureWatch will monitor and auto-scale your Azure applications from its cloud-based servers.  Desktop edition requires a special AzureWatch Monitoring agent to be installed on your premises.  In desktop-edition the agent is responsible for gathering metrics and initiating scaling events.

 

Part Two - Start Control Panel

After installation is complete, start AzureWatch ControlPanel and login with your newly created account.  You will be presented with a wizard to enter your Azure connection information. 

Subscription ID can be found on your Windows Azure developer portal.  If you do not already have the X.509 certificate, AzureWatch can create one for you.  Follow the hyperlink on the wizard to get detailed instructions on creation certificates.  It is a good idea to visit AzureWatch page to understand how your certificates and storage keys are kept secure.

After entering your account SubscriptionID and specifying a valid X.509 certificate, press Connect to Azure.  You will be presented with a list of storage accounts.  Storage account that is monitored by your Diagnostics Monitor is required.

 

On the next wizard page you can validate default settings for such things as throttle times, notification email, etc.

After the connection wizard is completed, AzureWatch will figure out what services, deployments and roles are present.  For each role found, you will be offered a chance to create simple predefined rules.

 

 

The few sample rules offered are simple rules that rely upon basic metrics. We will come back to these rules in a short while.  For now, wizards need to be completed.

 

Part Three - First time in Control Panel

After wizards complete, you are presented with a dashboard screen.  It likely contains empty historical charts since no data has been collected so far.  Navigation Explorer on the left shows various parameters that can be customized, while Instructions tab on the right shows context-sensitive instructions.

 

 

It is a good idea to visit the the Rules section to see the rules that have been defined by the wizard.  Two sample rules should be present and can be edited by double-clicking on each.  Rule Edit screen is simple yet powerful.  You can specify what formula needs to be evaluated, what happens when the evaluation returns TRUE, and what time of day should evaluation be restricted to.  To make formula entry easier, a list of already defined aggregated metrics is provided.  Hovering over the formula box will display allowed operands.

 

 

One last place to visit before starting the monitoring process is the screen that contains safety limits for the number of instances that AzureWatch can scale up to or down to.  By clicking on the appropriate Role name in the navigation explorer, you will be presented with a chance to modify these boundaries.

 

 

This is it.  If you are ready, press "Publish Changes" button.  Provided your AzureWatch Monitor service is running, it will pick up these configuration settings in the next iteration of its internal loop and instruct Azure Diagnostics Manager to start capturing the metrics required for formulas to work.  Windows Azure will need a few minutes to instruct your instances to start capturing those metrics afterwards, and then a few minutes more before these metrics will be transferred to your storage.  Thus, give AzureWatch at least 5-10 minutes before expecting to see anything on the Dashboard screen.

 

Part Four - A few tips and tricks

Some things to keep in mind while using AzureWatch

If you just started using AzureWatch and have not accumulated enough metric data, evaluation of your rules may be suspect as your aggregations will lack sufficient data.  It maybe prudent to disable scaling inside Rules in the beginning so that your scaling actions do not trigger unexpectedly.

Metric transfer occurs only when Monitor is running.  If you stopped Monitor service for an hour and then restarted it, it does not "go back" and send the missing hour's worth of metrics to AzureWatch.

When deploying new packages into Azure, you must use do it thru the same storage account that is specified in your DiagnosticsConnectionString and in AzureWatch System Settings.  Failure to do this, will result in incomplete or incorrect metrics.

AzureWatch will always instruct your instances to capture metrics that are defined in the Raw Metrics screen.  You do not need to do anything special with existing or newly started instances.  It may be worthwhile, however, to visit the System Settings screen to further configure how metric enforcement and gathering works.

AzureWatch will send a notification when it scales your instances up or down.  In it, it will provide values for all the aggregated metrics it knows about to help you understand why the scaling event occurred.

Locally installed AzureWatch components automatically self-update whenever a new version is released.



Dynamic scaling has arrived for Windows Azure

clock November 1, 2010 09:00 by author Igor Papirov

Paraleap Technologies, a Chicago-based emerging provider of cloud computing tools and services, is introducing technology preview of its flagship product, AzureWatch.

AzureWatch works with the Microsoft cloud platform and adds dynamic scaling capabilities to applications running under Windows Azure.  With AzureWatch, IT no longer has to worry about wasting money by over-provisioning resources, or experience slowdowns because Azure servers are overburdened.  This elastic scaling functionality that AzureWatch brings to the table is the key ingredient that brings out the power of Microsoft's cloud platform, allows for significant cost savings and guarantees consistent expectations to users. 

"We're absolutely thrilled to be coming out with AzureWatch on the heels of latest Azure-related announcements at Microsoft's PDC 2010", says Igor Papirov, founder of Paraleap Technologies.  "Windows Azure platform is getting better and more mature every day and our product completes the list of key features by providing automatic provisioning of compute resources to Windows Azure applications"

To learn more about AzureWatch or to participate in the free technology preview, log onto Paraleap Technologies’ web site at http://www.paraleap.com



Paraleap Technologies Joins Microsoft® BizSparkā„¢ program!

clock October 15, 2010 03:35 by author Igor Papirov

Paraleap Technologies is proud to announce that it became a partner in Microsoft® BizSpark™ program designed to accelerate the success of emerging startups by providing key resources such as software, support and visibility.

“We are very excited to participate in the BizSpark program,” says Igor Papirov, Founder of Paraleap Technologies. "Thanks to the program we now have better access to cutting edge software and services and can concentrate on building cutting edge products targeting Microsoft Windows Azure platform"


About Paraleap Technologies

Founded in 2010, Paraleap Technologies is an emerging Chicago-based startup, focused on providing tools and services for cloud computing technologies.
AzureWatch is Paraleap’s flagship product, designed to add dynamic scalability and monitoring to applications running in Microsoft Windows Azure cloud platform.



Release the power of the cloud

clock October 2, 2010 01:09 by author Igor Papirov

Cloud computing is more than just applications delivered over the Internet—creating those applications calls for a cloud-specific platform and cloud-specific tools.

Paraleap Technologies, an emerging provider of cloud computing tools, is preparing the beta release of its landmark AzureWatch program. AzureWatch, that works with the Microsoft Azure cloud platform to bring additional power and services to make the administration and development process more powerful, and the end-user cloud environment more reliable and robust.

“Cloud computing has always been a difficult thing to define,” said Igor Papirov, CEO of Paraleap Technologies. “It’s more than just applications that are located outside of your data center. There’s an infrastructure behind it, and a very unique development approach to creating those cloud-based applications.”

Cloud computing is traditionally defined as comprising three components: software as a service, infrastructure as a service, and platform as a service. Software as a service (SaaS) is the user-facing component of the cloud. It’s the applications that end users can access from their computers from any location. Infrastructure as a service is the delivery of compute resources such as servers and storage over the Internet. “But platform as a service is perhaps the least understood of all,” added Papirov. “Think of it as an operating system for the cloud, which provides a basic foundation for all of those cloud applications.”

Building on that foundation provided by Windows Azure, Paraleap Technologies’ AzureWatch adds dynamic scaling, so those applications automatically have all the resources they need, at any time. The elastic scaling functionality that AzureWatch brings to the table is the key ingredient that brings out the power of the cloud platform. With this feature, managers no longer have to worry about overprovisioning resources, or experiencing a slowdown because servers are overburdened. Access becomes more reliable.

To learn more about AzureWatch, log onto Paraleap Technologies’ web site at http://www.paraleap.com.



Paraleap Technologies Brings the Cloud Down to Earth

clock September 22, 2010 01:19 by author Igor Papirov

Paraleap Technologies, an emerging startup specializing in cloud computing technologies, empowers cloud developers with new tools. The company’s AzureWatch, a service designed to work in Microsoft Azure environments, adds a new dimension to the world of the Microsoft cloud.

Cloud computing is a strategy of making use of compute resources, including applications, infrastructure, and platforms, which are available from third-party providers’ data centers and delivered over a secure Internet connection.  In the early days of cloud computing, the advantage of cost savings was the “low hanging fruit” of the cloud, and businesses embraced it early on to take advantage of that bottom-line benefit. But the advantages go far beyond the cost factor, delivering remarkable ease of use, and access to resources from any location and any computer.

Because the cloud allows users to take advantage of a third-party resource pool, it immediately becomes possible to take advantage of more resources than would otherwise be available in a company’s own in-house data center. AzureWatch helps developers create applications to take full advantage of this potential, with the addition of instant and transparent scalability.

Older systems always required a defined set of resources to be dedicated to a task. The problem with that approach though, was that you never knew for certain whether that task would require more, or fewer resources at any given time. The inevitable result was over-provisioning—which led to wasted and underutilized resources and unnecessary expense. The cloud, as implemented with Microsoft Azure and enhanced with Paraleap Technologies’ AzureWatch, overcomes this wasteful approach by allowing for instant and automatic scalability of resources. If your Azure instances see a demand spike, your capacity will automatically increase—regardless of whether that spike was expected or a complete surprise. By the same token, if you experience a slowdown for any reason, the system will ramp down resource usage.

Applications that vary in usage demand by the month, week, or even by the hour are common. The automatic scaling functionality added by AzureWatch ensures that you always have the capacity you need, at any time.

Visit Paraleap Technologies online at http://www.paraleap.com for more information on AzureWatch.