Azure ExpressRoute Primer

What is Azure ExpressRoute?
ExpressRoute is an Microsoft Azure service that lets you create private connections between Microsoft datacenters and infrastructure that’s on your premises or in a colocation facility. ExpressRoute connections do not go over the public Internet, and offer higher security, reliability and speeds with lower latencies than typical connections over the Internet.

How to setup ExpressRoute Circuit?
ExpressRoute circuits are resources within Azure subscriptions. But before you setup Expressroute connection (or circuit as it’s normally referred as), you need to make decisions about setup parameters.
1) Connectivity Option – You can establish connection with Azure cloud either by extending your MPLS VPN (WAN), or you can leverage your Colocation provider and it’s cloud exchange or roll out an point-to-point ethernet your self. Most large enterprises would use the first option, medium size enterprise running in COLO would go with second option, and the last is more specialized scenario warranting higher level security
2) Port Speed – Bandwidth for your circuit
3) Service tier / SKU – standard or premium (more on this later)
4) Location – You might get multiple options for this depending on your choice for connectivity (#1). E.g. MPLS providers have multiple peering locations from which you can pick the one closet to you
5) Data Plan – Limited plan with pay as you go egress charges or unlimited plan with higher cost irrespective of egress volume

After you make these five choices, you can fire up PowerShell on your VM to execute ‘New-AzureDedicatedCircuit’. Remember to select the right Azure subscription (Add-AzureAccount / Select-AzureSubscription) where you want the circuit to be created. Please note you would need to import ExpressRoute module if not already (Import-Module ‘C:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\ExpressRoute\ExpressRoute.psd1’

New-AzureDedicatedCircuit -CircuitName $CircuitName -ServiceProviderName $ServiceProvider -Bandwidth $Bandwidth -Location $Location -sku Standard

As soon as this completes you will get a service key which is kind of a unique identifier for your circuit. At this step only your billing starts, as we have only completed Azure side of things. Now you need to work with your Network Service Provider, provide your service key and ask them to complete their side of configuration to Azure. This would also involve setting up a BGP session at your end. Once this done you are all set to leverage expressroute and connect the circuit to azure virtual networks – with the traffic flowing over private connection.

Connecting ExpressRoute Circuit to Azure Virtual Network
Once the circuit is configured it’s relatively straight forward to connect it to virtual network. Once again PowerShell is your friend. But before firing the below command ensure your VNET and the virtual gateway is created.

New-AzureDedicatedCircuitLink -ServiceKey “***” -VNetName “MyVNet”

ServiceKey parameter uniquely identifies your circuit. As circuits are part of the Azure Subscription (wish there was a way to view them in portal) your VNET should be part of the same subscription. This lead to the question – Can we connect expressroute circuits to VNETs across subscriptions? Answer is yes.

Connecting ExpressRoute Circuit to Azure Virtual Network across subscriptions
As we know circuit is part of a subscription, so as a subscription admin you will have to grant rights to other subscription admins so that they can link their VNETs to your circuit. Here’s the PowerShell cmdlet to do that.

New-AzureDedicatedCircuitLinkAuthorization -ServiceKey “***” -Description “AnotherProdSub” -Limit 2 -MicrosoftIds ‘devtest@contoso.com’

This commands allows 2 VNETs from AnotherProdSub to connect to the ExpressRoute circuit. You might see the last parameter MicrosoftId replaced by AzureAD Id (not sure what IDs are supported right now)

Once you have the authorization, you can query the servicekey from your subscription and link your VNET as appropriate.

Get-AzureAuthorizedDedicatedCircuit #This will get details of the circuit including ServiceKey

New-AzureDedicatedCircuitLink –servicekey “***” –VnetName ‘APSVNET’ #Link VNET in another subscription

Remember you can only connect 10 VNETs per circuit. Though this is a soft limit but you can grow only few folds. If you need to create 100 VNET instance you need to look at ExpressRoute Premium.

What is ExpressRoute Premium?
Premium tier for enterprises that need more VNETs per circuit, need their circuit to span geo-political region or have more than 4000 route prefixes. You will pay around 3000 USD more for the premium features, when compared to standard edition with same bandwidth.

How much it costs?
Express route costs boil down to price you pay to Microsoft and your service provider.
To Microsoft it’s
monthly fee depending on the port speed
Bandwidth consumed (unless you are in unlimited data where you flat 300 USD)
Virtual Gateway which you would provision in your VNET (mandatory for expressroute & S2S VPN)

To Network Service Provider:
It’s one time setup fee for the circuit
Bandwidth charges (how much data goes through their cloud to Microsoft)

How long does it take to setup connection?
Well it depends. If you already have a network service provider or an exchange provider supporting Azure, it shouldn’t take more than a day (excluding paperwork). Otherwise this can turn out to be a project in itself.

Can we use ExpressRoute to connect to office 365?
Answer is yes, but it actually depends on your provider. Apart from connecting to Azure VNET, expressroute allows you to establish public peering and Microsoft peering to route your Azure PaaS (public services) and Office 365 traffic over the private network. For more details refer to this link. Public peering allows you to route your traffic to public services like Azure Services and Azure SQL database over private tunnel.

Advertisements

Using a Single Windows Azure Active Directory tenant for All EA Azure Subscriptions

As you know by now Windows Azure Active Directory is at the root of every Azure subscription.

 Image

But in an EA setup you typically have multiple subscriptions and you definitely don’t want to create a different WAAD tenant for every other subscription. So here’s what you can do (there might be other ways too of achieving this). You can first create a Shared account and under that a Shared Subscription. Also create the WAAD tenant you want to use and ensure your shared subscription is under that WAAD tenant. In that WAAD tenant create all the account administrators.

Image

Now go to your EA portal, and add new accounts specifying the account administrators you just created. That’s it – next when you create subscriptions for those newly created accounts, these subscriptions will be by default part of the same WAAD tenant under which you created your shared subscription.

Image

It can’t get any easier, isn’t it 🙂 ?

Windows Azure Portals and Access Levels

When you sign up for Windows Azure you get a subscription and you are made the Service administrator of that subscription.

Image

While this creates a simple access model, things do get little complicated in an Enterprise where users need various levels of access. This blog post would help you understand these access levels. 

Enterprise Administrator
Enterprise Administrator has the ability to add or associate Accounts to the Enrollment and can view usage data across all Accounts. There is no limit to the number of Enterprise Administrators on an Enrollment.
Typical Audience: CIO, CTO, IT Director
URL to GO: https://ea.windowsazure.com

Account Owner
Account Owner can add Subscriptions for their Account, update the Service Administrator and Co-Administrator for an individual Subscription, and can view usage data for their Account. By default all subscriptions are named as ‘Enterprise’ on creation. You can edit the name post creation in the account portal. Under EA usage, only Account Administrators can sign up for Preview features. Recommendation for accounts to be created is either on functional, business or geographic divisions, though creating a hierarchy of accounts would help larger organizations.
Typical Audience: Business Heads, IT Divisional Heads
URL to GO: https://account.windowsazure.com

Service Administrator
Service Administrator and up to nine Co-Administrators per Subscription have the ability to access and manage Subscriptions and development projects within the Azure Management Portal. The Service Administrator does not have access to the Enterprise Portal unless they also have one of the other two roles. It’s recommended to create separate subscriptions for Development and Production, with production having strict restricted access.
Typical Audience: Project Manager, IT Operations
URL to GO: https://manage.windowsazure.com

Co-Administrators
Subscription co-administrators can perform all tasks that the service administrator for the subscription can perform. A co-administrator cannot remove the service administrator from a subscription. The service administrator and co-administrators for a subscription can add or remove co-administrators from the subscription.
Typical Audience: Test Manager, Technical Architect, Build Manager
URL to GO: https://manage.windowsazure.com

That’s it! With above know-how you can create an EA Setup like below

Image

Hope this helps 🙂

Recovering from Windows Azure Virtual Machines failures – Unable to establish RDP connection

Your ability to recover from failures is quintessential while working with any cloud platform and same applies to Windows Azure Virtual Machines. Being in preview mode you might experience some glitches around this offering. For instance, there were times when I would setup a VM and next day I wasn’t able to RDP (Remote Desktop) into it. It looked like my hard work of setting up the VM would go down the drain. But those initial struggles filled my knowledge gaps. While the stability of VMs is dramatically improvising as we approach the General Availability, below are few pointers to deal with such failures. Hope you find them useful. 

a. Restart VM: First is to restart your VM. That should do the trick whenever your VM becomes unresponsive.

b. Delete VM: In case restart doesn’t help or restart operation is failing you can try deleting and re-creating VM. If you are attempting a manual delete via Azure Portal, you should delete the underlying Cloud Service too. Cloud Service – is the container holding your VM and would become visible when you delete the VM (will detail this in an upcoming post). Deleting your Cloud Service would allow you to reuse your DNS name (as a best practice ensure that your VM and DNS names are prefixed with unique identifiers). I have seen few developers struggle and provide different names to their VMs every time they delete it, failing to realize that they need to delete the Cloud Service too. Below Figure shows how you can create a new VM by selecting an existing disk (New -> Compute -> Virtual Machine -> From Gallery)

Azure VM Disks

c. Resize VM: An alternative to deleting your VM is to change the size of your VM. This is less intrusive but might not be feasible in all the scenarios.

d. Unlock VHD / Disk: At times, after you delete your VM, the portal might show that Disk is still attached to the deleted VM. This would result into a quandary because you can’t reuse the Disk to create a new VM until it’s detached. The easier option to resolve this is to delete the disk object. Deleting the Disk object would still retain the underlying blob. After deletion you can re-create the disk object and use it to spawn a new VM. If you can’t delete the disk then your only option is to break the blob lease manually via PowerShell or using Storage Client Library (refer to this MSDN forum link for further information).

Please do leave a comment, if you have used other approaches.

Controlling Windows Azure VM lifetimes

Most of the Cloud computing resources are billable on an hourly basis and it’s important that you release these resources when you no longer need them. This typically applies to Windows Azure Virtual machines not running 24×7, example – there could be business workloads which requires an application to be available only twelve hours on week days. To limit running costs most users stop their VMs, just to realize that Windows Azure bills for VMs that are in a stopped state. So, your only option to control costs is to delete the VMs. But wouldn’t deleting VM cause any issues?

The answer is both No and Yes. When you delete a VM you are just deleting VM instance but the underlying OS and data disks are still intact (in fact, you still keeping paying for their storage which luckily is quite negligible). Hence, you can easily resurrect your VM without much harm. It’s important to note that when you delete the VM, you still retain the underlying Cloud Service container and its associated Site URL. The issue you might face though when you delete and re-create the VM, is the public IP address change. I work in an organization with strict IT security rules and locked down access. Static IP was necessary for me to raise an outbound RDP access request with my IT team. But with IP changing everyday it was definitely turning into a challenge. In end the solution I adopted was to create an extra small VM running 24×7 and bounce from there to other VMs.

To delete a VM you can use PowerShell cmdlets. PowerShell cmdlets allow you to export your VM configuration, delete the VM and then re-create VM using exported configuration.

Export-AzureVM -ServiceName ” -Name ” -Path ‘c:\vmconf.xml’

Remove-AzureVM -ServiceName ” -Name ”

Import-AzureVM -Path ‘c:\vmconf.xml’ | New-AzureVM -ServiceName ” -VNetName ‘’ -DnsSettings ‘’

Export configuration as shown above is stored in a XML file. It contains various properties of a VM including Endpoints, disks, VM size, subnet, etc. Below snapshot from Azure Portal shows an empty Cloud Service Container post deletion of the VM. Currently there is no cost associated with an empty cloud service. It’s important to note that when you retain Cloud Service you also retain the underlying DNS URL.

Passing Parameters to Hadoop Streaming

This weekend I would be presenting @ BDOTNET UG meet on topic – “Big Talk: Hadoop on Azure”. In case you are around and plan to attend here’s the facebook event link. In this talk I would help you get started with Hadoop and show how you can leverage it with Windows Azure. With that, let’s focus on the subject of this blog post.

Hadoop Streaming allows you to write and run MapReduce jobs in language of your choice. For Azure and Microsoft world this would be mostly C#. You can create your programs / executable in C#, read input from Console and write output to Console. Mapper task would feed input lines to your executable via console (standard input) and also collect output via console (standard output). It converts output into key value pairs. Reducer task on other hand converts key value pairs into input lines, feeds it to your executable (via console), and collects the output (via console) converting it back to key value pairs. For scenarios where you need only mapper you can emit reducer and set ‘numReduceTasks’ to zero as shown below:

call hadoop.cmd jar hadoop-streaming.jar -files "hdfs://10.186.36.85:9000/example/apps/Mapper.exe" -mapper "Mapper.exe" -input "asv://account/inputdata/account.data" -output "/example/data/StreamingOutput/mywc" -numReduceTasks=0

IP address in above case is that of Namenode (you can get by executing following command from Javascript console – #cat file:///apps/dist/conf/core-site.xml).

Now at times, your mapper program would need additional parameters to carry out its operations e.g. say you want mapper to filter data on few attributes. So, how do can we pass these attributes to Mapper executable? Simple – pass them as command line parameters and in your program read them from args.

call hadoop.cmd jar hadoop-streaming.jar -files "hdfs://10.186.36.85:9000/example/apps/Mapper.exe" -mapper "Mapper.exe param1 param2 param3" -input "asv://account/inputdata/ account.data" -output "/example/data/StreamingOutput/mywc" -numReduceTasks=0

static void Main(string[] args)
{
string line;

string parameterOne = args[0];
string parameterTwo = args[1];
string parameterThree = args[2];

Hope this helps!