Sunday, January 26, 2014

Automating IP addresses with TMG

In the past few months I've had the "opportunity" (note the quotes, maybe I'll write a post about all the troubles I've run into) to deal with some Office 365 integration projects.

When integrating one or more cloud services with Office 365 or Office 365 with on-prem Exchange, it is typical to limit the IP addresses that are aloud to communicate between services. This is an important layer on the security model and obviously reduces the attack surface. Microsoft maintains lists of IP addresses for the various services here:

That page links to other pages with more specific information about various other Microsoft cloud services such as the link below for Exchange online. for example.

At the time of this writing, you'll notice that the lists are inconsistent between pages. >:-(

Microsoft on two separate Premier Support Calls has told me that those lists are not kept up to date. As a result of this, I've experienced Journaling black-holes and Hybrid-configuration black-holes. In other words: I have discovered new IP addresses from Microsoft datacenters that were not published on their lists and so connections were being denied. In the case of the Journaling black-hole, Journaled messages were lost. This is bad, especially for litigation-hold situations. But I digress...

(NB: I have been told that there is an RSS feed that contains the various IP address lists which could be consumed via PowerShell or other automated task, but I've not been able to find it)

Anyway - all of this whining is tangential to the actual point of this post. The point of this post is that in one configuration I was working on recently, we were using a Threat Management Gateway (which was a cancelled product and replaced with UAG, which is a cancelled product and presumably to be partially replaced with WAP (? - conflicting information from Microsoft's team on this too)).

To configure a TMG rule to only accept traffic from specific hosts or subnets, those subnets must be entered as an Subnet Object (or some other type of range object supported by TMG). This isn't so hard right? Wrong. The user interface for TMG, when trying to bulk-add addresses, is terrible.  I imagine anyone who is reading this blog post already has some experience so I won't complain about it too much...

Here is how I handled the situation - some of it was more manual than I wanted - but at the time I was in a time crunch and haven't refined my process. That's mostly because I don't have a TMG in my lab. Anyway, here are steps I'd taken:

  1. Collect IP addresses from the web site referenced above - in my case, I put them in a Notepad++ window
  2. Convert the IP addresses to this format:
    "Unique Name","IP Address", "Subnet Mask"
    for example one address is "", convert that to
    Office IP Subnet 1,,
    I used regex's to perform this operation.
  3. Save that file as a CSV (make sure there are headers - I saved mine as 'SubnetList.csv')
Step 4 is a little more complicated - so here's what you can do from an elevated PowerShell:

$tmg = new-object -comobject "FPC.Root" -strict
$tmgarray  = $tmg.arrays | select-object -first 1
import-csv .\SubnetList.csv | % { $tmgarray.ruleelements.subnets.add($_.Name,$_.Network,$_.Subnet)}

Step 5
This will add all of the subnets as Objects within the TMG and then they can be easily applied to the rule. You could also add the subnets programatically to the rule(s) but I haven't tested it. I'm not sure if I'd recommend that outside of a lab environment first though since you don't get to easily and visually review the configuration when you do it that way. 

So that's it... while this post seems long, I assure you that this mechanism is much, much faster than performing this operation manually.

Good luck!