Wednesday, April 18, 2012
Office 365 with Exchange on-premises Hybrid Deployment and Migration - A COMPLETE HOW-TO GUIDE
- Sign in to the Microsoft Online Services Portal with your Office 365 Administrator Credentials
- On the Admin page, under Management, click Domains
- Click Add a domain
- Type the name of the domain you would like to add (e.g. itguy.gr)
- Click Next
- Click Verify domain
- Add a TXT record on your public DNS system
TTL:3600
Destination Host: autodiscover.outlook.com
TTL: 3600
Preference: 10
Mail server: contoso-com.mail.eo.outlook.com
- Two Windows 2008 or 2008 R2 hosts joined to your domain. One will server as the primary server (the one you will install ADFS first) and the other as backup.
- One IP address that will serve as your NLB cluster IP Address
- One DNS A record (in your internal DNS servers) in the form: sts.yourdomain.com
- One trusted third-party SSL certificate
- Install the Network Load Balancing Feature on both servers and set up an NLB cluster. Make sure you define a port rule just for HTTPS (TCP port 443)
- Go to the primary ADFS server (i.e. the one you will install ADFS first), go to IIS Manager and create a certificate request. Tho points to pay attention to:
- As a certificate common name use: sts.yourdomain.com
- As a bit length use: 2048
- Now use the certificate request you created to obtain an SSL certificate from a third-party provider. (hint: if you're creating a lab or you just don't care too much about security give www.startssl.com a go... one year SSL certificate 100% free!!!)
- Install the certificate in the primary ADFS server's IIS manager. Also go to bindings, and select the newly-imported certificate for HTTPS
- Now extract the certificate with its private key (.pfx) and repeat step 4 on the secondary ADFS server's IIS
- Create an ADFS service account
- Create it as a user in Active Directory Users and Computers
- Make sure "User cannot change password" and "Password never expires" is selected
- Set the SPN of the service account as described here
- Install ADFS 2.0 on both servers
- Obtain installation package here
- On the Server Role page of the ADFS 2.0 setup wizard, make sure you select "Federation Server"
- Install the ADFS Update Rollup 1
- Install Microsoft Online Services Sign-in Assistant
- Install Microsoft Online Services Module for Windows PowerShell
- Configure First Federation Server in Federation Server Farm
- Add Secondary Federation Server to Federation Server Farm
- Start Windows PowerShell as an administrator. To do this, right-click the Windows PowerShell shortcut, and then click Run As Administrator.
- To configure Windows PowerShell for remoting (this will enable Remote PowerShell on the ADFS server. Skipping this step will result in an authentication error), type the following command, and then press ENTER: Enable-PSRemoting -force
- Open Microsoft Online Services Module for Windows PowerShell as an administrator.
- Type cd\ and press Enter.
- Type $cred=Get-Credential and press Enter.
- Enter your Office 365 administrator name and password and click OK.
- Type Connect-MsolService –Credential $cred and press Enter.
- Type Set-MsolAdfscontext -Computer <AD FS 2.0 primary server> and press Enter
- Type New-MsolFederatedDomain -DomainName yourdomain.com and press Enter
- Now type Get-MsolFederationProperty -DomainName yourdomain.com and press Enter (this should bring up a bunch of data regarding certificates and sign in URLs)
- Download and run the Microsoft Office 365 Deployment Readiness Tool
If the tool has found any discrepancies, you'll need to fix them before continuing.
Great! Now log on to Microsoft Online Services, and:
- Go to Admin
- Click on Management, then Users
- Click on "Single sign-on: set up"
- On step 6 "Activate Active Directory Synchronization" click on "Activate"
- On step 7, download the Directory Synchronization Tool.
- Choose a machine to install the Directory Synchronization Tool on. The constraints are:
- This machine must be a domain joined computer
- It must be a highly-trusted PC or server that only trusted administrators have physical or remote access to.
- It must not be a domain controller
- It must not be one the of the ADFS servers
- If you run the Directory Synchronization Tool straight away it will fail! You'll need to do this first:
- Click Start, right-click Computer, and then click Manage.
- Under Computer Management, expand Local Users and Groups, and then expand Groups.
- Make sure that the MIISAdmins group exists. If the group is missing, create a group that is named MIISAdmins.
- Add yourself to the group.
- Log off from the computer, and then log on to establish the new group membership in the access token.
- Now install the Directory Synchronization Tool. After the installation has finished the Microsoft Online Services Directory Synchronization Configuration Wizard starts:
- You will need to type your Microsoft Online Services Admin Credentials as well as an on-premises Enterprise Admin Credentials
- On the Exchange hybrid deployment, check the box and click on Finish to Synchronize Directories
- To verify Directory Synchronization follow the instructions here
- To force Directory Synchronization:
- On the computer that is running the Directory Synchronization tool, navigate to the directory synchronization installation folder. By default, it is located here: %programfiles%\Microsoft Online Directory Sync.
- Double-click DirSyncConfigShell.psc1 to open a Windows PowerShell window with the cmdlets loaded.
- In the Windows PowerShell window, type Start-OnlineCoexistenceSync, and then press ENTER.
Configuring the ADFS Proxy
So, now that the integration with the on-premises Active Directory and Office 365 is complete, it's time to configure the ADFS Proxy. What Microsoft recommends, and you should take this recommendation seriously, is that you utilize 2 load-balanced proxy servers. For the purposes of this article we will use only one ADFS Proxy server, but all you need to do to get your load-balanced pair is essentially to configure a second ADFS Proxy and set up NLB on both as described previously in this article.
What you'll need for this part:
- One Windows 2008 or 2008 R2 server
- Access to modify your external DNS records
- The certificate you used on your internal ADFS servers
- A working DMZ Topology
- Appropriate rules on your corporate firewall
- First of all, configure your Windows server, position it on the DMZ and make sure there is connectivity both to the Internet and your internal network
- Install the IIS Role with the default options, import the certificate you installed to the internal ADFS servers, and configure the bindings for port 443
- Add the respective Host Name and IP address of the ADFS NLB Cluster you created earlier to the ADFS Proxy server's Hosts file
- Create a new A record on your public DNS server with the name sts.yourdomain.com (exactly like the one you created on you internal DNS servers) and point it to the public IP address you will use for your ADFS Proxy
- Configure your firewall so that HTTPS requests on this IP will be forwarded to your ADFS Proxy
- Make sure you can:
- ping the load-balanced internal ADFS server farm (sts.yourdomain.com)
- browse to this link: https://sts.yourdomain.com/federationmetadata/2007-06/federationmetadata.xml
- On IE 9, hit the 'Compatibility View' button, then you should be able to browse the XML contents
- Now run the ADFS 2.0 setup and select Federation Server Proxy
- Once completed, make sure the "Start the ADFS 2.0 Federation Server Proxy Configuration Wizard when this wizard closes" check box is selected and click finish.
- On the "Specify Federation Service Name" type: sts.yourdomain.com, i.e. the exact same name of the internal load-balanced ADFS server farm we configured earlier
- The "Use an HTTP proxy server when sending requests to this federation service" IS STUPID!!! Why on earth would we need yet another proxy, to proxy our proxy's requests!!!! Make sure this check box remains unselected and hit "Test Connection". If network connectivity is working you will get a "The specified Federation Service was contacted successfully" message.
- Hit Next, and you're prompted for the logon credentials for the internal ADFS server farm. Remember the ADFS service account you created? That's the one. You're supposed to enter the credentials in the form
yourdomain\adfsaccountpassword
But first go to Server Manager's Main Page, click on "Configure IE ESC" and turn it off for both users and administrators. Don't, and you'll get "unable to establish a trust between the federation server proxy and the federation service" errors - On "Ready to Apply Settings, click on Next and watch the magic happen!
- Open Event Viewer
- In the details pane, double-click Applications and Services Logs, expand AD FS 2.0, then click on Admin
- On the Event ID column, look for event ID 198
- Log on to a computer that sits outside your corporate network (so that it will be directed to your ADFS Proxy server
- In an Internet Explorer window, type https://portal.microsoftonline.com
- In the User ID field, type your corporate username (e.g. user@yourdomain.com)
- The web page should update with the "You are now required to sign in at yourdomain.com" message
- Click the sign-in link and in the sign-in page and enter your domain credentials
- If this takes you to the Microsoft Online Services Portal home screen, then you can pat yourself to the back; the ADFS system you just configured works
- If not, re-check what we've done so far and that you haven't skipped a step. You can also check Adam Conkle's great post on this TechNet thread: http://social.msdn.microsoft.com/Forums/en-US/Geneva/thread/bd815b4e-70be-4f1c-ac7f-a5e9ac5aea02?prof=required
Great, all done then?? Not quite! It's the on-premises Exchange 2010 server's turn!
- On your Hybrid Exchange server install Microsoft Online Services Sign-in Assistant (the one you installed on your ADFS Farm servers) AND Microsoft Online Services Module for Windows PowerShell
- Open the Exchange Management Console and click the Microsoft Exchange node (the top-most node in the tree)
- In the action pane, click Add Exchange Forest.
- In the friendly name type "Office 365". This name will display in the console tree.
- In the FQDN field just select "Exchange Online" from the drop-down list
- DO NOT check the "Logon with default credential" box
- Click OK and a credentials dialog box appears. Type the Office365 Admin account and corresponding password. For example:admin@yourdomain.onmicrosoft.compassword
- Watch in awe as your Office 365 organization appears on the EMC console!!
- On EMC, go to Organization Configuration and on the Actions pane, click on New Federation Trust
- Click on New and then on finish and you're done
- Right click under Organization Configuration and choose New Hybrid Configuration
- Go through the 2 steps of the wizard and click on finishThis was too easy, now comes the tricky part
- Right-click on the Hybrid Configuration and choose Manage. The Manage Hybrid Configuration opens
- On the credentials screen, type in the appropriate usernames and passwords. What I suggest you do is (since we've already synchronized Active Directory) to enable an Active Directory user that is member of the on-premises Organization Management role group, as a Global Administrator in Office 365. This way, you can use the same account for both fields
- Under Domains, choose your hybrid domain
- On the next step you will be asked to create a TXT record on your public DNS servers with a Record Value that is a lengthy string of alphanumeric characters. Check the box to confirm that the TXT record has been created
- Next up, choose your hybrid on-premises CAS and HUB servers
- Under Mail Flow Settings type the Public IP address of your on-premises Exchange server (i.e. the one on the public MX record), then type the FQDN of the HUB server you chose in the previous step
- Mail Flow Security is pretty much straightforward, the third-party certificate is already there, and for the Mail Flow Path choose the first option (as discussed in the beginning of this article).
- Click next but don't go any further just yet! If you continue you will most likely get an error and the wizard will fail with something like:
Execution of the Get-FederationInformation cmdlet had thrown an exception. Federation information could not be received from the external organization.
OR
Exception has been thrown by the target of an invocation
What on earth is wrong? Well, either the MRSProxy is disabled, or WSSecurity is disabled, or the MRSProxy is unreachable from the Internet.
Regardless of where exactly the fault lies, do all of the following:
- On your Hybrid Server's Exchange PowerShell, enter this command:Set-AutodiscoverVirtualDirectory -Identity ‘autodiscover (Default Web Site)’ –WSSecurityAuthentication $trueto enable WSSecurity authentication for autodiscover on your hybrid server, and...
- On your Hybrid Server's Exchange PowerShell, enter this command:Set-WebServicesVirtualDirectory -Identity "EWS (Default Web Site)" -MRSProxyEnabled $true -MRSProxyMaxConnections 100The rule of thumb is that the larger number you use for the MRSProxyMaxConnections parameter, the less chances there are that a timeout will occur for large simultaneous mailbox moves.
- Create a new publishing rule for Autodiscover in your TMG server.To get your hybrid scenario working, you will need to set up an additional ISA/TMG rule pointing to the hybrid server, using the proper public names (don’t forget autodiscover), bound to the OWA listener. You need to allow All Users (as opposed to Authenticated Users), set authentication to “No Authentication, but users can authenticate directly” and configure the following paths:
- /EWS/mrsproxy.svc/WSSecurity
- /EWS/Exchange.asmx/WSSecurity
- /Autodiscover/Autodiscover.svc
- /Autodiscover/Autodiscover.svc/WSSecurity
After configuring the rule, you need to put it above all the other Exchange rules, making it the first matching rule when federation traffic hits ISA/TMG.
Now you're ready to go through with the wizard and bask in the glory of your success!!!
We're almost done. What have we done so far?
- We've configured our SSO domain with Office 365
- set up public DNS records
- installed an internal ADFS server farm
- enabled Single Sign-On
- configured Active Directory Synchronization
- installed our ADFS proxy on the DMZ
- verified the ADFS bit is all interoperable
- configured our on-premises Exchange 2010 hybrid server for Office 365
- On the EMC console, go to the on-premises node --> Recipient Configuration --> Mailbox
- Right-click the mailbox you want to move and choose New Remote Move Request
- On the Connection Configurations windows, under "FQDN of the Microsoft Exchange Mailbox Replication service proxy" type the Internet FQDN of your hybrid server. Definitely consider typing in a domain admin credentials for this test.
- Under Move Settings, for the Target Delivery Domain browse for "yourdomain.mail.onmicrosoft.com"
- If you followed this article to the letter, the move should go without a hitch.
For good measure, test moving that mailbox back to the on-premises server:
- On the EMC console, go to the Office 365 node --> Recipient Configuration --> Mailbox
- Observe that the mailbox you just moved appears with a green arrow. This means that a move request exists for this mailbox (albeit a completed one). You will need to go to "Move Request" underneath select the move request and click on "Clear Move Request". This will enable you to move the mailbox back to the on-premises organization.
- Now right-click the test mailbox and click New Remote Move Request
- On the Connection Configurations windows, under "FQDN of the Microsoft Exchange Mailbox Replication service proxy" type the Internet FQDN of your hybrid server. Definitely type in a domain admin credentials for this test, then consider using delegated permissions
- Under Move Settings, for the Target Delivery Domain browse for "yourdomain.com". For Remote Target Database, type the on-premises database you want to move the mailbox to. Contrary to what other guides might say, just type in the database name without the server name.
- This should be enough to get the test mailbox moving back to your on-premises exchange organization.
- From Office 365 users to Internet (external) recipients
- From Office 365 users to on-premises recipients
- From on-premises users to Office 365 recipients
- From Internet (external) email users to Office 365 recipients
- On the TMG Management Console, to to Troubleshooting and on the Tasks pane click on "Control E-mail configuration Integration"
- Set the status to "Disabled" and enforce the policy.
- Go to Forefront Online Protection for Exchange Administration center and make a note of the IP addresses listed under "IP addresses to configure on your firewall".
- Login to https://portal.microsoftonline.com/ with a Global Administrator account
- Click on Outlook
- On the upper right-hand side of the Outlook page click on Options and then "See All Options"
- On the upper left-hand side click on the drop-down list arrow right from "Manage Myself" and under "Select What to Manage" click on "My Organization"
- Go to Mail Control and on the right-hand side of the page, under Forefront Online Protection for Exchange click on "Configure IP safelisting, perimeter message tracing and email policies"
- This takes you to Forefront Online Protection for Exchange for your organization. Under Information, click on Configuration.
- Copy-paste the IP addresses and subnet ranges to a text document, then arrange them in a comma-separated way, like so:12.12.12.0/24, 14.14.14.0/26, 34.23.34.156
- Now open the Exchange PowerShell on the TMG/Exchange Edge server and type the following command:New-ReceiveConnector -Name "From Cloud" -Usage Internet -RemoteIPRanges <FOPE Outbound IP Addresses> -Bindings 0.0.0.0:25 -FQDN mailserverexternalFQDN.yourdomain.com -TlsDomainCapabilities mail.messaging.microsoft.com:AcceptOorgProtocolwhere <FOPE Outbound IP Addresses>, copy-paste the comma-separated list from the text document
That all! This is all everything you need to do to get your hybrid Office 365 working.
Yours,
the ITGuy.
Subscribe to:
Post Comments (Atom)
About Me
Blog Archive
Categories
Musings
(3)
Virtualization
(3)
2012
(2)
SCVMM
(2)
SQL
(2)
2010
(1)
Active Directory
(1)
Cloud
(1)
DMZ
(1)
Errors
(1)
Hyper-V
(1)
Scripts
(1)
TMG
(1)
Threat Management Gateway
(1)
Powered by Blogger.
Popular Posts
- Office 365 with Exchange on-premises Hybrid Deployment and Migration - A COMPLETE HOW-TO GUIDE
- Workaround for Citrix EdgeSight Process Usage report showing average times instead of time sums and a quick insight into using MS SQL Report Builder with queries to generate custom SQL reports
- Using TMG 2010 with Hyper-V to support multiple Virtual DMZ hosts
- Annoying "You cannot access VMM management server scvmm.domain.local" 1604 error: "Contact the Virtual Machine Manager administrator to verify that your account is a member of a valid user role and then try the operation again."
- SCVMM 2012 locale issues and Error 24374 when adding new Library Share with default resources
- How to use msg.exe to send popup messages to multiple PCs / Workstations
- "The trust relationship between this computer and the primary domain failed" messing about with SCVMM 2012 RC
- With a mind on the cloud, eyes on virtualization and feet planted firmly on the (datacenter) ground...
- I'm an IT Manager... nobody loves me!
- Musings of an IT guy
13 comments:
Excellent post, I actually found it once I was done with my deployment and re checked several things.
Have you had issues with single sign-on? such as "Your organization could not sign you in to this service"
After numerous installations, no, SSO has worked without a hitch!
Are you using an ADFS Proxy? If so, have you tried my troubleshooting steps under
"Configuring the ADFS Proxy". It sounds like authentication issues
Very interesting tutorial. But I've got a problem. I create the rule on my TMG and I check it. And I've got an error to "/EWS/mrsproxy.svc/WSSecurity" it can't be verified. But for the others paths, no problem.
one question !! on the hybrid emc the tarnsport hub
fields are grayed out with a yellow lock.is this by design or i have failed to install the hybrid correctly.
If you mean for example, in Office365 --> Organization Configuration --> Hub Transport --> Transport Settings General Properties, fields like 'Maximum Receive Size' etc.
...then yes, it's by design. Some of the grayed-out options are configurable from the Office365 administrative portal, and others are not meant to be tampered with.
thank you for your answer , one last question the only way to change maximum receive size is by using power shell commands (set-recieveconnector). i could not find any other way.from office 365 is there any other option?
Hi, Tanks for the awesome article.
When clients with mailboxes in the cloud connect via Outlook Anywhere, is the data transferred via the on premise server internet connection or directly via the Microsoft datacentre?
Thanks,
Thomas
Hey,
Awesome post. Just loved it.
However have few questions for which help is much appreciated.
1. How do i make ADFS proxy requests forwarded to ADFS server? Should i do a port forwarding kind of thing for my ADFS Proxy?
2. I have already given firewall excemptions for Exchange Online IP address ranges. However, my ADFS confign was failing at Connect-Msolservice with the message "there was no endpoint listening at https://proviosioningapi.microsoftonline.com/..svc." Later the only way i could complete the activity was by having direct internet connectivity for my ADFS server. What could be the possible reason? Also should i retain this direct internet connnectivity forever/is there any work around?
Hello all, first of all apologies for the late replies, I've just been swamped with work over the past few months.
So here goes:
Regarging the 06/11/2012 Anonymous user comment:
OA works over HTTPS directly to the Microsoft datacenter for cloud mailboxes. This is true in all routing scenarios/topologies including the Hybrid Model we're discussing here.
What changes is the email flow which in the case of the Hybrid model, all email has to go through the on-premises server.
Regarding the 21/11/2012 coment:
1. Simple port forwarding to the 443 tcp port of the ADFS proxy IP will do. If you've followed the full ADFS topology/configuration. Ideally you shoulf use Microsoft TMG to publish the public IP address of your ADFS proxy
2. There are a lot of suggestions that come to mind... ADFS works over HTTPS (tcp port 443). It requires its own public IP address for which, at the very least, you should have a port forwarding rule on your firewall. Direct connectivity should be avoided at all costs, I cannot overemphasise this!
If you like you can give me your email address and look over your issue in more depth
hi there. i keep getting the error upon logging into the sts site, any insight?
There was a problem accessing the site. Try to browse to the site again.
If the problem persists, contact the administrator of this site and provide the reference number to identify the problem.
Reference number: 56e84ec1-a00e-459f-b3c0-65b691dd6a03
Does this happen internally, or on the ADFS Proxy? Are you using an LNB cluster for the internal ADFS server? Are you able to telnet to port 443?
Thanks for great post!!
I am getting the following error on Hybrid wizard.
>>Execution of the Get-FederationInformation cmdlet had thrown an exception. Federation information could not be received from the external organization.<<
I dont TMG/ISA in my network, I just have fortigate firewall and port 25,443 is allowed from external network.
My question is how can i add web publishing rule on fortigate firewall? is it require as I don't have ISA/TMG?
Post a Comment