Tony Redmond: Understanding how Exchange Online runs

Understanding how Office 365 operates is always an interesting challenge because Microsoft usually doesn’t say too much about how things work within the service. But the Exchange Online folks are pretty good at telling what they are up to, which brings us to the “Behind the Curtain: How we run Exchange Online” session at Microsoft Ignite in Chicago, featuring the talents of Vivek Sharma (Director of Office 365 product management) and Perry Clarke (VP of Exchange development). This provided an update to a similar session given at MEC in 2014 and is available on Microsoft’s Channel 9 service.

Both speakers are interesting people in their own right. In the past, Vivek had a lot to do with the implementation of PowerShell in Exchange 2007 and since then has focused on bringing Exchange Online through from the initial beginnings to BPOS to where it is today. Perry is one of Microsoft’s deep thinkers. A conversation with him is likely to explore what we’ll all be doing in five years’ time and it’s obvious that he has some pretty solid ideas on that point. He wrote the foreword for our just-published “Office 365 for Exchange Professionals” eBook and said some nice things about us, much to the amusement of some of the members of the Exchange development group.

The session began with some comments from Perry about how cloud services are changing the way people think about technology. Typically, companies look at three factors to assess a technology. Cost, risk, and user experience (or functionality). Perry maintains that the cost of an Exchange Online mailbox is at a point that no on-premises implementation can match, if costs are accurately calculated and everything is included. Part of this is because Microsoft has massive buying power for datacenters, storage, servers, and network to achieve price points that even the largest on-premises customer can only dream about.

A solid SLA track record (the most recent result was 99.99% for the first quarter of 2015) means that the perceived risk of companies putting their most important work on Office 365 is much less than it was four years ago when Microsoft launched the service. Finally, the functionality that can be delivered by a cloud service is so much ahead of what is possible for on-premises deployments because of the direct involvement of the engineering group (and some functionality, like Delve and Clutter is only available in the cloud). In a nutshell, Perry advanced a case that cloud services is the only way to achieve the desired combination of cost, risk, and functionality for a technology like email.

Returning back to the size and scale of Exchange Online, some data was offered to illustrate what Microsoft manages. It’s obvious from Microsoft’s financial results that they are enjoying growing revenue from commercial cloud services (the last quarter reported an annual growth of 106%). This growth is reflected in a 1350% increase in Exchange Online servers from Aug 1 2012 to 23 April 2015 compared to the 600% increase reported in 2014. The massive increase in servers is required to maintain capacity and to allow Microsoft to absorb new tenants who move to Office 365.

Interestingly, Exchange Online uses the same kind of rings to distribute new software. The rings are the developers, the Office 365 team, Microsoft in general, First Release Office 365 tenants, and finally, General availability. A similar approach is followed with the current Windows 10 insider program and is due to be used in the Windows Update for Business program announced at Monday’s Ignite keynote.

The increase in users means that Office 365 now deals with 55 billion client authentications annually. That kind of transactional volume cannot be handled when the infrastructure doesn’t scale efficiently.

Exchange Online uses 150 petabytes of storage, most of which is taken up by the 4 copies of the 1.2 million mailbox databases. The standard Office 365 mailbox quota is 50 GB, but naturally it takes time for users (maybe 90 million – Microsoft isn’t saying) to use this quota. The thought went through my mind of how many of the 8 TB 7200 rpm standard JBOD drives used by Office 365 fail daily and how they track and fix all the failures. The answer for how many is “a lot” and the management is done through a mixture of a very sophisticated service fabric and human intervention (to remove and replace the failed drives).

The service fabric controls and manages operations flowing across the service and deals with more than 500 million events that are collected hourly. In addition, 250 million synthetic test transactions are generated daily to validate that the Exchange Online service is working properly. The signals gathered by the transactions are analyzed by computers to detect and fix problems, just like the Managed Availability system in Exchange 2013. There’s no surprise here because Managed Availability is an obvious example of technology transfer from the cloud to on-premises (even more technology is being transferred in Exchange 2016). Machine learning is applied to correlate signals and compare them against known sets (that represent a satisfactory condition) to allow engineers to triangulate and identify the particular root problem.

An automation and orchestration workflow engine is used to maintain servers. The most common problems are hardware (disks and controllers), network, and software bugs. Problems can be automatically fixed or left to engineers, who can set off workflow items to address issues. Processes such as server deployments and upgrades are also dealt with through workflow in a way that allows Exchange Online to bring new capacity online within days of deciding that it’s needed. In this respect, new capacity means something like an additional 40 Database Availability Groups rather than a single server.

A DevOps model is used to run Exchange Online. In other words, development engineers don’t simply throw code over the wall to operations and then switch off. Instead, members of the development group up to and including VP level are on call to handle problems that arise in the service. This ensures that engineers take responsibility for the code that they write for the service. If they get it right, happiness and undisturbed nights. But if they get it wrong…

I find sessions that provide an insight into the trials and tribulations of operating a massive multi-tenant environment very interesting and worthwhile. Although you can follow them later online, there’s nothing quite like hearing someone speak in person. This session reinforced my view that something very special occurs to make Office 365 operate. Take the time to view it once the video is posted and make your own mind up.

Follow Tony @12Knocksinna

from Exchange News Full Article

msexchange.org: Exchange 2013 now supported eunning on Azure IaaS VMs

Deployment of Exchange 2013 on Infrastructure-as-a-Service (IaaS) providers is supported if all supportability requirements are met. In the case of providers who are provisioning virtual machines, these requirements include ensuring that the hypervisor being used for Exchange virtual machines is fully supported, and that the infrastructure to be utilized by Exchange meets the performance requirements that were determined during the sizing process. Deployment on Microsoft Azure virtual machines is supported if all storage volumes used for Exchange databases and database transaction logs (including transport databases) are configured for Azure Premium Storage.

from Exchange News Full Article

msexchange.org: What’s new in Office 365 administration from Microsoft Ignite

Today at Microsoft Ignite, we announced some exciting new Office 365 admin features. We focused on two key themes important to you—the Office 365 IT administrator—visibility and control. Enhancing these attributes, gives you greater capability to monitor how your organization is running and enable the necessary controls to keep your data safe.

from Exchange News Full Article

msexchange.org: Office 365 Import Service

You can use the Microsoft Office 365 Import service to transfer PST files from your on-premises environment to your Exchange Online mailboxes. You have two choices to move your data into Office 365: You can ship drives to Microsoft. You can upload files to Office 365.

from Exchange News Full Article

msexchange.org: MSIgnite First Day Review and Announcements

from Exchange News Full Article

Exchange Team Blog: Exchange Server 2016 Architecture

Exchange Server 2016 builds upon the architecture introduced in Exchange Server 2013, with the continued focus goal of improving the architecture to serve the needs of deployments at all scales.

Important: This article contains preliminary information that may be changed prior to final commercial release of the software described herein.

Building Block Architecture

In Exchange Server 2016, there is a single building block that provides the client access services and the high availability architecture necessary for any enterprise messaging environment.

Figure 1: Building Block Architecture

In our continuing quest to improve the product’s capabilities and simplify the architecture and its deployment, we have removed the Client Access server (CAS) role and added the client access services to the Mailbox role. Even without the CAS role, the system maintains loose coupling in terms of functionality, versioning, user partitioning and geographical affinity.

The Mailbox server role now:

1. Houses the logic to route protocol requests to the correct destination endpoint.
2. Hosts all of the components and/or protocols that process, render and store the data.

No clients connect directly to the back-end endpoints on the Mailbox server; instead, clients connect client access services and are routed (via local or remote proxy) to the Mailbox server that hosts the active database that contains the user’s mailbox.

Mailbox servers can be added to a Database Availability Group (DAG), thereby forming a high availability unit that can be deployed in one or more datacenters. DAGs in Exchange Server 2016 do have a few specific enhancements:

1. DatabaseAvailabilityGroupIpAddresses is no longer required when creating a DAG. By default, the failover cluster will be created without an administrative access point, as this is the recommended best practice.
2. Replay Lag Manager is enabled by default.
3. Lagged database copy play down can be delayed based on disk latency, thereby ensuring active users are not impacted.
4. Database failovers times are reduced by 33% when compared to Exchange Server 2013.

Removal of the separate CAS role does not affect how communication occurs between servers. Communication between servers still occurs at the protocol layer, effectively ensuring that every server is an island. For a given mailbox’s connectivity, the protocol being used is always served by the protocol instance that is local to the active database copy.

Figure 2: Inter-server communication in Exchange 2016

The load balancer configuration is also not affected by this architectural change. From a protocol perspective, the following will happen:

1. A client resolves the namespace to a load balanced virtual IP address.
2. The load balancer assigns the session to a Mailbox server in the load balanced pool.
3. The Mailbox server authenticates the request and performs a service discovery by accessing Active Directory to retrieve the following information:
   1. Mailbox version (for this discussion, we will assume an Exchange 2016 mailbox)
   2. Mailbox location information (e.g., database information, ExternalURL values, etc.)
4. The Mailbox server makes the decision to proxy the request or redirect the request to another Mailbox server in the infrastructure (within the same forest).
5. The Mailbox server queries an Active Manager instance that is responsible for the database to determine which Mailbox server is hosting the active copy.
6. The Mailbox server proxies the request to the Mailbox server hosting the active copy.

The protocol used in step 6 depends on the protocol used to connect to client access services. If the client request uses HTTP, then the protocol used between the servers is HTTP (secured via SSL using a self-signed certificate). If the protocol used by the client is IMAP or POP, then the protocol used between the servers is IMAP or POP.

Telephony requests are unique. Instead of proxying the request at step 6, the Mailbox server will redirect the request to the Mailbox server hosting the active copy of the user’s database, as the telephony devices support redirection and need to establish their SIP and RTP sessions directly with the Unified Messaging services on the Mailbox server.

Figure 3: Client Protocol Connectivity

And yes, the Edge Transport server role will ship in Exchange Server 2016 (and at RTM, to boot!). All the capabilities and features you had in the Edge Transport server role in Exchange Server 2013, remain in Exchange Server 2016.

Why did we remove the Client Access server role?

The Exchange Server 2016 architecture evolves the building block architecture that has been refined over the course of the last several releases. With this architecture, all servers in the Exchange environment (excluding Edge Transport) are exactly the same—the same hardware, the same configuration, and so forth. This uniformity simplifies ordering the hardware, as well as performing maintenance and management of the servers.  

As with Exchange 2010 and in Exchange 2013, we continue to recommend role co-location as a best practice. From a cost perspective, the overall goal is to ensure that the architecture is balanced for CPU and disk. Having separate server roles can result in long-term cost disadvantages as you may purchase more CPU, disk, and memory resources than you will actually use. For example, consider a server that hosts only the Client Access server role. Many servers enable you to add a given number of disks in a very economical fashion—when you are deploying and using that number of disks, the cost is essentially zero. But if you deploy a server role that uses far less than the given number of disks, you’re paying for a disk controller that is either under-used or not used at all.

This architecture is designed to enable you to have fewer physical Exchange servers in your environment. Fewer physical servers mean lower costs for a variety of reasons:

• Operational costs are almost always higher than the capital costs. It costs more to manage a server over its lifetime than it does to purchase it.
• You purchase fewer Exchange server licenses. This architecture only requires a license for one Exchange server and one operating system, while breaking out the roles required multiple Exchange server licenses and multiple operating system licenses.
• Deploying fewer servers has a trickle-down effect across the rest of the infrastructure. For example, deploying fewer physical servers may reduce the total rack and floor space required for the Exchange infrastructure, which in turn reduces power and cooling costs.

This architecture ultimately distributes the load across a greater number of servers than deploying single-role servers because all Mailbox servers also handle client access because:

• You’re distributing the load across a greater number of physical machines, which increases scalability. During a failure event, the load on the remaining servers only increases incrementally, which ensures the other functions the server is performing aren’t adversely affected.
• The solution can survive a greater number of Client Access role (or service) failures and still provide service, which increases resiliency.

Key Architectural Improvements

Exchange Server 2016 also includes a number of architectural improvements, beyond the server role consolidation, including search enhancements, document collaboration improvements, and more.

Search Improvements

One of the challenging areas for on-premises environment was the amount of data that was replicated with each database copy in previous releases. In Exchange Server 2016, we have reduced bandwidth requirements between the active copy and a passive copy by 40%. This was accomplished by enabling the local search instance to read data from its local database copy. As a result of this change, passive search instances no longer need to coordinate with their active counterparts in order to perform index updates.

Another area of investment in search has been around decreasing the length of time to return search results, especially in online mode clients like OWA. This is accomplished by performing multiple asynchronous disk reads prior to the user completing the search term, which populates the cache with the relevant information, providing sub-second search query latency for online mode clients.

Document Collaboration

In previous releases of Exchange, OWA included document preview for Office and PDF documents, reducing the need to have a full fidelity client. SharePoint had a similar feature, however it used the Office Web Apps Server to accomplish this capability. Within Office 365, we also leverage Office Web Apps Server to provide this capability, ensuring uniform document preview and editing capability across the suite.

In Exchange Server 2016, we leverage Office Web Apps Server to provide the rich document preview and editing capabilities for OWA. While this was a necessary change to ensure a homogenous experience across the Office Server suite, this does introduce additional complexity for environments that don’t have Office Web Apps Server.

The next generation of Office Web Apps Server will not be supported for co-location with Exchange. Therefore, you must deploy a separate server farm infrastructure. This infrastructure will require unique namespaces, and will require session affinity to be maintained at the load balancer.

While Exchange supports an unbound namespace model, the Office Web Apps Server will require a bound namespace for each site resilient datacenter pair. However, unlike the bound namespace model within Exchange, Office Web Apps Server will not require any namespace changes during a datacenter activation.

Figure 4: Office Web Apps Server Connectivity

Extensibility

Office 365 introduced the REST APIs (Mail, Calendar, and Contact APIs), and now these APIs are available in Exchange Server 2016. The REST APIs simplify programming against Exchange by providing a familiar syntax that is designed with openness (e.g., open standards support JSON, OAUTH, ODATA) and flexibility (e.g., granular, tightly scoped permission to access user data). These APIs allow developers to connect from any platform, whether it be web, PC, or mobile. SDKs exist for.NET, iOS, Android, NodeJS, Ruby, Python, Cordova, and CORS for use in single page JavaScript web apps.

What about Exchange Web Services (EWS)? All existing applications that leverage EWS will continue to work with Exchange Server 2016. We are, however, focusing new platform investments on the REST APIs and the apps for Office extensibility model. We expect to make significantly fewer investments in EWS so that we can focus our resources on investing in a single modern API that will, over time, enable most of the scenarios that our partners currently use EWS.

Outlook Connectivity

Introduced in Exchange Server 2013 Service Pack 1, MAPI/HTTP is the new standard in connectivity for Outlook. In Exchange Server 2016, MAPI/HTTP is enabled by default. In addition, Exchange Server 2016 introduces per-user control over this connectivity model, as well as, the ability to control whether the protocol (and Outlook Anywhere) is advertised to external clients.

Note: Exchange Server 2016 does not support connectivity via the MAPI/CDO library. Third-party products (and custom in-house developed solutions) need to move to Exchange Web Services (EWS) or the REST APIs to access Exchange data.

Coexistence with Exchange Server 2013

In Exchange Server 2013, the Client Access server role is simply an intelligent proxy that performs no processing/rendering of the content. That architectural tenet paid off in terms of forward coexistence. When you introduce Exchange Server 2016, you do not need to move the namespace. That’s right, the Exchange Server 2013 Client Access infrastructure can proxy the mailbox requests to the Exchange 2016 servers hosting the active database copy! For the first time ever, you get to decide when you move the namespace over to the new version. And not only that, you can even have load balancer pools contain a mix of Exchange Server 2013 and Exchange Server 2016. This means you can do a one-for-one swap – as you add Exchange 2016 servers, you can remove Exchange 2013 servers.

The Preferred Architecture

During my session at Microsoft Ignite, I revealed Microsoft’s preferred architecture (PA) for Exchange Server 2016. The PA is the Exchange Engineering Team’s best practice recommendation for what we believe is the optimum deployment architecture for Exchange 2016, and one that is very similar to what we deploy in Office 365.

While Exchange 2016 offers a wide variety of architectural choices for on-premises deployments, this architecture is our most scrutinized one ever. While there are other supported deployment architectures, they are not recommended.

The Exchange 2016 PA is very similar to the Exchange 2013 PA. A symmetrical DAG is deployed across a datacenter pair with active database copies distributed across all servers in the DAG. Database copies are deployed on JBOD storage, with four copies per-disk. One of the copies is a lagged database copy. Clients connect to a unified namespace that is equally distributed across the datacenters in the site resilient pair.

However, the Exchange 2016 PA differs in the following ways:

1. Exchange’s unbound namespace model is load balanced across the datacenters in a layer 7 configuration that does not leverage session affinity.
2. An Office Web Apps Server farm is deployed in each datacenter, with each farm having a unique namespace (bound model). Session affinity is managed by the load balancer.
3. The DAG is deployed without an administrative access point.
4. The commodity dual-socket server hardware platform contains 20-24 cores and up to 196GB of memory, and a battery-backed write cache controller.
5. All data volumes are formatted with ReFS.

As we get closer to release, we'll publish a complete Exchange 2016 Preferred Architecture article.

Summary

Exchange Server 2016 continues in the investments introduced in previous versions of Exchange by reducing the server role architecture complexity, aligning with the Preferred Architecture and Office 365 design principles, and improving coexistence with Exchange Server 2013.

These changes simplify your Exchange deployment, without decreasing the availability or the resiliency of the deployment. And in some scenarios, when compared to previous generations, the PA increases availability and resiliency of your deployment.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience

from Exchange News Full Article

EighTwOne: Ignite 2015 Session Download Script

Yesterday was the first day of Ignite, and Exchange fellow Tony Redmond put up a nice summary of the first day, keynoted included, here.

For those not attending Microsoft Ignite, attending different sessions or not able to enter a session because the room was full, Microsoft publishes Ignite sessions on Channel 9. Because you may want to watch sessions offline, some people created scripts to retrieve all session videos and slidedecks.

Here is a slightly modified script, originally from Claus Nielsen, to download all Microsoft Ignite 2015 videos and slidedecks as the become available on Channel9. You can select sessions based on category or speaker, which helps narrowing down the contents offered at Ignite to sessions you are interested in. The script also allows you to download other session videos and decks, for example from Build 2015 or last year’s TechEd NA.

You can download the script from the TechNet Gallery here.

Filed under: Misc Tagged: Event, Script

from Exchange News Full Article