For Exchange, NTFS has been the file system of choice since time immemorial. In 2012, Windows Server 2012 introduced a new file system: Resilient File System or just ReFS. ReFS was designed to overcome some of the limitations of NTFS, in particular in the area of maintaining data integrity. More information on ReFS in comparison to NTFS can be found here.
At that time Windows Server 2012 went RTM, the latest version of Exchange, Exchange 2010, was not supported to run on ReFS. Present day, Exchange 2010 still doesn’t support ReFS. However, when Exchange 2013 entered the arena shortly after Windows Server 2012, it came with support for both NTFS and ReFS file systems. NTFS was still considered best practice, with ReFS being a supported option with the added recommendation to turn off ReFS’ integrity checking feature, and disabling it for Content Index-exclusive volume is optional. It may therefor come as no surprise that nearly all customers are deploying Exchange 2013 on NTFS volumes only.
That may change with Exchange 2016. As announced at Ignite 2015, for Exchange 2016 more emphasis will be put on following the Preferred Architecture design when deploying Exchange on-premises. The Exchange 2016 Preferred Architecture contains guidance to use ReFS formatted, BitLocker encrypted data volumes with Exchange 2016. The latter option is of course to protect organizations against theft of physical storage devices.
With some time to spare, I was interested to see what the impact would be on the storage performance when using NTFS or ReFS, and especially the performance penalty when enabling BitLocker on a volume. Similar to a comparison I did between Exchange 2010 and Exchange 2013 on different operating systems, I ran a JetStress 2013 test utilizing these 3 file systems to get a sense of what to expect.
The ESE engine files from Exchange 2013 CU8 were used for testing, along with the following parameters:
| Mode | Test Disk Subsystem Throughput |
| Thread Count | 12 (fixed) |
| Min/Max DB Cache | 32 MB / 256 MB |
| Ins / Del / Repl / Read % | 40/20/5/35 |
| Lazy Commits | 70% |
| Run Background DB Maintenance | True |
| Databases | 1 x DB (186GB), 3 Copies |
| Running Time | 2 Hours |
Databases and logs were stored on a DAS SSD drive, and the volume was GPT partitioned and utilizing 64K allocation units. ReFS Integrity checking was disabled for the volume using:
Format-Volume –DriveLetter X -FileSystem ReFS -AllocationUnitSize 65536 -SetIntegrityStreams $false
The drive supported hardware encryption for BitLocker, which offloads encryption to the drive. You can verify that hardware encryption is used after enabling BitLocker on the volume by inspecting the BitLocker status using the manage-bde utility or Get-BitLockerVolume cmdlet:
As you can see from the EncryptionMethod property, this volume is protected using hardware-based BitLocker encryption. Perhaps needless to say, but when using BitLocker with software encryption, the CPU performance penalty is substantial, and this is not to be used with I/O intensive applications like Exchange.
The results from the JetStress tests are show in the following table:
| JetStress Version | 15.0.658.4 | ||||
| ESE.DLL | 15.0.1076.9 | ||||
| Operating System | 6.2.9200.0 | ||||
| Overall Test Result |
Passed |
Passed |
|
Passed |
|
| Achieved Transactional IOPS |
1,613.13 |
1,407.55 |
-13% |
1,379.98 |
-14% |
| Database Reads Average Latency (msec) |
8.53 |
10.50 |
-23% |
9.73 |
-14% |
| Database Writes Average Latency (msec) |
12.80 |
20.80 |
-63% |
19.98 |
-56% |
| Database Reads/sec |
895.25 |
787.08 |
-12% |
769.47 |
-14% |
| Database Writes/sec |
726.48 |
628.55 |
-13% |
618.65 |
-15% |
| Database Reads Average Bytes |
35,220.22 |
35,375.26 |
0% |
35,437.64 |
1% |
| Database Writes Average Bytes |
34,389.82 |
34,510.95 |
0% |
34,496.88 |
0% |
| Log Reads Average Latency (msec) |
4.64 |
5.06 |
-9% |
5.00 |
-8% |
| Log Writes Average Latency (msec) |
5.16 |
7.22 |
-40% |
6.73 |
-30% |
| Log Reads/sec |
18.64 |
16.29 |
-13% |
16.08 |
-14% |
| Log Writes/sec |
87.25 |
72.81 |
-17% |
73.82 |
-15% |
| Log Reads Average Bytes |
232,562.72 |
232,562.01 |
0% |
232,562.30 |
0% |
| Log Writes Average Bytes |
25,005.97 |
26,210.03 |
5% |
25,589.45 |
2% |
| Avg. % Processor Time |
4.28 |
3.66 |
14% |
3.60 |
16% |
Some observations and notes:
- ReFS took a near 15% IOPS performance hit when compared to NTFS.
- Usage of ReFS resulted in increased I/O latencies, especially for write operations.
- ReFS seem to have a positive impact on the processor utilization, lowering average utilization by around 15%.
- For some strange reason, average write latencies seemed lower when using ReFS with BitLocker than without BitLocker (~10%).
Given the impact of file system choice on I/O performance and CPU utilization, I hope next versions of Exchange Server Role Calculator will feature an option to select which file system will be used to store Exchange data, as the difference in I/O performance and CPU utilization between NTFS and ReFS seems significant.
The JetStress reports can be found here.
I will finish with a short disclaimer: This test was only performed to get an indication of performance impact of using different file systems with Exchange 2013 utilizing identical hardware. The results are purely indicative and not meant to provide guidance or proof. Your mileage may vary.
Filed under: Exchange 2013 Tagged: Design, Planning, Storage
from Exchange News Full Article
No comments:
Post a Comment