|
|
|
date: Mon, 17 Mar 2008 22:44:53 -0000,
group: microsoft.public.exchange.design
back
Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
Hi,
I am looking at some comments on a potential storage solution for around
5500 users on Exchange 2007.
Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two seperate
standalone mailbox servers (or 2 x SCC implementations), but I imagine this
should not impact the IOP requirements that much.
I have worked out the following for DB/Log IOP requirements, some of this
have been from John Fullbright's previous posts. I will be calling NetApp to
verify the configuration, but some real world experiences would be
appreciated.
One question though, with the FC 15k drives, is a realistic amount 180 IOPS
per spindle, should i drop to 150 IOPS per spindle?
5775 users @ 0.50 IOPS
5775 (+5%) users @ 0.50 IOPS/user
Peak DB IOPS : 2887
Peak Log IOPS : 288 (DB IOPS / 10)
+ 20% buffer
Peak DB IOPS : 3464
Peak Log IOPS : 345
Using RAID-DP (+ 2 disks)
Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not correct,
but makes sense to over supply for logs)
Total disks : 29 (+2 for extra aggregate)
I figure a couple of aggregates, I have not worked out the volumes yet.
Looking at a NetApp FAS3040 DC (3 shelves).
Thanks
Matthew
date: Mon, 17 Mar 2008 22:44:53 -0000
author: Matthew
Re: Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
On Mon, 17 Mar 2008 22:44:53 -0000, "Matthew" wrote:
>Hi,
>
>I am looking at some comments on a potential storage solution for around
>5500 users on Exchange 2007.
>
>Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two seperate
>standalone mailbox servers (or 2 x SCC implementations), but I imagine this
>should not impact the IOP requirements that much.
>
>I have worked out the following for DB/Log IOP requirements, some of this
>have been from John Fullbright's previous posts. I will be calling NetApp to
>verify the configuration, but some real world experiences would be
>appreciated.
>
>One question though, with the FC 15k drives, is a realistic amount 180 IOPS
>per spindle, should i drop to 150 IOPS per spindle?
>
>5775 users @ 0.50 IOPS
>5775 (+5%) users @ 0.50 IOPS/user
>Peak DB IOPS : 2887
>Peak Log IOPS : 288 (DB IOPS / 10)
>
>+ 20% buffer
>Peak DB IOPS : 3464
>Peak Log IOPS : 345
>
>Using RAID-DP (+ 2 disks)
>Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
>Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not correct,
>but makes sense to over supply for logs)
>
>Total disks : 29 (+2 for extra aggregate)
>
>I figure a couple of aggregates, I have not worked out the volumes yet.
>
>Looking at a NetApp FAS3040 DC (3 shelves).
>
>Thanks
>Matthew
>
>
You can do no better than to follow what John says. Personally I think
you don't have enough disks in your Logs aggregate. I've seen a lot of
problems where people supply the "right" number of disks to match the
IOPS they've worked out and end up with Log Latency issues,
particularly when BlackBerry's get near the place. I can (ok, I can't,
obviously) point you to a legal firm in the City who had a lot of
problems cured by no other measure than adding a half dozen disks to
an agg.
One thing I don't like is the mis-matched shelf count. My ex-boss
would wash his hands of you on that score alone but he's a purist.
Sure, it's not a problem for a basic install so you have two shelves
on one agg attached to one controller for the stores and the 3rd shelf
on the 2nd controller. I assume you have no plans for SnapMirror?
I don't know who supports you in the UK but get them to tap Paul
Hargreaves and whoever (someone new) works with him in NetApp Coventry
to validate you. That's a point though. Whoever sold you the Filer
shouldn't have let you do this on your own. When I worked for the UKs
No1 NetApp integrator we wouldn't allow you to do this kind of thing
without signing it off. It was free and part of the touchy feely
support but we made sure that you got the best possible configuration
done by the most experienced experts and with the best will in the
world we shipped more NetApp in a year than you're ever likely to see.
date: Mon, 17 Mar 2008 19:28:05 -0400
author: Mark Arnold [MVP]
Re: Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
"Mark Arnold [MVP]" wrote in message
news:0nutt39gqnl4urckedjbj71dnm5m8qr7o6@4ax.com...
> On Mon, 17 Mar 2008 22:44:53 -0000, "Matthew" wrote:
>
>>Hi,
>>
>>I am looking at some comments on a potential storage solution for around
>>5500 users on Exchange 2007.
>>
>>Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two
>>seperate
>>standalone mailbox servers (or 2 x SCC implementations), but I imagine
>>this
>>should not impact the IOP requirements that much.
>>
>>I have worked out the following for DB/Log IOP requirements, some of this
>>have been from John Fullbright's previous posts. I will be calling NetApp
>>to
>>verify the configuration, but some real world experiences would be
>>appreciated.
>>
>>One question though, with the FC 15k drives, is a realistic amount 180
>>IOPS
>>per spindle, should i drop to 150 IOPS per spindle?
>>
>>5775 users @ 0.50 IOPS
>>5775 (+5%) users @ 0.50 IOPS/user
>>Peak DB IOPS : 2887
>>Peak Log IOPS : 288 (DB IOPS / 10)
>>
>>+ 20% buffer
>>Peak DB IOPS : 3464
>>Peak Log IOPS : 345
>>
>>Using RAID-DP (+ 2 disks)
>>Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
>>Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not correct,
>>but makes sense to over supply for logs)
>>
>>Total disks : 29 (+2 for extra aggregate)
>>
>>I figure a couple of aggregates, I have not worked out the volumes yet.
>>
>>Looking at a NetApp FAS3040 DC (3 shelves).
>>
>>Thanks
>>Matthew
>>
>>
> You can do no better than to follow what John says. Personally I think
> you don't have enough disks in your Logs aggregate. I've seen a lot of
> problems where people supply the "right" number of disks to match the
> IOPS they've worked out and end up with Log Latency issues,
> particularly when BlackBerry's get near the place. I can (ok, I can't,
> obviously) point you to a legal firm in the City who had a lot of
> problems cured by no other measure than adding a half dozen disks to
> an agg.
>
> One thing I don't like is the mis-matched shelf count. My ex-boss
> would wash his hands of you on that score alone but he's a purist.
> Sure, it's not a problem for a basic install so you have two shelves
> on one agg attached to one controller for the stores and the 3rd shelf
> on the 2nd controller. I assume you have no plans for SnapMirror?
>
> I don't know who supports you in the UK but get them to tap Paul
> Hargreaves and whoever (someone new) works with him in NetApp Coventry
> to validate you. That's a point though. Whoever sold you the Filer
> shouldn't have let you do this on your own. When I worked for the UKs
> No1 NetApp integrator we wouldn't allow you to do this kind of thing
> without signing it off. It was free and part of the touchy feely
> support but we made sure that you got the best possible configuration
> done by the most experienced experts and with the best will in the
> world we shipped more NetApp in a year than you're ever likely to see.
>
>
Hi Mark,
Thanks for the reply.
We have not purchased it yet, I am just getting my feelers out there to make
sure I have the right idea so far as performance and scalability is
concerned.
Your points:
Shelf count mismatch, do you see this as a potential problem? I assume a
potential controller performance difference? Would you suggest purchasing 4
shelves and splitting the disks across the shelves?
Logs aggregate, I guess we can add extra drives to this.
These points are items for me to discuss with NetApp when I give them a call
tomorrow.
Btw, keep up the good work with the blog!
Matthew
date: Mon, 17 Mar 2008 23:44:24 -0000
author: Matthew
Re: Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
If
"Matthew" wrote in message
news:%23GiAGCIiIHA.6032@TK2MSFTNGP03.phx.gbl...
> Hi,
>
> I am looking at some comments on a potential storage solution for around
> 5500 users on Exchange 2007.
>
> Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two
> seperate standalone mailbox servers (or 2 x SCC implementations), but I
> imagine this should not impact the IOP requirements that much.
>
> I have worked out the following for DB/Log IOP requirements, some of this
> have been from John Fullbright's previous posts. I will be calling NetApp
> to verify the configuration, but some real world experiences would be
> appreciated.
>
> One question though, with the FC 15k drives, is a realistic amount 180
> IOPS per spindle, should i drop to 150 IOPS per spindle?
>
> 5775 users @ 0.50 IOPS
> 5775 (+5%) users @ 0.50 IOPS/user
> Peak DB IOPS : 2887
> Peak Log IOPS : 288 (DB IOPS / 10)
>
> + 20% buffer
> Peak DB IOPS : 3464
> Peak Log IOPS : 345
>
> Using RAID-DP (+ 2 disks)
> Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
> Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not correct,
> but makes sense to over supply for logs)
>
> Total disks : 29 (+2 for extra aggregate)
>
> I figure a couple of aggregates, I have not worked out the volumes yet.
>
> Looking at a NetApp FAS3040 DC (3 shelves).
>
> Thanks
> Matthew
>
>
>
date: Tue, 18 Mar 2008 17:30:20 -0700
author: John Fullbright fjohn@donotspamnetappdotcom
Re: Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
I would use 155 IOPS/data spindle. This assumes a host ops to disk ops
ratio of 1.1 or so. When you use the 180 figure, you're not taking into
account the NTFS file system on which the exchange data resides.
You'll find that your log IOPS are going to be higher. In Exchange 2007
designs with cached mode clients, the read/write ratio is close to 1:1.
Half of the IO is writes, and writes to the database mean writes to the
logs. Of course log writes are sequential, and not all log writes are 8K
( a log write can be as small as a single log buffer to as large as 90% of
log buffers). I'd count on about a third of the db IOPS. You can use the
MS sizing calculator to refine your requirement. In fact, I just ran them
in version 11 and get 3333 db iops and 1185 log iops (I actually used .48
IOPS/user)
With a FAS3040C, I'd go with 4 shelves and put two shelves/head. You could
drop down to 10K drives and still meet your performance requirement. The
size of disk will be determinded by your space requirements and the amount
of snapshot data you wish to retain. When it comes to the space
requirement, you'll generally wat to put the snapinfo with the logs and use
Copyless Transaction Log Archiving. This does increase the size of the log
lun, but gives you up to the minute restore capability no matter how many
snaps back you restore from.
John
"Matthew" wrote in message
news:%23GiAGCIiIHA.6032@TK2MSFTNGP03.phx.gbl...
> Hi,
>
> I am looking at some comments on a potential storage solution for around
> 5500 users on Exchange 2007.
>
> Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two
> seperate standalone mailbox servers (or 2 x SCC implementations), but I
> imagine this should not impact the IOP requirements that much.
>
> I have worked out the following for DB/Log IOP requirements, some of this
> have been from John Fullbright's previous posts. I will be calling NetApp
> to verify the configuration, but some real world experiences would be
> appreciated.
>
> One question though, with the FC 15k drives, is a realistic amount 180
> IOPS per spindle, should i drop to 150 IOPS per spindle?
>
> 5775 users @ 0.50 IOPS
> 5775 (+5%) users @ 0.50 IOPS/user
> Peak DB IOPS : 2887
> Peak Log IOPS : 288 (DB IOPS / 10)
>
> + 20% buffer
> Peak DB IOPS : 3464
> Peak Log IOPS : 345
>
> Using RAID-DP (+ 2 disks)
> Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
> Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not correct,
> but makes sense to over supply for logs)
>
> Total disks : 29 (+2 for extra aggregate)
>
> I figure a couple of aggregates, I have not worked out the volumes yet.
>
> Looking at a NetApp FAS3040 DC (3 shelves).
>
> Thanks
> Matthew
>
>
>
date: Tue, 18 Mar 2008 17:52:12 -0700
author: John Fullbright fjohn@donotspamnetappdotcom
Re: Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
Hi John,
Thanks for the reply.
Another question, do you think the FSA2050 would suffice for this
deployment?
Regards
Matthew
"John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message
news:OP6I5tViIHA.4468@TK2MSFTNGP03.phx.gbl...
>I would use 155 IOPS/data spindle. This assumes a host ops to disk ops
>ratio of 1.1 or so. When you use the 180 figure, you're not taking into
>account the NTFS file system on which the exchange data resides.
>
> You'll find that your log IOPS are going to be higher. In Exchange 2007
> designs with cached mode clients, the read/write ratio is close to 1:1.
> Half of the IO is writes, and writes to the database mean writes to the
> logs. Of course log writes are sequential, and not all log writes are 8K
> ( a log write can be as small as a single log buffer to as large as 90% of
> log buffers). I'd count on about a third of the db IOPS. You can use the
> MS sizing calculator to refine your requirement. In fact, I just ran them
> in version 11 and get 3333 db iops and 1185 log iops (I actually used .48
> IOPS/user)
>
> With a FAS3040C, I'd go with 4 shelves and put two shelves/head. You
> could drop down to 10K drives and still meet your performance requirement.
> The size of disk will be determinded by your space requirements and the
> amount of snapshot data you wish to retain. When it comes to the space
> requirement, you'll generally wat to put the snapinfo with the logs and
> use Copyless Transaction Log Archiving. This does increase the size of
> the log lun, but gives you up to the minute restore capability no matter
> how many snaps back you restore from.
>
> John
>
>
>
>
> "Matthew" wrote in message
> news:%23GiAGCIiIHA.6032@TK2MSFTNGP03.phx.gbl...
>> Hi,
>>
>> I am looking at some comments on a potential storage solution for around
>> 5500 users on Exchange 2007.
>>
>> Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two
>> seperate standalone mailbox servers (or 2 x SCC implementations), but I
>> imagine this should not impact the IOP requirements that much.
>>
>> I have worked out the following for DB/Log IOP requirements, some of this
>> have been from John Fullbright's previous posts. I will be calling NetApp
>> to verify the configuration, but some real world experiences would be
>> appreciated.
>>
>> One question though, with the FC 15k drives, is a realistic amount 180
>> IOPS per spindle, should i drop to 150 IOPS per spindle?
>>
>> 5775 users @ 0.50 IOPS
>> 5775 (+5%) users @ 0.50 IOPS/user
>> Peak DB IOPS : 2887
>> Peak Log IOPS : 288 (DB IOPS / 10)
>>
>> + 20% buffer
>> Peak DB IOPS : 3464
>> Peak Log IOPS : 345
>>
>> Using RAID-DP (+ 2 disks)
>> Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
>> Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not correct,
>> but makes sense to over supply for logs)
>>
>> Total disks : 29 (+2 for extra aggregate)
>>
>> I figure a couple of aggregates, I have not worked out the volumes yet.
>>
>> Looking at a NetApp FAS3040 DC (3 shelves).
>>
>> Thanks
>> Matthew
>>
>>
>>
>
>
date: Wed, 19 Mar 2008 12:16:55 -0000
author: Matthew
Re: Ex2k7 MBX using NetApp FAS3040 (5775 users @ 0.50 IOPS/user)
The 2050 is fully capable of supporting the number of spindles you need.
You can have 20 internal drives and up to 84 external drives (6 shelves)
attached to a head. In fact, we did an ESRP submission on a 5000 user (.33
iops/User) iscsi setup that is similar in many respects to what you are
attempting. You may want to take a look at that.
http://www.netapp.com/us/solutions/solution-partners/global-alliance/ms-esrp-fas2000.html
John
"Matthew" wrote in message
news:eSBHhsbiIHA.4684@TK2MSFTNGP06.phx.gbl...
> Hi John,
>
> Thanks for the reply.
>
> Another question, do you think the FSA2050 would suffice for this
> deployment?
>
> Regards
> Matthew
>
> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message
> news:OP6I5tViIHA.4468@TK2MSFTNGP03.phx.gbl...
>>I would use 155 IOPS/data spindle. This assumes a host ops to disk ops
>>ratio of 1.1 or so. When you use the 180 figure, you're not taking into
>>account the NTFS file system on which the exchange data resides.
>>
>> You'll find that your log IOPS are going to be higher. In Exchange 2007
>> designs with cached mode clients, the read/write ratio is close to 1:1.
>> Half of the IO is writes, and writes to the database mean writes to the
>> logs. Of course log writes are sequential, and not all log writes are 8K
>> ( a log write can be as small as a single log buffer to as large as 90%
>> of log buffers). I'd count on about a third of the db IOPS. You can use
>> the MS sizing calculator to refine your requirement. In fact, I just ran
>> them in version 11 and get 3333 db iops and 1185 log iops (I actually
>> used .48 IOPS/user)
>>
>> With a FAS3040C, I'd go with 4 shelves and put two shelves/head. You
>> could drop down to 10K drives and still meet your performance
>> requirement. The size of disk will be determinded by your space
>> requirements and the amount of snapshot data you wish to retain. When it
>> comes to the space requirement, you'll generally wat to put the snapinfo
>> with the logs and use Copyless Transaction Log Archiving. This does
>> increase the size of the log lun, but gives you up to the minute restore
>> capability no matter how many snaps back you restore from.
>>
>> John
>>
>>
>>
>>
>> "Matthew" wrote in message
>> news:%23GiAGCIiIHA.6032@TK2MSFTNGP03.phx.gbl...
>>> Hi,
>>>
>>> I am looking at some comments on a potential storage solution for around
>>> 5500 users on Exchange 2007.
>>>
>>> Config, SCC, 8 cores, 32GB RAM. We may consider splitting into two
>>> seperate standalone mailbox servers (or 2 x SCC implementations), but I
>>> imagine this should not impact the IOP requirements that much.
>>>
>>> I have worked out the following for DB/Log IOP requirements, some of
>>> this have been from John Fullbright's previous posts. I will be calling
>>> NetApp to verify the configuration, but some real world experiences
>>> would be appreciated.
>>>
>>> One question though, with the FC 15k drives, is a realistic amount 180
>>> IOPS per spindle, should i drop to 150 IOPS per spindle?
>>>
>>> 5775 users @ 0.50 IOPS
>>> 5775 (+5%) users @ 0.50 IOPS/user
>>> Peak DB IOPS : 2887
>>> Peak Log IOPS : 288 (DB IOPS / 10)
>>>
>>> + 20% buffer
>>> Peak DB IOPS : 3464
>>> Peak Log IOPS : 345
>>>
>>> Using RAID-DP (+ 2 disks)
>>> Database : 3464 / 180 = 19 disks + 2 (21) (5400GB)
>>> Logs : 345 / 180 = 6 disks (8) (1800GB) ( i realise this is not
>>> correct, but makes sense to over supply for logs)
>>>
>>> Total disks : 29 (+2 for extra aggregate)
>>>
>>> I figure a couple of aggregates, I have not worked out the volumes yet.
>>>
>>> Looking at a NetApp FAS3040 DC (3 shelves).
>>>
>>> Thanks
>>> Matthew
>>>
>>>
>>>
>>
>>
>
>
date: Wed, 19 Mar 2008 10:06:56 -0700
author: John Fullbright fjohn@donotspamnetappdotcom
|
|