|
|
|
date: Tue, 24 Jun 2008 20:01:04 +0000 (UTC),
group: microsoft.public.exchange.applications
back
Re: Named properties quota exceeded?
Steve-
Yes, we agree that it sucks. And while it likely doesn't change our options
for fixing the situation, I respectfully disagree that you (Microsoft) are
"doing our best to help everyone cope with it."
Check through your Exchange 2007 Planning and Architecture documentation,
and Deployment documentation, and Operations documentation. I have yet to
find a reference to the named properties exhaustion problem. There is a
bunch of information about sizing databases for I/O, backup and restore,
etc, but it all looks like a ghost town with tumbleweeds rolling through
when it comes to MAPI named properties. Where is the information about this
on the EHLO blog? Try searching for "MapiExceptionNamedPropsQuotaExceeded"
at support.microsoft.com; not much there, eh? Could you provide some details
on how to "download a copy of MFCMapi and logon to the information store and
dump out the Named Props table."?
My impression (opinion only) is that you (Microsoft) have a very
uncomfortable situation and are hoping that the spotlight doesn't shine in
this area. I mean, you have a great product in terms of stability,
scalability, fault tolerance, clustering and such, but the very real
possibility that any deployment receiving reasonably large amounts of
internet email is going to start dropping messages. Not exactly a strong
selling point, is it? But your customers need to be aware of this up front
during the design stage of their upgrade/rollout if the best you can offer
for mitigation is to move mailboxes to a new store. From an operational and
administrative standpoint, that's a remarkably poor solution and we need
something better.
--
Hunter
"Stephen Griffin" wrote in message
news:%23WtICK51IHA.552@TK2MSFTNGP06.phx.gbl...
> Yes - it sucks. We're all in agreement on this. But it's the way things
> are, and as long as MAPI property values are 32 bit numbers with only 16
> bits available for the unique identifier, that's the way things will
> continue to be. It's a bad situation and we're doing our best to help
> everyone cope with it.
>
> This scenario (and others) are part of why we don't recommend placing so
> many users in a single database. You can put them all on the same server -
> just break up the databases so you have far fewer users per database. The
> named prop issue will take much longer to appear, migration to a new
> database if it does appear will be much faster, and backup/restore times
> will be much shorter. I'm sure someone around here's got a link to our
> official guidance if you want some numbers to help.
>
> Steve
>
> "Karl Mitschke" wrote in message
> news:d66cd4c240ce58caa419e07da5c3@msnews.microsoft.com...
>> Hello Dgoldman [MSFT],
>>
>>> You dont want to increase the limit because down the road all you are
>>> going to do is hit it and have the same problem. You can download a
>>> copy of MFCMapi and logon to the information store and dump out the
>>> Named Props table. Essentially the only way to fix it (the correct
>>> way) is move the users to a new information store and also to stop the
>>> program from creating so many named props.
>>>
>>> Dgoldman
>>> http://blogs.msdn.com/dgoldman
>>> Download OABInteg
>>> (http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A
>>> 2338E73-F521-4071-9B1D-AAF49C346ACD)
>>
>> Are you kidding me?
>>
>> We get 1 million mail messages a month. If only a very small percentage
>> of thes have a rogue header like "x-david-mailscanner-spamcheck" and soon
>> we will have to move all 14000 users to new databases?
>>
>> That's insane!
>>
>>
>
>
date: Thu, 26 Jun 2008 10:05:30 -0600
author: Hunter Coleman
Re: Named properties quota exceeded?
Here's the EHLO article from three years ago talking about the problem:
http://msexchangeteam.com/archive/2005/08/08/408796.aspx.
Note that acknowledgement that named property exhaustion could be used as a
basis for a Denial of Service attack. We're not trying to hide this.
I also found these articles on technet and support.microsoft.com:
Cannot create any more MAPI named properties -
http://technet.microsoft.com/en-us/library/bb218047(EXCHG.80).aspx
Understanding the Impact of Named Property and Replica Identifier Limits on
Exchange Databases -
http://technet.microsoft.com/en-us/library/bb851492(EXCHG.80).aspx
How to configure quota settings for named properties and for replica
identifiers in Exchange Server 2003 and in Exchange Server 2007 -
http://support.microsoft.com/?kbid=820379
Events 9666, 9667, 9668, and 9669 Received When Named Properties or Replica
Identifiers Are Depleted for An Exchange Database -
http://technet.microsoft.com/en-us/library/bb851495.aspx
MFCMAPI can be downloaded from here:
http://codeplex.com/mfcmapi
If you're looking to dump the named property table, make sure you're using
an uncached profile (else you'll just be looking at the named prop table for
the OST), then open your mailbox and run "Property Pane"/"Find All Named
Props (SLOW)"
Steve
"Hunter Coleman" wrote in message
news:C3EB4069-2F3D-45A5-A991-C2E04157DDFB@microsoft.com...
> Steve-
>
> Yes, we agree that it sucks. And while it likely doesn't change our
> options for fixing the situation, I respectfully disagree that you
> (Microsoft) are "doing our best to help everyone cope with it."
>
> Check through your Exchange 2007 Planning and Architecture documentation,
> and Deployment documentation, and Operations documentation. I have yet to
> find a reference to the named properties exhaustion problem. There is a
> bunch of information about sizing databases for I/O, backup and restore,
> etc, but it all looks like a ghost town with tumbleweeds rolling through
> when it comes to MAPI named properties. Where is the information about
> this on the EHLO blog? Try searching for
> "MapiExceptionNamedPropsQuotaExceeded" at support.microsoft.com; not much
> there, eh? Could you provide some details on how to "download a copy of
> MFCMapi and logon to the information store and dump out the Named Props
> table."?
>
> My impression (opinion only) is that you (Microsoft) have a very
> uncomfortable situation and are hoping that the spotlight doesn't shine in
> this area. I mean, you have a great product in terms of stability,
> scalability, fault tolerance, clustering and such, but the very real
> possibility that any deployment receiving reasonably large amounts of
> internet email is going to start dropping messages. Not exactly a strong
> selling point, is it? But your customers need to be aware of this up front
> during the design stage of their upgrade/rollout if the best you can offer
> for mitigation is to move mailboxes to a new store. From an operational
> and administrative standpoint, that's a remarkably poor solution and we
> need something better.
>
> --
> Hunter
>
> "Stephen Griffin" wrote in message
> news:%23WtICK51IHA.552@TK2MSFTNGP06.phx.gbl...
>> Yes - it sucks. We're all in agreement on this. But it's the way things
>> are, and as long as MAPI property values are 32 bit numbers with only 16
>> bits available for the unique identifier, that's the way things will
>> continue to be. It's a bad situation and we're doing our best to help
>> everyone cope with it.
>>
>> This scenario (and others) are part of why we don't recommend placing so
>> many users in a single database. You can put them all on the same
>> server - just break up the databases so you have far fewer users per
>> database. The named prop issue will take much longer to appear, migration
>> to a new database if it does appear will be much faster, and
>> backup/restore times will be much shorter. I'm sure someone around here's
>> got a link to our official guidance if you want some numbers to help.
>>
>> Steve
>>
>> "Karl Mitschke" wrote in message
>> news:d66cd4c240ce58caa419e07da5c3@msnews.microsoft.com...
>>> Hello Dgoldman [MSFT],
>>>
>>>> You dont want to increase the limit because down the road all you are
>>>> going to do is hit it and have the same problem. You can download a
>>>> copy of MFCMapi and logon to the information store and dump out the
>>>> Named Props table. Essentially the only way to fix it (the correct
>>>> way) is move the users to a new information store and also to stop the
>>>> program from creating so many named props.
>>>>
>>>> Dgoldman
>>>> http://blogs.msdn.com/dgoldman
>>>> Download OABInteg
>>>> (http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A
>>>> 2338E73-F521-4071-9B1D-AAF49C346ACD)
>>>
>>> Are you kidding me?
>>>
>>> We get 1 million mail messages a month. If only a very small percentage
>>> of thes have a rogue header like "x-david-mailscanner-spamcheck" and
>>> soon we will have to move all 14000 users to new databases?
>>>
>>> That's insane!
>>>
>>>
>>
>>
>
date: Thu, 26 Jun 2008 14:49:36 -0400
author: Stephen Griffin
Re: Named properties quota exceeded?
Thank you for those links.
Are there any perfmon counters that will show what the named property count
is for a store?
When I dump the named property table for one store, I see entries such as
<NamedPropName>sz: "x-antivirus-konceptio" and
<NamedPropName>sz: "x-sender-reputation"
that are fairly obvious as to their source. However, I'm also seeing several
thousand PS_INTERNET_HEADERS formatted like:
<NamedPropName>sz: "m+c@x("tr."xt,"!41`hh16yd"
<NamedPropName>sz: "m(#(n.3(@5$0**"`p+c`p*2!4"
<NamedPropName>sz: "m+c0p(#(n.3(@5$0**"`p+c`p*2!4"
etc.
I want to identify the internet messages that have those 'randomly
generated' X-headers and look into the system/gateway that is creating them;
how can I associate those named property entries with the message(s) that
contain the corresponding X-headers?
--
Hunter
"Stephen Griffin" wrote in message
news:uPF8i171IHA.4188@TK2MSFTNGP04.phx.gbl...
> Here's the EHLO article from three years ago talking about the problem:
> http://msexchangeteam.com/archive/2005/08/08/408796.aspx.
> Note that acknowledgement that named property exhaustion could be used as
> a basis for a Denial of Service attack. We're not trying to hide this.
>
> I also found these articles on technet and support.microsoft.com:
> Cannot create any more MAPI named properties -
> http://technet.microsoft.com/en-us/library/bb218047(EXCHG.80).aspx
> Understanding the Impact of Named Property and Replica Identifier Limits
> on Exchange Databases -
> http://technet.microsoft.com/en-us/library/bb851492(EXCHG.80).aspx
> How to configure quota settings for named properties and for replica
> identifiers in Exchange Server 2003 and in Exchange Server 2007 -
> http://support.microsoft.com/?kbid=820379
> Events 9666, 9667, 9668, and 9669 Received When Named Properties or
> Replica Identifiers Are Depleted for An Exchange Database -
> http://technet.microsoft.com/en-us/library/bb851495.aspx
>
> MFCMAPI can be downloaded from here:
> http://codeplex.com/mfcmapi
>
> If you're looking to dump the named property table, make sure you're using
> an uncached profile (else you'll just be looking at the named prop table
> for the OST), then open your mailbox and run "Property Pane"/"Find All
> Named Props (SLOW)"
>
> Steve
>
> "Hunter Coleman" wrote in message
> news:C3EB4069-2F3D-45A5-A991-C2E04157DDFB@microsoft.com...
>> Steve-
>>
>> Yes, we agree that it sucks. And while it likely doesn't change our
>> options for fixing the situation, I respectfully disagree that you
>> (Microsoft) are "doing our best to help everyone cope with it."
>>
>> Check through your Exchange 2007 Planning and Architecture documentation,
>> and Deployment documentation, and Operations documentation. I have yet to
>> find a reference to the named properties exhaustion problem. There is a
>> bunch of information about sizing databases for I/O, backup and restore,
>> etc, but it all looks like a ghost town with tumbleweeds rolling through
>> when it comes to MAPI named properties. Where is the information about
>> this on the EHLO blog? Try searching for
>> "MapiExceptionNamedPropsQuotaExceeded" at support.microsoft.com; not much
>> there, eh? Could you provide some details on how to "download a copy of
>> MFCMapi and logon to the information store and dump out the Named Props
>> table."?
>>
>> My impression (opinion only) is that you (Microsoft) have a very
>> uncomfortable situation and are hoping that the spotlight doesn't shine
>> in this area. I mean, you have a great product in terms of stability,
>> scalability, fault tolerance, clustering and such, but the very real
>> possibility that any deployment receiving reasonably large amounts of
>> internet email is going to start dropping messages. Not exactly a strong
>> selling point, is it? But your customers need to be aware of this up
>> front during the design stage of their upgrade/rollout if the best you
>> can offer for mitigation is to move mailboxes to a new store. From an
>> operational and administrative standpoint, that's a remarkably poor
>> solution and we need something better.
>>
>> --
>> Hunter
>>
>> "Stephen Griffin" wrote in message
>> news:%23WtICK51IHA.552@TK2MSFTNGP06.phx.gbl...
>>> Yes - it sucks. We're all in agreement on this. But it's the way things
>>> are, and as long as MAPI property values are 32 bit numbers with only 16
>>> bits available for the unique identifier, that's the way things will
>>> continue to be. It's a bad situation and we're doing our best to help
>>> everyone cope with it.
>>>
>>> This scenario (and others) are part of why we don't recommend placing so
>>> many users in a single database. You can put them all on the same
>>> server - just break up the databases so you have far fewer users per
>>> database. The named prop issue will take much longer to appear,
>>> migration to a new database if it does appear will be much faster, and
>>> backup/restore times will be much shorter. I'm sure someone around
>>> here's got a link to our official guidance if you want some numbers to
>>> help.
>>>
>>> Steve
>>>
>>> "Karl Mitschke" wrote in message
>>> news:d66cd4c240ce58caa419e07da5c3@msnews.microsoft.com...
>>>> Hello Dgoldman [MSFT],
>>>>
>>>>> You dont want to increase the limit because down the road all you are
>>>>> going to do is hit it and have the same problem. You can download a
>>>>> copy of MFCMapi and logon to the information store and dump out the
>>>>> Named Props table. Essentially the only way to fix it (the correct
>>>>> way) is move the users to a new information store and also to stop the
>>>>> program from creating so many named props.
>>>>>
>>>>> Dgoldman
>>>>> http://blogs.msdn.com/dgoldman
>>>>> Download OABInteg
>>>>> (http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A
>>>>> 2338E73-F521-4071-9B1D-AAF49C346ACD)
>>>>
>>>> Are you kidding me?
>>>>
>>>> We get 1 million mail messages a month. If only a very small percentage
>>>> of thes have a rogue header like "x-david-mailscanner-spamcheck" and
>>>> soon we will have to move all 14000 users to new databases?
>>>>
>>>> That's insane!
>>>>
>>>>
>>>
>>>
>>
>
>
date: Fri, 27 Jun 2008 11:26:44 -0600
author: Hunter Coleman
Re: Named properties quota exceeded?
I don't think there's a perf counter for this - someone admin focused might
correct me.
The random headers: We've seen those, and not everything is understood about
them. We think they come from spam mail, but it's also possible they're
coming from non-english mail that may have used foreign character sets (like
asian character sets) in the headers. Headers like that did lead to a
problem we issued a hotfix for:
http://support.microsoft.com/kb/941060
This fix reduces the number of those randomly generated headers that can get
into the database. The cache referred to in the article is not actually the
named prop cache, but the fix has an effect on both. With that fix applied
(or any fix that puts exmime.dll >= 6.5.7653.20) you should see a reduction
in the number of those type of headers getting promoted to the named prop
cache. Of course, the fix won't clean up the ones that are already there -
to do that you'd need to move the mailboxes. :)
Steve
"Hunter Coleman" wrote in message
news:172298CB-BAAD-44C1-A29B-F554E5D50004@microsoft.com...
> Thank you for those links.
>
> Are there any perfmon counters that will show what the named property
> count is for a store?
>
> When I dump the named property table for one store, I see entries such as
> <NamedPropName>sz: "x-antivirus-konceptio" and
> <NamedPropName>sz: "x-sender-reputation"
> that are fairly obvious as to their source. However, I'm also seeing
> several thousand PS_INTERNET_HEADERS formatted like:
> <NamedPropName>sz: "m+c@x("tr."xt,"!41`hh16yd"
> <NamedPropName>sz: "m(#(n.3(@5$0**"`p+c`p*2!4"
> <NamedPropName>sz: "m+c0p(#(n.3(@5$0**"`p+c`p*2!4"
> etc.
>
> I want to identify the internet messages that have those 'randomly
> generated' X-headers and look into the system/gateway that is creating
> them; how can I associate those named property entries with the message(s)
> that contain the corresponding X-headers?
>
> --
> Hunter
>
> "Stephen Griffin" wrote in message
> news:uPF8i171IHA.4188@TK2MSFTNGP04.phx.gbl...
>> Here's the EHLO article from three years ago talking about the problem:
>> http://msexchangeteam.com/archive/2005/08/08/408796.aspx.
>> Note that acknowledgement that named property exhaustion could be used as
>> a basis for a Denial of Service attack. We're not trying to hide this.
>>
>> I also found these articles on technet and support.microsoft.com:
>> Cannot create any more MAPI named properties -
>> http://technet.microsoft.com/en-us/library/bb218047(EXCHG.80).aspx
>> Understanding the Impact of Named Property and Replica Identifier Limits
>> on Exchange Databases -
>> http://technet.microsoft.com/en-us/library/bb851492(EXCHG.80).aspx
>> How to configure quota settings for named properties and for replica
>> identifiers in Exchange Server 2003 and in Exchange Server 2007 -
>> http://support.microsoft.com/?kbid=820379
>> Events 9666, 9667, 9668, and 9669 Received When Named Properties or
>> Replica Identifiers Are Depleted for An Exchange Database -
>> http://technet.microsoft.com/en-us/library/bb851495.aspx
>>
>> MFCMAPI can be downloaded from here:
>> http://codeplex.com/mfcmapi
>>
>> If you're looking to dump the named property table, make sure you're
>> using an uncached profile (else you'll just be looking at the named prop
>> table for the OST), then open your mailbox and run "Property Pane"/"Find
>> All Named Props (SLOW)"
>>
>> Steve
>>
>> "Hunter Coleman" wrote in message
>> news:C3EB4069-2F3D-45A5-A991-C2E04157DDFB@microsoft.com...
>>> Steve-
>>>
>>> Yes, we agree that it sucks. And while it likely doesn't change our
>>> options for fixing the situation, I respectfully disagree that you
>>> (Microsoft) are "doing our best to help everyone cope with it."
>>>
>>> Check through your Exchange 2007 Planning and Architecture
>>> documentation, and Deployment documentation, and Operations
>>> documentation. I have yet to find a reference to the named properties
>>> exhaustion problem. There is a bunch of information about sizing
>>> databases for I/O, backup and restore, etc, but it all looks like a
>>> ghost town with tumbleweeds rolling through when it comes to MAPI named
>>> properties. Where is the information about this on the EHLO blog? Try
>>> searching for "MapiExceptionNamedPropsQuotaExceeded" at
>>> support.microsoft.com; not much there, eh? Could you provide some
>>> details on how to "download a copy of MFCMapi and logon to the
>>> information store and dump out the Named Props table."?
>>>
>>> My impression (opinion only) is that you (Microsoft) have a very
>>> uncomfortable situation and are hoping that the spotlight doesn't shine
>>> in this area. I mean, you have a great product in terms of stability,
>>> scalability, fault tolerance, clustering and such, but the very real
>>> possibility that any deployment receiving reasonably large amounts of
>>> internet email is going to start dropping messages. Not exactly a strong
>>> selling point, is it? But your customers need to be aware of this up
>>> front during the design stage of their upgrade/rollout if the best you
>>> can offer for mitigation is to move mailboxes to a new store. From an
>>> operational and administrative standpoint, that's a remarkably poor
>>> solution and we need something better.
>>>
>>> --
>>> Hunter
>>>
>>> "Stephen Griffin" wrote in message
>>> news:%23WtICK51IHA.552@TK2MSFTNGP06.phx.gbl...
>>>> Yes - it sucks. We're all in agreement on this. But it's the way things
>>>> are, and as long as MAPI property values are 32 bit numbers with only
>>>> 16 bits available for the unique identifier, that's the way things will
>>>> continue to be. It's a bad situation and we're doing our best to help
>>>> everyone cope with it.
>>>>
>>>> This scenario (and others) are part of why we don't recommend placing
>>>> so many users in a single database. You can put them all on the same
>>>> server - just break up the databases so you have far fewer users per
>>>> database. The named prop issue will take much longer to appear,
>>>> migration to a new database if it does appear will be much faster, and
>>>> backup/restore times will be much shorter. I'm sure someone around
>>>> here's got a link to our official guidance if you want some numbers to
>>>> help.
>>>>
>>>> Steve
>>>>
>>>> "Karl Mitschke" wrote in message
>>>> news:d66cd4c240ce58caa419e07da5c3@msnews.microsoft.com...
>>>>> Hello Dgoldman [MSFT],
>>>>>
>>>>>> You dont want to increase the limit because down the road all you are
>>>>>> going to do is hit it and have the same problem. You can download a
>>>>>> copy of MFCMapi and logon to the information store and dump out the
>>>>>> Named Props table. Essentially the only way to fix it (the correct
>>>>>> way) is move the users to a new information store and also to stop
>>>>>> the
>>>>>> program from creating so many named props.
>>>>>>
>>>>>> Dgoldman
>>>>>> http://blogs.msdn.com/dgoldman
>>>>>> Download OABInteg
>>>>>> (http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A
>>>>>> 2338E73-F521-4071-9B1D-AAF49C346ACD)
>>>>>
>>>>> Are you kidding me?
>>>>>
>>>>> We get 1 million mail messages a month. If only a very small
>>>>> percentage of thes have a rogue header like
>>>>> "x-david-mailscanner-spamcheck" and soon we will have to move all
>>>>> 14000 users to new databases?
>>>>>
>>>>> That's insane!
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
date: Fri, 27 Jun 2008 15:39:54 -0400
author: Stephen Griffin
Re: Named properties quota exceeded?
We're on E2k7 SP1 RU2, so I'm assuming that we're already getting any relief
that came with the exmime.dll hotfix.
The perfmon counter would be helpful for baselining and long-term growth
monitoring. As it is, we don't seem to have any way of knowing whether the
named prop cache is growing in a steady fashion, or random/bursty.
And there's no way to correlate a particular X-header (or any other entry)
in the cache to the message(s) that contain that header? So if we had an
ill-designed application generating custom named properties, or an external
SMTP host causing lots of custom named properties to generate, how should we
go about identifying that source?
--
Hunter
"Stephen Griffin" wrote in message
news:%23SBRU2I2IHA.4188@TK2MSFTNGP04.phx.gbl...
>I don't think there's a perf counter for this - someone admin focused might
>correct me.
>
> The random headers: We've seen those, and not everything is understood
> about them. We think they come from spam mail, but it's also possible
> they're coming from non-english mail that may have used foreign character
> sets (like asian character sets) in the headers. Headers like that did
> lead to a problem we issued a hotfix for:
> http://support.microsoft.com/kb/941060
>
> This fix reduces the number of those randomly generated headers that can
> get into the database. The cache referred to in the article is not
> actually the named prop cache, but the fix has an effect on both. With
> that fix applied (or any fix that puts exmime.dll >= 6.5.7653.20) you
> should see a reduction in the number of those type of headers getting
> promoted to the named prop cache. Of course, the fix won't clean up the
> ones that are already there - to do that you'd need to move the mailboxes.
> :)
>
> Steve
>
> "Hunter Coleman" wrote in message
> news:172298CB-BAAD-44C1-A29B-F554E5D50004@microsoft.com...
>> Thank you for those links.
>>
>> Are there any perfmon counters that will show what the named property
>> count is for a store?
>>
>> When I dump the named property table for one store, I see entries such as
>> <NamedPropName>sz: "x-antivirus-konceptio" and
>> <NamedPropName>sz: "x-sender-reputation"
>> that are fairly obvious as to their source. However, I'm also seeing
>> several thousand PS_INTERNET_HEADERS formatted like:
>> <NamedPropName>sz: "m+c@x("tr."xt,"!41`hh16yd"
>> <NamedPropName>sz: "m(#(n.3(@5$0**"`p+c`p*2!4"
>> <NamedPropName>sz: "m+c0p(#(n.3(@5$0**"`p+c`p*2!4"
>> etc.
>>
>> I want to identify the internet messages that have those 'randomly
>> generated' X-headers and look into the system/gateway that is creating
>> them; how can I associate those named property entries with the
>> message(s) that contain the corresponding X-headers?
>>
>> --
>> Hunter
>>
>> "Stephen Griffin" wrote in message
>> news:uPF8i171IHA.4188@TK2MSFTNGP04.phx.gbl...
>>> Here's the EHLO article from three years ago talking about the problem:
>>> http://msexchangeteam.com/archive/2005/08/08/408796.aspx.
>>> Note that acknowledgement that named property exhaustion could be used
>>> as a basis for a Denial of Service attack. We're not trying to hide
>>> this.
>>>
>>> I also found these articles on technet and support.microsoft.com:
>>> Cannot create any more MAPI named properties -
>>> http://technet.microsoft.com/en-us/library/bb218047(EXCHG.80).aspx
>>> Understanding the Impact of Named Property and Replica Identifier Limits
>>> on Exchange Databases -
>>> http://technet.microsoft.com/en-us/library/bb851492(EXCHG.80).aspx
>>> How to configure quota settings for named properties and for replica
>>> identifiers in Exchange Server 2003 and in Exchange Server 2007 -
>>> http://support.microsoft.com/?kbid=820379
>>> Events 9666, 9667, 9668, and 9669 Received When Named Properties or
>>> Replica Identifiers Are Depleted for An Exchange Database -
>>> http://technet.microsoft.com/en-us/library/bb851495.aspx
>>>
>>> MFCMAPI can be downloaded from here:
>>> http://codeplex.com/mfcmapi
>>>
>>> If you're looking to dump the named property table, make sure you're
>>> using an uncached profile (else you'll just be looking at the named prop
>>> table for the OST), then open your mailbox and run "Property Pane"/"Find
>>> All Named Props (SLOW)"
>>>
>>> Steve
>>>
>>> "Hunter Coleman" wrote in message
>>> news:C3EB4069-2F3D-45A5-A991-C2E04157DDFB@microsoft.com...
>>>> Steve-
>>>>
>>>> Yes, we agree that it sucks. And while it likely doesn't change our
>>>> options for fixing the situation, I respectfully disagree that you
>>>> (Microsoft) are "doing our best to help everyone cope with it."
>>>>
>>>> Check through your Exchange 2007 Planning and Architecture
>>>> documentation, and Deployment documentation, and Operations
>>>> documentation. I have yet to find a reference to the named properties
>>>> exhaustion problem. There is a bunch of information about sizing
>>>> databases for I/O, backup and restore, etc, but it all looks like a
>>>> ghost town with tumbleweeds rolling through when it comes to MAPI named
>>>> properties. Where is the information about this on the EHLO blog? Try
>>>> searching for "MapiExceptionNamedPropsQuotaExceeded" at
>>>> support.microsoft.com; not much there, eh? Could you provide some
>>>> details on how to "download a copy of MFCMapi and logon to the
>>>> information store and dump out the Named Props table."?
>>>>
>>>> My impression (opinion only) is that you (Microsoft) have a very
>>>> uncomfortable situation and are hoping that the spotlight doesn't shine
>>>> in this area. I mean, you have a great product in terms of stability,
>>>> scalability, fault tolerance, clustering and such, but the very real
>>>> possibility that any deployment receiving reasonably large amounts of
>>>> internet email is going to start dropping messages. Not exactly a
>>>> strong selling point, is it? But your customers need to be aware of
>>>> this up front during the design stage of their upgrade/rollout if the
>>>> best you can offer for mitigation is to move mailboxes to a new store.
>>>> From an operational and administrative standpoint, that's a remarkably
>>>> poor solution and we need something better.
>>>>
>>>> --
>>>> Hunter
>>>>
>>>> "Stephen Griffin" wrote in message
>>>> news:%23WtICK51IHA.552@TK2MSFTNGP06.phx.gbl...
>>>>> Yes - it sucks. We're all in agreement on this. But it's the way
>>>>> things are, and as long as MAPI property values are 32 bit numbers
>>>>> with only 16 bits available for the unique identifier, that's the way
>>>>> things will continue to be. It's a bad situation and we're doing our
>>>>> best to help everyone cope with it.
>>>>>
>>>>> This scenario (and others) are part of why we don't recommend placing
>>>>> so many users in a single database. You can put them all on the same
>>>>> server - just break up the databases so you have far fewer users per
>>>>> database. The named prop issue will take much longer to appear,
>>>>> migration to a new database if it does appear will be much faster, and
>>>>> backup/restore times will be much shorter. I'm sure someone around
>>>>> here's got a link to our official guidance if you want some numbers to
>>>>> help.
>>>>>
>>>>> Steve
>>>>>
>>>>> "Karl Mitschke" wrote in message
>>>>> news:d66cd4c240ce58caa419e07da5c3@msnews.microsoft.com...
>>>>>> Hello Dgoldman [MSFT],
>>>>>>
>>>>>>> You dont want to increase the limit because down the road all you
>>>>>>> are
>>>>>>> going to do is hit it and have the same problem. You can download a
>>>>>>> copy of MFCMapi and logon to the information store and dump out the
>>>>>>> Named Props table. Essentially the only way to fix it (the correct
>>>>>>> way) is move the users to a new information store and also to stop
>>>>>>> the
>>>>>>> program from creating so many named props.
>>>>>>>
>>>>>>> Dgoldman
>>>>>>> http://blogs.msdn.com/dgoldman
>>>>>>> Download OABInteg
>>>>>>> (http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A
>>>>>>> 2338E73-F521-4071-9B1D-AAF49C346ACD)
>>>>>>
>>>>>> Are you kidding me?
>>>>>>
>>>>>> We get 1 million mail messages a month. If only a very small
>>>>>> percentage of thes have a rogue header like
>>>>>> "x-david-mailscanner-spamcheck" and soon we will have to move all
>>>>>> 14000 users to new databases?
>>>>>>
>>>>>> That's insane!
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
date: Fri, 27 Jun 2008 15:31:24 -0600
author: Hunter Coleman
Re: Named properties quota exceeded?
You will not see a much better fix until Exchange 14. Just to add a little
more information to this as well. If you have Public Folders that are mail
enabled and addressable from the Internet this would lead to this problem..
Mailbox stores tend to get this more since it has to promote all props for
all emails coming in to the mailbox. Custom forms with custom props can also
cause this problem as well.
1. It is important to understand how Named Property gets added to the
information store. Named props do not come from a specific mailbox; they are
common to the mailbox store as a whole. Every time a mailbox in the store
gets a new message with a unique X-Header that's not in the props table, the
store will add it to the props table.
2. Most customers do add additionally 3rd party software such as spam
filtering software which also adds X-Headers and those will also get added
to the named props table.
3. Finally any 3rd party applications which create additional MAPI
properties, those properties are also added to the table.
4. There is no way to clear out this table unless you create a new mailbox
store!! As explained before you can move mailboxes off this store to a new
store and have a new named props table which is not full. Remember that you
will only have 32K of space available for named props.
To answer your question about the performance counter, you can monitor the
number of rows in the named props table by looking at the following
performance counter: MSExchangeIS Mailbox -> Rows in NamedProps Table ->
<Name of store>.
You will have to enabled advanced store counters to expose this value and
you can look at the following link on how to do this:
http://support.microsoft.com/kb/821912
Bare in mind that this performance counter will *only* help you monitor how
close to the edge you are getting as far as availability of rows in the
named props table but it will *NOT* tell you the origin of those props.
Steve gave you the best way to dump out these properties by using MFCMapi so
you can find out which X-Headers or MAPI properties take up the most space.
If you have X-Headers coming from the Internet then you may also want to use
a spam filtering solution to remove these X-Headers before they even reach
the Exchange organization.
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Dgoldman
http://blogs.msdn.com/dgoldman
Download OABInteg
(http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A2338E73-F521-4071-9B1D-AAF49C346ACD)
"Hunter Coleman" wrote in message
news:ukcUq0J2IHA.4140@TK2MSFTNGP03.phx.gbl...
> We're on E2k7 SP1 RU2, so I'm assuming that we're already getting any
> relief that came with the exmime.dll hotfix.
>
> The perfmon counter would be helpful for baselining and long-term growth
> monitoring. As it is, we don't seem to have any way of knowing whether the
> named prop cache is growing in a steady fashion, or random/bursty.
>
> And there's no way to correlate a particular X-header (or any other entry)
> in the cache to the message(s) that contain that header? So if we had an
> ill-designed application generating custom named properties, or an
> external SMTP host causing lots of custom named properties to generate,
> how should we go about identifying that source?
>
> --
> Hunter
>
>
>
> "Stephen Griffin" wrote in message
> news:%23SBRU2I2IHA.4188@TK2MSFTNGP04.phx.gbl...
>>I don't think there's a perf counter for this - someone admin focused
>>might correct me.
>>
>> The random headers: We've seen those, and not everything is understood
>> about them. We think they come from spam mail, but it's also possible
>> they're coming from non-english mail that may have used foreign character
>> sets (like asian character sets) in the headers. Headers like that did
>> lead to a problem we issued a hotfix for:
>> http://support.microsoft.com/kb/941060
>>
>> This fix reduces the number of those randomly generated headers that can
>> get into the database. The cache referred to in the article is not
>> actually the named prop cache, but the fix has an effect on both. With
>> that fix applied (or any fix that puts exmime.dll >= 6.5.7653.20) you
>> should see a reduction in the number of those type of headers getting
>> promoted to the named prop cache. Of course, the fix won't clean up the
>> ones that are already there - to do that you'd need to move the
>> mailboxes. :)
>>
>> Steve
>>
>> "Hunter Coleman" wrote in message
>> news:172298CB-BAAD-44C1-A29B-F554E5D50004@microsoft.com...
>>> Thank you for those links.
>>>
>>> Are there any perfmon counters that will show what the named property
>>> count is for a store?
>>>
>>> When I dump the named property table for one store, I see entries such
>>> as
>>> <NamedPropName>sz: "x-antivirus-konceptio" and
>>> <NamedPropName>sz: "x-sender-reputation"
>>> that are fairly obvious as to their source. However, I'm also seeing
>>> several thousand PS_INTERNET_HEADERS formatted like:
>>> <NamedPropName>sz: "m+c@x("tr."xt,"!41`hh16yd"
>>> <NamedPropName>sz: "m(#(n.3(@5$0**"`p+c`p*2!4"
>>> <NamedPropName>sz: "m+c0p(#(n.3(@5$0**"`p+c`p*2!4"
>>> etc.
>>>
>>> I want to identify the internet messages that have those 'randomly
>>> generated' X-headers and look into the system/gateway that is creating
>>> them; how can I associate those named property entries with the
>>> message(s) that contain the corresponding X-headers?
>>>
>>> --
>>> Hunter
>>>
>>> "Stephen Griffin" wrote in message
>>> news:uPF8i171IHA.4188@TK2MSFTNGP04.phx.gbl...
>>>> Here's the EHLO article from three years ago talking about the problem:
>>>> http://msexchangeteam.com/archive/2005/08/08/408796.aspx.
>>>> Note that acknowledgement that named property exhaustion could be used
>>>> as a basis for a Denial of Service attack. We're not trying to hide
>>>> this.
>>>>
>>>> I also found these articles on technet and support.microsoft.com:
>>>> Cannot create any more MAPI named properties -
>>>> http://technet.microsoft.com/en-us/library/bb218047(EXCHG.80).aspx
>>>> Understanding the Impact of Named Property and Replica Identifier
>>>> Limits on Exchange Databases -
>>>> http://technet.microsoft.com/en-us/library/bb851492(EXCHG.80).aspx
>>>> How to configure quota settings for named properties and for replica
>>>> identifiers in Exchange Server 2003 and in Exchange Server 2007 -
>>>> http://support.microsoft.com/?kbid=820379
>>>> Events 9666, 9667, 9668, and 9669 Received When Named Properties or
>>>> Replica Identifiers Are Depleted for An Exchange Database -
>>>> http://technet.microsoft.com/en-us/library/bb851495.aspx
>>>>
>>>> MFCMAPI can be downloaded from here:
>>>> http://codeplex.com/mfcmapi
>>>>
>>>> If you're looking to dump the named property table, make sure you're
>>>> using an uncached profile (else you'll just be looking at the named
>>>> prop table for the OST), then open your mailbox and run "Property
>>>> Pane"/"Find All Named Props (SLOW)"
>>>>
>>>> Steve
>>>>
>>>> "Hunter Coleman" wrote in message
>>>> news:C3EB4069-2F3D-45A5-A991-C2E04157DDFB@microsoft.com...
>>>>> Steve-
>>>>>
>>>>> Yes, we agree that it sucks. And while it likely doesn't change our
>>>>> options for fixing the situation, I respectfully disagree that you
>>>>> (Microsoft) are "doing our best to help everyone cope with it."
>>>>>
>>>>> Check through your Exchange 2007 Planning and Architecture
>>>>> documentation, and Deployment documentation, and Operations
>>>>> documentation. I have yet to find a reference to the named properties
>>>>> exhaustion problem. There is a bunch of information about sizing
>>>>> databases for I/O, backup and restore, etc, but it all looks like a
>>>>> ghost town with tumbleweeds rolling through when it comes to MAPI
>>>>> named properties. Where is the information about this on the EHLO
>>>>> blog? Try searching for "MapiExceptionNamedPropsQuotaExceeded" at
>>>>> support.microsoft.com; not much there, eh? Could you provide some
>>>>> details on how to "download a copy of MFCMapi and logon to the
>>>>> information store and dump out the Named Props table."?
>>>>>
>>>>> My impression (opinion only) is that you (Microsoft) have a very
>>>>> uncomfortable situation and are hoping that the spotlight doesn't
>>>>> shine in this area. I mean, you have a great product in terms of
>>>>> stability, scalability, fault tolerance, clustering and such, but the
>>>>> very real possibility that any deployment receiving reasonably large
>>>>> amounts of internet email is going to start dropping messages. Not
>>>>> exactly a strong selling point, is it? But your customers need to be
>>>>> aware of this up front during the design stage of their
>>>>> upgrade/rollout if the best you can offer for mitigation is to move
>>>>> mailboxes to a new store. From an operational and administrative
>>>>> standpoint, that's a remarkably poor solution and we need something
>>>>> better.
>>>>>
>>>>> --
>>>>> Hunter
>>>>>
>>>>> "Stephen Griffin" wrote in message
>>>>> news:%23WtICK51IHA.552@TK2MSFTNGP06.phx.gbl...
>>>>>> Yes - it sucks. We're all in agreement on this. But it's the way
>>>>>> things are, and as long as MAPI property values are 32 bit numbers
>>>>>> with only 16 bits available for the unique identifier, that's the way
>>>>>> things will continue to be. It's a bad situation and we're doing our
>>>>>> best to help everyone cope with it.
>>>>>>
>>>>>> This scenario (and others) are part of why we don't recommend placing
>>>>>> so many users in a single database. You can put them all on the same
>>>>>> server - just break up the databases so you have far fewer users per
>>>>>> database. The named prop issue will take much longer to appear,
>>>>>> migration to a new database if it does appear will be much faster,
>>>>>> and backup/restore times will be much shorter. I'm sure someone
>>>>>> around here's got a link to our official guidance if you want some
>>>>>> numbers to help.
>>>>>>
>>>>>> Steve
>>>>>>
>>>>>> "Karl Mitschke" wrote in message
>>>>>> news:d66cd4c240ce58caa419e07da5c3@msnews.microsoft.com...
>>>>>>> Hello Dgoldman [MSFT],
>>>>>>>
>>>>>>>> You dont want to increase the limit because down the road all you
>>>>>>>> are
>>>>>>>> going to do is hit it and have the same problem. You can download a
>>>>>>>> copy of MFCMapi and logon to the information store and dump out the
>>>>>>>> Named Props table. Essentially the only way to fix it (the correct
>>>>>>>> way) is move the users to a new information store and also to stop
>>>>>>>> the
>>>>>>>> program from creating so many named props.
>>>>>>>>
>>>>>>>> Dgoldman
>>>>>>>> http://blogs.msdn.com/dgoldman
>>>>>>>> Download OABInteg
>>>>>>>> (http://gotdotnet.com/Community/UserSamples/Download.aspx?SampleGuid=A
>>>>>>>> 2338E73-F521-4071-9B1D-AAF49C346ACD)
>>>>>>>
>>>>>>> Are you kidding me?
>>>>>>>
>>>>>>> We get 1 million mail messages a month. If only a very small
>>>>>>> percentage of thes have a rogue header like
>>>>>>> "x-david-mailscanner-spamcheck" and soon we will have to move all
>>>>>>> 14000 users to new databases?
>>>>>>>
>>>>>>> That's insane!
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
date: Mon, 30 Jun 2008 13:59:12 -0400
author: Dgoldman [MSFT]
|
|
