|
|
|
date: Thu, 1 Nov 2007 19:36:00 -0700,
group: microsoft.public.platformsdk.internet.server.isapi-dev
back
WSAENOTSOCK errors when IIS 5.1 is under heavy load
Hello.
We have an engine which does some specialized work. We've written adapters
for that engine that allow us to plug it into several different web server
platforms--Apache (1.3 through 2.2), Java (1.4+), .NET (1.1+). Last week I
wrote an ISAPI adapter for it. When the server is under load I see errors on
IIS that I haven't seen with the other web servers.
For my testing I'm using IIS 5.1 under Windows XP Pro SP2 on a 2GHz laptop.
I've used the technique mentioned on
http://www.codinghorror.com/blog/archives/000329.html to raise the IIS 5.1
concurrent connection limit from 10 to 40. I see these errors when I throw,
say, 20 to 25 concurrent requests at the server. (Our engine runs from memory
with no disk latency and a typical request takes less than ~20-30ms.)
What I see is a number of WSAENOTSOCK (10038) errors in the log. The errors
are generated by the first call (HSE_REQ_SEND_RESPONSE_HEADER_EX) that tries
to write data back to the client. If I add the ConnId to the log the pattern
I seem to see is that a request using that connection ID completes
successfully, and then shortly thereafter the two or so subsequent requests
with the same ConnID fail. That doesn't happen on every request; overall
roughly 10% of the requests fail. Of course, it's possible that the order I'm
seeing in the log doesn't *quite* match the order in which things are
happening, so all of that may be misleading.
I have not seen anyone reporting a WSAENOTSOCK error in this forum, so I
presume I've made a mistake somewhere. I've seen the messages about watching
error paths and the importance of the return values, but I've read through my
code many times now and I just haven't been able to spot the problem. No
doubt more experienced eyes will pick it up. Or perhaps my error is putting
IIS 5.1 through this kind of load on a Win XP Pro box.
Thanks for any help. Of course!
DWORD WINAPI HttpExtensionProc(
IN EXTENSION_CONTROL_BLOCK *pECB
)
{
DWORD result = HSE_STATUS_SUCCESS;
DWORD zz_result;
// <... get headers ...>
if( (zz_result = zz_acquire( &request, &response )) == ZZ_OK )
{
char status[ STATUS_BUFFER ];
char header[ HEADER_BUFFER ];
HSE_SEND_HEADER_EX_INFO HeaderExInfo;
<... fill status buffer ...>
<... fill header buffer ...>
if( result == HSE_STATUS_SUCCESS )
{
HeaderExInfo.pszStatus = status;
HeaderExInfo.pszHeader = header;
HeaderExInfo.cchStatus = strlen(HeaderExInfo.pszStatus);
HeaderExInfo.cchHeader = strlen(HeaderExInfo.pszHeader);
HeaderExInfo.fKeepConn = FALSE;
if( !pECB->ServerSupportFunction(
pECB->ConnID,
HSE_REQ_SEND_RESPONSE_HEADER_EX,
&HeaderExInfo,
NULL,
NULL) )
{
_log_error( pECB, "sending headers failed.",
GetLastError() );
result = HSE_STATUS_ERROR;
}
}
if( result == HSE_STATUS_SUCCESS )
{
if( (response.content_length > 0) &&
(response.content != NULL) )
{
UINT content_length = response.content_length;
PSZ content = _strdup( response.content );
if( result == HSE_STATUS_SUCCESS )
{
if( !content )
{
_log_error( pECB, "creating content failed.",
GetLastError() );
result = HSE_STATUS_ERROR;
}
}
if( result == HSE_STATUS_SUCCESS )
{
if( !pECB->ServerSupportFunction(
pECB->ConnID,
HSE_REQ_IO_COMPLETION,
HttpIOCompletionProc,
NULL,
(LPDWORD) content) )
{
_log_error( pECB, "creating i/o completion
failed.", GetLastError() );
free( content );
result = HSE_STATUS_ERROR;
}
}
if( result == HSE_STATUS_SUCCESS )
{
if( !pECB->WriteClient(
pECB->ConnID,
content,
&content_length,
HSE_IO_ASYNC | HSE_IO_NODELAY) )
{
_log_error( pECB, "sending content failed.",
GetLastError() );
free( content );
result = HSE_STATUS_ERROR;
}
else
{
result = HSE_STATUS_PENDING;
}
}
}
else
{
if( result == HSE_STATUS_SUCCESS )
{
if( ((response.content_length > 0) &&
((response.content == NULL) ||
(*response.content == '\0'))) ||
((response.content_length == 0) &&
((response.content != NULL) &&
(*response.content != '\0'))))
{
_log_error( pECB, "content/content-length
agreement failed.", GetLastError() );
result = HSE_STATUS_ERROR;
}
}
}
}
zz_release( &request, &response );
}
else
{
_log_error( pECB, "zz_acquire() failed.", zz_result );
result = HSE_STATUS_ERROR;
}
return result;
}
VOID WINAPI HttpIOCompletionProc(
IN EXTENSION_CONTROL_BLOCK *pECB,
IN PVOID pContext,
IN DWORD cbIO,
IN DWORD dwError
)
{
PSZ content = (PSZ) pContext;
free( content );
pECB->ServerSupportFunction(
pECB->ConnID,
HSE_REQ_DONE_WITH_SESSION,
NULL,
NULL,
NULL);
}
date: Thu, 1 Nov 2007 19:36:00 -0700
author: Michael Lenaghan Michael
Re: WSAENOTSOCK errors when IIS 5.1 is under heavy load
On Nov 1, 6:36 pm, Michael Lenaghan <Michael
Lenag...@discussions.microsoft.com> wrote:
> Hello.
>
> We have an engine which does some specialized work. We've written adapters
> for that engine that allow us to plug it into several different web server
> platforms--Apache (1.3 through 2.2), Java (1.4+), .NET (1.1+). Last week I
> wrote an ISAPI adapter for it. When the server is under load I see errors on
> IIS that I haven't seen with the other web servers.
>
> For my testing I'm using IIS 5.1 under Windows XP Pro SP2 on a 2GHz laptop.
> I've used the technique mentioned onhttp://www.codinghorror.com/blog/archives/000329.htmlto raise the IIS 5.1
> concurrent connection limit from 10 to 40. I see these errors when I throw,
> say, 20 to 25 concurrent requests at the server. (Our engine runs from memory
> with no disk latency and a typical request takes less than ~20-30ms.)
>
> What I see is a number of WSAENOTSOCK (10038) errors in the log. The errors
> are generated by the first call (HSE_REQ_SEND_RESPONSE_HEADER_EX) that tries
> to write data back to the client. If I add the ConnId to the log the pattern
> I seem to see is that a request using that connection ID completes
> successfully, and then shortly thereafter the two or so subsequent requests
> with the same ConnID fail. That doesn't happen on every request; overall
> roughly 10% of the requests fail. Of course, it's possible that the order I'm
> seeing in the log doesn't *quite* match the order in which things are
> happening, so all of that may be misleading.
>
> I have not seen anyone reporting a WSAENOTSOCK error in this forum, so I
> presume I've made a mistake somewhere. I've seen the messages about watching
> error paths and the importance of the return values, but I've read through my
> code many times now and I just haven't been able to spot the problem. No
> doubt more experienced eyes will pick it up. Or perhaps my error is putting
> IIS 5.1 through this kind of load on a Win XP Pro box.
>
> Thanks for any help. Of course!
>
> DWORD WINAPI HttpExtensionProc(
> IN EXTENSION_CONTROL_BLOCK *pECB
> )
> {
> DWORD result = HSE_STATUS_SUCCESS;
> DWORD zz_result;
>
> // <... get headers ...>
>
> if( (zz_result = zz_acquire( &request, &response )) == ZZ_OK )
> {
> char status[ STATUS_BUFFER ];
> char header[ HEADER_BUFFER ];
>
> HSE_SEND_HEADER_EX_INFO HeaderExInfo;
>
> <... fill status buffer ...>
>
> <... fill header buffer ...>
>
> if( result == HSE_STATUS_SUCCESS )
> {
> HeaderExInfo.pszStatus = status;
> HeaderExInfo.pszHeader = header;
> HeaderExInfo.cchStatus = strlen(HeaderExInfo.pszStatus);
> HeaderExInfo.cchHeader = strlen(HeaderExInfo.pszHeader);
> HeaderExInfo.fKeepConn = FALSE;
>
> if( !pECB->ServerSupportFunction(
> pECB->ConnID,
> HSE_REQ_SEND_RESPONSE_HEADER_EX,
> &HeaderExInfo,
> NULL,
> NULL) )
> {
> _log_error( pECB, "sending headers failed.",
> GetLastError() );
> result = HSE_STATUS_ERROR;
> }
> }
>
> if( result == HSE_STATUS_SUCCESS )
> {
> if( (response.content_length > 0) &&
> (response.content != NULL) )
> {
> UINT content_length = response.content_length;
> PSZ content = _strdup( response.content );
>
> if( result == HSE_STATUS_SUCCESS )
> {
> if( !content )
> {
> _log_error( pECB, "creating content failed.",
> GetLastError() );
> result = HSE_STATUS_ERROR;
> }
> }
>
> if( result == HSE_STATUS_SUCCESS )
> {
> if( !pECB->ServerSupportFunction(
> pECB->ConnID,
> HSE_REQ_IO_COMPLETION,
> HttpIOCompletionProc,
> NULL,
> (LPDWORD) content) )
> {
> _log_error( pECB, "creating i/o completion
> failed.", GetLastError() );
> free( content );
> result = HSE_STATUS_ERROR;
> }
> }
>
> if( result == HSE_STATUS_SUCCESS )
> {
> if( !pECB->WriteClient(
> pECB->ConnID,
> content,
> &content_length,
> HSE_IO_ASYNC | HSE_IO_NODELAY) )
> {
> _log_error( pECB, "sending content failed.",
> GetLastError() );
> free( content );
> result = HSE_STATUS_ERROR;
> }
> else
> {
> result = HSE_STATUS_PENDING;
> }
> }
>
> }
> else
> {
> if( result == HSE_STATUS_SUCCESS )
> {
> if( ((response.content_length > 0) &&
> ((response.content == NULL) ||
> (*response.content == '\0'))) ||
> ((response.content_length == 0) &&
> ((response.content != NULL) &&
> (*response.content != '\0'))))
> {
> _log_error( pECB, "content/content-length
> agreement failed.", GetLastError() );
> result = HSE_STATUS_ERROR;
> }
> }
> }
> }
>
> zz_release( &request, &response );
> }
> else
> {
> _log_error( pECB, "zz_acquire() failed.", zz_result );
> result = HSE_STATUS_ERROR;
> }
>
> return result;
>
> }
>
> VOID WINAPI HttpIOCompletionProc(
> IN EXTENSION_CONTROL_BLOCK *pECB,
> IN PVOID pContext,
> IN DWORD cbIO,
> IN DWORD dwError
> )
> {
> PSZ content = (PSZ) pContext;
> free( content );
>
> pECB->ServerSupportFunction(
> pECB->ConnID,
> HSE_REQ_DONE_WITH_SESSION,
> NULL,
> NULL,
> NULL);
>
>
>
> }- Hide quoted text -
>
> - Show quoted text -
Hmm... I don't see anything immediately wrong with what you have
shown. The logic with your code is fine (you're amongst the tiny
minority that actually correctly retrieve and account for error
conditions when making ISAPI function calls), though it is cumbersome
to read due to multiple nested if block tests all the time. You may
want to consider the following code flow using GOTO (yes, there is
good coding style with GOTO).
FYI, this is how the IIS6 codebase is written.
Some things to note:
1. Memory allocation and deallocation logic simple and clear in one
location. There is no such thing as forgetting to deallocate on error
code paths with this approach, while your approach forces every error
code path to duplicate identical code to deal with resource
deallocation -- and if you ever move code ordering around, you must
also add/remove the deallocation logic, which can be forgotten -- at
which point you either start leaking memory or crashing on touching
unallocated memory on error condition -- a common error.
2. At all times, the logic is clear. No nesting. And no need to
remember the condition from two parent blocks above and hundreds of
lines of code away.
Basically, you just follow the pattern of:
function
{
<initialize all variables to default values>
<do something, including allocate memory>
<if unsuccessful, goto finished>
<do something else>
<if unsuccessful, goto finished>
...
<everything is successful, set appropriate state>
Finished:
<if variable is not default value and it should NOT be allocated for
async IO, free it>
<if came here because of error (GetLastError() != NO_ERROR), bubble
it back to parent>
return;
}
And you will find that memory is magically deallocated correctly on
error, remain valid for async IO, and deallocated for success.
I recommend against load-testing against a client SKU like XP Pro.
Even if you have done the couple of hacks. Not only was XP Pro never
really load-tested (we *know* it does not handle the same load as IIS6
even if we keep the test platform the same), it runs a different
server core than IIS6 (and I'm assuming you want to target IIS6 server
for your product).
The sort of error pattern you describe sounds like the Windows Async
IO pattern not being correctly followed -- which is not what I see in
the code you have shown. I would be very interested to hear of your
results against IIS6 on Windows Server 2003.
The Async IO pattern in IIS is basically:
1. register the async IO completion function with the memory context
preserved during the async IO
2. make the async IO call. If success, immediately return
HSE_STATUS_PENDING. If failure, cleanup and do not return
HSE_STATUS_PENDING
3. after async IO call has been made, make sure to NOT touch the
memory used by async IO UNTIL inside the async IO completion function.
4. inside IO completion function, clean up the memory context
preserved in step #1 and call HSE_REQ_DONE_WITH_SESSION using the same
ECB->ConnId as #1 (which is basically the ECB passed into the IO
completion function).
I don't see you deviating from this pattern. So, I'm a little baffled
by your errors.
DWORD WINAPI HttpExtensionProc(
IN EXTENSION_CONTROL_BLOCK *pECB
)
{
DWORD result = HSE_STATUS_SUCCESS;
DWORD zz_result;
// <... get headers ...>
SetLastError( NO_ERROR );
zz_result = zz_acquire( &request, &response );
if ( ZZ_OK != zz_result )
{
_log_error( pECB, "zz_acquire() failed.", zz_result );
goto Finished;
}
char status[ STATUS_BUFFER ];
char header[ HEADER_BUFFER ];
HSE_SEND_HEADER_EX_INFO HeaderExInfo;
<... fill status buffer ...>
<... fill header buffer ...>
HeaderExInfo.pszStatus = status;
HeaderExInfo.pszHeader = header;
HeaderExInfo.cchStatus = strlen(HeaderExInfo.pszStatus);
HeaderExInfo.cchHeader = strlen(HeaderExInfo.pszHeader);
HeaderExInfo.fKeepConn = FALSE;
if( !pECB->ServerSupportFunction(
pECB->ConnID,
HSE_REQ_SEND_RESPONSE_HEADER_EX,
&HeaderExInfo,
NULL,
NULL) )
{
_log_error( pECB, "sending headers failed.", GetLastError() );
goto Finished;
}
if( ((response.content_length > 0) &&
((response.content == NULL) ||
(*response.content == '\0'))) ||
((response.content_length == 0) &&
((response.content != NULL) &&
(*response.content != '\0'))))
{
_log_error( pECB, "content/content-length agreement failed.",
GetLastError() );
goto Finished;
}
if( (response.content_length <= 0) ||
(response.content == NULL) )
{
SetLastError( ERROR_INVALID_PARAMETER );
goto Finished;
}
UINT content_length = response.content_length;
PSZ content = _strdup( response.content );
if( !content )
{
_log_error( pECB, "creating content failed.",
GetLastError() );
goto Finished;
}
if( !pECB->ServerSupportFunction(
pECB->ConnID,
HSE_REQ_IO_COMPLETION,
HttpIOCompletionProc,
NULL,
(LPDWORD) content) )
{
_log_error( pECB, "creating i/o completion failed.",
GetLastError() );
goto Finished;
}
if( !pECB->WriteClient(
pECB->ConnID,
content,
&content_length,
HSE_IO_ASYNC | HSE_IO_NODELAY) )
{
_log_error( pECB, "sending content failed.", GetLastError() );
goto Finished;
}
else
{
result = HSE_STATUS_PENDING;
}
zz_release( &request, &response );
Finished:
if ( NO_ERROR != GetLastError() )
{
result = HSE_STATUS_ERROR;
}
if ( NULL != content &&
HSE_STATUS_PENDING != result )
{
free( content );
}
return result;
}
VOID WINAPI HttpIOCompletionProc(
IN EXTENSION_CONTROL_BLOCK *pECB,
IN PVOID pContext,
IN DWORD cbIO,
IN DWORD dwError
)
{
DWORD dwStatus = HSE_STATUS_SUCCESS;
PSZ content = (PSZ) pContext;
if ( NULL != content )
{
free( content );
}
pECB->ServerSupportFunction(
pECB->ConnID,
HSE_REQ_DONE_WITH_SESSION,
&dwStatus,
NULL,
NULL );
// ...
}
//David
http://w3-4u.blogspot.com
http://blogs.msdn.com/David.Wang
//
date: Sun, 04 Nov 2007 10:30:31 -0000
author: David Wang
Re: WSAENOTSOCK errors when IIS 5.1 is under heavy load
Thanks, David, for your reply, and thanks for going out of your way to
recommend (and illustrate!) a different coding style.
In my code I don't call WriteClient if there isn't any content. I was
wondering if in that case I should specifically clear the IO completion
callback since it would have been set by a prior request on that ConnID. I
don't think that's necessary, but I was wondering. It's easy to do so I might
give it a try.
I noticed that in your code you set the optional return code param in
HSE_REQ_DONE_WITH_SESSION. I have a big long comment in my code about that.
(I removed that and other comments in my post.) The documentation on the
topic is a little, uh, sparse. My first version of the code set the return to
SUCCESS if the callback param dwError indicated success and ERROR otherwise.
Then I noticed that the ATL Server code set an error return if the status
code in the response was >= 400, so I added that to my code. After thinking
about it, though, I ripped all of that out.
My thinking was this: The param is optional, so clearly IIS knows what it
intends to do if I don't give it anything. Furthermore, IIS knows whether or
not the write I just asked it to perform was successful or not, and could in
theory use that info to set a response code of SUCCESS or ERROR as
appropriate. The reason IIS can't *always* fill in the response is that in
the callback I might perform some new operation that might fail, and if it
does I have to let IIS know that it should abort the whole thing. In all of
that the ATL Server code is something of an anamoly; why do they set it based
on the request's status code? I decided that either they made a mistake or
they were using that approach to achieve some side-effect, like error
logging. Anyway, after all of that I decided the best thing to do was to let
IIS decide what the correct value should be.
I actually went back to the HSE_REQ_DONE_WITH_SESSION page on MSDN to send
feedback about the documentation, but I didn't see a feedback link. (There
was a general comment link, but I wasn't convinced that anything I submitted
there would make it back to the IIS documentation team.) If you think the
comment link stuff will indeed make it back I'd be happy to write up this and
a few other things.
By the way, I will indeed test on IIS 6 & IIS 7; I just didn't want to
dismiss what I was seeing on 5.1. If I see funny behaviour on those platforms
I'll let you know. Well, I probably won't have a choice. :-)
Thanks again,
Michael.
date: Sun, 4 Nov 2007 10:06:01 -0800
author: Michael Lenaghan
Re: WSAENOTSOCK errors when IIS 5.1 is under heavy load
On Nov 4, 10:06 am, Michael Lenaghan
wrote:
> Thanks, David, for your reply, and thanks for going out of your way to
> recommend (and illustrate!) a different coding style.
>
> In my code I don't call WriteClient if there isn't any content. I was
> wondering if in that case I should specifically clear the IO completion
> callback since it would have been set by a prior request on that ConnID. I
> don't think that's necessary, but I was wondering. It's easy to do so I might
> give it a try.
>
> I noticed that in your code you set the optional return code param in
> HSE_REQ_DONE_WITH_SESSION. I have a big long comment in my code about that.
> (I removed that and other comments in my post.) The documentation on the
> topic is a little, uh, sparse. My first version of the code set the return to
> SUCCESS if the callback param dwError indicated success and ERROR otherwise.
> Then I noticed that the ATL Server code set an error return if the status
> code in the response was >= 400, so I added that to my code. After thinking
> about it, though, I ripped all of that out.
>
> My thinking was this: The param is optional, so clearly IIS knows what it
> intends to do if I don't give it anything. Furthermore, IIS knows whether or
> not the write I just asked it to perform was successful or not, and could in
> theory use that info to set a response code of SUCCESS or ERROR as
> appropriate. The reason IIS can't *always* fill in the response is that in
> the callback I might perform some new operation that might fail, and if it
> does I have to let IIS know that it should abort the whole thing. In all of
> that the ATL Server code is something of an anamoly; why do they set it based
> on the request's status code? I decided that either they made a mistake or
> they were using that approach to achieve some side-effect, like error
> logging. Anyway, after all of that I decided the best thing to do was to let
> IIS decide what the correct value should be.
>
> I actually went back to the HSE_REQ_DONE_WITH_SESSION page on MSDN to send
> feedback about the documentation, but I didn't see a feedback link. (There
> was a general comment link, but I wasn't convinced that anything I submitted
> there would make it back to the IIS documentation team.) If you think the
> comment link stuff will indeed make it back I'd be happy to write up this and
> a few other things.
>
> By the way, I will indeed test on IIS 6 & IIS 7; I just didn't want to
> dismiss what I was seeing on 5.1. If I see funny behaviour on those platforms
> I'll let you know. Well, I probably won't have a choice. :-)
>
> Thanks again,
> Michael.
I agree with you on HSE_REQ_DONE_WITH_SESSION. The unfortunate thing
is that there is a bit of a legacy behavior when it comes to
connection keep-alive, so the confusion continues. I'm just making the
decision unambiguous for IIS. You must set it to
HSE_STATUS_SUCCESS_KEEP_CONN if you wish to have keepalive *if* client
requests it *and* server allows it. Confused already? ;-)
Actually, you got me thinking -- how exactly are you filling in the
response headers? In particular, where is the Content-Length,
Transfer: chunked, or MultiPart header necessary for WriteClient to be
properly interpreted by the client in the successful case?
This can affect how your load client separates proper request/response
pairing and cause problems under load when the wrong request/response
pairs are interpreted by the client.
For example, this is how your current code works:
- Client sends request to server
- Server sends response with no indication of size
- Client has to wait for server to close connection before finishing
the request
- Client makes another request over the same TCP channel (since server
did not close it)
- Server thinks you are HTTP Pipelining when it is not expecting it
Depending on how your client races relative to server-side processing,
you may cause the server to think it is in HTTP Pipelining situation
(or not). Of course, if your client uses new connections for every
request, it may prolong the possibility of this confusion. I don't
remember if IIS 5.x core dealt with HTTP Pipelining properly. It was
reworked in IIS6.
Let's first make sure the ISAPI returns a proper HTTP response no
matter what happens.
//David
http://w3-4u.blogspot.com
http://blogs.msdn.com/David.Wang
//
date: Sun, 04 Nov 2007 17:04:28 -0800
author: David Wang
Re: WSAENOTSOCK errors when IIS 5.1 is under heavy load
Hello, David.
> I agree with you on HSE_REQ_DONE_WITH_SESSION. The unfortunate thing
> is that there is a bit of a legacy behavior when it comes to
> connection keep-alive, so the confusion continues. I'm just making the
> decision unambiguous for IIS. You must set it to
> HSE_STATUS_SUCCESS_KEEP_CONN if you wish to have keepalive *if* client
> requests it *and* server allows it. Confused already? ;-)
I think I understand the various cases where you'd *have* to return a status
code and why; my question was really about the "default" case.
For example, in your code you always set status to SUCCESS. But what if the
dwError parameter to the callback indicates that there was an error during
the last async write? Shouldn't we return ERROR in that case? But of course,
in that same case IIS is already aware of the error... That was the line of
thinking that led me to believe that for my relatively simple usage I should
leave it up to IIS.
Having said that, I did explore this issue before my first post. I set the
status code in the callback, and I also logged dwError and discovered (at
least I think I did) that there was never an error during write. But I was
still getting WSAENOTSOCK under load. I re-ran that test (minus logging)
after seeing your reply, and confirmed once again that setting the status
code doesn't change the result.
By the way, while we're on the topic here's the funny code I saw at line
1170 of Vc7\atlmfc\include\atlisapi.h:
m_pECB->dwHttpStatusCode = dwHttpStatusCode;
DWORD dwStatusCode = (dwHttpStatusCode >= 400) ? HSE_STATUS_ERROR :
HSE_STATUS_SUCCESS;
return m_pECB->ServerSupportFunction(m_pECB->ConnID,
HSE_REQ_DONE_WITH_SESSION, &dwStatusCode, NULL, NULL);
I believe that's wrong, but maybe they know something I don't.
> Actually, you got me thinking -- how exactly are you filling in the
> response headers? In particular, where is the Content-Length,
> Transfer: chunked, or MultiPart header necessary for WriteClient to be
> properly interpreted by the client in the successful case?
>
> This can affect how your load client separates proper request/response
> pairing and cause problems under load when the wrong request/response
> pairs are interpreted by the client.
>
> For example, this is how your current code works:
> - Client sends request to server
> - Server sends response with no indication of size
> - Client has to wait for server to close connection before finishing
> the request
> - Client makes another request over the same TCP channel (since server
> did not close it)
> - Server thinks you are HTTP Pipelining when it is not expecting it
>
> Depending on how your client races relative to server-side processing,
> you may cause the server to think it is in HTTP Pipelining situation
> (or not). Of course, if your client uses new connections for every
> request, it may prolong the possibility of this confusion. I don't
> remember if IIS 5.x core dealt with HTTP Pipelining properly. It was
> reworked in IIS6.
After reading this I went back to re-check everything. We always set and
send content-length, even if it's zero. That fulfills the HTTP requirement
for our usage pattern. Furthermore, we always set HeaderExInfo.fKeepConn to
FALSE. That causes IIS to send a "Connection: close" header, and that in turn
causes the client to close the connection after receiving the response.***
As a general reminder, I have the same engine running with, for example,
Apache 1.3, 2.0, and 2.2 adapters. Under the same (and even higher) load and
on the same copy of Win XP Pro I never see the problems I'm getting with IIS
5.1, and I'm using the same stress testing tools for all tests. I've made an
effort to send the same response headers from all web servers, and I've used
tools like Curl (and the equivalent from the IIS 6 resource toolkit) to check
them. They do indeed look the same, though it is of course possible I'm
missing some subtle detail.
At this point, though, my best guess is that even with the hack IIS 5.1 on
XP Pro can't *really* handle more than 10 concurrent connections. (The
proportion of errors rises with the number of concurrent connections.) That's
fair, the docs say you only get 10. I was just hoping that the hack would
allow me to push harder before moving it off to the server.
I should be able to run the server tests over the next two or three weeks.
I'm setting up a new server for those tests, and I have some other things
that are higher on the priority list.
Unless you have some other idea, I think my testing will come to a
standstill until I see the server test results.
(I might be able to borrow someone else's server for a quick test.)
Thanks, David.
Michael.
***Just to explain: our engine runs as a web-based service. Our protocol was
specifically designed to allow the client to get everything it needs in one
transaction. That assumption is why we close the connection after each
request; given our design it helps us conserve server resources.
date: Tue, 6 Nov 2007 12:17:01 -0800
author: Michael Lenaghan
|
|
