I've written an ISAPI extension to handle upload of large ( ~70MB ) files using HSE_REQ_ASYNC_READ_CLIENT. I'd like to add a private thread pool to the extension but am finding the code is hanging and the upload not completing on occasion so I'm wondering if I have the basic approach right. It seems to hang during the action described in below as in ThreadPool1. Is this the correct sequence? Any comments on a better design welcome or if this is over engineered may be time to simplify. Jim Kane ISAPI Thread calls HttpExtensionProc: On error WriteClient (plus header) then Return HSE_STATUS_ERROR or HSE_STATUS_SUCCESS Queue work item to my ThreadPool1 Note: HSE_REQ_IO_COMPLETION has NOT been called yet. HSE_REQ_DONE_WITH_SESSION is never used in this block Return HSE_STATUS_PENDING ThreadPool1 thread: - creates a context class - reads the registry and creates a disk file - decrypts a little of the available uploaded data (if any) and verifys it contains a signature. - writes data already available to file - Requests HSE_REQ_IO_COMPLETION via ServerSupportFunction - Request HSE_REQ_ASYNC_READ_CLIENT - On Error WriteClient + Request HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_ERROR or _SUCCESS even if HSE_REQ_IO_COMPLETION has not been requested yet. ( hangs with client not receiving full response when this happens sometimes ) - Thread returns to its pool. ISAPI Thread calls Callback Proc - Queue work item ( including buffer contents of cbIO bytes ) to ThreadPool2 thread - On error close the file, destroy context, write client + Request HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_ERROR - If last block was queued WriteClient + Request HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_Success ( and pray the last segment makes it to disk since we just reported success ) Return from callback ThreadPool2 thread: - Saves buffer to disk after waiting for prior I/O to finish - Closes the file and destroys the context class if this is the last disk write Thread returns to its pool.
I'll take a step back, and ask you what is the reason you are adding a thread pool? Is it because the decrypt/verify phase is CPU intensive or waits for some external application. Or is it because the current solution is too slow for your needs. Also, from your pseudo-code you appear to have 2 thread pools. Personally I'd try and use just the 1 for both parts, or avoid them all together if possible. Ian "jim kane" wrote in message news:eFrr5elOGHA.1028@TK2MSFTNGP11.phx.gbl... > I've written an ISAPI extension to handle upload of large ( ~70MB ) files > using HSE_REQ_ASYNC_READ_CLIENT. I'd like to add a private thread pool to > the extension but am finding the code is hanging and the upload not > completing on occasion so I'm wondering if I have the basic approach > right. It seems to hang during the action described in below as in > ThreadPool1. > Is this the correct sequence? Any comments on a better design welcome or > if this is over engineered may be time to simplify. > Jim Kane > > ISAPI Thread calls HttpExtensionProc: > On error WriteClient (plus header) then Return HSE_STATUS_ERROR or > HSE_STATUS_SUCCESS > Queue work item to my ThreadPool1 > Note: HSE_REQ_IO_COMPLETION has NOT been called yet. > HSE_REQ_DONE_WITH_SESSION is never used in this block > Return HSE_STATUS_PENDING > > > ThreadPool1 thread: > - creates a context class > - reads the registry and creates a disk file > - decrypts a little of the available uploaded data (if any) and verifys it > contains a signature. > - writes data already available to file > - Requests HSE_REQ_IO_COMPLETION via ServerSupportFunction > - Request HSE_REQ_ASYNC_READ_CLIENT > - On Error WriteClient + Request HSE_REQ_DONE_WITH_SESSION with > HSE_STATUS_ERROR or _SUCCESS > even if HSE_REQ_IO_COMPLETION has not been requested yet. ( hangs with > client not receiving full response when this happens sometimes ) > - Thread returns to its pool. > > ISAPI Thread calls Callback Proc > - Queue work item ( including buffer contents of cbIO bytes ) to > ThreadPool2 thread > - On error close the file, destroy context, write client + Request > HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_ERROR > - If last block was queued WriteClient + Request > HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_Success > ( and pray the last segment makes it to disk since we just reported > success ) > Return from callback > > ThreadPool2 thread: > - Saves buffer to disk after waiting for prior I/O to finish > - Closes the file and destroys the context class if this is the last disk > write > Thread returns to its pool. >
yes, the size of the file and amount of processing is a concern. Everything I read seems to say IIS wants it's threads back quickly. After coding the processing involved I decided the processing for the area of the flow where I indicated a 2nd thread pool was not as slow as I thought it might be so I dropped that. The reason for 2 thread pools is in the two areas of the programs the procedure run by the thread is different although a moot point now. Jim Kane "Ian" <Ixpah@newsgroup.nospam> wrote in message news:eiFQJp4OGHA.2604@TK2MSFTNGP09.phx.gbl... > I'll take a step back, and ask you what is the reason you are adding a > thread pool? Is it because the decrypt/verify phase is CPU intensive or > waits for some external application. Or is it because the current solution > is too slow for your needs. > > Also, from your pseudo-code you appear to have 2 thread pools. Personally > I'd try and use just the 1 for both parts, or avoid them all together if > possible. > > Ian > > > > "jim kane" wrote in message > news:eFrr5elOGHA.1028@TK2MSFTNGP11.phx.gbl... >> I've written an ISAPI extension to handle upload of large ( ~70MB ) files >> using HSE_REQ_ASYNC_READ_CLIENT. I'd like to add a private thread pool >> to the extension but am finding the code is hanging and the upload not >> completing on occasion so I'm wondering if I have the basic approach >> right. It seems to hang during the action described in below as in >> ThreadPool1. >> Is this the correct sequence? Any comments on a better design welcome or >> if this is over engineered may be time to simplify. >> Jim Kane >> >> ISAPI Thread calls HttpExtensionProc: >> On error WriteClient (plus header) then Return HSE_STATUS_ERROR or >> HSE_STATUS_SUCCESS >> Queue work item to my ThreadPool1 >> Note: HSE_REQ_IO_COMPLETION has NOT been called yet. >> HSE_REQ_DONE_WITH_SESSION is never used in this block >> Return HSE_STATUS_PENDING >> >> >> ThreadPool1 thread: >> - creates a context class >> - reads the registry and creates a disk file >> - decrypts a little of the available uploaded data (if any) and verifys >> it contains a signature. >> - writes data already available to file >> - Requests HSE_REQ_IO_COMPLETION via ServerSupportFunction >> - Request HSE_REQ_ASYNC_READ_CLIENT >> - On Error WriteClient + Request HSE_REQ_DONE_WITH_SESSION with >> HSE_STATUS_ERROR or _SUCCESS >> even if HSE_REQ_IO_COMPLETION has not been requested yet. ( hangs with >> client not receiving full response when this happens sometimes ) >> - Thread returns to its pool. >> >> ISAPI Thread calls Callback Proc >> - Queue work item ( including buffer contents of cbIO bytes ) to >> ThreadPool2 thread >> - On error close the file, destroy context, write client + Request >> HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_ERROR >> - If last block was queued WriteClient + Request >> HSE_REQ_DONE_WITH_SESSION with HSE_STATUS_Success >> ( and pray the last segment makes it to disk since we just reported >> success ) >> Return from callback >> >> ThreadPool2 thread: >> - Saves buffer to disk after waiting for prior I/O to finish >> - Closes the file and destroys the context class if this is the last disk >> write >> Thread returns to its pool. >> > >
