Hello, We are using SQL Server 2000 and we are doing Full-Text Indexing on a table that has 5 Text Columns (we have Full Text Indexing on these 5 columns). There are around 1.5 million records in this table. And the size of the Full-Text Catalog is around 1 GB. The performance issue we are facing is some searches are slow. If I search for âto kill a mockingbirdâ it returns results very quickly and if I search for ânumbersâ the search is slower. So we are looking to increase the search speed. And also we need to get paginated results in our application. So what options I have to try? I am thinking of these options: ⢠Upgrade to SQL Server 2005 and partition this table based on the year. Is it possible to create different Full-Text Catalogs for each partition? ⢠I just learnt about Lucene4DB. http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . Has anyone used this tool in their application? Will there be a gain in performance in the search speed and maintaining the indexes. Thanks Vijay
SQL 2005 is a great choice. Lucene is also one. SQL FTS has faster indexing and is searches are very fast. Lucene is optimized for returning small number of rows (as is SQL FTS). Lucene does trimming which allows you to filter on the full-text index by date (for example). Katmai (SQL 2008) should close this gap. -- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com "Vijay" wrote in message news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... > Hello, > > We are using SQL Server 2000 and we are doing Full-Text Indexing on a > table > that has 5 Text Columns (we have Full Text Indexing on these 5 columns). > There are around 1.5 million records in this table. And the size of the > Full-Text Catalog is around 1 GB. > > The performance issue we are facing is some searches are slow. If I search > for "to kill a mockingbird" it returns results very quickly and if I > search > for "numbers" the search is slower. So we are looking to increase the > search > speed. And also we need to get paginated results in our application. > > So what options I have to try? > > I am thinking of these options: > . Upgrade to SQL Server 2005 and partition this table based on the year. > Is > it possible to create different Full-Text Catalogs for each partition? > . I just learnt about Lucene4DB. > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . Has > anyone used this tool in their application? Will there be a gain in > performance in the search speed and maintaining the indexes. > > Thanks > Vijay >
Hi Hilary Cotter, Thanks for your quick reply. I have a few followup questions: Can we create a FT Catalog for each partition in a partitioned table in SQL 2005? (If the FT Catalog is smaller then hopefully the searches will be faster) Can you point me to any sources that show the comparision of perfomance in Seraches and Indexing between Lucene4DB and SQL Server? When will Katmai (SQL 2008) release? Thanks in advance, Vijay "Hilary Cotter" wrote: > SQL 2005 is a great choice. > > Lucene is also one. SQL FTS has faster indexing and is searches are very > fast. Lucene is optimized for returning small number of rows (as is SQL > FTS). Lucene does trimming which allows you to filter on the full-text index > by date (for example). > > Katmai (SQL 2008) should close this gap. > > -- > RelevantNoise.com - dedicated to mining blogs for business intelligence. > > Looking for a SQL Server replication book? > http://www.nwsu.com/0974973602.html > > Looking for a FAQ on Indexing Services/SQL FTS > http://www.indexserverfaq.com > "Vijay" wrote in message > news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... > > Hello, > > > > We are using SQL Server 2000 and we are doing Full-Text Indexing on a > > table > > that has 5 Text Columns (we have Full Text Indexing on these 5 columns). > > There are around 1.5 million records in this table. And the size of the > > Full-Text Catalog is around 1 GB. > > > > The performance issue we are facing is some searches are slow. If I search > > for "to kill a mockingbird" it returns results very quickly and if I > > search > > for "numbers" the search is slower. So we are looking to increase the > > search > > speed. And also we need to get paginated results in our application. > > > > So what options I have to try? > > > > I am thinking of these options: > > . Upgrade to SQL Server 2005 and partition this table based on the year. > > Is > > it possible to create different Full-Text Catalogs for each partition? > > . I just learnt about Lucene4DB. > > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . Has > > anyone used this tool in their application? Will there be a gain in > > performance in the search speed and maintaining the indexes. > > > > Thanks > > Vijay > > > > >
No, one catalog per table partition or unpartitioned. I have no stats for comparison's between Lucene4DB or SQL Server. Katmai is scheduled for release in early Feb 2008 (IIRC). -- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com "Vijay" wrote in message news:56C8AB41-5885-48E2-B9EC-8A8691DDB682@microsoft.com... > Hi Hilary Cotter, > > Thanks for your quick reply. > I have a few followup questions: > Can we create a FT Catalog for each partition in a partitioned table in > SQL > 2005? > (If the FT Catalog is smaller then hopefully the searches will be faster) > > Can you point me to any sources that show the comparision of perfomance in > Seraches and Indexing between Lucene4DB and SQL Server? > > When will Katmai (SQL 2008) release? > > Thanks in advance, > Vijay > > "Hilary Cotter" wrote: > >> SQL 2005 is a great choice. >> >> Lucene is also one. SQL FTS has faster indexing and is searches are very >> fast. Lucene is optimized for returning small number of rows (as is SQL >> FTS). Lucene does trimming which allows you to filter on the full-text >> index >> by date (for example). >> >> Katmai (SQL 2008) should close this gap. >> >> -- >> RelevantNoise.com - dedicated to mining blogs for business intelligence. >> >> Looking for a SQL Server replication book? >> http://www.nwsu.com/0974973602.html >> >> Looking for a FAQ on Indexing Services/SQL FTS >> http://www.indexserverfaq.com >> "Vijay" wrote in message >> news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... >> > Hello, >> > >> > We are using SQL Server 2000 and we are doing Full-Text Indexing on a >> > table >> > that has 5 Text Columns (we have Full Text Indexing on these 5 >> > columns). >> > There are around 1.5 million records in this table. And the size of the >> > Full-Text Catalog is around 1 GB. >> > >> > The performance issue we are facing is some searches are slow. If I >> > search >> > for "to kill a mockingbird" it returns results very quickly and if I >> > search >> > for "numbers" the search is slower. So we are looking to increase the >> > search >> > speed. And also we need to get paginated results in our application. >> > >> > So what options I have to try? >> > >> > I am thinking of these options: >> > . Upgrade to SQL Server 2005 and partition this table based on the >> > year. >> > Is >> > it possible to create different Full-Text Catalogs for each partition? >> > . I just learnt about Lucene4DB. >> > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . >> > Has >> > anyone used this tool in their application? Will there be a gain in >> > performance in the search speed and maintaining the indexes. >> > >> > Thanks >> > Vijay >> > >> >> >>
Thanks Hilary Vijay "Hilary Cotter" wrote: > No, one catalog per table partition or unpartitioned. > > I have no stats for comparison's between Lucene4DB or SQL Server. > > Katmai is scheduled for release in early Feb 2008 (IIRC). > > -- > RelevantNoise.com - dedicated to mining blogs for business intelligence. > > Looking for a SQL Server replication book? > http://www.nwsu.com/0974973602.html > > Looking for a FAQ on Indexing Services/SQL FTS > http://www.indexserverfaq.com > "Vijay" wrote in message > news:56C8AB41-5885-48E2-B9EC-8A8691DDB682@microsoft.com... > > Hi Hilary Cotter, > > > > Thanks for your quick reply. > > I have a few followup questions: > > Can we create a FT Catalog for each partition in a partitioned table in > > SQL > > 2005? > > (If the FT Catalog is smaller then hopefully the searches will be faster) > > > > Can you point me to any sources that show the comparision of perfomance in > > Seraches and Indexing between Lucene4DB and SQL Server? > > > > When will Katmai (SQL 2008) release? > > > > Thanks in advance, > > Vijay > > > > "Hilary Cotter" wrote: > > > >> SQL 2005 is a great choice. > >> > >> Lucene is also one. SQL FTS has faster indexing and is searches are very > >> fast. Lucene is optimized for returning small number of rows (as is SQL > >> FTS). Lucene does trimming which allows you to filter on the full-text > >> index > >> by date (for example). > >> > >> Katmai (SQL 2008) should close this gap. > >> > >> -- > >> RelevantNoise.com - dedicated to mining blogs for business intelligence. > >> > >> Looking for a SQL Server replication book? > >> http://www.nwsu.com/0974973602.html > >> > >> Looking for a FAQ on Indexing Services/SQL FTS > >> http://www.indexserverfaq.com > >> "Vijay" wrote in message > >> news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... > >> > Hello, > >> > > >> > We are using SQL Server 2000 and we are doing Full-Text Indexing on a > >> > table > >> > that has 5 Text Columns (we have Full Text Indexing on these 5 > >> > columns). > >> > There are around 1.5 million records in this table. And the size of the > >> > Full-Text Catalog is around 1 GB. > >> > > >> > The performance issue we are facing is some searches are slow. If I > >> > search > >> > for "to kill a mockingbird" it returns results very quickly and if I > >> > search > >> > for "numbers" the search is slower. So we are looking to increase the > >> > search > >> > speed. And also we need to get paginated results in our application. > >> > > >> > So what options I have to try? > >> > > >> > I am thinking of these options: > >> > . Upgrade to SQL Server 2005 and partition this table based on the > >> > year. > >> > Is > >> > it possible to create different Full-Text Catalogs for each partition? > >> > . I just learnt about Lucene4DB. > >> > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . > >> > Has > >> > anyone used this tool in their application? Will there be a gain in > >> > performance in the search speed and maintaining the indexes. > >> > > >> > Thanks > >> > Vijay > >> > > >> > >> > >> > > >
