Monday, March 24, 2008

Coveo Search vs. Ontolica Search for SharePoint

    Coveo vs. Ontolica.

    Search is very important part of a corporate Intranet portal or Internet site. Choosing the right search engine is critical, but can also be a difficult process because the most important questions like "is this search engine scalable, reliable and secure" take the longest to answer. I've worked with two of the more popular search engines for SharePoint over the past 6 months and have gathered enough data on the two to make some valuable, well-tested observations. Below I will compare Coveo and Ontolica.

  • Both Ontolica and Coveo are very good products. I have really been impressed with both. They offer fantastic, have-to-have features over out-of-the-box SharePoint. After you have used either Ontolica or Coveo, you will not want to go back to native SharePoint Search.
  • About Ontolica
    • Ontolica uses the SharePoint native search engine. Note that the native SharePoint search engine is included with the purchase of WSS 3.0 and MOSS.
    • Ontolica is installed as 4 SharePoint features and is administered through SharePoint application pages
  • About Coveo
    • Coveo has its own search index engine, and its own administrative interface.
  • Cost estimate for Ontolica:
    • Costs about $9,000 per production WFE (Web Front End) and an additional $4,500 for all development environments. So for example if you have a production farm with 2 WFE and a staging and a development environment and a VPC on your laptop, the cost would be 9k+9k + 4k = ~$20k.
    • Coveo offers two trial license options: either 1) you can have only two uses access the search index at a time and the license does not expire, or 2)you can have unlimited users use Ontolica for a limit of 30 days. Not that it is easy to add or remove a license without having to reinstall Ontolica.
  • Cost estimate for Coveo
    • Coveo licenses based on total documents indexed. They have a 100k document license, a 10k document license, 1M license etc. For example, a 100k document license is about $12k cost for the primary indexer which would run on the first WFE and the cost for a mirror is 60% of the primary indexer which would run on the 2nd WFE = $7k in our example. Development environment costs %50 of primary indexer = $6. $12k + $7k + $6k = ~$25k. This includes an annual support license.
  • Index browser
    • Coveo comes with an index browser, Ontolica does not have an index browser.
    • This is quite significant because it allows you to see what is in the index. It helps for troubleshooting and development purposes.

     

     

  • Customer Support
    • Customer support for Coveo is very responsive. They send you a livemeeting link and stay on the phone until the issue is resolved. I called support at least 15 times over the past two months, and they were very helpful each time. I actually have the direct phone number to the support rep that I use most frequently. Also, emails are acknowledged immediately and most often responded to within a few hours.
    • Ontolica support was slow to get back to me. When you call, you only get a receptionist. (granted my client had not purchased the product yet so the experience may be different after you have purchased the product and have a support license)
  • Administration interface
    • Coveo has a rich administration and configuration interface, with more options and quick, responsive navigation. It does take some getting used to, and is a lot to learn, but when you have learned it, it is very powerful and much more logical layout than MOSS native Search, in my opinion.
    • Ontolica uses the SharePoint infrastructure for its admin interface so if you are used to this interface, you will feel comfortable with the admin and configuration Ontolica interfaces. But there are fewer options available to you.
  • Search Results interface.
    • The single most important feature of both Ontolica and Coveo is results filtering. These allow the user to further filter results with a few single clicks. It also gives some great information about the result set by grouping results into categories like document Type, Author, SharePoint site, and displaying how many documents are in each group. (see screenshot)

    Screenshot of Coveo search result filtering user interface:

    • clip_image002[4]

    Screenshot of Ontolica search result filtering:

    clip_image003[4]

    • Coveo uses a nice Ajax interface which searches as you type. This provides a superior user experience. Instead of having to press enter to navigate to the results page, the user is able to see the results in a dropdown and click directly to the result.
    • clip_image004[4]
  • Security Trimming:

    ○ both Ontolica and Coveo do security trimming. (not showing results for documents and items you don't have permissions to view.

  • Transferring settings from one SharePoint environment to another
    • For Coveo there is a single config.txt file which holds all configuration settings. So moving settings from Dev to Stage to Prod is simply a matter of copying that file from one environment to another.
    • With Ontolica you have settings both MOSS search configuration and in Ontolica search configuration. If you backup a content database from dev to another environment, Ontolica configuration will be included, but MOSS search configuration (managed properties for example) will not be included and Microsoft does not provide a good tool that I know of to move those settings between environments, although there are some tools on codeplex.com that do this.
  • Index Crawler Performance:
    • Ontolica and MOSS native search have supported 64 bit platform since RTM of MOSS 2007.
    • Coveo crawler is faster than Ontolica crawler, as of version 5.1.2960 Coveo supports the 64 bit platform.

    - Results Settings

    • Coveo offers many more settings for results customization without having to write code
      • Options include turning off the advanced search panel, highlighting the title, displaying or hiding RSS feeds, etc.

    clip_image005[4]

  • Results presentation - Complete Customization of
    • Ontolica: XSLT results customization
      • Ontolica uses XSTL templates to control how results are displayed. If you do not have remote desktop access to the Web Front end servers, then Ontolica is probably the choice for you because all you need is a web browser to customize the display format of search results. There is one Ontolica Styles library per site collection.

    Screenshot of Ontolica Styles Library. This library contains XSLT templates which you can use to affect how search results are displayed.

    • clip_image006[4]
    • Coveo: custom ASCX files
      • Coveo uses custom ascx files (asp.net user controls) to allow you to customize search results.
      • Ultimately this ASCX technique will be more powerful for customizing search results because you can do server side scripting as well as client side scripting.
    • If you are more comfortable with ASP.NET development than with XSLT development, Coveo may be your best choice.
    • Note that with Coveo you need to have remote desktop access to the box to add an ascx file. But once that file is uploaded, it is available to all SharePoint sites in the Farm.
    • clip_image007[4]
  • User interface for search results.
    • Both Coveo and Ontolica have an clean results page. However, the Coveo search results returns a larger set of metadata and the layout is cleaner. The coveo results also display how many of each item per result group are contained in the result set.

    Coveo results:

    clip_image008[4]

    Ontolica results:

    clip_image009[4]

  • Managed Properties
    • Setting up managed properties is easier in Coveo than in MOSS/Ontolica Search. For example, you have to scroll though many pages to get to the managed property in MOSS, which is inconvenient. MOSS does have a "search crawled properties" which is helpful, but there is no "search managed properties".
    • Also in MOSS/Ontolica, you have to set up the properties in two places: once as a managed property in MOSS, and once in Ontolica.

    Setting up properties in Coveo:

    clip_image010[4]

    Setting up properties in MOSS/Ontolica:

    clip_image011[4]

  • Troubleshooting tools
    • The Coveo console is a nice real-time peek at what the index is doing. It allows you to view the indexer as it crawls and if any errors occur, you can see in real time what is going on.
    • MOSS/Ontolica view of current progress. Very bare bones. It only tells you the status of the crawler, and does not update real time.

    clip_image012

  • Scopes
    • Controlling and building Scopes is easier in Coveo. Moving between screens is faster,
    • MOSS prohibits two different content sources from pointing to the same URL. This is inconvenient for search engine tuning and testing when you want to index a subset of information (for example a particular document library) In MOSS because of this limitation you are forced to redraw the entire site collection each time you make a change instead of being able to crawl just the library you are working on.
  • Custom scripts
    • Both Coveo and MOSS/Ontolica offer scripts for customizing search result ranking.
    • Coveo offers the ability to write and run VB scripts which modify index content and ranking.
      • For example if you want to strip out the pound sign from a metadata value, or you want to boost the search index ranking for items that have a metadata value set to "premium=true".
    • Modifying the MOSS index involves using the SharePoint API. Using the SharePoint API is generally easy and powerful, but note that I have not yet used it to modify index content so I cannot comment on how easy or hard this would be with MOSS/Ontolica.
  • Misc
    • Coveo Webparts
    • clip_image013

Friday, March 21, 2008

Donate to SUSHI

If you would like to make a donation to the SharePoint SUSHI project, click the donate link below.















Saturday, March 15, 2008

SUSHI's popularity

SUSHI has become more popular than I had ever imagined. Codeplex has just added a fantastic statistics page which shows exactly how popular it is. The stats include number of downloads, visits and referring sites. SUSHI has been downloaded over 2000 times!

Figure 1: Downloads since the project began in November 2007:

image

 

It is interesting to see where most of my traffic comes from, and read the reviews on those sites. My favorite review is from a this German site: "Send a Test Email – Um die outbound Mail Funktion zu testen ist dieses Tool super handy." :) It looks like wssdemo.com has brought me a lot of traffic and of course google.

Figure 2: Top Referring Sites over last 30 days:

image

 

Final Comments

SUSHI fills a need for SharePoint in a way that very few other tools do and it is much more user friendly than command lines, or SharePoint features. But it is not yet finished. I still have a lot of additions to make. I would love to have community help with building SUSHI. If anyone is interested in contributing to SUSHI by writing code, testing code or suggesting features, I would love to hear from you. Please post  a comment on the SUSHI discussion boards, or contact shoot me an email at home@ fluckiger.org.

Monday, March 10, 2008

SUSHI Version 2.22 Released!

You can download the latest version of SUSHI here. Improvements in this version include:

 

*Improvements to the "Copy a view" feature.

    • Change copy a view so it doesn't require all fields to be present. This is a nice improvement! so that this feature can be used without having to have all the columns of the view present in the destination list.

**More tab for less popular features, click on the More tab to open up these features.
BugFix: For Import Documents feature, Upload button was disabled after pressing the validate. button. This is fixed now.

 

version 2.23 is in the works:

I've been working on improvements to the security report in SUSHI so that the report will also take into consideration active directory group membership in determining if a user has access to a site. To accomplish this, I've had to brush up on the System.DirectoryServices namespace used to access Active directory.

 

-Joseph

Tuesday, March 04, 2008

XSLT

I never thought that I would like XSLT and I hoped that I wouldn't have to learn this "wimpy web language" which is how I used to see it. But after this weekend, I am a big XSLT fan.  Which is appropriate because this month marks the 10 year anniversary of XML.

So why is XSLT so cool? I think I would have embraced it much sooner had someone made this simple comparison for me: XSTL is like TSQL. I love TSQL: it is so powerful for getting and manipulating data. It is blazing fast and it takes way way fewer lines of code than it would take in .NET. XSLT is similar because both TSQL and XSTL are declarative languages: they focus on saying what do do rather than how to do it. When writing TSQL or XSLT I only have to say what I want done, and the parsing engine is responsible for figuring out how to do it.

Another way to look at these two languages is that they do one thing really really well: TSQL manipulates data in a database, and XSLT transforms XML.

I strongly recommend the Wrox XML book: Beginning XML. I spent most of my Saturday looking at other XML books, but I always seem to come back to Wrox. They are just so so much better than everything else. Some of the other XML books are just embarrassing and it is a shame that bookstores sell them to poor unsuspecting victims who read them and end up being more confused than when they started reading those horrible books. My advice: stick with Wrox.

<xsl:template match="/">
  <html>
    <body>
      <h3>I love XSLT</h3>
    </body>
  </html>
</xsl:template>