Be very careful how you reference your SPListItem objects through the SharePoint API.  I’ll say it again.  Be very careful how you reference your SPListItem objects through the SharePoint API.  Ok, now that you get the point that this will be a “learn from my mistakes and don’t do unsmart things like I did” post, let’s dig into what it was that I did poorly.

Scenario

For the past year I’ve been building custom .Net applications that are hosted through SharePoint.  These application involve a number of SharePoint lists, external databases, custom web parts, and other SharePoint elements to provide functionality.  About two weeks ago I received a message from one of our end users that a custom application was performing slowly.  Specifically performance was slow when users were performing actions that interacted with the primary SharePoint list storing data for that app.

The Problem

I took a copy of the production site into a dev environment to investigate the code that was executing.  After attaching the debugger and running through the code I quickly found pieces of code referencing SPListItem objects (like below) that were performing very poorly:

SPListItem myItem = SPContext.Current.Web.Lists["List Name"].Items.GetItemById(value);

// do updates on SPListItem retrieved

As it turns out the SPList I was referencing was fairly large at ~1000 items and weighing in over 150 MB.  You see the problem with my above code is that I retrieved the SPListItem by first (unnecessarily) going through the Items member of the list.  As I understand it, when doing so the executing code will attempt to resolve that entity and pull it from the database and into RAM (all 150 MB.)  This causes the equivalent of a 50 car pile up in terms of performance with a single update taking more than 15 seconds.

The Solution

The solution is actually quite simple and I wish I had realized this during development.  Instead of going through the Items member it is possible to call GetItemById(…) directly on the SPList as in the example below:

SPListItem myItem = SPContext.Current.Web.Lists["List Name"].GetItemById(value);

// do updates on SPListItem retrieved

After making this simple change performance skyrocketed and updates were back to less than a second.

Conclusion

When given the option between two solutions, usually the simplest is the best solution.  In my scenario I was adding extra complexity going through the API the long way around to get to the objects I needed and it ended up hurting performance greatly.  Luckily we were able to find and resolve the performance issue in a relatively short amount of time.  Like I said at the beginning of the post, learn from my mistakes and hope it helps you.

-Frog Out

Image linked from http://www.freespirit.com/files/IMAGE/COVER/LARGE/BeCarefulSafe.jpg

    The SPSiteDataQuery class in SharePoint 2007 is very powerful, but it has a few limitations.  One of these limitations that I ran into this morning (and caused hours of frustration) is that you can only return results from one list type at a time.  For example, if you are trying to query items from an out of the box custom list (list type = 100) and document library (list type = 101) you will only get items from the custom list (SPSiteDataQuery defaults to list type = 100.)  In my situation I was attempting to query multiple lists (created from custom list templates 10001 and 10002) each with their own content types.

Solution

    Since I am only able to return results from one list type at a time, I was forced to run my query twice with each time setting the ServerTemplate (translates to ListTemplateId if you are defining custom list templates) before executing the query.  Below is a snippet of the code to accomplish this.

SPSiteDataQuery spDataQuery = new SPSiteDataQuery();

spDataQuery.Lists = "<Lists ServerTemplate='10001' />";

// ... set rest of properties for spDataQuery

var results = SPContext.Current.Web.GetSiteData(spDataQuery).AsEnumerable();

// only change to SPSiteDataQuery is Lists property for ServerTemplate attribute

spDataQuery.Lists = "<Lists ServerTemplate='10002' />";

// re-execute query and concatenate results to existing entity

results = results.Concat(SPContext.Current.Web.GetSiteData(spDataQuery).AsEnumerable());

Conclusion

    Overall this isn’t an elegant solution, but it’s a workaround for a limitation with the SPSiteDataQuery.  I am now able to return data from multiple lists spread across various list templates.  I’d like to thank those who commented on this MSDN page that finally pointed out the limitation to me.  Also a thanks out to Mark Rackley for “name dropping” me in his latest article (which I humbly insist I don’t belong in such company)  as well as encouraging me to write up a quick post on this issue above despite my busy schedule.  Hopefully this post saves some of you from the frustrations I experienced this morning using the SPSiteDataQuery.  Until next time, Happy SharePoint’ing all.

      -Frog Out

Links

MSDN Article for SPSiteDataQuery

http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spsitedataquery.lists.aspx