Check out this interesting blog from Dan Holme. Dan is a speaker at the European SharePoint Conference in Berlin from the 17th - 20th October. Check out Dan's session  on "Designing Governance: How Information Management and Security Must Drive Your Architecture".

In recent months, we've seen customers who are doing major migrations of documents-for example, from file servers-to SharePoint.  Far too often, one of the complaints I hear voiced in these scenarios is how slow the upload can be, particularly when uploading huge numbers of documents.  I thought it would be worth starting a discussion of lessons learned related to performance of mass uploads. This is a fairly targeted business scenario-most organizations don't do this too often-but it also brings up some key points to consider about performance in other SharePoint scenarios.

The following are among the factors that can cause performance of mass uploads to suffer:

•         The recovery model for the content database (a SQL Server setting) is set to full by default. While this is certainly the appropriate setting for a production environment, as it allows for recovery using transaction logs, it can slow the content loading of a migration. Set the recovery mode to simple, which causes the contents of the transaction log to be truncated each time a checkpoint is issued for the database.  Just remember two things: First, set it back to Full when finished.  Second, remember this mode means that the database recovery point can only be as recent as the last database backup, so you'll probably want to back up before your migration-and there are many good reasons for that, anyway.

•          Search indexing, if it kicks in, consumes resources that you might need on your WFEs and SQL servers for processing the migration of files. Make sure that search jobs are scheduled appropriately-or paused-while you do your mass upload.

•          Anti-virus software, if it is scanning every document that is uploaded, or is scanning the database or BLOB store directly, can slow things down tremendously. Assuming that your documents were scanned when they were uploaded to their original location, you probably don't need to incur that penalty when simply moving those documents to SharePoint.

•          BLOB storage can affect performance-for better or worse. As you know, I've done a lot of writing and speaking about BLOB storage and content database scalability.  BLOBs (binary large objects) are the binary, unstructured chunk of data that is the document as it is stored in SQL in the AllDocStreams table of your content database.  You can externalize BLOBs using EBS or RBS, which means you store BLOBs in a location other than your content database, and the database gets a pointer to the document.  When you externalize BLOBs, you reduce the writes to your database.  By default, when you upload a document, it gets written to the transaction log first, then gets committed to the database.  That's two writes for every document. By externalizing BLOBs, there is conceptually a performance benefit.  But it really depends on the performance of the storage tier to which you move BLOBs, and depending on the performance of the EBS or RBS provider (the software that manages the communication between EBS/RBS, which are Microsoft APIs, and your BLOB storage platform).  For example, if you're externalizing BLOBs to cloud storage-like Amazon or Rackspace for example, it's likely performance will be penalized.  But if you're externalizing to a high-performance storage tier, performance can definitely increase for this mass-upload scenario.

•          Database growth sizing. The default database size and growth settings for SQL databases are really not appropriate for most SharePoint databases, particularly those that will contain BLOBs.  Set the size of your content database to something that represents the size of the data you're going to upload. Consider the space that metadata will take, as well.  That way, SQL doesn't have to "grow" the database as you upload-the space is already there.  As a side note, size and growth affect performance as your environment scales-there are some great blog posts on the "interwebs" to help you determine an appropriate setting, but I recommend setting an initial size that represents your expected content size (including metadata and BLOBs, if stored in SQL) over the first few months of your service, and a growth setting of 10% of that size.  But be smart about it-there are a lot of variables in that calculation that all depend on your usage patterns.

Click here to view more:

Be part of the European SharePoint Conference in Berlin!