This blog post is an extract of a MacroView White Paper. Download the full White Paper here.
As more organizations explore the potential of using Microsoft SharePoint for Document Management and Email Management a question that arises frequently is “What is the best Information Architecture for a SharePoint DM store?” In other words, what is the best way to arrange the various SharePoint 2010 ‘building blocks’ – Site Collections, Sites, Document Libraries, Document Sets, Folders and metadata columns – to come up a design that is optimal in terms of volume handling, functionality and ease of use? That last objective – ease of use – is critical because if the users do not find it easy to interact with the document store they will likely store their documents in file shares or local drives and their emails in Outlook folders and the SharePoint DM project will not be successful.
Deeply Nested Folder Trees
There is now general recognition that deeply nested trees of SharePoint folders in one or two large document libraries is NOT a good design. When SharePoint 2007 was first released these designs appeared to offer an easy means of migrating existing File Shares into SharePoint – the tree of folders from the ‘P:’ drive were simply reproduced, one for one, with SharePoint folders. The fact that the full SharePoint folder names becomes part of the URL for a document stored in SharePoint rapidly led to URLs exceeding their maximum permitted length of 255 characters – especially if the Folder names were long and meaningful.
These designs also led to very large numbers of documents being stored in a single Document Library, which in turn caused poor performance when viewing the lists of documents in those libraries. In an attempt to prevent performance degradation, with SharePoint 2010 Microsoft introduced List View Throttling.
So if deeply nested folders are bad, what alternative design is best? How about using Document Sets instead of Folders? Isn’t it better to use metadata columns – particularly Managed Metadata columns – instead of either Folders or Document Sets? Don’t you need to have lots of Site Collections if you want to store a large volume of documents? Maybe we should just use Search and not attempt to have a hierarchy of storage containers at all?
The first point I would make is that there is no single design that is right for every organization. To come up with an optimal SharePoint DM design you do need to take specific business requirements into account.
A second point is that size does matter – lots of designs work well enough if you have only a small volume of documents or users, but good design makes a major difference when there are hundreds of terabytes of documents and / or thousands of users. A related observation is that document stores tend to grow very quickly so you should be designing your SharePoint DM store to cope with volume.
MacroView DMF Enables a Better Class of SharePoint DM Design
By the way it lets the user view and navigate a SharePoint document store, MacroView DMF enables a class of designs for SharePoint document stores that are optimal in terms of volume handling, functionality and ease of use.
We designed our MacroView Document Management Framework (DMF) to cope with larger SharePoint document stores. MacroView DMF shows you the complete tree structure of your SharePoint store – you simply register your SharePoint Web Applications and MacroView DMF then automatically (and efficiently) displays all areas of the store that contain document content for which you have access permission. The tree display extends down to the hierarchies of those Metadata Columns that have been defined as being available for Metadata Navigation.
MacroView DMF displays the full SharePoint site and library tree and enables efficient filtering of Sites and Libraries based on their Titles. The MacroView DMF tree display extends to show metadata navigation, such as this hierarchical term set.
Efficient Filtering and Searching of the Site / Library Tree
A key aspect of that MacroView DMF tree display is that it never attempts to display more than a threshold number of sub-nodes when you click to expand any node in the tree. For example if a top level site contains hundreds of sub-sites and document libraries, MacroView DMF will prompt you to enter some characters contained in the Titles of sub-sites and libraries and then display only those whose titles do contain the nominated characters. This filtering has excellent performance and minimal bandwidth consumption, thanks to purpose-built MacroView DMF components running at the SharePoint server.
MacroView DMF also enables filtering for Document Sets and Folders in a Library – it filters these nodes based on characters contained in their names. This is an important difference – because Sites and Document Libraries can have a Title as well as a URL, there is no pressure to keep Titles short (to avoid the 255 character limit on length of URLs). Instead Site and Library titles can be long and meaningful and they can be adjusted without having to change the URL of the Site or Library.
Speaking of searching, MacroView DMF also makes it convenient and intuitive to search for documents or emails that are stored in SharePoint. By using readily-configurable DMF Search Panels you can search based on metadata and / or keyword content while you work in Outlook, Word, Excel or PowerPoint.
Optimal Design Characteristics
Having encountered a wide range of alternative designs, MacroView consultants have determined that the designs that work best tend to use a tree of sites, each with a filterable / searchable title. The site tree reflects the fundamental structure of an organization – generally the various lines of business or other activity. Document libraries occur freely across those sites, again with each library having a filterable / searchable title.
The document libraries contain whatever metadata columns are relevant to the organization. Because they are centrally managed, user extensible and can be hierarchical, Managed Metadata columns are a good choice for those metadata attributes that are common across multiple libraries.
Wherever possible these metadata columns should be automatic – i.e. their values are recorded accurately to facilitate subsequent searching for documents, without needing to prompt the user as he / she saves / uploads to that library.
The document libraries can contain some Document Sets and Folders. However because the main business structure is reflected in the Site / Library tree, the use of Document Sets and Folders is not excessive to the point where issues with exceeding the List View Threshold emerge.
Why Doesn’t Everyone Design This Way?
If these MacroView-style designs are so good, how come everyone is not already using them?
I believe that the answer to this good question lies in the way the out-of-the-box SharePoint web browser UI operates. With that interface it is quite difficult for an end user to visualize the full tree of Sites in a SharePoint document store. You can easily view and click around the tree of libraries and folders and document sets that is contained within an individual Site, but navigating to another site is more difficult – unless it’s a sub-Site you typically need to key in a URL. In other words the shortcomings of the OOB web browser UI are leading folks to designs that use lots of Document Sets and / or Folders.
By showing the end user the full tree of sites and libraries for which he / she has permission, and making it easy and efficient to navigate across those Sites and Libraries, MacroView DMF enables designs that use more Sites and Libraries. These MacroView-style designs avoid volume handling issues such as the List View Threshold, minimise issues with excessively long URLs and eliminate the need to delete and re-create storage containers when key metadata such as names of Clients and Projects changes during the business lifecycle.
In the full form of this paper we compare this MacroView-style design approach to three alternative design concepts for a SharePoint document or email store:
- I need to use Document Sets because their Metadata helps me navigate…
- Why not use a managed metadata hierarchy to handle Client/Project structure?
- I want to use Search rather than a tree of SharePoint containers…