Their concern stemmed from reading a Technet article that lists the boundaries and limitations of SharePoint 2010, such as a maximum of 100GB per Site Collection, 300 Content Databases per Web Application and 2,000 Subsites per site view. Based on an initial reading of that article the firm had concluded that to cope with their volumes without exceeding these constraints they would need a Site Collections / Site / Library structure that was far from user-friendly, and which required so many Content Databases as to be extremely expensive to maintain.
MacroView was able to show that in addition to enhancing document management functionality and usability on SharePoint, MacroView DMF also enables the intuitive design of large-scale SharePoint document stores that have good performance and are efficient in terms of both machine resources and on-going administrative effort.
To demonstrate this MacroView created a large-scale SharePoint document store, which was code-named ‘ATLAS’. The ATLAS implementation has a single ‘Clients’ Site Collection containing over 30,000 subsites (each corresponding to a particular Client or Customer). Each Client sub-site has between 2 and 10 Document Libraries, each corresponding to a particular Project or Matter or Assignment. In total there are over 165,000 Document Libraries. Documents and emails were loaded into all 165,000 of these Document Libraries. In total over 3.2 Million documents and emails (total volume approximately 100GB) were loaded into ATLAS.
This is the sort of structure that you would find in a very large professional services office, such as a law firm or consulting organisation.
The ATLAS implementation uses a single Site Collection for all Sites and Libraries, but has Remote Blob Storage configured so that the bulk of document content is actually stored outside the SharePoint SQL Content Database. Vendors such as StoragePoint indicate that their RBS approach leads to a 20-fold increase in the volume of documents that can be managed using a single SharePoint Content Database. This means that the effective capacity of a Content Database becomes 2TB of documents, rather than the 100GB mentioned by the Technet article.
With SharePoint 2010 Microsoft has clearly paid attention to the need for efficient viewing of document libraries that contain large numbers of files. Relevant features in this regard are Resource Throttling and support for indexing of metadata columns (e.g. Managed Metadata columns). The resulting performance benefits apply when you use MacroView DMF to browse a large document library.
In addition MacroView DMF also improves efficiency of navigating TO a particular site or library within a document store that contains a large number of sites and libraries, as it typically the case in a large professional services firm implementation. The root site of the ‘Clients’ Site Collection in ATLAS contains a separate subsite for each of 30,000+ clients. Even though this design is way outside the ‘recommended’ limits shown in the Technet article, users accessing and navigating it via MacroView DMF enjoy excellent performance.
On closer examination it becomes clear that some of the limitations listed in the above Technet article refer to the out-of-the-box web browser UI for SharePoint, rather than to the SharePoint platform itself. For example the limitation re 2,000 subsites per Site actually reads:
Subsite |
2,000 per site view |
Threshold |
The interface for enumerating subsites of a given Web site does not perform well as the number of subsites surpasses 2,000. Similarly, the All Site Content page and the Tree View Control performance will decrease significantly as the number of subsites grows. |
In other words: “The out-of-the-box web browser end user interface for SharePoint does not perform well if there are more than 2,000 subsites of a site (web). Similarly the web browser page that is provided for administrators also suffers significantly decreased performance when there are a large number of subsites.”
Instead of attempting to display thousands of sub-sites, MacroView DMF uses filtering and search techniques to rapidly navigate to and display just the Client Site or Project Library that the user requires. From the user perspective the resulting filtered list of nodes is much preferable to having to cursor though a large number of nodes to find the one you want. The MacroView DMF approach is also efficient in terms of bandwidth and other machine resources.
The purpose-built MacroView DMF web service plays an important role in this regard, by providing specific support for DM-related tasks and dramatically reducing the volume of XML returned, compared to the out-of-the-box web services that ship as part of SharePoint. The DMF web service makes all access to the SharePoint content database via the SharePoint Object Model Layer, which ensures that DMF’s tree-view and file list displays fully respect the user’s permissions in SharePoint.
In addition to efficient browsing and navigating of the SharePoint Site / Library tree, MacroView DMF also provides a convenient and intuitive interface to the SharePoint Search and Fast Search engines. Search results are displayed with sub-second performance, even across the large volume of documents in the ATLAS document store. The DMF Search Panels can be used in conjunction with a federated search index, to find particular documents across a globally-distributed SharePoint environment.
The ATLAS implementation demonstrates that the combination of MacroView DMF +SharePoint 2010 enables large-scale document management solutions that are both attractive to use and efficient in operation.