With its native document management capabilities, SharePoint is attractive to many organizations. The IT area sees SharePoint as a way to improve upon storing emails in Outlook folders and other files in network file shares. From a management perspective extending the use of SharePoint to email and document management makes sense because it means that the organization is extracting more value from its investment in SharePoint. This is particularly the view when the organization is paying maintenance for a third-party DM system, such as Autonomy / iManage FileSite or OpenText / Hummingbird DM.

So why is it that so many 'SharePoint for DM' projects are not that successful? The basic reason is that the business end-users are frustrated and disappointed by the new SharePoint-based DM solution. From a user experience perspective, out-of-the-box SharePoint has a number of issues that must be addressed if the new SharePoint-based DM solution is to be genuinely successful.

In this article we will look at how these 'human factors' issues are addressed by a SharePoint add-on called MacroView Document Management Framework (DMF).

Managing Outlook Emails

"You can manage your emails by dragging and dropping them to an Outlook folder, but you can't drag and drop to save an email to SharePoint."

Email management is where most organisations start using the DM capabilities of SharePoint. This is not surprising because emails are these days such a major form of business communication, yet typically they are managed in a very haphazard fashion – if you are lucky John or Mary will have a copy of that important email in one of their personal Outlook folders. SharePoint appeals as a way to improve the management of Outlook emails – after all, SharePoint and Outlook are both Microsoft products, so they should work really well together, right?

As many organisations have found, the out-of-the-box integration of Outlook and SharePoint leaves a great deal to be desired. In response to this, various add-ons are now available on the market that allow you to drag and drop an email in Outlook to save it to SharePoint, preserving any attachments and even preventing duplicate copies from being saved by multiple recipients. MacroView DMF is one of these.

Figure 1: MacroView DMF adds a new pane to Outlook, which displays a full tree-view of SharePoint.

As this screen shot shows, MacroView DMF adds a new pane to the user's Outlook window, which displays all areas of the SharePoint document store for which the user has access permission, and allows saving of emails by drag to drop to any library or folder in SharePoint.

A distinctive feature of this DMF tree-view display is that it copes well with very large SharePoint document stores – tree-view display and navigation remains efficient even when the SharePoint document store grows to have many thousands of sites and libraries and folders – containing millions of documents. As you evaluate add-ons that improve the integration of Outlook and SharePoint it is worth remembering that your SharePoint document store will probably become quite large quite quickly.

User frustration with the out-of-the-box integration of Outlook and SharePoint is not limited to being able to save by dragging and dropping. As an example, when they right-clicked on a file in Windows Explorer they were able to create a new outgoing email with that file inserted as an attachment. MacroView DMF provides this same functionality when you right-click on a file or files stored in SharePoint.

Saving and Re-saving PDFs

"We scan our incoming paper documents and store the scanned images as PDFs in SharePoint. I often need to open a PDF from SharePoint and then save it back to SharePoint. It is very frustrating to have to provide the metadata again as I re-save."

The ready availability of 'smart' copiers and multi-function devices has made it much easier for user departments to scan to PDF. Storing the resulting PDFs in SharePoint enables business workflows to be created for these scanned images.

However Adobe Reader and Adobe Acrobat have only recently (with version X) offered a native ability to save to SharePoint. When they re-save a PDF that they have opened from SharePoint, what users really want is for any metadata that has already been captured for that PDF to be displayed so that it can be edited. Rather than completely replace the existing PDF they may also want to save a new version, in order to provide a form of audit trail.

Part of MacroView DMF is a plug-in for Adobe Reader / Acrobat that facilitates saving and re-saving of PDFs to SharePoint. As you re-save a PDF that you have opened from SharePoint this PDF SharePoint Save plug-in prompts to replace or create a new version (depending on the versioning settings in the destination library) and displays any already captured metadata for editing. The popularity of this PDF SharePoint Save plug-in is such that MacroView has made it available on a standalone basis, as well as including it in MacroView DMF.

Viewing and Navigating the SharePoint Document Store

"When we stored our files in the P: drive I could understand and navigate the folder structure; now we store them in SharePoint I find it MUCH harder to visualise and navigate the document store."

This statement comes from a user in an organisation that was attempting to use SharePoint to replace its file shares. The organization was hoping that its users would appreciate the more flexible ways that SharePoint provides for viewing and filtering based on metadata and for searching based on both metadata and content. Instead the overall user reaction to SharePoint was decidedly negative.

In large part this stemmed from the inability of the out-of-the-box SharePoint interfaces in the browser and in Word / Excel / PowerPoint to display the whole tree of SharePoint sites and libraries. The users found themselves having to enter URLs so that they could navigate to another Site. They were also confused by the way the available tree display stopped at the next break in permission inheritance.

What the users really wanted was to be able to view and navigate the hierarchy of the SharePoint document store in much the same way as they were previously able to view and navigate the tree of folders in the old network file shares, or the tree of folders in Outlook.

As the above screen shot shows, MacroView DMF is designed to display the complete SharePoint tree of sites, document libraries and folders. DMF provides that tree display in a range of popular user applications – Outlook, Word, Excel, PowerPoint, Windows Explorer and Adobe Reader / Adobe Acrobat. The SharePoint tree is also available in a new Windows application called MacroView DMF Explorer – as the name suggests a Windows Explorer-like tool for viewing and navigating SharePoint. The DMF tree actually makes navigating a large SharePoint tree structure even easier than navigating a large folder tree in Windows Explorer.

By making the whole SharePoint site tree readily visible and navigable MacroView DMF enables a whole new class of designs for your SharePoint document store, which are often far more popular with users than the designs that tend to occur when you have only the out-of-the-box SharePoint browser UI.

An example of these sub-optimal designs is a SharePoint document library with a deeply nested tree of SharePoint folders. This design allows the users to visualise the tree-structure of the SharePoint store when using the out-of-the-box SharePoint user interfaces, but it is not good for searching and it runs the real risk of issues associated with exceeding the threshold on the number of files in a SharePoint 2010 library and of exceeding the 255 character limit on the length of URLs for files.

Another style of design that is unpopular with users is document libraries with a large number of metadata columns. This design appeals to IT folks with a database background and also to some Records Managers (presumably because it provides a lot of flexibility for retrieval at discovery time). However users dislike the way this style of design makes them respond to multiple metadata prompts every time they save a document (or an email) to SharePoint.

What users really want is metadata to be recorded automatically as much as possible, with minimal or ideally no prompting. By recording email-related metadata automatically and by enabling designs with a tree of sites and libraries with automatic metadata columns, MacroView DMF greatly assists in minimising 'profiling fatigue'.

Searching

"It's not that SharePoint cannot search for documents and emails; it just that it doesn't let me search from where I want to work."

Improved searching is very relevant when it comes to better managing your files and emails. It's well known that SharePoint comes with a Search capability - in addition to SharePoint 2010 Search, there is also FAST Search for SharePoint 2010. These search engines, particularly FAST, have some great features, but the feedback from organisations that attempt to use out-of-the-box SharePoint (and FAST) Search for email and document management is that users find Search frustrating. Why is this?

It's not that SharePoint Search lacks the power to find documents in a large document store - it can index very large document stores and find specific documents very quickly. The problem is that this search power is not available where the user wants it to be available – i.e. while he / she continues to work in familiar applications like Outlook or Word, or Excel or PowerPoint. Instead, to do a search the user must jump out to the web browser. The search results are listed on a web page, but what the DM user really needs is for those results to be immediately available in context – e.g. to open as the next Word document or attach to an outgoing email.

Here are some screen shots – again from MacroView DMF – that illustrate the desired effect. While working in Word 2010 the user can click File, Open from SharePoint and do a search for the document that he wants to open.

MacroView DMF sends a query to the SharePoint Search engine (or to FAST Search if it is deployed) – so the same documents will be found as if you did the search in the Search Center web part.

Figure 2: MacroView DMF enables searching of SharePoint directly from Word 2010.

DMF also enables searching the SharePoint store directly from Outlook:

Figure 3: MacroView DMF Search pane and search results in Outlook 2007.

By enabling searching while working in familiar applications, MacroView DMF makes the search experience way more convenient for the user. And more intuitive too, because these DMF Search panels can readily be configured to reflect the way a particular organisation uses metadata in its management of documents. From a user perspective configuring search panels that are available conveniently in popular applications is far preferable than the organization spending a lot of money on building custom search web parts.

Security

"I don't want to save my sensitive documents to SharePoint unless I can control who else has access to them."

There is a lot of discussion about the lack of support in SharePoint for 'Deny-style' security. It's true that with out-of-the-box SharePoint you must specify security by identifying those users and groups that are permitted to access. It's also true that sometimes it would be more convenient to specify security by identifying who are denied access. However in my experience there is a security / permission issue that is of much greater impact in terms of avoiding pushback from users of a new SharePoint-based DM solution.

This issue is around making it easy for a user to make a document Private – i.e. for their eyes only – even though it is stored in the shared environment of SharePoint. This issue can arise when an organization seeks to use SharePoint to replace a previous DM system, such as Autonomy / iManage FileSite or OpenText / Hummingbird D. It can also arise when an organization attempts to force the move to SharePoint instead of private folders.

Users want setting document-level security to be as easy as ticking a box as they save a document to SharePoint. That is what you can do when using a traditional DM system. However with the user interfaces to SharePoint that are available out-of-the-box in the web browser and in Office, it is not at all easy for a regular user to set document level security. So instead of saving sensitive documents to SharePoint, users opt to save them to their C: drive or similar, where they might not be readily accessible by others users, but neither are they backed up or readily searchable. Overall the adoption of the new SharePoint DM solution drops.

In recognition of this human factor, MacroView has created an optional module of MacroView DMF called Enhanced Document Level Security. Deploying MacroView EDLS adds a Private check box to the profiling (metadata capture) dialog that MacroView DMF displays as you save a document to SharePoint. By simply checking that box, even a Contributor-level user can remove the permissions of all other users and groups that would otherwise have access based on the permissions for defined for the destination library or folder.

Thanks to EDLS a user can easily also grant specific levels of permission to selected other users or groups. EDLS maintains the permission level of the existing user – someone who is a Contributor at Library level cannot increase their own permission level, or grant others more than Contributor level access to the document. MacroView EDLS also makes it easy to re-inherit permissions from the library or folder level.

Checking Out, Checking In and Versioning

"I checked out a document from SharePoint and worked on it for three days, saving frequently; when I was finished I clicked 'Discard Checkout' and lost all my changes – I hate SharePoint!"

This cry of pain came from a user in an organization that was looking to replace their traditional DM system with SharePoint. The organization was pleased that SharePoint natively came with 'advanced' document management features such as check out / check in / discard checkout, versioning and unique document numbering. But their users found that these familiar-sounding features did not work as they expected, based on their experience with their previous DM system. The new SharePoint solution was not just different – sometimes it was dangerously different!

You check out and open a document from SharePoint. After editing the document you close it. The out-of-the-box integration of SharePoint and Word then presents a series of three screen dialogs: Do you want to save? Do you want to Check In / Discard Checkout? If you are checking in, do you want to create a new major version, a new minor version or replace an existing minor version?

Users generally find this succession of screen dialogs excessive and confusing. If an unsuspecting user elects to Discard the Checkout (thinking that this is appropriate because they have finished making changes) then SharePoint will discard not only the check out, but also all the changes that they have made since they checked out the document – regardless of how often they might have saved. This is particularly distressing to users that are coming to SharePoint from a traditional DM system, which displays a single screen dialog, which also allows them to save as a new document or replace an existing major version. This is both more convenient and safer than the out-of-the-box SharePoint experience.

MacroView DMF plugs the gap between the raw platform functionality of Microsoft SharePoint and the expectations of more advanced DM users. As you open a document from SharePoint, MacroView can automatically check it out. When the optional Unique Document Numbering module of MacroView DMF is deployed, the screen dialog that appears when you close that document can be customized to look just like that from a traditional DM system:

Figure 4: MacroView DMF customizes the Save As dialog and adds Document ID and Version number to the caption and footer.

Also like a traditional DM system, MacroView DMF can automatically update the footer and captions of Word, Excel and PowerPoint documents so that they display unique Document ID, current Version number and other metadata as required,

To summarise - MacroView DMF extends and enhances the document management capabilities of Microsoft SharePoint to enable solutions that have the functionality and usability that users expect. Meeting these expectations is a key success factor for document and email management solutions based on the SharePoint platform.