However just as important is to understand the purpose of the project, and its objectives.

If the objective is to present web pages to explain what it is your department does, then that can be done for a very low cost. But if the goal is to create complex and highly scalable online business applications, then costs will rise steeply. They can soar for a variety of reasons, most importantly that the development work is simply very difficult and time consuming. Just because what the citizen eventually sees is a website does not mean that money got spent solely on some design and a Web CMS; more likely it's extensive back-end business process automation that runs up the costs.

Poring through the detail of the Government's document there are clearly many questions that need to be asked. Questions such as whether current procurement systems that favor huge bundled deals, from equally huge suppliers, really deliver the benefits promised.  Over the past decade there has been a move to consolidate buying activities into as few cycles as possible, ranging from buying government-wide licenses to Microsoft products to mega-deals to build and source online systems through a single vendor.  Though there was likely good logic behind these initial moves, it's time to really examine if the benefits in terms of cost savings and efficiency gains were delivered, or not.

But then that is a different debate to the one emerging in the UK. That debate has collapsed into a a cry from the hordes that if only it had all been done on (take your choice) SharePoint, WordPress, Open Source...then everything would be fine.  Regardless of the fact that the cost of software is typically only a small component of larger projects, and though savings can and should be found, they are seldom anything like as substantial as their proponents would have you believe. The cost of software is not the primary problem.

Delivering online services requires a lot of pre-planning, change management, business analysis, testing and prototyping.  The fact that many e-government services either replace or work in conjunction with existing human/manual services ends up layering complexity on complexity -- plus politics. And when you consider that Government services are expected to meet higher regulatory standards than most (not the least of which accessibility), the costs mount even further. In short the move to electronic services is by its nature costly. What you are doing in reality is building complex business applications for citizens that are exposed through the web.

The move to e-government has been seen as an imperative for the past 10 years. The closing down of locally staffed offices of various government departments has been rapid and the rate of closure continues, to be replaced in many cases by online services.  A more valuable discussion may be had in examining the true value and cost of online services, where they work and where they don't.  At present we seem to be caught between a situation where it is taken to be indisputable that all manual and human based processes are old fashioned and must be automated as soon as possible. Contrast this with the dawning reality that automating  human processes can prove to be hugely expensive, and often less efficient.

The 600,000 employees that the UK government is set to shed in the next year or so will need to be replaced with something. The trouble is that we may not be as far down the technology path as big IT suppliers would have you believe. All we know for sure at this point is that most big e-government exercises have fallen well short of delivering on their promises. Sure we can scale systems to unbelievable heights and deliver speeds and feeds unthinkable just a few years ago, but our ability to navigate and automate well-proven, human oriented tasks hasn't progressed anything like as quickly. The result? Bloated budgets and failed projects.

That investment will not be forthcoming in many instances for the next few years. If that were the totality of the Information Managers lot, then things would really not be so terrible. But a big impact of the cuts will be on public (and politicians') expectations regarding access to information -- and will likely cause more grief. 

Demands have never been higher. For example, FOIA (Freedom of Information Act) requests have rocketed in the UK, and they are becoming ever more spurious and costly to fulfill.  Many FOIA requests come from the same small number of newspapers and reporters and amount to little more than wild fishing exercises.  The perception seems to be that if a government department cannot instantly locate and turn over requested information, then they must be trying to hide something. The reality is more prosaic: it is a hugely costly, complex, and tedious process to comb every electronic nook and cranny to unearth information.  From public officials I have spoken to in both local and national government, the reality is that come the cuts, decisions will have to be made whether to undertake essential day to day work, or whether to take staff away from these tasks and minister to FOIA requests. 

Likewise new web-based citizen access portals will in many cases not go ahead at all, or if they do will be unable to afford to make such portals DDA (Disability Discrimination Act) compliant, and accessible to all.

What the move to eGovernement in the UK has shown us is that though there are great benefits to be gained in terms of empowering and providing access via the web to government services to citizens, that this comes at a high cost.  The myth that electronic is by default cheaper has long been discredited. Government departments large and small are struggling to meet an ever longer list of regulatory and political demands, with an ever decreasing budget, whilst dealing with the unremitting growth of information volumes. Something will have to give.

Smarter government employees are already constructing their defenses, painstakingly detailing exactly why they can no longer deliver some services, meet requirements, regulations, and requests.  From what I can see, the politicians making the cuts have not quite grasped the fact that the "do more with less" boundaries were breached some time ago. That moving forward, their needs to be a recognition that the cuts will not just hurt, they will also cause damage. The equation therefore has to be one of short term pain plus mid term loss equals long term gain?

Yet there is money to be saved, Governments around the world are profilgate spenders on technology. The best place to my mind, both in the US as well as the UK, for saving public IT spending would be in identifying shelfware, renegotiating maintenance contracts, and reassessing the value of ELA's (enterprise license agreements). I have no doubt at all that doing so would deliver far more savings than canceling or cutting back on essential information management projects.

So whether you think these cuts are right and justified, or simply "reckless" as the Labour opposition states, the fact is they are going to happen. And as a public sector information manager you will almost certainly be impacted, yet that does not mean that you are powerless. Stalled or cancelled projects have consequences and it will be your job to detail those consequences, and where applicable to redirect cost saving efforts to more relevant and worthy targets where spending has been truly wasteful.

Advice that may be relevant to all....

Possibly the highlight discussion related to a major university considering a move from LiveLink to SharePoint. Senior management are convinced it will be a simple move and very cost effective, while others have their doubts, on the grounds that 10 years of customization and integration may make the migration phase alone an extraordinarily expensive challenge. I think they may be right...

So, tonight a drive from the northwest to to Birmingham, potentially traffic hell -- but with yet another interesting venue at the end of the drive, the Motorcycle Museum.

To give a practical example, a healthcare professional will need awareness of all the documents related to a particular patient. These documents and records are sorted and managed through their lifecycle as a "Case" even though they may reside in different locations, have different owners, other relationships, and different retention policies.  Other individuals may also need to interact with these documents for the purposes of billing or insurance. Same documents, different purpose. There may also be multiple legal and compliancy requirements to attend to.

In theory at least, Case Management provides you with the tools to pre-define and orchestrate those requirements. Permissions, rules, metadata, and processes all play a part in what can be a highly complex system. 

For some organizations, Case Management applications built from ECM platforms form the core of their business, and more will in the future. The need to better manage the massive volumes of transactional documentation is growing more acute, and Case Management will certainly play an increasingly important role. 

Yet almost more than any other scenario, Case Management demands good information governance and squeaky clean relevant data. Without it everything falls apart. The fact that so many organizations are lacking here is another key reason Case Management is not as widely deployed as it could be. 

Selecting the right software to meet your Case Management needs is difficult, since everyone claims to do it,  but very few do it well.  The nightmare scenario for a buyer of a Case Management system is to buy a vanilla ECM software system and then just bring in a .NET or Java developer. You are not only buying technical functionality you should also be buying deep and very specific domain expertise, and without the right combination of the two you can be in trouble quick.

ECM vendors such as Hyland, Objective, Open Text, EMC, Autonomy and IBM all have deep expertise and knowledge in the particular industry sectors that they design systems for (Pharma, Legal, Government, Intelligence, Healthcare, Insurance,  Law Enforcement, Retail etc). They know (mostly) what works and what does not, and they understand industry specific business processes right down to the task level. You are paying as much for that knowledge, as you are for their software.

Assuming though that you do have your document house in order, and already utilize Case Management, there are some interesting developments on the near horizon -- most notably the use of business intelligence and analytics tools to extract further value from what is already a rich information set.  Consider the possibilities of early fraud and discrepancy detection or new and emerging trend analysis from the very rich data within your documents.  BI has long been locked solely into the 20% of data that is structured in the enterprise, and is a valued tool set. But very large and clean volumes of documentation that have been given a tight structure can be mined these days too, and those documents theoretically at least, represent the other 80% of the data we deal with in business.  In fact some organizations are already starting to use tools like Cognos, Hyperion, and Business Objects in Case Management deployments, and they are liking what they see.

And remember it's our job here to ensure that technology buyers make the right decisions via the use of our research, and one of the best ways for us to do that is to continuously talk with buyers and users who are at the coalface.  So if you are an organization that is using Case Management along with some kind of BI tool, then  I would love to chat with you in confidence to hear more about what works and what does not just drop me a note and we can chat.

Ironically, it's early days for a combination of technologies that have been with us separately for many years. Yet this could prove to be a very good long term marriage.

In the context of the debate around a new U.S. Federal government OMB policy on the use of persistent cookies, as well as other national governments' interest in enabling site visitors to opt out of web analytics tracking, Google's stated goal is to make opting-out "easier."  The company has responded by developing a downloadable browser plug-in that disables Google Analytics data collection from all sites using Google Analytics.

Why is downloading a plug-in considered any easier than disabling cookies from within your browser options? Or adding websites to your exclusion lists? 

I can understand why some people may want to opt out -- especially for particular websites -- but it's important to understand that Google has selected the "nuclear" option here in lieu of using its market authority to promote a more modest, site-specific approach. (Competitor Omniture, now part of Adobe, provides a site-specific opt-out service, but doesn't promote it heavily.)  Adopted widely, the nuclear option will add to the myriad difficulties that already compromise web analytics accuracy. The impact will be magnified if Google's approach gets endorsed through the new Federal cookie policy and other government policies.

To be clear: I understand reasonable privacy concerns.  My alternative would be to provide site visitors with the option to use their browser to opt out of tracking from the specific website they are visiting, or opt out of all tracking from the particular web analytics solution, but not automatically default to the latter, as Google has done.

Why would Google promote the all-encompassing opt out? After all, it's the biggest web analytics service provider in the world by a large margin.  Global opt-outs hurt its customers.  Except that its customers don't pay anything for the service, whereas government goodwill is critical to Google's success in many other lucrative areas.  For Google Analytics customers, it's just another reminder: there's no free lunch.  

Case in point: Today I chatted with the good people of Elkhart County, Indiana, about their experience of trying to consolidate all their document management needs to a single provider. In this particular instance the vendors in question were Laserfiche and Docuware, but the lessons here would be the same regardless of the systems used.

The long and the short of it was that the County was advised by outside consultants to standardize on a single software provider. This is a common enough piece of advice, but it is not one that I give out too often.  The reason being that if you have 10 systems running, consolidating them under one umbrella sounds like a smart idea, but when it gets down to it, the effort may just not be worth it. And that's exactly what Elkhart County found out.

After a year of trying to consolidate two systems together, they gave up. Zero progress had been made, and so they simply continued to use Laserfiche (happily I might add), their legacy document application.  The reasons for the failure were many and complex, but two or three key aspects are well worth anyone contemplating the same move to bear in mind:

1. That consolidating systems requires an awful lot more skills than just document migration; you effectively have to reverse engineer everything.
2. Consultants and SIs who can implement a new system simply don't know where to start when dealing with decommissioning legacy applications
3. Finding people who understand the new system and the old ones equally well is near impossible
4. Legacy systems almost always have had some degree of tweaking away from vanilla
5. Legacy systems often integrate with and support other legacy applications and processes

Put these together and what seems like a simple task of "consolidation" becomes a very complex one -- oftentimes just not worth the effort.

Today it is most often Microsoft's SharePoint product set that is touted as the "one system to rule them all". In the past it used to be Documentum or FileNet. And over the years I have heard many a CTO declare that their organization will in the future consolidate all their disparate ECM needs under one vendor banner. Many have even gone as far as paying for new licenses up front. Many have found that this was a costly mistake to make.

Though not really a surprise that Documentum wants to build up its Case Management credentials in this burgeoning area (just think Healthcare, Government, Litigation and Insurance), it's far from clear why anyone would chose EMC Documentum over any other vendor's Case Management offering.

Case Management solutions vary widely, from pretenders to fully fledged players. Everyone from Autonomy to FabaSoft to Oracle has an offering,  There are even some third-party options available for SharePoint. But how each vendor approaches Case Management differs, since specific buyer requirements differ widely too.  Some enterprise buyers seek good BPM capabilities,  while others seek complex metadata management services, records management and retention or even the relative ability to manage files that can range from physical items through to rich media in the same context.  

There is much talk amongst consultants and vendors of the "commoditization" of ECM, but I see little evidence of that in the real world.  Sure, anyone can offer you check in/out and version control and workflow -- but not everyone does those things well, or even adequately -- and some do them too well and deliver you complete confusion and overkill.

One thing is for sure: one size most definitely does not fit all for Case Management.  Therefore, any buyer without a very solid handle on their business requirements and processes is walking into a selection minefield.

As for EMC Documentum, I gave up trying to understand their ever changing strategy(ies) some time back.  From once being the clear market leader they are now in an ever more crowded and ever more specialized and complex field (we cover about 30 in our evaluation research).  That said, EMC Documentum continues to be a key player, and we will continue to evaluate them, and on occasion recommend them to some advisory clients, assuming Documentum makes sense for that client's very specific needs.

People outside of highly regulated environments typically assume that records and retention management in sectors like healthcare, financial services, and energy is state of the art.  This is far from the case.

Most often Records Management is underfunded,  inadequately resourced, and unloved.  It is no cliche to say it is also often run from underground, literally in basements out of sight of those who need to do "real work." 

In places where records are actively managed, they are usually managed well, with detailed and well-maintained file plans and retention schedules.  But many organizations only manage a fraction of the records they should be.  For example very few records management departments include e-mail as part of their remit, yet e-mail is where all the "stuff" happens. 

But maybe, just maybe, the tide is starting to turn.

Here at the Real Story Group we currently support a number of large advisory customers who are looking at records management strategically across highly complex working environments.  We have others who are looking at the whole issue of information management and ECM more strategically than in the past, and have begun to include RM as a component or a recognized future element of their work. 

That might not sound like much, but just a year or so ago it seemed like nobody cared.  Things like e-government initiatives, healthcare reform, demand or need for more self-service applications, increased regulations, and so forth seldom have the immediate impact people expect. And in a world where decisions can get driven by today's opinion polls, it's easy to lose sight of the fact that many of the biggest and most profound changes we encounter in our society occur at a far slower pace.

The most overused phrase in my personal lexicon is, "time will tell." But I fervently hope that time will tell us that awareness of the importance of RM and archiving has slowly risen to the fore.

"Remember the clip board and all the forms you had to fill in, and remember also that the healthcare professional seemed to have a lot more paperwork in addition to yours?" I then explain that Document Management is the process and management of all that paperwork, often with the goal to improving the process and making the paper electronic.  As a working example it tends to resonate with people, and they typically "get it." 

Of course after that, they tend to say either out loud or to themselves, "Wow, that must be the most boring job in the world." 

Hence I could not help but be struck by a new marketing and sales campaign launched by Hyland this week. They're promoting a new offering that combines OnBase document management and workflow software with kiosk hardware from PFU, a Fujitsu group company. They have launched industry-specific (healthcare, government, higher education) kiosks for customers to self serve their forms and related paperwork.  It such a simple and smart concept that I am frankly surprised nobody has done it before. 

Hyland is one of the vendors we evaluate in depth in our ECM research stream, so if you are one of the organizations out there considering their products check out our critical reviews first. 

As an aside, I also hope is that Dr Gustafson on Ayer Road along with the Nashoba General Hospital in Groton will read this short post and agree with me that self service kiosks might be a good idea. 

To quote:

Workflow services can help minimize the cost and time required to coordinate common approval processes -- but only if the service does what you want it to do, and users don't "work around" the system....

Subscribers to our WCM and DAM research streams can download the workflow paper here.

And over the years, we've learned a hard truth: more details and more complexity in a scoring methodology may not deliver you the right vendor. It can occasionally do the exact opposite.

Ideally you would dispense with spreadsheet scoring entirely, in favor of a more practical, facilitated decision-making process.  However, the latter is not a universal panacea and needs to be carefully managed.  I have seen qualitative approaches become too loose and subjective, and fail to achieve the best-fit solution.

In some cases you may have no choice but to formally justify your decisions with a quantitative rationale. The level of detail and particular approach to this sort of thing can vary. For example the public sector or highly-regulated industries might require more detailed scoring of shortlisted vendor RFP responses due to legal requirements, or quite simply the need to CYA.

If you have a 17-sheet spreadsheet with 238 lines of requirements, grouped into logical categories and scientifically weighted, you might feel you are doing a thorough job.  But your scoring can only be based on what a vendor has supplied you in their RFP response document.  And vendors are experts are filing RFP response documents.

Thus, you might be undertaking a detailed review, but not necessarily an accurate one. Some vendors outright lie (I have caught some out on occasion myself), many if not most vendors massage the truth a bit (or a lot).  Some vendors are honest, and as a result are penalized heavily, leading to the victory of a rival vendor poorly qualified for the work, who did a great job of creative writing on their RFP response. 

One of my personal favorites was a vendor who had scored top (by quite a margin) in a very big records management RFP shortlist.  The buyer -- a research customer of ours -- grew a little suspicious and asked me to independently score the responses using their  methodology.  I scored the same vendor the lowest.  Why? Well, simply because when the vendor answered "Yes, with scripting" I took that to mean "No we cannot," whereas internal scorers scored those responses positively. 

This gets to the other big problem with quantitative scoring: it tends to favor questions and responses about what a product can do (or what the vendor says it can do) rather than what almost always constitutes the real discriminator: how a product works.

In the end, if you are stuck with a heavily quantitative selection process, make sure at least that you don't short-cut the rest of your diligence.  Make sure you speak directly to vendor references, and bring the vendor in to your premises to demonstrate the shipping product.  Just remember you can encounter equally as much slight of hand when it comes to demonstrating products and providing "references" as vendors exhibited playing the detailed RFP game.

Getting the balance right between the right functional versus the right technical fit is hard enough. Finding a vendor you really want to work with, and that really wants to work with you, can be even harder. Take your time and do your homework.

While some web managers in the Federal Government might wish to use Google Analytics, historically that solution has been more hamstrung than others because of Google's use of persistent cookies.

If you practice web analytics in the Federal government you know that persistent cookies are verboten by order of OMB -- unless you get approval from the director of your agency. In the real world of agency politics, that's like reaching the Mt. Everest summit. While other analytics solutions (such as Omniture and WebTrends) will certainly provide more accurate visitor data if they use persistent cookies, they aren't dependent on persistent cookies for data collection.

While Washington has seen some debate about loosening up the persistent cookie restriction, there has been no public movement since a flurry of activity and public comment during the summer of 2009. I expect that this will get resolved at some point before the end of 2010, but no one knows for sure.

Meanwhile, Google has issued a work-around.  According to the company, you can virtually eliminate the persistence in the GA cookie by using built-in tracking code functions to set the cookie time out to "0" in the two parameters below: 

Of course, this raises an interesting existential question: is a persistent cookie with time-to-live set to zero still a persistent cookie? Functionally, it becomes a session cookie, expiring when the browser closes.

Federal web managers have a more practical question: Why bother approving Google Analytics unless there was the possibility of being able to use it?  As for whether a non-persisting persistent cookie is acceptable to OMB...well, we'll have to await guidance on that.

Last week, I blogged about the increasing trend toward specialization in the Search & Information Access space. As you may know if you've been reading our Digital & Media Asset Management Research, the DAM industry is yet another area where specialization is ongoing. One trend that's helping drive the verticalization of DAM is the broader use of video in Web publishing and in enterprise scenarios.

Video, as a digital mode of communication, is nearly ubiquitous. This means video asset management (as a capability within DAM) will assume ever-greater importance in the months to come. If you're in the market for a DAM system, you'll want to think about what this may mean for your overall content management strategy -- and take video requirements into account when shopping for a DAM system.

Depending on the business you're in, your use of video may not be extensive today, but it may well become a key content category for you in the near future. Just as the podcast phenomenon suddenly found many companies in the business of managing MP3 files "overnight," pervasive video will likely find many Web CMS owners wishing they'd thought through the vicissitudes of Flash and MPEG4 ahead of time.

Video is becoming more important in broad intra-enterprise cases as well.  Many (if not most) companies have security cameras stationed in their stores, offices, or on company grounds. What happens to all the security-video footage? In some cases, old material is simply destroyed after a certain amount of time. But it still has to be cataloged and stored short-term (then dispositioned appropriately). Is it safe to just manage such footage in ad-hoc fashion? Maybe. But maybe not. What happens if an employee sues the company after (for example) suffering an accident on the job? If the accident was caught on video, the video becomes a key piece of evidence. What if the employee's lawyers claim that the accident was part of a series of similar events? If archival footage of all similar events, across time, is available (and can be found with the company's search technology), it could decide the case.

Video also plays an important (and increasingly critical) role in health care. Nowadays, at major hospitals, all surgical procedures are recorded, for legal reasons. This results in huge volumes of video files that need to be cataloged, archived, and dispositioned.

Highway-patrol cars are (more often than not) videocamera-equipped. Every traffic ticket, every arrest, every roadside assist, is video-recorded. All of that material has to end up somewhere. It's best if it ends up in a repository, managed.

Law enforcement agencies routinely videotape suspect interrogations. Again, this creates enormous quantities of video information that needs to be cataloged and managed -- preferably in such a way that footage can be semantically searched later on, if needed. According to Herndon, VA-based MediaSolv Corporation (which sells video asset management systems specifically designed for police use -- a prime example of the increased verticalization we're seeing in DAM), 28% of U.S. states currently require the recording of "custodial interviews" (i.e., police interrogations), and fully half of all states have already passed evidence-preservation legislation. This essentially amounts to state-mandated use of DAM.

Take a look at your own organization. Do you see video management in your future? If the answer is "yes" (and it probably is), you'll want to consider availing yourself of our Digital & Media Asset Management Research, where we rate each of the 20+ vendors we evaluate on their video-management capabilities. We can help you get a handle on your media management needs -- even if you're still deciding what they are.

Or as U.S. President Obama was quoted as saying, "This was not a failure to collect intelligence; it was a failure to integrate and understand the intelligence that we already had. The information was there, agencies and analysts who needed it had access to it, and our professionals were trained to look for it and to bring it all together."

How is it possible the dots weren't connected? We've grown accustomed to seeing movies and series where searches instantly bring up all the relevant information. Then there's hush-hush operations like ECHELON supposedly filtering all of our communications (we can say with some certainty it exists, but nobody really knows what it does). So we automatically assume that all the content is there, and through some inconceivably complex software, it leads to actionable information. You and I, of course, have no access to such technology, but certainly the intelligence agencies have it, right?

Maybe. In our Search & Information Access research, there's plenty of familiar software that we've heard about from major intelligence agencies in the U.S., UK and France. For instance, Autonomy will gladly tell you their IDOL is being used by many agencies (among them, the U.S. Department of Homeland Security, and also my local Dutch counter-terrorism agencies). Sinequa is used by the Ministry of Defense in France, and also correlates related crimes logged by Parisian police forces. Or there's the CIA, which has shown an interest in Lucene Solr; so they invested in Lucid Imagination (which specializes in that technology). And therein lies the point: in all these cases, the software doesn't just magically achieve the end result of successful crime or terrorism prevention. It takes work to be sure the technology is pulling the right information together, and even more work to make sure someone acts on it. 

So is a "lowered threshold for information considered important enough to put suspicious individuals on a no-fly list or revoke their visas" the real answer here? In information retrieval terms, that's increasing the recall at the cost of precision. And yes, a large part of this is an information access problem. It's about common problems such as uniquely identifying a person (especially if there are several different spellings of a name, as there was in this case). It's about connecting various databases and (geographically disparate) repositories, and federating search across them. And then once all that is achieved, and the relevant data is flagged: someone has to do something about it.

Your organization's need to retrieve, correlate and act on information may not be in an equally serious domain; still, you likely have a similar challenge. You need to not only decide what information is important, but how it should be weighted, correlated and acted upon. This is an ongoing challenge for those trying to get search right - and it's a challenge that never ends. Your information needs constant tending, trends need to be analyzed, and subsequent courses of action put in place.

It's the complete process of retrieving, correlating, and acting on relevant information -- not just finding it -- that makes all the difference. 

Here's an example that's been tried around the world with varying success.  Say you're running the web presence of some medical organization. As such, you have information on how to deal with various diseases, both on a general level (hygiene and disease prevention) and very specific (what to do when a flu breaks out).

Suddenly there is an outbreak of a new disease; let's say the elephant flu. You could prepare a news bulletin, which would automatically appear on information portals for medical professionals, consumers, etc. -- all the sites targeted to specific audiences for which this news might be of interest.

Better still, since you've already built up a large repository of information, it would be easy to launch an elephant flu theme site: just define the kind of content you'd want in there, and hey presto, with one click you've got an entire site with all the information (http://www.allaboutelephantflu.org). Content specifically on the elephant flu, but also the more generic topics on how to deal with a disease or whom to contact.

You can imagine why this is a compelling concept. Which is probably why I've seen attempts in many different areas, ranging from media companies, governments, product marketing companies, and insurance companies.

But, I said, elusive dream. Many have tried (I count myself among them), and many have failed (unfortunately, I can't really discount myself entirely from that group, either). That's because there are three major problems when you've actually implemented the infrastructure to do it. Since this is only a blog post, I'll pick the most obvious one for now.

The content needs metadata for this to work. Many will tell you that "people won't tag." No, seriously, they won't tag content with the right labels, add the right metadata, or correctly categorize, "even if threatened with being fired." And even if they do tag, it will be haphazard and inconsistent.

This is a very real problem. But at the same time it's complete nonsense. Because if this were the case, why would people meticulously tag and file their holiday snapshots on Flickr and Facebook? Somehow, in their spare time, they do identify the people in a picture, add keywords to a shot, give it a meaningful title, and actually describe it. Without having to be threatened with being fired, or even having to be beaten with a stick.

Partly this is because they get the feedback that makes it worth their while to do so. If you identify your friends in a picture on Facebook, they (and then their friends) will immediately find it and start commenting, which creates a positive feedback loop to tag some more. More importantly though, it's really easy.

If you get back to work the next day, and have to laboriously click ten times, scroll, add, categorize, while thinking what the right category within the taxonomy would be, it all feels like an insubordinate amount of trouble to go through. In most WCM systems and implementations (and dare I say it -- most ECM implementations are much, much worse) it's just too much trouble.

Fortunately, I'm beginning to see some change. There are now quite a few ways in which CMSs can make it easier on your editors to identify the content they're producing:

• Using a "free-for-all" folksonomy, where you can just quickly type in a few keywords. The problem of course, is that the tags will often be wildly inconsistent and ambiguous. Check a tag cloud near you for tags like "New York," "NY," "newyork," and of course, the typo that got away, "new yok." This can be made easier by type-ahead auto-completion of tags. Some systems will start listing suggestions as you type, and helpfully, with some, what you type doesn't have to be the beginning of the tag ("york" will also suggest "New York.") The auto-complete effectively normalizes the tags (i.e., at least all of them will be "New York.") It may still be ambiguous and inconsistent, but at least for many purposes, it'll be workable.
• Using suggestions. Usually with the help of an embedded search engine such as Lucene, the system comes up with tags and related links for your content (based on similar content it finds.) At first, this will need quite a bit of training, but the great thing is that the more content is accurately identified, the better the system gets. The suggestions can be used to completely automate the process, but since you'll still have the original author at hand in the editing screen, you can take advantage of this and ask them to validate the suggestions as they are saving the content. That's a lot easier than having to think them up themselves.

It's still more common to see any folksonomy functionality smoothly integrated into Social Software, and auto-categorization or auto-classification is an area where Search & Information Access systems are usually way ahead. But a few web CMSs are making headway into this territory, as well. For GOSS, which has mostly customers in the UK government, its ability to suggest related content, and categorize it in the IPSV (the Integrated Public Sector Vocabulary) is a unique selling point, and the company has been honing this for the past few years. Hippo (who, incidentally, recently won a large Dutch government account, so there may be a pattern there) is working on releasing similar functionality this fall.

There are others with such capabilities; but at the same time, many are lagging. A system that hides keywords and categories on the fourth tab, three items down under "metadata," and then makes users jump hoops to enter the information isn't likely to help. And some products are so thin on metadata, no amount of customization is ever going to make it work for your users. Carefully check before you buy.

But the good news is that if you share this dream of at least partly automating a metadata- driven architecture, the metadata part of that dream can be realized. Of course, that means there are at least two other major hurdles to take -- but that's enough for now.  I'll return to this topic again...

 I have to say, the idea of government web sites using Google Analytics makes me a bit uneasy from a privacy perspective. I just can't get comfortable with the idea of the government passing data to GA so Google can come up with new ways to advertise and market.  

There's a lot of activity going on these days around the subject of web analytics and government as it relates to the Obama Administration Open Government initiative and the release of the Web Analytics Association Report on Government and Web Analytics.

We don't usually think of the public sector as a hotbed of web analytics activity, but change has been long overdue. It's great to see that change may be on the way. Hopefully it will be sorted out so that there is a reasonable balance between analytics value and personal privacy protection.

The White House blog is inviting comment on how a new cookie policy should be shaped: http://www.whitehouse.gov/blog/Enhancing-Online-Citizen-Participation-Through-Policy/

I've posted my comments on the site and have reprinted them on:http://wam.typepad.com/wam/2009/06/us-federal-government-cookie-policy-under-review.html

My recommendations for a new policy are fairly straightforward:

• Allow the use of first party, persistent cookies for Web site measurement.
• Prominently disclose how Web site measurement is used and how the data is collected and analyzed.
• Provide instructions for how users may delete persistent cookies from their browser settings.
• Combination of PII and unique visitor ID (persistent cookie ID) will not be used for analysis.

What do you think? Is this enough? Not enough? I'm all for privacy protection as those of you who've read the commentary I wrote with Tony: www.cmswatch.com/Feature/191-Data-Ownership

However, I would advocate a balance between privacy and analysis, so that Federal web managers can provide more effective sites.

What's your take?

Let's first consider the positive implications: eXo will gain an audience outside of Europe.  And the JBoss Portal will have some nice new applications (content management, collaboration etc) to add to what was a fairly sparse framework.

As with any merger, the details could become inconvenient for existing licensees.

However, I have bigger doubts that Red Hat will really make the most of the opportunity, and I wonder if the likely outcome of the merger is really what buyers and implementers are even looking for in 2009. It is a merger that makes sense for eXo but maybe not so much for those in need of an enterprise portal.

If we exclude MySQL from the discussion, JBoss (acquired by Red Hat in 2006) and Red Hat, are by far the two biggest vendor names in the open source world. They are the names that come up most often in enterprise procurement discussions as alternatives to the likes of IBM and Oracle. Be it the Application Server, BPM, Directory Services, or indeed their own longstanding version of the Linux operating system -- RedHat and JBoss are synonymous with open source for the enterprise.

But the JBoss Portal has never really figured in that equation. What should have been a direct competitor to IBM WebSphere and the Oracle WebSuite and WebLogic portals, was seldom seen in action. And adding some functionality to the product is unlikely to really change that situation.

I am not favoring commercial systems such as those from Oracle or IBM over commercial open source systems such as Red Hat.  In fact, buyers deserve a truly level playing field from which to chose from, but currently that is not the case.

In fairness to the open source community, it's not for lack of trying. Just in the past year the Liferay open source project hooked up with an enterprise-level vendor (Sun Microsystems), only to see what looked like a very promising and competitive product emerge (GlassFish), and be then thrown into doubt after Sun's agreement to be acquired by Oracle.  Who knows how that one may resolve itself.

Red Hat now has an opportunity to fill that gap, and to give enterprise buyers a more competitive open source option. But the only way to do that is to build a truly enterprise-level portal framework, not by adding new widgets. The problem before wasn't a lack of functional bells and whistles. it was a lack of direction and depth.  Merging with eXo doesn't in itself fix that.

Of course, it's not my job to advise vendors, it is to advise buyers.  My current advice to buyers has to be to tread with extreme caution in the world of enterprise portals. As subscribers to our Enterprise Portal research well know, with Oracle now effectively owning 5 of the 12 major portal offerings and somehow attempting to make sense of them, IBM on the verge of a major upgrade, Vignette acquired by Open Text, SAP losing almost complete interest in the space, and open source alternatives either falling short or themselves going through a period of upheaval, it is a market that is full of treacherous waters to navigate.

Within that service we cover XML authoring tools: the software for creating and editing highly complex documentation, be it DITA structured technical documentation, complex translation work, or product information designed for multiple re-use. But our research goes beyond the editing and authoring options at the front end, and evaluates all the leading CCM (Component Content Management) tools that have been designed to manage the publishing and workflow of these complex XML document components throughout their lifecycle.

On other occasions we separate technologies into different categories, where others may have bundled them together. An obvious case here is our Web Content Management research and our Enterprise Content Management Suites research. Just in the past fortnight I have been asked by more than one person why we have two subscription services for the same technology. Well the simple answer is that it is not the same technology: the technology to manage outward (web) facing content and sites, versus the technology to manage inward-facing (documents and files) content and filing systems is quite different. But nevertheless it is a very fair question to ask, because to someone outside the industry (and even some insiders) that distinction is not always obvious.

Confusion can come about due to vendor branding strategies or a plethora of nebulous acronyms used within the industry. For example Interwoven (recently acquired by Autonomy) did a good job of branding their WorkSite (document & files) offering quite separately from their TeamSite (web content) offering. Other vendors such as EMC Documentum or Open Text use a single brand moniker (such as "digital media") to cover all their content-focused software offerings regardless of each component's specific purpose, while in other instances product names can even be synoymous with the firm's brand, as with ADAM and Sitecore.

Then there is the issue of acronyms. We use the term ECM as it is the most commonly used term for document-based technologies, however in different regions and industry segments ECM is referred to as EDMS, EDM, EDRM, Document Control, IM, IDM even KM. As an advisory firm we have to call our research and services something, and we are faced with picking and choosing amongst a myriad of terms, typically choosing the most well-known term currently in use.

Our job at CMS Watch is to make sense of the complexity of the technology offerings out there for you, the buyer - and that can be a challenge at the best of times. We believe it's safe to say that we have more research available to our subscribers on content technologies than all of our competitors combined. But volume, depth and breadth is not all that separates us from our competitors, we are in addition avowedly a firm that follows the market rather than "makes" it. Thus, we don't try and come up with new acronyms, terminology or market segments, we don't cheerlead for the industry or get involved in market sizing, we stick to the basics of evaluating current release products side by side and evaluating the importance of current trends.

So here is my challenge to you. We evaluate over 200 products and sub-divide those into 10 subscription services, which in turn are sub-divided into about 5-8 sub-categories each. We try to use common terminology so that you the buyer can relate to and locate the research that meets your needs, but that does not always work as well as it could. So if you find the product evaluation(s) you need, but locate them in a place - or grouped in a way you find 'surprising' or 'interesting', let us know. Tell us how you and your colleagues perceive these technologies, what you call them, how you would categorize them differently, how you explain them to your internal clients or project teams. Truly we would love to hear from you as this industry remains a dynamic and very moveable feast at times....

The 'Records Management Services Technical Specification' provides a standard for the Federal US Government, but just like DOD 5015 this might well have much broader reach and value outside of Government. In essence this is a SOA (Services Oriented Architecture) specification that provides a framework for defining and ultimately re-using web services throughout an enterprise to provide full end-to-end record lifecycle management.

As subscribers to our ECM Suites research know, RM technologies are often limited in their reach to particular repositories or systems. The theory here is to provide some common web services that can be utilized wherever RM functionality is needed, regardless of location, and to enable truly centralized RM. It's a lofty but important goal, and frankly RM as of today is not a consideration in most SOA architectures. If this succeeds as a standard that could start to change.

I for one hope it does succeed, but without further studying the documentation and seeing it tested in the real world we can't know for sure how well it might work or be adopted. Nevertheless, it is an interesting development and one we will watch with interest.

EMC has made a developer edition of Content Server freely available, and they have also provided a similar free developer environment for xDB. At the same time, each free developer edition will have an accompanying online community that provides code samples, tutorials and full documentation.

All in all, this represents quite a significant turnaround for EMC Documentum and one that is to be applauded. I can speak from personal past experience of the nightmare of finding basic technical advice regarding Documentum releases, and being reduced to scouring the web for potential (and often unreliable) information. This announcement will also provide a sigh of relief for those who have had to pay license fees for Documentum development environments. More importantly it shows EMC 'opening the kimono' (as that awful phrase goes) more widely than we might have expected.

We often comment that Microsoft has also gone from being a secretive firm (even about current releases) to (in the case of SharePoint) a very positive level of openness, community support and involvement. As always it will be time that decides the success of the EMC initiative, but I suspect it will be just as successful as Microsoft's community initiative around SharePoint, as there is an established, vibrant and very sizeable Documentum community out there who have long awaited this day.

From today's release:

Web CMS vendors and Social Software suppliers can talk a good game about offering a unified solution to this challenge, but their architectures have not caught up with their marketing here.

Today also inaugurates a kind of "dot-release" for our Web CMS Report 2009, as we move to a rolling update schedule with vendor evaluations revised more frequently as developments warrant. We've updated several major vendor chapters, including Oracle, Alfresco, FatWire, and more. If you subscribe to our Web CMS research service, or you purchased the report singly during the past ten weeks, you'll receive this update automatically.

GCN: What do agency systems developers sometimes forget when purchasing an ECM system?

BYRNE: ... The third thing is a tendency not to test the tools properly before they sign a contract. The federal government is way behind the commercial space here and doesn't need to be. There is a misperception among contracting officers that the Federal Acquisition Regulation doesn't allow this, but the FAR actually does. Some of the smarter agencies do these [competitive bake-offs], but too few of them do.

As someone committed to the value of proper research, I'll always suggest doing your homework before you start to procure technology, but then -- after getting as smart as you can -- be sure to test....test....test...before you buy.

Of course, it's the job of the SI to keep scope under control, if they are going to finish on time and under budget. But the tenor of the project has changed. To continue the marriage metaphor that I frequently roll out to describe vendor selection best practices, the honeymoon is long over, baby.

I leave it to real experts (like Graham Oakes) to explain the various ways to keep projects running smoothly and meeting business objectives. It's been my experience, though, that even with the best project controls (and there are some good ones in place here), there comes a time in a systems project where the conversation turns subtlely but fatefully towards what the software package can and can't do, rather than what the business needs to accomplish. It usually starts with the implementation team and soon carries over to everyone else. Some of that attitude is really just being practical, and doubtless some of it reflects exhaustion with the process. A savvy project leader on the customer side recognizes this as a marathon, and keeps reserves of energy to advocate on behalf of original objectives.

To me, this also reinforces the primacy of due diligence and broad testing before you sign on to a solution. In a large project, you are going to encounter surprises and disappointments. The question is, how big...and how often?