Lots of information - not enough critiques

We always follow the communities around specific products quite closely. Whether an open source or commercial closed source system, product communities can give us great input on the real story. But one thing to remember is that the most active and enthusiastic communities are also a really biased source of information.

Think about it. If someone has spent the last ten years mastering a particular product, and one product only, are they likely to be critical of it? Their expertise is their livelihood, and that livelihood is closely tied to the success of a particular system. Also, it may be the only tool in that category they really know. The effect gets worse when they have gone through extensive training to become fully certifiable. (Pun intended.) And the more proprietary a system is, the stronger the seasoned veteran will proclaim that way to be the best way. Not all product experts are this way, but many follow the pattern.

I'm not going to name examples because this isn't specific to any specific vendor or project. If you browse through the comments on blog posts here I'm sure you can find several, often contradictory, examples of what I mean. Sometimes commenters will be honest in stating their ties to the product's success. But quite often they'll simply deride anything that in their mind seems to attack their beloved system. Pretending that unlike everything else, which "of course" has major flaws, their system is without a fault. Unsurprisingly, these aren't particularly useful voices to listen to.

By contrast, we have nothing to gain by promoting one system over another. We don't work for vendors in any way (we don't speak at their events, we don't write whitepapers for them, we don't get a commission when a particular system wins the selection procedure). That means we can afford to be very critical (though we spend a lot of effort to make sure the criticism is well founded). If you want to get the cheerleading, "this is the most awesome thing since sliced bread" story, you can easily get that from the vendors -- and increasingly their consulting partners. For us, there's little point in putting out a weather alert for a nice summer's day. Our point will often be to warn of the cloud on the horizon and to bring an umbrella just in case.

Some observers don't seem to get that at all. As the proverb has it, "to the crooked eye, all things are crooked." If one commenter angrily shouts that obviously we're against system X and totally on the take of Y, and a week later another reader accuses us of always being pro-X and not taking Y seriously, that doesn't really reflect on us. It reflects on the communities making those allegations. (Which is why we don't usually remove those comments.)

My colleague Tony Byrne has been praised for being "an equal opportunity critic." I like that; it's our job to stand by the content technology buyer, and to warn them of pitfalls. Not because we're pessimists, but because forewarned is forearmed. When our readers use that knowledge to navigate the minefields towards success, it makes it all worthwhile. And I feel privileged to work for a company that values integrity over easy money -- even if that means being criticized for being a critic.

Excerpt from the SharePoint 2010 Evaluation

Enterprise Collaboration & Social Software Evaluation Stream looks at... Limitations of SharePoint

"SharePoint could fit the bill for you, as long as you understand its functional limitations and weak Outlook integration. Where your team members may really feel these limitations is when they participate in multiple Team Sites, and cannot easily merge or roll up tasks and events across them...."
(p. 149)

Get All Collaboration Vendor Evaluations

Learn the real strengths and weaknesses of major Collaboration vendors from around the world, in our Enterprise Collaboration & Social Software evaluation research stream.

Real Story Group. Make Better Technology Decisions.