Initially, vendors were skeptical about providing information for a vendor neutral, independent analyst operation. Things have changed over the years, primarily because buyers like you have come to trust our evaluations as a complete and objective resource. The vendors are now more forthcoming with access to customers and solutions. It's made our job a lot easier, and it has also contributed to making the research more complete.

This is one of the big reasons I'm really pleased with the 5th edition we released earlier this month. It's been rounded out by lots of input from practitioners who are putting these solutions through their paces every day in large and small deployments and for both simple and complex analytics scenarios. My updates this year were also informed by my colleagues at Semphonic, who spend a lot of time "hands on" with Omniture, Webtrends, Coremetrics, Google Analytics, Yahoo! Analytics and Unica, and much of my own vendor selection work.

While we've seen change in the market over the last year, notably the acquisition of Omniture and Coremetrics, I don't think of this as consolidation. I think of this as part of a continuum in the evolution of analytics that was started when Google acquired Urchin, and continued with Yahoo's acquisition of IndexTools.  Specifically, these moves represent large, well-funded companies purchase analytics because of their perception that analytics can help them add value to their existing offerings and create upsell opportunities within their existing client base.

What I find interesting and perhaps more challenging from a consumer perspective, is that the base line functionality of these tools can no longer be taken in a vacuum, but has to be considered in context with the influence and direction of the larger company. So, the question you face in the future may go something like this: "I like Coremetrics functionality but I also like the idea of being able to embed Omniture tags in our Adobe Creative Suite content. What's the trade off if I go with one instead of the other?"

Trade offs in functionality and price already exist today. What I see frequently is that tool selection often comes down to nuanced differences in functionality and price. Sure, it sounds clever to say "all analytics tools are basically the same" or "you can get 80% of what you need from Google Analytics." But if you look really closely at the solutions you'll find small but important differences...differences that translate into solution acceptance or rejection by your user base (such as ease of use), or complete data analysis (such as ease of analytics data export).  

I think you'll find our evaluations helpful as you begin to explore those trade-offs.

In the context of the debate around a new U.S. Federal government OMB policy on the use of persistent cookies, as well as other national governments' interest in enabling site visitors to opt out of web analytics tracking, Google's stated goal is to make opting-out "easier."  The company has responded by developing a downloadable browser plug-in that disables Google Analytics data collection from all sites using Google Analytics.

Why is downloading a plug-in considered any easier than disabling cookies from within your browser options? Or adding websites to your exclusion lists? 

I can understand why some people may want to opt out -- especially for particular websites -- but it's important to understand that Google has selected the "nuclear" option here in lieu of using its market authority to promote a more modest, site-specific approach. (Competitor Omniture, now part of Adobe, provides a site-specific opt-out service, but doesn't promote it heavily.)  Adopted widely, the nuclear option will add to the myriad difficulties that already compromise web analytics accuracy. The impact will be magnified if Google's approach gets endorsed through the new Federal cookie policy and other government policies.

To be clear: I understand reasonable privacy concerns.  My alternative would be to provide site visitors with the option to use their browser to opt out of tracking from the specific website they are visiting, or opt out of all tracking from the particular web analytics solution, but not automatically default to the latter, as Google has done.

Why would Google promote the all-encompassing opt out? After all, it's the biggest web analytics service provider in the world by a large margin.  Global opt-outs hurt its customers.  Except that its customers don't pay anything for the service, whereas government goodwill is critical to Google's success in many other lucrative areas.  For Google Analytics customers, it's just another reminder: there's no free lunch.  

While some web managers in the Federal Government might wish to use Google Analytics, historically that solution has been more hamstrung than others because of Google's use of persistent cookies.

If you practice web analytics in the Federal government you know that persistent cookies are verboten by order of OMB -- unless you get approval from the director of your agency. In the real world of agency politics, that's like reaching the Mt. Everest summit. While other analytics solutions (such as Omniture and WebTrends) will certainly provide more accurate visitor data if they use persistent cookies, they aren't dependent on persistent cookies for data collection.

While Washington has seen some debate about loosening up the persistent cookie restriction, there has been no public movement since a flurry of activity and public comment during the summer of 2009. I expect that this will get resolved at some point before the end of 2010, but no one knows for sure.

Meanwhile, Google has issued a work-around.  According to the company, you can virtually eliminate the persistence in the GA cookie by using built-in tracking code functions to set the cookie time out to "0" in the two parameters below: 

Of course, this raises an interesting existential question: is a persistent cookie with time-to-live set to zero still a persistent cookie? Functionally, it becomes a session cookie, expiring when the browser closes.

Federal web managers have a more practical question: Why bother approving Google Analytics unless there was the possibility of being able to use it?  As for whether a non-persisting persistent cookie is acceptable to OMB...well, we'll have to await guidance on that.

There's a certain logic to this argument that I understand. One of the biggest issues that larger organizations have with analytics is getting reports out to people in a timely and coherent manner so that they can use the data to make decisions. You'd think that after this many years, it would be pretty straightforward.  It's a gap that Google Analytics has done a wonderful job at exploiting. But getting reports out to lots of people still doesn't address the gap of being able to understand how to use the data. Once your managers, marketers and content writers understand the basic “out of the box” reports, they want analysis that guides their decision making. So, by following Google's lead, WebTrends is solving only half the problem.

If you're an experienced WebTrends analyst or have one on your staff, there's a decent argument for staying the course with WebTrends and seeing what they come up with in February at their user conference. Based on the improvements already made in the product, using the new interface and alert system will help you with report distribution and doing very basic traffic analysis.

On the other hand, if you’re managing a WebTrends installation and folks aren’t using it, I’m not sure Analytics 9 will address the issue enough for you to justify a business case for continued use of the product. Of course, the same could be said of any fee-based solution, but WebTrends is making a much stronger pitch to this level of user than the other fee based vendors.

As I described in a recent post, there are a few interesting new offerings that address business questions based on analytics data. I anticipate that there will be growth in point solutions that focus on automating the extraction of actionable information out of web analytics data.  These solutions don't ask you to analyze data; they tell you what you need to know. As this level of automation becomes "smarter", the need for web analytics reporting, could quickly lose currency.

Come to think of it, it wouldn't be a huge surprise to me if a lot of the Genesis partnership integrations, many of which are not fully baked, are left to whither. I understand from conversations that I've had with customers that the e-mail integrations are really the only ones that work well. Great opportunity to cull the Genesis program down to what works and forget about the grand scheme. Or just let it mature slowly as the market catches up to it, but focusing on the growth areas in analytics...video, mobile and RIA.

So, if you're a customer how does this play out? My expectation is that things won't change much. Perhaps you'll be getting less pressure from you sales rep to move into Genesis. I don't think customer support will improve...could it get worse? Depends. Adobe says Omniture will operate as an independent business unit, but it will remain to be seen how this might affect headcount or integration with existing Adobe infrastructure.

If you're considering a purchase of SiteCatalyst, I'd normally suggest that you have a heart to heart with your rep, but I'm not sure what they'll be able to tell you at this point. As I said in my post from the other day, it can be a bit unpredictable during times of acquisition. As a company that has done its share of acquisitions and migrating customers into its own solution, it will be interesting to hear what they'll be saying over the next few weeks.

Interesting times ahead...what do you think?

Unica, manufacturer of NetInsight, announced a disappointing quarter of earnings and then indicated that it would be shifting its R&D to India. Is this a sign that the company may be not be financially stable? Personally I don't think so (although I know of at least one competitor that suggested this).  And while there has been much consolidation and churn within the web analytics market, we don't tend to see companies disappear. We see them acquired and their offerings sunset or evolved over the course of time. And the decision to be acquired doesn't necessarily have anything to do with financial stability. Sometimes management just wants to sell the company.

And then there's WebTrends, the oldest independent web analytics firm still doing business. It has perhaps the rockiest history of any vendor on the market. First independent, then becoming a public company, then bought by Macromedia, then bought by a venture capital firm, and going through more management changes than I can count off-hand. WebTrends just came out with a new release that is intended to be the first step in a re-engineered product. But even here, because the company is part of a venture capital firm's portfolio, it is certain that at some point a transition of some kind will occur. How will that impact the work that is currently being done?

Does ownership of an analytics tool by a huge company like Google or Yahoo, ensure stability? Well, remember Microsoft Adcenter (formerly Gatineau)? This was based on the acquisition of DeepMetrix and initially ballyhooed as a competitor to Google Analytics. After little take up by the market, Microsoft decided to discontinue Adcenter support after the end of 2009.

So, no matter what tool you use, and no matter how comfortable the vendor makes you feel about using it, the possibility of being thrown a curve through merger, acquisition, or divestment always lurks. To stay on top of your alternatives, you can subscribe to our Web Analytics research

But it’s the history of web analytics that I find truly fascinating. For an industry that is about 13 years old, there’s been a lot going on in terms of technology and company evolution. In a way, the irony is that the industry evolution has occurred far more quickly and dynamically than the use of the tools themselves.

So, with this as background, I am really looking forward to the Web Analytics Founding Fathers key note presentation at the upcoming X Change conference in San Francisco from September 9-11.

This is going to be a lot of fun (if you’re into web analytics). For the first time, Matt Cutler, co-founder of NetGenesis (first generation log file analysis) and now on the management team at Visible Measures (video measurement); Bob Page, co-founder of  Accrue Software (first generation packet sniffing analysis) and now head of analytics engineering at Yahoo; John Pestana, co-founder of Omniture, and now co-founder of ObservePoint (page tag auditing),  and Brett Crosby, co-founder of Urchin, and now Group Manager of Google Analytics will be together and talking about where web analytics has been and where they think it’s going.

I’m also looking forward to some of this year’s sessions…notably James Robinson’s (from the NY Times) on  Creating a Metrics Driven Organization, and Greg Dowling’s (from Nokia) on Measuring the Mobile Internet.

If you’re thinking of coming, it’d be great to see you at the advanced training workshop I’ll be teaching on Managing Web Analytics on a Shoestring, where I’ll be talking about how you can maximize your web analytics impacts within the organization when you’re short on time, resources, and budget.

Hope to see you at X Change. CMS Watch readers get a 15 percent discount on the conference and training by using the promo code: CMSWATCH

 I have to say, the idea of government web sites using Google Analytics makes me a bit uneasy from a privacy perspective. I just can't get comfortable with the idea of the government passing data to GA so Google can come up with new ways to advertise and market.  

There's a lot of activity going on these days around the subject of web analytics and government as it relates to the Obama Administration Open Government initiative and the release of the Web Analytics Association Report on Government and Web Analytics.

We don't usually think of the public sector as a hotbed of web analytics activity, but change has been long overdue. It's great to see that change may be on the way. Hopefully it will be sorted out so that there is a reasonable balance between analytics value and personal privacy protection.

The White House blog is inviting comment on how a new cookie policy should be shaped: http://www.whitehouse.gov/blog/Enhancing-Online-Citizen-Participation-Through-Policy/

I've posted my comments on the site and have reprinted them on:http://wam.typepad.com/wam/2009/06/us-federal-government-cookie-policy-under-review.html

My recommendations for a new policy are fairly straightforward:

• Allow the use of first party, persistent cookies for Web site measurement.
• Prominently disclose how Web site measurement is used and how the data is collected and analyzed.
• Provide instructions for how users may delete persistent cookies from their browser settings.
• Combination of PII and unique visitor ID (persistent cookie ID) will not be used for analysis.

What do you think? Is this enough? Not enough? I'm all for privacy protection as those of you who've read the commentary I wrote with Tony: www.cmswatch.com/Feature/191-Data-Ownership

However, I would advocate a balance between privacy and analysis, so that Federal web managers can provide more effective sites.

What's your take?

But it was the earnings call that was telling. Here are the quotes that jumped out for me:

" ...over 1 trillion transactions (are) being captured"

"A very small percentage of these 1 trillion transactions are being optimized today"

"...customers now have an average of 1.39 [Omniture] products per customer."

Taken together this can look like a glass half empty or full. In Omniture's favor, with its suite of 10 products (depending on how you count), one would think there's plenty of room to cross-sell -- but on the other hand, it seems very few Omniture customers are taking the company up on that.

And what are those products? SiteCatalyst is Omniture's core analytics product. DataWarehouse is usually the next "must have," followed by Discover, and then one of the marketing and conversion modules, such as Test & Target, Search Center, Merchandising, or a Genesis partnership.

So, Omniture has created a great vision of how analytics should be the common thread among all web marketing. But if you're an Omniture customer, are you ready? Most of the Omniture customers I speak to are not at the point of adoption beyond Discover. And the Genesis partnerships? These take commitment and resources to get underway. As we've reported in our Web Analytics Research, this isn't quite the "plug and play" process touted in the marketing material.

I think Web analytics software has evolved more quickly than the ability of most people to use it effectively -- and justify paying for it. Interesting factoids from the Web Analytics Association 2009 membership survey released in January 2009:

- Over 62% of respondents said they have fewer than three years of web analytics experience.

- Optimizing web site functionality and analysis of past performance both scored over 70 percent as the purpose of web analytics in organizations

- Measuring Web 2.0 Technologies, targeting and segmentation and A/B and multivariate testing scored highest as subjects that people had the highest interest in learning more about

So, if you're an Omniture customer, I wonder how long it will take for you to make a sound business case for purchasing the other products in Omniture's suite. Certainly optimizing site functionality and analyzing performance comes within the purview of the basic web analytics tool kit; so does measuring Web 2.0, segmentation, and A/B testing; targeting as well. Do you really need the other modules? I'm thinking in most cases and in the near term, probably not.

Are there any other reasons that might be impacting Omniture's growth? Don't discount the Google Analytics factor.

This is a high-profile standoff around the value of analytics vs. personal opinion when it comes to web design, but the underlying conflict is quite common. Those on the design side may say that Google has gone too far with data, I know for a fact that design decisions being made by "the boss" on a unilateral basis is a common scenario. You know the one I'm talking about....where "the boss" says he wants the color red, even though the professional designers he has hired contend that the color should be blue. And even though there's user experience testing to back the professional designers' decisions.

Yes, it is one of the benefits of analytics to put these types of debates to an end. Analytics provides the opportunity to test the colors and provide the validation one way or the other. So, while I totally get the idea that a designer would rather not be constrained by analytics, I suggest that this is part of the "new world." Somewhat like the world content managers are in when it comes to proving web site effectiveness. Or the additional accountability marketers must adhere to in the online world versus the offline world. All because now, we can measure what's going on, and coordinate this with usability testing to find out why it is happening.

There were many good discussions around web analytics at the conference. I've added a few more takeaways at http://wam.typepad.com/wam/2009/05/web-analytics-insights-from-jboye-09.html.

The API enables a download of Analytics data in the form of Google Data API feeds. The client application uses the Data Export API to request data from an existing Analytics profile for an authorized user, and refines the results of the request using query parameters.

Here are a few examples that I found intriguing:

- A heat map of purchases per keyword search referral by country based on crossing Google Analytics with Mathematica and Dabble DB.

- A long-tail search analytics tool from Juice Analytics that exports Google Analytics keyword data and applies a pattern identification algorithm that condenses the long tail of search into keyword phrases with similar structures.

- A CMS analytics application by Axiom that enables content managers to see metrics for the pages they're updating within the CMS itself.

So, the bottom line here is that if you're a Google Analytics customer, you'll have potentially some powerful new applications to consider for your tool box. As you may recall, WebTrends announced their API in April, and Omniture publically released their Reporting API in October 2008. Coremetrics, Nedstat and Unica also have data export APIs.  We cover them all in some detail in our Web Analytics research.

Yes, APIs are certainly becoming commonplace, but the key is how to really benefit from them. As my colleague Gary Angel points out, working with APIs and analytics could require a considerable amount of time and effort.

So, in keeping with the quirky nature for how web analytics is evolving, if you're a Google Analytics customer working for yourself -- or a small business, or non-profit organization -- you may get more benefit from Google Analytics' API than your peer at a large company that has licensed WebTrends or Omniture, who is trying to develop an application using their APIs.

But, if you are using Google Analytics, it's equally as likely that analytics comprises but a small part of your job, and you won't have time to really use some of these new API-driven applications created by the community. Ironic, isn't it?

It was a fair criticism, and we decided to address this in the 2009 edition of our research. We asked all twenty vendors we evaluated to provide the same point-by-point explanation of their solution in the manner set out by IndexTools' (now Yahoo! Web Analytics) Dennis Mortensen.

The results: 50 percent of the vendors responded.  They are
Amethon
Bango Analytics
Coremetrics
Google Analytics
Intellitracker
Lyris HQ: ClickTracks
Mobilytics
Nedstat
Unica
WebTrends

We put all their responses into a matrix that's included as an appendix in our report.  But you can now download the matrix here for free.  See our release about it here.

We were pleased to see that most vendors we contacted wanted to participate. At the same time it was disappointing that not all did, especially mega-vendor Omniture. Omniture indicated that they would be releasing this information on their own sometime this year.  We gave a pass to the video analytics vendors, Visible Measures and GlanceGuide, since the WAA Standards did not apply to them in a complete manner.   While there is certainly debate among web analytics insiders as to the relative value of the Web Analytics Definitions, we think you can derive some “on the ground” benefits from using them as a reference in a couple of key ways.

1. Vendor evaluations
You’ll see when comparing responses that not all vendors answer "Yes" in their being able to calculate all of the WAA standard metrics. This may come as a bit of a surprise to those who have sat through demos where vendors spin out beautiful reports, or to those who don’t really think about their metrics requirements until after the vendor selection process. For example, Repeat Visitors is not a metric you can get in Google Analytics, Lyris HQ:ClickTracks or Nedstat. This becomes even more interesting in viewing the responses of mobile analytics vendors who indicate that the metrics are collected, but not available in reports.And, those vendors who do indicate compliance with the WAA standards, may choose wording that you not find to be entirely clear, or may by omission of details, invite additional questions, such as: "What do you mean by that?" Or, "Can you explain that in more detail?" Or, "Could you give me an example?"  The definition of a "Page" provides some illustration. Most vendors answer that their definition of a Page is the same as the WAA, "A page is an analyst definable unit of content." That in of itself is a bit obtuse, but the vendors by and large don’t try to explain in any detail how "they" define a page; much less files or media content. Only WebTrends describes in their definition, "Images, sound files and video are not considered pages."

You may also ask vendors whether they provide reports that are compliant with the definitions "out of the box", or whether they require calculation

2. Report analysis
How many times have you been asked by your web analytics report consumers, "how are unique visitors calculated" or "how is visit duration defined," or a range of other questions that are central to coming to terms with report accuracy and intention? Referring to the vendor responses provides a basis for your reference, as well as establishing a benchmark for your own report development. You may also find that using these definitions as the basis for report specifications prior to development helps educate your report consumers and helps to constrain multiple iterations of report review cycles caused by a lack of understanding of metrics definitions and terminology.   Are the Web Analytics Association Web Analytics Definitions a gold standard? No, but they are a useful tool in your evaluation of vendors, as well as your understanding of the metrics and reports you can produce once you have the solution in production. We invite you to take a look for yourself and check out how your current solution stacks up on its own and against the other vendors. We think it makes for some interesting reading.

Again, you can download the WAA Compatibility ratings here.

Things have moved a more slowly on the product development front. While the openness vision has been articulated at industry conferences such as eMetrics, it wasn't exactly clear what this meant -- other than that the company's "Marketing Warehouse" was the key to data access. So, there was great expectations for Engage, WebTrends' customer conference last week.

Now that the smoke has cleared, let's see what happened:

A Big Pre-Announcement

Specifically, a New WebTrends Open Exchange Yes, Open Exchange has been around since September 2007, but primarily as a partnership program. The revamped Open Exchange will provide API access that builds on WebTrends' Tag Builder functionality. At first impression this is all a good thing: a fully published API should help extend data export to third party solutions (like Web CMS tools); provide an alternative to the ODBC driver for exporting data into Excel; and enable better access to both Analytics and Marketing Warehouse data. These last two points are significant because -- as our Web Analytics research subscribers know -- for years WebTrends has insisted that there are no issues with the ODBC driver even though many in the marketplace have expressed unhappiness with the product. On the second point, there seemed to be an overemphasis on developing additional reporting and querying capabilities for Marketing Warehouse at the expense of the core Analytics product.

Is there a catch? Well, the critical component of the web services is the Data Collection API, and this is available in beta only right now, with general release for Analytics slated for June and for Marketing Warehouse in Q4 2009. Any beta users out there? I'd definitely like to hear about your experiences after you've used the APIs to see what you think.

Quick take: Big build up, but with pay-off still to be determined.

An Interesting Partnership

Next, a co-branded Radian 6 offering. Radian 6 is one of the new breed of analytics tools that monitors social media impact. WebTrends announced WebTrends Social Measurement Powered by Radian 6. The promise is that through Open Exchange, you'll be able to integrate Radian 6 data with WebTrends Analytics. However, you'll need to wait on this -- probably at least another quarter for any integration. In the meantime, you can purchase the Social Measurement solution through WebTrends and for now; see basically the same reports as you would if you purchase Radian 6 directly.

Quick take: Big build up, with pay-off to be determined.

A Packaged Approach

Finally, a Solution-Oriented Vertical Offering. Throughout its history, WebTrends has lacked cohesiveness to its product and service offerings. Yes, it could do a lot, but it's been difficult to figure out what part of the solution would fit your needs. The company is addressing this with eight vertical focused solutions. Nothing like the Coremetrics reporting or the Omniture playbook approaches, these service/solution offerings are presented as complete offerings that encompass strategic, technical and tactical components. Of course it will be curious how these complex engagements are estimated, planned, delivered and managed.

Quick take: Big build up and pay off to be determined.

So, You Tell Me...

As I mentioned, WebTrends is doing a lot these days to convey a new image. Management is very forthcoming with information, and you can sense a real energy -- so different than years past. But I wonder what you the customer have seen, in your demos with WebTrends, and your experiences with tech support. Do you think the company has changed for the better? Is this management enthusiasm and sense of purpose evident in your day-to-day dealings with the company? Or is it pretty much the same old, same old?

The next few quarters should be telling as to whether Engage will be seen as the public turning point for the company, or simply a pep rally for the company faithful.

Why was it a great conversation?

He understood the importance of web analytics to his organization, and made the commitment to support web analytics with his resources and authority.

He stated that web analytics is a discipline that all content managers need to use to plan new content and remove old content. He views web analytics as a “value add” service that his office can offer to internal stakeholders. He committed to having his development team learn how to tag web sites, and wants to encourage site owners to initiate web analytics on their sites. He understood the business benefits of his IT staff moving out of the log file processing business and into a SAAS solution that enabled better access to analytics data by potential analytics consumers throughout the enterprise.

Yes, I found this conversation to be somewhat of a revelation because the ederal government is by and large at a very early stage in using web analytics. This is an environment where older log file solutions are commonplace and entrenched, where IT departments still “own” analytics, where reams of page views and visit reports define web metrics. I speak with Federal web analysts who have literally spent years trying to institutionalize web analytics at their agencies and have met extreme resistance and/or apathy. The common denominator…no senior management support. True, lack of senior management commitment hampers many web analytics programs in the private sector, but the impact within the Federal sector seems to have more impact. Historically, there has been no revenue-based goals driving an interest in web analytics. Fortunately, this seems to be changing due to tighter budgets, and zero based budgets (ZBB). According to Forbes Investopedia:

ZBB allows top-level strategic goals to be implemented into the budgeting process by tying them to specific functional areas of the organization, where costs can be first grouped, then measured against previous results and current expectations.

If Federal CIOs needed a business case for institutionalizing web analytics as a program office, the ability to tie all web initiatives to a ZBB model and initiate accountability through meaningful web metrics seems to make a lot of sense.

Will Federal CIOs take a leadership role in developing web analytics programs at their agencies? I hope so. While there has been a lot of attention about cookies and privacy, I think that in the scheme of things, this is a somewhat overstated issue when it comes to making web analytics useful within the Federal government. (For a longer discussion of cookies and how they're used or mis-used across solutions, consult our Web Analytics evaluation research.) The issue of importance as I see it is in establishing viable and relevant web analytics programs within Federal agencies. After all, if you resolve the cookie or no cookie issue, and there’s no senior management commitment to support a web analytics organization, no one to implement the web analytics solution, develop meaningful metrics, create reports and provide analysis, then what’s the point whether persistent cookies are used or not?

Case in point:  Microsoft's "adAnalytics" beta service originally launched with great expectations of duking it out with Google Analytics -- only to be quietly shelved earier this month. (Current customers will receive support through the end of this year.) While Redmond didn't give an official reason, I couldn't ever find a really big following for the solution. Indeed the lack of response to a feedback request on the adAnalytics blog back in July makes you wonder whether adAnalytics ever garnered much of a clientele.

Given Microsoft's weight and ambition to challenge Google, as well as Yahoo's evident potential in this space, I'm surprised that adAnalytics remained such a non-factor in the analytics space. If you have theories, feel free to comment below (just remember: no anonymous posts, please).

Perhaps adAnalytics will be best remembered for its ability to provide a limited amount of demographic data on about 25% of users based on MS Passport Live IDs -- an intriguing concept, but not complete enough to draw any meaningful conclusions about your site traffic.

While it may be entertaining to speculate on whether Micosoft will pursue the analytics game going forward, I'm not sure there's enough business value for Redmond here. Sure, companies such as Omniture, Coremetrics and WebTrends are all possible acquisition candidates, but if it's data that they want, Redmond can get that through the rest of their web and PC-based applications and services.

Anyway, just goes to show you: big vendors kill products and services just as freely -- if not more frequently -- than small. Plan contingencies accordingly.

The Public Sector Special Interest Group of the Web Analytics Association (WAA) started asking these and other questions -- both to find out whether web analytics in the "non-private" sector is really that different than the private sector, and to help web site managers, marketers, and analysts develop benchmark metrics, usage, and presentation methods that are common to this corner of the web universe.

The result is the Web Analytics Association (WAA) Public Sector Web Analytics Survey. I like to think that this is the first step in developing a greater presence and recognition for this segment, which often seems to be regarded as somewhat of a "backwater" by the larger web analytics community. I'm hopeful that the survey results and communication of the results will help foster a greater dialogue between the public and private sector web analytics communities. And wouldn't it be cool if there was enough participation among public sector organizations to ultimately foster the type of index we've seen in the retail area, such as the Coremetrics Benchmark?

If you work for a public sector (including military), not-for--profit, or educational organization, take the survey now; you don't need to be a member of the WAA.

http://www.surveymonkey.com/s.aspx?sm=QWRWnODN1hbqLUaNyy52Lg_3d_3d

Warren makes the point about lack of transparency into "free tools" when he writes,

The fact that I can't have access to the raw data, the logs, has bothered me everytime I came across a problem. I just cannot accept that the only validation criterium of the numbers I'm seeing is... the brand! “It's right, because Google says so.”

Now, it is true that Warren, who is one of the smartest web analysts out there, probably understands the intricacies and nuances of web data better than most people, but there are broader issues around data trust that Google's situation is just exposing. Yes, if you use Google Analytics, you can't easily address core data accuracy questions. But does it become easier or more difficult to do than with a tool that has more apparent data transparency?

Ironically, many enterprises use Google Analytics to “audit” their fee-based analytics tool. I don't quite get the logic in this. How about doing an audit of the data collection for the fee-based tool that has already received investment?

In an odd turn of events, I have to wonder if companies' dual deployment of Google in addition to the fee-based solution has actually helped perpetuate poor implementations of fee-based solutions. It used to be that when organizations got frustrated with their analytics solutions, they'd be more prone to switch solutions. Since the availability of Google Analytics, they simply use Google Analytics in tandem. I've got nothing other than anecdotal evidence to prove this, as well as using Stephane Hamel's WASP, but I'd be curious to hear what you have to say about this.

Hamel's comment makes a good point regarding the concept of web analytics maturity within an organization and use of the “right” solution. This is something we've sought to educate readers of the Web Analytics Report through the use of business and resource commitment scenarios. I also share Stephane's concern that “as the power of Google increase, the options are becoming scarcer.”

Then again, although this economy may impact recognized commercial vendors in the analytics space, I think that there will be a growing number of organizations and analysts who will mature into appreciating the value of fee over free.

The ensuing discussion had dozens of posts with most of the focus on reporting features and capabilities.

But it was what didn't get mentioned that caught my attention: data usage as a differentiator. (This is a topic we explore in the Web Analytics Report.)

Google and Yahoo! use the data they collect from your website as fuel to develop new products, services, offerings and marketing programs. Omniture and all other fee-based vendors do not use the data they collect.

To be fair, both Google and Yahoo! clearly state how they use the data in their Service Agreements. Then again we all know about carefully reading the fine print.

I just find it surprising that folks tend not to mention data usage as a differentiator in the many World vs. Google Analytics/Yahoo! Analytics discussions or vendor evaluations that occur. The question for you is: does "free" makes it OK? For some enterprises the answer will be "no."

In my last post I mentioned that I'd be curious how web analytics vendors might respond to Yahoo! Web Analytics at the eMetrics Summit in Washington DC. In fact it was Google Analytics that "stole the show" with the introductions of a host of long-awaited features, including an API and Advanced Segmentation.

The buzz at the conference was overwhelmingly positive, and both independent practitioners and Google Analytics partners have been providing plenty of color commentary on the features and cool things available, and so on and so on.

From what I've seen, I agree for the most part...sure I'd quibble about the value of a segmentation capability that focuses on visits and not visitors, and I found others at eMetrics who discussed issues they had with how Google Analytics cookies identify visitor status through a session or with how the segmentation report presentation was not to their liking, but this is almost besides the point in looking at how the analytics vendor landscape is changing.

On one hand, there is, and will continue to be fragmentation in the market because web analytics has a low barrier of entry for the creation of new tools. This was my take after the last eMetrics, and this continues today.

On the other hand, Google and now Yahoo! are using web analytics as part of online product platforms to drive revenue to their advertising and online store products. AdMob and Bango Analytics are doing this in the mobile analytics space; AdMob with its advertiser network and Bango with its payment services. This is all part of the "platformization" in web analytics.

OK, not an elegant term, but something that many of you will find an attractive option in looking for an analytics solution, although hard to find outside of Google and Yahoo!. You could say that Affinium NetInsight in addition to one of the other products in the Unica suite, such as Campaign, meets this criteria because it's part of a larger platform.

Now, I'll bet that I'll be getting vendor e-mails pointing to all their partnerships and so on, but I don't think these partnerships have a clear value proposition associated with them because of the customization that needs to occur to make many of them work. There are inconsistent levels of productization and a great deal of competition with offerings outside the partner networks. So, there are no "must have" reasons for purchasing vendor partner options in common areas such as search marketing, e-mail marketing or content management.

In my opinion, easy integration of web analytics data with other data and web tools, such as those listed above and the ability to do complex queries into the unaggregated data sets should have been one of the top priorities among analytics vendors. This was possible even with the NetGenesis tool I used in 1998, but has been such an issue with the SaaS and page tag vendors. It reminds me of how the US car industry spent their time in the 1950s putting tail fins on cars instead of making cars more efficient. Now with Google and Yahoo! both offering fairly robust analytics tools within their platforms, the rest of the industry is starting to look tired and in a game of catch up. I'm not going to say that there aren't bright spots, but why did it take Omniture so long to open up its APIs? Why is WebTrends all of a sudden preaching the importance of getting data out of their system? Why did it take so long for Coremetrics and Nedstat to provide query access to their databases?

Some might argue, myself included, that there hasn't been a groundswell of market take up for advanced features, due to lack of organizational understanding regarding the business value of analytics. But I'd further argue that vendors haven't helped this by generally focusing on product features, rather than real education...not just the kind that conveys how to use the tool, but demonstrating the value of the tool.

I'm not given to crystal ball predictions, but I see some tough times ahead for these companies if they can not focus on and execute on a value proposition that validates the reason to pay for primarily stand alone tools in a world that is going to both "fragmentation" and "platformization."

What do you think? I'd love to get your take.

Readers of the Web Analytics Report, and my post about web analytics leadership know that I'd agree with the comment about a competitive marketplace in analytics.

This was borne out in even starker terms by the long awaited announcement later in the week on the launch of Yahoo! Web Analytics, the solution based on IndexTools. Yahoo! Web Analytics is a no-cost solution that is geared to serious web analysts who want to do behavioral segmentation on unaggregated data. If you're an experienced web analyst, this would be worth a closer look.  One important note to this: Yahoo! Web Analytics is not freely available to the entire market. As described on the Yahoo blog, the initial rollout is limited to Yahoo! Small Business' 13,000 hosted e-commerce customers, Yahoo! Web Analytics out to advertisers who seek Yahoo!’s help to build custom micro-sites, and third-party application developers who build widgets and other mini-apps for Yahoo! users via the Yahoo developer network.

Just as Nedstat raised the bar by offering analysis across unaggregated data at no additional cost in its "Live Segmentation" release, the Yahoo! Web Analytics offering puts pressure on all web analytics vendors to provide what had been previously considered a premium option, at no cost. Web analytics vendors have publically stated that Google Analytics didn't cause much concern because it is considered more of a reporting product than serious analytics platform, however, the Yahoo! Web Analytics offering potentially creates pricing pressure at the higher end of tool functionality.

So what's the next move from the web analytics vendors?

Look to see a more meaningful push to integrate web analytics data with third party marketing vendors from Coremetrics,WebTrends, ClickTracks and Omniture. When I say more meaningful, I mean productized integrations. Also expect to see greater emphasis from Unica and WebTrends on exporting data out of their solutions into corporate data warehouses for use in business intelligence scenarios with non-web data.

The upcoming eMetrics conference in Washington, DC on October 20-23 will be quite interesting, as the vendors will have their first major venue to articulate their positions in the wake of the Yahoo! Web Analytics release.

You see, I'm asking all of the vendors I'm evaluating in Web Analytics Report to explain how they comply with the standards. The timing of this blog was perhaps no coincidence, as I've been working with Cutroni on updating my evaluation of Google Analytics.

While I consider CMS Watch the consumer watchdog of web and content technology, it's nice to see that some of our work may actually help encourage vendor transparency.

As I wrote when Dennis Mortensen matched up IndexTools to the Standards, all Web Analytics vendors should step up.

Now that Google Analytics has taken the step to "go public," perhaps the other vendors will soon follow.

If you're a web analytics manager for a large organization, one of the biggest challenges you likely face in creating enterprise-wide standards is at the very heart of web analytics: data collection.

Typically lots of energy goes into data accuracy, but it's easy to forget that it accuracy is tied to complete data collection. Most issues of data accuracy can be traced back to tags not being applied to all pages, or within applications, or that variables aren't set up to capture the required data.

As web analytics becomes more critical to senior managers, the need to create "apples to apples" comparison of site performance among different regions, different countries, or multitudes of company divisions increases in importance. To develop this level of macro-reporting, you need to make sure that the data collected from your Japanese site maps to the same variables as the data from your site in Spain, the USA, and so on. This will give you the confidence to develop roll-up views of activity across the entire enterprise, as well as individual site performance reports that may be judged through use of the same metrics and with a data set that all stakeholders agree to be complete and accurate.

To address the issue of complete data collection across multiple sites, I'm finding an increasing number of enterprises spending time developing a global tagging standard. Simply put, this tag includes the key common variables required for standardized data collection to provide agreed upon management metrics. Local teams may then customize the tag with additional variables if they require more advanced reporting. I'm seeing this especially with international implementations of Omniture. One of the gotchas a global tagging standard of SiteCatalyst helps avoid is the possibility of sites using different variable mappings, which could yield very different report results.

Think about it: standard data collection drives standard reporting, which you then can base standard report consumer training, which can then drive enterprise-wide agreement about web analytics definitions and results. Makes a lot of sense to me...what do you think?

Start Making Sense: Web Analytics Metrics and Reporting will be held on 04 November at 13:00.

In this half-day boot camp, you'll learn how to develop effective web metrics and presentation methods that address your business needs as well as those of your management, content managers and marketers.

In addition to the tutorial, I'll be giving a presentation on the web analytics vendor marketplace and be sharing selected research from Web Analytics Report, including new vendors who address mobile and video analytics.