I wrapped my brain around this question and after noodling it for some time I’ve come to believe you can take most paid tools out there and throw them into 3 essential categories:

1. Basic adhoc tools: These tools are quick and simple. The output is predictable almost to a default; what you put in controls ( and in someways limits) what you get out. These tools rely on very specific searches and the best types of these tools are Boolean based.

Pros: Cheap, quick and nimble, any one can use it, web-based (no installation)
Cons: Difficult to drill down, no data warehouse capabilities, limited data display options ( charts, graphs, etc), poor sentiment, limited historical data, user must know what to search for, lots of spam results

2. Database/ CRM Tools: These tools are for your number crunchers; they integrate easily with an external database ( sometimes yours/ sometimes the tool’s). Usually they also allow you to append additional data to you  searches, like web traffic. These tools are fundamental for a long-term social data storage and provide a central data source to query using internal systems.

Pros: Strong analysis capabilities, integration with internal databases, more robust data
Con: Expensive, high learning curve to adoption, must be embraced by non-social teams, poor sentiment, user must know what to search for, lots of spam results

3. Text Analytics/ Ontological tools: These tools answer the magical questions “What are people saying about my brand that I don’t know about.” Instead of being key word based ( like 1 & 2) these tools are either algorithmically [stats] or  ontologically [natural language] based. Rather than filtering content in or out, like a keyword tool does, they group content into hot topic sections. Essentially they find your everyday topics as well as topics you didn’t even know to look for. They are also great for answering ” what topics are people talking about most?”, ” what are unforeseen potential issues”, “how do people group our products together in discussion?”

Pros: Strong analysis capabilities, unearths hidden hot topics, auto generates categories, superb sentiment results, less spam content
Con: Expensive, high learning curve to adoption, must be embraced by non-social teams, not a solely social media tool & is best deployed in conjunction with other customer facing channels ( service, emails, etc), long  set up process, does not provided base level brand metrics, analysis in not immediate, not a long term data storage solution

In a perfect measurement world a company would have tool from each category type. But, for most companies that isn’t an option. The bigger questions becomes, which of these three type of tools does your company really need? [Read: Which of these tools can I persuade finance to give me budget for]. I wish there was an easy way to preemptively know which you need and which you don’t. There’s not.   Excellence in Social Media measurement like many things in the Social Media space is, well, difficult.

Watch this video. Ignore everything except the equations and how they progress. Yes, some of the metrics are complete nonsense. Yes, subscribers are just like followers. Yes, the video transition effects were nifty. I am asking you look past all of this.  Focus solely on the thought progression to the final equation.

You likely didn’t take notes while you watched that, did you?  If you didn’t, you missed all the messy bits. But, have no fear, because I am going to take you though the proposed equation step by step so you can see it for what it is- (mess x epicfail)/lack-o-judgment.

First, the host introduced you to three major metric categories (volume, engagement, & conversions). Under those three categories she listed several proposed metrics. Do you remember this? Or were you to focused on the super hero t-shirt? I know; it was a great t-shirt.

Now to make following the progression of this proposed equation easier, I broke each of those metrics into two groups. For example, I split the proposed Volume metrics into those that represent Reach & those that show Frequency. For Engagement I parsed the metrics into Time and Content engagement types. Lastly, I broke the metrics mentioned in the Conversion category into Responses & Revenue related metrics.  If you’re reading all this and are confused, the picture below should clear everything up.

In the next section the presenter outlined how you should divide these metrics, regardless of which category they were in within the previous section, into two groups: Hot & Cold metrics. Try to ignore that these groups make about as much sense as splitting the metrics by what which letter of the alphabet they start with. I know it’s hard, but just try.

The metrics the video host allocated to the Cold side are those that fall into the standard media buyers equation (reach x frequency) / timespent.  Additionally, revenue & responses are also fall under  Cold metrics. NOTE: it is not specified where in the equation these metrics belong, just that they belong there. Somewhere. The Warm side contains “harder to measure” Content Engagement metrics such as Sentiment & Ecosystem.  See below for the picture-fun version.

Now does anyone see some major possible issues yet? You should. If not, I am more than happy to point them out.

1. The actual metrics in the previous sections are replaced with less specific category titles. This begs the question why were those category groups originally defined? 
2. Responses & Revenue aren’t included in the Cold metrics equation & seem to be mentioned as an after thought?
3. Many of the data rich metrics in Content category of Engagement  are completely ignored, instead favoring to rely solely on Sentiment analysis
4. The host has introduced a new metric to the equation which was not included in the previous metric categorization: Ecosystem

So after all few more sultry glances and mirage of smoke, mirrors, and excessive metric organization, the video host presented the final equation:

You’ll likely note I added a little color to the above graphic. Here’s why. The yellow represents those metrics the host mentioned previously in each part of the discussion leading up to the grand reveal of the master equation. The plum section signifies metrics that were introduced in the second part of equation definition. Those red highlights those metrics which may have been alluded to, but were not brought out specifically during the previous discussions.

So now let’s play point out the possible problem points again (it’s a fun game isn’t it?).

1.  What are the Page Views & Visits in the Social Media world? Up until this point we have not see anything in this video that mentions these metrics or what defines what metrics would represent them.
2. Where did Frequency & Reach go? Are they supposed to be represented by Page Views & Visits? Are those two things really the same thing in social media measurement?
3. Why did the equation change from Timespent being in the denominator of the Cold metrics portion of the equation, to being an additive?
4. Why are Responses and Revenue removed, especially since they’re the only metrics which track possible financial return?
5. What exactly is the resulting metric meauring?

Now let’s combine everything together into a pretty picture. It’s a beautiful disaster.