Shortly after my article was published, I got a tweet, from Eugene Mandel, who invited me to discuss the whole concept of data mining on Twitter.  That tweet led to a very collegial telephone conversation between us, in which we exchanged ideas which will provide the basis for this article.

Eugene is a very engaging young entrepreneur in Silicon Valley, who has been involved with a number of startups.  His current plans integrate very nicely with my concept of Twitter data mining.  My prediction is that his newest venture, mustexist.com, is destined to become an important tool for Twitter users.  He recently started the company with his pal and co-founder, Alex Sherstinsky, a Ph.D. from MIT.  These guys are very serious players.

Mustexist’s current product is called list tags. It allows one to take a twitter name, and creates a ‘cloud’ of lists in which that Twitter name appears, roughly signifying the importance of keywords in that person’s list audience by the size of the type in the cloud.

Here’s a copy of Glenn Beck’s tag cloud.  I shrunk it down to accommodate the page size, and its contextual importance in this article.  Beck is on some 3800 lists.  Politics, News, and political head the parade, but further down the list are lists named things like ‘nutcase’, and other less desirable things to be associated with.  Mandel goes through the Twitter API, and finds all the lists on which Glenn Beck is mentioned.  Then, he queries those lists and finds the most popular keywords in their title.  Then he ranks those keywords by their frequency of appearance in Beck’s universe of lists, and performs a simple statistical conversion to interpolate the list keyword frequency to the type size in the cloud.

Pretty neat trick, eh?

But there’s more: list_tags enumerates some of the lists on which the Twitter member appears.  Here’s a picture of the TopLists for “Politics” in which Glenn Beck appears.  Now think, for a moment how valuable this information can be.  Many of these “top” lists contain 400–500 members.  These are lists for the top topic which have the largest number of members – regardless of whether or not the subject appears on those lists.  That’s valuable information for marketers, or for researchers and even job seekers.  But more than that– everyone who is maintaining such a list is someone who has enough interest in your subject to go to the trouble of maintaining a list of people who he/she considers are the leaders in the field.  And if that’s not enough, the lists editors is another valuable source of fertile relationships to cultivate.

The second. longer list is a list of which Glenn is a member.  So, in the case of a celebrity, it will not be unlikely that the same names will be on both lists.  I’ll not reproduce the Lists for ‘politics’ because it is too long – and because in appearance it strongly resembles in format the ‘Top List’.

So the stage is set now with a resource now to empower Twitter users with access to many lists.  A little resourcefulness will enable you to harvest those lists, and, by extension, to knowledge of all of their membership.  This will give you first hand access to communicating with the most well-regarded people in any given field!  And with the LIstimonkey resource, you can monitor the conversations of those experts for keywords.  With a little software jiggering and less than an hour’s effort, using our Glenn Beck example, we could identify, harvest and communicate with thousands of political junkies.

Mandel, however, is talking about taking Mustexist to a whole other level.  Using the database he has assembled for the list-tag project, he is planning to offer in the near future an interactive, iterative way of ‘surfing’ the tweetstream of these lists, in much the same was that Google lets you surf for articles.  He made the analogy of a newspaper in our conversation.  Each newspaper has several sections: sport, business, local, etc.  If you accept the proposition that a list’s tweetstream is like a newspaper, then it would be very productive to skim through the newspaper, looking only for articles that interested you.  You could skip the car ads, and dating club ads, and focus on baseball, or international news.

Similarly, over a period of time, a list’s tweetstream will have a lot of chaff: someone’s opinion of last night’s American Idol, inclement weather, mother-in-law coming for a visit…  But what if you could adopt a list, or amalgam of similar interest lists?  What if you could in real time and on an ad hoc basis query the stream for stuff like: climategate?  cap and trade?  world health organization?  BTU content of coal?  And how powerful would it be if you could pursue your line of inquiry immediately, based on feedback you got from the previous question?

I think that MustExist is on the verge of somthing quite important that will add a new dimension of empowerment to the serious Twitter user.  There are currently some 23 million Twitter users, 75% of which are either classified as ‘addicts’ or ‘regulars’.  That’s a reliable 18 million pairs of eyeballs scanning and interacting with Twitter several times a week – sometimes several times a day.  Any tool that can reliably and easily segment relevant portions of that population and then address them is bound to be very successful.

Please add your comments.  It is helpful to hear the opinions of our readers!

Far and away the most important of these meta tags is the <Title> tag.  Left click on your web page and select ‘View Page Source’  from the popup menu.  This will open a copy of the source code for that page in your browser window.  Near the top of the page will (or should) be the Title tag.  Google and other search engines attach a large degree of importance to the contents of this tag, and assume that it summarized the purpose of the page.  It is considered good form for the context of the Title to be consistent with the context of the text elsewhere on the page.  If those two contexts do NOT agree, you have a problem, because Google will deduct points from its appraisal of your page, and demote your page to the lower regions of their rankings for those subjects you are trying to promote.  An even more egregious error occurs when a page has no title.  This is the mark of an unprofessional developer.

Titles should be limited to 10 words at maximum.  And one should bear in mind that the search engines attach more importance to words at the beginning of a Title tag, as well as at the top of the page of text.

The Title tag should contain the name of the firm, and the keywords describing the items you are trying to promote on that page.  In addition, if your business is local in nature it would be wise to include the locale as part of the Title tag.  Don’t waste words like ‘the’ or ‘and’ or generic adverbs or prepositions in Title tags, because robots are trained to ignore them, and because they take up room that can be better employed.

Next in importance is the meta Description tag.  The text in this tag is the description that appears in the Google search results just below the clickable title of the page, which appears in the search results.  Obviously, the text in the meta description tag should be consistent with the text in the main body of the page.  If it is NOT consistent, see above: another deduction in Google’s appraisal of your offering.  Think of this tag as your opportunity to make a small tasteful commercial about your product.  It’s OK to be a little provocative, and give the reader a reason to buy/investigate further.  Meta description text should be limited to 160 characters (the limit Google will read).

Meta Keywords, are now of just marginal importance.  In the past, this tag was a catch-all summary of the keywords in the article/page.  As a result, developers would stuff this tag with repetitions of keywords and phrases that they thought users would search for.  But in recent years, search engines have gotten more sophisticated, and actually the text on the page itself is indexed.  So meta keywords has been relegated to a redundant opportunity for ambiguity.

The meta Robots tag tells the spiders whether or not to follow and index that page.  The default condition is to index and follow.

In our last SEO article, Effective Web Marketing: Common Sense SEO Choose Your Battles and Your Terrain, we made the point that a micro-targeting strategy, using the long tail principle would reward you with more visitors, and more qualified visitors to your site.  In essence, the strategy is to choose very specific key words to describe the product you are offering on any given page, and concentrate on marketing only that item on that page.  We used the example of ‘electric swizzle sticks’ instead of ‘appliances’ to capture a surer position in the search engine for those who were searching for ‘household bar appliances’.

An obvious objection might be, ‘But swizzle sticks are only a small portion of what I sell, and their profit margin is very small!’  While that’s true, there is no real limit on the number of pages you can have.  If you’re in the appliance business, you can have a separate page for blenders, another for ice machines, another for bar fridges, etc.  And each of these pages will be dedicated to the specific appliance you want to target.

You don’t need to sell everything on your home page – in fact nobody really tries to do this anymore.  In fact – every one of your pages on your site is an opportunity to sell a different product or service.  And to neglect this strategy is to condemn your site to be far down the list in search engine offerings for any product!  Now that you have adopted the long tail strategy of positioning your web pages, it’s time to get down to the fine tuning of each page that is important in getting you the results you want.
Returning to our ‘electric swizzle stick’ example, it’s obvious that not everyone who is looking for such an item will refer to it by the same name.  Some are looking for an ‘electric bar stirrer’, some a ‘drink mixer’, still others a ‘hand drink mixed’, and perhaps a dozen other terms, taking into account regional language and terminology differences.

So your job is to brainstorm with other members of your organization, and come up with a list of a dozen or so synonymous terms which you feel that the searching public might enter in the Google search box and expect to see your product.   Sit around, and try to imagine that many ways the public might end up looking for your product.  Often, this is NOT the way you refer to your product.  For instance, if your marketing folks come up with a process which uses ultrasound to hostile the molecules in a drink to make a more thoroughly mixed drink experience, it is not necessarily true that the public will be searching for that ‘ultrasound drink experience’, even though that might be the phrase that the marketers hope to make famous.

Once you have compiled your list (actually, there may be multiple lists because you may have multiple products to be found by different searches), now you have to validate and quantify your brainstorming choices.  The will require you to employ the use of one or several tools and techniques to assist you.  For organic searches (those that are NOT paid), you want to choose terms that are relatively infrequently found in web pages, and relatively frequently searched for.  So if you quantified the number of pages containing each phrase, and divided THAT number into the number of times that term had been searched for over a period of time, al other things being equal, you would select the terms having the higher quotients as your ‘favorite’ key words to optimize for.

Here are a few of those tools available on the web:

Google AdWords Keyword Tool: Enter a search term or terms, to display other keywords related to that term.

Google Trends:Insights into broad search patterns.

Google Insights for Search: Compares search volume patterns.

Wordtracker Free Keyword Suggestion Tool: Returns related terms rated by popularity. Data is collected from the Dogpile and Metacrawler meta search engines.

Trellian Free Search Term Suggestion Tool: Ranking similar to WordTracker

SEO Book Keyword Suggestion Tool: a wrapper for the Yahoo! keywords tool