DataSift debuts a way to find the tweets you need

A Twitter data organization tool zeros in on tweets that impact your business the most

With social media outlets exploding with real-time feedback on countless events and topics, businesses are scrambling to find ways to harness the pulse of their audiences, but there is so much noise that they have to sort through to get to the valuable data they are after.

A data organization service launched Wednesday to help companies interested in using information from Twitter to better their brand and consumer awareness.

DataSift, has spent roughly the last four years creating a software that would allow simple coding commands to filter through the billions of tweets and organize them for easier analysis and action.

The San Francisco and UK-based company is one of only a handful of companies that has access to the entire Twitter fire hose (not just the tweets but the information on each user's profile and Twitter history.)

Since its inception, DataSift has gathered together $8 million in funding from various investors and financial groups and is gearing up to staff aggressively within the U.S.

The co-founder and CTO Nick Halstead has been a passionate coder since he was a child and wanted to harness the immense information available on social platforms, such as Twitter, but it took a while to get the system to the point were it could launch.

"As Twitter grew, it weathered some serious pains and people became familiar with the infamous fail whale," said Halstead, who has founded and worked for several technology companies, including Techmeme. "We wanted to make sure that we had the infrastructure to handle the sheer data we were going to be sifting through before we went live."

What it does

A client or company that wants to track a particular trend or discussion topic through Twitter would create an account with DataSift under one of two payment structures:

1. On demand -- where the user will pay for the data that returns from each search and meets the criteria set forth. This pay system would often look like $0.10 per thousand tweets returned.
2. Prepaid package -- where you choose the budget you wish to spend and limit the number of searches you aggregate from.

Depending on the level of difficultly and the magnitude of the searches a company is interested in, either someone within the user's company can manage the searches or a DataSift employee would spearhead the project.

If a company, say Starbucks, is interested in getting feedback about a new special they are running in particular regions, a DataSift search can be created that target geographical areas, key search terms and then analyzes whether the responses are positive or negative. The data can also understand and sort through 31 languages.

Overall reports on the general public sentiment or the amount of feedback for a given search can be found in mere moments and then, if the client is so inclined, they can dig deeper and look at each individual response or organize them based on gender or location or the level of influence they have on Twitter.

This can be a very powerful tool for companies that want instant feedback or are looking to reach out to people that are most influential about their product.

"We are able to go through this massive mound of data and take out all the spam and duplicates that make up the noise," said DataSift CEO Rob Bailey, formerly of SimpleGeo. "And use very precise tools to give users insightful feedback of what people are thinking and saying."

How is it different from other search tools

Because DataSift has access to the entire Twitter fire hose, it can search through far more data. What, you may ask? Well one key element is the shortened url links that are all over the Twitter ecosystem. Standard searched on Twitter, or through sites that run more advanced searches, cannot read the data in the shorted links.

DataSift is about to unravel the links and filter the search terms through headlines of the articles linked to on the site. Without such a feature, countless valid search returns would be lost because key words were not in the tweet, but in the linked article headline or description.

Other search options are also limited to the seven day window that Twitter keeps on the search section.

This means that data more than seven days old is not returned in a search. DataSift is currently backfilling its servers with information that will, soon, go back to Twitter's debut. Providing historical data to companies and showing changes over time is yet another asset in the analysis they can provide their marketing and development teams.

Who can use this

As I watched a few demonstrations that Halstead and Bailey conducted for me in their offices, it became clear that not just global brands like Pepsi and Starbucks would find this helpful, but anyone who wants to understand public opinion on anything.

From news organizations looking at the mood on social movements (such as the Occupy movement) to political campaigns that want to gauge a change in the influence of given candidates to financial investors that want to see when to sell or buy a product -- anyone could find a use for this service.

Bailey told me that the search results could even come in handy for company presentations and educational studies on opinion and social media.

DataSift has already partnered with companies like Klout, to add more search options to the service and has signed on several global brands that will begin using its search tools immediately for marketing and analysis purposes.

Currently, DataSift is looking to move deeper into the Twitter space, and hopes that soon after its launch it will have more features to offer in the sorting of Twitter data. Eventually they would like to expand to other platforms, however, such as Facebook, Google+ and blogs.

"Our goal is to make data available to everyone interested," said Bailey, who was pointed to the ability that paid users have of sharing their data results with anyone for free. "We think that data should be disseminated and help everyone better understand the climate we are in, moment by moment."