Delivering the Twitter Firehose, DataSift funded

Data mining 200 million tweets per day isn't easy, but this startup gives you the tools to do it

DataSift, a data mining platform for the real-time social Web, announced Monday that it has raised a $6 million round of funding co-led by IA Ventures and GRP Partners.

“DataSift was among the first to use the Twitter Firehose to provide meaningful insight for brands and publishers, and now they share their expertise with developers across the ecosystem who need an easy way to access Twitter data," said Dick Costolo, CEO of Twitter. "We’re delighted to see DataSift receive this funding, which is indicative of the value the venture community sees in the Twitter ecosystem as a whole."

It first emerged back in April that DataSift had partnered up with Twitter to make the full Twitter Firehose commercially available. Since then, the company must have acquired a healthy number of customers (though they haven't gotten back to me with exact figures) to have secured this new round of venture capital.

"Companies in data-intensive industries," according to DataSift are those who are most likely to benefit from access to the stream of Twitter posts. That's why its clients come from news, publishing, marketing, advertising and financial sectors.

DataSift isn't the only one offering access to the Twitter Firehose. In fact, Gnip first revealed its product in November 2010, almost a half year before DataSift.

That said, DataSift is a bit different in that it offers clients a programming language called Curated Stream Definition Language (CSDL), which allows for much more complex, higher-order filtering of the Firehose. It's not just your standard text query filter.

Having powerful ways of parsing the stream is extremely important when you're dealing with the kind of load that Twitter delivers: over 200 million tweets daily, as of a week ago.

Read more about the differences between DataSift and Gnip directly from company representatives over at Quora.