October 5, 2010
Twitter search sees one billion queries/day
Engineers quietly launched new search structure during same period as redesign rollout
Twitter revealed Wednesday afternoon that during the last few weeks, while most users explored the ins and outs of the site’s redesign, a team of engineers were implementing a completely new search backend.
If you’re wondering why nobody ever complained about outages or even slowdowns related to search on Twitter, that just shows how great a job the search team did in making the switch. (Developers curious to read about the technical details involved should click through to the Twitter Engineering Blog.)
The accomplishment is even more astonishing when one considers how much data Twitter search must sift through. Users send over 1,000 tweets/second (that’s 86.4 million tweets/day). Seeking pearls in this sea of tweets, users search 12,000 queries/sec (that’s over one billion queries/day).
In fact, it’s precisely these ballooning statistics that forced Twitter to restructure its search in the first place. The search team estimates that now they’re “only using about 5% of the available backend resources,” meaning they’ve scaled search to handle at least a couple more years of growth.
All Twitter search needs now are some extra filters added to search, something that will probably land on the site sooner than later:
“[...] the new system is extremely versatile and extensible, which will allow us to build cool new features faster and better.”
It’s a bit confounding that a site sitting on such a large amount of data doesn’t have any sort of advanced search options already. After performing a search, one can filter by “Tweets,” “Tweets with links,” “Tweets near you,” and “People,” but that’s a fairly rudimentary foundation for what could be on the way.
Twitter should take notes on the recently launched LinkedIn Signal, which lets users filter imported tweets by network, industry, company, time published, region, school, or hashtag. Obviously, Twitter doesn’t have access to all the same information as LinkedIn, but the point remains that users want more options.
Why can’t we search by location or time or even (though this might sound silly) tweet length? After all, Twitter users do love their data.
Related Companies, Investors, and Entrepreneurs
Startup/Business
Joined Vator on
What is Twitter?
Twitter is an online information network that allows anyone with an account to post 140 character messages, called tweets. It is free to sign up. Users then follow other accounts which they are interested in, and view the tweets of everyone they follow in their "timeline." Most Twitter accounts are public, where one does not need to approve a request to follow, or need to follow back. This makes Twitter a powerful "one to many" broadcast platform where individuals, companies or organizations can reach millions of followers with a single message. Twitter is accessible from Twitter.com, our mobile website, SMS, our mobile apps for iPhone, Android, Blackberry, our iPad application, or 3rd party clients built by outside developers using our API. Twitter accounts can also be private, where the owner must approve follower requests.
Where did the idea for Twitter come from?
Twitter started as an internal project within the podcasting company Odeo. Jack Dorsey, and engineer, had long been interested in status updates. Jack developed the idea, along with Biz Stone, and the first prototype was built in two weeks in March 2006 and launched publicly in August of 2006. The service grew popular very quickly and it soon made sense for Twitter to move outside of Odea. In May 2007, Twitter Inc was founded.
How is Twitter built?
Our engineering team works with a web application framework called Ruby on Rails. We all work on Apple computers except for testing purposes.
We built Twitter using Ruby on Rails because it allows us to work quickly and easily--our team likes to deploy features and changes multiple times per day. Rails provides skeleton code frameworks so we don't have to re-invent the wheel every time we want to add something simple like a sign in form or a picture upload feature.
How do you make money from Twitter?
There are a few ways that Twitter makes money. We have licensing deals in place with Google, Yahoo!, and Microsoft's Bing to give them access to the "firehose" - a stream of tweets so that they can more easily incorporate those tweets into their search results.
In Summer 2010, we launched our Promoted Tweets product. Promoted Tweets are a special kind of tweet which appear at the top of search results within Twitter.com, if a company has bid on that keyword. Unlike search results in search engines, Promoted Tweets are normal tweets from a business, so they are as interactive as any other tweet - you can @reply, favorite or retweet a Promoted Tweet.
At the same time, we launched Promoted Trends, where companies can place a trend (clearly marked Promoted) within Twitter's Trending Topics. These are especially effective for upcoming launches, like a movie or album release.
Lastly, we started a Twitter account called @earlybird where we partner with other companies to provide users with a special, short-term deal. For example, we partnered with Virgin America for a special day of fares on Virginamerica.com that were only accessible through the link in the @earlybird tweet.
What's next for Twitter?
We continue to focus on building a product that provides value for users.
We're building Twitter, Inc into a successful, revenue-generating company that attracts world-class talent with an inspiring culture and attitude towards doing business.
Related News
October 5, 2010
Twitter roundup: new CEO, Promoted Accounts
September 29, 2010