sorted by: recent | see : popular

Content Tagged with User:dolander + report

This past weekend I attended the O’Reilly Social Graph FOO Camp and got to meet a bunch of folks who I’ve only “known” via their blogs or news stories about them. My favorite moment was talking to Mark Zuckerberg about stuff I think is wrong with Facebook and he stops for a second while I’m telling hin the story of naked pictures in my Facebook news feed then says “Dare? I read your blog”. Besides that my favorite part of the experience was learning new things from folks with different perspectives and technical backgrounds from me. Whether it was hearing different perspectives on the social graph problem from folks like Joseph Smarr and Blaine Cook, getting schooled on the various real-world issues around using OpenID/OAuth in practice from John Panzer and Eran Hammer-Lahav or getting to ask getting to Q&A Brad Fitzpatrick about the Google Social Graph API, it was a great learning experience all around.

There have been some ideas tumbling around in my head all week and I wanted to wait a few days before blogging to make sure I’d let the ideas fully marinate. Below are a few of the more important ideas I took away from the conference.

Social Network Discovery vs. Social Graph Portability

One of the most startling realizations I made during the conference is a lot of my assumptions about why developers of social applications are interested in what has been mistakenly called “social graph portability” were incorrect. I had assumed a lot of social networking sites that utilize the password anti-pattern to screen scrape a user’s Hotmail/Y! Mail/Gmail/Facebook address book were doing that as a way to get a list of the user’s friends to spam invite to join the service. However a lot of the folks I met at the SG FOO Camp made me realize how much of a bad idea this would be if they actually did that. Sending out a lot of spam would lead to negativity being associated with their service and brand (Plaxo is still dealing with a lot of the bad karma they generated from their spammy days).

Instead the way social applications often use the contacts from a person’s email address book is to satisfy the scenario in Brad Fitzpatrick’s blog post URLs are People, Too where he wrote

So you've just built a totally sweet new social app and you can't wait for people to start using it, but there's a problem: when people join they don't have any friends on your site. They're lonely, and the experience isn't good because they can't use the app with people they know.  

I then thought of my first time using Twitter and Facebook, and how I didn’t consider them of much use until I started interacting with people I already knew that used those services. More than once someone has told me, “I didn’t really get why people like Facebook until I got over a dozen friends on the site”.

So the issue isn’t really about “portability”. After all, my “social graph” of Hotmail or Gmail contacts isn’t very useful on Twitter if none of my friends use the service. Instead it is about “discovery”.

Why is this distinction important? Let’s go back to the complaint that Facebook doesn’t expose email addresses in it’s API. The site actually hides all contact information from their API which is understandable. However since email addresses are also the only global identifiers we can rely on for uniquely identifying users on the Web, they are useful as way of being able to figure out if Carnage4Life on Twitter is actually Dare Obasanjo on Facebook since you can just check if they are backed by the same email address.

I talked to both John Panzer and Brad Fitzpatrick about how we could bridge this gap and Brad pointed out something really obvious which he takes advantage of in the Google Social Graph API. We can just share email addresses using foaf:mbox_sha1sum (i.e. cryptographical one-way hashes of email addresses). That way we all have a shared globally unique identifier for a user but services don’t have to reveal their user’s email addresses.

I wonder how we can convince the folks working on the Facebook platform to consider adding this as one of the properties returned by Users.getInfo?

You Aren’t Really My Friend Even if Facebook Says We Are

In a post entitled A proposal: email to URL mapping Brad Fitzpatrick wrote

People have different identifiers, of different security, that they give out depending on how much they trust you. Examples might include:

• Homepage URL (very public)
• Email address (little bit more secret)
• Mobile phone number (perhaps pretty secretive)

When I think back to Robert Scoble getting kicked off of Facebook for screen scraping his friends’s email addresses and dates of birth into Plaxo, I wonder how many of his Facebook friends are comfortable with their personal contact information including email addresses, cell phone numbers and home addresses being utilized by Robert in this manner. A lot of people argued at SG FOO Camp that “If you’ve already agreed to share your contact info with me, why should you care whether I write it down on paper or download it into some social networking site?”.

That’s an interesting question.

I realized that one of my answers is that I actually don’t even want to share this info with the majority of the people in my Facebook friends list in the first place [as Brad points out]. The problem is that Facebook makes this a somewhat binary decision. Either I’m your “friend” and you get all my private personal details or I’ve faceslammed you by ignoring your friend request or only giving you access to my Limited Profile. I once tried to friend Andrew ‘Boz’ Bosworth (a former Microsoft employee who works at Facebook) and he told me he doesn’t accept friend requests from people he didn’t know personally so he ignored the friend request. I thought it was fucking rude even though objectively I realize it makes sense since it would mean I could view all his personal wall posts as well as his contact info. Funny enough, I always thought that it was a flaw in the site’s design that we had to have such an awkward social interaction.

I think the underlying problem again points to Facebook’s poor handling of multiple social contexts. In the real world, I separate my interactions with co-workers from that with my close friends or my family. For an application that wants to be the operating system underlying my social interactions, Facebook doesn’t do a good job of handling this fundamental reality of adult life.

Now playing: D12 - Revelation

These are my notes from the talk Lessons in Building Scalable Systems by Reza Behforooz.

The Google Talk team have produced multiple versions of their application. There is

• a desktop IM client which speaks the Jabber/XMPP protocol.
• a Web-based IM client that is integrated into GMail
• a Web-based IM client that is integrated into Orkut
• An IM widget which can be embedded in iGoogle or in any website that supports embedding Flash.

Google Talk Server Challenges

The team has had to deal with a significant set of challenges since the service launched including

• Support displaying online presence and sending messages for millions of users. Peak traffic is in hundreds of thousands of queries per second with a daily average of billions of messages handled by the system.

• routing and application logic has to be applied to each message according to the preferences of each user while keeping latency under 100ms.

• handling surge of traffic from integration with Orkut and GMail.

• ensuring in-order delivery of messages

• needing an extensibile architecture which could support a variety of clients

Lessons

The most important lesson the Google Talk team learned is that you have to measure the right things. Questions like "how many active users do you have" and "how many IM messages does the system carry a day" may be good for evaluating marketshare but are not good questions from an engineering perspective if one is trying to get insight into how the system is performing.

Specifically, the biggest strain on the system actually turns out to be displaying presence information. The formula for determining how many presence notifications they send out is

total_number_of_connected_users * avg_buddy_list_size * avg_number_of_state_changes

Sometimes there are drastic jumps in these numbers. For example, integrating with Orkut increased the average buddy list size since people usually have more friends in a social networking service than they have IM buddies.

Other lessons learned were

1. Slowly Ramp Up High Traffic Partners: To see what real world usage patterns would look like when Google Talk was integrated with Orkut and GMail, both services added code to fetch online presence from the Google Talk servers to their pages that displayed a user's contacts without adding any UI integration. This way the feature could be tested under real load without users being aware that there were any problems if there were capacity problems. In addition, the feature was rolled out to small groups of users at first (around 1%).

2. Dynamic Repartitioning: In general, it is a good idea to divide user data across various servers (aka partitioning or sharding) to reduce bottlenecks and spread out the load. However, the infrastructure should support redistributing these partitions/shards without having to cause any downtime.

3. Add Abstractions that Hide System Complexity: Partner services such as Orkut and GMail don't know which data centers contain the Google Talk servers, how many servers are in the Google Talk cluster and are oblivious of when or how load balancing, repartitioning or failover occurs in the Google Talk service.

4. Understand Semantics of Low Level Libraries: Sometimes low level details can stick it to you. The Google Talk developers found out that using epoll worked better than the poll/select loop because they have lots of open TCP conections but only a relatively small number of them are active at any time.

5. Protect Against Operational Problems: Review logs and endeavor to smooth out spikes in activity graphs. Limit cascading problems by having logic to back off from using busy or sick servers.

6. Any Scalable System is a Distributed System: Apply the lessons from the fallacies of distributed computing. Add fault tolerance to all your components. Add profiling to live services and follow transactions as they flow through the system (preferably in a non-intrusive manner). Collect metrics from services for monitoring both for real time diagnosis and offline generation of reports.

Recommended Software Development Strategies

Compatibility is very important, so making sure deployed binaries are backwards and forward compatible is always a good idea. Giving developers access to live servers (ideally public beta servers not main production servers) will encourage them to test and try out ideas quickly. It also gives them a sense of empowerement. Developers end up making their systems easier to deploy, configure, monitor, debug and maintain when they have a better idea of the end to end process.

Building an experimentation platform which allows you to empirically test the results of various changes to the service is also recommended.

These are my notes from the keynote session MapReduce, BigTable, and Other Distributed System Abstractions for Handling Large Datasets by Jeff Dean.

The talk was about the three pillars of Google's data storage and processing platform; GFS, BigTable and MapReduce.

GFS

The developers at Google decided to build their own custom distributed file system because they felt that they had unique requirements. These requirements included

• scalable to thousands of network nodes
• massive read/write bandwidth requirements
• ability to handle large blocks of data which are gigabytes in size.
• need exremely efficient distribution of operations across nodes to reduce bottlenecks

One benefit the developers of GFS had was that since it was an in-house application they could control the environment, the client applications and the libraries a lot better than in the off-the-shelf case.

There are two server types in the GFS system.

There are currently over 200 GFS clusters at Google, some of which have over 5000 machines. They now have pools of tens of thousands of machines retrieving data from GFS clusters that run as large as 5 petabytes of storage with read/write throughput of over 40 gigabytes/second across the cluster.

MapReduce

At Google they do a lot of processing of very large amounts of data. In the old days, developers would have to write their own code to partition the large data sets, checkpoint code and save intermediate results, handle failover in case of server crashes, and so on as well as actually writing the business logic for the actual data processing they wanted to do which could have been something straightforward like counting the occurence of words in various Web pages or grouping documents by content checksums. The decision was made to reduce the duplication of effort and complexity of performing data processing tasks by building a platform technology that everyone at Google could use which handled all the generic tasks of working on very large data sets. So MapReduce was born.

MapReduce is an application programming interface for processing very large data sets. Application developers feed in a key/value pair (e.g. {URL,HTML content} pair) then use the map function to extract relevant information from each record which should produce a set of intermediate key/value pairs (e.g. {word, 1 } pairs for each time a word is encountered) and finally the reduce function merges the intermediate values associated with the same key to produce the final output (e.g. {word, total count of occurences} pairs).

A developer only has to write their specific map and reduce operations for their data sets which could run as low as 25 - 50 lines of code while the MapReduce infrastructure deals with parallelizing the task and distributing it across different machines, handling machine failures and error conditions in the data, optimizations such as moving computation close to the data to reduce I/O bandwidth consumed, providing system monitoring and making the service scalable across hundreds to thousands of machines.

Currently, almost every major product at Google uses MapReduce in some way. There are 6000 MapReduce applications checked into the Google source tree with the hundreds of new applications that utilize it being written per month. To illustrate its ease of use, a graph of new MapReduce applications checked into the Google source tree over time shows that there is a spike every summer as interns show up and create a flood of new MapReduce applications that are then checked into the Google source tree.

There are three server types in the MapReduce system.

One of the main issues they have to deal with in the MapReduce system is problem of stragglers. Stragglers are servers that run slower than expected for one reason or the other. Sometimes stragglers may be due to hardware issues (e.g. bad harddrive conttroller causes reduced I/O throughput) or may just be from the server running too many complex jobs which utilize too much CPU. To counter the effects of stragglers, they now assign multiple servers the same jobs which counterintuitively ends making tasks finish quicker. Another clever optimization is that all data transferred between map and reduce servers is compressed since the servers usually aren't CPU bound so compression/decompression costs are a small price to pay for bandwidth and I/O savings.

BigTable

After the creation of GFS, the need for structured and semi-structured storage that went beyond opaque files became clear. Examples of situations that could benefit from this included

• associating metadata with a URL such as when it was crawled, its PageRank™, contents, links to it, etc
• associating data with a user such as the user's search history and preferences
• geographical data such as information about roads and sattelite imagery

The system required would need to be able to scale to storing billions of URLs, hundreds of terabytes of satellite imagery, data associated preferences with hundreds of millions of users and more. It was immediately obvious that this wasn't a task for an off-the-shelf commercial database system due to the scale requirements and the fact that such a system would be prohibitively expensive even if it did exist. In addition, an off-the-shelf system would not be able to make optimizations based on the underlying GFS file system. Thus BigTable was born.

BigTable is not a relational database. It does not support joins nor does it support rich SQL-like queries. Instead it is more like a multi-level map data structure. It is a large scale, fault tolerant, self managing system with terabytes of memory and petabytes of storage space which can handle millions of reads/writes per second. BigTable is now used by over sixty Google products and projects as the platform for storing and retrieving structured data.

The BigTable data model is fairly straightforward, each data item is stored in a cell which can be accessed using its {row key, column key, timestamp}. The need for a timestamp came about because it was discovered that many Google services store and compare the same data over time (e.g. HTML content for a URL). The data for each row is stored in one or more tablets which are actually a sequence of 64KB blocks in a data format called SSTable.

There are three primary server types of interest in the BigTable system.

There are a number of optimizations which applications can take advantage of in BigTable. One example is the concept of locality groups. For example, some of the simple metadata associated with a particular URL which is typically accessed together (e.g. language, PageRank™ , etc) can be physically stored together by placing them in a locality group while other columns (e.g. content) are in a separate locality group. In addition, tablets are usually kept in memory until the machine is running out of memory before their data is written to GFS as an SSTable and a new in memory table is created. This process is called compaction. There are other types of compactions where in memory tables are merged with SSTables on disk to create an entirely new SSTable which is then stored in GFS.

Current Challenges Facing Google's Infrastructure

Although Google's infrastructure works well at the single cluster level, there are a number of areas with room for improvement including

• support for geo-distributed clusters
• single global namespace for all data since currently data is segregated by cluster
• more and better automated migration of data and computation
• lots of consistency issues when you couple wide area replication with network partitioning (e.g. keeping services up even if a cluster goes offline for maintenance or due to some sort of outage).

Recruiting Sales Pitch

[The conference was part recruiting event so some of the speakers ended their talks with a recruiting spiel - Dare]

Having access to lots of data and computing power is a geek playground. You can build cool, seemingly trivial apps on top of the data such which turn out to be really useful such as Google Trends and catching misspellings of "britney spears. Another example of the kinds of apps you can build when you have enough data treating the problem of language translation as a statistical modeling problem which turns out to be one of the most successful methods around.

Google hires smart people and lets them work in small teams of 3 to 5 people. They can get away with teams being that small because they have the benefit of an infrastructure that takes care of all the hard problems so devs can focus on building interesting, innovative apps.

Linking to Niall Kennedy's blog reminded me that I owed him an email response to a question he asked about a month ago. The question asked what I thought about the diversity of speakers at the Widgets Live conference given my comments on the topic in my blog post entitled Who Attends 'Web 2.0' Conferences. 

After thinking about it off and on for a month, I realize that I liked the conference primarily because of its content and focus. The speakers weren't the usual suspects you see at Web conferences nor were they homogenous in gender and ethnic background. I assume the latter is a consequence of the fact that the conference was about concrete technical topics as opposed to a gathering to gab with the hip Web 2.0 crowd which meant that the people who actually build stuff were there...and guess what they aren't all caucasian males in their 20s to 30s, regardless of how much conferences like The Future of Web Apps and Office 2.0 pretend otherwise.

This is one of the reasons I decided to pass on the Web 2.0 conference this year. It seems I may have made the right choice given John Battelle's comments on the fact that a bunch of the corporate VP types that spoke at the conference ended up losing their jobs the next week. ;)

These are my notes from the session on Success Story: PhotoBucket.

PhotoBucket is a video and image hosting site that sees 7 million photos and 30,000 videos uploaded daily. They serve over 3 billion pieces of media a day. The site has 15 million unique users in the U.S. (20 million worldwide) and has 80,000 new accounts created daily. There is now a staff of 55 people whose job it is to moderate content submissions to ensure they meet their guidelines.

The top sites their images used to be linked from used to be eBay and LiveJournal but now the key drivers of traffic are now social networking sites such as MySpace and Xanga. There is 30% - 40% overlap between their user base and social network website users

There was some general advise about widgets such as being careful about hosting costs which may pile up quick if your widgets become popular and also about trying to monetize users via your widgets because some sites frown upon that behavior such as eBay. However well designed and compelling widgets can drive a lot of traffic back to your site, the best example of this to date being YouTube.

The speaker then gave a timeline of notable occurences in the MySpace widgets world such as MySpace blocking Revver & YouTube to the recent explosion of new widgets in the past few months from MeeboMe to a number photo slideshow widgets from the major image hosting services.

Pete Cashmore over at Mashable.com has compiled some statistics on the most popular widgets on MySpace which shows the relative popularity of PhotoBucket's widgets in comparison to other services.

So what's in a name? They've renamed the feature from BucketFeatures to Widgets and now to 'Slide Shows' because none of their non-Silicon Valley users knew what widgets were. After the rename from 'Widgets' to 'Slide Shows', the usage of the feature almost doubled within a month.

They've also designed a JWidget which allows people to log-in to their PhotoBucket account to access their videos and images. Users can upload images and videos . This way people can outsource both image upload and content moderation to PhotoBucket. Now have 16,000,000 logins a month via JWidget from about 500 partner sites. It is amed JWidget because the developer's name begins with 'J'. :)

During the Q&A someone asked if they support you have tagging & open APIs. The response wa sthat they don't do tagging and their user base has never asked for tagging. With 2500 support tickets a day, none of them have ever been about tagging. Also, since it is just image hosting service, tagging is probably more appropriate for the blog post or profile the image is appearing in than on the hosted image. They don't have an No API primarily due to resource constraints, there are only 40 people at the company working on it.

These are my notes from the session on Konfabulator by Arlo Rose.

He started with answering the question, why name them 'widgets'? At Apple, a UI control was called a widget. He thought the name meant something more and has always wanted to build widgets that do more.

He was the creator of Kaleidoscope which was one of the key customization and theming applications on the Macintosh. The application was so popular that the CEO of Nokia mentioned it as the inspiration for customization in cell phones.

When Apple announced Mac OS X, he became nervous that this would spell the end of Kaleidoscope and it was because they couldn't make the transition to Cocoa so they killed the product. Arlo then looked around for a new kind of application to build and came across the Control Strip and Control Strip Modules on the Mac which he thought were useful but had a bad user experience. He had also discovered an MP3 player for the Mac named Audion which used cool UI effects to create little UI components on the desktop which seemed transparent. Arlo thought it would be a great idea to build a better Control Strip using Audion-like UI. He talked to his partner from Kaleidoscope but he wasn't interested in the idea. He also talked to the developers of Audion but they weren't interested either. So arlo gave up on the idea and wandered from startup to startup until he ended up at Sun Microsystems

At Sun, he was assigned to a project related to the Cobalt Qube which eventually was cancelled. He then had time to work on a side project and so he resurected his idea for building a better Control Strip with an Audion-like user interface. He originally wanted to develop the project using Perl and XML as the development languages but he soon got some feedback that creative types on the Web are more familiar with Javascript. So in 2002 he started on Konfabulator and released version 1.0 the following year. They also created a widget gallery that enabled developers to upload widgets they've built to share with other users. However they didn't get a lot of submissions from developers so they talked to developers and got a lot of feedback on features to add to their platform such as drag and drop, mouse events, keyboard events and so on. Once they did that they started getting dozens and dozens of develper submissions.

After they got so much praise for the Mac version, they decided to work on a Windows version. While working on the Windows version, he got a call from a friend at Apple who said while he was at a design meeting and he heard "We need to steamroll Konfabulator". He started calling all his friends at Apple and eventually it turned out that the Apple product that was intended to steamroll Konfabulator was Dashboard. The products are different, Dashboard uses standard DHTML while Konfabulator uses proprietary markup. Arlo stated that their use of proprietary technologies gave them advantages over using straight DHTML.

Unfortunately, even though they got millions of downloads of the Windows version not a lot of people paid for the software. There were a number of reasons for this. The first was that in general there is a less of a culture of paying for shareware in the Windows world than in the Mac world. Secondly, there were free alternatives to their product on Windows that had sprung up while there was only a Mac version. In looking for revenue, they sought out partnerships and formed one with Yahoo!. He also talked to people at Microsoft in Redmond who let him know that they were planning to add gadgets to Longhorn Windows Vista. Microsoft made him an offer to come work on Windows Vista but he turned it down. Later on, he was pinged by a separate group at MSN that expressed an interest in buying Konfabulator. Once this happened, Arlo contacted Google and Yahoo! to see if theyd make counter offers and Yahoo! won the bidding war.

They started working on the Yahoo! Widget Engine and the goal was to make it a platform for accessing Yahoo! APIs as part of the Yahoo! Developer Network. However consumers still wanted a consumer product like Konfabulator and eventually they left the YDN and went to the Connected Life group at Yahoo! which works on non-Web consumer applications such as desktop and mobile applications.

There are now 4000 3rd party widgets in the Yahoo! widget gallery and they are the only major widget platform which is cross platform. Also they are the only widget platform that has total access to Yahoo! data.

Q: What's next?
A: The next question is to see how far widgets can scale as mini-applications. Can a picture frame widget become something more but not a full replacement for Flickr or Photoshop?

Q: What do you think of the Apollo project from Adobe?
A: Doesn't know what it is.

Q: Did he ever figure out a business model for widgets?
A: He planned to make deals with companies like J.Crew, Staples, and Time Warner for movie tie-ins.

Q: Why move from YDN to Connected Life?
A: They were 3 people and they couldn't do both the developer side & the consumer application. Also it turned out that the Yahoo! Developer Network turned out not to have the clout that they thought they would in that Yahoo! applications would refuse to provide APIs that could be accessed by 3rd party developers but would create special APIs for writing Konfabulator widgets.