Inspired by a post<\/a> there, I wrote a simple web-crawler that looked at the age, membership length and reputation of the users of StackOverflow.<\/p>\n I describe the method below, for people who like hearing about software. I graph the results below for people who like looking at colourful graphs. At the bottom, I have a quick conclusion for people who just want the summary.<\/p>\n I wrote a simple Python script that fetched each user’s profile page, and snipped out the user’s age (if present), their reputation and the numbers of days they had been a member of StackOverflow. (Code and data available upon request<\/a>.)<\/p>\n The script ran for around 18 hours around October 23-24, 2008.<\/p>\n Deleted users (having missing user pages) were ignored leaving 57,144 users.<\/p>\n Using feedback from the original post, I treated all ages 8 and below, and also the maximum permissible age of 88 as missing. This removed several outliers. People’s ages are self-reported, and may be wrong or biased. Just over 30,000 users had valid ages.<\/p>\n I grouped reputations into several buckets. The range of the buckets were sized on a logarithmic scale. A reputation score of 1 was put in its own bucket, as it represents the default reputation. Probably, the majority of these users are not active.<\/p>\n I then hastily constructed some Excel charts. I am using the default appearance for Excel charts; sorry for any perceived unprofessionalism, but this was a hasty, and unprofessional, excercise.<\/p>\n Here are the results, presented as charts.<\/p>\n The first graph reproduces the original post. It shows the age of the users.<\/p>\n This graph shows over half of the users are sitting on a reputation score of 1.<\/p>\n This graph shows how StackOverflow has grown in terms of members. The sharp spike almost certainly represents the growth after the site moved from Private Beta to Public Beta. You may have heard Jeff Atwood and Joel Spolsky discussing this curve on their StackOverflow podcast.<\/p>\n This chart is a little more complex. It shows how people’s reputations are associated with the time they have been on the site.<\/p>\n The left part of the graph represents recent joiners. The right part of the graph represents the old-timers.<\/p>\n The gentle slope down to the right represents the obvious fact that, the longer you have been a member, the more likely you are to have a higher reputation.<\/p>\n The sudden drop downwards in the middle represents the date of the public beta starting. Data to the right of the spike represent beta testers.<\/p>\n One way to use it is to find out how long you have been a member along the bottom, and rule a line straight up. The different coloured sections show what percentage of people have what reputation, so you can see your very rough percentile position compared to others.<\/p>\n This is the result which originally inspired this project. It shows the relationship (or the lack of it) between age and reputation.<\/p>\n For this chart, I removed all ages which had less than 10 data points to ensure the data is meaningful. (Why 10? I used the sophisticated statistically modelling technique called “Feels about right”.)<\/p>\nMethod<\/h2>\n
Results<\/h2>\n
Number of Users By Age<\/h3>\n
![\"\"](\"http:\/\/www.somethinkodd.com\/oddthinking\/wp-content\/uploads\/2008\/10\/number-of-users-by-age-300x184.png\" "\\"number-of-users-by-age\\"")
<\/a><\/p>\nNumber of Users By Reputation<\/h3>\n
![\"\"](\"http:\/\/www.somethinkodd.com\/oddthinking\/wp-content\/uploads\/2008\/10\/user-count-by-reputation-256x300.png\" "\\"user-count-by-reputation\\"")
<\/a><\/p>\nMembership Growth<\/h3>\n
![\"\"](\"http:\/\/www.somethinkodd.com\/oddthinking\/wp-content\/uploads\/2008\/10\/membership-growth-300x184.png\" "\\"membership-growth\\"")
<\/a><\/p>\nReputation by Membership Length<\/h3>\n
![\"\"](\"http:\/\/www.somethinkodd.com\/oddthinking\/wp-content\/uploads\/2008\/10\/reputation-profile-by-membership-length-300x184.png\" "\\"reputation-profile-by-membership-length\\"")
<\/a><\/p>\nReputation by Age<\/h3>\n
![\"\"](\"http:\/\/www.somethinkodd.com\/oddthinking\/wp-content\/uploads\/2008\/10\/reputation-profile-by-age-300x184.png\" "\\"reputation-profile-by-age\\"")
<\/a><\/p>\n