1. I think operative word is scaling. The implication that using a single SQL (Oracle) server would somehow solve F* speed problem is to not understand what the word "scale" means. (BTW, I wrote the blog comment entry ref. in the MIT report on F* speed problem). And also not all queries to a DB are the same. A SQL request for a typical photo.net page isn't nearly as involved as SQL stmt in F*. It's prob like 10-100 times more CPU/disk intensive.

It's like folks who think solving problems like SETI has anything to do with "scaling". Unlike this and other trivial distributed problems (like code cracking) that deal with computing on isolated data or serving static content (you name it, 99.99% of the Web is static in a given period of time; it doesn't matter if it comes from RDBMS), apps like F* has to serve data that's constantly changing. What this means is that server #15 needs to tell "99" other servers in a cluster of "100" servers that they need to update their DB or cache whenever someone adds a new connection or updates his/her profile.

This of course is what the big sites do all the time (and companies like Akamai provides such services). BUT the difference is that F* app requires updates to the whole cluster ALL the time, whereas a site like Yahoo! or NYTIMES.com needs to update their news items every few hours.

So, there's no way you can scale an SNS app (btw, online dating sites don't have this problem, 'cause you don't need to know within the min. a person updated/posted their profile) without doing one of two things:
1) spend like $10+ million on a 64 CPU Oracle server with like 10 X Gbit ethernet connections;
2) OR cache like crazy so that you don't constantly pound on the poor DB server.

Another point of fact. Getting 50% speedup by using PHP over JSP (or whatever) is a waste of time for F*, 'cause F* needs 100X speedup to deal with potentially 100 million users. And if you know anything about scaling, getting 100X speedup on non-trivial problems is really, really hard.

FYI: there'll be a JavaOne BOF on "Building High Performance SNS" that you might want to check out for more info.

Cheers,
jko

Posted at 8:04PM on Dec 18th 2005 by John Ko

2. Heh. I spelled my name right in my submission (Jon is short for Jonathan). And what happened to the link?

Posted at 8:04PM on Dec 18th 2005 by Jon Gales

3. corrected your name Jon, and the 'link' you included is in the related links...

Posted at 8:04PM on Dec 18th 2005 by judith

4. Thanks Judith. I meant my personal link that the submit form says you will link to (usually just link the name I believe). It's no big deal.

Posted at 8:04PM on Dec 18th 2005 by Jon Gales

5. LOL! Aren't they taking a step backward here? You think it's slow now? Just wait until you launch a PHP hack site. You better figure out how to do some serious connection pooling because PHP sucks at caching.

Posted at 8:04PM on Dec 18th 2005 by Joe

6. ...but that is very beside the point.

Joe...I think that you're prolly not understanding what has to happen on a SN site, and confusing something like Friendster with a site that is a plain regular database-driven website. It isn't just that, and I don't think that John Ko's post above suggested that the caching be on the front end, at the php (i.e. serving) level.

The main challenge for real social networking sites is not the database, and not the serving. It is, like Ko's post above, is being able to update the computation of a network (i.e. in math terms, the graph)...which is not trivial to do, especially on the scale of Friendster. This calculation (who is connected to who thru whom) needs to be done in almost real-time, otherwise, friends who add or remove connections won't see them right away. This means that it's not feasible to make that calculation via a database query. The algorithms used to calculate the shortest path thru a networked group of nodes is not something that is easily done (and shouldn't be) by a simple database query. My math background is showing thru at this point. :)

So there is a middle tier that most web developers aren't thinking about when it comes to social networking.

Posted at 8:04PM on Dec 18th 2005 by sty

7. LOL, again. So, you're saying embedding queries and business logic within a PHP template will improve performance? LOL. You may have a math background, but obviously you lack architectural and enterprise software experience. It's obvious friendster data model sucks the big one, and they probably don't have a middle tier.

Posted at 8:04PM on Dec 18th 2005 by Joe

8. Ha. No. That's not what I said. Like I said, and John Ko's post agrees, the graph calculations need to be updated almost real-time, and of course php/mysql is just not going to cut it, due to database replication. A compute engine written in something like C (I assume their current Java engine is obviously going away) would probably be the middle tier, and I agree with Ko that caching around that tier would help abstract the database writes from the reads, and making replication across a mysql cluster more feasible.

To help you understand what I'm talking about, check out some lecture notes from one of their engineers....it's obviously talking about the computation from a Java standpoint, but you get the idea:

http://www.sims.berkeley.edu/academics/courses/ is255/f03/lectures/lecture24.pdf
http://www.sims.berkeley.edu/academics/courses/ is255/f03/lectures/lecture22.pdf

In other words, I'm sure the folks at Friendster know enough about php to know what its limitations are when it comes to scaling.

As to my background, I'll volunteer that I've had a major part in building some pretty large-scale sites in my time, which I'll keep to myself. :)

Posted at 8:04PM on Dec 18th 2005 by sty

9. Now you are making more sense. :) I still think PHP is not a real programming language.

Posted at 8:04PM on Dec 18th 2005 by Joe

10. the P in LAMP stands for PHP/Perl. In this case I am pretty certain they primarily mean mod_perl.
Slashdot and Amazon run on mod_perl. I rest my case.

Posted at 8:04PM on Dec 18th 2005 by Nicolas

11. nope:

http://www.craigslist.org/sby/eng/27162929.html

Amazon and Slashdot are not social networking sites, they're content and ecommerce, which are *very* different.

Posted at 8:04PM on Dec 18th 2005 by mfc

12. The P is for both PHP and Perl for Friendster. Also nope has it right... It is all about the graph.

Posted at 8:04PM on Dec 18th 2005 by Nicolas

13. My guess is that F* is looking into impl native code to speed their graph calc, and they figure they might as well just use PHP since they're simpler to use compared to JSP/Servlet and a bit faster. I guess they're shooting for ~10x improvement from their current farm of servers, not the 100x that I would think they need.

Posted at 8:04PM on Dec 18th 2005 by John Ko

14. More power to them if they THINK PHP is faster than JSP. Which is a joke. Spoke, Tribe, LinkedIn , and several other sites are done in JSP/Java and they don't think have performance issues like Friendster. It has got to be crappy code, maybe they should invest in good developers vs. hackers? Get ride of that stupid recursive code.

Posted at 8:04PM on Dec 18th 2005 by Joe

15. LinkedIn, Spoke, Tribe, and others have *piddly* number of user load compared to Friendster. It's apples and oranges. I would suspect that even if added together, they don't have the user load/traffic of Friendster.

And as to their developers...unless they've left the company, Jason Brittain is one of them, who is the author of "Tomcat: The Definitive Guide":

http://tinyurl.com/2f7gp

and so is Ian McFarland, who is the author of "Mastering Tomcat Development":

http://tinyurl.com/2jpvk

So I'd say that Friendster probably has enough JSP brainpower to know what's good and what's not, wouldn't you ? I really doubt that they are hiring PHP people because of just a whim that PHP will be faster.

This clue, for you, today only, is free.

Posted at 8:04PM on Dec 18th 2005 by Stephen

16. Friendster made a bad decision initially to go with Java.

The problem compounded when they decided to do a re-write which was further compounded by sticking with Java.

While re-writes are generally a bad idea, that Friendster is now re-writing in PHP is OK.

The site is molasses because all of their engineering has been working on a re-write instead of tuning the live service.


PHP is fine for Friendster. PHP is fine for the vast majority of web apps.

Posted at 8:04PM on Dec 18th 2005 by pb

17. Currently, friendster does not even update connections in real time (haven't you ever gotten the 'connection is currently unavailable' message?) And I think that they could as well have a dedicated "connection processor" machine if the network model is a major performance bottleneck for them. Written in pure C++ data structures or a main memory database (like intersystems cache?) and running on a 64-bit Opteron, this connection processor will contain just enough data to process queries dealing with who's connected to whom. People don't add and remove new friends so often.

Posted at 8:04PM on Dec 18th 2005 by Seun Osewa

18. Use both! I use both. Each has different strengths. I use Perl regular expressions to parse info into databases and then PHP to provide the front end to that data.

Posted at 8:04PM on Dec 18th 2005 by Scott Malec

19. Does anyone know if there is any open source social network software that resembles LinkedIn or Ecademy?

I need it to develop a community of people with common intests for our site http://www.perupolitica.com

Posted at 8:04PM on Dec 18th 2005 by Esteban

20. They went with a LAMP solution so that they could work with the source code of their back-end.

http://www.infoworld.com/article/05/04/04/14FEfriendster_2.html

Posted at 8:04PM on Dec 18th 2005 by xenlab