LessThanDot Site Logo

LessThanDot

A Technical Community for IT Professionals

Less Than Dot is a community of passionate IT professionals and enthusiasts dedicated to sharing technical knowledge, experience, and assistance. Inside you will find reference materials, interesting technical discussions, and expert tips and commentary.

LTD Social Sitings

Lessthandot twitter Lessthandot Linkedin Lessthandot facebook Lessthandot rss

Note: Watch for social icons on posts by your favorite authors to follow their postings on these and other social sites.

Highly Rated Users

Forum
No Posts Rated

Top 50
Given
Received

Links

Wiki
Blog

Forum Statistics

Users
Members:
1882
Members Online:
2
Guests Online:
159

Total Post History
Posts:
81456
Topics:
18718

7-Day Post History
New Posts:
0
New Topics:
0
Active Topics:
0

Our newest member
justanails28

Other

FAQ
All times are UTC [ DST ]

Building Scalable Websites

Building Solutions, Composition of Applications, etc
Please wait...

Building Scalable Websites

Postby damber on Sun Jun 01, 2008 9:32 pm

Building websites that scale is difficult - especially with limited funds. There are many ways to approach scalability, from vertical scaling to horizontal load balancing, to application design and network/hardware design. Here's a great site for news and examples of highly scaled architectures - some are much better than others, but it's good to see what other people are doing:

http://highscalability.com/

You'll probably want to get your claws into these first:

YouTube: http://highscalability.com/youtube-architecture
Amazon: http://highscalability.com/amazon-architecture
Google: http://highscalability.com/google-architecture
Twitter: http://highscalability.com/scaling-twit ... ent-faster
a smile is worth a thousand kind words, so smile, it's easy! :-)


CODE: $5
WORKING CODE: $500
PROPERLY DESIGNED & WORKING CODE: Priceless
User avatar
damber
LTD Admin
LTD Admin
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
 
Posts: 3138
Joined: Tue Oct 09, 2007 1:48 pm
Location: North Wales, UK
Unrated

Re: Building Scalable Websites

Postby AlexCuse on Sun Jun 01, 2008 10:40 pm

I love that article on twitter. People are so quick to jump on technologies used that they often miss the big picture (remember all the "ruby on rails sucks" articles?). You could have the fastest platform in the world, whatever that is, and if your app is poorly designed, then it is not going to be so fast.

I guess this "scalability problem" is a result of developers not asking themselves "what am I going to do when this site has umpteen-million users per day" up front. You can't expect to solve the problem then and there (after all, you are looking to get your project done on schedule), but I think everyone is happier if they keep this in mind from the get-go. After all, this is the problem you want to have ;)
Say what you like about the tenets of National Socialism Dude, at least it's an ethos
User avatar
AlexCuse
LTD Admin
LTD Admin
LTD Gold - Rating: 1031LTD Gold - Rating: 1031LTD Gold - Rating: 1031LTD Gold - Rating: 1031LTD Gold - Rating: 1031
LTD Gold - Rating: 1031LTD Gold - Rating: 1031LTD Gold - Rating: 1031LTD Gold - Rating: 1031LTD Gold - Rating: 1031
LTD Gold - Rating: 1031
 
Posts: 5524
Joined: Tue Oct 09, 2007 5:26 pm
Location: Pennsylvania, US
Unrated

Re: Building Scalable Websites

Postby damber on Tue Jun 17, 2008 9:20 pm

Well, there have been a lot of complaints about Ruby on Rails as you've noted, but not having worked with it myself, it's a bit difficult to comment - except to say that in many cases developing things fast does not always equate to efficient applications. Rapid prototyping is kind of commonplace, especially in ideology of IDE vendors and frameworks like .NET and Ruby on Rails etc. What that tends to lead to is poor design decisions - not necessarily raw performance of the language itself for equivalent functions etc, but if you don't ask the right questions, you rarely get the right answer - and the IDE or framework has more than one way to crack a nut, so to speak. Don't get me wrong, there are a lot of good things about truly great frameworks, but a) not all frameworks are great and b) not all users of those frameworks understand the implications of their choices.

This is generally why the language/platform/etc is one of the LAST things you should decide. Design the solution first, then fit the relevant technology to it. That way, you get the fundamentals right for different aspects of the solution, and especially the non-functional requirements etc.
a smile is worth a thousand kind words, so smile, it's easy! :-)


CODE: $5
WORKING CODE: $500
PROPERLY DESIGNED & WORKING CODE: Priceless
User avatar
damber
LTD Admin
LTD Admin
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
 
Posts: 3138
Joined: Tue Oct 09, 2007 1:48 pm
Location: North Wales, UK
Unrated

Re: Building Scalable Websites

Postby SQLDenis on Tue Jun 17, 2008 9:28 pm

The problem with twitter is that it was not designed to be a messaging bus app, but people are using it that way
It was designed to have more of a blog architecture

Take Robert Scoble, he updates his status, 27,338 writes have to be done to the DB for one update....that is a big mistake of course...it should be 1 write and then 27,338 reads
User avatar
SQLDenis
LTD Admin
LTD Admin
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
 
Posts: 21784
Joined: Wed Oct 10, 2007 6:43 pm
Location: Princeton, New Jersey, USA,World, Solar System, Milky Way, Universe and Beyond
Unrated

Re: Building Scalable Websites

Postby SQLDenis on Tue Jun 17, 2008 9:31 pm

Here are two interesting quotes

At this
point in time there’s no facility in Rails to talk to more than one
database at a time.


Once you hit a certain threshold of
traffic, either you need to strip out all the costly neat stuff that
Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move
the slow parts of your application out of Rails, or both.It’s also worth mentioning that there shouldn’t be doubt in anybody’s
mind at this point that Ruby itself is slow. It’s great that people
are hard at work on faster implementations of the language, but right
now, it’s tough. If you’re looking to deploy a big web application
and you’re language-agnostic, realize that the same operation in Ruby
will take less time in Python. All of us working on Twitter are big
Ruby fans, but I think it’s worth being frank that this isn’t one of
those relativistic language issues. Ruby is slow.



http://www.radicalbehavior.com/5-questi ... lex-payne/
User avatar
SQLDenis
LTD Admin
LTD Admin
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
 
Posts: 21784
Joined: Wed Oct 10, 2007 6:43 pm
Location: Princeton, New Jersey, USA,World, Solar System, Milky Way, Universe and Beyond
Unrated

Re: Building Scalable Websites

Postby SQLDenis on Tue Jun 17, 2008 9:35 pm

Another interesting article is the one when MySpace moved from ColdFusion to .NET (BlueDragon)

MySpace.com is now processing 1.5 Billion page views per day
MySpace.com handles 2.3 million concurrent users during the day
MySpace.com’s average server CPU utilization went from 85% to 27% after moving (from another technology) to ASP.NET 2.0


here is a real fascinating article
http://www.baselinemag.com/c/a/Projects ... ySpacecom/


also ScottGu has one here: http://weblogs.asp.net/scottgu/archive/ ... T-2.0.aspx
User avatar
SQLDenis
LTD Admin
LTD Admin
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
LTD Gold - Rating: 3467LTD Gold - Rating: 3467LTD Gold - Rating: 3467
 
Posts: 21784
Joined: Wed Oct 10, 2007 6:43 pm
Location: Princeton, New Jersey, USA,World, Solar System, Milky Way, Universe and Beyond

Re: Building Scalable Websites

Postby damber on Tue Jun 17, 2008 9:38 pm

you're absolutely right Denis - pub/sub messaging has been around a long time, and dealing with larger numbers than twitter does. one of the issues though, is that twitter is very much a push architecture, rather than pull due to the integration with existing clients etc. I suspect (without any in depth knowledge of their approach,reasons etc) that they would benefit from high level architectural changes, as well as a change in technology... I've read those quotes before, and those kind of things alone scared me away from RoR.. because that's always going to be your problem. It might be great for rapid prototyping, but it should be understood that that is what it is.. and at some point it will need rebuilding.
a smile is worth a thousand kind words, so smile, it's easy! :-)


CODE: $5
WORKING CODE: $500
PROPERLY DESIGNED & WORKING CODE: Priceless
User avatar
damber
LTD Admin
LTD Admin
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
 
Posts: 3138
Joined: Tue Oct 09, 2007 1:48 pm
Location: North Wales, UK
Unrated

Re: Building Scalable Websites

Postby damber on Tue Jun 17, 2008 10:01 pm

Thanks for the link to the myspace article, it was an interesting read indeed - such a common pattern of change for web scale applications, but in someways you can't always avoid it - if the funding was there to build out all the improvements from day 1 it may be a different story.. but as we know with this site, we've not got a bottomless pocket filled with gold to apply any architectural pattern we wish :-).
a smile is worth a thousand kind words, so smile, it's easy! :-)


CODE: $5
WORKING CODE: $500
PROPERLY DESIGNED & WORKING CODE: Priceless
User avatar
damber
LTD Admin
LTD Admin
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663LTD Silver - Rating: 663
 
Posts: 3138
Joined: Tue Oct 09, 2007 1:48 pm
Location: North Wales, UK
Unrated

Re: Building Scalable Websites

Postby Karato on Mon Jul 27, 2009 1:26 am

The titlescreen now has full credits and also the loading screens prior.
sonnerie portable mp3 - Sonnerie portable MP3 est en effet unpouvoir de l'innovation technologique. Êtes-vous familier avec cette sonnerie portable?

*URL Removed from Spam - Tarwn
Karato
Newbie
Newbie
 
Posts: 1
Joined: Mon Jul 27, 2009 1:21 am
Unrated