How large eCommerce sites are so fast?
I have been working on asp.net mvc3 e-commerce application based on NopCommerce.
Recently we have been working on performance side to improve the performance of the site. The site have more than 200000 products and 1200 categories and large no. of brands. This efforts has resulted into 20 to 200 times the performance increment, but still when bench marked against the e-commerce majors like, amazon, flipkart, jabong, letsbuy, ebay, shopping.indiatimes.com,etc it is still slower.
Now looking at just few sites, like Amazon, Flipkart & Jabong... Their page starts loading with almost zero waiting time, and images and other resources load almost instantaneously.
Also for search, Nop Commerce is dyeing slow, just look at the FlipKart & Jabong or Amazon, they very fast. No clue how? What do they do? Do they perform a search in db or something else?
My question is what do they do to have this kind of superb performance? I know they have load balancing servers with mem cached like memory implementation implemented to cache may be the entire site.
But what are the best practices for creating such large scalable web site? And how do they do it? Are any of them use any opensource e-commerce platform like nop commerce or magento as their base? Or do they all prefer custom made?
Would like to learn how to scale a web application like them with its best practices to implement. (Note that this is a general question and not a nop-commerce related one, it is one of the best e-commerce application we used till date.)
Thanks
Answer
The database tier is usually always a bottleneck, and often the significant one. That means it’s imperative that you have a caching layer to minimize database access and remove it from the critical path. Memcached may be a good option when a key-value store is sufficient.
However, many real-world scenarios involve complex domain models. In these cases, it’s much better to use an in-memory data grid that works with domain objects and which can handle embedded object relationships.
An in-memory data grid is also very helpful if you need high-availability, elastic scaling, maximum performance, and perhaps want to create an event-driven architecture or pre-process data before it hits the database. Here’s a good video describing how some of the biggest players do it: http://youtu.be/1AR2WWaP8CE
Another major optimization would be in reducing the time to download content, so you’ll also want to use a content delivery network. Start with these 2 optimizations and you may find that performance is no longer an issue.