Why is Symfony2 performing so bad in benchmarks and does it matter?
My colleagues and I are in the process of choosing a web framework to develop a high traffic web site. We are really good with node.js + express and php + symfony2. Both are great frameworks but we are a bit concerned about Symfony2 because it seems to be outperformed by most web frameworks out there.
Here is the benchmarks that proves it: http://www.techempower.com/benchmarks/
For this reason we will probably use node.js + express but I still wonder why is Symfony2 performing so bad in benchmarks.
Answer
In the end it all comes down to correct cache handling...
symfony or PHP in general IS slower than other languages or frameworks thus providing you with the tools to create rich, secure and testable web applications really fast.
If you use a reverse proxy like Varnish and ESI ( edge side includes ) and end up serving the parts of your templates you really need to have updated through symfony. you will have a blazingly fast experience.
Furthermore if you use an opcode cache like APC and an optimized database a human user won't actually notice the difference of a few ms in a real world application.
As per request i will dive a bit deeper and give you a few more things to think about.
Caching & Performance
With cloud-services (s3,ec2,gae,...) at almost no cost paired with load-balancers, easy provisioning (chef,puppet,...) and all this funky stuff avilable it has become easy and affordable even for small companies to run and manage large data and/or high traffic applications.
More storage means more space for cache - more computing power means quicker cache warmimg.
things you will often hear if people are talking about php or framework-performance:
- facebook runs with php
- youp**n was developed with symfony
- ...
So why do these sites not break down completely? Because their caching routines are clever.
Did you know for instance what facebook does if you write a status update?
It does not save it right into a database table with all your status updates and if a friend visits his stream all the statuses from all his friend are being fetched from database prior to being served.
facebook writes your status to all of your friends news streams and starts warming their cache. Now all the streams are being prepared for serving and whenever one of your friends visits his stream he will be served a cached version; instantly with almost no code execution involved. The stream will only show your newly created status when the cache warming has finished. we are talking about ms here ...
What does this tell us? In modern highly-frequented applications almost everything is being served from cache and the user will not notice if the actual computing of the page took 1ms or 5 seconds.
In a "real-world" scenario the end-user will notice no difference in req/sec between frameworks. Even with simple stuff like micro-caching you can have your vps hosted blog not go down instantly once you made it on hackernews's landing page.
In the end the more important thing is ... does my framework provide the tools, the documentation and the tutorials and examples ... to get this whole thing up and running quick & easy. symfony does for me!
If you're stuck ... how many people are there willing and able to answer your performance-related questions? How many real-world applications have already been or will in the near future be created with this framework?
you choose a community by choosing a framework !
... okay thats for the does it matter part ... now back to these benchmarks :)
Benchmarks & Setups
Over all these shiny colors and fancy graphs in the benchmark you easily miss the fact that there is only one setup ( webserver, database, ... ) tested with each of these frameworks while you can have a wide variety of configurations for each of them.
Example: instead of using symfony2+doctrineORM+mysql you could also use symfony+doctrineODM+MongoDB.
MySQL ... MongoDB ... Relational Databases ... NoSQL Databases ... ORM ... micro ORMs ... raw SQL ... all mixed up in these configurations ------> apples and oranges.
Benchmarks & Optimization
A common problem with almost all benchmarks - even those only comparing php frameworks - found around the web and also those "TechEmpower Web Framework Benchmarks" is unequal optimization.
These benchmarks don't make use of possible (and by experienced developers well known) optimizations on those frameworks ... at least for symfony2 and their tests this is a fact.
A few examples regarding the symfony2 setup used in their latest tests:
- "composer install" is not being called with the -o flag to dump an optimized classmap autoloader (code)
- Symfony2 will not use APC cache for Doctrine metadata annotations without apc_cli = 1 ( issue )
- the whole DI container is injected in the controller instead of only the few necessary services
- hereby setter injection is used -> creates object then call setContainer() method instead of injecting the container directly into the constructor (see: BenchController extends Controller extends ContainerAware)
- an alias ( $this->get('service_name') ) is used to retrieve services from the container instead of accessing it directly ($this->container->get('service_name') ). ( code )
- ...
the list continues ... but i guess you understood where this is leading. 90 open issues by now ... an endless story.
Development & Ressources
Ressources like servers and storage are cheap. Really cheap ... compared to development time.
I am a freelancer charging considerably common rates. you can either get 2-3 days of my time ... or a sh**load of computing power and storage!
When choosing a framework, you are also choosing a toolkit for rapid development - a weapon for your fight against the never completely satisfied, feature-creeping customer ... who will pay you well for his wishes.
As an agency (or a freelancer) you want to build feature-rich applications in short time. You will face points where you are stuck with something ... maybe a performance related issue. But you are facing development costs and time as well.
What will be more expensive? An additional server or an additional developer?