What are the advantages of using one over the other ? I know POCO classes are more optimal but are they worth the overkill ? And should we always be using POCO's or is there a time when you should prefer entity framework classes ?
The EF default classes all inherit from EF base class, while POCO don't (hence the name). While inheriting from EF base class, the logic behind change tracking is hidden from you and all of the logic is stored inside the context that's holding references to your objects. This is preferred if you're working in a connected state, ie, you have context at the same time you have entities. This is usually the case where you build 'fat' client, so the client and the database are the only two tiers.
On the other hand, if you're working with web services / web forms, the entities you pass around don't have a context and have to track the state themselves - then the POCO is the better option, since they have change tracking object as their property that can be transferred around until you decide to apply the changes to the context and save. One other benefit would be that your clients don't have to be .NET and don't have to have the EF .dll to deserialize your objects.