The question is in the title : How to remove buttons from CKeditor 4 .
Documentation does not answer it clearly
Based on reinmar answer and tested here is the better answer. Add this in your ckeditor config.js :
config.removeButtons = 'Underline,JustifyCenter';
For reference you can find the complete list of CKeditor 4 buttons there : http://ckeditor.com/comment/123266#comment-123266