I found that when clicking "Submit Patch Set" in the Gerrit web interface, it'll either simply add a commit to that branch, or create a merge commit if another commit was submitted just before.
Example that creates 2 commit: The actual commit and a merge commit:
There is a "Rebase Change" button which is great but it means that to submit a patch set everyone should always do:
The only reason I see a merge commit useful is to keep dates of the commits (but I do understand why it is required without a rebase).
Isn't there an automatic rebase or at least a check to avoid generating an undesired merge commit?
Yes. Change the Submit Action for your project(s) to Cherry Pick. This will do roughly the same as rebasing when the submit button is pressed. It will keep the clean history you are looking for with no merge commits when submitting changes.