label.font.pointSize
is read-only, so I'm not sure how to change it.
You can do it like this:
label.font = UIFont(name: label.font.fontName, size: 20)
Or like this:
label.font = label.font.withSize(20)
This will use the same font. 20 can be whatever size you want of course.
Note: The latter option will overwrite the current font weight to regular
so if you want to preserve the font weight use the first option.
Swift 3 Update:
label.font = label.font.withSize(20)
Swift 4 Update:
label.font = label.font.withSize(20)
or
label.font = UIFont(name:"fontname", size: 20.0)
and if you use the system fonts
label.font = UIFont.systemFont(ofSize: 20.0)
label.font = UIFont.boldSystemFont(ofSize: 20.0)
label.font = UIFont.italicSystemFont(ofSize: 20.0)