I searched a lot, but all are guessed answers. Help me to find the exact answer.
An MSI is a Windows Installer database. Windows Installer (a service installed with Windows) uses this to install software on your system (i.e. copy files, set registry values, etc...).
A setup.exe may either be a bootstrapper or a non-msi installer. A non-msi installer will extract the installation resources from itself and manage their installation directly. A bootstrapper will contain an MSI instead of individual files. In this case, the setup.exe will call Windows Installer to install the MSI.
Some reasons you might want to use a setup.exe: