If I create an Azure Linux VM using PowerShell, how can I get its new ssh host key, so that I can install it in my local ssh/putty? Preferably the solution is also PowerShell code.
Old question, but for newcomers there is nowadays an alternative available by using run-command in Azure CLI. There is probably an equivalent for PowerShell too, but I have not investigated that.
az vm run-command invoke --name <your-vm-name> --command-id RunShellScript --scripts "cat /etc/ssh/ssh_host_ecdsa_key.pub"
will output a json document from which you can extract the public key. Beware though that this process is incredibly slow (~30 seconds per host), but you only need to run it once. See this gist for an example of how to update the known_hosts file with Ansible.