A little background on what's running:
I have a Windows 10 desktop running Ubuntu through VM.. On this verision 16 of Ubuntu, I have the guacamole server, guacd, and tomcat running all nicely with no problems... I can access the guacamole client through my localhost on the VM... now I can see the RDP connection I created inside the user-mapping.xml file when I load the guacamole client, but whenevr I try to access it, within 2 seconds I get a "connection error: an internation error has occurred within the guacamole server, and the connection has been terminated."
System logs has no info other than what the terminal is saying (which is "listening on host x.x.x.x, port xxxx")
Now I'm not entirely sure if i set up the connection properly.. what I've done was enable remote access on my desktop (which is where the VM for ubuntu is) and took my computers IP (google whats my ip) and used that... heres a sample of my user-mapping.xml file
<user-mapping>
<authorize
username="test"
password="testtest">
<connection name="Windows 10">
<protocol>rdp</protocol>
<param name="hostname">x.x.x.x</param> //my desktops ip address (From google)
<param name="port">3389</param>
</connection>
</authorize>
</user-mapping>
You should check the IP address, because using "google what is my ip" is not reliable method of getting the ip address of machine you are working on. Google will return the public IP visible to Google, what is most probably not your machine IP. A better method to find ip is with commands 'ipconfig' in Windows or 'ifconfig' on Linux.
Next, it might be easier to debug RDP connection parameters and IP address using some desktop RDP client, because you can avoid possible errors in installation of Guacamole. Check if the RDP connection and IP address are correctly set using e.g. 'xfreerdp /v:' (should be run from terminal).
If the connection works from xfreerdp, then you can only debug Guacamole logs, try increasing debug level (explained here for both web app and guacd daemon https://guacamole.incubator.apache.org/doc/gug/configuring-guacamole.html)
If I understood correctly, you are trying to RDP from VM to the host; this will probably show the host desktop with VM on it, showing host desktop and so on recursively... I am not sure if this will work, better try from the other machine.