Currently I'm using the following function
file.Delete();
But how can I use this function to send a file to the recycle bin instead of just deleting it outright?
Use FileSystem.DeleteFile and specify the right RecycleOption.
While this will work with UI Interactive Apps, it will not work with non UI interactive apps like a Windows Service app.