I am debugging one of my apps and noticed that the RAM on my SQL Server 2005 x64 box (running on a Windows 2003 R2 x64 ) is pegged and even going into the paging file.
I understand that SQL Server 2005 x64 just grabs everything it can, so this might be normal (???). How can I check how much memory it is actually using?
You need to run Perfmon (start->control panel->administrative tools->performance). The counters you want, I believe, are in SQL Server:Memory Manager. Target Server Memory is how much it's allowed to take if needed, Total Server Memory is how much it's currently using. If you click on Explain it'll pop open a box with the details.
By default it's uncapped, and should use all but a couple gb of memory (it leaves that for the OS). However, there are various bugs and memory leaks that can cause it to go into paging.