How does Microsoft Detours work and how do I use it to get a stack trace?
I am new to Microsoft Detours. I have installed it to trace the system calls a process makes. I run the following commands which I got from the web
syelogd.exe /q C:\Users\xxx\Desktop\log.txt
withdll.exe /d:traceapi.dll C:\Program Files\Google\Google Talk\googletalk.exe
I get the log file. The problem is I don't fully understand what is happening here. How does detours work? How does it trace the system calls? Also I don't know how to read the output in log.txt. Here is one line in log.txt
20101221060413329 2912 50.60: traceapi: 001 GetCurrentThreadId()
Finally I want to get the stack trace of the process. How can I get that?
Answer
Detours lets you intercept any function. It places a jmp in the address that you specify creating a trampoline to your code. Finally, you call the old function if you want to do it. To use Detours you have to inject your code in the process you want to intercept.
To simplify this process you can use Deviare API Hook which does all the injection staff and you can use intercept applications from any programming language that supports COM technology, including .NET, Delphi, C++, Python, etc.. After downloading the package you will find some examples in it. There is a console named DeviareCSharpConsole that let you intercept any API of any process showing full stack trace information.
This is the way Deviare API Hook works but is what you need to do if you want to create an application that hooks another process:
An agent should be created in the target process to intercept the APIs you want. To intercept these APIs you can use Detours but you have to code IPC staff that is not included in that library.
If you need to write code inside the target process using Deviare API Hook you can use Deviare Custom Hooks. This feature lets you intercept APIs and handle processed parameters asynchronously.