No console output when using AllocConsole and target architecture x86

Question

I have a WinForms project, and if the user want's a debug console, I allocate a console with AllocConsole().

All console output works normally with the target architecture set to "Any CPU", but when I change it to "x86" it doesn't output anything (Console.Read() still works as expected). If I open the EXE directly, the output works. It looks like Visual Studio redirects it into it's own "Output" window.

I also tried this answer, but it didn't work, I also tried Console.SetOut(GetStdHandle(-11)), which didn't work either.

Setting the target architecture to 'Any CPU' is no option for me.

So here are my two questions:

• Why is this only the case when the target architecture is set to x86?
• How can I output to my console when running inside of Visual Studio?

Answer 1

When "Enable native code debugging" is enabled, output from consoles crated with AllocConsole is redirected to the debug output window instead.

The reason this only happens in x86 and not AnyCPU is because you can only debug native code in an x86 application.

Note that this behavior only occurs with consoles created with AllocConsole. A console application's output is not redirected.

EDIT: The other reason for the console not outputting text is when you've written to the console before calling AllocConsole.

Regardless of the reason, this code will restore output if it was redirected, and reopen the console in case it's invalid. It uses the magic number 7 which is what the handle of stdout usually equals to.

using System;
using System.IO;
using System.Runtime.InteropServices;

public static class ConsoleHelper
{
    public static void CreateConsole()
    {
        AllocConsole();

        // stdout's handle seems to always be equal to 7
        IntPtr defaultStdout = new IntPtr(7);
        IntPtr currentStdout = GetStdHandle(StdOutputHandle);

        if (currentStdout != defaultStdout)
            // reset stdout
            SetStdHandle(StdOutputHandle, defaultStdout);

        // reopen stdout
        TextWriter writer = new StreamWriter(Console.OpenStandardOutput()) 
        { AutoFlush = true };
        Console.SetOut(writer);
    }

    // P/Invoke required:
    private const UInt32 StdOutputHandle = 0xFFFFFFF5;
    [DllImport("kernel32.dll")]
    private static extern IntPtr GetStdHandle(UInt32 nStdHandle);
    [DllImport("kernel32.dll")]
    private static extern void SetStdHandle(UInt32 nStdHandle, IntPtr handle);
    [DllImport("kernel32")]
    static extern bool AllocConsole();
}

See How to detect if Console.In (stdin) has been redirected? for another way to detect if the console handles have been redirected.