Hi,

I was wondering, is it possible to make a program which uses a console user interface when started from prompt and GUI if its started from explorer for example?

I think I need to make a GUI app and somehow detect if its started from console and then make that console (not a new one) to accept my prints. But how to do that?

Sami
Posted on 2001-12-06 15:33:44 by SamiP
One thing you may be able to use is that when you run the program from a command prompt, it will have a console window, and when it's run from the GUI, it won't.
However I did a quick test with GetStdHandle, too see if it returned INVALID_HANDLE_VALUE when run from the GUI, but it didn't, although GetLastError returned "The handle is invalid"...

Thomas
Posted on 2001-12-06 15:45:44 by Thomas
I did a check of cmd.exe, and it has a huge amount of stuff that can be passed to it on the command line, my guess is it wouldn't be too hard to ShellExecute it, and pass it a command on the command line, i.e. to print some stuff to stdout. If you have a bunch of commands to execute, you could try putting them in a batch file, and execute that. Check the attached file for the command line options available for cmd.exe.

I would be curious to know how you get on with this one.
Posted on 2001-12-06 18:29:12 by sluggy
Thomas, if the subsystem is set to IMAGE_SUBSYSTEM_WINDOWS_CUI,
you should always get a console no matter if you run the app from
a shell or from explorer.

So the solution is probably to make a GUI app, and make it AllocConsole
if it's run from a shell. But I don't know How to detect whether it's
run from a console or a GUI app :/.
Posted on 2001-12-06 19:30:10 by f0dder
Thomas, if the subsystem is set to IMAGE_SUBSYSTEM_WINDOWS_CUI,
you should always get a console no matter if you run the app from a shell or from explorer.
.

Yes but when the subsystem is the default GUI subsystem, it will only have a console when run from the shell, as it takes over the shell command line. It won't have one in the GUI.
However the problem is detecting it. Although getlasterror returned the handle was invalid, it wasn't (i.e. it wasn't invalid_handle_value what GetStdHandle returned).

Thomas
Posted on 2001-12-07 01:20:11 by Thomas
Afternoon, SamiP.

Why do you want to have your proggy output to the console if it's executed from there? I'm just curious.

This all seems obvious. However, it's the obvious stuff which I'm always overlooking:tongue: , so I thought I might mention it:

You do realise that a Win32 proggy will run perfectly fine whether it's executed with Explorer or from a console.

Plus, remember: a console is not DOS. If you restarted your 'puter in DOS mode, your proggy won't run.

I, also, do not know how to get a proggy to detect how it's been executed. So I'm no help:rolleyes: . I'd be interested to know how to do this as well.

Cheers,
Scronty
Posted on 2001-12-07 03:07:59 by Scronty
Hi, I know it can be done (but I really don't know HOW!), and it can really be useful.

I have a tool, made by MS, that manipulates images (it adds mipmaps, and converts formats for DirectX compression).
If you run it via explorer, it has a gui, and you can see the differences while you edit the pic.

If instead you need to canvert a large number of textures to other formats, you can simply write a batch, launch the program via console, and you can process several of them without constant input from the user.
Useful, isn't it?

'Bye, Kefren
Posted on 2001-12-07 05:04:42 by kefren
This is just an idea, I have never actually done anything even near to it, but the GetConsoleTitle API can help. If I understand it correctly, it returns the title bar of the current console window. If it does, it means we are starting from one. If not, we are starting from GUI.

This comes from Docs:

The GetConsoleTitle function retrieves the title bar string for the current console window.

DWORD GetConsoleTitle(

LPTSTR lpConsoleTitle, // address of buffer for title
DWORD nSize // size of the buffer

);
Parameters

lpConsoleTitle

Points to a buffer that receives a null-terminated string containing the text that appears in the title bar of the console window.

nSize

Specifies the size, in characters, of the buffer pointed to by the lpConsoleTitle parameter.

Return Value

If the function succeeds, the return value is the length, in characters, of the string copied to the buffer.
If the function fails, the return value is zero. To get extended error information, call GetLastError.

Remarks

To set the title bar string for a console window, use the SetConsoleTitle function.


HTH, Kefren
Posted on 2001-12-07 05:16:43 by kefren
When I make a CON program it allways opens console, even when executed from explorer. So GetConsoleTitle and/or GetConsoleWindow both returns as they return when executed from console. So no luck...

When I make a GUI program GetConsoleTitle return allways empty string... so this cuold be used to determine if program is started from explorer or from console.

Now the problem is how to tell to GUI program that it needs to use existing console and not to allocate a new one?

Scronty: I need this because when I work in console its annoying that I can't use my programs from there. And sometimes I'm so lazy that I like just clickin different options :) Well, I can make two programs, one which runs in console mode and one which runs with gui... but it would be nice if I can join both programs to one exe.

Sami
Posted on 2001-12-07 05:39:23 by SamiP
Of course, you need to compile your prog as GUI, otherwise it will always open a console, then return the title bar of that console.

The CreateFile API can retrieve the console's handle, then you go on with the norma Console APIs.

HTH, Kefren
Posted on 2001-12-07 05:52:52 by kefren
Afternoon, SamiP.

yep. It'd definitely be useful. i.e. for an example as kefren suggested.

Where are you using "GetConsoleTitle" in your GUI proggy ? is it at the "start:" or in WinMain, or somewhere else?

Once you've found out whether the proggy's been started in a console, you don't have to create a new one. Just don't create a window/message loop/etc. and send all ouput to STDout.

(I haven't tried this, so it could be all BS);)

Cheers,
Scronty
Posted on 2001-12-07 05:55:16 by Scronty
When I make a GUI program GetConsoleTitle return allways empty string... so this cuold be used to determine if program is started from explorer or from console.


I wasn't thinking clearly. GetConsoleTitle returns allways empty string if it is called from program that is compiled to run from gui.

So it can't be used to determine if program is started from gui or con.

Because of that we are again at step one :(

I can't use CreateFile to obtain console handle if I call it from gui program. Atleast it doesn't obtain current consoles output handle.
Posted on 2001-12-07 06:48:40 by SamiP
The easy solution... if commandline arguments are passed, work
in console mode. If not, start in GUI mode =). Still the issue of using
the existing console remains...
Posted on 2001-12-07 10:24:16 by f0dder
I need to give up :(

I can't find a way to make console and GUI application on one exe.

If you have done it, I appreciate if you can mail to me or post here how this happens.

Sami
Posted on 2001-12-13 16:46:25 by SamiP
Afternoon, SamiP.

Don't give up yet ;).

There'll always be some work-around.
All I can think of at the moment:

Use a GUI proggy.
If a file is in the commandline, create a console (using CreateFile) and process it (as kefren suggested).
Otherwise, carry on with the usual GUI setup.

If a file is "drag/drop"ed onto your proggy, it'll process it in a console. Plus, you could use it in a batch file.

With regards to starting the proggy from the command prompt (console), I can't see why it'd be necessary to process a file in the *same* console.

Cheers,
Scronty
Posted on 2001-12-13 18:34:52 by Scronty
The reason GetStdHandle doesn't return INVALID_HANDLE_VALUE is because (from MSDN)


If an application does not have associated standard handles, such as a service running on an interactive desktop, and has not redirected them, the return value is NULL.


This article may be helpful, it suggests two methods and has some example code.

http://msdn.microsoft.com/msdnmag/issues/04/02/CQA/

N.B. there is a mistake in that article and MSDN (or my headers), the correct value to pass to AttachConsole is not ATTACH_PARENT_CONSOLE but rather ATTACH_PARENT_PROCESS, which is ((DWORD)-1)
Posted on 2005-06-07 09:54:43 by stormix
Unfortunately there is still the same problem... I need two files one .com and one .exe.

Now I just wonder what was the story about "own startup code"

Best regards,
Sami
Posted on 2005-06-08 14:09:43 by SamiP
After some fiddling I have come up with a better solution (I hope).

The idea is that you load have a generic "loader" as the .com application, to save you having to write your app twice. So the loader works by using LoadLibraryEx to map the .exe file into the address space (for this reason, the loader is based at 0x300000), then builds the .exe file's IAT (for some reason this is not done by LoadLibraryEx, nor LoadLibrary), then jumps to the .exe's entrypoint.

Here is the code, attached below is the whole vc.net 2003 project. If anyone wants then I'll comment it better :)


int main()
{
TCHAR szMod[256];
GetModuleFileName(GetModuleHandle(NULL), szMod, 256);
_tcscpy(&szMod[_tcslen(szMod)-4], _T(".exe"));
HMODULE hExe = LoadLibraryEx(szMod, NULL, DONT_RESOLVE_DLL_REFERENCES);
IMAGE_NT_HEADERS *nth = (IMAGE_NT_HEADERS*)(DWORD_PTR(hExe) + PIMAGE_DOS_HEADER(hExe)->e_lfanew);
DWORD_PTR pEntry = nth->OptionalHeader.AddressOfEntryPoint + DWORD_PTR(hExe);
// have to build IAT ourself
// unprotect IAT
LPVOID pIat = (LPVOID)(DWORD_PTR(hExe) + nth->OptionalHeader.DataDirectory.VirtualAddress);
DWORD cbIat = nth->OptionalHeader.DataDirectory.Size;
DWORD flOldProt;
VirtualProtect(pIat, cbIat, PAGE_EXECUTE_READWRITE, &flOldProt);
IMAGE_IMPORT_DESCRIPTOR* idt = (IMAGE_IMPORT_DESCRIPTOR*)(
DWORD_PTR(hExe) + nth->OptionalHeader.DataDirectory.VirtualAddress);
while(idt->Characteristics)
{
// load this library
HMODULE hLib = LoadLibrary((LPCTSTR)(DWORD_PTR(hExe)+idt->Name));
IMAGE_THUNK_DATA *th1 = (IMAGE_THUNK_DATA*)(DWORD_PTR(hExe) + idt->FirstThunk);
IMAGE_THUNK_DATA *th2 = (IMAGE_THUNK_DATA*)(DWORD_PTR(hExe) + idt->OriginalFirstThunk);
while(th1->u1.Function)
{
if(th2->u1.Ordinal & 0x80000000)
{
th1->u1.Function = (DWORD)(DWORD_PTR)GetProcAddress(hLib, (LPCSTR)(DWORD_PTR)(th2->u1.Ordinal & 0xffff));
}
else
{
LPCSTR szFuncName = (LPCSTR)(DWORD_PTR(hExe) + th2->u1.AddressOfData + 2);
th1->u1.Function = (DWORD)(DWORD_PTR)GetProcAddress(hLib, szFuncName);
}
// next thunk
th1++;
th2++;
}
// next descriptor
idt++;
}
// restore IAT protection
DWORD dummy;
VirtualProtect(pIat, cbIat, flOldProt, &dummy);
// jump to our program's entrypoint
__asm {
call dword ptr
mov dummy,eax
}
return (int)dummy;
}


And I tested it on Windows XP SP2, using this as the .exe file


int WINAPI WinMain(HINSTANCE hInst, HINSTANCE, LPSTR lpCmdLine, int nCmdShow)
{
HANDLE hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
HANDLE hStdIn = GetStdHandle(STD_INPUT_HANDLE);
if(hStdIn == NULL) // we don't have a console
{
MessageBox(HWND_DESKTOP, "Hi from GUI!", "GUI", MB_OK | MB_ICONINFORMATION);
}
else // use the console
{
DWORD dw;
WriteFile(hStdOut, "Hi from exe\r\n", 13, &dw, NULL);
WriteFile(hStdOut, "Enter something: ", 17, &dw, NULL);
char buf[256];
ReadFile(hStdIn, buf, 256, &dw, NULL);
WriteFile(hStdOut, buf, dw, &dw, NULL);
}
return 0;
}


EDIT: changed example code to detect whether to use GUI or console
Attachments:
Posted on 2005-06-08 21:01:10 by stormix
well a brief scan over the thread shows this blog wasnt linked
it has some interesting obsevation regarding the topic
hxxp://blogs.msdn.com/junfeng/archive/2004/02/06/68531.aspx

also i remembered having seen some kind of enumeration here regarding console vs gui though i cant locate it now over here

hxxp://www.catch22.net/tuts/undoc01.asp

make sure you edit xx to tt :)

apart from that you can also look into gflags utility from reskit
it works in both console as well as gui mode
it looks for arguments if there is an argument it runs console mode else it opens a gui :)

also in console mode it uses a different set of apis to achieve the result and in gui it uses another set of apis

for example if you do gflags -k -sls
it will use NtSetSystemInformation (undocumented) api but if you set the same flag via
gui it will use some Propertysheethandler and use commcontrol.dll to do some jumping here and there
Posted on 2005-06-09 07:18:55 by bluffer

well a brief scan over the thread shows this blog wasnt linked
it has some interesting obsevation regarding the topic
hxxp://blogs.msdn.com/junfeng/archive/2004/02/06/68531.aspx

also i remembered having seen some kind of enumeration here regarding console vs gui though i cant locate it now over here

hxxp://www.catch22.net/tuts/undoc01.asp

make sure you edit xx to tt :)

apart from that you can also look into gflags utility from reskit
it works in both console as well as gui mode
it looks for arguments if there is an argument it runs console mode else it opens a gui :)

also in console mode it uses a different set of apis to achieve the result and in gui it uses another set of apis

for example if you do gflags -k -sls
it will use NtSetSystemInformation (undocumented) api but if you set the same flag via
gui it will use some Propertysheethandler and use commcontrol.dll to do some jumping here and there



Thank you but that seems to all be rather irrelevant.

As the link I provided says, its easy to just make a console app that also pops up a GUI. gflags doesn't even get rid of its old console when it pops up the GUI, and what does NtSetSystemInformation have to do with anything? And it's also no problem checking if you have a console or not, you just check the return values of GetStdHandle: they are NULL if there's no console. The problem is how to use the parent console if you're using the WINDOWS subsystem.

The second link, about createprocess has nothing to do with this topic??!?!

Also, why on earth do you use hxxp for your links?
Posted on 2005-06-09 10:11:30 by stormix