G'day All,

I'm trying to display a dialog with Unicode/wide charactersets.

The dialog works OK in normal mode (DialogBoxParam), but does not display the
wide (Cyrillic-Russian) characters correctly in wide mode (DialogBoxParamW).

The wide (Cyrillic-Russian) characters do display correctly in a test message box
(MessageBoxW).

Also the dialog title in Cyrillic-Russian (SendMessageW) is also displayed correctly.

The problem is with the Edit and Combo controls in wide mode. These
controls display garbage in wide mode :(

Can anyone see what I am doing wrong?

Thanks
dorf

Setup:
Win2K
Masm32 v.8.2 SP2a
RadASM v.2.2.0.3

Dialog resource - RadSM generated:

#define IDD_UNIDLG 4000
#define IDC_UNIEDTCTL 4002
#define IDC_UNIEDTLBL 4001
#define IDC_UNICBOLBL 4003
#define IDC_UNICBOCTL 4004
IDD_UNIDLG DIALOGEX 6,6,248,134
CAPTION "My Dialog"
FONT 8,"MS Sans Serif",400,0
STYLE 0x10CF0000
EXSTYLE 0x00000000
BEGIN
  CONTROL "",IDC_UNIEDTCTL,"Edit",0x50010000,122,24,116,11,0x00000200
  CONTROL "My Edit",IDC_UNIEDTLBL,"Static",0x50000000,6,24,96,11,0x00000000
  CONTROL "My Combo",IDC_UNICBOLBL,"Static",0x50000000,6,59,100,11,0x00000000
  CONTROL "",IDC_UNICBOCTL,"ComboBox",0x50010002,122,59,112,62,0x00000080
END



.code

invoke IsWindowsNT
.if(eax)
invoke DialogBoxParamW, hInstance, IDD_UNIDLG, hWndMain, offset SciUniDialogProc, FALSE
.else
; Not WinNT or Win2000
invoke DialogBoxParam, hInstance, DD_UNIDLG, hWndMain, offset SciUniDialogProc, FALSE
.endif


SciUniDialogProc proc uses ebx esi edi, hDlg:HWND, uMsg:UINT, wParam:WPARAM, lParam:LPARAM
LOCAL codePage:dword
LOCAL uniStr[128]:word
LOCAL hCtlEdit:dword
LOCAL hCtlCombo:dword

mov eax, uMsg
.if eax==WM_INITDIALOG
mov eax, hDlg
mov hFindReplace, eax
; Set the codepage to Cyrillic-Russian
mov codePage, 1251

invoke GetDlgItem, hFindReplace, IDC_UNIEDTCTL
mov hCtlEdit, eax
invoke GetDlgItem, hFindReplace, IDC_UNICBOCTL
mov hCtlCombo, eax

invoke IsWindowUnicode, hDlg
.if(eax)
invoke Beep, 100, 100
; Get the buffer length (in wide characters)
invoke MultiByteToWideChar, codePage, 0, addr sciTEBase.findWhat, \
-1, addr uniStr, 0
; Map the character string to a wide-character (Unicode) string.
invoke MultiByteToWideChar, codePage, 0, addr sciTEBase.findWhat, \
-1, addr uniStr, eax

; Messagebox (wide) displays the Cyrillic text <<< OK
invoke MessageBoxW,0, addr uniStr, CStr('Cyrillic Characters:'), MB_OK

; Set the dialog title <<< OK
invoke SendMessageW, hDlg, WM_SETTEXT, 0, addr uniStr

; Set the dialog Edit control: <<< Garbage
invoke SetDlgItemTextW, hDlg, IDC_UNIEDTLBL, addr uniStr
invoke SetDlgItemTextW, hDlg, IDC_UNIEDTCTL, addr uniStr

; Set the dialog Combo control: <<< Garbage
invoke SetDlgItemTextW, hDlg, IDC_UNICBOLBL, addr uniStr
invoke SendMessageW, hCtlCombo, WM_SETTEXT, 0, addr uniStr
invoke SendMessageW, hCtlCombo, CB_ADDSTRING, 0, addr uniStr

.else
; ASCII 8-Bit - ALL OK
; Set the dialog title - OK
invoke SendMessage, hDlg, WM_SETTEXT, 0, CStr('Cyrillic Dialog')

; Set the dialog Edit control:
invoke SetDlgItemText, hDlg, IDC_UNIEDTLBL, CStr('Cyrillic Edit Control:')
invoke SetDlgItemText, hDlg, IDC_UNIEDTCTL, CStr('Cyrillic Edit Control:')

; Set the dialog Combo control:
invoke SetDlgItemText, hDlg, IDC_UNICBOLBL, CStr('Cyrillic Combo Control:')
invoke SendMessage, hCtlCombo, WM_SETTEXT, 0, CStr('Cyrillic Combo Control:')
invoke SendMessage, hCtlCombo, CB_ADDSTRING, 0, CStr('Cyrillic Item 1')
.endif

return TRUE

.elseif eax==WM_COMMAND
; ...
;.elseif eax==WM_ACTIVATE
; ...
.elseif eax==WM_CLOSE
; ...Sci
.else
return FALSE
.endif

return TRUE

SciUniDialogProc endp

Posted on 2005-08-28 22:08:11 by dorf
when i had a lang problem i'v put this line in start of <project>.rc file.
LANGUAGE LANG_CROATIAN, SUBLANG_NEUTRAL

default is
LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US
Posted on 2005-08-29 07:05:49 by drizz
You can also try setting the code page using project options.

This example sets codepage 1250:

4,O,$B\RC.EXE /c1250 /v,1

KetilO
Posted on 2005-08-29 07:20:23 by KetilO
i think you can try add manifest in you resource file , it useful with me on winxp .
Posted on 2005-08-29 08:17:09 by secmask
2.1.0.2
- Added LANGUAGE settings to resources.


OMG i just noticed that i didn't update my radasm.ini file with this section.

KetilO: do i need to manually add "#include <Res\xxxxxxLng.rc>" to project.rc or ?
Posted on 2005-08-29 08:33:04 by drizz
Thanks guys

I should have also mentioned that I have to change codepage and
chararacter sets on-the-fly (user selectable).

KetilO:
4,O,$B\RC.EXE /c1251 /v,1
Sorry :( didn't work

drizz:
Do you mean something like below:
This will not compile for me :(
But I found an interesting link on MS relating to LANGUAGE definition:
http://www.microsoft.com/globaldev/handson/dev/muiapp.mspx
I'll check it out, but I don't think it is quite what I need. I need to
change CP/CS on-the-fly.


IDD_UNIDLG DIALOGEX 6,6,251,134
CAPTION "My Dialog"
FONT 8,"MS Sans Serif",400,0
STYLE 0x10CF0000
LANGUAGE LANG_CROATIAN, SUBLANG_NEUTRAL
EXSTYLE 0x00000000
BEGIN
  CONTROL "",IDC_UNIEDTCTL,"Edit",0x50010000,122,24,116,11,0x00000200
  CONTROL "My Edit",IDC_UNIEDTLBL,"Static",0x50000000,6,24,96,11,0x00000000
  CONTROL "My Combo",IDC_UNICBOLBL,"Static",0x50000000,6,59,100,11,0x00000000
  CONTROL "",IDC_UNICBOCTL,"ComboBox",0x50010002,122,59,112,62,0x00000080
END


secmask:
Can you please post a small example of how to do this. I don't think this is what
I need here, but am interested in how to do it anyway.


About the original concept. The Scintilla guys gave got this to work
well (in C++). SciTE can switch CP/CS on-the-fly. I'm trying to port the dialog
to masm32.

Ref:
A free source code editing component for Win32 and GTK+
www.scintilla.org
Download Scintilla and SciTE
http://scintilla.sourceforge.net/SciTEDownload.html

Here is the original dialog from the SciTE source:
Their setup procedure is much as I mine above.

IDD_FIND DIALOG 30, 73, 275, 84
STYLE DS_MODALFRAME | DS_3DLOOK | WS_POPUP | WS_CAPTION | WS_SYSMENU
CAPTION "Find"
FONT 8, "MS Shell Dlg"
BEGIN
LTEXT          "Fi&nd what:",-1,5,7,45,8
COMBOBOX        IDFINDWHAT,50,5,145,50, WS_TABSTOP | CBS_DROPDOWN | CBS_AUTOHSCROLL
AUTOCHECKBOX    "Match &whole word only", IDWHOLEWORD,5,22,120,10, WS_GROUP | WS_TABSTOP
AUTOCHECKBOX    "Match &case", IDMATCHCASE,5,34,130,10, WS_GROUP | WS_TABSTOP
AUTOCHECKBOX    "Regular &expression", IDREGEXP,5,46,120,10, WS_GROUP | WS_TABSTOP
AUTOCHECKBOX    "Wrap aroun&d", IDWRAP,5,58,120,10, WS_GROUP | WS_TABSTOP
AUTOCHECKBOX    "Transform &backslash expressions", IDUNSLASH,5,70,160,10, WS_GROUP | WS_TABSTOP
GROUPBOX        "Direction", -1, 135,22,60,34,WS_GROUP
AUTORADIOBUTTON "&Up",IDDIRECTIONUP,140,30,45,12, WS_GROUP
AUTORADIOBUTTON "&Down",IDDIRECTIONDOWN,140,42,45,12
DEFPUSHBUTTON  "&Find Next",IDOK,205,5,65,14,WS_GROUP
PUSHBUTTON      "&Mark All",IDMARKALL,205,23,65,14
PUSHBUTTON      "Cancel",IDCANCEL,205,41,65,14
END


Thanks
dorf
Posted on 2005-08-29 09:10:34 by dorf
dorf: check if this works for you, i have never written anything multilingual.

#define LANG_RUSSIAN 0x19
#define SUBLANG_NEUTRAL 0x00
#define LANG_CROATIAN 0x1A
#define SUBLANG_CROATIAN_CROATIA 0x01
LANGUAGE LANG_RUSSIAN,SUBLANG_NEUTRAL
100 DIALOGEX 6,6,251,134
CAPTION "?????????"
FONT 8,"MS Sans Serif",400,0
STYLE 0x10CF0000
BEGIN
END
LANGUAGE LANG_CROATIAN,SUBLANG_CROATIAN_CROATIA
100 DIALOGEX 6,6,251,134
CAPTION "Hrvatski"
FONT 8,"MS Sans Serif",400,0
STYLE 0x10CF0000
BEGIN
END


MAKELANGID macro p,s
;#define MAKELANGID(p, s) ((((WORD  )(s)) << 10) | (WORD  )(p))
exitm %((s shl 10) or p)
endm
.if Lang == LANG_RUSSIAN
mov eax,MAKELANGID(LANG_RUSSIAN,0)
.elseif Lang == LANG_CROATIAN
mov eax,MAKELANGID(LANG_CROATIAN,1)
.endif
invoke FindResourceEx,hInstance,RT_DIALOG,100,eax
.if eax
mov eax,.IMAGE_RESOURCE_DATA_ENTRY.OffsetToData
add eax,hInstance
invoke DialogBoxIndirectParam,hInstance,eax,0,offset DlgProc,0
.endif

btw setthreadlocale won't work on 2k/XP
Posted on 2005-08-29 19:51:03 by drizz
drizz:
I got your RUSSIAN/CROATIAN dialogs to display OK
The CROATIAN text displays correctly
The RUSSIAN text is still broken :sad:

But that leads to more interesting research:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceinternational5/html/wce50lrfmakelangid.asp
MAKELANGID macro
Language Identifiers and Locales
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceinternational5/html/wce50conlanguageidentifiersandlocales.asp

This multilingual subject is certainly a head-banger  :)

Thanks
dorf
Posted on 2005-08-29 21:58:08 by dorf
Got it figured  :D

Here's how I figured it out:
I actually gave up on this for now. So I thought I would have a browse for a few minutes.

The "program breaks" topic by czDrillard looks interesting, so I had a quick browse through
the posts. The link "exception handling - Jeremy Gordon" by donkey in looks good so I'll check
it out.

After reading the "exception handling" page I'll check out what else Jeremy
Gordon has on his site (http://www.jorgon.freeserve.co.uk/). Ahhh some stuff
on Unicode!

Taking a look at "Hello Unicode 1 A simple program with a Unicode UTF-8" under
the Advanced heading, see:
---
; If you see question marks instead of Russian in the MS-DOS (Command prompt)
; window (console) right click on the title bar and change font to Lucida Console.
; That font is capable of drawing Unicode characters to the console.
---

That looks something like the problem!

Try changing the font of the dialog to "Arial" (in RadASM project->dialog->Font) and
re-build! Bingo ... thats it :))

So - here is what the dialog definition should look like (Using a font that can do
the Unicode character set on a particular system):


#define IDD_UNIDLG 4000
#define IDC_UNIEDTCTL 4002
#define IDC_UNIEDTLBL 4001
#define IDC_UNICBOLBL 4003
#define IDC_UNICBOCTL 4004
IDD_UNIDLG DIALOGEX 5,5,218,118
CAPTION "My Dialog"
FONT 9,"Arial",400,0
STYLE 0x10CF0000
EXSTYLE 0x00000000
BEGIN
  CONTROL "",IDC_UNIEDTCTL,"Edit",0x50010000,105,21,100,10,0x00000200
  CONTROL "My Edit",IDC_UNIEDTLBL,"Static",0x50000000,5,21,82,9,0x00000000
  CONTROL "My Combo",IDC_UNICBOLBL,"Static",0x50000000,5,51,85,9,0x00000000
  CONTROL "",IDC_UNICBOCTL,"ComboBox",0x50010002,105,51,96,54,0x00000080
END



Heres a bit on Unicode/fonts if anyone is interested:

Unicode Fonts
http://www.jorgon.freeserve.co.uk/GoasmHelp/Unicode.htm

--- cut ---
The trick is to find the correct font and character set to draw the characters
which need to be drawn.

Not all fonts and character sets will be installed on all machines. What fonts
and character sets are loaded on installation will depend on which version of
Windows is being used, whether it is an English or other language version, and
what applications are loaded (they can come with their own fonts).

Every Windows machine will have installed Courier New, Arial, Times New Roman,
Symbol, Wingdings, MS Serif, MS Sans Serif and (on later versions) Tahoma.
Depending on the Windows version, between them these fonts can handle at least
Western and Central European, Hebrew, Arabic, Greek, Turkish, Baltic, Cyrillic and
Vietnamese scripts.

You can see what fonts are loaded on your machine by looking in the Windows\fonts
folder or at "Control Panel, fonts".
--- cut ---

Regards
dorf

Posted on 2005-08-30 22:42:23 by dorf
A good site for people developing International software:

Michael Kaplan's - Sorting It All Out Blog
http://blogs.msdn.com/michkap/

Lot of indepth Localization / character encoding discussion and security issues

Here's some security (buffer overflow) issues to watch for when using encoding

Encoding APIs and Security Concerns
http://blogs.msdn.com/michkap/archive/category/8717.aspx
---cut---
A few of the gotchas of WideCharToMultiByte and A few of the gotchas of MultiByteToWideChar.

Both API topics (MultiByteToWideChar and WideCharToMultiByte) include "Security Alert" warnings
to which developers should pay heed:

security note Security Alert  Using the WideCharToMultiByte function incorrectly can compromise
the security of your application. Calling the WideCharToMultiByte function can easily cause a
buffer overrun because the size of the In buffer equals the number of WCHARs in the string,
while the size of the Out buffer equals the number of bytes. To avoid a buffer overrun, be sure
to specify a buffer size appropriate for the data type the buffer receives. For more information,
see Security Considerations: International Features.

security note Security Alert  Using the MultiByteToWideChar function incorrectly can compromise
the security of your application. Calling the MultiByteToWideChar function can easily cause a buffer
overrun because the size of the In buffer equals the number of bytes in the string, while the size
of the Out buffer equals the number of WCHARS. To avoid a buffer overrun, be sure to specify a
buffer size appropriate for the data type the buffer receives. For more information, see Security
Considerations: International Features.

It seems obvious from the parameter names (cbMultiByte vs. cchWideChar) that one parameter is a
count of bytes and the other is a count of characters. But if you mess it up then it is easy to cause
a buffer overrun as the alerts indicate, which is obviously bad. Another problem that can happen is
that the call could fail due to insufficient buffer size.

Now historically people seem to like calling these APIs without checking the return value. Or alternately
they call with a NULL target buffer to get the size, allocate a target buffer based on that size, and then
call the API again without checking the return value. Obviously both of the possible problems can be bad
...
--- cut ---

dorf

Posted on 2005-09-01 17:50:58 by dorf
oh , i don't think add manifest resource is new .people add this to resource to has XP look control . but when i try to type some unicode text , it won't look as unicode character if i didn't add this resource . if you aren't family with this you can see Radasm box , there are some easy way to add manifest resource there . good luck.
Posted on 2005-09-04 09:01:00 by secmask