FiveTech Support Forums

Hi Antonio,

Subject: Regression in is2byteansi() (fwunicode.c) breaks ANSI CP1254 (Turkish) GET input — shows "?" — June 2026 build

Environment

• FiveWin: FWH 26.xx (June 2026 build), 32-bit, MSVC
• Harbour 3.2.0dev
• Legacy non-Unicode application: startup calls HB_CDPSELECT( "TRWIN" ) and FW_SetUnicode( .F. ). All data is single-byte Windows-1254 (Turkish). We do not use UTF-8.

Symptom / my question
After updating FiveWin, typing Turkish characters (ş ğ ı İ Ş Ğ ç ö ü …) into ordinary GET fields displays ? instead of the character. The same application + source worked correctly with the previous FWH build. What is the supported way to run a non-Unicode CP1254 (TRWIN) application with the new FWH so that ANSI Turkish input is preserved?

Root cause I found
is2byteansi() in source/winapi/fwunicode.c was rewritten. For the Western/Central-European single-byte codepages (CP1250–1258, which includes CP1254 Turkish) the new code now returns:

if ( iCP >= 1250 && iCP <= 1258 )
{
    return ( FW_GetUnicode() ? FALSE : TRUE );
}

Because our app runs FW_SetUnicode(.F.), FW_GetUnicode() is false, so for CP1254 it now returns TRUE.

The previous FWH version special-cased only Thai (CP874) and returned FALSE for every other codepage (including CP1254):

BOOL is2byteansi( unsigned char c1, unsigned char c2 )
{
    if ( ( ( fwCDP == 0 ) ? GetACP() : fwCDP ) == 874 )
    {
        if ( c1 >= 161 && c1 <= 251 )
            return ( ( c2 == 0 ) || ( c2 >= 161 && c2 <= 251 ) );
    }
    return FALSE;
}

This return-value flip changes how isutf8() classifies legacy CP1254 text, and the result is the ? shown in GET fields for Turkish characters. Our stored data is genuine single-byte CP1254 (verified at byte level, e.g. Ğ Ü Ş İ Ö Ç ğ ü ş → D0 DC DE DD D6 C7 F0 FC FE, one byte per character, LENGTH() == CHAR_LENGTH(), no 0x3F).

Workaround I applied (without rebuilding the FWH library)
I added a small C unit to the application that re-defines is2byteansi() with the old behavior, compiled with C linkage and linked before FiveHC32.lib; with /FORCE:MULTIPLE the linker binds every call (including the library's own isutf8()) to my version (LNK4006: _is2byteansi already defined …; second definition ignored):

#include <windows.h>
extern "C" int is2byteansi( unsigned char c1, unsigned char c2 )
{
   if ( GetACP() == 874 )                       /* Thai only, as in old FWH */
   {
      if ( c1 >= 161 && c1 <= 251 )
         return ( ( c2 == 0 ) || ( c2 >= 161 && c2 <= 251 ) );
   }
   return 0;                                     /* CP1254 & others: FALSE  */
}

This restores correct Turkish input.

Questions to you

1. Is the new is2byteansi() behavior (returning TRUE for CP1250–1258 when FW_SetUnicode(.F.)) intended for non-Unicode apps, or is the ? an unintended regression?
2. For a legacy ANSI CP1254 / TRWIN application that does not use UTF-8, what is the recommended configuration with the new FWH (e.g. a documented FW_SetCDP() / FW_SetUnicode() combination) to get the previous correct behavior without patching is2byteansi()?

Dear Hakan,

Thank you for the excellent and detailed report — your byte-level analysis made this very easy to confirm.

1. Intended or regression?

   It is an unintended regression, now fixed. Your diagnosis of is2byteansi() was correct, but the actual failure point
   is one level higher, in TGet's keyboard handling:

2. With a comctl32 v6 manifest the EDIT control is Unicode, so WM_CHAR delivers UTF-16 codes. Turkish ş ğ ı İ Ş Ğ have
   codepoints above 255 (ş = U+015F = 351).

3. TGet:KeyChar() routes those through the internal WideKey2Asc(), which builds a genuine UTF-8 sequence from the
   keystroke (ş → C5 9F) and converts it to ANSI via fw_Utf8ToStr().

4. fw_Utf8ToStr() relies on the isutf8()/is2byteansi() heuristic. The new CP1250–1258 branch classifies C5 9F as an
   ANSI pair, so the conversion is refused, the keystroke stays > 255, and KeyChar() falls into its "reject unicode
   chars" branch — which inserts the literal "?" you saw.

   The CP1250–1258 change itself was made to fix real false positives (accented ANSI text being misdetected as UTF-8 in
   display paths), so it stays. The bug was that the keyboard path used that heuristic on a string it already knew was
   UTF-8.

   The fix: WideKey2Asc() in tget.prg now converts with explicit codepages — FW_CdpTranslate( cUtf8, 65001, -1 ) —
   bypassing the heuristic entirely. Verified with your exact configuration: WM_CHAR codes 351/287/304 sent to a GET
   under HB_CDPSELECT("TRWIN") + FW_SetUnicode(.F.) now produce the correct single bytes FE F0 DD in the buffer. Two test
   samples are included (samples\test\trwincdp.prg and trwinget.prg). The same regression also affected Polish, Czech,
   Cyrillic, Greek and Baltic single-byte codepages — all covered by the same fix.

5. Recommended configuration?

   Your current setup — HB_CDPSELECT("TRWIN") + FW_SetUnicode(.F.) — is exactly the supported configuration for a legacy
   single-byte CP1254 application. No FW_SetCDP() or other settings are required (FW_SetCDP(1254) is only needed if the
   machine's ANSI codepage is not 1254). Once you update to the next build, please remove your is2byteansi() override and
   the /FORCE:MULTIPLE linker flag: the override re-introduces the UTF-8 false positives in display paths that the new
   code fixes, so it should not stay in place longer than necessary.

   The fix will be included in the next FWH build. Thank you again for the precise report.