No mapping for the Unicode character exists in the target multi-byte code page

Question

i have a soap builded with Delphi 2007 and it work!

After a conversion in Delphi 10.1 Berlin i have a lot of exceptions:

No mapping for the Unicode character exists in the target multi-byte code page

the cause seem, when webbroker HttpApp parsing the request parameters, it raise this exception if request has some url encoded chars in the query string, eg.:

http://localhost/soap/soap.dll/action?param=%E0

in fact if i call URL.Decode() with %E0 (à url encoded):

TNetEncoding.URL.Decode('%E0');

it raise the same exception:

No mapping for the Unicode character exists in the target multi-byte code page

The problem seem unit System.NetEncoding on method TURLEncoding.DoDecode(const Input: string): string. This method try to convert url encoded chars only in UTF-8 without any fallback on Windows-1252. String %E0 is Windows-1252 encoding for à, but delphi only conver UTF-8 version: %C3%A0.

A small (not perfect, not elegant) fix is add a fallback:

try
  Result := TEncoding.UTF8.GetString(Bytes); // original Delphi 10.1 line
except
  on E: EEncodingError do Result := string(PChar(Bytes)); // fallback
end;

full code:

function TURLEncoding.DoDecode(const Input: string): string;

  function DecodeHexChar(const C: Char): Byte;
  begin
    case C of
       '0'..'9': Result := Ord(C) - Ord('0');
       'A'..'F': Result := Ord(C) - Ord('A') + 10;
       'a'..'f': Result := Ord(C) - Ord('a') + 10;
    else
      raise EConvertError.Create('');
    end;
  end;

  function DecodeHexPair(const C1, C2: Char): Byte; inline;
  begin
    Result := DecodeHexChar(C1) shl 4 + DecodeHexChar(C2)
  end;

var
  Sp, Cp: PChar;
  I: Integer;
  Bytes: TBytes;

begin
  SetLength(Bytes, Length(Input) * 4);
  I := 0;
  Sp := PChar(Input);
  Cp := Sp;
  try
    while Sp^ <> #0 do
    begin
      case Sp^ of
        '+':
          Bytes[I] := Byte(' ');
        '%':
          begin
            Inc(Sp);
            // Look for an escaped % (%%)
            if (Sp)^ = '%' then
              Bytes[I] := Byte('%')
            else
            begin
              // Get an encoded byte, may is a single byte (%<hex>)
              // or part of multi byte (%<hex>%<hex>...) character
              Cp := Sp;
              Inc(Sp);
              if ((Cp^ = #0) or (Sp^ = #0)) then
                raise EHTTPException.CreateFmt(sErrorDecodingURLText, [Cp - PChar(Input)]);
              Bytes[I] := DecodeHexPair(Cp^, Sp^)
            end;
          end;
      else
        // Accept single and multi byte characters
        if Ord(Sp^) < 128 then
          Bytes[I] := Byte(Sp^)
        else
          I := I + TEncoding.UTF8.GetBytes([Sp^], 0, 1, Bytes, I) - 1

      end;
      Inc(I);
      Inc(Sp);
    end;
  except
    on E: EConvertError do
      raise EConvertError.CreateFmt(sInvalidURLEncodedChar, [Char('%') + Cp^ + Sp^, Cp - PChar(Input)])
  end;
  SetLength(Bytes, I);

  // ------> MY FIX <------
  try
     Result := TEncoding.UTF8.GetString(Bytes); // Original line
  except
     on E: EEncodingError do Result := string(PChar(Bytes));
  end;
  // END FIX
end;

After a lot of search i founds the Bug fix list for RAD Studio 10.1 Berlin and it says this bug is fixed:

Webbroker HttpApp parsing request parameters and getting error "No mapping for the Unicode character exists in the target multi-byte code page"

but not work for me...

Answer 1

try use WEB.ReqMulti;

I have the same exception when I user WebBroker to handle a POST method from a web page when there are multi-byte character in form.

And after I added use WEB.ReqMulti in WebBroker's project, this exception was gone.