String.getBytes(charset) has errors for EBCDIC-charset

Ulrich picture Ulrich · Aug 11, 2013 · Viewed 7.8k times · Source

The String-conversion to EBCDIC via String.getBytes(charset) supplys at least one false result. The character "a" becomes a 0x3f but should be 0x81.

public static void  convert() throws UnsupportedEncodingException {
    String data="abcABC";
    String ebcdic = "IBM-1047";
    String ascii  = "ISO-8859-1";

    System.out.printf("Charset %s is supported: %s\n", ebcdic, Charset.isSupported(ebcdic));
    String result= new String(data.getBytes(ebcdic));
    System.out.printf("EBCDIC: %s\n",asHex(result.getBytes()));

    System.out.printf("Charset %s is supported: %s\n", ascii, Charset.isSupported(ascii));
    result= new String(data.getBytes(ascii));
    System.out.printf("ASCII: %s\n",asHex(result.getBytes()));
}

public static String asHex(byte[] buf) {
    char[] HEX_CHARS = "0123456789abcdef".toCharArray();
    char[] chars = new char[2 * buf.length];
    for (int i = 0; i < buf.length; ++i)
    {
        chars[2 * i] = HEX_CHARS[(buf[i] & 0xF0) >>> 4];
        chars[2 * i + 1] = HEX_CHARS[buf[i] & 0x0F];
    }
    return new String(chars);
}

The result ist:

  • Charset IBM-1047 is supported: true
  • EBCDIC: 3f8283c1c2c3
  • Charset ISO-8859-1 is supported: true
  • ASCII: 616263414243

Anything I can do about this?

Answer

Joni picture Joni · Aug 11, 2013

When you call

data.getBytes(ebcdic)

You are encoding the text in data into EBCDIC bytes. Then you create a string from these bytes as if they stood for some string in the default character encoding for your system: this causes breakage because the bytes don't have to encode valid text in any other encoding than EBCDIC.

To fix this, keep bytes as bytes:

byte[] result= data.getBytes(ebcdic);
System.out.printf("EBCDIC: %s\n",asHex(result));