l10N and I18N of Perl

A Brief History of time()

Character ≡ Byte
- $string =~ /(漢字)/; # Does not work!
You could still treat multibyte characters as binary data.
- print "漢字\n"; # Does work!
But there are so many ways to spell "漢字" in Japan!

Shift_JIS Most widely used 8a bf 8e 9a

EUC-JP Most elegant b4 c1 bb fa

ISO-2022-JP SMTP-safe 1b 24 42 34 41 3b 7a 1b 28 42

UTF-8: New kid on the block e6 bc a2 e5 ad 97

You need at least a transcoder just to print() right!
FYI, other CJKT worlds do not suffer as much as Japan.
- China: GB2312 (EUC-coded)
- Korea: KSC5601 (EUC-coded)
- Taiwan: Big5 (Shift_JISish)

  require "jcode.pl";
  while(<>){
    jcode::convert(*_, 'euc', 'sjis');
    print;
  }

First Japanese transcoder for perl.
For Perl 4.
Written in 100% perl.
- Conversions are done algorithmically.
- No Unicode support.
- No Perl5 Features like OO.

  while(<>){
        print Jcode->new($_, 'sjis')->euc;
  }

jcode.pl for Perl 5
- 100% compatible w/ jcode.pl -- well, almost.
- Perl5 OO savvy.
- Unicode support added via XS.
- MIME B encoding supported for mails.
Pretty Comprehensive -- so long as you are Japanese.
- just a L10N, no I18N.
- still just a converter -- $string =~ /(漢字)/;# won't work
- Unicode support added via XS.
- MIME B encoding supported for mails.

Character ≡ UTF-8 ∋ Byte
- $string =~ /(漢字)/; # works at last!
It ONLY supports UTF-8!
- while most text data in the world are in legacy encodings.
- so it is still a hard thing that is just possible.
You still need a transcoder!
We wanted it to be an easy thing that can be done easily!

A suit of modules to handle utf8-legacy transcodings

  use Encode;
  while(<>){
    from_to($_, 'shiftjis', 'euc-jp');
    print;
  }

Tightly coupled w/ perl core so you can use it implicitly.

  binmode STDIN  => ":encoding(shiftjis)";
  binmode STDOUT => ":encoding(euc-jp)";
  print while(<>);

Characters are now characters, not just series of bytes!

  binmode STDIN  => ":utf8";
  binmode STDOUT => ":utf8";
  while(<STDIN>){
    s/(.)/[$1]/g; print;
  };

謝謝合作！