View on
INABA Hitoshi > Char-GBK-1.06 > GBK



Annotate this POD


Open  0
View/Report Bugs
Module Version: 1.06   Source   Latest Release: Char-GBK-1.08


GBK - Source code filter to escape GBK script

Install and Usage ^

There are two steps there:


  use GBK;
  use GBK ver.sion;             --- require minimum version
  use GBK ver.sion.0;           --- expects version (match or die)
  use GBK qw(ord reverse getc); --- demand enhanced feature of ord, reverse, and getc
  use GBK ver.sion qw(ord reverse getc);
  use GBK ver.sion.0 qw(ord reverse getc);

  # "no GBK;" not supported


  $ perl >


  $ perl  --- script written in GBK --- escaped script



  dummy functions:


GBK software is "middleware" between perl interpreter and your Perl script written in GBK.

Perl is optimized for problems which are about 90% working with text and about 10% everything else. Even if this "text" doesn't contain GBK, Perl3 or later can treat GBK as binary data.

By "use GBK;", it automatically interpret your script as GBK. The various functions of perl including a regular expression can treat GBK now. The function length treats length per byte. This software does not use UTF8 flag.

Yet Another Future Of ^

JPerl is very useful software. -- Oops, note, this "JPerl" means "Japanized Perl" or "Japanese Perl". Therefore, it is unrelated to JPerl of the following.

 JPerl is an implementation of Perl written in Java.
 jPerl - Perl on the JVM
 Jamie's PERL scripts for bioinformatics
 jperl (Jonathan Perl)

Now, the last version of JPerl is 5.005_04 and is not maintained now.

Japanization modifier WATANABE Hirofumi said,

  "Because WATANABE am tired I give over maintaing JPerl."

at Slide #15: "The future of JPerl" of

in The Perl Confernce Japan 1998.

When I heard it, I thought that someone excluding me would maintain JPerl. And I slept every night hanging a sock. Night and day, I kept having hope. After 10 years, I noticed that white beard exists in the sock :-)

This software is a source code filter to escape Perl script encoded by GBK given from STDIN or command line parameter. The character code is never converted by escaping the script. Neither the value of the character nor the length of the character string change even if it escapes.

I learned the following things from the successful software.

I am excited about this software and Perl's future --- I hope you are too.

JRE: JPerl Runtime Environment ^

  |        JPerl Application Script       | Your Script
  |  Source Code Filter, Runtime Routine  | ex.,
  |          PVM 5.00503 or later         | ex. perl 5.00503

A Perl Virtual Machine (PVM) enables a set of computer software programs and data structures to use a virtual machine model for the execution of other computer programs and scripts. The model used by a PVM accepts a form of computer intermediate language commonly referred to as Perl byteorientedcode. This language conceptually represents the instruction set of a byte-oriented, capability architecture.

Basic Idea of Source Code Filter ^

I discovered this mail again recently.

[] jus Benkyoukai

save as:

  package SJIS;
  use Filter::Util::Call;
  sub multibyte_filter {
      my $status;
      if (($status = filter_read()) > 0 ) {
  sub import {

I am glad that I could confirm my idea is not so wrong.

Command-line Wildcard Expansion on DOS-like Systems ^

The default command shells on DOS-like systems (COMMAND.COM or cmd.exe or Win95Cmd.exe) do not expand wildcard arguments supplied to programs. Instead, import of works well.

   # @ARGV wildcard globbing
   sub import {

       if ($^O =~ /\A (?: MSWin32 | NetWare | symbian | dos ) \z/oxms) {
           my @argv = ();
           for (@ARGV) {

               # has space
               if (/\A (?:$q_char)*? [ ] /oxms) {
                   if (my @glob = Egbk::glob(qq{"$_"})) {
                       push @argv, @glob;
                   else {
                       push @argv, $_;

               # has wildcard metachar
               elsif (/\A (?:$q_char)*? [*?] /oxms) {
                   if (my @glob = Egbk::glob($_)) {
                       push @argv, @glob;
                   else {
                       push @argv, $_;

               # no wildcard globbing
               else {
                   push @argv, $_;
           @ARGV = @argv;

Software Composition ^               --- source code filter to escape GBK              --- run-time routines for

Upper Compatibility by Escaping ^

This software adds the function by 'Escaping' it always, and nothing of the past is broken. Therefore, 'Possible job' never becomes 'Impossible job'. This approach is effective in the field where the retreat is never permitted. It means incompatible upgrade of Perl should be rewound.

Escaping Your Script (You do) ^

You need write 'use GBK;' in your script.

  You do
  use GBK;

Escaping Multiple-Octet Code (GBK software provides) ^

Insert chr(0x5c) before @ [ \ ] ^ ` { | and } in multiple-octet of

  ex. Japanese Katakana "SO" like [ `/ ] code is "\x83\x5C" in SJIS
                  see     hex dump
  source script   "`/"    [83 5c]
  Here, use SJIS;
                          hex dump
  escaped script  "`\/"   [83 [5c] 5c]
                    ^--- escape by SJIS software
  by the by       see     hex dump
  your eye's      "`/\"   [83 5c] [5c]
  perl eye's      "`\/"   [83] \[5c]
                          hex dump
  in the perl     "`/"    [83] [5c]

Multiple-Octet Anchoring of Regular Expression (GBK software provides) ^

GBK software applies multiple-octet anchoring at beginning of regular expression.

  Before                  After
  m/regexp/               m/${Egbk::anchor}(?:regexp).../

Escaping Second Octet (GBK software provides) ^

GBK software escapes second octet of multiple-octet character in regular expression.

  Before                  After
  m<...`/...>             m<...`/\...>

Multiple-Octet Character Regular Expression (GBK software provides) ^

GBK software clusters multiple-octet character with quantifier, makes cluster from multiple-octet custom character classes. And makes multiple-octet version metasymbol from classic Perl character class shortcuts and POSIX-style character classes.

  Before                  After
  m/...MULTIOCT+.../      m/...(?:MULTIOCT)+.../
  m/...[AN-EM].../        m/...(?:A[N-Z]|[B-D][A-Z]|E[A-M]).../
  m/...\D.../             m/...(?:${Egbk::eD}).../
  m/...[[:^digit:]].../   m/...(?:${Egbk::not_digit}).../

Calling 'Egbk::ignorecase()' (GBK software provides) ^

GBK software applies calling 'Egbk::ignorecase()' instead of /i modifier.

  Before                  After
  m/...$var.../i          m/...@{[Egbk::ignorecase($var)]}.../

Character-Oriented Regular Expression ^

Regular expression works as character-oriented that has no /b modifier.

  Before                  After
  /regexp/                /ditto$Egbk::matched/
  m/regexp/               m/ditto$Egbk::matched/
  ?regexp?                m?ditto$Egbk::matched?
  m?regexp?               m?ditto$Egbk::matched?
  $_ =~                   ($_ =~ m/ditto$Egbk::matched/) ?
  s/regexp/replacement/   CORE::eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; 1 } :
  $_ !~                   ($_ !~ m/ditto$Egbk::matched/) ?
  s/regexp/replacement/   1 :
                          CORE::eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; undef }
  split(/regexp/)         Egbk::split(qr/regexp/)
  split(m/regexp/)        Egbk::split(qr/regexp/)
  split(qr/regexp/)       Egbk::split(qr/regexp/)
  qr/regexp/              qr/ditto$Egbk::matched/

Byte-Oriented Regular Expression ^

Regular expression works as byte-oriented that has /b modifier.

  Before                  After
  /regexp/b               /(?:regexp)$Egbk::matched/
  m/regexp/b              m/(?:regexp)$Egbk::matched/
  ?regexp?b               m?regexp$Egbk::matched?
  m?regexp?b              m?regexp$Egbk::matched?
  $_ =~                   ($_ =~ m/(\G[\x00-\xFF]*?)(?:regexp)$Egbk::matched/) ?
  s/regexp/replacement/b  CORE::eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; 1 } :
  $_ !~                   ($_ !~ m/(\G[\x00-\xFF]*?)(?:regexp)$Egbk::matched/) ?
  s/regexp/replacement/b  1 :
                          CORE::eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; undef }
  split(/regexp/b)        split(qr/regexp/)
  split(m/regexp/b)       split(qr/regexp/)
  split(qr/regexp/b)      split(qr/regexp/)
  qr/regexp/b             qr/(?:regexp)$Egbk::matched/

Escaping Character Classes ( provides) ^

The character classes are redefined as follows to backward compatibility.

  Before        After
   .            ${Egbk::dot}
                ${Egbk::dot_s}    (/s modifier)
  \d            [0-9]              (universally)
  \s            \s
  \w            [0-9A-Z_a-z]       (universally)
  \D            ${Egbk::eD}
  \S            ${Egbk::eS}
  \W            ${Egbk::eW}
  \h            [\x09\x20]
  \v            [\x0A\x0B\x0C\x0D]
  \H            ${Egbk::eH}
  \V            ${Egbk::eV}
  \C            [\x00-\xFF]
  \X            X                  (so, just 'X')
  \R            ${Egbk::eR}
  \N            ${Egbk::eN}

Also POSIX-style character classes.

  Before        After
  [:alnum:]     [\x30-\x39\x41-\x5A\x61-\x7A]
  [:alpha:]     [\x41-\x5A\x61-\x7A]
  [:ascii:]     [\x00-\x7F]
  [:blank:]     [\x09\x20]
  [:cntrl:]     [\x00-\x1F\x7F]
  [:digit:]     [\x30-\x39]
  [:graph:]     [\x21-\x7F]
  [:lower:]     [\x61-\x7A]
                [\x41-\x5A\x61-\x7A]     (/i modifier)
  [:print:]     [\x20-\x7F]
  [:punct:]     [\x21-\x2F\x3A-\x3F\x40\x5B-\x5F\x60\x7B-\x7E]
  [:space:]     [\s\x0B]
  [:upper:]     [\x41-\x5A]
                [\x41-\x5A\x61-\x7A]     (/i modifier)
  [:word:]      [\x30-\x39\x41-\x5A\x5F\x61-\x7A]
  [:xdigit:]    [\x30-\x39\x41-\x46\x61-\x66]
  [:^alnum:]    ${Egbk::not_alnum}
  [:^alpha:]    ${Egbk::not_alpha}
  [:^ascii:]    ${Egbk::not_ascii}
  [:^blank:]    ${Egbk::not_blank}
  [:^cntrl:]    ${Egbk::not_cntrl}
  [:^digit:]    ${Egbk::not_digit}
  [:^graph:]    ${Egbk::not_graph}
  [:^lower:]    ${Egbk::not_lower}
                ${Egbk::not_lower_i}    (/i modifier)
  [:^print:]    ${Egbk::not_print}
  [:^punct:]    ${Egbk::not_punct}
  [:^space:]    ${Egbk::not_space}
  [:^upper:]    ${Egbk::not_upper}
                ${Egbk::not_upper_i}    (/i modifier)
  [:^word:]     ${Egbk::not_word}
  [:^xdigit:]   ${Egbk::not_xdigit}

\b and \B are redefined as follows to backward compatibility.

  Before      After
  \b          ${Egbk::eb}
  \B          ${Egbk::eB}

Definitions in

  After                    Definition
  ${Egbk::anchor}         qr{\G(?>[^\x81-\xFE]|[\x81-\xFE][\x00-\xFF])*?}
                           for over 32766 octets string on ActivePerl5.6 and Perl5.10 or later
  ${Egbk::dot}            qr{(?>[^\x81-\xFE\x0A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::dot_s}          qr{(?>[^\x81-\xFE]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::eD}             qr{(?>[^\x81-\xFE0-9]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::eS}             qr{(?>[^\x81-\xFE\s]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::eW}             qr{(?>[^\x81-\xFE0-9A-Z_a-z]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::eH}             qr{(?>[^\x81-\xFE\x09\x20]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::eV}             qr{(?>[^\x81-\xFE\x0A\x0B\x0C\x0D]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::eR}             qr{(?>\x0D\x0A|[\x0A\x0D])};
  ${Egbk::eN}             qr{(?>[^\x81-\xFE\x0A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_alnum}      qr{(?>[^\x81-\xFE\x30-\x39\x41-\x5A\x61-\x7A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_alpha}      qr{(?>[^\x81-\xFE\x41-\x5A\x61-\x7A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_ascii}      qr{(?>[^\x81-\xFE\x00-\x7F]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_blank}      qr{(?>[^\x81-\xFE\x09\x20]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_cntrl}      qr{(?>[^\x81-\xFE\x00-\x1F\x7F]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_digit}      qr{(?>[^\x81-\xFE\x30-\x39]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_graph}      qr{(?>[^\x81-\xFE\x21-\x7F]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_lower}      qr{(?>[^\x81-\xFE\x61-\x7A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_lower_i}    qr{(?>[^\x81-\xFE\x41-\x5A\x61-\x7A]|[\x81-\xFE][\x00-\xFF])}; # Perl 5.16 compatible
# ${Egbk::not_lower_i}    qr{(?>[^\x81-\xFE]|[\x81-\xFE][\x00-\xFF])};                   # older Perl compatible
  ${Egbk::not_print}      qr{(?>[^\x81-\xFE\x20-\x7F]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_punct}      qr{(?>[^\x81-\xFE\x21-\x2F\x3A-\x3F\x40\x5B-\x5F\x60\x7B-\x7E]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_space}      qr{(?>[^\x81-\xFE\s\x0B]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_upper}      qr{(?>[^\x81-\xFE\x41-\x5A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_upper_i}    qr{(?>[^\x81-\xFE\x41-\x5A\x61-\x7A]|[\x81-\xFE][\x00-\xFF])}; # Perl 5.16 compatible
# ${Egbk::not_upper_i}    qr{(?>[^\x81-\xFE]|[\x81-\xFE][\x00-\xFF])};                   # older Perl compatible
  ${Egbk::not_word}       qr{(?>[^\x81-\xFE\x30-\x39\x41-\x5A\x5F\x61-\x7A]|[\x81-\xFE][\x00-\xFF])};
  ${Egbk::not_xdigit}     qr{(?>[^\x81-\xFE\x30-\x39\x41-\x46\x61-\x66]|[\x81-\xFE][\x00-\xFF])};
  # This solution is not perfect. I beg better solution from you who are reading this.
  ${Egbk::eb}             qr{(?:\A(?=[0-9A-Z_a-z])|(?<=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF])(?=[0-9A-Z_a-z])|(?<=[0-9A-Z_a-z])(?=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF]|\z))};
  ${Egbk::eB}             qr{(?:(?<=[0-9A-Z_a-z])(?=[0-9A-Z_a-z])|(?<=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF])(?=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF]))};

Un-Escaping \ of \b{}, \B{}, \N{}, \p{}, \P{}, and \X (GBK software provides) ^

GBK software removes '\' at head of alphanumeric regexp metasymbols \b{}, \B{}, \N{}, \p{}, \P{} and \X. By this method, you can avoid the trap of the abstraction.

See also, Deprecate literal unescaped "{" in regexes.

  Before           After
  \b{...}          b\{...}
  \B{...}          B\{...}
  \p{L}            p\{L}
  \p{^L}           p\{^L}
  \p{\^L}          p\{\^L}
  \pL              pL
  \P{L}            P\{L}
  \P{^L}           P\{^L}
  \P{\^L}          P\{\^L}
  \PL              PL
  \X               X

Escaping Built-in Functions (GBK software provides) ^

Insert 'Egbk::' at head of function name. provides your script Egbk::* subroutines.

  Before      After            Works as
  length      length           Byte
  substr      substr           Byte
  pos         pos              Byte
  split       Egbk::split     Character
  tr///       Egbk::tr        Character
  tr///b      tr///            Byte
  tr///B      tr///            Byte
  y///        Egbk::tr        Character
  y///b       tr///            Byte
  y///B       tr///            Byte
  chop        Egbk::chop      Character
  index       Egbk::index     Character
  rindex      Egbk::rindex    Character
  lc          Egbk::lc        Character
  lcfirst     Egbk::lcfirst   Character
  uc          Egbk::uc        Character
  ucfirst     Egbk::ucfirst   Character
  fc          Egbk::fc        Character
  chr         Egbk::chr       Character
  glob        Egbk::glob      Character
  lstat       Egbk::lstat     Character
  opendir     Egbk::opendir   Character
  stat        Egbk::stat      Character
  unlink      Egbk::unlink    Character
  chdir       Egbk::chdir     Character
  do          Egbk::do        Character
  require     Egbk::require   Character

  Before                   After
  use Perl::Module;        BEGIN { Egbk::require 'Perl/'; Perl::Module->import() if Perl::Module->can('import'); }
  use Perl::Module @list;  BEGIN { Egbk::require 'Perl/'; Perl::Module->import(@list) if Perl::Module->can('import'); }
  use Perl::Module ();     BEGIN { Egbk::require 'Perl/'; }
  no Perl::Module;         BEGIN { Egbk::require 'Perl/'; Perl::Module->unimport() if Perl::Module->can('unimport'); }
  no Perl::Module @list;   BEGIN { Egbk::require 'Perl/'; Perl::Module->unimport(@list) if Perl::Module->can('unimport'); }
  no Perl::Module ();      BEGIN { Egbk::require 'Perl/'; }

Escaping File Test Operators (GBK software provides) ^

Insert 'Egbk::' instead of '-' of operator.

  Available in MSWin32, MacOS, and UNIX-like systems
  Before   After      Meaning
  -r       Egbk::r   File or directory is readable by this (effective) user or group
  -w       Egbk::w   File or directory is writable by this (effective) user or group
  -e       Egbk::e   File or directory name exists
  -x       Egbk::x   File or directory is executable by this (effective) user or group
  -z       Egbk::z   File exists and has zero size (always false for directories)
  -f       Egbk::f   Entry is a plain file
  -d       Egbk::d   Entry is a directory
  -t       -t         The filehandle is a TTY (as reported by the isatty() system function;
                      filenames can't be tested by this test)
  -T       Egbk::T   File looks like a "text" file
  -B       Egbk::B   File looks like a "binary" file
  -M       Egbk::M   Modification age (measured in days)
  -A       Egbk::A   Access age (measured in days)
  -C       Egbk::C   Inode-modification age (measured in days)
  -s       Egbk::s   File or directory exists and has nonzero size
                      (the value is the size in bytes)
  Available in MacOS and UNIX-like systems
  Before   After      Meaning
  -R       Egbk::R   File or directory is readable by this real user or group
  -W       Egbk::W   File or directory is writable by this real user or group
  -X       Egbk::X   File or directory is executable by this real user or group
  -l       Egbk::l   Entry is a symbolic link
  -S       Egbk::S   Entry is a socket
  Not available in MSWin32 and MacOS
  Before   After      Meaning
  -o       Egbk::o   File or directory is owned by this (effective) user
  -O       Egbk::O   File or directory is owned by this real user
  -p       Egbk::p   Entry is a named pipe (a "fifo")
  -b       Egbk::b   Entry is a block-special file (like a mountable disk)
  -c       Egbk::c   Entry is a character-special file (like an I/O device)
  -u       Egbk::u   File or directory is setuid
  -g       Egbk::g   File or directory is setgid
  -k       Egbk::k   File or directory has the sticky bit set

-w only inspects the read-only file attribute (FILE_ATTRIBUTE_READONLY), which determines whether the directory can be deleted, not whether it can be written to. Directories always have read and write access unless denied by discretionary access control lists (DACLs). (MSWin32) -R, -W, -X, -O are indistinguishable from -r, -w, -x, -o. (MSWin32) -g, -k, -l, -u, -A are not particularly meaningful. (MSWin32) -x (or -X) determine if a file ends in one of the executable suffixes. -S is meaningless. (MSWin32)

As of Perl 5.00503, as a form of purely syntactic sugar, you can stack file test operators, in a way that -w -x $file is equivalent to -x $file && -w _ .

  if ( -w -r $file ) {
      print "The file is both readable and writable!\n";

Escaping Function Name (You do) ^

You need write 'GBK::' at head of function name when you want character- oriented subroutine. See 'Character-Oriented Subroutines'.

  Function   Character-Oriented   Description
  ord        GBK::ord
  reverse    GBK::reverse
  getc       GBK::getc
  length     GBK::length
  substr     GBK::substr
  index      GBK::index          See 'About Indexes'
  rindex     GBK::rindex         See 'About Rindexes'
  eval       GBK::eval

  About Indexes
  Function       Works as    Returns as   Description
  index          Character   Byte         JPerl semantics (most useful)
  (same as Egbk::index)
  GBK::index    Character   Character    Character-oriented semantics
  CORE::index    Byte        Byte         Byte-oriented semantics
  (nothing)      Byte        Character    (most useless)

  About Rindexes
  Function       Works as    Returns as   Description
  rindex         Character   Byte         JPerl semantics (most useful)
  (same as Egbk::rindex)
  GBK::rindex   Character   Character    Character-oriented semantics
  CORE::rindex   Byte        Byte         Byte-oriented semantics
  (nothing)      Byte        Character    (most useless)

Character-Oriented Subsroutines ^

Byte-Oriented Functions ^

Yada Yada Operator (GBK software provides) ^

  The yada yada operator (noted ...) is a placeholder for code. Perl parses it
  without error, but when you try to execute a yada yada, it throws an exception
  with the text Unimplemented:

  sub unimplemented { ... }
  eval { unimplemented() };
  if ( $@ eq 'Unimplemented' ) {
      print "I found the yada yada!\n";

  You can only use the yada yada to stand in for a complete statement. These
  examples of the yada yada work:

  { ... }
  sub foo { ... }
  eval { ... };
  sub foo {
      my( $self ) = shift;
  do { my $n; ...; print 'Hurrah!' };

  The yada yada cannot stand in for an expression that is part of a larger statement
  since the ... is also the three-dot version of the range operator
  (see "Range Operators"). These examples of the yada yada are still syntax errors:

  print ...;
  open my($fh), '>', '/dev/passwd' or ...;
  if ( $condition && ... ) { print "Hello\n" };

  There are some cases where Perl can't immediately tell the difference between an
  expression and a statement. For instance, the syntax for a block and an anonymous
  hash reference constructor look the same unless there's something in the braces that
  give Perl a hint. The yada yada is a syntax error if Perl doesn't guess that the
  { ... } is a block. In that case, it doesn't think the ... is the yada yada because
  it's expecting an expression instead of a statement:

  my @transformed = map { ... } @input;  # syntax error

  You can use a ; inside your block to denote that the { ... } is a block and not a
  hash reference constructor. Now the yada yada works:

  my @transformed = map {; ... } @input; # ; disambiguates
  my @transformed = map { ...; } @input; # ; disambiguates

Un-Escaping bytes::* Subroutines (GBK software provides) ^

GBK software removes 'bytes::' at head of subroutine name.

  Before           After     Works as
  bytes::chr       chr       Byte
  bytes::index     index     Byte
  bytes::length    length    Byte
  bytes::ord       ord       Byte
  bytes::rindex    rindex    Byte
  bytes::substr    substr    Byte

Ignore Pragmas and Modules ^

  Before                    After
  use strict;               use strict; no strict qw(refs);
  use 5.12.0;               use 5.12.0; no strict qw(refs);
  require utf8;             # require utf8;
  require bytes;            # require bytes;
  require charnames;        # require charnames;
  require I18N::Japanese;   # require I18N::Japanese;
  require I18N::Collate;    # require I18N::Collate;
  require I18N::JExt;       # require I18N::JExt;
  require File::DosGlob;    # require File::DosGlob;
  require Wild;             # require Wild;
  require Wildcard;         # require Wildcard;
  require Japanese;         # require Japanese;
  use utf8;                 # use utf8;
  use bytes;                # use bytes;
  use charnames;            # use charnames;
  use I18N::Japanese;       # use I18N::Japanese;
  use I18N::Collate;        # use I18N::Collate;
  use I18N::JExt;           # use I18N::JExt;
  use File::DosGlob;        # use File::DosGlob;
  use Wild;                 # use Wild;
  use Wildcard;             # use Wildcard;
  use Japanese;             # use Japanese;
  no utf8;                  # no utf8;
  no bytes;                 # no bytes;
  no charnames;             # no charnames;
  no I18N::Japanese;        # no I18N::Japanese;
  no I18N::Collate;         # no I18N::Collate;
  no I18N::JExt;            # no I18N::JExt;
  no File::DosGlob;         # no File::DosGlob;
  no Wild;                  # no Wild;
  no Wildcard;              # no Wildcard;
  no Japanese;              # no Japanese;

  Comment out pragma to ignore utf8 environment, and provides these

Environment Variable ^

 This software uses the flock function for exclusive control. The execution of the
 program is blocked until it becomes possible to read or write the file.
 You can have it not block in the flock function by defining environment variable
 (The value '1' doesn't have the meaning)


I have tested and verified this software using the best of my ability. However, a software containing much regular expression is bound to contain some bugs. Thus, if you happen to find a bug that's in GBK software and not your own program, you can try to reduce it to a minimal test case and then report it to the following author's address. If you have an idea that could make this a more useful tool, please let everyone share it.


INABA Hitoshi <>

This project was originated by INABA Hitoshi.


This software is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

My Goal ^

P.401 See chapter 15: Unicode of ISBN 0-596-00027-8 Programming Perl Third Edition.

Before the introduction of Unicode support in perl, The eq operator just compared the byte-strings represented by two scalars. Beginning with perl 5.8, eq compares two byte-strings with simultaneous consideration of the UTF8 flag.

/* You are not expected to understand this */

  Information processing model beginning with perl 5.8
    |     Text strings     |                     |
    +----------+-----------|    Binary strings   |
    |  UTF-8   |  Latin-1  |                     |
    | UTF8     |            Not UTF8             |
    | Flagged  |            Flagged              |

  Confusion of Perl string model is made from double meanings of
  "Binary string."
  Meanings of "Binary string"
  1. Non-Text string
  2. Digital octet string

  Let's draw again using those term.
    |     Text strings     |                     |
    +----------+-----------|   Non-Text strings  |
    |  UTF-8   |  Latin-1  |                     |
    | UTF8     |            Not UTF8             |
    | Flagged  |            Flagged              |
    |            Digital octet string            |

There are people who don't agree to change in the character string processing model of Perl 5.8. It is impossible to get to agree it to majority of Perl user who hardly ever use Perl. How to solve it by returning to a original method, let's drag out page 402 of the old dusty Programming Perl, 3rd ed. again.

  Information processing model beginning with perl3 or this software
  of UNIX/C-ism.

    |    Text string as Digital octet string     |
    |    Digital octet string as Text string     |
    |       Not UTF8 Flagged, No Mojibake        |

  In UNIX Everything is a File
  - In UNIX everything is a stream of bytes
  - In UNIX the filesystem is used as a universal name space

  Native Encoding Scripting
  - native encoding of file contents
  - native encoding of file name on filesystem
  - native encoding of command line
  - native encoding of environment variable
  - native encoding of API
  - native encoding of network packet
  - native encoding of database

Ideally, I'd like to achieve these five Goals:

Back when Programming Perl, 3rd ed. was written, UTF8 flag was not born and Perl is designed to make the easy jobs easy. This software provides programming environment like at that time.

Perl's motto ^

   Some computer scientists (the reductionists, in particular) would
  like to deny it, but people have funny-shaped minds. Mental geography
  is not linear, and cannot be mapped onto a flat surface without
  severe distortion. But for the last score years or so, computer
  reductionists have been first bowing down at the Temple of Orthogonality,
  then rising up to preach their ideas of ascetic rectitude to any who
  would listen.
   Their fervent but misguided desire was simply to squash your mind to
  fit their mindset, to smush your patterns of thought into some sort of
  Hyperdimensional Flatland. It's a joyless existence, being smushed.
  --- Learning Perl on Win32 Systems

  If you think this is a big headache, you're right. No one likes
  this situation, but Perl does the best it can with the input and
  encodings it has to deal with. If only we could reset history and
  not make so many mistakes next time.
  --- Learning Perl 6th Edition

   The most important thing for most people to know about handling
  Unicode data in Perl, however, is that if you don't ever use any Uni-
  code data -- if none of your files are marked as UTF-8 and you don't
  use UTF-8 locales -- then you can happily pretend that you're back in
  Perl 5.005_03 land; the Unicode features will in no way interfere with
  your code unless you're explicitly using them. Sometimes the twin
  goals of embracing Unicode but not disturbing old-style byte-oriented
  scripts has led to compromise and confusion, but it's the Perl way to
  silently do the right thing, which is what Perl ends up doing.
  --- Advanced Perl Programming, 2nd Edition


 Larry Wall, Randal L.Schwartz, Yoshiyuki Kondo
 December 1997
 ISBN 4-89052-384-7

 Programming Perl, Second Edition
 By Larry Wall, Tom Christiansen, Randal L. Schwartz
 October 1996
 Pages: 670
 ISBN 10: 1-56592-149-6 | ISBN 13: 9781565921498

 Programming Perl, Third Edition
 By Larry Wall, Tom Christiansen, Jon Orwant
 Third Edition  July 2000
 Pages: 1104
 ISBN 10: 0-596-00027-8 | ISBN 13: 9780596000271

 The Perl Language Reference Manual (for Perl version 5.12.1)
 by Larry Wall and others
 Paperback (6"x9"), 724 pages
 Retail Price: $39.95 (pound 29.95 in UK)
 ISBN-13: 978-1-906966-02-7

 Perl Pocket Reference, 5th Edition
 By Johan Vromans
 Publisher: O'Reilly Media
 Released: July 2011
 Pages: 102

 Programming Perl, 4th Edition
 By: Tom Christiansen, brian d foy, Larry Wall, Jon Orwant
 Publisher: O'Reilly Media
 Formats: Print, Ebook, Safari Books Online
 Released: March 2012
 Pages: 1130
 Print ISBN: 978-0-596-00492-7 | ISBN 10: 0-596-00492-3
 Ebook ISBN: 978-1-4493-9890-3 | ISBN 10: 1-4493-9890-1

 Perl Cookbook
 By Tom Christiansen, Nathan Torkington
 August 1998
 Pages: 800
 ISBN 10: 1-56592-243-3 | ISBN 13: 978-1-56592-243-3

 Perl Cookbook, Second Edition
 By Tom Christiansen, Nathan Torkington
 Second Edition  August 2003
 Pages: 964
 ISBN 10: 0-596-00313-7 | ISBN 13: 9780596003135

 Perl in a Nutshell, Second Edition
 By Stephen Spainhour, Ellen Siever, Nathan Patwardhan
 Second Edition  June 2002
 Pages: 760
 Series: In a Nutshell
 ISBN 10: 0-596-00241-6 | ISBN 13: 9780596002411

 Learning Perl on Win32 Systems
 By Randal L. Schwartz, Erik Olson, Tom Christiansen
 August 1997
 Pages: 306
 ISBN 10: 1-56592-324-3 | ISBN 13: 9781565923249

 Learning Perl, Fifth Edition
 By Randal L. Schwartz, Tom Phoenix, brian d foy
 June 2008
 Pages: 352
 Print ISBN:978-0-596-52010-6 | ISBN 10: 0-596-52010-7
 Ebook ISBN:978-0-596-10316-3 | ISBN 10: 0-596-10316-6

 Learning Perl, 6th Edition
 By Randal L. Schwartz, brian d foy, Tom Phoenix
 June 2011
 Pages: 390
 ISBN-10: 1449303587 | ISBN-13: 978-1449303587

 Advanced Perl Programming, 2nd Edition
 By Simon Cozens
 June 2005
 Pages: 300
 ISBN-10: 0-596-00456-7 | ISBN-13: 978-0-596-00456-9

 Futato, Irving, Jepson, Patwardhan, Siever
 ISBN 10: 1-56592-370-7

 By Daisuke Maki
 Pages: 344
 ISBN 10: 4798119172 | ISBN 13: 978-4798119175

 Understanding Japanese Information Processing
 By Ken Lunde
 January 1900
 Pages: 470
 ISBN 10: 1-56592-043-0 | ISBN 13: 9781565920439

 CJKV Information Processing
 Chinese, Japanese, Korean & Vietnamese Computing
 By Ken Lunde
 First Edition  January 1999
 Pages: 1128
 ISBN 10: 1-56592-224-7 | ISBN 13: 9781565922242

 Mastering Regular Expressions, Second Edition
 By Jeffrey E. F. Friedl
 Second Edition  July 2002
 Pages: 484
 ISBN 10: 0-596-00289-0 | ISBN 13: 9780596002893

 Mastering Regular Expressions, Third Edition
 By Jeffrey E. F. Friedl
 Third Edition  August 2006
 Pages: 542
 ISBN 10: 0-596-52812-4 | ISBN 13:9780596528126

 Regular Expressions Cookbook
 By Jan Goyvaerts, Steven Levithan
 May 2009
 Pages: 512
 ISBN 10:0-596-52068-9 | ISBN 13: 978-0-596-52068-7

 Regular Expressions Cookbook, 2nd Edition
 By Jan Goyvaerts, Steven Levithan
 Final Release Date: August 2012
 Pages: 612
 ISBN: 978-1-4493-1943-4 | ISBN 10:1-4493-1943-2

 By Kouji Shibano
 Pages: 1456
 ISBN 4-542-20129-5

 1993 Aug
 Pages: 172
 T1008901080816 ZASSHI 08901-8

 By YAMAGATA Hiroo, Stephen J. Turnbull, Craig Oda, Robert J. Bickel
 June, 2000
 Pages: 376
 ISBN 4-87311-016-5

 MacPerl Power and Ease
 By Vicki Brown, Chris Nandor
 April 1998
 Pages: 350
 ISBN 10: 1881957322 | ISBN 13: 978-1881957324

 Windows NT Shell Scripting
 By Timothy Hill
 April 27, 1998
 Pages: 400
 ISBN 10: 1578700477 | ISBN 13: 9781578700479

 Windows(R) Command-Line Administrators Pocket Consultant, 2nd Edition
 By William R. Stanek
 February 2009
 Pages: 594
 ISBN 10: 0-7356-2262-0 | ISBN 13: 978-0-7356-2262-3

 Kaoru Maeda, Perl's history Perl 1,2,3,4

 nurse, What is "string"

 NISHIO Hirokazu, What's meant "string as a sequence of characters"?

 nurse, History of Japanese EUC 22:00

 Mike Whitaker, Perl And Unicode

 Ricardo Signes, Perl 5.14 for Pragmatists

 Ricardo Signes, What's New in Perl? v5.10 - v5.16

 CPAN Directory INABA Hitoshi


 Recent Perl packages by "INABA Hitoshi"


This software was made referring to software and the document that the following hackers or persons had made. I am thankful to all persons.

 Rick Yamashita, Shift_JIS
 ttp://!1pmWgsL289nm7Shn7cS0jHzA!2225.entry (dead link)
 (add 'h' at head)

 Larry Wall, Perl

 Kazumasa Utashiro,

 Jeffrey E. F. Friedl, Mastering Regular Expressions

 SADAHIRO Tomoyuki, The right way of using Shift_JIS

 Yukihiro "Matz" Matsumoto, YAPC::Asia2006 Ruby on Perl(s)

 jscripter, For jperl users

 Bruce., Unicode in Perl

 Hiroaki Izumi, Perl5.8/Perl5.10 is not useful on the Windows.

 TSUKAMOTO Makio, Perl memo/file path of Windows

 chaichanPaPa, Matching Shift_JIS file name

 SUZUKI Norio, Jperl

 WATANABE Hirofumi, Jperl

 Chuck Houpt, Michiko Nozu, MacJPerl

 Kenichi Ishigaki, Pod-PerldocJp, Welcome to modern Perl world

 Fuji, Goro (gfx), Perl Hackers Hub No.16

 Dan Kogai, Encode module (video) (audio)

 Takahashi Masatuyo, JPerl Wiki

 Juerd, Perl Unicode Advice

 daily dayflower, 2008-06-25 perluniadvice

 Jesse Vincent, Compatibility is a virtue

 Tokyo-pm archive

 Error: Runtime exception on jperl 5.005_03


 Object-oriented with Perl
syntax highlighting: