The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

NAME

GBK - Source code filter to escape GBK script

Install and Usage

There are two steps there:

  • You'll have to download GBK.pm and Egbk.pm and put it in your perl lib directory.

  • You'll need to write "use GBK;" at head of the script.

SYNOPSIS

  use GBK;
  use GBK ver.sion;             --- require minimum version
  use GBK ver.sion.0;           --- expects version (match or die)
  use GBK qw(ord reverse getc); --- demand enhanced feature of ord, reverse, and getc
  use GBK ver.sion qw(ord reverse getc);
  use GBK ver.sion.0 qw(ord reverse getc);

  # "no GBK;" not supported

  or

  $ perl GBK.pm GBK_script.pl > Escaped_script.pl.e

  then

  $ perl Escaped_script.pl.e

  GBK_script.pl  --- script written in GBK
  Escaped_script.pl.e --- escaped script

  functions:
    GBK::ord(...);
    GBK::reverse(...);
    GBK::getc(...);
    GBK::length(...);
    GBK::substr(...);
    GBK::index(...);
    GBK::rindex(...);
    <*>
    glob(...);
    CORE::chop(...);
    CORE::ord(...);
    CORE::reverse(...);
    CORE::getc(...);
    CORE::index(...);
    CORE::rindex(...);

  emulate Perl5.6 on perl5.00503
    use warnings;
    use warnings::register;

  emulate Perl5.16
    use feature qw(fc);

  dummy functions:
    utf8::upgrade(...);
    utf8::downgrade(...);
    utf8::encode(...);
    utf8::decode(...);
    utf8::is_utf8(...);
    utf8::valid(...);
    bytes::chr(...);
    bytes::index(...);
    bytes::length(...);
    bytes::ord(...);
    bytes::rindex(...);
    bytes::substr(...);

ABSTRACT

GBK software is "middleware" between perl interpreter and your Perl script written in GBK.

Perl is optimized for problems which are about 90% working with text and about 10% everything else. Even if this "text" doesn't contain GBK, Perl3 or later can treat GBK as binary data.

By "use GBK;", it automatically interpret your script as GBK. The various functions of perl including a regular expression can treat GBK now. The function length treats length per byte. This software does not use UTF8 flag.

Yet Another Future Of

JPerl is very useful software. -- Oops, note, this "JPerl" means "Japanized Perl" or "Japanese Perl". Therefore, it is unrelated to JPerl of the following.

 JPerl is an implementation of Perl written in Java.
 http://www.javainc.com/projects/jperl/
 
 jPerl - Perl on the JVM
 http://www.dzone.com/links/175948.html
 
 Jamie's PERL scripts for bioinformatics
 http://code.google.com/p/jperl/
 
 jperl (Jonathan Perl)
 https://github.com/jperl

Now, the last version of JPerl is 5.005_04 and is not maintained now.

Japanization modifier WATANABE Hirofumi said,

  "Because WATANABE am tired I give over maintaing JPerl."

at Slide #15: "The future of JPerl" of

ftp://ftp.oreilly.co.jp/pcjp98/watanabe/jperlconf.ppt

in The Perl Confernce Japan 1998.

When I heard it, I thought that someone excluding me would maintain JPerl. And I slept every night hanging a sock. Night and day, I kept having hope. After 10 years, I noticed that white beard exists in the sock :-)

This software is a source code filter to escape Perl script encoded by GBK given from STDIN or command line parameter. The character code is never converted by escaping the script. Neither the value of the character nor the length of the character string change even if it escapes.

I learned the following things from the successful software.

  • Upper Compatibility like Perl4 to Perl5

  • Maximum Portability like jcode.pl

  • Remains One Language Handling Raw GBK, Doesn't Use UTF8 flag like JPerl

  • Remains One Interpreter like Encode module

  • Code Set Independent like Ruby

  • There's more than one way to do it like Perl itself

I am excited about this software and Perl's future --- I hope you are too.

JRE: JPerl Runtime Environment

  +---------------------------------------+
  |        JPerl Application Script       | Your Script
  +---------------------------------------+
  |  Source Code Filter, Runtime Routine  | ex. GBK.pm, Egbk.pm
  +---------------------------------------+
  |          PVM 5.00503 or later         | ex. perl 5.00503
  +---------------------------------------+

A Perl Virtual Machine (PVM) enables a set of computer software programs and data structures to use a virtual machine model for the execution of other computer programs and scripts. The model used by a PVM accepts a form of computer intermediate language commonly referred to as Perl byteorientedcode. This language conceptually represents the instruction set of a byte-oriented, capability architecture.

Basic Idea of Source Code Filter

I discovered this mail again recently.

[Tokyo.pm] jus Benkyoukai

http://mail.pm.org/pipermail/tokyo-pm/1999-September/001854.html

save as: SJIS.pm

  package SJIS;
  use Filter::Util::Call;
  sub multibyte_filter {
      my $status;
      if (($status = filter_read()) > 0 ) {
          s/([\x81-\x9f\xe0-\xef])([\x40-\x7e\x80-\xfc])/
              sprintf("\\x%02x\\x%02x",ord($1),ord($2))
          /eg;
      }
      $status;
  }
  sub import {
      filter_add(\&multibyte_filter);
  }
  1;

I am glad that I could confirm my idea is not so wrong.

Command-line Wildcard Expansion on DOS-like Systems

The default command shells on DOS-like systems (COMMAND.COM or cmd.exe) do not expand wildcard arguments supplied to programs. Instead, import of Egbk.pm works well.

   in Egbk.pm
   #
   # @ARGV wildcard globbing
   #
   sub import {

       if ($^O =~ /\A (?: MSWin32 | NetWare | symbian | dos ) \z/oxms) {
           my @argv = ();
           for (@ARGV) {

               # has space
               if (/\A (?:$q_char)*? [ ] /oxms) {
                   if (my @glob = Egbk::glob(qq{"$_"})) {
                       push @argv, @glob;
                   }
                   else {
                       push @argv, $_;
                   }
               }

               # has wildcard metachar
               elsif (/\A (?:$q_char)*? [*?] /oxms) {
                   if (my @glob = Egbk::glob($_)) {
                       push @argv, @glob;
                   }
                   else {
                       push @argv, $_;
                   }
               }

               # no wildcard globbing
               else {
                   push @argv, $_;
               }
           }
           @ARGV = @argv;
       }
   }

Software Composition

   GBK.pm               --- source code filter to escape GBK
   Egbk.pm              --- run-time routines for GBK.pm
   perl5.bat             --- find and run perl5    without %PATH% settings
   perl55.bat            --- find and run perl5.5  without %PATH% settings
   perl56.bat            --- find and run perl5.6  without %PATH% settings
   perl58.bat            --- find and run perl5.8  without %PATH% settings
   perl510.bat           --- find and run perl5.10 without %PATH% settings
   perl512.bat           --- find and run perl5.12 without %PATH% settings
   perl514.bat           --- find and run perl5.14 without %PATH% settings
   perl516.bat           --- find and run perl5.16 without %PATH% settings
   perl64.bat            --- find and run perl64   without %PATH% settings
   perl64512.bat         --- find and run perl5.12 (x64) without %PATH% settings
   perl64514.bat         --- find and run perl5.14 (x64) without %PATH% settings
   perl64516.bat         --- find and run perl5.16 (x64) without %PATH% settings
   aperl58.bat           --- find and run ActivePerl 5.8  without %PATH% settings
   aperl510.bat          --- find and run ActivePerl 5.10 without %PATH% settings
   aperl512.bat          --- find and run ActivePerl 5.12 without %PATH% settings
   aperl514.bat          --- find and run ActivePerl 5.14 without %PATH% settings
   aperl516.bat          --- find and run ActivePerl 5.16 without %PATH% settings
   aperl64512.bat        --- find and run ActivePerl 5.12 (x64) without %PATH% settings
   aperl64514.bat        --- find and run ActivePerl 5.14 (x64) without %PATH% settings
   aperl64516.bat        --- find and run ActivePerl 5.16 (x64) without %PATH% settings
   sperl58.bat           --- find and run Strawberry Perl 5.8  without %PATH% settings
   sperl510.bat          --- find and run Strawberry Perl 5.10 without %PATH% settings
   sperl512.bat          --- find and run Strawberry Perl 5.12 without %PATH% settings
   sperl514.bat          --- find and run Strawberry Perl 5.14 without %PATH% settings
   sperl516.bat          --- find and run Strawberry Perl 5.16 without %PATH% settings
   sperl64512.bat        --- find and run Strawberry Perl 5.12 (x64) without %PATH% settings
   sperl64514.bat        --- find and run Strawberry Perl 5.14 (x64) without %PATH% settings
   sperl64516.bat        --- find and run Strawberry Perl 5.16 (x64) without %PATH% settings
   strict.pm_            --- dummy strict.pm
   warnings.pm_          --- poor warnings.pm
   warnings/register.pm_ --- poor warnings/register.pm
   feature.pm_           --- dummy feature.pm

Upper Compatibility by Escaping

This software adds the function by 'Escaping' it always, and nothing of the past is broken. Therefore, 'Possible job' never becomes 'Impossible job'. This approach is effective in the field where the retreat is never permitted. It means incompatible upgrade of Perl should be rewound.

Escaping Your Script (You do)

You need write 'use GBK;' in your script.

  ---------------------------------
  Before      You do
  ---------------------------------
  (nothing)   use GBK;
  ---------------------------------

Escaping Multiple-Octet Code (GBK.pm provides)

Insert chr(0x5c) before @ [ \ ] ^ ` { | and } in multiple-octet of

  • string in single quote ('', q{}, <<'END' and qw{})

  • string in double quote ("", qq{}, <<END, <<"END", ``, qx{} and <<`END`)

  • regexp in single quote (m'', s''', split(''), split(m'') and qr'')

  • regexp in double quote (//, m//, ??, s///, split(//), split(m//) and qr//)

  • character in tr/// (tr/// and y///)

  ex. Japanese Katakana "SO" like [ `/ ] code is "\x83\x5C" in SJIS
 
                  see     hex dump
  -----------------------------------------
  source script   "`/"    [83 5c]
  -----------------------------------------
 
  Here, use SJIS;
                          hex dump
  -----------------------------------------
  escaped script  "`\/"   [83 [5c] 5c]
  -----------------------------------------
                    ^--- escape by SJIS software
 
  by the by       see     hex dump
  -----------------------------------------
  your eye's      "`/\"   [83 5c] [5c]
  -----------------------------------------
  perl eye's      "`\/"   [83] \[5c]
  -----------------------------------------
 
                          hex dump
  -----------------------------------------
  in the perl     "`/"    [83] [5c]
  -----------------------------------------

Multiple-Octet Anchoring of Regular Expression (GBK.pm provides)

GBK.pm applies multiple-octet anchoring at beginning of regular expression.

  --------------------------------------------------------------------------------
  Before                  After
  --------------------------------------------------------------------------------
  m/regexp/               m/${Egbk::anchor}(?:regexp).../
  --------------------------------------------------------------------------------

Escaping Second Octet (GBK.pm provides)

GBK.pm escapes second octet of multiple-octet character in regular expression.

  --------------------------------------------------------------------------------
  Before                  After
  --------------------------------------------------------------------------------
  m<...`/...>             m<...`/\...>
  --------------------------------------------------------------------------------

Multiple-Octet Character Regular Expression (GBK.pm provides)

GBK.pm clusters multiple-octet character with quantifier, makes cluster from multiple-octet custom character classes. And makes multiple-octet version metasymbol from classic Perl character class shortcuts and POSIX-style character classes.

  --------------------------------------------------------------------------------
  Before                  After
  --------------------------------------------------------------------------------
  m/...MULTIOCT+.../      m/...(?:MULTIOCT)+.../
  m/...[AN-EM].../        m/...(?:A[N-Z]|[B-D][A-Z]|E[A-M]).../
  m/...\D.../             m/...(?:${Egbk::eD}).../
  m/...[[:^digit:]].../   m/...(?:${Egbk::not_digit}).../
  --------------------------------------------------------------------------------

Calling 'Egbk::ignorecase()' (GBK.pm provides)

GBK.pm applies calling 'Egbk::ignorecase()' instead of /i modifier.

  --------------------------------------------------------------------------------
  Before                  After
  --------------------------------------------------------------------------------
  m/...$var.../i          m/...@{[Egbk::ignorecase($var)]}.../
  --------------------------------------------------------------------------------

Character-Oriented Regular Expression

Regular expression works as character-oriented that has no /b modifier.

  --------------------------------------------------------------------------------
  Before                  After
  --------------------------------------------------------------------------------
  /regexp/                /ditto$Egbk::matched/
  m/regexp/               m/ditto$Egbk::matched/
  ?regexp?                m?ditto$Egbk::matched?
  m?regexp?               m?ditto$Egbk::matched?
 
  $_ =~                   ($_ =~ m/ditto$Egbk::matched/) ?
  s/regexp/replacement/   eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; 1 } :
                          undef
 
  $_ !~                   ($_ !~ m/ditto$Egbk::matched/) ?
  s/regexp/replacement/   1 :
                          eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; undef }
 
  split(/regexp/)         Egbk::split(qr/regexp/)
  split(m/regexp/)        Egbk::split(qr/regexp/)
  split(qr/regexp/)       Egbk::split(qr/regexp/)
  qr/regexp/              qr/ditto$Egbk::matched/
  --------------------------------------------------------------------------------

Byte-Oriented Regular Expression

Regular expression works as byte-oriented that has /b modifier.

  --------------------------------------------------------------------------------
  Before                  After
  --------------------------------------------------------------------------------
  /regexp/b               /(?:regexp)$Egbk::matched/
  m/regexp/b              m/(?:regexp)$Egbk::matched/
  ?regexp?b               m?regexp$Egbk::matched?
  m?regexp?b              m?regexp$Egbk::matched?
 
  $_ =~                   ($_ =~ m/(\G[\x00-\xFF]*?)(?:regexp)$Egbk::matched/) ?
  s/regexp/replacement/b  eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; 1 } :
                          undef
 
  $_ !~                   ($_ !~ m/(\G[\x00-\xFF]*?)(?:regexp)$Egbk::matched/) ?
  s/regexp/replacement/b  1 :
                          eval{ Egbk::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; undef }
 
  split(/regexp/b)        split(qr/regexp/)
  split(m/regexp/b)       split(qr/regexp/)
  split(qr/regexp/b)      split(qr/regexp/)
  qr/regexp/b             qr/(?:regexp)$Egbk::matched/
  --------------------------------------------------------------------------------

Escaping Character Classes (Egbk.pm provides)

The character classes are redefined as follows to backward compatibility.

  ---------------------------------------------------------------
  Before        After
  ---------------------------------------------------------------
   .            ${Egbk::dot}
                ${Egbk::dot_s}    (/s modifier)
  \d            [0-9]
  \s            [\x09\x0A\x0C\x0D\x20]
  \w            [0-9A-Z_a-z]
  \D            ${Egbk::eD}
  \S            ${Egbk::eS}
  \W            ${Egbk::eW}
  \h            [\x09\x20]
  \v            [\x0A\x0B\x0C\x0D]
  \H            ${Egbk::eH}
  \V            ${Egbk::eV}
  \C            [\x00-\xFF]
  \X            X (so, just 'X')
  \R            ${Egbk::eR}
  \N            ${Egbk::eN}
  ---------------------------------------------------------------

Also POSIX-style character classes.

  ---------------------------------------------------------------
  Before        After
  ---------------------------------------------------------------
  [:alnum:]     [\x30-\x39\x41-\x5A\x61-\x7A]
  [:alpha:]     [\x41-\x5A\x61-\x7A]
  [:ascii:]     [\x00-\x7F]
  [:blank:]     [\x09\x20]
  [:cntrl:]     [\x00-\x1F\x7F]
  [:digit:]     [\x30-\x39]
  [:graph:]     [\x21-\x7F]
  [:lower:]     [\x61-\x7A]
                [\x41-\x5A\x61-\x7A]     (/i modifier)
  [:print:]     [\x20-\x7F]
  [:punct:]     [\x21-\x2F\x3A-\x3F\x40\x5B-\x5F\x60\x7B-\x7E]
  [:space:]     [\x09\x0A\x0B\x0C\x0D\x20]
  [:upper:]     [\x41-\x5A]
                [\x41-\x5A\x61-\x7A]     (/i modifier)
  [:word:]      [\x30-\x39\x41-\x5A\x5F\x61-\x7A]
  [:xdigit:]    [\x30-\x39\x41-\x46\x61-\x66]
  [:^alnum:]    ${Egbk::not_alnum}
  [:^alpha:]    ${Egbk::not_alpha}
  [:^ascii:]    ${Egbk::not_ascii}
  [:^blank:]    ${Egbk::not_blank}
  [:^cntrl:]    ${Egbk::not_cntrl}
  [:^digit:]    ${Egbk::not_digit}
  [:^graph:]    ${Egbk::not_graph}
  [:^lower:]    ${Egbk::not_lower}
                ${Egbk::not_lower_i}    (/i modifier)
  [:^print:]    ${Egbk::not_print}
  [:^punct:]    ${Egbk::not_punct}
  [:^space:]    ${Egbk::not_space}
  [:^upper:]    ${Egbk::not_upper}
                ${Egbk::not_upper_i}    (/i modifier)
  [:^word:]     ${Egbk::not_word}
  [:^xdigit:]   ${Egbk::not_xdigit}
  ---------------------------------------------------------------

\b and \B are redefined as follows to backward compatibility.

  ---------------------------------------------------------------
  Before      After
  ---------------------------------------------------------------
  \b          ${Egbk::eb}
  \B          ${Egbk::eB}
  ---------------------------------------------------------------

Definitions in Egbk.pm.

  ---------------------------------------------------------------------------------------------------------------------------------------------------------
  After                    Definition
  ---------------------------------------------------------------------------------------------------------------------------------------------------------
  ${Egbk::anchor}         qr{\G(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE])*?}
                           for over 32766 octets string on ActivePerl5.6 and Perl5.10 or later
                           qr{\G(?(?=.{0,32766}\z)(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE])*?|(?(?=[$sbcs]+\z).*?|(?:.*?[$sbcs](?:$tbcs_1st[^$sbcs]{2})*?)))}oxms;
  ${Egbk::dot}            qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x0A])}
  ${Egbk::dot_s}          qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE])}
  ${Egbk::eD}             qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE0-9])}
  ${Egbk::eS}             qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x09\x0A\x0C\x0D\x20])}
  ${Egbk::eW}             qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE0-9A-Z_a-z])}
  ${Egbk::eH}             qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x09\x20])}
  ${Egbk::eV}             qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x0A\x0B\x0C\x0D])}
  ${Egbk::eR}             qr{(?:\x0D\x0A|[\x0A\x0D])}
  ${Egbk::eN}             qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x0A])}
  ${Egbk::not_alnum}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x30-\x39\x41-\x5A\x61-\x7A])}
  ${Egbk::not_alpha}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x41-\x5A\x61-\x7A])}
  ${Egbk::not_ascii}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x00-\x7F])}
  ${Egbk::not_blank}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x09\x20])}
  ${Egbk::not_cntrl}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x00-\x1F\x7F])}
  ${Egbk::not_digit}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x30-\x39])}
  ${Egbk::not_graph}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x21-\x7F])}
  ${Egbk::not_lower}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x61-\x7A])}
  ${Egbk::not_lower_i}    qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE])}
  ${Egbk::not_print}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x20-\x7F])}
  ${Egbk::not_punct}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x21-\x2F\x3A-\x3F\x40\x5B-\x5F\x60\x7B-\x7E])}
  ${Egbk::not_space}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x09\x0A\x0B\x0C\x0D\x20])}
  ${Egbk::not_upper}      qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x41-\x5A])}
  ${Egbk::not_upper_i}    qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE])}
  ${Egbk::not_word}       qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x30-\x39\x41-\x5A\x5F\x61-\x7A])}
  ${Egbk::not_xdigit}     qr{(?:[\x81-\xFE][\x00-\xFF]|[^\x81-\xFE\x30-\x39\x41-\x46\x61-\x66])}
  ${Egbk::eb}             qr{(?:\A(?=[0-9A-Z_a-z])|(?<=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF])(?=[0-9A-Z_a-z])|(?<=[0-9A-Z_a-z])(?=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF]|\z))}
  ${Egbk::eB}             qr{(?:(?<=[0-9A-Z_a-z])(?=[0-9A-Z_a-z])|(?<=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF])(?=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF]))}
  ---------------------------------------------------------------------------------------------------------------------------------------------------------

Un-Escaping \ of \N, \p, \P and \X (GBK.pm provides)

GBK.pm removes '\' at head of alphanumeric regexp metasymbols \N, \p, \P and \X. By this method, you can avoid the trap of the abstraction.

See also, Deprecate literal unescaped "{" in regexes. http://perl5.git.perl.org/perl.git/commit/2a53d3314d380af5ab5283758219417c6dfa36e9

  ------------------------------------
  Before           After
  ------------------------------------
  \N{CHARNAME}     N\{CHARNAME}
  \p{L}            p\{L}
  \p{^L}           p\{^L}
  \p{\^L}          p\{\^L}
  \pL              pL
  \P{L}            P\{L}
  \P{^L}           P\{^L}
  \P{\^L}          P\{\^L}
  \PL              PL
  \X               X
  ------------------------------------

Escaping Built-in Functions (GBK.pm and Egbk.pm provide)

Insert 'Egbk::' at head of function name. Egbk.pm provides your script Egbk::* functions.

  -------------------------------------------
  Before      After            Works as
  -------------------------------------------
  length      length           Byte
  substr      substr           Byte
  pos         pos              Byte
  split       Egbk::split     Character
  tr///       Egbk::tr        Character
  tr///b      tr///            Byte
  tr///B      tr///            Byte
  y///        Egbk::tr        Character
  y///b       tr///            Byte
  y///B       tr///            Byte
  chop        Egbk::chop      Character
  index       Egbk::index     Character
  rindex      Egbk::rindex    Character
  lc          Egbk::lc        Character
  lcfirst     Egbk::lcfirst   Character
  uc          Egbk::uc        Character
  ucfirst     Egbk::ucfirst   Character
  fc          Egbk::fc        Character
  chr         Egbk::chr       Character
  glob        Egbk::glob      Character
  lstat       Egbk::lstat     Character
  opendir     Egbk::opendir   Character
  stat        Egbk::stat      Character
  unlink      Egbk::unlink    Character
  chdir       Egbk::chdir     Character
  do          Egbk::do        Character
  require     Egbk::require   Character
  -------------------------------------------

  ------------------------------------------------------------------------------------------------------------------------
  Before                   After
  ------------------------------------------------------------------------------------------------------------------------
  use Perl::Module;        BEGIN { Egbk::require 'Perl/Module.pm'; Perl::Module->import() if Perl::Module->can('import'); }
  use Perl::Module @list;  BEGIN { Egbk::require 'Perl/Module.pm'; Perl::Module->import(@list) if Perl::Module->can('import'); }
  use Perl::Module ();     BEGIN { Egbk::require 'Perl/Module.pm'; }
  no Perl::Module;         BEGIN { Egbk::require 'Perl/Module.pm'; Perl::Module->unimport() if Perl::Module->can('unimport'); }
  no Perl::Module @list;   BEGIN { Egbk::require 'Perl/Module.pm'; Perl::Module->unimport(@list) if Perl::Module->can('unimport'); }
  no Perl::Module ();      BEGIN { Egbk::require 'Perl/Module.pm'; }
  ------------------------------------------------------------------------------------------------------------------------

Escaping File Test Operators (GBK.pm and Egbk.pm provide)

Insert 'Egbk::' instead of '-' of operator.

  Available in MSWin32, MacOS, and UNIX-like systems
  --------------------------------------------------------------------------
  Before   After      Meaning
  --------------------------------------------------------------------------
  -r       Egbk::r   File or directory is readable by this (effective) user or group
  -w       Egbk::w   File or directory is writable by this (effective) user or group
  -e       Egbk::e   File or directory name exists
  -x       Egbk::x   File or directory is executable by this (effective) user or group
  -z       Egbk::z   File exists and has zero size (always false for directories)
  -f       Egbk::f   Entry is a plain file
  -d       Egbk::d   Entry is a directory
  -t       -t         The filehandle is a TTY (as reported by the isatty() system function;
                      filenames can't be tested by this test)
  -T       Egbk::T   File looks like a "text" file
  -B       Egbk::B   File looks like a "binary" file
  -M       Egbk::M   Modification age (measured in days)
  -A       Egbk::A   Access age (measured in days)
  -C       Egbk::C   Inode-modification age (measured in days)
  -s       Egbk::s   File or directory exists and has nonzero size
                      (the value is the size in bytes)
  --------------------------------------------------------------------------
  
  Available in MacOS and UNIX-like systems
  --------------------------------------------------------------------------
  Before   After      Meaning
  --------------------------------------------------------------------------
  -R       Egbk::R   File or directory is readable by this real user or group
  -W       Egbk::W   File or directory is writable by this real user or group
  -X       Egbk::X   File or directory is executable by this real user or group
  -l       Egbk::l   Entry is a symbolic link
  -S       Egbk::S   Entry is a socket
  --------------------------------------------------------------------------
  
  Not available in MSWin32 and MacOS
  --------------------------------------------------------------------------
  Before   After      Meaning
  --------------------------------------------------------------------------
  -o       Egbk::o   File or directory is owned by this (effective) user
  -O       Egbk::O   File or directory is owned by this real user
  -p       Egbk::p   Entry is a named pipe (a "fifo")
  -b       Egbk::b   Entry is a block-special file (like a mountable disk)
  -c       Egbk::c   Entry is a character-special file (like an I/O device)
  -u       Egbk::u   File or directory is setuid
  -g       Egbk::g   File or directory is setgid
  -k       Egbk::k   File or directory has the sticky bit set
  --------------------------------------------------------------------------

-w only inspects the read-only file attribute (FILE_ATTRIBUTE_READONLY), which determines whether the directory can be deleted, not whether it can be written to. Directories always have read and write access unless denied by discretionary access control lists (DACLs). (MSWin32) -R, -W, -X, -O are indistinguishable from -r, -w, -x, -o. (MSWin32) -g, -k, -l, -u, -A are not particularly meaningful. (MSWin32) -x (or -X) determine if a file ends in one of the executable suffixes. -S is meaningless. (MSWin32)

As of Perl 5.00503, as a form of purely syntactic sugar, you can stack file test operators, in a way that -w -x $file is equivalent to -x $file && -w _ .

  if ( -w -r $file ) {
      print "The file is both readable and writable!\n";
  }

Escaping Function Name (You do)

You need write 'GBK::' at head of function name when you want character- oriented function. See 'Character-Oriented Functions'.

  --------------------------------------------------------
  Function   Character-Oriented   Description
  --------------------------------------------------------
  ord        GBK::ord
  reverse    GBK::reverse
  getc       GBK::getc
  length     GBK::length
  substr     GBK::substr
  index      GBK::index          See 'About Indexes'
  rindex     GBK::rindex         See 'About Rindexes'
  --------------------------------------------------------

  About Indexes
  -------------------------------------------------------------------------
  Function       Works as    Returns as   Description
  -------------------------------------------------------------------------
  index          Character   Byte         JPerl semantics (most useful)
  (same as Egbk::index)
  GBK::index    Character   Character    Character-oriented semantics
  CORE::index    Byte        Byte         Byte-oriented semantics
  (nothing)      Byte        Character    (most useless)
  -------------------------------------------------------------------------

  About Rindexes
  -------------------------------------------------------------------------
  Function       Works as    Returns as   Description
  -------------------------------------------------------------------------
  rindex         Character   Byte         JPerl semantics (most useful)
  (same as Egbk::rindex)
  GBK::rindex   Character   Character    Character-oriented semantics
  CORE::rindex   Byte        Byte         Byte-oriented semantics
  (nothing)      Byte        Character    (most useless)
  -------------------------------------------------------------------------

Character-Oriented Functions

  • Ordinal Value of Character

      $ord = GBK::ord($string);
    
      This function returns the numeric value (ASCII or GBK character) of the
      first character of $string, not Unicode. If $string is omitted, it uses $_.
      The return value is always unsigned.
    
      If you import ord "use GBK qw(ord);", ord of your script will be rewritten in
      GBK::ord. GBK::ord is not compatible with ord of JPerl.
  • Reverse List or String

      @reverse = GBK::reverse(@list);
      $reverse = GBK::reverse(@list);
    
      In list context, this function returns a list value consisting of the elements of
      @list in the opposite order.
    
      In scalar context, the function concatenates all the elements of @list and then
      returns the reverse of that resulting string, character by character.
    
      If you import reverse "use GBK qw(reverse);", reverse of your script will be
      rewritten in GBK::reverse. GBK::reverse is not compatible with reverse of
      JPerl.
    
      Even if you do not know this function, there is no problem. This function can
      be created with
    
      $rev = join('', reverse(split(//, $jstring)));
    
      as before.
    
      See:
      P.558 JPerl (Japanese Perl)
      Appendix C Supplement the Japanese version
      ISBN 4-89052-384-7 PERL PUROGURAMINGU
  • Returns Next Character

      $getc = GBK::getc(FILEHANDLE);
      $getc = GBK::getc($filehandle);
      $getc = GBK::getc;
    
      This function returns the next character from the input file attached to FILEHANDLE.
      It returns undef at end-of-file, or if an I/O error was encountered. If FILEHANDLE
      is omitted, the function reads from STDIN.
    
      This function is somewhat slow, but it's occasionally useful for single-character
      input from the keyboard -- provided you manage to get your keyboard input
      unbuffered. This function requests unbuffered input from the standard I/O library.
      Unfortunately, the standard I/O library is not so standard as to provide a portable
      way to tell the underlying operating system to supply unbuffered keyboard input to
      the standard I/O system. To do that, you have to be slightly more clever, and in
      an operating-system-dependent fashion. Under Unix you might say this:
    
      if ($BSD_STYLE) {
          system "stty cbreak </dev/tty >/dev/tty 2>&1";
      }
      else {
          system "stty", "-icanon", "eol", "\001";
      }
    
      $key = GBK::getc;
    
      if ($BSD_STYLE) {
          system "stty -cbreak </dev/tty >/dev/tty 2>&1";
      }
      else {
          system "stty", "icanon", "eol", "^@"; # ASCII NUL
      }
      print "\n";
    
      This code puts the next character typed on the terminal in the string $key. If your
      stty program has options like cbreak, you'll need to use the code where $BSD_STYLE
      is true. Otherwise, you'll need to use the code where it is false.
    
      If you import getc "use GBK qw(getc);", getc of your script will be rewritten in
      GBK::getc. GBK::getc is not compatible with getc of JPerl.
  • Length by GBK Character

      $length = GBK::length($string);
      $length = GBK::length();
    
      This function returns the length in characters (programmer-visible characters) of
      the scalar value $string. If $string is omitted, it returns the GBK::length of
      $_.
    
      Do not try to use GBK::length to find the size of an array or hash. Use scalar
      @array for the size of an array, and scalar keys %hash for the number of key/value
      pairs in a hash. (The scalar is typically omitted when redundant.)
    
      To find the length of a string in bytes rather than characters, say simply:
    
      $bytes = length($string);
    
      Even if you do not know this function, there is no problem. This function can
      be created with
    
      $len = split(//, $jstring);
    
      as before.
    
      See:
      P.558 JPerl (Japanese Perl)
      Appendix C Supplement the Japanese version
      ISBN 4-89052-384-7 PERL PUROGURAMINGU
  • Substr by GBK Character

      $substr = GBK::substr($string,$offset,$length,$replacement);
      $substr = GBK::substr($string,$offset,$length);
      $substr = GBK::substr($string,$offset);
    
      This function extracts a substring out of the string given by $string and returns
      it. The substring is extracted starting at $offset characters from the front of
      the string.
      If $offset is negative, the substring starts that far from the end of the string
      instead. If $length is omitted, everything to the end of the string is returned.
      If $length is negative, the length is calculated to leave that many characters off
      the end of the string. Otherwise, $length indicates the length of the substring to
      extract, which is sort of what you'd expect.
    
      For bytes, use the substr from built-in Perl functions.
    
      An alternative to using GBK::substr as an lvalue is to specify the $replacement
      string as the fourth argument. This lets you replace parts of the $string and return
      what was there before in one operation, just as you can with splice. The next
      example also replaces the last character of $var with "Curly" and puts that replaced
      character into $oldstr: 
    
      $oldstr = GBK::substr($var, -1, 1, "Curly");
    
      To prepend the string "Larry" to the current value of $var, use:
    
      GBK::substr($var, 0, 0, "Larry");
    
      To instead replace the first character of $var with "Moe", use:
    
      GBK::substr($var, 0, 1, "Moe");
    
      And, finally, to replace the last character of $var with "Curly", use:
    
      GBK::substr($var, -1, 1, "Curly");
  • Index by GBK Character

      $index = GBK::index($string,$substring,$offset);
      $index = GBK::index($string,$substring);
    
      This function searches for one string within another. It returns the character
      position of the first occurrence of $substring in $string. The $offset, if
      specified, says how many characters from the start to skip before beginning to
      look. Positions are based at 0. If the substring is not found, the function
      returns one less than the base, ordinarily -1. To work your way through a string,
      you might say:
    
      $pos = -1;
      while (($pos = GBK::index($string, $lookfor, $pos)) > -1) {
          print "Found at $pos\n";
          $pos++;
      }
  • Rindex by GBK Character

      $rindex = GBK::rindex($string,$substring,$offset);
      $rindex = GBK::rindex($string,$substring);
    
      This function works just like GBK::index except that it returns the character
      position of the last occurrence of $substring in $string (a reverse GBK::index).
      The function returns -1 if $substring is not found. $offset, if specified, is
      the rightmost character position that may be returned. To work your way through
      a string backward, say:
    
      $pos = GBK::length($string);
      while (($pos = GBK::rindex($string, $lookfor, $pos)) >= 0) {
          print "Found at $pos\n";
          $pos--;
      }
  • Filename Globbing

      @glob = glob($expr);
      $glob = glob($expr);
      @glob = glob;
      $glob = glob;
      @glob = <*>;
      $glob = <*>;
    
      Performs filename expansion (globbing) on $expr, returning the next successive
      name on each call. If $expr is omitted, $_ is globbed instead.
    
      This operator is implemented via the Egbk::glob() function. See Egbk::glob of
      Egbk.pm for details.

Byte-Oriented Functions

  • Chop Byte String

      $byte = CORE::chop($string);
      $byte = CORE::chop(@list);
      $byte = CORE::chop;
    
      This function chops off the last byte of a string variable and returns the
      byte chopped. The CORE::chop operator is used primarily to remove the newline
      from the end of an input record, and is more efficient than using a
      substitution (s/\n$//). If that's all you're doing, then it would be safer to
      use chomp, since CORE::chop always shortens the string no matter what's there,
      and chomp is more selective.
    
      You cannot CORE::chop a literal, only a variable.
    
      If you CORE::chop a @list of variables, each string in the list is chopped:
    
      @lines = `cat myfile`;
      CORE::chop @lines;
    
      You can CORE::chop anything that is an lvalue, including an assignment:
    
      CORE::chop($cwd = `pwd`);
      CORE::chop($answer = <STDIN>);
    
      This is different from:
    
      $answer = CORE::chop($temp = <STDIN>); # WRONG
    
      which puts a newline into $answer because CORE::chop returns the byte chopped,
      not the remaining string (which is in $tmp). One way to get the result
      intended here is with substr:
    
      $answer = substr <STDIN>, 0, -1;
    
      But this is more commonly written as:
    
      CORE::chop($answer = <STDIN>);
    
      In the most general case, CORE::chop can be expressed in terms of substr:
    
      $last_byte = CORE::chop($var);
      $last_byte = substr($var, -1, 1, ""); # same thing
    
      Once you understand this equivalence, you can use it to do bigger chops. To
      CORE::chop more than one byte, use substr as an lvalue, assigning a null
      string. The following removes the last five bytes of $caravan:
    
      substr($caravan, -5) = "";
    
      The negative subscript causes substr to count from the end of the string
      instead of the beginning. If you wanted to save the bytes so removed, you
      could use the four-argument form of substr, creating something of a quintuple
      CORE::chop:
    
      $tail = substr($caravan, -5, 5, "");
    
      If no argument is given, the function chops the $_ variable.
  • Ordinal Value of Byte

      $ord = CORE::ord($expr);
    
      This function returns the numeric value of the first byte of $expr, regardless
      of "use GBK qw(ord);" exists or not. If $expr is omitted, it uses $_.
      The return value is always unsigned.
    
      If you want a signed value, use unpack('c',$expr). If you want all the bytes of
      the string converted to a list of numbers, use unpack('C*',$expr) instead.
  • Reverse List or Byte String

      @reverse = CORE::reverse(@list);
      $reverse = CORE::reverse(@list);
    
      In list context, this function returns a list value consisting of the elements
      of @list in the opposite order.
    
      In scalar context, the function concatenates all the elements of @list and then
      returns the reverse of that resulting string, byte by byte, regardless of
      "use GBK qw(reverse);" exists or not.
  • Returns Next Byte

      $getc = CORE::getc(FILEHANDLE);
      $getc = CORE::getc($filehandle);
      $getc = CORE::getc;
    
      This function returns the next byte from the input file attached to FILEHANDLE.
      It returns undef at end-of-file, or if an I/O error was encountered. If
      FILEHANDLE is omitted, the function reads from STDIN.
    
      This function is somewhat slow, but it's occasionally useful for single-byte
      input from the keyboard -- provided you manage to get your keyboard input
      unbuffered. This function requests unbuffered input from the standard I/O library.
      Unfortunately, the standard I/O library is not so standard as to provide a portable
      way to tell the underlying operating system to supply unbuffered keyboard input to
      the standard I/O system. To do that, you have to be slightly more clever, and in
      an operating-system-dependent fashion. Under Unix you might say this:
    
      if ($BSD_STYLE) {
          system "stty cbreak </dev/tty >/dev/tty 2>&1";
      }
      else {
          system "stty", "-icanon", "eol", "\001";
      }
    
      $key = CORE::getc;
    
      if ($BSD_STYLE) {
          system "stty -cbreak </dev/tty >/dev/tty 2>&1";
      }
      else {
          system "stty", "icanon", "eol", "^@"; # ASCII NUL
      }
      print "\n";
    
      This code puts the next single-byte typed on the terminal in the string $key.
      If your stty program has options like cbreak, you'll need to use the code where
      $BSD_STYLE is true. Otherwise, you'll need to use the code where it is false.
  • Index by Byte String

      $index = CORE::index($string,$substring,$offset);
      $index = CORE::index($string,$substring);
    
      This function searches for one byte string within another. It returns the position
      of the first occurrence of $substring in $string. The $offset, if specified, says
      how many bytes from the start to skip before beginning to look. Positions are based
      at 0. If the substring is not found, the function returns one less than the base,
      ordinarily -1. To work your way through a string, you might say:
    
      $pos = -1;
      while (($pos = CORE::index($string, $lookfor, $pos)) > -1) {
          print "Found at $pos\n";
          $pos++;
      }
  • Rindex by Byte String

      $rindex = CORE::rindex($string,$substring,$offset);
      $rindex = CORE::rindex($string,$substring);
    
      This function works just like CORE::index except that it returns the position of
      the last occurrence of $substring in $string (a reverse CORE::index). The function
      returns -1 if not $substring is found. $offset, if specified, is the rightmost
      position that may be returned. To work your way through a string backward, say:
    
      $pos = CORE::length($string);
      while (($pos = CORE::rindex($string, $lookfor, $pos)) >= 0) {
          print "Found at $pos\n";
          $pos--;
      }

Un-Escaping bytes::* Functions (GBK.pm provides)

GBK.pm removes 'bytes::' at head of function name.

  ---------------------------------------
  Before           After     Works as
  ---------------------------------------
  bytes::chr       chr       Byte
  bytes::index     index     Byte
  bytes::length    length    Byte
  bytes::ord       ord       Byte
  bytes::rindex    rindex    Byte
  bytes::substr    substr    Byte
  ---------------------------------------

Escaping Built-in Standard Module (Egbk.pm provides)

Egbk.pm does "BEGIN { unshift @INC, '/Perl/site/lib/GBK' }" at head. Store the standard module modified for GBK software in this directory to override built-in standard modules.

Escaping Standard Module Content (You do)

You need copy built-in standard module to /Perl/site/lib/GBK and change 'use utf8;' to 'use GBK;' in its. You need help yourself for now.

Back to and see 'Escaping Your Script'. Enjoy hacking!!

Ignore Pragmas and Modules

  -----------------------------------------------------------
  Before                    After
  -----------------------------------------------------------
  use strict;               use strict; no strict qw(refs);
  use 5.12.0;               use 5.12.0; no strict qw(refs);
  require utf8;             # require utf8;
  require bytes;            # require bytes;
  require charnames;        # require charnames;
  require I18N::Japanese;   # require I18N::Japanese;
  require I18N::Collate;    # require I18N::Collate;
  require I18N::JExt;       # require I18N::JExt;
  require File::DosGlob;    # require File::DosGlob;
  require Wild;             # require Wild;
  require Wildcard;         # require Wildcard;
  require Japanese;         # require Japanese;
  use utf8;                 # use utf8;
  use bytes;                # use bytes;
  use charnames;            # use charnames;
  use I18N::Japanese;       # use I18N::Japanese;
  use I18N::Collate;        # use I18N::Collate;
  use I18N::JExt;           # use I18N::JExt;
  use File::DosGlob;        # use File::DosGlob;
  use Wild;                 # use Wild;
  use Wildcard;             # use Wildcard;
  use Japanese;             # use Japanese;
  no utf8;                  # no utf8;
  no bytes;                 # no bytes;
  no charnames;             # no charnames;
  no I18N::Japanese;        # no I18N::Japanese;
  no I18N::Collate;         # no I18N::Collate;
  no I18N::JExt;            # no I18N::JExt;
  no File::DosGlob;         # no File::DosGlob;
  no Wild;                  # no Wild;
  no Wildcard;              # no Wildcard;
  no Japanese;              # no Japanese;
  -----------------------------------------------------------

  Comment out pragma to ignore utf8 environment, and Egbk.pm provides these
  functions.
  • Dummy utf8::upgrade

      $num_octets = utf8::upgrade($string);
    
      Returns the number of octets necessary to represent the string.
  • Dummy utf8::downgrade

      $success = utf8::downgrade($string[, FAIL_OK]);
    
      Returns true always.
  • Dummy utf8::encode

      utf8::encode($string);
    
      Returns nothing.
  • Dummy utf8::decode

      $success = utf8::decode($string);
    
      Returns true always.
  • Dummy utf8::is_utf8

      $flag = utf8::is_utf8(STRING);
    
      Returns false always.
  • Dummy utf8::valid

      $flag = utf8::valid(STRING);
    
      Returns true always.
  • Dummy bytes::chr

      This function is same as chr.
  • Dummy bytes::index

      This function is same as index.
  • Dummy bytes::length

      This function is same as length.
  • Dummy bytes::ord

      This function is same as ord.
  • Dummy bytes::rindex

      This function is same as rindex.
  • Dummy bytes::substr

      This function is same as substr.

Environment Variable

 This software uses the flock function for exclusive control. The execution of the
 program is blocked until it becomes possible to read or write the file.
 You can have it not block in the flock function by defining environment variable
 SJIS_NONBLOCK.
 
 Example:
 
   SET SJIS_NONBLOCK=1
 
 (The value '1' doesn't have the meaning)

Perl5.6 Emulation on perl5.005

  Using warnings pragma on perl5.00503 by rename files.

  warnings.pm_ --> warnings.pm
  warnings/register.pm_ --> warnings/register.pm

Perl5.16 Emulation

  Using feature pragma by rename files.

  feature.pm_ --> feature.pm

BUGS, LIMITATIONS, and COMPATIBILITY

I have tested and verified this software using the best of my ability. However, a software containing much regular expression is bound to contain some bugs. Thus, if you happen to find a bug that's in GBK software and not your own program, you can try to reduce it to a minimal test case and then report it to the following author's address. If you have an idea that could make this a more useful tool, please let everyone share it.

  • format

    Function "format" can't handle multiple-octet code same as original Perl.

  • cloister of regular expression

    The cloister (?s) and (?i) of a regular expression will not be implemented for the time being. Cloister (?s) can be substituted with the .(dot) and \N on /s modifier. Cloister (?i) can be substituted with \F...\E.

  • chdir

    Function chdir() can always be executed with perl5.005.

    There are the following limitations for DOS-like system(any of MSWin32, NetWare, symbian, dos).

    On perl5.006 or perl5.00800, if path is ended by chr(0x5C), it needs jacode.pl library.

    On perl5.008001 or later, perl5.010, perl5.012, perl5.014, perl5.016, if path is ended by chr(0x5C), chdir succeeds when a short path name (8dot3name) can be acquired according to COMMAND.COM or cmd.exe. However, leaf-subdirectory of the current directory is a short path name (8dot3name).

      see also,
      Bug #81839
      chdir does not work with chr(0x5C) at end of path
      http://bugs.activestate.com/show_bug.cgi?id=81839
  • GBK::substr as Lvalue

    GBK::substr differs from CORE::substr, and cannot be used as a lvalue. To change part of a string, you can use the optional fourth argument which is the replacement string.

    GBK::substr($string, 13, 4, "JPerl");

  • Special Variables $` and $& need /( Capture All )/

      Because $` and $& use $1.
    
      -------------------------------------------------------------------------------------------
      Before          After                Works as
      -------------------------------------------------------------------------------------------
      $`              Egbk::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      $PREMATCH       Egbk::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      ${^PREMATCH}    Egbk::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      $&              Egbk::MATCH()       $1
      $MATCH          Egbk::MATCH()       $1
      ${^MATCH}       Egbk::MATCH()       $1
      $'              Egbk::POSTMATCH()   $'
      $POSTMATCH      Egbk::POSTMATCH()   $'
      ${^POSTMATCH}   Egbk::POSTMATCH()   $'
      -------------------------------------------------------------------------------------------
  • Limitation of Regular Expression

    This software has limitation from \G in multibyte anchoring. Only the following Perl can treat the character string which exceeds 32766 octets with a regular expression.

    perl 5.6 or later --- ActivePerl on MSWin32

    perl 5.10 or later --- other Perl

      see also,
      
      [perl #116379] \G can't treat over 32767 octet
      http://www.nntp.perl.org/group/perl.perl5.porters/2013/01/msg197320.html
      
      perlre - Perl regular expressions
      http://perldoc.perl.org/perlre.html
      
      perlre length limit
      http://stackoverflow.com/questions/4592467/perlre-length-limit
      
      Japanese Document
      GBK/JA.pm
  • Empty Variable in Regular Expression

    Unlike literal null string, an interpolated variable evaluated to the empty string can't use the most recent pattern from a previous successful regular expression.

  • Limitation of ?? and m??

    Multibyte character needs ( ) which is before {n,m} {n,} {n} * and + in ?? or m??. As a result, you need to rewrite a script about $1,$2,$3,... You cannot use (?: ) ? {n,m}? {n,}? and {n}? in ?? and m??, because delimiter of m?? is '?'.

  • Look-behind Assertion

    The look-behind assertion like (?<=[A-Z]) is not prevented from matching trail octet of the previous multiple-octet code.

  • Modifier /a /d /l and /u of Regular Expression

    The concept of this software is not to use two or more encoding methods at the same time. Therefore, modifier /a /d /l and /u are not supported. \d means [0-9] always.

  • ${^WIN32_SLOPPY_STAT} is ignored

    Even if ${^WIN32_SLOPPY_STAT} is set to a true value, file test functions Egbk::*(), Egbk::lstat(), and Egbk::stat() on Microsoft Windows open the file for the path which has chr(0x5c) at end.

AUTHOR

INABA Hitoshi <ina@cpan.org>

This project was originated by INABA Hitoshi.

LICENSE AND COPYRIGHT

This software is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

My Goal

P.401 See chapter 15: Unicode of ISBN 0-596-00027-8 Programming Perl Third Edition.

Before the introduction of Unicode support in perl, The eq operator just compared the byte-strings represented by two scalars. Beginning with perl 5.8, eq compares two byte-strings with simultaneous consideration of the UTF8 flag.

  Information processing model beginning with perl 5.8
 
    +----------------------+---------------------+
    |     Text strings     |                     |
    +----------+-----------|    Binary strings   |
    |   UTF8   |  Latin-1  |                     |
    +----------+-----------+---------------------+
    | UTF8     |            Not UTF8             |
    | Flagged  |            Flagged              |
    +--------------------------------------------+
    http://perl-users.jp/articles/advent-calendar/2010/casual/4
 
    You should memorize this figure.
 
    (Why is only Latin-1 special?)

This change consequentially made a big gap between a past script and new script. Both scripts cannot re-use the code mutually any longer. Because a new method puts a strain in the programmer, it will still take time to replace all the in existence scripts.

The biggest problem of new method is that the UTF8 flag can't synchronize to real encode of string. Thus you must debug about UTF8 flag, before your script. How to solve it by returning to a this method, let's drag out page 402 of the old dusty Programming Perl, 3rd ed. again.

  Information processing model beginning with perl3 or this software.

    +--------------------------------------------+
    |       Text strings as Binary strings       |
    |       Binary strings as Text strings       |
    +--------------------------------------------+
    |              Not UTF8 Flagged              |
    +--------------------------------------------+

Ideally, I'd like to achieve these five Goals:

  • Goal #1:

    Old byte-oriented programs should not spontaneously break on the old byte-oriented data they used to work on.

    This goal has been achieved by that this software is additional code for perl like utf8 pragma. Perl should work same as past Perl if added nothing.

  • Goal #2:

    Old byte-oriented programs should magically start working on the new character-oriented data when appropriate.

    Still now, 1 octet is counted with 1 by embedded functions length, substr, index, rindex and pos that handle length and position of string. In this part, there is no change. The length of 1 character of 2 octet code is 2.

    On the other hand, the regular expression in the script is added the multibyte anchoring processing with this software, instead of you.

    figure of Goal #1 and Goal #2.

                                   GOAL#1  GOAL#2
                            (a)     (b)     (c)     (d)     (e)
          +--------------+-------+-------+-------+-------+-------+
          | data         |  Old  |  Old  |  New  |  Old  |  New  |
          +--------------+-------+-------+-------+-------+-------+
          | script       |  Old  |      Old      |      New      |
          +--------------+-------+---------------+---------------+
          | interpreter  |  Old  |              New              |
          +--------------+-------+-------------------------------+
          Old --- Old byte-oriented
          New --- New character-oriented

    There is a combination from (a) to (e) in data, script and interpreter of old and new. Let's add the Encode module and this software did not exist at time of be written this document and JPerl did exist.

                            (a)     (b)     (c)     (d)     (e)
                                          JPerl           Encode,GBK
          +--------------+-------+-------+-------+-------+-------+
          | data         |  Old  |  Old  |  New  |  Old  |  New  |
          +--------------+-------+-------+-------+-------+-------+
          | script       |  Old  |      Old      |      New      |
          +--------------+-------+---------------+---------------+
          | interpreter  |  Old  |              New              |
          +--------------+-------+-------------------------------+
          Old --- Old byte-oriented
          New --- New character-oriented

    The reason why JPerl is very excellent is that it is at the position of (c). That is, it is not necessary to do a special description to the script to process new character-oriented string.

    JPerl is the only software attained to this goal.

  • Goal #3:

    Programs should run just as fast in the new character-oriented mode as in the old byte-oriented mode.

    It is impossible. Because the following time is necessary.

    (1) Time of escape script for old byte-oriented perl.

    (2) Time of processing regular expression by escaped script while multibyte anchoring.

  • Goal #4:

    Perl should remain one language, rather than forking into a byte-oriented Perl and a character-oriented Perl.

    JPerl remains one Perl language by forking to two interpreters. However, the Perl core team did not desire fork of the interpreter. As a result, Perl language forked contrary to goal #4.

    A character-oriented perl is not necessary to make it specially, because a byte-oriented perl can already treat the binary data. This software is only an application program of byte-oriented Perl, a filter program.

    And you will get support from the Perl community, when you solve the problem by the Perl script.

    GBK software remains one language and one interpreter.

  • Goal #5:

    JPerl users will be able to maintain JPerl by Perl.

    May the JPerl be with you, always.

Back when Programming Perl, 3rd ed. was written, UTF8 flag was not born and Perl is designed to make the easy jobs easy. This software provide programming environment like at that time.

Words of Learning Perl

   Some computer scientists (the reductionists, in particular) would
  like to deny it, but people have funny-shaped minds. Mental geography
  is not linear, and cannot be mapped onto a flat surface without
  severe distortion. But for the last score years or so, computer
  reductionists have been first bowing down at the Temple of Orthogonality,
  then rising up to preach their ideas of ascetic rectitude to any who
  would listen.
 
   Their fervent but misguided desire was simply to squash your mind to
  fit their mindset, to smush your patterns of thought into some sort of
  Hyperdimensional Flatland. It's a joyless existence, being smushed.
 
  --- Learning Perl on Win32 Systems
 
  If you think this is a big headache, you're right. No one likes
  this situation, but Perl does the best it can with the input and
  encodings it has to deal with. If only we could reset history and
  not make so many mistakes next time.
 
  --- Learning Perl 6th Edition

SEE ALSO

 PERL PUROGURAMINGU
 Larry Wall, Randal L.Schwartz, Yoshiyuki Kondo
 December 1997
 ISBN 4-89052-384-7
 http://www.context.co.jp/~cond/books/old-books.html

 Programming Perl, Second Edition
 By Larry Wall, Tom Christiansen, Randal L. Schwartz
 October 1996
 Pages: 670
 ISBN 10: 1-56592-149-6 | ISBN 13: 9781565921498
 http://shop.oreilly.com/product/9781565921498.do

 Programming Perl, Third Edition
 By Larry Wall, Tom Christiansen, Jon Orwant
 Third Edition  July 2000
 Pages: 1104
 ISBN 10: 0-596-00027-8 | ISBN 13: 9780596000271
 http://shop.oreilly.com/product/9780596000271.do

 The Perl Language Reference Manual (for Perl version 5.12.1)
 by Larry Wall and others
 Paperback (6"x9"), 724 pages
 Retail Price: $39.95 (pound 29.95 in UK)
 ISBN-13: 978-1-906966-02-7
 http://www.network-theory.co.uk/perl/language/

 Perl Pocket Reference, 5th Edition
 By Johan Vromans
 Publisher: O'Reilly Media
 Released: July 2011
 Pages: 102
 http://shop.oreilly.com/product/0636920018476.do

 Programming Perl, 4th Edition
 By: Tom Christiansen, brian d foy, Larry Wall, Jon Orwant
 Publisher: O'Reilly Media
 Formats: Print, Ebook, Safari Books Online
 Released: March 2012
 Pages: 1130
 Print ISBN: 978-0-596-00492-7 | ISBN 10: 0-596-00492-3
 Ebook ISBN: 978-1-4493-9890-3 | ISBN 10: 1-4493-9890-1
 http://shop.oreilly.com/product/9780596004927.do

 Perl Cookbook, Second Edition
 By Tom Christiansen, Nathan Torkington
 Second Edition  August 2003
 Pages: 964
 ISBN 10: 0-596-00313-7 | ISBN 13: 9780596003135
 http://shop.oreilly.com/product/9780596003135.do

 Perl in a Nutshell, Second Edition
 By Stephen Spainhour, Ellen Siever, Nathan Patwardhan
 Second Edition  June 2002
 Pages: 760
 Series: In a Nutshell
 ISBN 10: 0-596-00241-6 | ISBN 13: 9780596002411
 http://shop.oreilly.com/product/9780596002411.do

 Learning Perl on Win32 Systems
 By Randal L. Schwartz, Erik Olson, Tom Christiansen
 August 1997
 Pages: 306
 ISBN 10: 1-56592-324-3 | ISBN 13: 9781565923249
 http://shop.oreilly.com/product/9781565923249.do

 Learning Perl, Fifth Edition
 By Randal L. Schwartz, Tom Phoenix, brian d foy
 June 2008
 Pages: 352
 Print ISBN:978-0-596-52010-6 | ISBN 10: 0-596-52010-7
 Ebook ISBN:978-0-596-10316-3 | ISBN 10: 0-596-10316-6
 http://shop.oreilly.com/product/9780596520113.do

 Learning Perl, 6th Edition
 By Randal L. Schwartz, brian d foy, Tom Phoenix
 June 2011
 Pages: 390
 ISBN-10: 1449303587 | ISBN-13: 978-1449303587
 http://shop.oreilly.com/product/0636920018452.do

 Perl RESOURCE KIT UNIX EDITION
 Futato, Irving, Jepson, Patwardhan, Siever
 ISBN 10: 1-56592-370-7
 http://shop.oreilly.com/product/9781565923706.do

 MODAN Perl NYUMON
 By Daisuke Maki
 2009/2/10
 Pages: 344
 ISBN 10: 4798119172 | ISBN 13: 978-4798119175
 http://www.seshop.com/product/detail/10250/

 Understanding Japanese Information Processing
 By Ken Lunde
 January 1900
 Pages: 470
 ISBN 10: 1-56592-043-0 | ISBN 13: 9781565920439
 http://shop.oreilly.com/product/9781565920439.do

 CJKV Information Processing
 Chinese, Japanese, Korean & Vietnamese Computing
 By Ken Lunde
 First Edition  January 1999
 Pages: 1128
 ISBN 10: 1-56592-224-7 | ISBN 13: 9781565922242
 http://shop.oreilly.com/product/9781565922242.do

 Mastering Regular Expressions, Second Edition
 By Jeffrey E. F. Friedl
 Second Edition  July 2002
 Pages: 484
 ISBN 10: 0-596-00289-0 | ISBN 13: 9780596002893
 http://shop.oreilly.com/product/9780596002893.do

 Mastering Regular Expressions, Third Edition
 By Jeffrey E. F. Friedl
 Third Edition  August 2006
 Pages: 542
 ISBN 10: 0-596-52812-4 | ISBN 13:9780596528126
 http://shop.oreilly.com/product/9780596528126.do

 Regular Expressions Cookbook
 By Jan Goyvaerts, Steven Levithan
 May 2009
 Pages: 512
 ISBN 10:0-596-52068-9 | ISBN 13: 978-0-596-52068-7
 http://shop.oreilly.com/product/9780596520694.do

 JIS KANJI JITEN
 By Kouji Shibano
 Pages: 1456
 ISBN 4-542-20129-5
 http://www.webstore.jsa.or.jp/lib/lib.asp?fn=/manual/mnl01_12.htm

 UNIX MAGAZINE
 1993 Aug
 Pages: 172
 T1008901080816 ZASSHI 08901-8
 http://ascii.asciimw.jp/books/books/detail/978-4-7561-5008-0.shtml

 LINUX NIHONGO KANKYO
 By YAMAGATA Hiroo, Stephen J. Turnbull, Craig Oda, Robert J. Bickel
 June, 2000
 Pages: 376
 ISBN 4-87311-016-5
 http://www.oreilly.co.jp/books/4873110165/

 MacPerl Power and Ease
 By Vicki Brown, Chris Nandor
 April 1998
 Pages: 350
 ISBN 10: 1881957322 | ISBN 13: 978-1881957324
 http://www.amazon.com/Macperl-Power-Ease-Vicki-Brown/dp/1881957322

 Windows NT Shell Scripting
 By Timothy Hill
 April 27, 1998
 Pages: 400
 ISBN 10: 1578700477 | ISBN 13: 9781578700479
 http://search.barnesandnoble.com/Windows-NT-Shell-Scripting/Timothy-Hill/e/9781578700479/

 Windows(R) Command-Line Administrators Pocket Consultant, 2nd Edition
 By William R. Stanek
 February 2009
 Pages: 594
 ISBN 10: 0-7356-2262-0 | ISBN 13: 978-0-7356-2262-3
 http://shop.oreilly.com/product/9780735622623.do

 Other Tools
 http://search.cpan.org/dist/jacode/
 http://search.cpan.org/dist/Char/

 BackPAN
 http://backpan.perl.org/authors/id/I/IN/INA/

ACKNOWLEDGEMENTS

This software was made referring to software and the document that the following hackers or persons had made. I am thankful to all persons.

 Rick Yamashita, Shift_JIS
 ttp://furukawablog.spaces.live.com/Blog/cns!1pmWgsL289nm7Shn7cS0jHzA!2225.entry (dead link)
 ttp://shino.tumblr.com/post/116166805/1981-us-jis
 (add 'h' at head)
 http://www.wdic.org/w/WDIC/%E3%82%B7%E3%83%95%E3%83%88JIS

 Larry Wall, Perl
 http://www.perl.org/

 Kazumasa Utashiro, jcode.pl
 ftp://ftp.iij.ad.jp/pub/IIJ/dist/utashiro/perl/
 http://log.utashiro.com/pub/2006/07/jkondo_a580.html

 Jeffrey E. F. Friedl, Mastering Regular Expressions
 http://regex.info/

 SADAHIRO Tomoyuki, The right way of using Shift_JIS
 http://homepage1.nifty.com/nomenclator/perl/shiftjis.htm
 http://search.cpan.org/~sadahiro/

 Yukihiro "Matz" Matsumoto, YAPC::Asia2006 Ruby on Perl(s)
 http://www.rubyist.net/~matz/slides/yapc2006/

 jscripter, For jperl users
 http://homepage1.nifty.com/kazuf/jperl.html

 Bruce., Unicode in Perl
 http://www.rakunet.org/tsnet/TSabc/18/546.html

 Hiroaki Izumi, Perl5.8/Perl5.10 is not useful on the Windows.
 http://dl.dropbox.com/u/23756062/perlwin.html
 https://sites.google.com/site/hiroa63iz/perlwin

 TSUKAMOTO Makio, Perl memo/file path of Windows
 http://digit.que.ne.jp/work/wiki.cgi?Perl%E3%83%A1%E3%83%A2%2FWindows%E3%81%A7%E3%81%AE%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB%E3%83%91%E3%82%B9

 chaichanPaPa, Matching Shift_JIS file name
 http://d.hatena.ne.jp/chaichanPaPa/20080802/1217660826

 SUZUKI Norio, Jperl
 http://homepage2.nifty.com/kipp/perl/jperl/

 WATANABE Hirofumi, Jperl
 http://www.cpan.org/src/5.0/jperl/
 http://search.cpan.org/~watanabe/
 ftp://ftp.oreilly.co.jp/pcjp98/watanabe/jperlconf.ppt

 Chuck Houpt, Michiko Nozu, MacJPerl
 http://habilis.net/macjperl/index.j.html

 Kenichi Ishigaki, Pod-PerldocJp, Welcome to modern Perl world
 http://search.cpan.org/dist/Pod-PerldocJp/
 http://gihyo.jp/dev/serial/01/modern-perl/0031
 http://gihyo.jp/dev/serial/01/modern-perl/0032
 http://gihyo.jp/dev/serial/01/modern-perl/0033

 Fuji, Goro (gfx), Perl Hackers Hub No.16
 http://gihyo.jp/dev/serial/01/perl-hackers-hub/001602

 Dan Kogai, Encode module
 http://search.cpan.org/dist/Encode/
 http://www.archive.org/details/YAPCAsia2006TokyoPerl58andUnicodeMythsFactsandChanges (video)
 http://yapc.g.hatena.ne.jp/jkondo/ (audio)

 Juerd, Perl Unicode Advice
 http://juerd.nl/site.plp/perluniadvice

 daily dayflower, 2008-06-25 perluniadvice
 http://d.hatena.ne.jp/dayflower/20080625/1214374293

 Jesse Vincent, Compatibility is a virtue
 http://www.nntp.perl.org/group/perl.perl5.porters/2010/05/msg159825.html

 Tokyo-pm archive
 http://mail.pm.org/pipermail/tokyo-pm/
 http://mail.pm.org/pipermail/tokyo-pm/1999-September/001844.html
 http://mail.pm.org/pipermail/tokyo-pm/1999-September/001854.html

 ruby-list
 http://blade.nagaokaut.ac.jp/ruby/ruby-list/index.shtml
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/2440
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/2446
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/2569
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/9427
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/9431
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/10500
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/10501
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/10502
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/12385
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/12392
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/12393
 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-list/19156