The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

HDR EXAMPLES

This document will help you understand how to use the hdr command which is a part of the Data::HexDump::Range module. Some advanced examples use the module directly.

The latest version is at http://search.cpan.org/search?query=Data%3A%3AHexDump%3A%3ARange.

This document contains many embeded examples in HTML format. It is mean to be processed by pod2html or equivalent.

When in the hexdump land, do as hexdumper do

Without range definition hdr act like hexdump.

 >$ hdr file_name

00000000 63 6f 6d 6d 69 74 20 38 38 30 39 63 38 65 38 34  commit 8809c8e84 
00000010 65 62 33 32 33 62 62 61 61 33 36 36 39 65 39 34  eb323bbaa3669e94 
00000020 38 62 36 38 35 30 32 36 63 33 35 66 61 35 39 20  8b685026c35fa59  
00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A 
00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe 
00000050 6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir  
00000060 67 3e 0a 44 61 74 65 3a 20 20 20 53 75 6e 20 41  g>.Date:   Sun A 
00000070 70 72 20 31 38 20 30 33 3a 33 32 3a 30 33 20 32  pr 18 03:32:03 2 
00000080 30 31 30 20 2b 30 32 30 30 0a 0a 20 20 20 20 41  010 +0200..    A 

Output format

The default output format is ANSI. ASCII is like ANSI without the colors.

ASCII

  00000000 63 6f 6d 6d 69 74 20 38 38 30 39 63 38 65 38 34  commit 8809c8e84
  00000010 65 62 33 32 33 62 62 61 61 33 36 36 39 65 39 34  eb323bbaa3669e94
  00000020 38 62 36 38 35 30 32 36 63 33 35 66 61 35 39 20  8b685026c35fa59
  00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A
  00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe
  00000050 6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir <nkh@cpan.or
  00000060 67 3e 0a 44 61 74 65 3a 20 20 20 53 75 6e 20 41  g>.Date:   Sun A
  00000070 70 72 20 31 38 20 30 33                          pr 18 03

ANSI

ANSI can only be rendered on a terminal but it look like the HTML format below

HTML

00000000 63 6f 6d 6d 69 74 20 38 38 30 39 63 38 65 38 34  commit 8809c8e84 
00000010 65 62 33 32 33 62 62 61 61 33 36 36 39 65 39 34  eb323bbaa3669e94 
00000020 38 62 36 38 35 30 32 36 63 33 35 66 61 35 39 20  8b685026c35fa59  
00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A 
00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe 
00000050 6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir  
00000060 67 3e 0a 44 61 74 65 3a 20 20 20 53 75 6e 20 41  g>.Date:   Sun A 
00000070 70 72 20 31 38 20 30 33                          pr 18 03         

Range descritption

                  format                          range example
                  
  normal range => integer                         header, 4, bright_blue
  comment      => #                               data section start, # 
  bitfield     => [XInteger][xInteger]bInteger    bitfield, X2x4b4         # X is byte offset, x is bit offset
  skip range   => XInteger                        boring, X256,, comment

Direct command line defintion

You can define your range description on the command line by passing a string to the --range_description (short -r) option.

  hdr -r 'magic, 5,   :other,37  :bf,b8   :skip,X32,, I skipped :more, 20'

Definition from a file

The --range_description option also accept a file namewhich contain a range description written in Perl.

  hdr -r my_range_description_file

simple file

        [
          ['data header', 5, 'blue on_cyan'],
          ['data', 20, 'blue on_bright_yellow'],
        ] ;

more structured file

 use strict ;
 use warnings ;
 
 # any perl code you may need to generate your range definition
 
 my $data_range = # definition to re-use
        [
          ['data header', 5, 'blue on_cyan'],
          ['data', 20, 'blue on_bright_yellow'],
        ] ;
 
 my $structured_range = 
        [
          ['zero size', 0],
          [
            ['magic cookie', 12, 'red'],
            ['padding', 32, 'yellow'],
            $data_range, 
          ],
          ['A comment', '#'],           
          [
            ['extra data', 18, undef],
              [
              $data_range, 
              ['footer', 4, 'bright_yellow on_red'],
            ]
          ],
        ] ; 
        
 $structured_range ; # return the range description

See Also "Advanced usage"

Decimal dump

You can get a decimal dump instead for an hexidecimal dump.

 hdr  -r 'header,20 :DATA,4  :c,b24  :e,x4b5 :uninterresting, x16  :more,20' -display_dec_dump 1 -display_hex_dump 0 -rul -col -o ver 
RANGE_NAME       OFFSET   CUMULATI DEC_DUMP                                                         ASCII_DUMP        
                                   0   1   2   3   4   5   6   7   8   9   0   1   2   3   4   5    0123456789012345  
header           00000000 00000000 010 112 097 099 107 097 103 101 032 068 097 116 097 058 058 072  .package Data::H 
header           00000010 00000010 101 120 068 117                                                  exDu             
DATA             00000014 00000000 109 112 058 058                                                  mp::             
.c               00 .. 23          --- 112 058 058 value: 7354938                                   .bitfield: -p::  
.e               04 .. 08          --- --- --- 003 value: 3                                         .bitfield: ---.  
>>uninterresting 00000018 00000027 000 000 000 016 skipped: 16 bytes                                                 
more             00000028 00000000 105 099 116 059 010 117 115 101 032 119 097 114 110 105 110 103  ict;.use warning 
more             00000038 00000010 115 032 059 010                                                  s ;.             

You can also have both dumps simulteanously. Check the display option of the hdr command.

Colors and Headers

Showing the column header and the rulers

  $>hdr -col -rul your_file
OFFSET   HEX_DUMP                                         ASCII_DUMP        
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345  
00000000 63 6f 6d 6d 69 74 20 38 38 30 39 63 38 65 38 34  commit 8809c8e84 
00000010 65 62 33 32 33 62 62 61 61 33 36 36 39 65 39 34  eb323bbaa3669e94 
00000020 38 62 36 38 35 30 32 36 63 33 35 66 61 35 39 20  8b685026c35fa59  

Defining your own colors

Define a custom color file containing a data in the format below andd pass it as argument to option --colors

  {
  HTML =>
        {
        # custom color name
        custom => 'color:#f00; background-color:#ff0; ',
        
        # the normal ansi color
        reset => '',
        white => 'color:#aaa; ',
        black => 'color:#000; ',
        green => 'color:#0a0; ',
        ...
        },
  ANSI =>
        {
        # custom color name, it's a good idea to define them on both formats
        custom => 'bright_red on_bright_yellow; ',
        
        reset => '',
        white => 'white',
        ...
        },
  }

Colorize only the range you give color to

By default hdr will assign colors to ranges you don't give a color to.

  hdr -r 'A,10:B,10,red:C,10:D,10,green' -col -rul
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_  
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345         
00000000 63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 A, B,  
00000010 62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 B, C, D,  
00000020 63 34 30 62 64 65 31 39                          c40bde19         D,  

If you only want the ranges you gave a color to to be colorized,use the --color bw option

  hdr -r 'A,10:B,10,red:C,10:D,10,green' -col -rul -color bw
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_  
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345         
00000000 63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 A, B,  
00000010 62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 B, C, D,  
00000020 63 34 30 62 64 65 31 39                          c40bde19         D,  

Input

from file or stdin

  hdr -r 'magic cooookie, 5,  ...'  -rul -o ver -col my_file

or pipe your data into hdr

  cat my_file | hdr -r 'magic, 5,  ...' -rul -o ver -col

Too little data

hdr will display one or more warnings is there is not enough data for all the defined ranges.

  Warning: range 'xxx' size 32 was shortened to 8 due to maximum size limit
  Warning: More ranges to display but no more data.

Too much data

hdr only uses the part that is used by your range descriptions. The rest is ignored.

If you want to use an existing range description that contains many range definitions, you can use the --maximum_size option to limit the amount of data hdr will use.

  $>hdr -r 'A,50:B,50:C,50:D,50' --maximum_size 100

will display a warning:

  Warning: More ranges to display but no more data.

and this dump:

OFFSET   HEX_DUMP                                         ASCII_DUMP       RAN  
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345      
00000000 63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 A,  
00000010 62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 A,  
00000020 63 34 30 62 64 65 31 39 62 61 31 33 38 63 34 20  c40bde19ba138c4  A,  
00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A A, B,  
00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe B,  
00000050 6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir  B,  
00000060 67 3e 0a 44                                      g>.D             B,  

Offset and Offset start

Say you have the following hexdump

  hdr -r 'A,55:B,5:bf,x3b5:C,18:D,10' -col -rul
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_NAME        
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345                   
00000000 63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 A,  
00000010 62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 A,  
00000020 63 34 30 62 64 65 31 39 62 61 31 33 38 63 34 20  c40bde19ba138c4  A,  
00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A A, B, C,  
00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe C, D,  
00000050 6d 69 72 20 3c 6e 6b 68                          mir <nkh         D,  

Range A is of no interrest to you so you decide to remove it and the dump to start at range B.

  hdr -r 'B,5:bf,x3b5:C,18:D,10' -col -rul -offset 55
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_NAME        
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345                   
00000000 6d 61 73 74 65 72 29 0a 41 75 74 68 6f 72 3a 20  master).Author:  B, C,  
00000010 6e 61 64 69 6d 20 6b 68 65 6d 69 72 20 3c 6e 6b  nadim khemir <nk C, D,  
00000020 68                                               h                D,  

If you want the displayed offset to correspond to the offset in your binary file use the --offset_start option

 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -offset_start 55
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_NAME        
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345                   
00000037 6d 61 73 74 65 72 29 0a 41 75 74 68 6f 72 3a 20  master).Author:  B, C,  
00000047 6e 61 64 69 6d 20 6b 68 65 6d 69 72 20 3c 6e 6b  nadim khemir <nk C, D,  
00000057 68                                               h                D,  

Start color

if you do not assign colors to your ranges, DATA::HexDump::Range will conveniently assign a default color. Although this is what you want most of the time, in some cases you want to control what default color is used first.

In the above example, the first dump had range B displayed in bright_yellow and the second dump had it displayed in bright_green.

You can use option --start_color to force the color to match the dump that included the A range.

  hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -offset 55 -offset_start 55 -start_color bright_yellow
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_NAME        
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345                   
00000037 6d 61 73 74 65 72 29 0a 41 75 74 68 6f 72 3a 20  master).Author:  B, C,  
00000047 6e 61 64 69 6d 20 6b 68 65 6d 69 72 20 3c 6e 6b  nadim khemir <nk C, D,  
00000057 68                                               h                D,  

See Also "Skip Ranges".

Bitfields

  $> hdr  -r 'header,1 :DATA,4  :c,b9  :d,b3 :e,x4b5  :the end,8 :x,b3 :data,47 :v,x90b8' -o ver

Vertical mode

header           00000000 00000000 0a                                               .                
DATA             00000001 00000000 70 61 63 6b                                      pack             
.c               00 .. 08          -- -- 01 6b    -----------------------101101011  .bitfield: --.k  
.d               00 .. 02          -- -- -- 03    -----------------------------011  .bitfield: ---.  
.e               04 .. 08          -- -- -- 16    -----------------------10110----  .bitfield: ---.  
the end          00000005 00000000 61 67 65 20 44 61 74 61                          age Data         
.x               00 .. 02          -- -- -- 01    -----------------------------001  .bitfield: ---.  
data             0000000d 00000000 3a 3a 48 65 78 44 75 6d 70 3a 3a 52 61 6e 67 65  ::HexDump::Range 
data             0000001d 00000010 20 3b 0a 0a 75 73 65 20 73 74 72 69 63 74 3b 0a   ;..use strict;. 
data             0000002d 00000020 75 73 65 20 77 61 72 6e 69 6e 67 73 20 3b 0a     use warnings ;.  
.v               90 .. 97          -- -- -- 48    01001000------------------------  .bitfield: ---H  

Horizontal mode

By default, bitmaps are not displayed in horizontal mode as they tend to clutter the display. You can force the display of bitfields in vertical mode by seting the --display_bitfields option to true.

without --display_bitfields option set:

00000000 0a 70 61 63 6b 61 67 65 20 44 61 74 61 3a 3a 48  .package Data::H header, DATA, the end, data,  
00000010 65 78 44 75 6d 70 3a 3a 52 61 6e 67 65 20 3b 0a  exDump::Range ;. data,  
00000020 0a 75 73 65 20 73 74 72 69 63 74 3b 0a 75 73 65  .use strict;.use data,  
00000030 20 77 61 72 6e 69 6e 67 73 20 3b 0a               warnings ;.     data,  

with --display_bitfields option set to true:

00000000          0a 70 61 63 6b 61 67 65 20 44 61 74 61 3a 3a 48  .package Data::H header, DATA, the end, data,  
00 .. 08 DATA     -- -- 01 6b    -----------------------101101011  .bitfield: --.k  .c               
00 .. 02 DATA     -- -- -- 03    -----------------------------011  .bitfield: ---.  .d               
04 .. 08 DATA     -- -- -- 16    -----------------------10110----  .bitfield: ---.  .e               
00 .. 02 the end  -- -- -- 01    -----------------------------001  .bitfield: ---.  .x               
00000010          65 78 44 75 6d 70 3a 3a 52 61 6e 67 65 20 3b 0a  exDump::Range ;. data,  
00000020          0a 75 73 65 20 73 74 72 69 63 74 3b 0a 75 73 65  .use strict;.use data,  
00000030          20 77 61 72 6e 69 6e 67 73 20 3b 0a               warnings ;.     data,  
90 .. 97 data     -- -- -- 48    01001000------------------------  .bitfield: ---H  .v               

Not enough data for the bitfield

If you define an offset and a bit amout that sets the bitfields outside the data range, or you define a bitfield with more than 32 bits, The bitfields will be displayed as a '?'

   hdr  -r 'data,1 :DATA,4  :c,b50  :d,x30b3 :e,x4b5  :the end,8' -o ver
data             00000000 00000000 0a                                               .                
DATA             00000001 00000000 70 61 63 6b                                      pack             
.c               00 .. 49          ?                                                ?                
.d               30 .. 32          ?                                                ?                
.e               04 .. 08          -- -- -- 16    -----------------------10110----  .bitfield: ---.  
the end          00000005 00000000 61 67 65 20 44 61 74 61                          age Data         

Bitfield offset

The format for a bitfield is [XInteger][xInteger]b[integer]. you can use X for byte offset and x for bit offset. If no size is given after b, the bitfield consists of a single bit

Examples:

  • X1x1b - second bit of the second byte

  • X3x8b4 - 4 first bits of the fifth byte

  • X4b8 - fifth byte

  • X1x0b8 - second byte

Remember that bit zero is on the right side by default. Also notice that since we have no fifth byte, the corrsponding bitfields are displayed with a '?'.

 RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP       USER_INFORMATION
 4:data           00000000 00000000 41 05 30 31                                      A.01
 .X1x1b           09 .. 09          -- -- -- 00    ----------------------0---------  .bitfield: ---.
 .X3x8b4          32 .. 35          ?                                                ?
 .X4b8            32 .. 39          ?                                                ?
 .X1x0b8          08 .. 15          -- -- -- 30    ----------------00110000--------  .bitfield: ---0
 8:the end        00000004 00000000 32 33 34 35 36 37 38 39                          23456789

With BIT_ZERO_ON_LEFT set

 RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP       USER_INFORMATION
 4:data           00000000 00000000 41 05 30 31                                      A.01
 .X1x1b           09 .. 09          -- -- -- 00    ---------0----------------------  .bitfield: ---.
 .X3x8b4          32 .. 35          ?                                                ?
 .X4b8            32 .. 39          ?                                                ?
 .X1x0b8          08 .. 15          -- -- -- 05    --------00000101----------------  .bitfield: ---.
 8:the end        00000004 00000000 32 33 34 35 36 37 38 39                          23456789

Skip Ranges

Say you have data with data that you are not interrested in seeing in the hexdump.

  $>hdr -r 'magic, 5  :other,37  :uninterresting,45     :more, 20'
00000000          63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 magic, other,  
00000010          62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 other,  
00000020          63 34 30 62 64 65 31 39 62 61 31 33 38 63 34 20  c40bde19ba138c4  other, uninterresting,  
00000030          28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A uninterresting,  
00000040          75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe uninterresting,  
00000050          6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir h@cpan.or uninterresting, more,  
00000060          67 3e 0a 44 61 74 65 3a 20 20 20                 g>.Date:         more,  

Change the uninterresting range definition to skip the data.

  $>hdr -r 'magic, 5  :other,37  :uninterresting,X45     :more, 20'
00000000          63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 magic, other,  
00000010          62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 other,  
00000020          63 34 30 62 64 65 31 39 62 61                    c40bde19ba       other, >>uninterresting 
00000057          68 40 63 70 61 6e 2e 6f 72 67 3e 0a 44 61 74 65  h@cpan.org>.Date more,  
00000067          3a 20 20 20                                      :                more,  

In vertical mode

magic            00000000 00000000 63 6f 6d 6d 69                                   commi            
other            00000005 00000000 74 20 31 39 39 61 34 62 31 32 37 62 32 39 66 39  t 199a4b127b29f9 
other            00000015 00000010 31 64 32 65 36 33 66 39 35 66 38 63 34 30 62 64  1d2e63f95f8c40bd 
other            00000025 00000020 65 31 39 62 61                                   e19ba            
>>uninterresting 0000002a 00000056 00 00 00 2d bytes skipped                                         
more             00000057 00000000 68 40 63 70 61 6e 2e 6f 72 67 3e 0a 44 61 74 65  h@cpan.org>.Date 
more             00000067 00000010 3a 20 20 20                                      :                

Comment

You can use comments to introduce the ranges that are going to be displayed next and thus make it easier to understand

        [
          ['data header', 5, 'blue on_cyan'],
          ['Next is a very interresting data section', #, 'bright_white on_red'],
          ['data', 20, 'blue on_bright_yellow'],
        ] ;


  $>hdr -o vertical -r 'magic, 5  :first interresting data section,#, black on_bright_cyan    :other,37  :something,45 :second interresting data section,#,black on_bright_cyan   :other,37  :something,45 :We are done, #, black on_bright_cyan  :more, 20'
magic            00000000 00000000 63 6f 6d 6d 69                                   commi            
"first interresting data section" 
other            00000005 00000000 74 20 31 39 39 61 34 62 31 32 37 62 32 39 66 39  t 199a4b127b29f9 
other            00000015 00000010 31 64 32 65 36 33 66 39 35 66 38 63 34 30 62 64  1d2e63f95f8c40bd 
other            00000025 00000020 65 31 39 62 61                                   e19ba            
something        0000002a 00000000 31 33 38 63 34 20 28 48 45 41 44 2c 20 6d 61 73  138c4 (HEAD, mas 
something        0000003a 00000010 74 65 72 29 0a 41 75 74 68 6f 72 3a 20 6e 61 64  ter).Author: nad 
something        0000004a 00000020 69 6d 20 6b 68 65 6d 69 72 20 3c 6e 6b           im khemir  
"second interresting data section" 
other            00000057 00000000 68 40 63 70 61 6e 2e 6f 72 67 3e 0a 44 61 74 65  h@cpan.org>.Date 
other            00000067 00000010 3a 20 20 20 54 75 65 20 41 70 72 20 32 30 20 30  :   Tue Apr 20 0 
other            00000077 00000020 31 3a 30 36 3a                                   1:06:            
something        0000007c 00000000 30 33 20 32 30 31 30 20 2b 30 32 30 30 0a 0a 20  03 2010 +0200..  
something        0000008c 00000010 20 20 20 46 49 58 45 44 3a 20 6d 61 78 69 6d 75     FIXED: maximu 
something        0000009c 00000020 6d 5f 73 69 7a 65 20 61 70 70 6c 69 65           m_size applie    
"We are done" 
more             000000a9 00000000 64 20 77 69 74 68 69 6e 20 61 20 72 61 6e 67 65  d within a range 
more             000000b9 00000010 0a 20 20 20                                      .                

Advanced Usage

First, read the module documentation.

dynamic range generation

The parser below will alternatively return a 'from 0' or 'from 1' range description depending on the first byte of the data to display

 sub my_parser 
        {
        my ($dumper, $data, $offset) = @_ ;
        
        my $first_byte = unpack ("x$offset C", $data) ;
        
        $offset < length($data)
                ?  $first_byte == ord(0)
                        ? ['from "0"', 5, 'bright_yellow']
                        : ['from "1"', 3, 'bright_green']
                : undef ;
        }       
 my $data = '01' x 20 ;
 print $hdr->dump(\&my_parser, $data) ;
00000000 30 31 30 31 30 31 30 31 30 31 30 31 30 31 30 31  0101010101010101 5:from "0", 3:from "1", 5:from "0", 3:from "1",  
00000010 30 31 30 31 30 31 30 31 30 31 30 31 30 31 30 31  0101010101010101 5:from "0", 3:from "1", 5:from "0", 3:from "1",  
00000020 30 31 30 31 30 31 30 31                          01010101         5:from "0", 3:from "1",  

IPV4 header

Below is the range description for an IPV4 header:

 use strict ;
 use warnings ;
 
 my $IPV4_header =
        [
        ['IPV4_header', '#', 'bright_white on_red'],
                [
                        ['Version, Header length,  Differentiated Services, Total Length', '#', 'blue on_cyan'],
                        ['First 32 bits', 4], 
                        ['Version', 'b4'],
                        ['Header length', 'x4b4'],
                        ['Differentiated Services', 'x8b8'],
                                # Differentiated Services (DS) 
                                # # bit 3: 0 = Normal Delay, 1 = Low Delay
                                # # bit 4: 0 = Normal Throughput, 1 = High Throughput
                                # # bit 5: 0 = Normal Reliability, 1 = High Reliability
                                # # bit 6: 0 = Normal Cost, 1 = Minimize Monetary Cost
                                # (defined by RFC 1349)
                                # # bit 7: never defined
                        ['Total length', 'x16b16'],
                ],

                [
                        ['Identification, Flags, Fragment Offset', '#', 'blue on_cyan'],
                        ['Second 32 bits', 4], 
                        ['Identification', 'b16'],
                        
                        ['reserved', 'x16b1'],
                        ["Don't Fragment", 'x17b1'],
                        ['More Fragments', 'x18b1'],
                                # bit 0: Reserved; must be zero. As an April Fools joke, proposed for use in RFC 3514 as the "Evil bit".
                                # bit 1: Don't Fragment (DF)
                                # bit 2: More Fragments (MF)

                        ['Fragment Offset', 'x19b13'],
                ],

                [
                        ['Time to Live, Protocol, Header Checksum', '#', 'blue on_cyan'],
                        ['Third 32 bits', 4], 
                        ['Time to Live', 'b8'],
                        ['Protocol', 'x8b8'],
                        ['Header Checksum', 'x16b16'],
                        [],
                ],
                
                ['Addresses and options', '#', 'blue on_cyan'],
                ['Source Address', 4],
                ['Destination Address', 4], 
                [
                sub
                        {
                        my ($self, $data, $offset, $size, $range) = @_ ;
                        
                        my ($header_length_chunk) = grep {$_->{NAME} eq 'Header length'} @{$self->{GATHERED}} ;
                        my $header_length = ord(substr($header_length_chunk->{DATA}, 0, 1)) & 0x0f  ;
                        
                        if($header_length > 5)
                                {
                                return ['Options', 4, undef ,'header length > 5, extracting options'] ;
                                }
                        else
                                {
                                # Note that we could choose to return a comment range instead
                                #~ return ['header length < 5, no options', '#', undef, undef] ;
                                
                                return undef, 'Skipping option field header length < 5';
                                }
                        }
                ],
                ['Data', '#', 'blue on_cyan'],
                ['Data', 128] # display some of the data
        ] ;

Note that the Options field is an optional field that is present when the Header length is greater than 5. For the Options field, we define subroutine which verifies the value of the Header length and selectively returns a range description or nothing.

In vertical mode

 $> hdr -r t/parse_ipv4.pl -display_ascii_dump 0 -bit_zero_on_left -o ver
 
"IPV4_header"  
"Version, Header length,  Differentiated Services, Total Length"  
First 32 bits    00000000 00000000 71 5f 63 68                                                                                          
.Version         00 .. 03          -- -- -- 07    0111----------------------------                        
.Header length   04 .. 07          -- -- -- 01    ----0001------------------------                        
.Differentiated  08 .. 15          -- -- -- 5f    --------01011111----------------                        
.Total length    16 .. 31          -- -- 63 68    ----------------0110001101101000                        
"Identification, Flags, Fragment Offset"  
Second 32 bits   00000004 00000000 61 6e 67 65                                                                                          
.Identification  00 .. 15          -- -- 61 6e    0110000101101110----------------                        
.reserved        16 .. 16          -- -- -- 00    ----------------0---------------                        
.Don't Fragment  17 .. 17          -- -- -- 01    -----------------1--------------                        
.More Fragments  18 .. 18          -- -- -- 01    ------------------1-------------                        
.Fragment Offset 19 .. 31          -- -- 07 65    -------------------0011101100101                        
"Time to Live, Protocol, Header Checksum"  
Third 32 bits    00000008 00000000 5f 6f 5f 74                                                                                          
.Time to Live    00 .. 07          -- -- -- 5f    01011111------------------------                        
.Protocol        08 .. 15          -- -- -- 6f    --------01101111----------------                        
.Header Checksum 16 .. 31          -- -- 5f 74    ----------------0101111101110100                        
"Addresses and options"  
Source Address   0000000c 00000000 6f 5f 71 5f                                                                                          
Destination Addr 00000010 00000000 62 6c 61 68                                                                                          
"Data"  
128->26:Data     00000014 00000000 5f 62 6c 61 68 5f 62 6c 61 68 5f 62 6c 61 68 5f                                                      
128->26:Data     00000024 00000010 62 6c 61 68 5f 62 6c 61 68 0a                                                                        
 

In horizontal mode

 $> hdr -r t/parse_ipv4.pl -display_ascii_dump 0 -bit_zero_on_left -display_bitfields 1 -display_comment_range 0
 
00000000          71 5f 63 68 61 6e 67 65 5f 6f 5f 74 6f 5f 71 5f  First 32 bits, Second 32 bits, Third 32 bits, Source Address,   
00 .. 03 First 32 -- -- -- 07    0111----------------------------  .Version          
04 .. 07 First 32 -- -- -- 01    ----0001------------------------  .Header length    
08 .. 15 First 32 -- -- -- 5f    --------01011111----------------  .Differentiated   
16 .. 31 First 32 -- -- 63 68    ----------------0110001101101000  .Total length     
00 .. 15 Second 3 -- -- 61 6e    0110000101101110----------------  .Identification   
16 .. 16 Second 3 -- -- -- 00    ----------------0---------------  .reserved         
17 .. 17 Second 3 -- -- -- 01    -----------------1--------------  .Don't Fragment   
18 .. 18 Second 3 -- -- -- 01    ------------------1-------------  .More Fragments   
19 .. 31 Second 3 -- -- 07 65    -------------------0011101100101  .Fragment Offset  
00 .. 07 Third 32 -- -- -- 5f    01011111------------------------  .Time to Live     
08 .. 15 Third 32 -- -- -- 6f    --------01101111----------------  .Protocol         
16 .. 31 Third 32 -- -- 5f 74    ----------------0101111101110100  .Header Checksum  
00000010          62 6c 61 68 5f 62 6c 61 68 5f 62 6c 61 68 5f 62  Destination Addr, 128->26:Data,   
00000020          6c 61 68 5f 62 6c 61 68 5f 62 6c 61 68 0a        128->26:Data,   
 

Comments that help

Remember that you can, while generating your ranges, assign the comment field

  ['range_name', 4, 'red', 'comment of great help']
  

You can also pass a sub reference in place of the comment. The sub will be called just before the range is displayed. It is passed information about the data to be displayed (data and offset). This allows you to dynamically generate a comment string.

        ['size', 1, undef, 
                sub
                        {
                        my ($self, $data, $offset, $size, $range) = @_ ;
                        
                        my $size = ord(substr($data, $offset, 1))  ;
                        
                        for($size)
                                {
                                $_ == 1 and do {return 'size is: S' } ;
                                $_ == 2 and do {return  'size is: M'} ;
                                $_ == 3 and do {return 'size is: L'} ;
                                $_ == 4 and do {return  'size is: XL'} ;
                                
                                return 'Error in size range!' ;
                                }
                        }
        ],
        

Which would generate :

 ...
 size             00000022 00000000 32                                               2                Error in size range!
 size             00000023 00000000 01                                               .                size is: S
 ...

Let the computer do the heavy lifting

hdr may be a nice tool to run from the command line but you can also use it in shell scripts to generate reports. I generally write pod so I let hdr generate my pod snippets with embeded html hexdumps.

 printf "=head1 MY REPORT\n\n" > report.pod
 printf "=head2 The data\n\n" >> report.pod
 
 hdr -r 'A,55:B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -format HTML -display_command_line -start_tag '=begin html' -end_tag '=end html' >> report.pod
 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -format HTML -display_command_line -start_tag '=begin html' -end_tag '=end html' >> report.pod
 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -offset_start 55 -format HTML -display_command_line -start_tag '=begin html' -end_tag '=end html' >> report.pod

 echo "=head2 Nicely synchronized\n\n" >> report.pod 
 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -offset_start 55 -format HTML -display_command_line -start_color bright_yellow -start_tag '=begin html' -end_tag '=end html' >> report.pod
 
 pod2html report.pod > report.html

Bash completion script

hdr can generate a bash completion script

 $> hdr --generate_bash_completion > hdr_completion
 $> source hdr_completion

Helping out

You contribute when you:

  • report errors

  • send some examples of your data/output

  • send parsers or other scripts in which you use hdr or Data::HexDump::Range

  • send new requirement, patches, documentation enhancements

  • pass the word around so more people use hdr and I get more feedback

  • plant a tree, save a cat/dog or both, bike instead for driving, spread some love around you, :)

AUTHOR

        Nadim ibn hamouda el Khemir
        CPAN ID: NKH
        mailto: nadim@cpan.org

COPYRIGHT & LICENSE

Copyright 2010 Nadim Khemir.

This program is free software; you can redistribute it and/or modify it under the terms of either:

  • the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or

  • the Artistic License version 2.0.

SUPPORT

You can also look for information at:

SEE ALSO

hexd from libma http://www.ioplex.com/~miallen/libmba/