NAME

balance.pl - Create a balanced Senseval-2 data file that has the same number of instances for each possible sense.

SYNOPSIS

 balance.pl [OPTIONS] DATA N > Balanced-DATA

This is the original distribution of senses in a Senseval-2 data file:

 frequency.pl begin-v.test.xml

Output =>

 <sense id="begin%2:30:00::" percent="64.31"/>
 <sense id="begin%2:30:01::" percent="14.51"/>
 <sense id="begin%2:42:04::" percent="21.18"/>
 Total Instances = 255
 Total Distinct Senses=3
 Distribution={64.31,21.18,14.51}
 % of Majority Sense = 64.31

Here they are balanced with 20 instances per sense.

 balance.pl begin.v-test.xml 20 > bal-output

 frequency.pl bal-output

Output =>

 <sense id="begin%2:30:00::" percent="33.33"/>
 <sense id="begin%2:30:01::" percent="33.33"/>
 <sense id="begin%2:42:04::" percent="33.33"/>
 Total Instances = 60
 Total Distinct Senses=3
 Distribution={33.33,33.33,33.33}
 % of Majority Sense = 33.33

Here they are balanced with 50 instances per sense. Please note that any sense with less than 50 instances is removed from the data.

 balance.pl begin.v-test.xml 50 > bal-output

 frequency.pl bal-output

Output =>

 <sense id="begin%2:30:00::" percent="50.00"/>
 <sense id="begin%2:42:04::" percent="50.00"/>
 Total Instances = 100
 Total Distinct Senses=2
 Distribution={50.00,50.00}
 % of Majority Sense = 50.00

You can find begin-v.test.xml in samples/Data

Type balance.pl --help for a quick summary of options

DESCRIPTION

This program will choose exactly the same number of instances for each sense found in a given Senseval-2 file. Unless a value is specified, it will choose the number of instances present in the least frequent sense. Output is to STDOUT, so the original input data is unchanged.

INPUT

Required Arguments:

DATA

balance.pl accepts a Senseval-2 data file.

N

Specifies the number of instances to be selected from each sense.

Optional Arguments:

--count COUNT

Balances the COUNT file created by SenseTool's preprocess.pl along with the DATA file. COUNT file is balanced such that it stays consistent with the new balanced DATA file and contains only those instances left after balancing, in the same order as they appear in the output.

Balanced COUNT is written to file COUNT.balanced and every ith line in COUNT.balanced is instance data within <context> and </context> tags for the ith instance in the output of balance.

Other Options :

--help

Displays this message.

--version

Displays the version information.

OUTPUT

Output is a sense balanced Senseval-2 file and is displayed to stdout. Output will show exactly N instances of each sense that has at least N instances. All senses in the output Senseval-2 will have equal number of instances meaning the senses will be equally distributed.

BUGS

The output of balance.pl will have un-balanced distribution of senses when some of the instances have multiple sense tags in the given DATA file.

AUTHORS

 Ted Pedersen,  University of Minnesota, Duluth
 tpederse at d.umn.edu

 Amruta Purandare, University of Pittsburgh

COPYRIGHT

Copyright (c) 2002-2008, Amruta Purandare and Ted Pedersen

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to

 The Free Software Foundation, Inc.,
 59 Temple Place - Suite 330,
 Boston, MA  02111-1307, USA.

cpanm Text::SenseClusters

perl -MCPAN -e shell
install Text::SenseClusters