CHANGES - Revision history for WordNet::Similarity
Fix make test error in lesktrace.t due to overlap results returning in unpredictable orders - problem is documented here : https://rt.cpan.org/Ticket/Display.html?id=86437 and fix is provided by Phil Goetz, philgoetz@gmail.com and involves sorting overlaps in lesk.pm to guarantee order in testing. Note that keys had to be regenerated after this fix installed using perl t/trace.t --key (TDP)
Install patch to fix WordNet version detection issues in Windows. Problem description and patch provided here : https://rt.cpan.org/Ticket/Display.html?id=79065
add doc/update-pod.sh in order to create plain text documentation (TDP)
fix WordNet download location in install.pod (TDP)
update prereqs in Makefile.PL (TDP)
Created new module WordNet::Similarity::FrequencyCounter containing common support code for information content programs. (Sid)
Updated all the frequency counting programs in /utils (*Freq.pl) to use the common code in WordNet::Similarity::FrequencyCounter. (Sid)
Changed the default path to Perl from /usr/local/bin to /usr/bin in all scripts and tests in the package. (Sid)
Fixed incorrect handling of BNC header information. (Sid)
Modified the compoundify() method in WordNet::Tools to include compounds containing special characters (period, hyphen, forward-slash, single-quote). (Sid)
Updated compoundify() to handle larger compounds. (Sid)
04/23/08
Fixed the "excessive ROOTs" bug in *Freq.pl. (Sid)
Fixed the extra verb concept counts in *Freq.pl. (Sid)
04/17/08
Reorganized similarity_server initialization. (Sid)
The similarity server now prints more intuitive messages. (Sid)
Attached timestamps to log messages. (Sid)
Added additional checks to input strings from clients. (Sid)
04/12/08
Added more detailed description of information content to rawtextFreq.pl, and made minor copy editing and formatting changes to other /utils files (TDP)
Made minor copy editing and formatting changes to files in /doc (TDP)
04/10/08
Moved get_wn_info, stem and vectorFile modules under WordNet, i.e., they are now WordNet::get_wn_info, WordNet::stem and WordNet::vectorFile. (Sid)
Updated all the modules and programs using the above modules. (Sid)
Added copyright notices in all module and program headers. (Sid)
Added method getCompoundsList() to WordNet::Tools. (Sid)
Made a more distrtibutable version of simialrity_server. The similarity_server is now "daemonized", and is installed in /usr/bin along with the other utils. (Sid)
03/23/08
Added SIGNATURE to distrribution to enable package verification. (Sid)
Updated MANIFEST to reflect new SIGNATURE. (Sid)
Set the LICENSE to gpl in META.yml and Makefile.PL. (Sid)
03/17/08
Added NO_META option to Makefile.PL to prevent automatic generation of META.yml during 'make dist'. (Sid)
Removed unused variable "loaded" from Makefile.PL. (Sid)
03/07/08
Removed all references to WordNet::QueryData from Makefile.PL. This is based on the following advice present in the ExtUtils::MakeMaker documentation: "Module installation tools have ways of resolving unmet dependencies but to do that they need a Makefile". By checking for the presence of WordNet::QueryData during 'perl Makefile.PL', we are preventing any opportunity for automated dependency resolution. (Sid)
The WordNet path (if specified by the WNHOME option during 'perl Makefile.PL') is not checked for validity beforehand, and is now directly provided as-is to build/Infocontent.PL and build/Depthfiles.PL. In case of a WNHOME error, now 'make' should fail instead of 'perl Makefile.PL' (which is more appropriate). (Sid)
Corrected a typo in DepthFinder.pm synopsis that refered to getTaxonomyRoot rather than getTaxonomies. Removed some cut and paste documentation from the templated used for GlossFinder.pm and PathFinder.pm (Ted)
Made synopsis examples WordNet version independent by not hard coding offsets, etc. Did this in Depthfinder.pm, PathFinder.pm, ICFinder, and GlossFinder.pm (Ted)
Made minor changes in path names and file names in the /samples directory and the /config-files subdirectory. (Ted)
03/04/08
Applied patch from Ben Haskell to fix a bug report (submitted by Quang Do Xuan) about failing self-similarity of tilde#n#1 using wup and lch measures. (Sid)
Added tests for above bug to t/wup.t and t/lch.t. (Sid)
Added WordNet::Similarity package version info to similarity.pl --version. (Sid)
01/31/08
Changed some default options in the similarity_server.conf configuration. (Sid)
Reformatted some of the similarity_server code. (Sid)
01/10/08
Reduced version requirements of some of the PREREQ_PM modules. (Sid)
Changed WordNet::QueryData requirements to v1.40 in the documentation. (Sid)
10/13/07
Fixed error in loading WordNet::Tools for similarity_server.pl. (Sid)
Removed the use of default (hardcoded) stoplist and word-vectors file for similarity_server.pl. (Sid)
Print WordNet hash-code instead of WordNet version, for similarity.cgi WordNet version information. (Sid)
10/09/07
Updated the Pathfinder code to handle loops in the WordNet is-a hierarchy (like the one in WN3.0). (Sid)
Updated MANIFEST, changelog and documentation to reflect the new changes. (Sid)
10/08/07
The modules now are not dependent on the version() method of WordNet::QueryData (which is no longer reliable). Instead they now use a 'hash-code' representing a specific WordNet distribution. (Sid)
Added module WordNet::Tools which provides the hashCode and compoundify methods used by most of the other modules and utilities. (Sid)
Completely modified the build procedure to generate data files during the 'make' step instead of the 'perl Makefile.PL' step. (Sid)
Removed the WordNet version numbers appended to synsetdepths.dat and treedepths.dat. (Sid)
Added two "build" utilities -- build/Infocontent.PL and build/Depthfiles.PL -- which are run during the 'make' step to generate data files. (Sid)
The default WordNet version is now v3.0. Changed all documentation, code and examples to reflect this. (Sid)
The package now requires WordNet::QueryData version 1.46 or above. (Sid)
Revised all tests and test-keys for the new code and new version of WordNet and QueryData. (Sid)
Removed the multiple pieces of code implementing "compoundify" and moved it all into a single method in WordNet::Tools. (Sid)
10/04/07
Included a default word vectors file in the distribution and eliminated the creation of a default word vectors file at install time. (Sid)
02/25/07
Fixed documentation where module WordNet::Similarity::path was referred to as WordNet::Similarity::edge (old name). (Sid)
01/30/07
Fixed wnDepths.pl man-page to display the wnpath option consistently in the usage and the description. (Sid)
Fixed the "deep recursion" error (only with WN3.0) in the findWPSDepths() subroutine in the wnDepths.pl script. (Sid)
12/13/06
Fixed major bug reported in vector_pairs, where every alternate function is skipped because of a loop variable being incremented twice. (Sid)
04/21/06
The web-interface was still not working for the vector measure, because only one side of the client-server interface had been updated. Updated the similarity server with code to support both, vector and vector_pairs measures. (Sid)
Updated the description of the Gloss Vector measure in measures.html (web interface). (Sid)
04/14/06
Applied Ben Haskell's patch to ICFinder.pm (to make the behaviour of the probability() and IC() functions consistent with their comments).
04/05/06
Updated the names for the Extended Gloss Overlaps measure and the Gloss Vector measure in the documentation. (Sid)
02/19/06
Updated PODs for all modules. (Sid)
Added tests for POD errors and for POD coverage. (Sid)
03/31/06
Changed "hash-style" constants (Perl v5.8) to single line constants (Perl v5.6) for compatibility with Perl v5.6.0. (Sid)
02/06/06
Added utility rankFormat.pl for ranking the output of similarity.pl and making the output suitable for input to rank.pl (to compute Spearman's correlation coefficient) of the Text::NSP package. (Sid)
01/15/06
Fixed issue in lesk.pm where undefined values for $wc1 and $wc2 caused errors with the normalize option. (Sid)
Fixed minor UI issues in wnDepths.pl. (Sid)
12/09/05
Modified get_wn_info.pm with Wybo Wiersma's changes. (Sid)
Modified lesk.pm, vector.pm and vector_pairs.pm to be compatible with above changes. (Sid)
12/07/05
Updated all utilities to use WordNet 2.1 (WordNet::QueryData 1.39 or above). (Sid)
Updated all modules and test cases for WordNet 2.1. (Sid)
12/05/05
Changed order of authors in package documentation. (Sid)
12/01/05
Added Wybo Wiersma's super-gloss caching code to GlossFinder.pm. (Sid)
Updated documentation to reflect above changes. (Sid)
12/11/05
tar file unpacked as WordNet-Similarity for June 12, v 0.15, now unpacks as WordNet-Similarity-0.15, which is consistent with all previous versions. (Ted)
Similarity.pm version was shown as 0.14, is now 0.15. Our general convention for modules is that their version number only change when the module itself changes, so the module version number can tell you when was the last time a module changed. However, for Similarity.pm this is needlessly confusing, so it will always carry the same version number as the release. (Ted)
06/10/05
Fixed a minor bug in MANIFEST. (Sid)
Updated modules.pod and developers.pod to reflect new software architecture. (Jason)
06/08/05
Re-introduced the previous (non-pairwise-comparison) vector. (Sid)
Updated documentation and test cases to support the new vector measure. (Sid)
Added default relation file for new vector measure. (Sid)
Expunged erroneous references to LCSFinder, esp. in test scripts. (JM)
04/21/05
removed LCSFinder module; moved LCS methods to DepthFinder, ICFinder, and PathFinder (JM)
renamed vector measure vector_pairs (JM)
03/24/05
Modified the documentation to reflect the relation file format for vector and for lesk. (Sid)
03/02/05
Set up selective test cases for "make test", depending upon the default data files installed by user. (Sid)
02/24/05
Reinstated default relation files for vector and lesk. In case the default relation files (vector-relation.dat and lesk-relation.dat) are missing, both modules would default to the glosexample-glosexample relation. (Sid)
Modified Makefile.PL to query the user before installing default data files. (Sid)
Removed infocontent file generation code from Makefile.PL. Now Makefile.PL simply calls utilities from the /utils directory (wnDepths.pl, semCorFreq.pl and wordVectors.pl) to generate the all default data files. (Sid)
Installation process now generates a default word vectors file. The vectordb configuration variable for vector is now optional. (Sid)
Earlier, the WNHOME option was given to Makefile.PL as --WNHOME <path>, whereas the PREFIX option was written as PREFIX=<path>. This inconsistent (and potentially confusing) notation has now been fixed. Now, the WNHOME option is provided to Makefile.PL as WNHOME=<path>. (Sid)
Added some basic tests for vector in t/vector.t.
12/11/04
Created WordNet::Similarity::GlossFinder.pm, a super-class of WordNet::Similarity::vector and WordNet::Similarity::lesk. (Sid)
Removed default relation file for lesk. Vector and lesk both default to glosexample-glosexample. (Sid)
10/29/04
Added vector to the CGI interface. (JM)
Incorporated a configuration file into similarity_server.pl. (JM)
10/28/04
Removed readDB.pl. (JM)
10/27/04
Modified string overlap finding in lesk to use the Text::OverlapFinder module. Removed string_compare.pm. This fixed an old bug where the relatedness of word1 and word2 wasn't always equal to the relatedness of word2 and word1. (JM)
Updated Makefile.PL, INSTALL, and doc/install.pod to reflect new dependency on Text::OverlapFinder. (JM)
Removed lib/dbInterface.pm and lib/string_compare.pm from MANIFEST. (JM)
10/19/04
Word vectors no longer stored in a BerkeleyDB database, a plain text file is now used. Modified wordVectors.pl, WordNet::Similarity::vector to use the plain text word vectors file. New module vectorFile.pm now used to access this plain text database. Module dbInterace.pm is obsolete. (Sid)
Modified Makefile.PL to no longer check for BerkeleyDB dependency. All modules are installed. (Sid)
09/23/04
Fixed bug in wup that allowed some relatedness scores to be greater than 1. This bug is discussed in the archives of the mailing list. (JM)
09/01/04
Modified vector to look like the other measures. It now is derived from WordNet::Similarity.pm. (Sid)
Updated the MANIFEST. (Sid)
Fixed some minor typos in Makefile.PL. (Sid)
Added single test case (for vector) to t/access.t. (Sid)
Fixed config option name conflict in WordNet::Similarity.pm. (JM)
Fixed WNHOME and WNSEARCHDIR related bugs. (JM)
Updated documentation for the web interface. (JM)
05/19/04
Fixed over-counting problem in *Freq.pl programs. Under certain conditions, word senses would sometimes get counted twice. (JM)
Updated *Freq.pl programs to use WordNet 2.0. (JM)
Input files to rawtextFreq.pl are now specified with the --infile option. (JM)
Improved speed of compound identification in rawtextFreq.pl by adding ',', ';', and ':' to the list of characters that we consider to be the end of a sentence (compound identification time is proportional to the square of the length of the sentence). (JM)
04/28/2004
Created a CGI-based web interface for the relatedness modules. (JM)
04/19/2004
Fixed problem with path to Perl interpreter in Makefile.PL. This was causing problems during installation if there was no /usr/local/bin/perl. (JM)
wnDepths.pl had forgotten that on Windows some filenames are different; for example, data.noun is noun.dat. (JM)
03/23/2004
In /t, save diff files between 0.06 and 0.07. Make sure to run diff tests for path/0.07 and edge/0.06.
03/16/2004
make sure that every .pm and .pl file has the same GNU copyleft language. Use PathFinder.pm as a template.
make sure that documentation is clear that vector and lesk require different format relation files (ie they are not interchangeable).
convert README into a series of pod documents in doc directory. In the intro.pod, provide a table of contents like structure (much like perldoc perl does).
Make sure that each pod documents follows the cpan style (name, synopsis, etc.) This should be true of any pod documentation in the package.
Modify INSTALL to describe local install correctly. In particular, the description of how to do a 'use lib' or -I may need adjustment.
03/12/2004
Make developers.pod into a self contained document that provides a step by step tutorial on how to write a measure of relatedness. The file NewStats.txt in NSP provides an example of the style of presentation that is expected.
developers.pod should be a tutorial that explains how to create a new measure. It should take the reader through a complete example, such as creating a measure that returns the sum of the information content of the concpets found in the shortest path between two concepts. This should include an example of how to use all of the available configuration options, and also adding a new one.
03/11/2004
document measure modules (lch.pm, wup.pm, etc.) with information about effect of hypo root node. (Take discussion from email explaining why it has an effect, and why it doesn't have an effect) and make it a part of the .pm perldoc. This will eventually be used in thesis writing, so it should be complete and detailed. Of particular important is the behavior of lch.pm, but all of the modules should have their expected behaviour with and without the hypo root node clearly documented. Also, you should note what the behavior was in 0.06 for both nouns and verbs, and if this has changed.
03/09/2004
lch.pm does not yet support not having a hypo root. Remember that the lack of hypo root will change (potentially) the max path length found for each taxonomy.
03/08/2004
depth finding code should be contained with DepthFinder.pm. We should not do any depth finding on the fly, rather that should all be precomputed (like we do info content). That includes the depth of individual concepts, and the max depths of taxonomies.
When wup.pm encounters two or more paths to the root, the trace output "condenses" those paths into a single path. It would be better to show all paths in the trace (as res does, for example). Also, make sure that the depth reported in such cases is always the minimum (shortest path to root).
03/05/2004
Modify wnDepths such that it shows both the depths of individual concepts, as well as the max distance from a root node. In the case of multiple inheritance, wndepths should show the depth of the concept in each case, and also the relevant root node. wnDepths should sort these depths from shortest to longest. The output of wndepths should be formatted like infocontent.dat, anticipating an eventual merger.
03/02/2004
in docs, update/replace current discussion of modules. Include example usage as well. Make sure that path length is clearly defined for lch, edge, and wup.
02/25/2004
In PathFinder.pm, Infocontent.pm, Similarity.pm, and LCSFinder.pm each function should be documented in perldoc form such that their input, output and basic functionality is described. This should then appear in the DESCRIPTION portion of the perldoc. The SYNOPSIS should contain examples or templates of each function being used.
02/23/2004
redo random pairs testing such that we have 60 noun-noun pairs, 25 verb-verb pairs, and 15 mixed pairs.
02/20/2004
Revisit the distance versus similarity issue in jcn.pm. It maybe be that simply inverting the distance is too extreme a solution. One possibility is to make it a linear transformation via maxdist - dist instead. (JM - we'll stick with inverting the distance, but added a discussion of this issue to the documentation)
02/18/2004
document all multiple inheritance issues that are being handled for measures.
02/16/2004
validateSynset should check wps format fairly closely, and issue descriptive errors if the wps is ill formed. Words can apparently be about anything (except #) but pos should be lower case nvra, and senses should be digits. Error messages should point out which field is the problem, or if there are too few or too many fields.
place all hypo root handling node code in PathFinder.pm. The measures should not have any hypo root handling code in them.
PathFinder.pm should include a function getAllPaths.pm that returns all paths between two concepts, their length, and their "tops" (the candidate LCSs). This should be used as the main source of input for the getLCS* functions, and for getShortestPath.
remove all "input verifcation" code from the measures. That should be inherited from Similarity.pm.
There is replicated code in the measure modules that checks validity of input. This should be removed to a common module that can be called by all of the measures. Any other replicated code should be removed as well. The goal of 0.07 is to largely eliminate replicated code via the use of inheritance, and to make the writing of new measures simpler.
02/13/2004
add pod/perldoc to lib/ICFinder.pm. Should also be done for all other files as they are modified for other reasons. In particular, introductory material that appears in source code comments, author information, GPL, etc. should be moved into pod and removed from source code comments. See similarity.pl for an example.
path should use getShortestPath from PathFinder.pm.
02/09/2004
getLCSDepth, getLCSInfo, getLCSPath should appear in LCSFinder.pm, which should inherit from both ICFinder and Pathfiner.
The measures (lch, path, jcn, lin, res, wup) should default to having the hypo root node turned on (for both nouns and verbs). This will eventually be true of hso, but is not currently. hypo root nodes could also be used for lesk and vector, although they are not currently.
02/04/2004
Wps and offsets will be supported internally. The user can request either mode via an option to getRelatedness. offset is our default. profiling has shown wps to be somewhat faster, in that it makes fewer calls to getSense, although it does make some. For input, we only support wps. For trace output we support wps and offset. For output we support wps and offset.
01/29/2004
modify option in config files such that an option without a value reverts to the default in all cases (except vectordb).
01/24/2004
Provide support for undefined values in the path finding and info content measures (path, wup, lch, res, lin, jcn). If two concepts are not in the same taxonomy then an error should be issued and a large negative integer should be returned. This can occur in two cases, between the same part of speech (noun-noun, verb-verb), or between nouns and verbs. Distinct error messsages should be indicated in both cases.
01/20/2004
Clean up configuration file examples (in samples). Make them consistent by having a master list (all-options.conf) that is what we make changes to. Then specific example files can be created via copy and paste. Make sure all possible options for a measure are included, and that the explanations describe all possible values as well as default handling. (TDP updated all-options.conf on 12/10/03, use this as source of cut and paste).
01/19/2004
Create test scripts that can be run to verify the correctness of output - they should include "correct" answers that can be compared to (automatically) and rerun as the system changes. We should use the CPAN module Test::More, and create .t files in a /t directory that test specific situations/problems, etc. The .t files themselves should be documented with an explanation of what is being tested. We should have lots of smaller, specific .t tests (rather than a few big test files). Whenever a bug is found and fixed, a .t file should be created that tests the fix, and this should be mentioned in the source code comments where the fix is made (this fix is tested by t/xyz.t).
Make sure that the testing system can be easily extended/modified, and that it can support the use of multiple input files and configuration files. We should have multiple *.t files to run our tests, and each module and utility should have at least its own *.t file (maybe more than one in some cases). We should also have *.t files that are dedicated to particular situations that affect a number of measures (like what happens when info content is zero for one concept, what happens if one of the concepts being compared is the lcs of the other, what if the two concepts are the same (self similarity), and so forth.
Test cases for configuration file handling should include:
repeated options in configuration file, as in
trace::0 trace::1
bad values in configuration file, as in
trace::nothankyou
bad options in configuration file, as in
tracer::0
Test cases for similarity.pl should include:
ill formed file input for similarity.pl, as in
cat#dog#1 cat#n#2 cat#n#n cat#n#2 cat
Test cases for measures should include:
show that wps and offset methods of path finding are equivalent
check trace output for each of the measures. use wps format, as that is subject to fewer changes than offsets.
a "big" file of word pairs (maybe 100 pairs) that run all the measures and compare values to what is obtained in 0.6. If there are differences, let's see what they are.
Test cases for information content programs should include:
an information content file based on one of our resident text files that is large enough to be interesting (readme, gpl, etc.) as computed in 0.6/0.7 (should be the same). This can be used as a reference point when we make changes in future.
Information content computed with a very small number of concepts, to expose the counting problem that ted mentions below.
Test cases for wnDepth...
Generate output for 0.07 to use as a point of reference. A few specific manual checks would be good too (leather_carp, entity, etc.)
run tests to determine where the system now provides different results from version 0.06 - make sure to document these cases (that are different).
01/12/2004
document configuration options extensively in a separate pod called doc/config.pod. Organize such that you have options that are used with all measures, and then those that are used with certain classes of measures. Then, use this as a master copy to update .pm files with.
01/09/2004
modify option handling such that multiple occurrences of an option in a config file cause an error. For example
trace:: trace::1
should cause an error.
12/17/2003
SemCor1.7Freq.pl and SemTagFreq.pl need to be renamed. They are now called semCorRawFreq.pl and SemCorFreq.pl. semCorRawFreq.pl counts without sense tags and SemCorFreq.pl counts the sense tags. (TDP)
12/09/2003
In similarity.pl cache error strings that indicate that two input synsets are from different parts of speech so that we only print out a warning once for each unique word1#pos1 word2#pos2 combination (JM)
Enhance similarity.pl file handling (for input files). Comments should be allowed - this will help in creation of test data (we can explain in the comment what "case" is being tested by a particular set of pairs. Use standard perl commenting style line starting with a # is a comment. Note that I don't think we can use the convention of # anywhere in a line as being the start of a comment (due to w#p#s) but I think any line that starts with a # can be safely treated as a comment. (JM -- we are using // to indicated the start of a comment)
Enhance similarity.pl file handling (for input files). At present if a single word (not a pair) appears on a line, no error is issued. It silently ignores this case. This should result in an error to the effect that the input format is invalid, only one word. Also, I'm not sure what happens if you have more than two words on a line. An error of some sort would also be necessary in that case. Also, I am not sure if similarity.pl checks to see that the words pairs are "well formed", that is to say do they adhere to the word, word#pos, or word#pos#number format. It would be good to have a simple check that verifies we have alphanumeric words, pos of n, v, a, or r, and numeric numbers. (JM)
12/08/2003
Clean up configuration file examples (in samples). Make them consistent by having a master list (all-options.conf) that is what we make changes to. Then specific example files can be created via copy and paste. Make sure all possible options for a measure are included, and that the explanations describe all possible values as well as default handling. (JM)(TDP updated all-options.conf on 12/10/03, use this as source of cut and paste).
Determine if it is feasible (not too difficult or time consuming) to modify --version option so it can display both the version of similarity.pl and the version of the module used when --type is specified. (JM -- version will show module version as well if a module is specified)
12/05/2003
all configuration options are now printed to traceString after module initialization. (JM)
explain the distinction between compounds and collocations raised in sample README. (Drop the distinction, and clarify what we mean by Wordnet compounds. TDP Dec 3). (JM)
12/04/2003
document caching for random (normally random uses an unlimited cache size) (JM -- random now uses the same default as all other measures)
determine a reasonable default cache size. Should not be unlimited. Current default is 1000, maybe it can be increased to 5000 or 10000. Let lesk with trace be the standard as to what is reasonable. (JM -- default is now 5,000).
Improve error handling when processing config files. Make sure the values specified are valid and that filenames refer to extant files. All options should allow the value to be omitted, in which case the default is used. (JM)
12/01/2003
Adjust Makefile.PL to account for new contents of samples directory. Added entries to MANIFEST as well. (JM)
update samples/sample.pl to run with the new files (and organization) provided in the samples directory. This was also a problem in 0.06, where it did not run for hso properly due to a mismatch in the name specified in sample.pl and the configuration file.
Rename infocontent.dat in Makefile.PL to use our standard name for semcor information content files. Name should reflect options used in computing information content values (if any). JM
relation.dat is in lib/WordNet. Should be referred to as lesk-relation.dat. Should also have vector-relation.dat I would think. (if not, what does vector do?). JM (vector doesn't try finding a default relation file--it fails silently).
/sample/vector-relation.dat is wrong. Calls itself LeskRelationFile. JM
In intro.pod, provide instruction on how to convert to html or whatever if user wishes (just point them to documentation that describes this elsewhere even). JM
11/28/2003
remove wordnet 1.7.1 compounds from samples directory. (TDP)
change comment in Similarity.pm to explain the pluses and minuses of using/not using a unique root node. (JM)
11/26/2003
added info content files in samples/Infocontent
changed version numbers to 0.07 in all modules and utils
fixed bug in wup: if user supplies car#n#1 and auto#n#1, the LCS found by wup is motor_vehicle#n#1, not car#n#1
added POD to all programs in /samples
11/24/2003
added documentation (in the form of POD) to /doc
11/21/2003
added /doc directory to contain documentation
11/18/2003
ensured that each measure initializes a part-of-speech list in _initialize
all measures (except vector) now use fetchFromCache and storeToCache
updated README:
Replaces most references to WordNet 1.7.1 with 2.0
Add some documentation on how to write a new measure
added an INSTALL file
cleaned up /samples. relation.dat is now named lesk-relation.dat and added vector-relation.dat. A sample config file is also provided for each measure (in /samples/config-files)
11/15/2003
updated jcn, hso, random, and lesk to use the funcitions that have been moved to Similarity.pm (such as the cache management functions).
cleaned up the /samples directory. Removed outdated files. Put sample config files in samples/config-files. Added README in /samples.
11/12/2003
Added fetchFromCache() and storeToCache() to Similarity.pm to make caching easier and cleaner.
Updated wup, edge, lch, res, and lin to use fetchFromCache() and storeToCache().
10/25/2003
Reduced the amount of duplication code in the measure modules by moving some common code to WordNet::Similarity. WordNet::Similarity is now a base class for all the measures. Also added a module called infocontent.pm from which all information content measures are descended (i.e., res, lin, jcn).
Removed @ symbol from all email addresses in all files (I think). This might help keep spammers from harvesting our email addresses.
10/18/2003
Removed dependence of the vector measure on PDL. Implemented "in-house" sparse vector manipulation functions.
Modified the README with updated documentation of similarity.pl (--interact option) and wordVectors.pl.
10/15/2003
Changed Makefile.PL so that it checks for version 1.30 of QueryData
10/13/2003
Added "maxCacheSize" option to all measures.
Added "maxCacheSize" option info to the man/pod documentation.
Used the new dataPath() method of QueryData 1.31 in all the utilities to obtain the path of the WordNet data files.
Modified Makefile.PL to check for PDL and BerkeleyDB dependency during installation. vector.pm is not installed on failed dependencies.
10/11/2003
Replaced instances of deprecated WordNet::QueryData::query with WordNet::QueryData::queryWord in hso.pm
made hso.pm check QueryData version. queryWord was broken in QueryData 1.29 and earlier
added support for new relations in WordNet 2.0 to get_wn_info.pm
updated test scripts to work with WN 2.0 (and WN 1.7.1)
10/06/2003
Added rootNode option to wup.pm
09/27/2003
Fixed syntax error in wordVectors.pl.
Added readDB.pl to utils.
Changed contact information in docs.
Re-organized the samples subdirectory.
Fixed typo in random.pm.
Updated the MANIFEST.
09/21/2003
Updated POD for WordNet::Similarity::wup
Added option to wup to specify a cache size in a configuration file.
similarity.pl now 'use's QueryData 1.30 or later. Previous versions of QueryData will not work. t/access.t also 'use's QueryData 1.30. get_wn_info.pm and lesk.pm both check for QueryData 1.30 and will die if it not found.
Reorganized the bibliography in README and slightly re-worded part of the introduction.
09/18/2003
Added new Wu Palmer measure of similarity (lib/WordNet/Similarity/wup.pm)
Updated README to mention wup
Added t/wup.t
Updated POD for WordNet::Similarity to mention wup
Updated the help message of similarity.pl to mention wup
Added t/wup.t and lib/WordNet/Similarity/wup.pm to MANIFEST
09/05/2003
Added '--interact' option to similarity.pl.
Changed the structure of the Vector Relation File.
Fixed a minor bug in similarity.pl. (s///g)
Updated the perldocs for the measures.
Incorporated some new features into the 'wordVectors.pl' utility. These features were used for thesis experiments.
Added documentation about the Lesk and Vector relation files (they have different formats now).
06/03/2003
Added new measure of semantic relatedness, based on co-occurrence vectors of WordNet glosses.
Set up the package so that similarity.pl and the other perl utilities get installed in "/usr/local/bin".
Complete rewrite of similarity.pl with cleaner code and added functionality:
Multiple parts of speech can be specified as car#nv (noun and verb forms of car) or cool#nar (noun, adjective and adverb forms of cool).
Word senses can now be specified as car#n#2, jump#v#2, etc.
Added functionality to similarity.pl to use a local install of WordNet::Similarity modules (in non-standard directories).
Output of similarity.pl now specifies the senses that represent the relatedness of two words.
Enforced limit on the cache size of modules.
Updated README to reflect the changes and to specify options for local installs of similarity.pl and the other utilities.
Fixed the perl docs (remove leading spaces).
Added mailing list address to documentation -- (http://groups.yahoo.com/group/wn-similarity).
Improved jcn and lin tracing ("bird-crane" problem obvious now).
Added new utility wordVectors.pl required for WordNet::Similarity::vector module.
05/02/2003
*Fixed* newline in traces.
*Fixed* blank line bug in brownFreq.pl.
*Fixed* "--offset" option bug in similarity.pl.
*Fixed* lin measure non-normalized scores... added zero infocontent handling in jcn and lin.
New utility rawtextFreq.pl, to generate information content files from plain text.
similarity.pl supports option to specify part-of-speech of input words while measuring relatedness.
Added option to specify (conifuration / information content) file in similarity.pl.
Added Resnik counting option to the information content generation utilities.
More documentation on information content utilities.
Added Add-1 smoothing option to the information content generation utilities.
03/10/2003
Removed trace bug in hso.pm.
Added test cases for all modules.
02/10/2003
Created CPAN modules from distance ver 0.11.
Modules are completely object oriented.
Added Adapted Lesk semantic relatedness measure -- lesk.pm.
Added simple edge counting semantic relatedness measure -- edge.pm.
Added a random relatedness measure -- random.pm.
jcn, res and lin measures now support verb hierarchies.
Information content files can now be specified as parameters to the modules.
Tools provided to build information content files from various publicly available corpora.
Various parameters now control the behavior of the modules. These parameters are passed to the modules through 'configuration files'.
Ted Pedersen, University of Minnesota, Duluth tpederse at d.umn.edu Siddharth Patwardhan, University of Utah, Salt Lake City sidd at cs.utah.edu Satanjeev Banerjee, Carnegie Mellon University, Pittsburgh banerjee+ at cs.cmu.edu Jason Michelizzi
todo.pod
Copyright (c) 2005, Ted Pedersen, Siddharth Patwardhan, Satanjeev Banerjee and Jason Michelizzi
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
Note: a copy of the GNU Free Documentation License is available on the web at http://www.gnu.org/copyleft/fdl.html and is included in this distribution as FDL.txt.
To install WordNet::Similarity, copy and paste the appropriate command in to your terminal.
cpanm
cpanm WordNet::Similarity
CPAN shell
perl -MCPAN -e shell install WordNet::Similarity
For more information on module installation, please visit the detailed CPAN module installation guide.