tmx-tokenize - tokenizes translation units on a tmx file.
tmx-tokenize file.tmx # creates t_file.tmx tmx-tokenize -o=out.tmx file.tmx
Although this script is bundled in XML::TMX, it has a soft dependency on Lingua::FreeLing3. Soft means that the dependency is not ensured at install time, and other features of the module can still be used without Lingua::FreeLing3. Nevertheless, if you want to use this tool you should install that module.
XML::TMX
Lingua::FreeLing3
At the moment the supported languages are the same as supported by FreeLing3: English, Spanish, Russian, Portuguese and Italian.
It your TMX file includes any other language, they will be maintained without a change. This behavior can change in the future, as a basic regexp based tokenizer might be implemented.
XML::TMX, Lingua::FreeLing3
Alberto Manuel Brandão Simões, <ambs@cpan.org>
Copyright (C) 2012 by Alberto Manuel Brandão Simões
To install XML::TMX, copy and paste the appropriate command in to your terminal.
cpanm
cpanm XML::TMX
CPAN shell
perl -MCPAN -e shell install XML::TMX
For more information on module installation, please visit the detailed CPAN module installation guide.