NAME

Games::Freelancer::UTF - Perl extension for working with Microsoft UTF Files used in the Game Freelancer.

Synopsis

        use Games::Freelancer::UTF;
        open FILE,"model.cmp"; #or .utf, .3db, .txm, .mat, .ale, .vms, .dfm or maybe some more
        binmode FILE;
        my $content = do {local $/; <FILE>};
        close FILE;
        my $tree=UTFread($content);
        $code = UTFwriteUTF($tree);
        open FILE, ">out.cmp"
        binmode FILE;
        print FILE, $code;
        close FILE;

DESCRIPTION

This Module provides the ability to decode UTF files for the Mircrosoft game "Freelancer"

Those are named UTF files because of their header.

They are just trees that are encoded in binary, there might be a possibility that these files are used somewhere else, too.

In "Freelancer" they are used for models, meshes, materials, textures, effects and a lot more.

You can even use this to save hashes of hashes, but I highly recommend using something else, like Storable for this.

WARNING

The read routines return a tied InsertOrderHash, so just keep the reference you got and work with it instead of using

        my %mysuperhash = %{UTFread($crypted)}
        %mysuperhash{NewEntry} = "Data"
        # Now the order is destroyed
        # This is better:
        my $tree=UTFread($crypted);
        $tree->{NewEntry} = "Data"
        

I have no idea how important the order of the elements is for Freelancer, but better keep it this way. Of course all subhashes are also tied #Bad code example $tree->{copyme}={%{$tree}}; #Good code: tie my %newhash,'Tie::InsertOrderHash'; %newhash=(%{$tree}) #Copy tree, preserves order, at least last time I tested $tree->{copyme}=\%newhash; #not this: $tree->{copyme}={%newhash}; #Looses tiedness too.

FUNCTIONS

$tree = UTFread ($data);

Reads an UTF file content into a tied tree:

        use Games::Freelancer::UTF;
        use Data::Dumper;
        open FILE,"model.cmp";
        binmode FILE;
        my $content = do {local $/; <FILE>};
        close FILE;
        my $tree=UTFread($content);
        print Data::Dumper->Dump([$tree]);

$data = UTFwrite ($tree);

Reads an UTF file content into a tied tree:

        use Games::Freelancer::UTF;
        open FILE,"model.cmp";
        binmode FILE;
        my $content = do {local $/; <FILE>};
        close FILE;
        my $tree=UTFread($content);
        #... Do something with $tree ..., for example moving a hardpoint:
        foreach my $entr (grep /\.3db/,keys %{$tree->{"\\"}}) {
                foreach (keys %{$tree->{"\\"}->{$entr}->{Hardpoints}->{"Fixed"}}) {
                        #moves all fixed hardpoints along (0.2,0.2,0.2):
                        $tree->{"\\"}->{$entr}->{Hardpoints}->{"Fixed"}->{$_}->{Position}=pack("f*",map {$_+0.2} unpack("f*",$tree->{"\\"}->{$entr}->{Hardpoints}->{"Fixed"}->{$_}->{Position})); 
                }
        }
        # Now write it down.
        open FILE,">model2.cmp";
        binmode FILE;
        print FILE UTFwrite($tree);
        close FILE;

INTERNAL FUNCTIONS (use with care)

get (SOURCE, OFFSET, LENTGH)

Extracts a string of LENGTH at OFFSET of the SOURCE.

Used for general file reading.

string (OFFSET)

Extracts a string \0 delimited string at OFFSET out of the stringlibrary of the file.

Used for key names

data (OFFSET, LENGTH)

Extracts data from an OFFSET with specific LENGTH out of the datalibrary of the file.

Used for data nodes

UTFreadUTFrek( TREE, NODEID )

Parses a node with NODEID out of the binary TREE, then calls itself with all the childnodes and siblingnodes

packString (STRING)

Saves a STRING to the stringlib and returns an offset. Tests if the string already exists.

Used for writing Nodenames (keys)

packData (DATA)

Saves a string with DATA to the datalib and returns an offset. Tests if the data already exists.

Used for writing nodedata (values)

UTFwriteUTFrek (DATA, NAME, SIBLING)

Writes hashref or a scalar into the tree.

SIBLING is true if there is a next sibling node (different output on nodes without a next one)

Calls itself again for each entry of a hashref. (Writes the children)

UTFreadUTF ( DATA )

Extracts and parses an UTF header from the scalar DATA.

Splits the file in TREE, STRINGLIB and DATALIB according to the header and then calls UTFreadUTFrek on the TREE.

UTFwriteUTF (TREE(HASHREF) )

Calls UTFwriteUTFrek and then return the header, TREE, STRINGLIB and DATALIB

Example tree

This is an example of deparsed tree of a very basic model (.cmp) a friend of mine made, there are however other types of UTF files.

Every UTF file seems to have a root node called \.

        $tree = {
            '\\' => {
                'VMeshLibrary' => {
                    'jc_defender.lod0.vms' => { #Contains the model data
                        'VMeshData' => 'Some Vmeshdata' #Removed by me because you can't see anything useful here and its large.
                    }
                },
                'Cmpnd' => {
                    'Root' => {
                        'File name' => 'jc_defender.3db�', #Links the frist model
                        'Index' => '��������', #Index 0
                        'Object name' => 'Root�' #Name of that model.
                    }
                },
                'jc_defender.3db' => { #A model, a file can have multiple .3db (model) files in them
                    'Hardpoints' => { #The game lets you mount things on this Points
                        'Fixed' => { #Not rotatebale
                            'HpEngine01' => {
                                #These are just packed vectors, can be easily decoded using unpack("f*",this)
                                'Orientation' => '��€?��������������€?��������������€?',
                                'Position' => '����IK�¿×A' #also a vector: unpack("f*",this)
                            },
                        }
                        'Revolute' => { #For weapons mostly, follow the cursor.
                            'HpWeapon01' => {
                                'Axis' => '������€?����', #also a vector: unpack("f*",this)
                                'Max' => '’†>����', #an angle in radians: unpack("f*",this)
                                'Min' => '’†¾����',
                                'Orientation' => 'Z|¿âÐ1¾���€âÐ1>Z|¿�����������€��€?', #also a vector: unpack("f*",this)
                                'Position' => '­L€?Œø®>¿+±À' #also a vector: unpack("f*",this)
                            },
                            'HpWeapon02' => {
                                'Axis' => '������€?����',
                                'Max' => '’†>����',
                                'Min' => '’†¾����',
                                'Orientation' => 'Z|¿âÐ1>����âÐ1¾Z|¿��������������€?', 
                                'Position' => 'r¿Œø®>¿+±À'
                            }
                        }
                    },
                    'MultiLevel' => {
                        'Level0' => {
                            'VMeshPart' => {
                                #This contains a reference to the VMeshdata up there
                                'VMeshRef' => '<���Úý��x��n
��
��¯Ë@Ä�ËÀ“°š?d÷¤¿-¬Aï7Á�`Ž:m$½€ð¢½ä�,A'
                            }
                        }
                    }
                }
            }
        };

I cut out the VMesh part which is just binary data, but it can be read using Games::Freelancer::VMesh

SEE ALSO

Games::Freelancer::VMesh for in UTF files included VMeshData and VMeshRef values.

Games::Freelancer::BINI for another type of file used by Freelancer.

LICENSE

UTF.pm is published under the terms of the MIT license, which basically means "Do with it whatever you want". For more information, see the license.txt file that should be enclosed with this distribution. A copy of the license is (at the time of this writing) also available at http://www.opensource.org/licenses/mit-license.php.

AUTHOR

Marc "Maluku" Sebastian Lucksch perl@marc-s.de

cpanm Games::Freelancer::UTF

perl -MCPAN -e shell
install Games::Freelancer::UTF