Chapter 24. Data Service

Table of Contents

24.1. .update_description structure
24.2. Data Objects
24.3. Download/upload speed

To enable even users with a slow internet connection to start playing as soon as possible, it is important to avoid making them download many megabytes of data needed for the presentation before they can connect. The first way is to distribute all the data on a CD. This has many drawbacks, in particular problems with the distribution to the potentially "random" locations in the world. The second way is to enable them downloading the data dynamically, when they need them, during playing the game.

The class Massiv::Core::DataManager implements the functionality of the data management. It provides methods which can be used to work with data such as textures, sound data, maps, texts and other textual and binary data.

A global instance of the class Massiv::Core::DataManager is created on each Massiv node and plays a different role according to the hosting node type (Master data service, Server or Client).

On a specific node the data is stored in a subtree of one separate directory in files. This directory is called the data directory. The files can be located in subdirectories of the data directory and are called the data files. A data file name from the point of view of the data manager is not just a plain file name, but it is a relative path to the data directory.

Besides the data files, there are also two files in the data directory that describe the whole data system (in a textual way). These files are called .description and .filelist. On the master data service node, there is even another directory called the source data directory and a textual description of the source data in the file .update_description. The structure of this file will be described below. It is the only file that must be changed manually by the administrator; the other two mentioned files are updated automatically.

An administrator of the Massiv data server modifies the data in the source data directory (for example adds new data files, modifies the existing ones, modifies the source data description file, etc.). After that he should update the changes from the source data directory into the data directory. This update is processed during the master data service startup, or when the method DataManager::request_master_data_update() is called. After data update information about update is propagated to servers and clients they will update their local data.

The administrator needs to edit file .update_description which describes the source data system. See section Structure of .update_description file.

Each data file is versioned (i.e. contains information about its version) as well as the whole data system. This is essential for the data manager to be able to find out if some data has changed (new texture added, old texture modified, etc.). Clients and servers compare their local version information with the master data service to decide if it is necessary to download some new data.

24.1. .update_description structure

To be able to create and modify .update_description file it is necessary to know the structure of this file. Let's look at an example to understand.

###
# Images
node image_default
    type necessary_client
    file "image/default.png"

    node image_font_console
        type necessary_client
        file "image/font.png"
    endnode

    node image_background
        type client
        file "image/background.jpg"
    endnode
endnode # image_default

node model_default
    type necessary_client
    file "model/default.md2"

    node monster
        type client
        file "model/monster.md2"

        node player
            type client
            file "model/player.md2"
        endnode
    endnode # monster
endnode # model_default


###
# Data objects
dataobject image/default = /image_default
dataobject image/font_console = /image_default/image_font_console
dataobject image/background = /image_default/image_background
dataobject model/default = /model_default
dataobject model/monster = /model_default/monster
dataobject model/monster_dark = /model_default/monster
dataobject model/player = /model_default/monster/player

The structure consists of two parts - data nodes and data objects. The data nodes create a tree. Each data node is associated with exactly one data file.

A data node ancestor in the tree can substitute its descendant data nodes. This mechanism enables to use more general data temporarily until the specific data is actually downloaded. For example, in the tree above, image/default.png would be used instead of image/background.jpg until the later would be downloaded by the relevant client.

Data objects are mapped to data nodes, usually leaf data nodes (BUG TO SE MI TEDA NEZDA!). If a programmer wants to obtain a data object, the data manager algorithm searches for the first usable data node on a path from the corresponding data node to the root data node. It is guaranteed that the root data node is always present in the data system of each client (i.e. it never needs to be downloaded).

The consequence of the described mechanism is that the player can see (for example) "walking black-and-white boxes" instead of the real models of figures until the they are actually are available. This is a big advantage, because any player can start playing very quickly. However, if he doesn't like the default models and images, he can still wait until all the data is available.

Let's describe structure of a file. Comments are parts of lines starting with the # character and ending at the end of the line. A data node declaration begins with the node keyword and ends with keyword endnode.

Between these keywords there are located properties of the relevant data node as well as other (submerged) nodes. Submerging of the data nodes declarations is the way how to create the tree.

Each data node has several mandatory properties:

  • name stands for a data node name; it must be located on the same line after the node keyword.

  • file specifies a name of the data file associated with the relevant data node; only one data file is allowed in one specifiec data node.

There exist also some optional properties:

  • type specifies the data node type; the possible values are necessary_client, necessary_server, client, server; there can be several types for one data node; the purpose of this property is to declare which data is client or server specific and which is necessary for the simulation.

  • obsolete declares the relevant data node as obsolete; it cannot be used in the simulation any more.

The second part of the .update_description file consists of the data objects. A data object definition begins with the dataobject keyword followed by the data object name (the / character is permitted), = and the data node path to the existing data node. Don't forget to use space between all of the parts.

[Note]Note

Two data objects can be mapped to the same data nodes. For example models of two characters are identical, but textures will differ.

[Note]Note

Data files are unique for data nodes. Different data nodes must own different data files.