Table of Contents
The Massiv provides facilities for automatic load balancing. It is implemented by the Balancer class which monitors data flow among migration groups and nodes registered into the simulation, performs transparent object migrations in order to decrease network or CPU load, swaps out not-recently-used objects and migrates objects to less loaded nodes in order to keep uniform resource loads. To make the balancing possible transparent object migrations without application's assistance had to be implemented. This is a key feature of the object model and probably the most important object model design goal. As a result objects can migrate not only due to application logic decisions. However the migrations triggered by the load balancing subsystems are completely transparent to the application. If you are not interested in how the load balancer works you can safely ignore this appendix.
There are three different optimization techniques. They run independently and are often triggered when there is enough information to perform the optimization (for example data flow statistics between migration groups and remote nodes is available):
Local data flow optimization
The optimizer keeps track of data flow generated by local migration groups and nodes that consume the generated data. Whenever an object migrates, local SRPC call is made or a replication update is sent, the data amount, that would have to be transmitted over the network, is accounted to a migration group responsible for the production of the data flow. For example if an object A issues a RPC call to object B, RPCObject is created and migrated to B. The migration cost of the RPCObject is charged to the object A as the migration was initiated by that object.
![[Note]](images/note.png)
Note When data flow needs be accounted its producer must be identified first. To simplify the process the data is always treated as if it was produced by the last object upcalled from the Core. Direct local calls made by the application since the last upcall are ignored. Although the producer may not be identified reliably it will always be an object from the "correct" migration group unless local weak pointers are used to call other objects directly. This semantics is similar to CORBA _this().
Migration groups that produce data flow consumed by "single major" nodes (it is not directed to other nodes or the consumed portion of the data is negligible with respect to the major nodes) will be migrated to that nodes. This will reduce data flow between the local node and the major nodes.
The statistics are gathered locally on a per-object basis. When enough data is collected the statistics are aggregated by migration groups and major consumer nodes are identified. Migration groups are then migrated to the major consumer nodes unless it is cheaper to leave them on the local node.
Note If we were running this kind of optimization only all objects would finally end up on a single node.
Global resource optimization
Unlike the local data flow optimization whose aim is to decrease network traffic locally, from a local node to remote nodes, the global resource optimization tries to optimize resource load in the global. The optimizer keeps track of resource loads on all nodes and migrates objects among the nodes to ensure uniform loads.
Each node "ranks" its resource load which is in turn distributed to all other nodes. The most loaded node periodically elects a subset of its local objects and migrates them to the least loaded node. Since the number of objects to be migrated must be sufficiently small (in order to converge), the optimization can be a lengthy process and it may take a while to find an equilibrium.
Note Objects are elected by a random process which implies that the sets need not be "compact" (with respect to some data flow criterion). However if the optimization is combined with the local data flow optimization, the other optimizer will push "companion" objects to that node soon.
Local memory optimization
Migration groups that were not touched for some time are transparently swapped out and read back into memory when needed, upon a Core request. Swap-out operation is technically equal to a "migration to a swap file", which means that swapped-out objects are not local.
Note This is completely different from memory swapping implemented by various operating systems. Our approach ensures that idle objects do not pollute Core tables and that "working object set" is kept sufficiently small. This allows to build large distributed worlds as most of them would probably be idle.
The swapping always operates on a per migration group basis. Objects are read into memory when their scheduled events should be dispatched or an object is delivered to them. As a result RPC calls wake up the objects too.
If load balancing must be disabled for some reason one can do so via the appropriate settings in the registry (see Section 27.4, “Balancer”). An another option is to mark classes as no-balance (see Section 10.9.3, “Class Attributes”) which prevents the Balancer from performing optimization on instances of such classes. In the most cases no_balance IDL attribute is used as a hint to the Balancer that says that instances of the class are transient and will be collected by the Garbage Collector soon thus no optimization is required.