Page MenuHomeAnticto

Mutable for Unreal Engine | Technical documentation
Updated 613 Days AgoPublic

Version 22 of 66: You are viewing an older version of this document, as it appeared on May 21 2020, 12:07 PM.

Integrating

Architecture

The system is structured as an Unreal plugin with two additional Unreal modules: one for the editor and one for the runtime. These modules use external libraries for the Mutable runtime, which is a standalone library with no dependencies.

Mutable Unreal Architecture.png (839×797 px, 38 KB)

The next diagram shows the basic folder structure of an Unreal Engine game that uses Mutable. The hierarchy only shows the relevant folders for Mutable source code.

Mutable Unreal Folder Structure.png (548×560 px, 24 KB)

The folder “source” is optional (and by default not included with all licenses) and it contains the source code of the standalone runtime and tools libraries. These are already included in the “lib” folder as precompiled binaries for each supported platform, with the required headers in the “include” folder.

Packaging a project

Mutable will be automatically linked in your game executable. It doesn't require any additional files or dynamic-link libraries. In order to package the data for your Customizable Objects, an additional folders needs to be added in your project settings. The plugin will try to do this for you, but if you need to do it manually you can add "MutableStreamedData" to the list of "Additional non-asset directories to package" in the project settings. You can do this in the DefaultGame.ini file of your project:

+DirectoriesToAlwaysStageAsUFS=(Path="MutableStreamedData") )

or from the project settings in the Unreal Editor:

ProjectSettings.png (1×1 px, 160 KB)

Engine Changes

In most versions of Mutable, some changes in the Unreal Engine codebase are required in order to improve the performance. These are modifications that cannot be implemented through plugins. In some platforms like Windows, Mutable will still work without the engine changes, but it may have lower performance and some editor features may be disabled, like automatic compilation of models when packaging.

The engine modifications are distributed through a patch file that you can find with every release of the plugin. They are also available in a GitHub branch of the official engine repository, as per Epic Game's rules. In order to apply the patch you can use standard tools like TODO.

Compilation warnings

The patch with engine changes includes some helpers to try to detect if any modification is missing. If these warnings are visible it is a sign that some of the changes have not been applied, or may have been overwritten by other changes or merges.

These are for the runtime:

#define ANTICTO_MUTABLE_ENGINE_PATCH_TEXTURESTREAMING
#define ANTICTO_MUTABLE_ENGINE_PATCH_VERTEXBUFFERS
#define ANTICTO_MUTABLE_ENGINE_PATCH_MORPHTARGETVERTEX

These are for the editor:

#define ANTICTO_MUTABLE_ENGINE_PATCH_COOKHOOK

You don't have to manually use these defines but if they are not present, the plugin will show warnings at compile time.

Please note that these are just simple preprocessor defines, so they do not guarantee that the actual changes are present, but capture most error cases.


Using Mutable from C++

Creating an Instance of a CustomizableObject

A CustomizableObjectInstance contains a set of values for the parameters of a CustomizableObject, these values can be used by actors to create in-game meshes and materials.

To create an instance of a CustomizableObject, you need a reference to the Object and a CustomizableObjectInstance. Getting the Object reference can be done either by adding a pointer as UPROPERTY() to the actor’s class and then setting the value from blueprints or by loading it using the asset’s path. When you have both, calling the “SetObject” function tells the instance which object to represent. This way the instance knows what parameters to store and what their default values are.

UCustomizableObject* CustomizableObject = 
        LoadObject<UCustomizableObject>(nullptr, TEXT("/Game/MyCustomizableObject"));

if(CustomizableObject)
{
    CustomizableObjectInstance = NewObject<UCustomizableObjectInstance>();
    CustomizableObjectInstance->SetObject(CustomizableObject);
}

Connecting Instances with Actors

The best way to link an instance to an actor is to use the CustomizableSkeletalComponent, this helper can be attached to the actor’s SkeletalMeshComponent in orter to update and replace the mesh asset with a Mutalbe-generated mesh.

If you create the component as shown below, when the actor is added to a level, it should automaticaly show the customized skeletal mesh.


CSkeletalComponent = 
        NewObject<UCustomizableSkeletalComponent>(UCustomizableSkeletalComponent::StaticClass());

// Set the instance that will be used by the actor
CSkeletalComponent->CustomizableObjectInstance = CustomizableObjectInstance;

// Attach the CustomizableSkeletalComponent to the Actor's SkeletalMeshComponent
CSkeletalComponent->AttachToComponent(GetMesh(), FAttachmentTransformRules::KeepRelativeTransform);

Changing Parameters

Parameters are stored in public arrays within each instance. These parameters can be modified directly, although it is recommended to use the API functions to avoid invalid values. Below you have an example of how to modify parameters and update the instance afterwards.


// Sets the bool value of the parameter "Frackles"
CustomizableMeshFactory->SetBoolParameterSelectedOption(FString("Freckles"), true);

// Sets the int value of the parameter "Shirt" (Enum Type)
CustomizableMeshFactory->SetIntParameterSelectedOption(FString("Shirt"), FString("BasicShirt"));

// Sets the float value of the parameter "Fatness", range from 0 to 1
CustomizableMeshFactory->SetFloatParameterSelectedOption(FString("Fatness"), 0.5f);

// Sets the color value of the parameter "EyeColor"
CustomizableMeshFactory->SetColorParameterSelectedOption(FString("EyeColor"), FLinearColor(FColor::Blue));

// Sets the vector value of the parameter "VParam"
CustomizableMeshFactory->SetVectorParameterSelectedOption(FString("VParam"), FLinearColor(10.0f, 5.f, 5.f));

// Sets the projectors values of the parameter "Tatto"
CustomizableMeshFactory->SetProjectorValue(FString("Tatto"), LocalPosition, Direction, Up, Scale, Angle, 
                                             ECustomizableObjectProjectorType::Planar);

// Update Instance
CSkeletalComponent->UpdateSkeletalMeshAsync();

Using Mutable from Blueprint


Performance tuning

Memory usage

Mutable uses memory mainly in 3 different ways:

  • Streaming of source mutable data : The data required to build instaces of customizable objects needs to be loaded from disk. This data is then processed into a set of skeletal meshes and textures that compose the final object. The source data may be very big if you have many customization options and it is kept in disk. A cache of this data is maintained in memory to speed up object construction. The size of this cache can be controlled by per-platform scalability settings as explained in the Mutable settings section below.
  • Working memory: This is the amount of data used while an object is being constructed, and it is freed while the system is idle. Mutable tries to minimise this amount of data, and its size depends entirely on the kind of objects to be built.
  • Object memory: This is the data used by the actual Unreal resources that compose an instance of a customizable object. Mutable will try to reuse textures that are identical, but other than this, these resources can be treated and measured exactly like any standard Unreal Skeletal Mesh or Texture.

Memory can always be reduced by tuning the assets provided to Mutable. For example, lower polygon counts in meshes, lower resolution in the reference textures or fewer customizable parts. Lower quality LODs can reduce their memory usage by disconnecting less important features like earrings and projections/tattoos. If these strategies have already been used or they are not an option, here are some memory reduction techniques that Mutable offers:

Set a maximum radius of visibility of Customizable Characters

If too much memory is being used because there are just too many different characters using lots of different skeletal meshes and textures, and most of them are too far from the camera to appreciate their differences or are hidden from view, a viable option is to set a maximum visibility radius around the player (or central actor). Only characters inside this radius will be generated and created, and the ones outside it will either be invisible or using the reference skeletal mesh. The API to control this functionality is located in the UCustomizableObjectSystem singleton class that can always be accessed with the following call:

UCustomizableObjectSystem::GetInstance()

The control functions for this functionality are as follows:

To enable/disable the Customization visibility radius and set the actual radius use:

void EnableOnlyUpdateCloseCustomizableObjects(float CloseDist);
void DisableOnlyUpdateCloseCustomizableObjects();

By default characters outside the radius will receive a null skeletal mesh and will be invisible. If you prefer to replace them with a common generic skeletal mesh, use the next two functions. The generic mesh is the reference skeletal mesh defined in the properties of the base Customizable Object of the character.

void SetReplaceDiscardedWithReferenceMeshEnabled(bool bIsEnabled);
bool IsReplaceDiscardedWithReferenceMeshEnabled() const;

The visibility radius is defined around the central actor of the scene. By default it is the first player controller actor, and can be changed with the following call:

void DefineCentralActor(const AActor* centralActor)
Limit the number of instances with all LODs

Sometimes most characters are too far away from the camera to effectively use all levels of detail, especially the higher quality ones like LOD0 and LOD1. A lot of memory is wasted in detail that most of the time will not be seen until the player gets close to those characters. In this case in makes sense to limit how many high-quality LODs Mutable generates for these far-away characters.

Mutable dynamically keeps track of every instance that is live in the game and how far they are from the player or central actor so it always knows which instances are more important and deserve having the most detailed LODs. There are some scalability/console variables (b.NumGeneratedInstancesLimit*) that control how many instances will have full LODs (LOD0 + LOD1 + LOD2), how may will have only two (LOD1 + LOD2) and finally how many will only have the low resolution LOD2 with minimal memory consumption. See the Scalability settings section for more information about these variables.

Only the closest instances to the player will get the full LODs. The next closest will get the two lower detail LODs and finally the rest will only get LOD2. If there are more instances than the sum of the values of b.NumGeneratedInstancesLimit*, the remaining furthest instances will not be generated and will be invisible.

At every tick Mutable will recalculate which instances deserve each level of detail and automatically issue and enqueue updates to discard LODs for the instances that have gone too far from the camera and updates to generate LODs for the ones that have got closer to the camera.

Although these updates are processed by prioritizing the closest instances to the camera or central actor, if the characters frequently move relative to it they can generate a lot of LOD change updates and quickly fill-up the queue. If the CPU is busy it may take a few moments to process the queue and new updates can take a few seconds to become visible. In this case, LOD updates can make customization updates (ex: putting on a jacket) lag for a while. This can be prevented with the following functions accessed through the UCustomizableObjectSystem:

void SetNumberOfPriorityUpdateInstances(int32 NumPriorityUpdateInstances);
int32 GetNumberOfPriorityUpdateInstances() const;

They can be used to set how many of the nearest instances to the player will have updates with priority over LOD changes, and thus update much faster. This is especially useful for multiplayer games with teams, where the player wants to see customization updates faster for teammates.

Reuse the memory resources of mutable textures

By default, when a character changes a color in a texture or it moves a projector, the old texture is discarded and a new one is generated. The old texture's resources will not be release until the next Garbage Collection (GC) UE4 issues, so texture memory usage will go up until it happens. There are situations when a lot of textures will be created and destroyed in a short amount of time, for example in a customization lobby. The player moving a slider to control the color of a customizable object or moving a projector over a character's body will trigger a quick surge in texture usage until the next GC.

Immediately release mutable textures after instance destruction

Mutable Settings

General settings

These are settings that affect mutable in general ways. They can be defined in the DefaultMutable.ini file of your project. See the plugin Config folder for an example.

  • Features section
bExtraBoneInfluencesEnabledIf this is enabled, CustomizableObjects will be allowed to use 8 bone influences per vertex. Otherwise only the default 4 will be used. This is used at object compile time.
Scalability settings

These settings can be modified by the standard Scalability ini file hierarchy in Unreal, or with Console Variables or game code. These are the variables for each section:

  • ViewDistanceQuality section
b.MutableStreamingMemoryIf different than 0, limit the amount of memory (in KB) to use to cache streaming data when building characters. 0 means no limit, -1 means use default (40Mb in PC and 20Mb in consoles).
b.NumGeneratedInstancesLimitIf different than 0, limit the number of mutable instances with full LODs to have at a given time.
b.NumGeneratedInstancesLimitLOD1If different than 0, limit the number of mutable instances with LOD 1 to have at a given time.
b.NumGeneratedInstancesLimitLOD2If different than 0, limit the number of mutable instances with LOD 2 to have at a given time.
b.DistanceForFixedLOD2If NumGeneratedInstancesLimit is different than 0, sets the distance at which the system will fix the LOD of an instance to the lowest res one (LOD2) to prevent unnecessary LOD changes and memory

Statistics widget

A good starting point to see the performance of Mutable in your game, is to use the provided statistics UMG widget. You can find it in the plugin Content folder with the name UI/MutableStats.

imatge.png (535×1 px, 469 KB)

When using these statistics in the editor, the numbers will also capture all the instances built in all open levels, editors, and even content browser thumbnails, if enabled.

If your project is not using UMG you can access these statistics and log them or display them from the central singleton object UCustomizableObjectSystem. These are some of the relevant methods:


	// Get the singleton object. It will be created if it doesn't exist yet.
	UFUNCTION(BlueprintCallable, BlueprintPure, Category = Status)
	static UCustomizableObjectSystem* GetInstance();

	// Find out if the performance engine patches have been applied
	UFUNCTION(BlueprintCallable, Category = Status)
	bool AreEnginePatchesPresent() const;

	// Find out the version of the plugin
	UFUNCTION(BlueprintCallable, Category = Status)
	FString GetPluginVersion() const;

	// Get the number of instances built and alive.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetNumInstances() const;

	// Get the number of instances waiting to be updated.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetNumPendingInstances() const;

	// Get the total number of instances including built and not built.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetTotalInstances() const;

	// Get the amount of memory in use for textures generated by mutable.
	UFUNCTION(BlueprintCallable, Category = Status)
	int64 GetTextureMemoryUsed() const;

	// Return the average build/update time of an instance in ms.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetAverageBuildTime() const;

Customizable Object States

When an application uses complex Customizable Objects with many parameters, updating the instances may be a costly process that takes many milliseconds. However there are some usage scenarios for games that require these updates to be interactive, and big delays are not acceptable. In order to optimize your data to solve these problem, Mutable has the concept of States.

A State represents a specific use case of the Customizable Object in your game. For example, at some point during character creation you may want to let the player customize the face and hair of a character. During this stage, you show a close up camera of the character head and display a user interface for the related parameters: hair colour, nose size, hair style, etc. During this stage, you will not be modifying other parameters, like t-shirt colour, torso tattoos, etc. In order for Mutable to provide maximum performance, you can create a State in your Customizable Object, indicating that the subset of parameters that you will modify in these stage, and the system will generate an optimised version of the data that updates faster.

States also give you more options in order to optimise the instance construction time. For instance, in the previous example of the head customization stage, your game may have more graphic resources available because instead of being inside a level, you are in a smaller lobby scene. This means that you can afford to temporarily use more memory for your character. For each individual State Mutable gives you these optimisation options:

  • List of parameters that may be modified.
  • Option to avoid texture compression for textures that may change in this state.
  • Option to generate only LOD 0 of the object.

States are created in any Base Object Node. If no state is created, a default state with no optimized parameters and sensible optimisation options will be automatically created.

A normal game will want to create a "default" state to create customizable objects to be used in-game (no optimized parameters), and then several states to create and update objects in the different in-game customization scenarios.

The editor user interface for preview instances has the object state selection combobox at the top.


Data-driven User Interfaces

Last Author
alexei
Last Edited
May 21 2020, 12:07 PM

Event Timeline

jordi edited the content of this document. (Show Details)
alexei edited the content of this document. (Show Details)
alexei published a new version of this document.
alexei published a new version of this document.May 21 2020, 12:51 PM
jordi changed the visibility from "All Users" to "Public (No Login Required)".May 26 2020, 12:04 PM
pere published a new version of this document.
alexei edited the content of this document. (Show Details)
alexei published a new version of this document.
pere published a new version of this document.
gerard published a new version of this document.Oct 8 2021, 4:07 PM
gerard edited the content of this document. (Show Details)