Integrating

Architecture

The system is structured as an Unreal plugin with two additional Unreal modules: one for the editor and one for the runtime. These modules use external libraries for the Mutable runtime, which is a standalone library with no dependencies.

The next diagram shows the basic folder structure of an Unreal Engine game that uses Mutable. The hierarchy only shows the relevant folders for Mutable source code.

The folder “source” is optional (and by default not included with all licenses) and it contains the source code of the standalone runtime and tools libraries. These are already included in the “lib” folder as precompiled binaries for each supported platform, with the required headers in the “include” folder.

Packaging a project

Mutable will be automatically linked in your game executable. It doesn't require any additional files or dynamic-link libraries. In order to package the data for your Customizable Objects, an additional folders needs to be added in your project settings. The plugin will try to do this for you, but if you need to do it manually you can add "MutableStreamedData" to the list of "Additional non-asset directories to package" in the project settings. You can do this in the DefaultGame.ini file of your project:

+DirectoriesToAlwaysStageAsUFS=(Path="MutableStreamedData") )

or from the project settings in the Unreal Editor:

Engine Changes

In most versions of Mutable, some changes in the Unreal Engine codebase are required in order to improve the performance. These are modifications that cannot be implemented through plugins. In some platforms like Windows, Mutable will still work without the engine changes, but it may have lower performance and some editor features may be disabled, like automatic compilation of models when packaging.

The engine modifications are distributed through a patch file that you can find with every release of the plugin. In order to apply the patch you can use the Git tool. Also, the engine with the modifications already applied is also available in a GitHub branch of the engine repository.

Compilation warnings

The patch with engine changes includes some helpers to try to detect if any modification is missing. If these warnings are visible it is a sign that some of the changes have not been applied, or may have been overwritten by other changes or merges.

These are for the runtime:

#define ANTICTO_MUTABLE_ENGINE_PATCH_TEXTURESTREAMING
#define ANTICTO_MUTABLE_ENGINE_PATCH_VERTEXBUFFERS
#define ANTICTO_MUTABLE_ENGINE_PATCH_MORPHTARGETVERTEX

These are for the editor:

#define ANTICTO_MUTABLE_ENGINE_PATCH_COOKHOOK

It's not mandatory to manually use these defines but if they are not present, the plugin will show warnings at compile time.

Using Mutable from C++

Creating an Instance of a CustomizableObject

A CustomizableObjectInstance contains a set of values for the parameters of a CustomizableObject, these values can be used by actors to create in-game meshes and materials.

To create an instance of a CustomizableObject, we need a reference to the Object and a CustomizableObjectInstance. Getting the Object reference can be done either by adding a pointer as UPROPERTY() to the actor’s class and then setting the value from blueprints or by loading it using the asset’s path. Having both, then calling the “SetObject” function tells the instance which object to represent. This way the instance knows what parameters to store and what their default values are.

UCustomizableObject* CustomizableObject = 
        LoadObject<UCustomizableObject>(nullptr, TEXT("/Game/MyCustomizableObject"));

if(CustomizableObject)
{
    CustomizableObjectInstance = NewObject<UCustomizableObjectInstance>();
    CustomizableObjectInstance->SetObject(CustomizableObject);
}

Connecting Instances with Actors

The best way to link an instance to an actor is to use the CustomizableSkeletalComponent, which can be attached to the actor’s SkeletalMeshComponent in order to update and replace the mesh asset with a Mutable-generated mesh.

Creating the component as shown below, when the actor is added to a level, it will automatically show the customized skeletal mesh.

CSkeletalComponent = 
        NewObject<UCustomizableSkeletalComponent>(UCustomizableSkeletalComponent::StaticClass());

// Set the instance that will be used by the actor
CSkeletalComponent->CustomizableObjectInstance = CustomizableObjectInstance;

// Attach the CustomizableSkeletalComponent to the Actor's SkeletalMeshComponent
CSkeletalComponent->AttachToComponent(GetMesh(), FAttachmentTransformRules::KeepRelativeTransform);

Changing Parameters

Parameters are stored in public arrays within each instance. These parameters can be modified directly, although it is recommended to use the API functions to avoid invalid values. Below is an example of how to modify different types of parameters.

// Sets the bool value of the parameter "Frackles"
CustomizableObjectInstance->SetBoolParameterSelectedOption(FString("Freckles"), true);

// Sets the int value of the parameter "Shirt" (Enum Type)
CustomizableObjectInstance->SetIntParameterSelectedOption(FString("Shirt"), FString("BasicShirt"));

// Sets the float value of the parameter "Fatness", range from 0 to 1
CustomizableObjectInstance->SetFloatParameterSelectedOption(FString("Fatness"), 0.5f);

// Sets the color value of the parameter "EyeColor"
CustomizableObjectInstance->SetColorParameterSelectedOption(FString("EyeColor"), 
                                                             FLinearColor(FColor::Blue));

// Sets the vector value of the parameter "VParam"
CustomizableObjectInstance->SetVectorParameterSelectedOption(FString("VParam"), 
                                                             FLinearColor(120.f, 50.f, 180.f));

// Sets the projectors values of the parameter "Tatto"
CustomizableObjectInstance->SetProjectorValue(FString("Tatto"), LocalPosition, Direction, Up, Scale, Angle, 
                                               ECustomizableObjectProjectorType::Planar);

Changing States

As explained in the States page, Mutable has the concept of States that enable certain optimizations depending on the use case. Given an Customizable Object Instance, its State can be queried and changed using the API functions:

// Get the current State
CustomizableObjectInstance->GetCurrentState(FString("InGame"));

// Set the current State. Requires the Customizable Object Instance to be updated
FString State = CustomizableObjectInstance->SetCurrentState();

Given a Customizable Object, we can query the available States and the parameters in each State:

// Get the number of States
int32 Count = CustomizableObject->GetStateCount()

// Get the State Name of the given State index
FString Name = CustomizableObject->GetStateName(1);

// Get the number of parameters of a given State
int32 ParameterCount = CustomizableObject->GetStateParameterCount(FString("InGame"));

// Get the name of parameter index in a given State
FString ParameterName = CustomizableObject->GetStateParameterName(FString("InGame"), 1);

Updating Instances

Changing parameters or States does not update the Instance automatically. To apply these changes, the instance must be updated by calling the UpdateSkeletalMeshAsync() method, within the CustomizableSkeletalComponent class. Doing so will replace the SkeletalMesh of all actors using the same CustomizableObjectInstance with the generated from those changes.

// Update Instances
CSkeletalComponent->UpdateSkeletalMeshAsync();

Here is a diagram of what goes on "under the hood" when an update is taking place. Most of the CPU work is done in Mutable's own thread, and for most projects most of the time is spent waiting for assets to stream from disk.

If your project's CPU threads or disk are highly saturated all the time, it may slow down the updates of Mutable characters because of resource contention.

Instance Updated Delegate

Sometimes, you might want to run a specific method after the mesh has been updated, either to trigger an animation, make the mesh visible, etc.
To make it easy, Mutable provides a delegate where callbacks can be registered. This delegate will broadcast the completion of the skeletal mesh update to the registered callbacks.

// Bind "this" UObject's "OnCustomizableSkeletalUpdated" method as a callback for the delegate 
CSkeletalComponent->UpdatedDelegate.BindUObject(this, &MyCustomCharacter::OnCustomizableSkeletaUpdated);

---

// Unbind callbacks 
CSkeletalComponent->UpdatedDelegate.Unbind();

Using Mutable from Blueprint

Creating a Customizable Character

1. Create a new blueprint with "character" as parent class and open it. Alternatively, you can use a class/BP that has a skeletal mesh component.

2. In the blueprint editor, select the Mesh component and add a CustomizableSkeletal component as a child.

3. Having selected the newly created component, go to the details panel and chose the instance to be used.

The reference mesh or the custom mesh, if generated, will show up in the viewport.

Changing Parameters

Parameters are stored by instances and can be accessed or modified using the API nodes. Here are examples on how to set parameter values based on their type:

• Boolean Parameter

• Int Parameter:

It's important to make sure that the desired option actually exists within the instance. Search existing parameters using the FindParameter node and then get the available option with GetIntParameterAvailableOption. Both nodes have the CustomizableObject as target, which can be accessed through the CustomizableObjectInstance.

• Float Parameter

• Color Parameter

• Projector Parameter

Update Instances

To apply recent changes in parameters, the instance needs to be updated. This can be achieved by adding an UpdateSkeletalMeshAsync node after one or more changes.

Updated Delegate

Events can be registered to this delegate. The broadcasting will take place after the completion of the skeletal mesh update.

Bind Event	Unbind Event

Parameter Information

Sometimes, additional information like the amount of parameters within an instance, their type or the name of an int parameter's option might come in handy. This information is kept in the source CustomizableObject, accessible through the instance, and can be retrieved using the following nodes:

Get Parameter Count	Get Parameter Name
Get Parameter Type by Name	Find Parameter
* Get Int Parameter Num Options	* Get Int Parameter Available Option

Changing States

States can also be queried and changed using the node API:

• Set Current State
  

• Get Current State
  

State Information

Information about the number and name of States a CustomizableObject has and the number and names of parameters in a State can be useful. This information is stored per CustomizableObject, accessible through a CustomizableObjectInstance and can be retrieved using the following nodes:

Get State Count	Get State Name
Get State Parameter Count	Get State Parameter Name

Performance tuning

Memory usage

Mutable uses memory mainly in 3 different ways:

• Streaming of source mutable data : The data required to build instances of customizable objects needs to be loaded from disk. This data is then processed into a set of skeletal meshes and textures that compose the final object. The source data may be very big if you have many customization options and it is kept in disk. A cache of this data is maintained in memory to speed up object construction. The size of this cache can be controlled by per-platform scalability settings as explained in the Mutable settings section below.

• Working memory: This is the amount of data used while an object is being constructed, and it is freed while the system is idle. Mutable tries to minimize this amount of data, and its size depends entirely on the kind of objects to be built.

• Object memory: This is the data used by the actual Unreal resources that compose an instance of a customizable object. Mutable will try to reuse textures that are identical, but other than this, these resources can be treated and measured exactly like any standard Unreal Skeletal Mesh or Texture.

Memory can always be reduced by tuning the assets provided to Mutable. For example, lower polygon counts in meshes, lower resolution in the reference textures or fewer customizable parts. Lower quality LODs can reduce their memory usage by disconnecting less important features like earrings and projections/tattoos. If these strategies have already been used or they are not an option, here are some memory reduction techniques that Mutable offers:

Set a maximum radius of visibility of Customizable Characters

If too much memory is being used because there are just too many different characters using lots of different skeletal meshes and textures, and most of them are too far from the camera to appreciate their differences or are hidden from view, a viable option is to set a maximum visibility radius arround one or more view centers, for example the player. Only characters inside this radius will be generated and created, and the ones outside it will either be invisible or using the reference skeletal mesh. The API to control this functionality is located in the UCustomizableInstanceLODManagement class, accessible through the UCustomizableObjectSystem singleton with the following call:

auto LODManagmentBase = UCustomizableObjectSystem::GetInstance()->GetInstanceLODManagement();

Cast the LODManagmentBase to a UCustomizableInstanceLODManagement to access the LOD system provided by Mutable.

if(auto MutableLODManagment = Cast<UCustomizableInstanceLODManagement>( LODManagmentBase)
{ }

The control functions for this functionality are as follows:

To enable/disable the Customization visibility radius and set the actual radius use:

void EnableOnlyUpdateCloseCustomizableObjects(float CloseDist);
void DisableOnlyUpdateCloseCustomizableObjects();

By default characters outside the radius will receive a null skeletal mesh and will be invisible. If you prefer to replace them with a common generic skeletal mesh, use the next two functions. The generic mesh is the reference skeletal mesh defined in the properties of the base Customizable Object of the character.

void SetReplaceDiscardedWithReferenceMeshEnabled(bool bIsEnabled);
bool IsReplaceDiscardedWithReferenceMeshEnabled() const;

The visibility radius is defined around the view center/s of the scene. By default it is the first player controller actor, and can be changed with the following calls:

void AddViewCenter(const AActor* const InViewCenter);
void RemoveViewCenter(const AActor* const InViewCenter);
void ClearAllViewCenters();

Sometimes most characters are too far away from the camera to effectively use all levels of detail, especially the higher quality ones like LOD0 and LOD1. A lot of memory is wasted in detail that most of the time will not be seen until the player gets close to those characters. In this case it makes sense to limit how many high-quality LODs Mutable generates for these far-away characters. To adjust the number of generated LODs, Mutable provides different two methods.

Automatic adjustment of generated LODs per instance

Once the maximum display radius has been defined, an automatic system will come into operation to limit the LODs present in each instance according to the screen size of the meshes.

The system works similarly to the rendering system. Mutable dynamically tracks all active instances in the game, if they are within the maximum radius of visibility it estimates the level of detail they use and, if necessary, issue a request to discard or add LODs. Using this system minimizes the memory usage used by instances without limiting the number of instances with all LODs.

In cases where the density of unique instances is very high and the character moves frequently, there may be cases where the number of requests to generate instances with all LODs accumulates in the tasks queue. Since requests are processed based on the distance to the view centers, in these circumstances, it may take a few seconds to process updates for far-away instances.

Limit the number of instances with all LODs

Another method of control the number of LODs generated in order to reduce the memory being used is to limit how many instances will have full LODS (LOD0 + LOD1 + LOD2), how many will have only two (LOD1 + LOD2) and finally how many will only have the low resolution LOD2 with minimal memory consumption. There are some scalability/console variables (b.NumGeneratedInstancesLimit*) to customize this option. See the Scalability settings section for more information about these variables.

Similar to the automatic adjustment of generated LODs, Mutable will recalculate in each frame which level of detail belongs to each instance. In this case however the calculation will be based on the distance to the view centers and a the limit on the number of instances with full LODs. Only the closest instances to the view centers will get the full LODs. The next closest will get the two lower detail LODs and finally the rest will only get LOD2. If there are more instances than the sum of the values of b.NumGeneratedInstancesLimit*, the remaining furthest instances will not be generated and will be either invisible or replaced by the reference skeletal mesh.

At every tick Mutable will recalculate which instances deserve each level of detail and automatically issue and enqueue updates to discard LODs for the instances that have gone too far from the camera and updates to generate LODs for the ones that have got closer to the camera.

Although these updates are processed by prioritizing the closest instances to the view centers, if the characters frequently move relative to it they can generate a lot of LOD change updates and quickly fill-up the queue. If the CPU is busy it may take a few moments to process the queue and new updates can take a few seconds to become visible. In this case, LOD updates can make customization updates (ex: putting on a jacket) lag for a while. This can be prevented with the following functions accessed through the UCustomizableObjectSystem:

void SetNumberOfPriorityUpdateInstances(int32 NumPriorityUpdateInstances);
int32 GetNumberOfPriorityUpdateInstances() const;

They can be used to set how many of the nearest instances to the player will have updates with priority over LOD changes, and thus update much faster. This is especially useful for multiplayer games with teams, where the player wants to see customization updates faster for teammates.

Reuse the memory resources of mutable textures during realtime parameter editing

By default, when a character changes a color in a texture or it moves a projector, the old texture is discarded and a new one is generated. The old texture's resources will not be released until the next Garbage Collection (GC) UE4 issues, so texture memory usage will go up until it happens. There are situations when a lot of textures will be created and destroyed in a short amount of time, for example in a customization lobby. The player moving a slider to control the color of a customizable object or moving a projector over a character's body will trigger a quick surge in texture usage until the next GC.

Mutable offers an option to reuse textures for such situations on a per-instance basis. It is not enabled by default because there are certain precautions that must be taken for it to be safe imposed by UE4's texture system. It is extremely important not to keep references to the textures of the instances that have texture reuse enabled, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable.

1. First of all, make sure that the Customizable Object Instance is in a Mutable State where only the Mutable parameters of the currently edited object are being updated. See the section on "Customizable Object States" for more information. For instance, if a color slider is being edited, set the Mutable parameter that controls the color in the list of parameters of the Mutable State. Only use texture reuse in a customization lobby State, never in a gameplay State. In addition, make sure that this State disables texture compression and streaming. UE4 doesn't support the update of compressed and/or streaming textures.

Something like the following must be called before enabling texture reuse:

CustomizableObjectInstance->SetCurrentState(FString("LobbyStateName_WhereOnlyOneParameterIsEdited"));

2. Activate texture reuse for the lobby instance before any parameter manipulation:

CustomizableObjectInstance->SetReuseInstanceTextures(true);

3. Remember to disable texture reuse after the parameter editing session:

CustomizableObjectInstance->SetReuseInstanceTextures(false);

Immediately release mutable textures after instance destruction

As discussed in the previous section, by default textures created by Mutable cannot be released until a UE4 Garbage collection (GC) happens. In scenes where a lot of instances are being continuously created and destroyed or updated, there can be a spike in memory usage between GC events.

Mutable has an option to enable the immediate release of texture memory upon texture/instance destruction without having to wait for GC. Like the previous section, it is not enabled by default because the programmer has to take some precautions in order to make it safe, but this time the requirements are less strict.

Textures can be compressed and streamed, and the instances do not have to be set in a special State. The only thing have in mind is not to keep references to the textures of the instances, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable.

If some texture or skeletal mesh generated by Mutable needs to be used in something that is not the character itself and attached to the Customizable Skeletal Component, for example in the UI or in any other copy, please make a full deep copy of the UObjects and resources that has no reference to the ones generated by Mutable.

To enable the immediate release of texture resources call on the UCustomizableObjectSystem:

void SetReleaseMutableTexturesImmediately(bool bReleaseTextures)

GPU Usage

Mesh Morph Baking

In Unreal Engine, when a mesh has a morph (for example a fat/slim morph that lets the player select the character's fitness), it is computed for every vertex in the GPU every frame, which can be expensive. With Mutable, you can pre-compute the morphs in the CPU once and store the result in the vertices, which means the GPU will not have to compute them any more.

Mutable Settings

General settings

These are settings that affect mutable in general ways. They can be defined in the DefaultMutable.ini file of your project. See the plugin Config folder for an example.

• Features section

Scalability settings

These settings can be modified by the standard Scalability ini file hierarchy in Unreal, or with Console Variables or game code. These are the variables for each section:

• ViewDistanceQuality section

Statistics widget

A good starting point to see the performance of Mutable in your game, is to use the provided statistics UMG widget. You can find it in the plugin Content folder with the name UI/MutableStats.

If your project is not using UMG you can access these statistics and log them or display them from the central singleton object UCustomizableObjectSystem. These are some of the relevant methods:

	// Get the singleton object. It will be created if it doesn't exist yet.
	UFUNCTION(BlueprintCallable, BlueprintPure, Category = Status)
	static UCustomizableObjectSystem* GetInstance();

	// Find out if the performance engine patches have been applied
	UFUNCTION(BlueprintCallable, Category = Status)
	bool AreEnginePatchesPresent() const;

	// Find out the version of the plugin
	UFUNCTION(BlueprintCallable, Category = Status)
	FString GetPluginVersion() const;

	// Get the number of instances built and alive.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetNumInstances() const;

	// Get the number of instances waiting to be updated.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetNumPendingInstances() const;

	// Get the total number of instances including built and not built.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetTotalInstances() const;

	// Get the amount of memory in use for textures generated by mutable.
	UFUNCTION(BlueprintCallable, Category = Status)
	int64 GetTextureMemoryUsed() const;

	// Return the average build/update time of an instance in ms.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetAverageBuildTime() const;

Troubleshooting an Object

There are 3 ways of looking in detail at objects generated by Mutable to see the resources they use, or try to spot why a desired effect is not achieved.

• You can bake an instance into standard Unreal assets.
• You can open the preview mesh, material and textures by double-clicking the Customizable Instance SkeletalMesh in the Preview Instance Properties panel.
• You can use the Performance Report to see at a glance the parameter combinations that require the most texture memory, the ones that have the most triangles and the parameters or states that are the slowest to update when changed.

The second way is quicker, but since the dynamic preview assets don't exist in the Content Browser and disk, some editors may no behave as expected. In that case, we recommend using the baking option.

States

When using States, it's advised to split character edition into multiple coherent States as more parameters are added to the model. Parameters usually modified together should have their own State. For example, parameters such as facial features should be in a separate State from clothing. It's uncommon to be modifying the face and the clothes at the same time. This division into States with less parameters frees resources and will allow the parameter updates to be faster.

Merging of physical assets

In some situations you require the Mutable Instances to not only have the physics assets located on the reference object but to also include the physics assets located on the meshes that the Customizable Object uses in its graph. This is why this option exists.

Some cases where this might be useful is when, for example, you want to work with simulated cloth in your CO, or when you require the instances to have an accurate physics representation based on the configuration of them. An example of the later is having a character that does have the option to wield a shield and you want that shield, if enabled, to have its own physics assets present on the Mutable Instance.

In order to enable this behavior you need to call the next method from UCustomizableObjectInstance:

SetReplacePhysicsAssets(true)

When this option is not enabled the generated instances do not take in consideration the physics assets that are included with the meshes that form the CO.

Tips

• To refresh all material related nodes on a tree more efficiently, you can right click on a group and refresh all children: