Page MenuHomeAnticto

Mutable for Unreal Engine 4 | Technical documentation
Updated 48 Days AgoPublic

Integrating

Architecture

The system is structured as an Unreal plugin with two additional Unreal modules: one for the editor and one for the runtime. These modules use external libraries for the Mutable runtime, which is a standalone library with no dependencies.

The next diagram shows the basic folder structure of an Unreal Engine game that uses Mutable. The hierarchy only shows the relevant folders for Mutable source code.

The folder “source” is optional (and by default not included with all licenses) and it contains the source code of the standalone runtime and tools libraries. These are already included in the “lib” folder as precompiled binaries for each supported platform, with the required headers in the “include” folder.

Packaging a project

Mutable will be automatically linked in your game executable. It doesn't require any additional files or dynamic-link libraries. In order to package the data for your Customizable Objects, an additional folders needs to be added in your project settings. The plugin will try to do this for you, but if you need to do it manually you can add "MutableStreamedData" to the list of "Additional non-asset directories to package" in the project settings. You can do this in the DefaultGame.ini file of your project:

+DirectoriesToAlwaysStageAsUFS=(Path="MutableStreamedData") )

or from the project settings in the Unreal Editor:

Engine Changes

In most versions of Mutable, some changes in the Unreal Engine codebase are required in order to improve the performance. These are modifications that cannot be implemented through plugins. In some platforms like Windows, Mutable will still work without the engine changes, but it may have lower performance and some editor features may be disabled, like automatic compilation of models when packaging.

The engine modifications are distributed through a patch file that you can find with every release of the plugin. They are also available in a GitHub branch of the official engine repository, as per Epic Game's rules. In order to apply the patch you can use standard tools like TODO.

Compilation warnings

The patch with engine changes includes some helpers to try to detect if any modification is missing. If these warnings are visible it is a sign that some of the changes have not been applied, or may have been overwritten by other changes or merges.

These are for the runtime:

#define ANTICTO_MUTABLE_ENGINE_PATCH_TEXTURESTREAMING
#define ANTICTO_MUTABLE_ENGINE_PATCH_VERTEXBUFFERS
#define ANTICTO_MUTABLE_ENGINE_PATCH_MORPHTARGETVERTEX

These are for the editor:

#define ANTICTO_MUTABLE_ENGINE_PATCH_COOKHOOK

It's not mandatory to manually use these defines but if they are not present, the plugin will show warnings at compile time.

Please note that these are just simple preprocessor defines, so they do not guarantee that the actual changes are present, but capture most error cases.


Using Mutable from C++

Creating an Instance of a CustomizableObject

A CustomizableObjectInstance contains a set of values for the parameters of a CustomizableObject, these values can be used by actors to create in-game meshes and materials.

To create an instance of a CustomizableObject, we need a reference to the Object and a CustomizableObjectInstance. Getting the Object reference can be done either by adding a pointer as UPROPERTY() to the actor’s class and then setting the value from blueprints or by loading it using the asset’s path. Having both, then calling the “SetObject” function tells the instance which object to represent. This way the instance knows what parameters to store and what their default values are.

UCustomizableObject* CustomizableObject = 
        LoadObject<UCustomizableObject>(nullptr, TEXT("/Game/MyCustomizableObject"));

if(CustomizableObject)
{
    CustomizableObjectInstance = NewObject<UCustomizableObjectInstance>();
    CustomizableObjectInstance->SetObject(CustomizableObject);
}

Connecting Instances with Actors

The best way to link an instance to an actor is to use the CustomizableSkeletalComponent, which can be attached to the actor’s SkeletalMeshComponent in order to update and replace the mesh asset with a Mutable-generated mesh.

Creating the component as shown below, when the actor is added to a level, it will automatically show the customized skeletal mesh.

CSkeletalComponent = 
        NewObject<UCustomizableSkeletalComponent>(UCustomizableSkeletalComponent::StaticClass());

// Set the instance that will be used by the actor
CSkeletalComponent->CustomizableObjectInstance = CustomizableObjectInstance;

// Attach the CustomizableSkeletalComponent to the Actor's SkeletalMeshComponent
CSkeletalComponent->AttachToComponent(GetMesh(), FAttachmentTransformRules::KeepRelativeTransform);

Changing Parameters

Parameters are stored in public arrays within each instance. These parameters can be modified directly, although it is recommended to use the API functions to avoid invalid values. Below is an example of how to modify different types of parameters.

// Sets the bool value of the parameter "Frackles"
CustomizableObjectInstance->SetBoolParameterSelectedOption(FString("Freckles"), true);

// Sets the int value of the parameter "Shirt" (Enum Type)
CustomizableObjectInstance->SetIntParameterSelectedOption(FString("Shirt"), FString("BasicShirt"));

// Sets the float value of the parameter "Fatness", range from 0 to 1
CustomizableObjectInstance->SetFloatParameterSelectedOption(FString("Fatness"), 0.5f);

// Sets the color value of the parameter "EyeColor"
CustomizableObjectInstance->SetColorParameterSelectedOption(FString("EyeColor"), 
                                                             FLinearColor(FColor::Blue));

// Sets the vector value of the parameter "VParam"
CustomizableObjectInstance->SetVectorParameterSelectedOption(FString("VParam"), 
                                                             FLinearColor(120.f, 50.f, 180.f));

// Sets the projectors values of the parameter "Tatto"
CustomizableObjectInstance->SetProjectorValue(FString("Tatto"), LocalPosition, Direction, Up, Scale, Angle, 
                                               ECustomizableObjectProjectorType::Planar);

Updating Instances

Changing parameters does not update the Instance automatically. To apply new values, the instance must be updated by calling the UpdateSkeletalMeshAsync() method, within the CustomizableSkeletalComponent class. Doing so will replace the SkeletalMesh of all actors using the same CustomizableObjectInstance with the generated from the new parameter values.

// Update Instances
CSkeletalComponent->UpdateSkeletalMeshAsync();

Instance Updated Delegate

Sometimes, you might want to run a specific method after the mesh has been updated, either to trigger an animation, make the mesh visible, etc.
To make it easy, Mutable provides a delegate where callbacks can be registered. This delegate will broadcast the completion of the skeletal mesh update to the registered callbacks.

// Bind "this" UObject's "OnCustomizableSkeletalUpdated" method as a callback for the delegate 
CSkeletalComponent->UpdatedDelegate.BindUObject(this, &MyCustomCharacter::OnCustomizableSkeletaUpdated);

---

// Unbind callbacks 
CSkeletalComponent->UpdatedDelegate.Unbind();

Using Mutable from Blueprint

Creating a Customizable Character

  1. Create a new blueprint with "character" as parent class and open it. Alternatively, you can use a class/BP that has a skeletal mesh component.

  1. In the blueprint editor, select the Mesh component and add a CustomizableSkeletal component as a child.

  1. Having selected the newly created component, go to the details panel and chose the instance to be used.

The reference mesh or the custom mesh, if generated, will show up in the viewport.

Changing Parameters

Parameters are stored by instances and can be accessed or modified using the API nodes. Here are examples on how to set parameter values based on their type:

  • Boolean Parameter

  • Int Parameter:

It's important to make sure that the desired option actually exists within the instance. Search existing parameters using the FindParameter node and then get the available option with GetIntParameterAvailableOption. Both nodes have the CustomizableObject as target, which can be accessed through the CustomizableObjectInstance.

  • Float Parameter

  • Color Parameter

  • Projector Parameter

Update Instances

To apply recent changes in parameters, the instance needs to be updated. This can be achieved by adding an UpdateSkeletalMeshAsync node after one or more changes.

Updated Delegate

Events can be registered to this delegate. The broadcasting will take place after the completion of the skeletal mesh update.

  • Bind Event

  • Unbind Event

Parameter Information

Sometimes, additional information like the amount of parameters within an instance, their type or the name of an int parameter's option might come in handy. This information is kept in the source CustomizableObject, accessible through the instance, and can be retrieved using the following nodes.

  • Get Parameter Count

  • Get Parameter Name

  • Get Parameter Type by Name

  • Find Parameter

  • Get Int Parameter Num Options

  • Get Int Parameter Available Option

---

Performance tuning

Memory usage

Mutable uses memory mainly in 3 different ways:

  • Streaming of source mutable data : The data required to build instances of customizable objects needs to be loaded from disk. This data is then processed into a set of skeletal meshes and textures that compose the final object. The source data may be very big if you have many customization options and it is kept in disk. A cache of this data is maintained in memory to speed up object construction. The size of this cache can be controlled by per-platform scalability settings as explained in the Mutable settings section below.
  • Working memory: This is the amount of data used while an object is being constructed, and it is freed while the system is idle. Mutable tries to minimize this amount of data, and its size depends entirely on the kind of objects to be built.
  • Object memory: This is the data used by the actual Unreal resources that compose an instance of a customizable object. Mutable will try to reuse textures that are identical, but other than this, these resources can be treated and measured exactly like any standard Unreal Skeletal Mesh or Texture.

Memory can always be reduced by tuning the assets provided to Mutable. For example, lower polygon counts in meshes, lower resolution in the reference textures or fewer customizable parts. Lower quality LODs can reduce their memory usage by disconnecting less important features like earrings and projections/tattoos. If these strategies have already been used or they are not an option, here are some memory reduction techniques that Mutable offers:

Set a maximum radius of visibility of Customizable Characters

If too much memory is being used because there are just too many different characters using lots of different skeletal meshes and textures, and most of them are too far from the camera to appreciate their differences or are hidden from view, a viable option is to set a maximum visibility radius around the player (or central actor). Only characters inside this radius will be generated and created, and the ones outside it will either be invisible or using the reference skeletal mesh. The API to control this functionality is located in the UCustomizableObjectSystem singleton class that can always be accessed with the following call:

UCustomizableObjectSystem::GetInstance()

The control functions for this functionality are as follows:

To enable/disable the Customization visibility radius and set the actual radius use:

void EnableOnlyUpdateCloseCustomizableObjects(float CloseDist);
void DisableOnlyUpdateCloseCustomizableObjects();

By default characters outside the radius will receive a null skeletal mesh and will be invisible. If you prefer to replace them with a common generic skeletal mesh, use the next two functions. The generic mesh is the reference skeletal mesh defined in the properties of the base Customizable Object of the character.

void SetReplaceDiscardedWithReferenceMeshEnabled(bool bIsEnabled);
bool IsReplaceDiscardedWithReferenceMeshEnabled() const;

The visibility radius is defined around the central actor of the scene. By default it is the first player controller actor, and can be changed with the following call:

void DefineCentralActor(const AActor* centralActor)
Limit the number of instances with all LODs

Sometimes most characters are too far away from the camera to effectively use all levels of detail, especially the higher quality ones like LOD0 and LOD1. A lot of memory is wasted in detail that most of the time will not be seen until the player gets close to those characters. In this case it makes sense to limit how many high-quality LODs Mutable generates for these far-away characters.

Mutable dynamically keeps track of every instance that is live in the game and how far they are from the player or central actor so it always knows which instances are more important and deserve having the most detailed LODs. There are some scalability/console variables (b.NumGeneratedInstancesLimit*) that control how many instances will have full LODs (LOD0 + LOD1 + LOD2), how may will have only two (LOD1 + LOD2) and finally how many will only have the low resolution LOD2 with minimal memory consumption. See the Scalability settings section for more information about these variables.

Only the closest instances to the player will get the full LODs. The next closest will get the two lower detail LODs and finally the rest will only get LOD2. If there are more instances than the sum of the values of b.NumGeneratedInstancesLimit*, the remaining furthest instances will not be generated and will be invisible.

At every tick Mutable will recalculate which instances deserve each level of detail and automatically issue and enqueue updates to discard LODs for the instances that have gone too far from the camera and updates to generate LODs for the ones that have got closer to the camera.

Although these updates are processed by prioritizing the closest instances to the camera or central actor, if the characters frequently move relative to it they can generate a lot of LOD change updates and quickly fill-up the queue. If the CPU is busy it may take a few moments to process the queue and new updates can take a few seconds to become visible. In this case, LOD updates can make customization updates (ex: putting on a jacket) lag for a while. This can be prevented with the following functions accessed through the UCustomizableObjectSystem:

void SetNumberOfPriorityUpdateInstances(int32 NumPriorityUpdateInstances);
int32 GetNumberOfPriorityUpdateInstances() const;

They can be used to set how many of the nearest instances to the player will have updates with priority over LOD changes, and thus update much faster. This is especially useful for multiplayer games with teams, where the player wants to see customization updates faster for teammates.

Reuse the memory resources of mutable textures during realtime parameter editing

By default, when a character changes a color in a texture or it moves a projector, the old texture is discarded and a new one is generated. The old texture's resources will not be released until the next Garbage Collection (GC) UE4 issues, so texture memory usage will go up until it happens. There are situations when a lot of textures will be created and destroyed in a short amount of time, for example in a customization lobby. The player moving a slider to control the color of a customizable object or moving a projector over a character's body will trigger a quick surge in texture usage until the next GC.

Mutable offers an option to reuse textures for such situations on a per-instance basis. It is not enabled by default because there are certain precautions that must be taken for it to be safe imposed by UE4's texture system. It is extremely important not to keep references to the textures of the instances that have texture reuse enabled, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable.

  1. First of all, make sure that the Customizable Object Instance is in a Mutable state where only the Mutable parameters of the currently edited object are being updated. See the section on "Customizable Object States" for more information. For instance, if a color slider is being edited, set the Mutable parameter that controls the color in the list of parameters of the Mutable state. Only use texture reuse in a customization lobby state, never in a gameplay state. In addition, make sure that this state disables texture compression and streaming. UE4 doesn't support the update of compressed and/or streaming textures.

Something like the following must be called before enabling texture reuse:

CustomizableObjectInstance->SetCurrentState(FString("LobbyStateName_WhereOnlyOneParameterIsEdited"));
  1. Activate texture reuse for the lobby instance before any parameter manipulation:
CustomizableObjectInstance->SetReuseInstanceTextures(true);
  1. Remember to disable texture reuse after the parameter editing session:
CustomizableObjectInstance->SetReuseInstanceTextures(false);
Immediately release mutable textures after instance destruction

As discussed in the previous section, by default textures created by Mutable cannot be released until a UE4 Garbage collection (GC) happens. In scenes where a lot of instances are being continuously created and destroyed or updated, there can be a spike in memory usage between GC events.

Mutable has an option to enable the immediate release of texture memory upon texture/instance destruction without having to wait for GC. Like the previous section, it is not enabled by default because the programmer has to take some precautions in order to make it safe, but this time the requirements are less strict.

Textures can be compressed and streamed, and the instances do not have to be set in a special state. The only thing have in mind is not to keep references to the textures of the instances, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable.

If some texture or skeletal mesh generated by Mutable needs to be used in something that is not the character itself and attached to the Customizable Skeletal Component, for example in the UI or in any other copy, please make a full deep copy of the UObjects and resources that has no reference to the ones generated by Mutable.

To enable the immediate release of texture resources call on the UCustomizableObjectSystem:

void SetReleaseMutableTexturesImmediately(bool bReleaseTextures)

Mutable Settings

General settings

These are settings that affect mutable in general ways. They can be defined in the DefaultMutable.ini file of your project. See the plugin Config folder for an example.

  • Features section
bExtraBoneInfluencesEnabledIf this is enabled, CustomizableObjects will be allowed to use 8 bone influences per vertex. Otherwise only the default 4 will be used. This is used at object compile time.
Scalability settings

These settings can be modified by the standard Scalability ini file hierarchy in Unreal, or with Console Variables or game code. These are the variables for each section:

  • ViewDistanceQuality section
b.MutableStreamingMemoryIf different than 0, limit the amount of memory (in KB) to use to cache streaming data when building characters. 0 means no limit, -1 means use default (40Mb in PC and 20Mb in consoles).
b.NumGeneratedInstancesLimitIf different than 0, limit the number of mutable instances with full LODs to have at a given time.
b.NumGeneratedInstancesLimitLOD1If different than 0, limit the number of mutable instances with LOD 1 to have at a given time.
b.NumGeneratedInstancesLimitLOD2If different than 0, limit the number of mutable instances with LOD 2 to have at a given time.
b.DistanceForFixedLOD2If NumGeneratedInstancesLimit is different than 0, sets the distance at which the system will fix the LOD of an instance to the lowest res one (LOD2) to prevent unnecessary LOD changes and memory

Statistics widget

A good starting point to see the performance of Mutable in your game, is to use the provided statistics UMG widget. You can find it in the plugin Content folder with the name UI/MutableStats.

When using these statistics in the editor, the numbers will also capture all the instances built in all open levels, editors, and even content browser thumbnails, if enabled.

If your project is not using UMG you can access these statistics and log them or display them from the central singleton object UCustomizableObjectSystem. These are some of the relevant methods:


	// Get the singleton object. It will be created if it doesn't exist yet.
	UFUNCTION(BlueprintCallable, BlueprintPure, Category = Status)
	static UCustomizableObjectSystem* GetInstance();

	// Find out if the performance engine patches have been applied
	UFUNCTION(BlueprintCallable, Category = Status)
	bool AreEnginePatchesPresent() const;

	// Find out the version of the plugin
	UFUNCTION(BlueprintCallable, Category = Status)
	FString GetPluginVersion() const;

	// Get the number of instances built and alive.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetNumInstances() const;

	// Get the number of instances waiting to be updated.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetNumPendingInstances() const;

	// Get the total number of instances including built and not built.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetTotalInstances() const;

	// Get the amount of memory in use for textures generated by mutable.
	UFUNCTION(BlueprintCallable, Category = Status)
	int64 GetTextureMemoryUsed() const;

	// Return the average build/update time of an instance in ms.
	UFUNCTION(BlueprintCallable, Category = Status)
	int32 GetAverageBuildTime() const;

Troubleshooting an Object

There are 2 ways of looking in detail at objects generated by Mutable to see the resources they use, or try to spot why a desired effect is not achieved.

  • You can bake an instance into standard Unreal assets.
  • You can open the preview mesh, material and textures by double-clicking the Customizable Instance SkeletalMesh in the Preview Instance Properties panel.

The second way is quicker, but since the dynamic preview assets don't exist in the Content Browser and disk, some editors may no behave as expected. In that case, we recommend using the baking option.

Customizable Object States

When an application uses complex Customizable Objects with many parameters, updating the instances may be a costly process that takes many milliseconds. However there are some usage scenarios for games that require these updates to be interactive, and big delays are not acceptable. In order to optimize your data to solve these problem, Mutable has the concept of States.

A State represents a specific use case of the Customizable Object in your game. For example, at some point during character creation you may want to let the player customize the face and hair of a character. During this stage, you show a close up camera of the character head and display a user interface for the related parameters: hair colour, nose size, hair style, etc. During this stage, you will not be modifying other parameters, like t-shirt colour, torso tattoos, etc. In order for Mutable to provide maximum performance, you can create a State in your Customizable Object, indicating that the subset of parameters that you will modify in these stage, and the system will generate an optimised version of the data that updates faster.

States also give you more options in order to optimise the instance construction time. For instance, in the previous example of the head customization stage, your game may have more graphic resources available because instead of being inside a level, you are in a smaller lobby scene. This means that you can afford to temporarily use more memory for your character. For each individual State Mutable gives you these optimisation options:

  • List of parameters that may be modified.
  • Option to avoid texture compression for textures that may change in this state.
  • Option to generate only LOD 0 of the object.

States are created in any Base Object Node. If no state is created, a default state with no optimized parameters and sensible optimisation options will be automatically created.

A normal game will want to create a "default" state to create customizable objects to be used in-game (no optimized parameters), and then several states to create and update objects in the different in-game customization scenarios.

The editor user interface for preview instances has the object state selection combobox at the top.

Last Author
max
Last Edited
Aug 10 2020, 7:37 PM

Event Timeline

jordi created this object.Mar 9 2020, 6:45 PM
jordi created this object in space S6 Mutable for Unreal Engine 4.
jordi edited the content of this document. (Show Details)Mar 26 2020, 9:42 AM
jordi edited the content of this document. (Show Details)Mar 26 2020, 9:46 AM
jordi edited the content of this document. (Show Details)Mar 26 2020, 9:53 AM
jordi edited the content of this document. (Show Details)Mar 26 2020, 6:56 PM
jordi edited the content of this document. (Show Details)Mar 26 2020, 8:34 PM
jordi edited the content of this document. (Show Details)
jordi edited the content of this document. (Show Details)Mar 31 2020, 4:53 PM
jordi edited the content of this document. (Show Details)Mar 31 2020, 8:30 PM
jordi edited the content of this document. (Show Details)Apr 16 2020, 12:02 PM
pere edited the content of this document. (Show Details)Apr 29 2020, 12:52 PM
pere edited the content of this document. (Show Details)Apr 29 2020, 11:14 PM
alexei edited the content of this document. (Show Details)May 21 2020, 10:06 AM
alexei edited the content of this document. (Show Details)May 21 2020, 10:13 AM
alexei edited the content of this document. (Show Details)May 21 2020, 10:24 AM
alexei edited the content of this document. (Show Details)
alexei edited the content of this document. (Show Details)May 21 2020, 10:38 AM
alexei edited the content of this document. (Show Details)May 21 2020, 11:10 AM
alexei edited the content of this document. (Show Details)May 21 2020, 11:30 AM
alexei edited the content of this document. (Show Details)May 21 2020, 11:44 AM
alexei edited the content of this document. (Show Details)May 21 2020, 11:46 AM
alexei edited the content of this document. (Show Details)May 21 2020, 12:07 PM
alexei edited the content of this document. (Show Details)May 21 2020, 12:23 PM
alexei edited the content of this document. (Show Details)May 21 2020, 12:27 PM
alexei edited the content of this document. (Show Details)May 21 2020, 12:29 PM
alexei published a new version of this document.
alexei edited the content of this document. (Show Details)May 21 2020, 12:46 PM
alexei published a new version of this document.May 21 2020, 12:51 PM
pere edited the content of this document. (Show Details)May 21 2020, 1:03 PM
pere edited the content of this document. (Show Details)May 22 2020, 3:37 PM
pere edited the content of this document. (Show Details)May 22 2020, 9:00 PM
pere edited the content of this document. (Show Details)May 25 2020, 12:27 AM
ricard edited the content of this document. (Show Details)May 26 2020, 11:31 AM
ricard edited the content of this document. (Show Details)May 26 2020, 11:46 AM
jordi changed the visibility from "All Users" to "Public (No Login Required)".May 26 2020, 12:04 PM
pere edited the content of this document. (Show Details)May 26 2020, 1:55 PM
pere edited the content of this document. (Show Details)May 26 2020, 6:07 PM
pere published a new version of this document.
pere edited the content of this document. (Show Details)May 26 2020, 8:09 PM
jordi edited the content of this document. (Show Details)Jun 2 2020, 9:56 AM
alexei edited the content of this document. (Show Details)Jun 2 2020, 4:12 PM
ricard edited the content of this document. (Show Details)Jun 19 2020, 11:28 AM
ricard edited the content of this document. (Show Details)Jun 22 2020, 3:57 PM
max edited the content of this document. (Show Details)Aug 10 2020, 7:34 PM
max edited the content of this document. (Show Details)Aug 10 2020, 7:37 PM