== Integrating == === Architecture === The system is structured as an Unreal plugin with two additional Unreal modules: one for the editor and one for the runtime. These modules use external libraries for the Mutable runtime, which is a standalone library with no dependencies. {F85832, size=full} The next diagram shows the basic folder structure of an Unreal Engine game that uses Mutable. The hierarchy only shows the relevant folders for Mutable source code. {F85834, size=full} The folder “source” is optional (and by default not included with all licenses) and it contains the source code of the standalone runtime and tools libraries. These are already included in the “lib” folder as precompiled binaries for each supported platform, with the required headers in the “include” folder. === Packaging a project === Mutable will be automatically linked in your game executable. It doesn't require any additional files or dynamic-link libraries. In order to package the data for your Customizable Objects, an additional folders needs to be added in your project settings. The plugin will try to do this for you, but if you need to do it manually you can add "MutableStreamedData" to the list of "Additional non-asset directories to package" in the project settings. You can do this in the DefaultGame.ini file of your project: ``` +DirectoriesToAlwaysStageAsUFS=(Path="MutableStreamedData") ) ``` or from the project settings in the Unreal Editor: {F85836} === Engine Changes === In most versions of Mutable, some changes in the Unreal Engine codebase are required in order to improve the performance. These are modifications that cannot be implemented through plugins. In some platforms like Windows, Mutable will still work without the engine changes, but it may have lower performance and some editor features may be disabled, like automatic compilation of models when packaging. The engine modifications are distributed through a patch file that you can find with every release of the plugin. They are also available in a GitHub branch of the official engine repository, as per Epic Game's rules. In order to apply the patch you can use standard tools like TODO. === Compilation warnings {anchor #warnings} === The patch with engine changes includes some helpers to try to detect if any modification is missing. If these warnings are visible it is a sign that some of the changes have not been applied, or may have been overwritten by other changes or merges. These are for the runtime: ``` lang=c++ #define ANTICTO_MUTABLE_ENGINE_PATCH_TEXTURESTREAMING #define ANTICTO_MUTABLE_ENGINE_PATCH_VERTEXBUFFERS #define ANTICTO_MUTABLE_ENGINE_PATCH_MORPHTARGETVERTEX ``` These are for the editor: ``` lang=c++ #define ANTICTO_MUTABLE_ENGINE_PATCH_COOKHOOK ``` It's not mandatory to manually use these defines but if they are not present, the plugin will show warnings at compile time. {icon exclamation-triangle color=orange} Please note that these are just simple preprocessor defines, so they do not guarantee that the actual changes are present, but capture most error cases. --- == Using Mutable from C++ {anchor #cpp} == === Creating an Instance of a CustomizableObject === A CustomizableObjectInstance contains a set of values for the parameters of a CustomizableObject, these values can be used by actors to create in-game meshes and materials. To create an instance of a CustomizableObject, we need a reference to the Object and a CustomizableObjectInstance. Getting the Object reference can be done either by adding a pointer as UPROPERTY() to the actor’s class and then setting the value from blueprints or by loading it using the asset’s path. Having both, then calling the “SetObject” function tells the instance which object to represent. This way the instance knows what parameters to store and what their default values are. ``` lang=cpp UCustomizableObject* CustomizableObject = LoadObject<UCustomizableObject>(nullptr, TEXT("/Game/MyCustomizableObject")); if(CustomizableObject) { CustomizableObjectInstance = NewObject<UCustomizableObjectInstance>(); CustomizableObjectInstance->SetObject(CustomizableObject); } ``` === Connecting Instances with Actors === The best way to link an instance to an actor is to use the CustomizableSkeletalComponent, which can be attached to the actor’s SkeletalMeshComponent in order to update and replace the mesh asset with a Mutable-generated mesh. Creating the component as shown below, when the actor is added to a level, it will automatically show the customized skeletal mesh. ``` lang=cpp CSkeletalComponent = NewObject<UCustomizableSkeletalComponent>(UCustomizableSkeletalComponent::StaticClass()); // Set the instance that will be used by the actor CSkeletalComponent->CustomizableObjectInstance = CustomizableObjectInstance; // Attach the CustomizableSkeletalComponent to the Actor's SkeletalMeshComponent CSkeletalComponent->AttachToComponent(GetMesh(), FAttachmentTransformRules::KeepRelativeTransform); ``` === Changing Parameters === Parameters are stored in public arrays within each instance. These parameters can be modified directly, although it is recommended to use the API functions to avoid invalid values. Below is an example of how to modify different types of parameters. ``` lang=cpp // Sets the bool value of the parameter "Frackles" CustomizableObjectInstance->SetBoolParameterSelectedOption(FString("Freckles"), true); // Sets the int value of the parameter "Shirt" (Enum Type) CustomizableObjectInstance->SetIntParameterSelectedOption(FString("Shirt"), FString("BasicShirt")); // Sets the float value of the parameter "Fatness", range from 0 to 1 CustomizableObjectInstance->SetFloatParameterSelectedOption(FString("Fatness"), 0.5f); // Sets the color value of the parameter "EyeColor" CustomizableObjectInstance->SetColorParameterSelectedOption(FString("EyeColor"), FLinearColor(FColor::Blue)); // Sets the vector value of the parameter "VParam" CustomizableObjectInstance->SetVectorParameterSelectedOption(FString("VParam"), FLinearColor(120.f, 50.f, 180.f)); // Sets the projectors values of the parameter "Tatto" CustomizableObjectInstance->SetProjectorValue(FString("Tatto"), LocalPosition, Direction, Up, Scale, Angle, ECustomizableObjectProjectorType::Planar); ``` === Updating Instances === Changing parameters does not update the Instance automatically. To apply new values, the instance must be updated by calling the UpdateSkeletalMeshAsync() method, within the CustomizableSkeletalComponent class. Doing so will replace the SkeletalMesh of all actors using the same CustomizableObjectInstance with the generated from the new parameter values. ``` lang=cpp // Update Instances CSkeletalComponent->UpdateSkeletalMeshAsync(); ``` Here is a diagram of what goes on "under the hood" when an update is taking place. Most of the CPU work is done in Mutable's own thread, and for most projects most of the time is spent waiting for assets to stream from disk. {F161570, size=full} If your project's CPU threads or disk are highly saturated all the time, it may slow down the updates of Mutable characters because of resource contention. === Instance Updated Delegate === Sometimes, you might want to run a specific method after the mesh has been updated, either to trigger an animation, make the mesh visible, etc. To make it easy, Mutable provides a delegate where callbacks can be registered. This delegate will broadcast the completion of the skeletal mesh update to the registered callbacks. ``` lang=cpp // Bind "this" UObject's "OnCustomizableSkeletalUpdated" method as a callback for the delegate CSkeletalComponent->UpdatedDelegate.BindUObject(this, &MyCustomCharacter::OnCustomizableSkeletaUpdated); --- // Unbind callbacks CSkeletalComponent->UpdatedDelegate.Unbind(); ``` --- == Using Mutable from Blueprint {anchor #blueprint} == === Creating a Customizable Character {anchor #create-character} === 1. Create a new blueprint with "character" as parent class and open it. Alternatively, you can use a class/BP that has a skeletal mesh component. {F102097} 2. In the blueprint editor, select the Mesh component and add a CustomizableSkeletal component as a child. {F102099} 3. Having selected the newly created component, go to the details panel and chose the instance to be used. {F102105} The reference mesh or the custom mesh, if generated, will show up in the viewport. {F102109} === Changing Parameters === Parameters are stored by instances and can be accessed or modified using the API nodes. Here are examples on how to set parameter values based on their type: * Boolean Parameter {F102123} * Int Parameter: It's important to make sure that the desired option actually exists within the instance. Search existing parameters using the FindParameter node and then get the available option with GetIntParameterAvailableOption. Both nodes have the CustomizableObject as target, which can be accessed through the CustomizableObjectInstance. {F102113} * Float Parameter {F102119} * Color Parameter {F102129} * Projector Parameter {F102131} === Update Instances === To apply recent changes in parameters, the instance needs to be updated. This can be achieved by adding an UpdateSkeletalMeshAsync node after one or more changes. {F102256} === Updated Delegate === Events can be registered to this delegate. The broadcasting will take place after the completion of the skeletal mesh update. * Bind Event {F102368} * Unbind Event {F102370} === Parameter Information === Sometimes, additional information like the amount of parameters within an instance, their type or the name of an int parameter's option might come in handy. This information is kept in the source CustomizableObject, accessible through the instance, and can be retrieved using the following nodes. * Get Parameter Count {F102356} * Get Parameter Name {F102358} * Get Parameter Type by Name {F102360} * Find Parameter {F102362} * Get Int Parameter Num Options {F102364} * Get Int Parameter Available Option {F102366} === Changing States === As will be shown in a section below, Mutable has the concept of states that enable certain optimizations depending on the use case. The state an instance is in can be queried and changed using the node API. * Set Current State Similarly to when changing parameters, the instance needs to be updated with a UpdateSkeletalMeshAsync node after a state change. {F161529} * Get Current State {F161531} === State Information === Information about the number and name of states a CustomizableObject has and the number and names of parameters in a state can be useful. This information is stored per CustomizableObject, accessible through a CustomizableObjectInstance and can be retrieved using the following nodes. * Get State Count {F161543} * Get State Name {F161539} * Get State Parameter Count {F161533} * Get State Parameter Name {F161535} --- == Performance tuning == === Memory usage === Mutable uses memory mainly in 3 different ways: - **Streaming of source mutable data** : The data required to build instances of customizable objects needs to be loaded from disk. This data is then processed into a set of skeletal meshes and textures that compose the final object. The source data may be very big if you have many customization options and it is kept in disk. A cache of this data is maintained in memory to speed up object construction. The size of this cache can be controlled by per-platform scalability settings as explained in the //Mutable settings// section below. - **Working memory**: This is the amount of data used while an object is being constructed, and it is freed while the system is idle. Mutable tries to minimize this amount of data, and its size depends entirely on the kind of objects to be built. - **Object memory**: This is the data used by the actual Unreal resources that compose an instance of a customizable object. Mutable will try to reuse textures that are identical, but other than this, these resources can be treated and measured exactly like any standard Unreal Skeletal Mesh or Texture. Memory can always be reduced by tuning the assets provided to Mutable. For example, lower polygon counts in meshes, lower resolution in the reference textures or fewer customizable parts. Lower quality LODs can reduce their memory usage by disconnecting less important features like earrings and projections/tattoos. If these strategies have already been used or they are not an option, here are some memory reduction techniques that Mutable offers: ==== Set a maximum radius of visibility of Customizable Characters ==== If too much memory is being used because there are just too many different characters using lots of different skeletal meshes and textures, and most of them are too far from the camera to appreciate their differences or are hidden from view, a viable option is to set a maximum visibility radius arround one or more view centers, for example the player. Only characters inside this radius will be generated and created, and the ones outside it will either be invisible or using the reference skeletal mesh. The API to control this functionality is located in the UCustomizableInstanceLODManagement class, accessible through the UCustomizableObjectSystem singleton with the following call: ``` lang=cpp auto LODManagmentBase = UCustomizableObjectSystem::GetInstance()->GetInstanceLODManagement(); ``` Cast the LODManagmentBase to a UCustomizableInstanceLODManagement to access the LOD system provided by Mutable. ``` lang=cpp if(auto MutableLODManagment = Cast<UCustomizableInstanceLODManagement>( LODManagmentBase) { } ``` The control functions for this functionality are as follows: To enable/disable the Customization visibility radius and set the actual radius use: ``` lang=cpp void EnableOnlyUpdateCloseCustomizableObjects(float CloseDist); void DisableOnlyUpdateCloseCustomizableObjects(); ``` By default characters outside the radius will receive a null skeletal mesh and will be invisible. If you prefer to replace them with a common generic skeletal mesh, use the next two functions. The generic mesh is the reference skeletal mesh defined in the properties of the base Customizable Object of the character. ``` lang=cpp void SetReplaceDiscardedWithReferenceMeshEnabled(bool bIsEnabled); bool IsReplaceDiscardedWithReferenceMeshEnabled() const; ``` The visibility radius is defined around the view center/s of the scene. By default it is the first player controller actor, and can be changed with the following calls: ``` lang=cpp void AddViewCenter(const AActor* const InViewCenter); void RemoveViewCenter(const AActor* const InViewCenter); void ClearAllViewCenters(); ``` Sometimes most characters are too far away from the camera to effectively use all levels of detail, especially the higher quality ones like LOD0 and LOD1. A lot of memory is wasted in detail that most of the time will not be seen until the player gets close to those characters. In this case it makes sense to limit how many high-quality LODs Mutable generates for these far-away characters. To adjust the number of generated LODs, Mutable provides different two methods. ==== Automatic adjustment of generated LODs per instance ==== Once the maximum display radius has been defined, an automatic system will come into operation to limit the LODs present in each instance according to the screen size of the meshes. The system works similarly to the rendering system. Mutable dynamically tracks all active instances in the game, if they are within the maximum radius of visibility it estimates the level of detail they use and, if necessary, issue a request to discard or add LODs. Using this system minimizes the memory usage used by instances without limiting the number of instances with all LODs. In cases where the density of unique instances is very high and the character moves frequently, there may be cases where the number of requests to generate instances with all LODs accumulates in the tasks queue. Since requests are processed based on the distance to the view centers, in these circumstances, it may take a few seconds to process updates for far-away instances. ==== Limit the number of instances with all LODs ==== (WARNING) {icon exclamation-triangle color=orange} This method disables the Automatic LOD system Another method of control the number of LODs generated in order to reduce the memory being used is to limit how many instances will have full LODS (LOD0 + LOD1 + LOD2), how many will have only two (LOD1 + LOD2) and finally how many will only have the low resolution LOD2 with minimal memory consumption. There are some scalability/console variables (b.NumGeneratedInstancesLimit*) to customize this option. See the Scalability settings section for more information about these variables. Similar to the automatic adjustment of generated LODs, Mutable will recalculate in each frame which level of detail belongs to each instance. In this case however the calculation will be based on the distance to the view centers and a the limit on the number of instances with full LODs. Only the closest instances to the view centers will get the full LODs. The next closest will get the two lower detail LODs and finally the rest will only get LOD2. If there are more instances than the sum of the values of b.NumGeneratedInstancesLimit*, the remaining furthest instances will not be generated and will be either invisible or replaced by the reference skeletal mesh. At every tick Mutable will recalculate which instances deserve each level of detail and automatically issue and enqueue updates to discard LODs for the instances that have gone too far from the camera and updates to generate LODs for the ones that have got closer to the camera. Although these updates are processed by prioritizing the closest instances to the view centers, if the characters frequently move relative to it they can generate a lot of LOD change updates and quickly fill-up the queue. If the CPU is busy it may take a few moments to process the queue and new updates can take a few seconds to become visible. In this case, LOD updates can make customization updates (ex: putting on a jacket) lag for a while. This can be prevented with the following functions accessed through the UCustomizableObjectSystem: ``` lang=cpp void SetNumberOfPriorityUpdateInstances(int32 NumPriorityUpdateInstances); int32 GetNumberOfPriorityUpdateInstances() const; ``` They can be used to set how many of the nearest instances to the player will have updates with priority over LOD changes, and thus update much faster. This is especially useful for multiplayer games with teams, where the player wants to see customization updates faster for teammates. ==== Reuse the memory resources of mutable textures during realtime parameter editing ==== By default, when a character changes a color in a texture or it moves a projector, the old texture is discarded and a new one is generated. The old texture's resources will not be released until the next Garbage Collection (GC) UE4 issues, so texture memory usage will go up until it happens. There are situations when a lot of textures will be created and destroyed in a short amount of time, for example in a customization lobby. The player moving a slider to control the color of a customizable object or moving a projector over a character's body will trigger a quick surge in texture usage until the next GC. Mutable offers an option to reuse textures for such situations on a per-instance basis. It is not enabled by default because there are certain precautions that must be taken for it to be safe imposed by UE4's texture system. It is extremely important not to keep references to the textures of the instances that have texture reuse enabled, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable. 1. First of all, make sure that the Customizable Object Instance is in a Mutable state where only the Mutable parameters of the currently edited object are being updated. See the section on "Customizable Object States" for more information. For instance, if a color slider is being edited, set the Mutable parameter that controls the color in the list of parameters of the Mutable state. Only use texture reuse in a customization lobby state, never in a gameplay state. In addition, make sure that this state disables texture compression and streaming. UE4 doesn't support the update of compressed and/or streaming textures. Something like the following must be called before enabling texture reuse: ``` lang=cpp CustomizableObjectInstance->SetCurrentState(FString("LobbyStateName_WhereOnlyOneParameterIsEdited")); ``` 2. Activate texture reuse for the lobby instance before any parameter manipulation: ``` lang=cpp CustomizableObjectInstance->SetReuseInstanceTextures(true); ``` 3. Remember to disable texture reuse after the parameter editing session: ``` lang=cpp CustomizableObjectInstance->SetReuseInstanceTextures(false); ``` ==== Immediately release mutable textures after instance destruction ==== As discussed in the previous section, by default textures created by Mutable cannot be released until a UE4 Garbage collection (GC) happens. In scenes where a lot of instances are being continuously created and destroyed or updated, there can be a spike in memory usage between GC events. Mutable has an option to enable the immediate release of texture memory upon texture/instance destruction without having to wait for GC. Like the previous section, it is not enabled by default because the programmer has to take some precautions in order to make it safe, but this time the requirements are less strict. Textures can be compressed and streamed, and the instances do not have to be set in a special state. The only thing have in mind is not to keep references to the textures of the instances, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable. If some texture or skeletal mesh generated by Mutable needs to be used in something that is not the character itself and attached to the Customizable Skeletal Component, for example in the UI or in any other copy, please make a full deep copy of the UObjects and resources that has no reference to the ones generated by Mutable. To enable the immediate release of texture resources call on the UCustomizableObjectSystem: ``` lang=cpp void SetReleaseMutableTexturesImmediately(bool bReleaseTextures) ``` === Mutable Settings === ==== General settings ==== These are settings that affect mutable in general ways. They can be defined in the DefaultMutable.ini file of your project. See the plugin Config folder for an example. * **Features** section | bExtraBoneInfluencesEnabled | If this is enabled, CustomizableObjects will be allowed to use 8 bone influences per vertex. Otherwise only the default 4 will be used. This is used at object compile time. | ====Scalability settings==== These settings can be modified by the [[ https://docs.unrealengine.com/en-US/Engine/Performance/Scalability/ScalabilityReference/index.html | standard Scalability ini file hierarchy in Unreal ]], or with Console Variables or game code. These are the variables for each section: * **ViewDistanceQuality** section | b.MutableStreamingMemory | If different than 0, limit the amount of memory (in KB) to use to cache streaming data when building characters. 0 means no limit, -1 means use default (40Mb in PC and 20Mb in consoles). | | b.NumGeneratedInstancesLimit | If different than 0, limit the number of mutable instances with full LODs to have at a given time. | | b.NumGeneratedInstancesLimitLOD1 | If different than 0, limit the number of mutable instances with LOD 1 to have at a given time. | | b.NumGeneratedInstancesLimitLOD2 | If different than 0, limit the number of mutable instances with LOD 2 to have at a given time. | | b.DistanceForFixedLOD2 | If NumGeneratedInstancesLimit is different than 0, sets the distance at which the system will fix the LOD of an instance to the lowest res one (LOD2) to prevent unnecessary LOD changes and memory | === Statistics widget === A good starting point to see the performance of Mutable in your game, is to use the provided statistics UMG widget. You can find it in the plugin //Content// folder with the name //UI/MutableStats//. {F85822} (WARNING) {icon exclamation-triangle color=orange} When using these statistics in the editor, the numbers will also capture all the instances built in all open levels, editors, and even content browser thumbnails, if enabled. If your project is not using UMG you can access these statistics and log them or display them from the central singleton object UCustomizableObjectSystem. These are some of the relevant methods: ``` lang=cpp // Get the singleton object. It will be created if it doesn't exist yet. UFUNCTION(BlueprintCallable, BlueprintPure, Category = Status) static UCustomizableObjectSystem* GetInstance(); // Find out if the performance engine patches have been applied UFUNCTION(BlueprintCallable, Category = Status) bool AreEnginePatchesPresent() const; // Find out the version of the plugin UFUNCTION(BlueprintCallable, Category = Status) FString GetPluginVersion() const; // Get the number of instances built and alive. UFUNCTION(BlueprintCallable, Category = Status) int32 GetNumInstances() const; // Get the number of instances waiting to be updated. UFUNCTION(BlueprintCallable, Category = Status) int32 GetNumPendingInstances() const; // Get the total number of instances including built and not built. UFUNCTION(BlueprintCallable, Category = Status) int32 GetTotalInstances() const; // Get the amount of memory in use for textures generated by mutable. UFUNCTION(BlueprintCallable, Category = Status) int64 GetTextureMemoryUsed() const; // Return the average build/update time of an instance in ms. UFUNCTION(BlueprintCallable, Category = Status) int32 GetAverageBuildTime() const; ``` === Troubleshooting an Object === There are 2 ways of looking in detail at objects generated by Mutable to see the resources they use, or try to spot why a desired effect is not achieved. * You can [[ mutable/unreal-engine-4/user-documentation/baking-instances/ | bake an instance ]] into standard Unreal assets. * You can open the preview mesh, material and textures by double-clicking the //Customizable Instance SkeletalMesh// in the //Preview Instance Properties// panel. {F103397} The second way is quicker, but since the dynamic preview assets don't exist in the Content Browser and disk, some editors may no behave as expected. In that case, we recommend using the baking option. === Customizable Object States === When an application uses complex Customizable Objects with many parameters, updating the instances may be a costly process that takes many milliseconds. However there are some usage scenarios for games that require these updates to be interactive, and big delays are not acceptable. In order to optimize your data to solve these problem, Mutable has the concept of States. A State represents a specific use case of the Customizable Object in your game. For example, at some point during character creation you may want to let the player customize the face and hair of a character. During this stage, you show a close up camera of the character head and display a user interface for the related parameters: hair colour, nose size, hair style, etc. During this stage, you will not be modifying other parameters, like t-shirt colour, torso tattoos, etc. In order for Mutable to provide maximum performance, you can create a State in your Customizable Object, indicating that the subset of parameters that you will modify in these stage, and the system will generate an optimised version of the data that updates faster. States also give you more options in order to optimise the instance construction time. For instance, in the previous example of the head customization stage, your game may have more graphic resources available because instead of being inside a level, you are in a smaller lobby scene. This means that you can afford to temporarily use more memory for your character. For each individual State Mutable gives you these optimisation options: * List of parameters that may be modified. * Option to avoid texture compression for textures that may change in this state. * Option to generate only LOD 0 of the object. States are created in any Base Object Node. If no state is created, a default state with no optimized parameters and sensible optimisation options will be automatically created. A normal game will want to create a "default" state to create customizable objects to be used in-game (no optimized parameters), and then several states to create and update objects in the different in-game customization scenarios. The editor user interface for preview instances has the object state selection combobox at the top.

== Integrating == === Architecture === The system is structured as an Unreal plugin with two additional Unreal modules: one for the editor and one for the runtime. These modules use external libraries for the Mutable runtime, which is a standalone library with no dependencies. {F85832, size=full} The next diagram shows the basic folder structure of an Unreal Engine game that uses Mutable. The hierarchy only shows the relevant folders for Mutable source code. {F85834, size=full} The folder “source” is optional (and by default not included with all licenses) and it contains the source code of the standalone runtime and tools libraries. These are already included in the “lib” folder as precompiled binaries for each supported platform, with the required headers in the “include” folder. === Packaging a project === Mutable will be automatically linked in your game executable. It doesn't require any additional files or dynamic-link libraries. In order to package the data for your Customizable Objects, an additional folders needs to be added in your project settings. The plugin will try to do this for you, but if you need to do it manually you can add "MutableStreamedData" to the list of "Additional non-asset directories to package" in the project settings. You can do this in the DefaultGame.ini file of your project: ``` +DirectoriesToAlwaysStageAsUFS=(Path="MutableStreamedData") ) ``` or from the project settings in the Unreal Editor: {F85836} === Engine Changes === In most versions of Mutable, some changes in the Unreal Engine codebase are required in order to improve the performance. These are modifications that cannot be implemented through plugins. In some platforms like Windows, Mutable will still work without the engine changes, but it may have lower performance and some editor features may be disabled, like automatic compilation of models when packaging. The engine modifications are distributed through a patch file that you can find with every release of the plugin. They are also available in a GitHub branch of the official engine repository, as per Epic Game's rules. In order to apply the patch you can use standard tools like TODO. === Compilation warnings {anchor #warnings} === The patch with engine changes includes some helpers to try to detect if any modification is missing. If these warnings are visible it is a sign that some of the changes have not been applied, or may have been overwritten by other changes or merges. These are for the runtime: ``` lang=c++ #define ANTICTO_MUTABLE_ENGINE_PATCH_TEXTURESTREAMING #define ANTICTO_MUTABLE_ENGINE_PATCH_VERTEXBUFFERS #define ANTICTO_MUTABLE_ENGINE_PATCH_MORPHTARGETVERTEX ``` These are for the editor: ``` lang=c++ #define ANTICTO_MUTABLE_ENGINE_PATCH_COOKHOOK ``` It's not mandatory to manually use these defines but if they are not present, the plugin will show warnings at compile time. {icon exclamation-triangle color=orange} Please note that these are just simple preprocessor defines, so they do not guarantee that the actual changes are present, but capture most error cases. --- == Using Mutable from C++ {anchor #cpp} == === Creating an Instance of a CustomizableObject === A CustomizableObjectInstance contains a set of values for the parameters of a CustomizableObject, these values can be used by actors to create in-game meshes and materials. To create an instance of a CustomizableObject, we need a reference to the Object and a CustomizableObjectInstance. Getting the Object reference can be done either by adding a pointer as UPROPERTY() to the actor’s class and then setting the value from blueprints or by loading it using the asset’s path. Having both, then calling the “SetObject” function tells the instance which object to represent. This way the instance knows what parameters to store and what their default values are. ``` lang=cpp UCustomizableObject* CustomizableObject = LoadObject<UCustomizableObject>(nullptr, TEXT("/Game/MyCustomizableObject")); if(CustomizableObject) { CustomizableObjectInstance = NewObject<UCustomizableObjectInstance>(); CustomizableObjectInstance->SetObject(CustomizableObject); } ``` === Connecting Instances with Actors === The best way to link an instance to an actor is to use the CustomizableSkeletalComponent, which can be attached to the actor’s SkeletalMeshComponent in order to update and replace the mesh asset with a Mutable-generated mesh. Creating the component as shown below, when the actor is added to a level, it will automatically show the customized skeletal mesh. ``` lang=cpp CSkeletalComponent = NewObject<UCustomizableSkeletalComponent>(UCustomizableSkeletalComponent::StaticClass()); // Set the instance that will be used by the actor CSkeletalComponent->CustomizableObjectInstance = CustomizableObjectInstance; // Attach the CustomizableSkeletalComponent to the Actor's SkeletalMeshComponent CSkeletalComponent->AttachToComponent(GetMesh(), FAttachmentTransformRules::KeepRelativeTransform); ``` === Changing Parameters === Parameters are stored in public arrays within each instance. These parameters can be modified directly, although it is recommended to use the API functions to avoid invalid values. Below is an example of how to modify different types of parameters. ``` lang=cpp // Sets the bool value of the parameter "Frackles" CustomizableObjectInstance->SetBoolParameterSelectedOption(FString("Freckles"), true); // Sets the int value of the parameter "Shirt" (Enum Type) CustomizableObjectInstance->SetIntParameterSelectedOption(FString("Shirt"), FString("BasicShirt")); // Sets the float value of the parameter "Fatness", range from 0 to 1 CustomizableObjectInstance->SetFloatParameterSelectedOption(FString("Fatness"), 0.5f); // Sets the color value of the parameter "EyeColor" CustomizableObjectInstance->SetColorParameterSelectedOption(FString("EyeColor"), FLinearColor(FColor::Blue)); // Sets the vector value of the parameter "VParam" CustomizableObjectInstance->SetVectorParameterSelectedOption(FString("VParam"), FLinearColor(120.f, 50.f, 180.f)); // Sets the projectors values of the parameter "Tatto" CustomizableObjectInstance->SetProjectorValue(FString("Tatto"), LocalPosition, Direction, Up, Scale, Angle, ECustomizableObjectProjectorType::Planar); ``` === Updating Instances === Changing parameters does not update the Instance automatically. To apply new values, the instance must be updated by calling the UpdateSkeletalMeshAsync() method, within the CustomizableSkeletalComponent class. Doing so will replace the SkeletalMesh of all actors using the same CustomizableObjectInstance with the generated from the new parameter values. ``` lang=cpp // Update Instances CSkeletalComponent->UpdateSkeletalMeshAsync(); ``` Here is a diagram of what goes on "under the hood" when an update is taking place. Most of the CPU work is done in Mutable's own thread, and for most projects most of the time is spent waiting for assets to stream from disk. {F161570, size=full} If your project's CPU threads or disk are highly saturated all the time, it may slow down the updates of Mutable characters because of resource contention. === Instance Updated Delegate === Sometimes, you might want to run a specific method after the mesh has been updated, either to trigger an animation, make the mesh visible, etc. To make it easy, Mutable provides a delegate where callbacks can be registered. This delegate will broadcast the completion of the skeletal mesh update to the registered callbacks. ``` lang=cpp // Bind "this" UObject's "OnCustomizableSkeletalUpdated" method as a callback for the delegate CSkeletalComponent->UpdatedDelegate.BindUObject(this, &MyCustomCharacter::OnCustomizableSkeletaUpdated); --- // Unbind callbacks CSkeletalComponent->UpdatedDelegate.Unbind(); ``` --- == Using Mutable from Blueprint {anchor #blueprint} == === Creating a Customizable Character {anchor #create-character} === 1. Create a new blueprint with "character" as parent class and open it. Alternatively, you can use a class/BP that has a skeletal mesh component. {F102097} 2. In the blueprint editor, select the Mesh component and add a CustomizableSkeletal component as a child. {F102099} 3. Having selected the newly created component, go to the details panel and chose the instance to be used. {F102105} The reference mesh or the custom mesh, if generated, will show up in the viewport. {F102109} === Changing Parameters === Parameters are stored by instances and can be accessed or modified using the API nodes. Here are examples on how to set parameter values based on their type: * Boolean Parameter {F102123} * Int Parameter: It's important to make sure that the desired option actually exists within the instance. Search existing parameters using the FindParameter node and then get the available option with GetIntParameterAvailableOption. Both nodes have the CustomizableObject as target, which can be accessed through the CustomizableObjectInstance. {F102113} * Float Parameter {F102119} * Color Parameter {F102129} * Projector Parameter {F102131} === Update Instances === To apply recent changes in parameters, the instance needs to be updated. This can be achieved by adding an UpdateSkeletalMeshAsync node after one or more changes. {F102256} === Updated Delegate === Events can be registered to this delegate. The broadcasting will take place after the completion of the skeletal mesh update. * Bind Event {F102368} * Unbind Event {F102370} === Parameter Information === Sometimes, additional information like the amount of parameters within an instance, their type or the name of an int parameter's option might come in handy. This information is kept in the source CustomizableObject, accessible through the instance, and can be retrieved using the following nodes. * Get Parameter Count {F102356} * Get Parameter Name {F102358} * Get Parameter Type by Name {F102360} * Find Parameter {F102362} * Get Int Parameter Num Options {F102364} * Get Int Parameter Available Option {F102366} === Changing States === As will be shown in a section below, Mutable has the concept of states that enable certain optimizations depending on the use case. The state an instance is in can be queried and changed using the node API. * Set Current State Similarly to when changing parameters, the instance needs to be updated with a UpdateSkeletalMeshAsync node after a state change. {F161529} * Get Current State {F161531} === State Information === Information about the number and name of states a CustomizableObject has and the number and names of parameters in a state can be useful. This information is stored per CustomizableObject, accessible through a CustomizableObjectInstance and can be retrieved using the following nodes. * Get State Count {F161543} * Get State Name {F161539} * Get State Parameter Count {F161533} * Get State Parameter Name {F161535} --- == Performance tuning == {anchor #performance} === Memory usage === Mutable uses memory mainly in 3 different ways: - **Streaming of source mutable data** : The data required to build instances of customizable objects needs to be loaded from disk. This data is then processed into a set of skeletal meshes and textures that compose the final object. The source data may be very big if you have many customization options and it is kept in disk. A cache of this data is maintained in memory to speed up object construction. The size of this cache can be controlled by per-platform scalability settings as explained in the //Mutable settings// section below. - **Working memory**: This is the amount of data used while an object is being constructed, and it is freed while the system is idle. Mutable tries to minimize this amount of data, and its size depends entirely on the kind of objects to be built. - **Object memory**: This is the data used by the actual Unreal resources that compose an instance of a customizable object. Mutable will try to reuse textures that are identical, but other than this, these resources can be treated and measured exactly like any standard Unreal Skeletal Mesh or Texture. Memory can always be reduced by tuning the assets provided to Mutable. For example, lower polygon counts in meshes, lower resolution in the reference textures or fewer customizable parts. Lower quality LODs can reduce their memory usage by disconnecting less important features like earrings and projections/tattoos. If these strategies have already been used or they are not an option, here are some memory reduction techniques that Mutable offers: ==== Set a maximum radius of visibility of Customizable Characters ==== If too much memory is being used because there are just too many different characters using lots of different skeletal meshes and textures, and most of them are too far from the camera to appreciate their differences or are hidden from view, a viable option is to set a maximum visibility radius arround one or more view centers, for example the player. Only characters inside this radius will be generated and created, and the ones outside it will either be invisible or using the reference skeletal mesh. The API to control this functionality is located in the UCustomizableInstanceLODManagement class, accessible through the UCustomizableObjectSystem singleton with the following call: ``` lang=cpp auto LODManagmentBase = UCustomizableObjectSystem::GetInstance()->GetInstanceLODManagement(); ``` Cast the LODManagmentBase to a UCustomizableInstanceLODManagement to access the LOD system provided by Mutable. ``` lang=cpp if(auto MutableLODManagment = Cast<UCustomizableInstanceLODManagement>( LODManagmentBase) { } ``` The control functions for this functionality are as follows: To enable/disable the Customization visibility radius and set the actual radius use: ``` lang=cpp void EnableOnlyUpdateCloseCustomizableObjects(float CloseDist); void DisableOnlyUpdateCloseCustomizableObjects(); ``` By default characters outside the radius will receive a null skeletal mesh and will be invisible. If you prefer to replace them with a common generic skeletal mesh, use the next two functions. The generic mesh is the reference skeletal mesh defined in the properties of the base Customizable Object of the character. ``` lang=cpp void SetReplaceDiscardedWithReferenceMeshEnabled(bool bIsEnabled); bool IsReplaceDiscardedWithReferenceMeshEnabled() const; ``` The visibility radius is defined around the view center/s of the scene. By default it is the first player controller actor, and can be changed with the following calls: ``` lang=cpp void AddViewCenter(const AActor* const InViewCenter); void RemoveViewCenter(const AActor* const InViewCenter); void ClearAllViewCenters(); ``` Sometimes most characters are too far away from the camera to effectively use all levels of detail, especially the higher quality ones like LOD0 and LOD1. A lot of memory is wasted in detail that most of the time will not be seen until the player gets close to those characters. In this case it makes sense to limit how many high-quality LODs Mutable generates for these far-away characters. To adjust the number of generated LODs, Mutable provides different two methods. ==== Automatic adjustment of generated LODs per instance ==== Once the maximum display radius has been defined, an automatic system will come into operation to limit the LODs present in each instance according to the screen size of the meshes. The system works similarly to the rendering system. Mutable dynamically tracks all active instances in the game, if they are within the maximum radius of visibility it estimates the level of detail they use and, if necessary, issue a request to discard or add LODs. Using this system minimizes the memory usage used by instances without limiting the number of instances with all LODs. In cases where the density of unique instances is very high and the character moves frequently, there may be cases where the number of requests to generate instances with all LODs accumulates in the tasks queue. Since requests are processed based on the distance to the view centers, in these circumstances, it may take a few seconds to process updates for far-away instances. ==== Limit the number of instances with all LODs ==== (WARNING) {icon exclamation-triangle color=orange} This method disables the Automatic LOD system Another method of control the number of LODs generated in order to reduce the memory being used is to limit how many instances will have full LODS (LOD0 + LOD1 + LOD2), how many will have only two (LOD1 + LOD2) and finally how many will only have the low resolution LOD2 with minimal memory consumption. There are some scalability/console variables (b.NumGeneratedInstancesLimit*) to customize this option. See the Scalability settings section for more information about these variables. Similar to the automatic adjustment of generated LODs, Mutable will recalculate in each frame which level of detail belongs to each instance. In this case however the calculation will be based on the distance to the view centers and a the limit on the number of instances with full LODs. Only the closest instances to the view centers will get the full LODs. The next closest will get the two lower detail LODs and finally the rest will only get LOD2. If there are more instances than the sum of the values of b.NumGeneratedInstancesLimit*, the remaining furthest instances will not be generated and will be either invisible or replaced by the reference skeletal mesh. At every tick Mutable will recalculate which instances deserve each level of detail and automatically issue and enqueue updates to discard LODs for the instances that have gone too far from the camera and updates to generate LODs for the ones that have got closer to the camera. Although these updates are processed by prioritizing the closest instances to the view centers, if the characters frequently move relative to it they can generate a lot of LOD change updates and quickly fill-up the queue. If the CPU is busy it may take a few moments to process the queue and new updates can take a few seconds to become visible. In this case, LOD updates can make customization updates (ex: putting on a jacket) lag for a while. This can be prevented with the following functions accessed through the UCustomizableObjectSystem: ``` lang=cpp void SetNumberOfPriorityUpdateInstances(int32 NumPriorityUpdateInstances); int32 GetNumberOfPriorityUpdateInstances() const; ``` They can be used to set how many of the nearest instances to the player will have updates with priority over LOD changes, and thus update much faster. This is especially useful for multiplayer games with teams, where the player wants to see customization updates faster for teammates. ==== Reuse the memory resources of mutable textures during realtime parameter editing ==== By default, when a character changes a color in a texture or it moves a projector, the old texture is discarded and a new one is generated. The old texture's resources will not be released until the next Garbage Collection (GC) UE4 issues, so texture memory usage will go up until it happens. There are situations when a lot of textures will be created and destroyed in a short amount of time, for example in a customization lobby. The player moving a slider to control the color of a customizable object or moving a projector over a character's body will trigger a quick surge in texture usage until the next GC. Mutable offers an option to reuse textures for such situations on a per-instance basis. It is not enabled by default because there are certain precautions that must be taken for it to be safe imposed by UE4's texture system. It is extremely important not to keep references to the textures of the instances that have texture reuse enabled, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable. 1. First of all, make sure that the Customizable Object Instance is in a Mutable state where only the Mutable parameters of the currently edited object are being updated. See the section on "Customizable Object States" for more information. For instance, if a color slider is being edited, set the Mutable parameter that controls the color in the list of parameters of the Mutable state. Only use texture reuse in a customization lobby state, never in a gameplay state. In addition, make sure that this state disables texture compression and streaming. UE4 doesn't support the update of compressed and/or streaming textures. Something like the following must be called before enabling texture reuse: ``` lang=cpp CustomizableObjectInstance->SetCurrentState(FString("LobbyStateName_WhereOnlyOneParameterIsEdited")); ``` 2. Activate texture reuse for the lobby instance before any parameter manipulation: ``` lang=cpp CustomizableObjectInstance->SetReuseInstanceTextures(true); ``` 3. Remember to disable texture reuse after the parameter editing session: ``` lang=cpp CustomizableObjectInstance->SetReuseInstanceTextures(false); ``` ==== Immediately release mutable textures after instance destruction ==== As discussed in the previous section, by default textures created by Mutable cannot be released until a UE4 Garbage collection (GC) happens. In scenes where a lot of instances are being continuously created and destroyed or updated, there can be a spike in memory usage between GC events. Mutable has an option to enable the immediate release of texture memory upon texture/instance destruction without having to wait for GC. Like the previous section, it is not enabled by default because the programmer has to take some precautions in order to make it safe, but this time the requirements are less strict. Textures can be compressed and streamed, and the instances do not have to be set in a special state. The only thing have in mind is not to keep references to the textures of the instances, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable. If some texture or skeletal mesh generated by Mutable needs to be used in something that is not the character itself and attached to the Customizable Skeletal Component, for example in the UI or in any other copy, please make a full deep copy of the UObjects and resources that has no reference to the ones generated by Mutable. To enable the immediate release of texture resources call on the UCustomizableObjectSystem: ``` lang=cpp void SetReleaseMutableTexturesImmediately(bool bReleaseTextures) ``` === Mutable Settings === ==== General settings ==== These are settings that affect mutable in general ways. They can be defined in the DefaultMutable.ini file of your project. See the plugin Config folder for an example. * **Features** section | bExtraBoneInfluencesEnabled | If this is enabled, CustomizableObjects will be allowed to use 8 bone influences per vertex. Otherwise only the default 4 will be used. This is used at object compile time. | ====Scalability settings==== These settings can be modified by the [[ https://docs.unrealengine.com/en-US/Engine/Performance/Scalability/ScalabilityReference/index.html | standard Scalability ini file hierarchy in Unreal ]], or with Console Variables or game code. These are the variables for each section: * **ViewDistanceQuality** section | b.MutableStreamingMemory | If different than 0, limit the amount of memory (in KB) to use to cache streaming data when building characters. 0 means no limit, -1 means use default (40Mb in PC and 20Mb in consoles). | | b.NumGeneratedInstancesLimit | If different than 0, limit the number of mutable instances with full LODs to have at a given time. | | b.NumGeneratedInstancesLimitLOD1 | If different than 0, limit the number of mutable instances with LOD 1 to have at a given time. | | b.NumGeneratedInstancesLimitLOD2 | If different than 0, limit the number of mutable instances with LOD 2 to have at a given time. | | b.DistanceForFixedLOD2 | If NumGeneratedInstancesLimit is different than 0, sets the distance at which the system will fix the LOD of an instance to the lowest res one (LOD2) to prevent unnecessary LOD changes and memory | === Statistics widget === A good starting point to see the performance of Mutable in your game, is to use the provided statistics UMG widget. You can find it in the plugin //Content// folder with the name //UI/MutableStats//. {F85822} (WARNING) {icon exclamation-triangle color=orange} When using these statistics in the editor, the numbers will also capture all the instances built in all open levels, editors, and even content browser thumbnails, if enabled. If your project is not using UMG you can access these statistics and log them or display them from the central singleton object UCustomizableObjectSystem. These are some of the relevant methods: ``` lang=cpp // Get the singleton object. It will be created if it doesn't exist yet. UFUNCTION(BlueprintCallable, BlueprintPure, Category = Status) static UCustomizableObjectSystem* GetInstance(); // Find out if the performance engine patches have been applied UFUNCTION(BlueprintCallable, Category = Status) bool AreEnginePatchesPresent() const; // Find out the version of the plugin UFUNCTION(BlueprintCallable, Category = Status) FString GetPluginVersion() const; // Get the number of instances built and alive. UFUNCTION(BlueprintCallable, Category = Status) int32 GetNumInstances() const; // Get the number of instances waiting to be updated. UFUNCTION(BlueprintCallable, Category = Status) int32 GetNumPendingInstances() const; // Get the total number of instances including built and not built. UFUNCTION(BlueprintCallable, Category = Status) int32 GetTotalInstances() const; // Get the amount of memory in use for textures generated by mutable. UFUNCTION(BlueprintCallable, Category = Status) int64 GetTextureMemoryUsed() const; // Return the average build/update time of an instance in ms. UFUNCTION(BlueprintCallable, Category = Status) int32 GetAverageBuildTime() const; ``` === Troubleshooting an Object === There are 2 ways of looking in detail at objects generated by Mutable to see the resources they use, or try to spot why a desired effect is not achieved. * You can [[ mutable/unreal-engine-4/user-documentation/baking-instances/ | bake an instance ]] into standard Unreal assets. * You can open the preview mesh, material and textures by double-clicking the //Customizable Instance SkeletalMesh// in the //Preview Instance Properties// panel. {F103397} The second way is quicker, but since the dynamic preview assets don't exist in the Content Browser and disk, some editors may no behave as expected. In that case, we recommend using the baking option. === Customizable Object States === When an application uses complex Customizable Objects with many parameters, updating the instances may be a costly process that takes many milliseconds. However there are some usage scenarios for games that require these updates to be interactive, and big delays are not acceptable. In order to optimize your data to solve these problem, Mutable has the concept of States. A State represents a specific use case of the Customizable Object in your game. For example, at some point during character creation you may want to let the player customize the face and hair of a character. During this stage, you show a close up camera of the character head and display a user interface for the related parameters: hair colour, nose size, hair style, etc. During this stage, you will not be modifying other parameters, like t-shirt colour, torso tattoos, etc. In order for Mutable to provide maximum performance, you can create a State in your Customizable Object, indicating that the subset of parameters that you will modify in these stage, and the system will generate an optimised version of the data that updates faster. States also give you more options in order to optimise the instance construction time. For instance, in the previous example of the head customization stage, your game may have more graphic resources available because instead of being inside a level, you are in a smaller lobby scene. This means that you can afford to temporarily use more memory for your character. For each individual State Mutable gives you these optimisation options: * List of parameters that may be modified. * Option to avoid texture compression for textures that may change in this state. * Option to generate only LOD 0 of the object. States are created in any Base Object Node. If no state is created, a default state with no optimized parameters and sensible optimisation options will be automatically created. A normal game will want to create a "default" state to create customizable objects to be used in-game (no optimized parameters), and then several states to create and update objects in the different in-game customization scenarios. The editor user interface for preview instances has the object state selection combobox at the top.

== Integrating == === Architecture === The system is structured as an Unreal plugin with two additional Unreal modules: one for the editor and one for the runtime. These modules use external libraries for the Mutable runtime, which is a standalone library with no dependencies. {F85832, size=full} The next diagram shows the basic folder structure of an Unreal Engine game that uses Mutable. The hierarchy only shows the relevant folders for Mutable source code. {F85834, size=full} The folder “source” is optional (and by default not included with all licenses) and it contains the source code of the standalone runtime and tools libraries. These are already included in the “lib” folder as precompiled binaries for each supported platform, with the required headers in the “include” folder. === Packaging a project === Mutable will be automatically linked in your game executable. It doesn't require any additional files or dynamic-link libraries. In order to package the data for your Customizable Objects, an additional folders needs to be added in your project settings. The plugin will try to do this for you, but if you need to do it manually you can add "MutableStreamedData" to the list of "Additional non-asset directories to package" in the project settings. You can do this in the DefaultGame.ini file of your project: ``` +DirectoriesToAlwaysStageAsUFS=(Path="MutableStreamedData") ) ``` or from the project settings in the Unreal Editor: {F85836} === Engine Changes === In most versions of Mutable, some changes in the Unreal Engine codebase are required in order to improve the performance. These are modifications that cannot be implemented through plugins. In some platforms like Windows, Mutable will still work without the engine changes, but it may have lower performance and some editor features may be disabled, like automatic compilation of models when packaging. The engine modifications are distributed through a patch file that you can find with every release of the plugin. They are also available in a GitHub branch of the official engine repository, as per Epic Game's rules. In order to apply the patch you can use standard tools like TODO. === Compilation warnings {anchor #warnings} === The patch with engine changes includes some helpers to try to detect if any modification is missing. If these warnings are visible it is a sign that some of the changes have not been applied, or may have been overwritten by other changes or merges. These are for the runtime: ``` lang=c++ #define ANTICTO_MUTABLE_ENGINE_PATCH_TEXTURESTREAMING #define ANTICTO_MUTABLE_ENGINE_PATCH_VERTEXBUFFERS #define ANTICTO_MUTABLE_ENGINE_PATCH_MORPHTARGETVERTEX ``` These are for the editor: ``` lang=c++ #define ANTICTO_MUTABLE_ENGINE_PATCH_COOKHOOK ``` It's not mandatory to manually use these defines but if they are not present, the plugin will show warnings at compile time. {icon exclamation-triangle color=orange} Please note that these are just simple preprocessor defines, so they do not guarantee that the actual changes are present, but capture most error cases. --- == Using Mutable from C++ {anchor #cpp} == === Creating an Instance of a CustomizableObject === A CustomizableObjectInstance contains a set of values for the parameters of a CustomizableObject, these values can be used by actors to create in-game meshes and materials. To create an instance of a CustomizableObject, we need a reference to the Object and a CustomizableObjectInstance. Getting the Object reference can be done either by adding a pointer as UPROPERTY() to the actor’s class and then setting the value from blueprints or by loading it using the asset’s path. Having both, then calling the “SetObject” function tells the instance which object to represent. This way the instance knows what parameters to store and what their default values are. ``` lang=cpp UCustomizableObject* CustomizableObject = LoadObject<UCustomizableObject>(nullptr, TEXT("/Game/MyCustomizableObject")); if(CustomizableObject) { CustomizableObjectInstance = NewObject<UCustomizableObjectInstance>(); CustomizableObjectInstance->SetObject(CustomizableObject); } ``` === Connecting Instances with Actors === The best way to link an instance to an actor is to use the CustomizableSkeletalComponent, which can be attached to the actor’s SkeletalMeshComponent in order to update and replace the mesh asset with a Mutable-generated mesh. Creating the component as shown below, when the actor is added to a level, it will automatically show the customized skeletal mesh. ``` lang=cpp CSkeletalComponent = NewObject<UCustomizableSkeletalComponent>(UCustomizableSkeletalComponent::StaticClass()); // Set the instance that will be used by the actor CSkeletalComponent->CustomizableObjectInstance = CustomizableObjectInstance; // Attach the CustomizableSkeletalComponent to the Actor's SkeletalMeshComponent CSkeletalComponent->AttachToComponent(GetMesh(), FAttachmentTransformRules::KeepRelativeTransform); ``` === Changing Parameters === Parameters are stored in public arrays within each instance. These parameters can be modified directly, although it is recommended to use the API functions to avoid invalid values. Below is an example of how to modify different types of parameters. ``` lang=cpp // Sets the bool value of the parameter "Frackles" CustomizableObjectInstance->SetBoolParameterSelectedOption(FString("Freckles"), true); // Sets the int value of the parameter "Shirt" (Enum Type) CustomizableObjectInstance->SetIntParameterSelectedOption(FString("Shirt"), FString("BasicShirt")); // Sets the float value of the parameter "Fatness", range from 0 to 1 CustomizableObjectInstance->SetFloatParameterSelectedOption(FString("Fatness"), 0.5f); // Sets the color value of the parameter "EyeColor" CustomizableObjectInstance->SetColorParameterSelectedOption(FString("EyeColor"), FLinearColor(FColor::Blue)); // Sets the vector value of the parameter "VParam" CustomizableObjectInstance->SetVectorParameterSelectedOption(FString("VParam"), FLinearColor(120.f, 50.f, 180.f)); // Sets the projectors values of the parameter "Tatto" CustomizableObjectInstance->SetProjectorValue(FString("Tatto"), LocalPosition, Direction, Up, Scale, Angle, ECustomizableObjectProjectorType::Planar); ``` === Updating Instances === Changing parameters does not update the Instance automatically. To apply new values, the instance must be updated by calling the UpdateSkeletalMeshAsync() method, within the CustomizableSkeletalComponent class. Doing so will replace the SkeletalMesh of all actors using the same CustomizableObjectInstance with the generated from the new parameter values. ``` lang=cpp // Update Instances CSkeletalComponent->UpdateSkeletalMeshAsync(); ``` Here is a diagram of what goes on "under the hood" when an update is taking place. Most of the CPU work is done in Mutable's own thread, and for most projects most of the time is spent waiting for assets to stream from disk. {F161570, size=full} If your project's CPU threads or disk are highly saturated all the time, it may slow down the updates of Mutable characters because of resource contention. === Instance Updated Delegate === Sometimes, you might want to run a specific method after the mesh has been updated, either to trigger an animation, make the mesh visible, etc. To make it easy, Mutable provides a delegate where callbacks can be registered. This delegate will broadcast the completion of the skeletal mesh update to the registered callbacks. ``` lang=cpp // Bind "this" UObject's "OnCustomizableSkeletalUpdated" method as a callback for the delegate CSkeletalComponent->UpdatedDelegate.BindUObject(this, &MyCustomCharacter::OnCustomizableSkeletaUpdated); --- // Unbind callbacks CSkeletalComponent->UpdatedDelegate.Unbind(); ``` --- == Using Mutable from Blueprint {anchor #blueprint} == === Creating a Customizable Character {anchor #create-character} === 1. Create a new blueprint with "character" as parent class and open it. Alternatively, you can use a class/BP that has a skeletal mesh component. {F102097} 2. In the blueprint editor, select the Mesh component and add a CustomizableSkeletal component as a child. {F102099} 3. Having selected the newly created component, go to the details panel and chose the instance to be used. {F102105} The reference mesh or the custom mesh, if generated, will show up in the viewport. {F102109} === Changing Parameters === Parameters are stored by instances and can be accessed or modified using the API nodes. Here are examples on how to set parameter values based on their type: * Boolean Parameter {F102123} * Int Parameter: It's important to make sure that the desired option actually exists within the instance. Search existing parameters using the FindParameter node and then get the available option with GetIntParameterAvailableOption. Both nodes have the CustomizableObject as target, which can be accessed through the CustomizableObjectInstance. {F102113} * Float Parameter {F102119} * Color Parameter {F102129} * Projector Parameter {F102131} === Update Instances === To apply recent changes in parameters, the instance needs to be updated. This can be achieved by adding an UpdateSkeletalMeshAsync node after one or more changes. {F102256} === Updated Delegate === Events can be registered to this delegate. The broadcasting will take place after the completion of the skeletal mesh update. * Bind Event {F102368} * Unbind Event {F102370} === Parameter Information === Sometimes, additional information like the amount of parameters within an instance, their type or the name of an int parameter's option might come in handy. This information is kept in the source CustomizableObject, accessible through the instance, and can be retrieved using the following nodes. * Get Parameter Count {F102356} * Get Parameter Name {F102358} * Get Parameter Type by Name {F102360} * Find Parameter {F102362} * Get Int Parameter Num Options {F102364} * Get Int Parameter Available Option {F102366} === Changing States === As will be shown in a section below, Mutable has the concept of states that enable certain optimizations depending on the use case. The state an instance is in can be queried and changed using the node API. * Set Current State Similarly to when changing parameters, the instance needs to be updated with a UpdateSkeletalMeshAsync node after a state change. {F161529} * Get Current State {F161531} === State Information === Information about the number and name of states a CustomizableObject has and the number and names of parameters in a state can be useful. This information is stored per CustomizableObject, accessible through a CustomizableObjectInstance and can be retrieved using the following nodes. * Get State Count {F161543} * Get State Name {F161539} * Get State Parameter Count {F161533} * Get State Parameter Name {F161535} --- == Performance tuning == {anchor #performance} === Memory usage === Mutable uses memory mainly in 3 different ways: - **Streaming of source mutable data** : The data required to build instances of customizable objects needs to be loaded from disk. This data is then processed into a set of skeletal meshes and textures that compose the final object. The source data may be very big if you have many customization options and it is kept in disk. A cache of this data is maintained in memory to speed up object construction. The size of this cache can be controlled by per-platform scalability settings as explained in the //Mutable settings// section below. - **Working memory**: This is the amount of data used while an object is being constructed, and it is freed while the system is idle. Mutable tries to minimize this amount of data, and its size depends entirely on the kind of objects to be built. - **Object memory**: This is the data used by the actual Unreal resources that compose an instance of a customizable object. Mutable will try to reuse textures that are identical, but other than this, these resources can be treated and measured exactly like any standard Unreal Skeletal Mesh or Texture. Memory can always be reduced by tuning the assets provided to Mutable. For example, lower polygon counts in meshes, lower resolution in the reference textures or fewer customizable parts. Lower quality LODs can reduce their memory usage by disconnecting less important features like earrings and projections/tattoos. If these strategies have already been used or they are not an option, here are some memory reduction techniques that Mutable offers: ==== Set a maximum radius of visibility of Customizable Characters ==== If too much memory is being used because there are just too many different characters using lots of different skeletal meshes and textures, and most of them are too far from the camera to appreciate their differences or are hidden from view, a viable option is to set a maximum visibility radius arround one or more view centers, for example the player. Only characters inside this radius will be generated and created, and the ones outside it will either be invisible or using the reference skeletal mesh. The API to control this functionality is located in the UCustomizableInstanceLODManagement class, accessible through the UCustomizableObjectSystem singleton with the following call: ``` lang=cpp auto LODManagmentBase = UCustomizableObjectSystem::GetInstance()->GetInstanceLODManagement(); ``` Cast the LODManagmentBase to a UCustomizableInstanceLODManagement to access the LOD system provided by Mutable. ``` lang=cpp if(auto MutableLODManagment = Cast<UCustomizableInstanceLODManagement>( LODManagmentBase) { } ``` The control functions for this functionality are as follows: To enable/disable the Customization visibility radius and set the actual radius use: ``` lang=cpp void EnableOnlyUpdateCloseCustomizableObjects(float CloseDist); void DisableOnlyUpdateCloseCustomizableObjects(); ``` By default characters outside the radius will receive a null skeletal mesh and will be invisible. If you prefer to replace them with a common generic skeletal mesh, use the next two functions. The generic mesh is the reference skeletal mesh defined in the properties of the base Customizable Object of the character. ``` lang=cpp void SetReplaceDiscardedWithReferenceMeshEnabled(bool bIsEnabled); bool IsReplaceDiscardedWithReferenceMeshEnabled() const; ``` The visibility radius is defined around the view center/s of the scene. By default it is the first player controller actor, and can be changed with the following calls: ``` lang=cpp void AddViewCenter(const AActor* const InViewCenter); void RemoveViewCenter(const AActor* const InViewCenter); void ClearAllViewCenters(); ``` Sometimes most characters are too far away from the camera to effectively use all levels of detail, especially the higher quality ones like LOD0 and LOD1. A lot of memory is wasted in detail that most of the time will not be seen until the player gets close to those characters. In this case it makes sense to limit how many high-quality LODs Mutable generates for these far-away characters. To adjust the number of generated LODs, Mutable provides different two methods. ==== Automatic adjustment of generated LODs per instance ==== Once the maximum display radius has been defined, an automatic system will come into operation to limit the LODs present in each instance according to the screen size of the meshes. The system works similarly to the rendering system. Mutable dynamically tracks all active instances in the game, if they are within the maximum radius of visibility it estimates the level of detail they use and, if necessary, issue a request to discard or add LODs. Using this system minimizes the memory usage used by instances without limiting the number of instances with all LODs. In cases where the density of unique instances is very high and the character moves frequently, there may be cases where the number of requests to generate instances with all LODs accumulates in the tasks queue. Since requests are processed based on the distance to the view centers, in these circumstances, it may take a few seconds to process updates for far-away instances. ==== Limit the number of instances with all LODs ==== (WARNING) {icon exclamation-triangle color=orange} This method disables the Automatic LOD system Another method of control the number of LODs generated in order to reduce the memory being used is to limit how many instances will have full LODS (LOD0 + LOD1 + LOD2), how many will have only two (LOD1 + LOD2) and finally how many will only have the low resolution LOD2 with minimal memory consumption. There are some scalability/console variables (b.NumGeneratedInstancesLimit*) to customize this option. See the Scalability settings section for more information about these variables. Similar to the automatic adjustment of generated LODs, Mutable will recalculate in each frame which level of detail belongs to each instance. In this case however the calculation will be based on the distance to the view centers and a the limit on the number of instances with full LODs. Only the closest instances to the view centers will get the full LODs. The next closest will get the two lower detail LODs and finally the rest will only get LOD2. If there are more instances than the sum of the values of b.NumGeneratedInstancesLimit*, the remaining furthest instances will not be generated and will be either invisible or replaced by the reference skeletal mesh. At every tick Mutable will recalculate which instances deserve each level of detail and automatically issue and enqueue updates to discard LODs for the instances that have gone too far from the camera and updates to generate LODs for the ones that have got closer to the camera. Although these updates are processed by prioritizing the closest instances to the view centers, if the characters frequently move relative to it they can generate a lot of LOD change updates and quickly fill-up the queue. If the CPU is busy it may take a few moments to process the queue and new updates can take a few seconds to become visible. In this case, LOD updates can make customization updates (ex: putting on a jacket) lag for a while. This can be prevented with the following functions accessed through the UCustomizableObjectSystem: ``` lang=cpp void SetNumberOfPriorityUpdateInstances(int32 NumPriorityUpdateInstances); int32 GetNumberOfPriorityUpdateInstances() const; ``` They can be used to set how many of the nearest instances to the player will have updates with priority over LOD changes, and thus update much faster. This is especially useful for multiplayer games with teams, where the player wants to see customization updates faster for teammates. ==== Reuse the memory resources of mutable textures during realtime parameter editing ==== By default, when a character changes a color in a texture or it moves a projector, the old texture is discarded and a new one is generated. The old texture's resources will not be released until the next Garbage Collection (GC) UE4 issues, so texture memory usage will go up until it happens. There are situations when a lot of textures will be created and destroyed in a short amount of time, for example in a customization lobby. The player moving a slider to control the color of a customizable object or moving a projector over a character's body will trigger a quick surge in texture usage until the next GC. Mutable offers an option to reuse textures for such situations on a per-instance basis. It is not enabled by default because there are certain precautions that must be taken for it to be safe imposed by UE4's texture system. It is extremely important not to keep references to the textures of the instances that have texture reuse enabled, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable. 1. First of all, make sure that the Customizable Object Instance is in a Mutable state where only the Mutable parameters of the currently edited object are being updated. See the section on "Customizable Object States" for more information. For instance, if a color slider is being edited, set the Mutable parameter that controls the color in the list of parameters of the Mutable state. Only use texture reuse in a customization lobby state, never in a gameplay state. In addition, make sure that this state disables texture compression and streaming. UE4 doesn't support the update of compressed and/or streaming textures. Something like the following must be called before enabling texture reuse: ``` lang=cpp CustomizableObjectInstance->SetCurrentState(FString("LobbyStateName_WhereOnlyOneParameterIsEdited")); ``` 2. Activate texture reuse for the lobby instance before any parameter manipulation: ``` lang=cpp CustomizableObjectInstance->SetReuseInstanceTextures(true); ``` 3. Remember to disable texture reuse after the parameter editing session: ``` lang=cpp CustomizableObjectInstance->SetReuseInstanceTextures(false); ``` ==== Immediately release mutable textures after instance destruction ==== As discussed in the previous section, by default textures created by Mutable cannot be released until a UE4 Garbage collection (GC) happens. In scenes where a lot of instances are being continuously created and destroyed or updated, there can be a spike in memory usage between GC events. Mutable has an option to enable the immediate release of texture memory upon texture/instance destruction without having to wait for GC. Like the previous section, it is not enabled by default because the programmer has to take some precautions in order to make it safe, but this time the requirements are less strict. Textures can be compressed and streamed, and the instances do not have to be set in a special state. The only thing have in mind is not to keep references to the textures of the instances, either directly through a pointer to the texture or indirectly through references to the skeletal mesh or materials generated by the instance. In other words, the Customizable Object Instance owns the textures and the generated skeletal mesh should be treated as a black box that is managed by Mutable. If some texture or skeletal mesh generated by Mutable needs to be used in something that is not the character itself and attached to the Customizable Skeletal Component, for example in the UI or in any other copy, please make a full deep copy of the UObjects and resources that has no reference to the ones generated by Mutable. To enable the immediate release of texture resources call on the UCustomizableObjectSystem: ``` lang=cpp void SetReleaseMutableTexturesImmediately(bool bReleaseTextures) ``` === Mutable Settings === ==== General settings ==== These are settings that affect mutable in general ways. They can be defined in the DefaultMutable.ini file of your project. See the plugin Config folder for an example. * **Features** section | bExtraBoneInfluencesEnabled | If this is enabled, CustomizableObjects will be allowed to use 8 bone influences per vertex. Otherwise only the default 4 will be used. This is used at object compile time. | ====Scalability settings==== These settings can be modified by the [[ https://docs.unrealengine.com/en-US/Engine/Performance/Scalability/ScalabilityReference/index.html | standard Scalability ini file hierarchy in Unreal ]], or with Console Variables or game code. These are the variables for each section: * **ViewDistanceQuality** section | b.MutableStreamingMemory | If different than 0, limit the amount of memory (in KB) to use to cache streaming data when building characters. 0 means no limit, -1 means use default (40Mb in PC and 20Mb in consoles). | | b.NumGeneratedInstancesLimit | If different than 0, limit the number of mutable instances with full LODs to have at a given time. | | b.NumGeneratedInstancesLimitLOD1 | If different than 0, limit the number of mutable instances with LOD 1 to have at a given time. | | b.NumGeneratedInstancesLimitLOD2 | If different than 0, limit the number of mutable instances with LOD 2 to have at a given time. | | b.DistanceForFixedLOD2 | If NumGeneratedInstancesLimit is different than 0, sets the distance at which the system will fix the LOD of an instance to the lowest res one (LOD2) to prevent unnecessary LOD changes and memory | === Statistics widget === A good starting point to see the performance of Mutable in your game, is to use the provided statistics UMG widget. You can find it in the plugin //Content// folder with the name //UI/MutableStats//. {F85822} (WARNING) {icon exclamation-triangle color=orange} When using these statistics in the editor, the numbers will also capture all the instances built in all open levels, editors, and even content browser thumbnails, if enabled. If your project is not using UMG you can access these statistics and log them or display them from the central singleton object UCustomizableObjectSystem. These are some of the relevant methods: ``` lang=cpp // Get the singleton object. It will be created if it doesn't exist yet. UFUNCTION(BlueprintCallable, BlueprintPure, Category = Status) static UCustomizableObjectSystem* GetInstance(); // Find out if the performance engine patches have been applied UFUNCTION(BlueprintCallable, Category = Status) bool AreEnginePatchesPresent() const; // Find out the version of the plugin UFUNCTION(BlueprintCallable, Category = Status) FString GetPluginVersion() const; // Get the number of instances built and alive. UFUNCTION(BlueprintCallable, Category = Status) int32 GetNumInstances() const; // Get the number of instances waiting to be updated. UFUNCTION(BlueprintCallable, Category = Status) int32 GetNumPendingInstances() const; // Get the total number of instances including built and not built. UFUNCTION(BlueprintCallable, Category = Status) int32 GetTotalInstances() const; // Get the amount of memory in use for textures generated by mutable. UFUNCTION(BlueprintCallable, Category = Status) int64 GetTextureMemoryUsed() const; // Return the average build/update time of an instance in ms. UFUNCTION(BlueprintCallable, Category = Status) int32 GetAverageBuildTime() const; ``` === Troubleshooting an Object === There are 2 ways of looking in detail at objects generated by Mutable to see the resources they use, or try to spot why a desired effect is not achieved. * You can [[ mutable/unreal-engine-4/user-documentation/baking-instances/ | bake an instance ]] into standard Unreal assets. * You can open the preview mesh, material and textures by double-clicking the //Customizable Instance SkeletalMesh// in the //Preview Instance Properties// panel. {F103397} The second way is quicker, but since the dynamic preview assets don't exist in the Content Browser and disk, some editors may no behave as expected. In that case, we recommend using the baking option. === Customizable Object States === When an application uses complex Customizable Objects with many parameters, updating the instances may be a costly process that takes many milliseconds. However there are some usage scenarios for games that require these updates to be interactive, and big delays are not acceptable. In order to optimize your data to solve these problem, Mutable has the concept of States. A State represents a specific use case of the Customizable Object in your game. For example, at some point during character creation you may want to let the player customize the face and hair of a character. During this stage, you show a close up camera of the character head and display a user interface for the related parameters: hair colour, nose size, hair style, etc. During this stage, you will not be modifying other parameters, like t-shirt colour, torso tattoos, etc. In order for Mutable to provide maximum performance, you can create a State in your Customizable Object, indicating that the subset of parameters that you will modify in these stage, and the system will generate an optimised version of the data that updates faster. States also give you more options in order to optimise the instance construction time. For instance, in the previous example of the head customization stage, your game may have more graphic resources available because instead of being inside a level, you are in a smaller lobby scene. This means that you can afford to temporarily use more memory for your character. For each individual State Mutable gives you these optimisation options: * List of parameters that may be modified. * Option to avoid texture compression for textures that may change in this state. * Option to generate only LOD 0 of the object. States are created in any Base Object Node. If no state is created, a default state with no optimized parameters and sensible optimisation options will be automatically created. A normal game will want to create a "default" state to create customizable objects to be used in-game (no optimized parameters), and then several states to create and update objects in the different in-game customization scenarios. The editor user interface for preview instances has the object state selection combobox at the top.