Skip to content

How To

Jump to bottom
Pravus edited this page Sep 25, 2024 · 21 revisions

Design systems

Systems must be the only authoring point for any logic executed on entities. It's strictly forbidden to initiate any manipulation with entities outside of systems. Though, they can call into their dependencies and pieces of logic isolated in their own files and classes.

Systems can't contain any collections of components or entities that persist through multiple frames: everything should be stored in ECS directly.

Systems can contain temporary collections used for data aggregation: take a look at DeferredLoadingSystem.cs.

Systems can't contain a state. All states should be written and stored in ECS Worlds.

How to construct

Systems should have an internal constructor, it clearly indicates that we can't instantiate the system directly but are obliged to use ArchSystemsWorldBuilder.

Systems may accept shared dependencies in the .ctor such as:

  • Settings that apriori exist in a single instance: Quality, Partitioning, etc.
  • Pool Providers
  • Utility functionality (that for some reason is not static)
  • Configuration dedicated to the given system only, strategies and factories that are injected from the upper level: e.g. IConcurrentBudgetProvider

Every system should be inherited from BaseUnityLoopSystem: it provides common functionality for profiling and error reporting.

Follow Single Resposibility

Every system should execute a limited scope of responsibilities. It should be reflected in its name. There is no strict rule of how many queries it should have but if it grows beyond 200 lines of code consider splitting it into static counterparts.

Normally, every feature is represented by multiple systems that are bound by a certain execution order.

Distribute in groups

Decide in which Game Loop moment (SystemGroup) it will be executed. It purely depends on the system's designation, e.g.:

  • Physics manipulation should happen in PhysicsSystemGroup
  • Actions based on Transform.position or Transform.rotation - in PresentationSystemGroup as it is executed after transformation is applied in Unity

Consider creating your own group for a given feature: it will simplify defining dependencies between other groups and systems

How to write queries

There are four ways of writing queries:

  1. Automatic generation is the most preferred one. It can be used in the systems only. But if you have a generic system it's impossible to use it as generic attributes are not supported in the version of C# used in Unity.
  2. Iterating over chunks manually: GetChunkIterator(). The same code is generated by the source generator. You can consider this option in a generic class in some special cases.
  3. World.InlineQuery can be used outside of the system itself and in generic cases. Its performance is very close to generated queries. See ReleasePoolableComponentSystem<T, TProvider> for a reference.
  4. World.Query is the least preferred way of doing things as it uses delegates and can lead to closures unintentionally.

Good practices

  • Filter out by DeleteEntityIntention: it's undesirable to execute logic over entities marked for destruction

Performance Implication

System's Update should be allocation-free. In order to ensure this consider profiling before sending a feature for review.

When you define a system that operates in a scene context (not a global world) there will be as many instances of this system as worlds are loaded. Thus its Update may be executed many times in the same frame. You should keep the logic as simple as possible so every step of the system takes negligible time.

Every query produces an overhead. Try avoiding introducing multiple queries in the same system with the same filter. Invoke several different methods from one handler instead:

  • e.g. take a look at CalculateCharacterVelocitySystem: it uses a single entry point ResolveVelocity to calculate every kind of velocity in isolated pieces of logic: ApplyCharacterMovementVelocity, ApplyJump, ApplyGravity, ApplyAirDrag, etc.
  • another example is FinalizeGltfContainerLoadingSystem: in FinalizeLoading the static method ConfigureGltfContainerColliders.SetupColliders is called to execute logic encapsulated in its own class

Throttling

In order to optimize it further there is a concept of throttling: Systems registered in a Scene World do not execute unless there is a CRDT change from the JavaScript scene. This behavior is implemented in SystemGroupsUpdateGate.cs.

Throttling must be enabled manually by annotating with ThrottlingEnabled attribute. Not every system is suitable for throttling: for example Promises resolution should happen as soon as data is ready.

Enabling Throttling will significantly relieve CPU pressure.

How to manipulate components

World.Get API and Queries provide a ref access to the component. It makes it possible to modify a value type directly without the necessity of setting it back.

⚠️ You must use ref var, otherwise the value will be copied and changes won't be reflected, e.g.: ref var meshRendererComponent = ref world.Get<PrimitiveMeshRendererComponent>(entity);

⚠️ A severe ECS pitfall you may fall into: E.g. you have a query

        protected void TestQuery(in Entity entity, ref StreamableLoadingState state)
        {
            World.Add(entity, new StreamableLoadingResult<TAsset>());
            state.Value = StreamableLoadingState.Status.Finished;
        }

state.Value = StreamableLoadingState.Status.Finished; will not apply the change to the value you expect because you make a structural change (World.Add) before that line and moving between archetypes invalidates ref StreamableLoadingState state

You should be very cautious and apply all structural changes last!

It's super hard to detect as ref StreamableLoadingState state will not throw any exception but will silently point to another cell (in fact the same cell but affected by memcpy) in the reserved array in the archetype's chunk.

So the change will apply eventually to an indefinite component: lucky you if it is just an empty reserved cell but it can be also another valid entity that will be modified accidentally!

⚠️ Passing Entity by reference

E.g. you have a query

        protected void TestQuery(in Entity entity)
        {
            World.Add(entity, new Component1());
            World.Add(entity, new Component2());
        }

Component1 will be properly added according to World.Add(entity, new Component1()). However it leads to a structural change and in Entity entity will start to refer to another region of memory so the second operation won't be executed properly World.Add(entity, new Component2()).

Yet it is possible to execute ir properly if Entity is passed as a value:

        protected void TestQuery(Entity entity)
        {
            World.Add(entity, new Component1());
            World.Add(entity, new Component2());
        }

In this case Entity is copied to the stack and won't be modified after the first structural change.

Mutable vs immutable reference

  • You should clarify your intentions: if you expect your method to modify the value passed you should use ref modifier. Otherwise use in modifier. Thus, you state explicitly whether your logic changes the original value or not. In both cases value type instance won't be copied and will be passed by reference.
  • It's possible to declare ref readonly var to specify immutability of the reference. E.g. ref readonly var c = ref world.Get<PromiseTrackingInfoComponent>(entity);

Reference vs Value Copy

  • Reference size is equal to the OS' bit-depth. Nowadays we don't consider 32-bit systems at all, so the size is 64 bits
  • It does not make sense to pass by reference any structures of size equal or below that, e.g. Entity contains a single int id so its size is 32 bits. Structures slightly above 64 bits are handled by stack with ease as well
  • Stack performance can struggle significantly if large memory blocks are copied, it's noticeable in Profiler as string.memcpy. A good example is MaterialData that contains a lot of fields and nested structures. In case you have no intentions to modify the original value prefer using in modifier instead

⚠️ Every Protobuf component needs to be registered at the ComponentsContainer in order to be correctly detected by systems

How to use "IsDirty" property on Protobuf Components

The partial definition of the specific Protobuf Component should be defined at IDirtyMarker.

ResetDirtyFlagSystem<T>.InjectToWorld(ref builder); should be called before the pertinent System InjectToWorld() call

How to clean up components

Cleaning up can result in (but not limited to) the following actions:

  • Returning to the pool

    E.g.:

    if (poolsRegistry.TryGetPool(component.ColliderType, out IComponentPool componentPool))
              componentPool.Release(component.Collider);

    in ReleaseOutdatedColliderSystem.cs

  • Invalidating previously created Promises. Promises represent asynchronous loading operations. On component cleaning-up, you must ensure they don't leak and are properly interrupted.

    E.g.:

    component.Promise.ForgetLoading(world);

    in CleanUpGltfContainerSystem

  • Dereferencing assets in a corresponding cache. Check Resources Unloading for more details.

          private void TryReleaseAsset(ref GltfContainerComponent component)
      {
          if (component.Promise.TryGetResult(World, out StreamableLoadingResult<GltfContainerAsset> result) && result.Succeeded)
          {
              cache.Dereference(component.Source, result.Asset);
              entityCollidersSceneCache.Remove(result.Asset);
          }
      }

    in ResetGltfContainerSystem.cs

  • Any custom logic

    entityCollidersSceneCache.Remove(result.Asset); to sync available colliders in the scene.

    In AvatarInstantiatorSystem.cs complex logic connected to the Custom Skinning:

          private void InternalDestroyAvatar(ref AvatarShapeComponent avatarShapeComponent,
          ref AvatarCustomSkinningComponent skinningComponent, ref AvatarTransformMatrixComponent avatarTransformMatrixComponent,
          AvatarBase avatarBase)
      {
          CommonAvatarRelease(avatarShapeComponent, skinningComponent);
          avatarTransformMatrixComponent.Dispose();
          avatarPoolRegistry.Release(avatarBase);
      }

When to clean up components

  • When the component is removed

    Component removal is initiated by the JavaScript scene logic. The common pattern to detect it is to apply an ECS query in the following scheme:

    • The original SDK (Protobug) Component no longer exists. It will only happen when the scene deletes the component. When the whole scene is unloaded it's not the case
    • A purely client-side component complementing an SDK one does
    • DeleteEntityIntention does not as it should be handled by another query

    E.g:

       [Query]
   [None(typeof(PBMeshRenderer), typeof(DeleteEntityIntention))]
   private void HandleComponentRemoval(ref PrimitiveMeshRendererComponent rendererComponent)
   {
       ReleaseMaterial.TryReleaseDefault(ref rendererComponent);

       if (poolsRegistry.TryGetPool(rendererComponent.PrimitiveMesh.GetType(), out IComponentPool componentPool))
           componentPool.Release(rendererComponent.PrimitiveMesh);
   }

  • When the entity is destroyed

    Entity destruction is initiated by the JavaScript scene logic.

    It can be detected by the presence of DeleteEntityIntention component. DeleteEntityIntention is not placed on any entities when the whole scene goes out of scope. DeleteEntityIntention survives only 1 frame. The automatic destruction of marked entities is performed in DestroyEntitiesSystem

    E.g:

          [Query]
      [All(typeof(DeleteEntityIntention))]
      private void TryRelease(ref MaterialComponent materialComponent)
      {
          ReleaseMaterial.Execute(World, ref materialComponent, destroyMaterial);
      }

    In some scenarios, clean-up logic may be heavy and can produce hiccups. In this case it's advisable to rely on frame-time IConcurrentBudgetProvider. If the budget is not available (and thus the clean-up logic is not executed) it's necessary to set the flag DeferDeletion on the DeleteEntityIntention component so the entity will survive.

    E.g:

          [Query]
      private void DestroyAvatar(ref AvatarShapeComponent avatarShapeComponent, ref AvatarTransformMatrixComponent avatarTransformMatrixComponent,
          AvatarBase avatarBase, AvatarCustomSkinningComponent skinningComponent, ref DeleteEntityIntention deleteEntityIntention)
      {
          // Use frame budget for destruction as well
          if (!instantiationFrameTimeBudgetProvider.TrySpendBudget())
          {
              avatarBase.gameObject.SetActive(false);
              deleteEntityIntention.DeferDeletion = true;
              return;
          }

          InternalDestroyAvatar(ref avatarShapeComponent, ref skinningComponent, ref avatarTransformMatrixComponent, avatarBase);
          deleteEntityIntention.DeferDeletion = false;
      }

  • When the whole world is disposed of

    Disposal of the world may happen when the player gets far enough away from the scene.

    The only way to detect is to implement IFinalizeWorldSystem and add into the finalizeWorldSystems list on the world creation in a plugin.

    E.g in GltfContainerPlugin:

    var cleanUpGltfContainerSystem =
              CleanUpGltfContainerSystem.InjectToWorld(ref builder, assetsCache, sharedDependencies.EntityCollidersSceneCache);

finalizeWorldSystems.Add(cleanUpGltfContainerSystem);

    void FinalizeComponents(in Query query); of IFinalizeWorldSystem:

    • Currently, this method is ensured to be called on the main thread. However in the future for performance reason, we may revise it. So try to design the implementation in a thread-safe manner
    • query corresponds to all entities with CRDTEntity component. Namely all SDK entities. You can safely ignore it
    • You can provide your own query for the component of your interest

    E.g

    private static readonly QueryDescription ENTITY_DESTROY_QUERY = new QueryDescription()
         .WithAll<DeleteEntityIntention, GltfContainerComponent>();

    from the previous example

Generalized scenarios

Some clean-up behaviour is common enough so it's generalized and can be reused across different components.

  • Returning reference-type components to the pool

    All SDK components and some custom ones can be registered in ComponentPoolsRegistry. Then all components that correspond to SDK entities are grabbed by ReleaseReferenceComponentsSystem:

    • This system automatically returns them to the pool when the entity gets destroyed and the scene dies (by implementing IFinalizeWorldSystem)
    • This system does not return an SDK component to the pool when the component is removed by the scene (as it does not have knowledge about the client-side counterpart based on which it can infer if the component was ever processed). It should be done by custom code.

    For all SDK (Protobuf) components Get and Release behaviour is pretty general and provided by the following extensions:

    public static SDKComponentBuilder<T> WithPool<T>(this SDKComponentBuilder<T> sdkComponentBuilder, Action<T> onGet = null, Action<T> onRelease = null) where T: class, new()
      {
          sdkComponentBuilder.pool = new ComponentPool<T>(onGet: onGet, onRelease: onRelease);
          return sdkComponentBuilder;
      }

      /// <summary>
      ///     Provide a custom pool behavior for SDK components, it is a must
      /// </summary>
      public static SDKComponentBuilder<T> WithPool<T>(this SDKComponentBuilder<T> sdkComponentBuilder, IComponentPool<T> componentPool) where T: class, new()
      {
          sdkComponentBuilder.pool = componentPool;
          return sdkComponentBuilder;
      }

      /// <summary>
      ///     A shortcut to create a standard suite for Protobuf components
      /// </summary>
      /// <returns></returns>
      public static SDKComponentBridge AsProtobufComponent<T>(this SDKComponentBuilder<T> sdkComponentBuilder)
          where T: class, IMessage<T>, IDirtyMarker, new() =>
          sdkComponentBuilder.WithProtobufSerializer()
                             .WithPool(SetAsDirty)
                             .Build();

    But it's useful to keep in mind that you can provide custom Get and Release behaviour for components being pooled without introducing a whole new system or query. Though the behavior should be simple and "static", and can rely on the data of that component only. For that the following methods exist

    public static class ComponentPoolsRegistryExtensions
  {
      public static void AddComponentPool<T>(this IComponentPoolsRegistry componentPoolsRegistry, Action<T> onGet = null, Action<T> onRelease = null) where T: class, new()
      {
          componentPoolsRegistry.AddComponentPool(new ComponentPool<T>(onGet, onRelease));
      }
  }
...
public interface IComponentPoolsRegistry
  { 
      void AddGameObjectPool<T>(Func<T> creationHandler = null, Action<T> onRelease = null, int maxSize = 1024) where T: Component;

      void AddGameObjectPoolDCL<T>(Func<T> creationHandler = null, Action<T> onRelease = null, int maxSize = 1024) where T: Component;

      void AddComponentPool<T>(IComponentPool<T> componentPool) where T: class;
  }

  • Returning reference-type components to the pool indirectly

    You may store poolable references in another component which is not pooled on its own.

    E.g:

    public struct PrimitiveColliderComponent : IPoolableComponentProvider<Collider>
  {
      public Collider Collider;
      public Type ColliderType;
      public PBMeshCollider.MeshOneofCase SDKType;

      Collider IPoolableComponentProvider<Collider>.PoolableComponent => Collider;

      Type IPoolableComponentProvider<Collider>.PoolableComponentType => ColliderType;

      public void Dispose() { }
  }

    In order to provide basic clean-up behaviour for such scenarios do the following:

    • Make your component implement IPoolableComponentProvider<out T>:
      • It can implement as many different T as needed
      • If your component is a structure you should implement Type PoolableComponentType => typeof(T); explicitly. Otherwise it will be boxed to take the default implementation from the interface
    • Inject class ReleasePoolableComponentSystem<T, TProvider> with final arguments' types and add it to the list

    E.g

    finalizeWorldSystems.Add(ReleasePoolableComponentSystem<IPrimitiveMesh, PrimitiveMeshRendererComponent>.InjectToWorld(ref builder, componentPoolsRegistry));
    • This system automatically returns components to the pool when the entity gets destroyed and the scene dies (by implementing IFinalizeWorldSystem)
    • This system does not return an SDK component to the pool when the component is removed by the scene (as it does not have knowledge about the client-side counterpart based on which it can infer if the component was ever processed). It should be done by custom code.

If you require more complex clean up logic you can implement the system on your own and don't rely on this shortcut.

How to test systems

UnitySystemTestBase<TSystem> provides basic functionality for world creation and disposal.

In Tests you can create systems directly by calling a constructor. Consider exposing them by [InternalsVisibleTo] to tests.

Design components

  • In terms of ECS there is no difference between SDK components (from Proto) and written by us
  • If you need to enrich an entity (created with an SDK component) with additional data create a separate component: by filtering you will be able to recognize which components are not processed yet
  • Keep the balance between separate components and state:
    • structural changes are expensive operations, if the logic supposes frequent/uncontrolled Adding or Removing components, it's preferred to have a single component and change its state instead.
    • otherwise, it's advised to maintain a reasonable segregation and responsibilities distribution between different components
  • If you need to wait for data that is retrieved asynchronously create an AssetPromise<TAsset, TLoadingIntention>, e.g.:
    • Asset Bundles
    • GLTF
    • Textures
    • Any other data from web requests
  • You may have as many AssetPromises as needed and store them in a component or add them to an entity directly. Keep in mind it's a value type as well so whatever you do, ensure you operate with it by ref, otherwise the state won't be reflected.

Singletons

  • Some components can be natural singletons (e.g. PhysicsTickComponent): in our case, it means they exist in a single instance per World
  • They are created by systems in ctor or in Initialize
  • Then they can be used in Update by other systems
  • Instead of making a query every time such a component is needed consider caching it and save into a SingleInstanceEntity field

How to connect to test scene environments (Unity Editor)

Goerli Plaza Test scenes

That repo contains example scenes of different SDK components combinations and common usages.

  1. Set the Main Scene Loader -> Startup Config -> Initial Realm to "Custom".
  2. Set the initial position to one of the existent scenes in that repo, for example "74, -9".
  3. Set Custom Realm to "https://sdk-team-cdn.decentraland.org/ipfs/goerli-plaza-main-latest".
  4. Hit PLAY and that test environment should load with all of its deployed scenes

SDK7 Test scenes

That repo contains test scenes for specific components or features of the SDK.

Beware that environment scenes are currently not being converted to Asset Bundles, so only primitives will be visible on the scenes...

  1. Set the Main Scene Loader -> Startup Config -> Initial Realm to "Custom".
  2. Set the initial position to one of the existent scenes in that repo, for example "0, 2".
  3. Set Custom Realm to "https://sdk-team-cdn.decentraland.org/ipfs/sdk7-test-scenes-main-next".
  4. Hit PLAY and that test environment should load with all of its deployed scenes

How to connect to a local scene (Unity Editor / Custom Build / Latest Released Build)

Running the scene

1. Download the desired SDK7 scene: e.g. SDK7 Scene Template / Any scene from SDK7 Test Scenes / Any scene from the Goerli Plaza Test Scenes.

2. Run the scene locally: Open a console at the root of the scene folder and run npm i (alternatively if you run npm i @dcl/sdk@latest you can make sure you are using the latest SDK released version). Once it finishes, execute npm run start -- --explorer-alpha. This will create the compiled bin file in sdk7-scene-template/bin/.

3. The Latest Released Build of the Explorer opens: The command ran in step 2 ends up openiong the Latest Released Build (that gets downloaded by the launcher) and connects it to the locally running scene. If you want to test the scene with the latest released build of the Explorer then next steps are not needed.

4. Close the Launcher / Explorer that auto-opened: Since you will be connecting to it with Unity Editor or a custom build.

Note: Modify the scene's code to your liking: Modify the code files inside sdk7-scene-template/src/ according to your needs and it will be automatically updated on the connected Explorer. Remember that you have the SDK Documentation available to learn how to implement each component from the SDK side.

(Optional) run a local test environment with several scenes (AKA SDK Workspace)

For example you could be interested in testing some scenes from the SDK Goerli Plaza Test Scenes and you want to some/all of them together.

  1. Clone the sdk7-goerli-plaza repo.
  2. Modify the dcl-workspace.json file in order to have ONLY the list of scene paths that you want to test. For example:
{
  "folders": [
    {
      "path": "advanced-avatar-swap"
    },
    {
      "path": "avatar-swap"
    },
    {
      "path": "Cube"
    }
  ]
}
  1. In the console, run npm i and then npm run start -- --explorer-alpha to run the workspace locally.

Connecting Unity Editor to the scene

  1. Set the Main Scene Loader -> Startup Config -> Initial Realm to "Localhost".
  2. Set the initial position to one of the locally running scene.
  3. Toggle the "Use Remote Asset Bundles" according to your need. If it's off then the original raw GLTFs from the locally running scene will be loaded instead of downloading their asset bundle version (if it exists)
  4. Hit PLAY and the editor should connect to the locally running scene

Connecting A custom build to the scene

Custom builds can be just locally built executables or for example the PR builds exposed by the CI.

Run the build from a console/terminal specifying the needed parameters, for example:

WinOS

"C:\Users\[YOUR-USER]\Downloads\Decentraland_windows64\Decentraland.exe" --realm http://127.0.0.1:8000 --position 54,-55 --local-scene true --debug

MacOS

open Decentraland.app --args --realm http://127.0.0.1:8000 --position 54,-55 --local-scene true --debug

How to open the Explorer with parameters

Deep Link

Regardless of being on WinOS or MacOS, when opening a Decentraland deep link the launcher will be opened and so the latest released build will be used.

Any relevant parameter can be used through the deep link, some examples:

  • For connecting to a locally running scene: decentraland://?realm=http://127.0.0.1:8000/&position=-139,-28&local-scene=true&debug
  • For connecting to genesis city directly on the specified position: decentraland://?position=66,66
  • For connecting directly into a specified world: decentraland://?realm=pravus.dcl.eth

Command Arguments (custom/PR builds)

By running the build from a console/terminal you can specifying the pertinent parameters.

WinOS

  • For connecting to a locally running scene: "C:\Users\[YOUR-USER]\Downloads\Decentraland_windows64\Decentraland.exe" --realm http://127.0.0.1:8000 --position -139,-28 --local-scene true --debug
  • For connecting to genesis city directly on the specified position: "C:\Users\[YOUR-USER]\Downloads\Decentraland_windows64\Decentraland.exe" --position 66,66 --debug
  • For connecting directly into a specified world: "C:\Users\[YOUR-USER]\Downloads\Decentraland_windows64\Decentraland.exe" --realm pravus.dcl.eth --debug

MacOS

  • For connecting to a locally running scene: open Decentraland.app --args --realm http://127.0.0.1:8000 --position -139,-28 --local-scene true --debug
  • For connecting to genesis city directly on the specified position: open Decentraland.app --args --position 66,66 --debug
  • For connecting directly into a specified world: open Decentraland.app --args --realm pravus.dcl.eth --debug

Test media streaming

In the current project, media streaming is implemented via external package, namely AVPro. This package is imported during the build process on CI. In order to test media streaming in editor

  1. Import AVPro package. Trial/preview version of it can be downloaded from the official github repository. At the moment of writing this documentation AVPro version used in this project was 2.8.0.
  2. Add csc.rsp file to the Assets folder with only one line -define:AV_PRO_PRESENT in this file.
  3. Close and re-open Unity Editor, so the define symbols are updated.
  4. Verify that symbol is recognized by checking DCL.MediaPlayer.asmdef in inspector. You should not see red sign in front of Define Constraints AV_PRO_PRESENT there.

After importing trial package you can

  1. run MediaStreaming sdk-scene from StaticSceneLoader.unity scene to verify that media streaming is working. This scene includes both audio- and video streams.
  2. run Main.unity scene on https://sdk-team-cdn.decentraland.org/ipfs/streaming-world-main realm and run to coordinates (-12,-3) to observe video played on the big screen (in the open cinema scene)
Clone this wiki locally