Feedback Obi 8: what's coming up

[josemendez]

Hi all!

I've been working on Obi 8 for a few weeks now, and wanted to talk about what we're doing with it and get some feedback if possible.

The main driving force behind Obi 8 is making it more flexible. Currently, the only way to customize particle behavior is to add external forces and/or modify their positions outside the main solver loop. This is fine in many cases, but will fail miserably in others, specially when tight coupling with existing constraints is required and doing stuff every substep/iteration is a necessity. In these cases you'd be forced to crack open the engine and mess with internals, which isn't an appealing prospect.

So the idea is to rewrite a large part of the engine and expose an API that allows you to write entirely custom constraints. These get solved along with all built-in constraints, in fact, we're rewriting all built-in constraints using this new API. This is how it looks:

Constraints are split into 3 classes: a data container, a burst implementation (optional) and a compute implementation (optional). For instance, bend constraints would consist of:

• ObiBendConstraintData (derived from ObiConstraintData)
  
• ObiBurstBendConstraints (derived from ObiBurstConstraints)
  
• ObiComputeBendConstraints (derived from ObiComputeConstraints)
  

To register a new constraint type in a solver, there's a generic RegisterConstraintType method that takes the name used to display constraint parameters in the solver and the parameters that should be shown in the ObiSolver inspector:

Code:

solver.RegisterConstrainType<T>(string name, ConstraintParameters params) where T : ObiConstraintData

The ability to pass custom constraint parameters allows your constraints to have their own mini-inspector in the ObiSolver component, to have all related parameters neatly presented along built-in ones. This has allowed us to move what were solver-global parameters (like ambient wind direction which is used by aerodynamic constraints, or collision margin which is used by collisions) to the constraints that actually depend on them, which is less confusing:



for example, bend constraints register themselves like so:

Code:

solver.RegisterConstrainType<ObiBendConstraintData>("Bend", new BendConstraintParameters());

Once you've registered a new type of constraints, you can add implementations for the Burst and Compute backends. These are optional, eg. if you don't provide a Compute implementation then these constraints won't do anything when using the Compute backend. I don't recommend not adding any implementation though!

Code:

solver.RegisterBurstConstraintsImplementation<T, U>() where T : ObiConstraintData where U : ObiBurstConstraints;
solver.RegisterComputeConstraintsImplementation<T, U>() where T : ObiConstraintData where U : ObiComputeConstraints

The constraint data class is very simple: you just declare data buffers (ObiNativeLists) to hold data for each constraint, and provide implementations to add constraints, resize the buffers, clear and dispose them. For instance bend constraint data looks like this:

Code:

using UnityEngine;
using System.Collections;
using System.Collections.Generic;
using System;

namespace Obi
{
    [Serializable]
    public class ObiBendConstraintData : ObiConstraintData
    {
        /// <summary>
        /// one float per constraint: the rest bend distance.
        /// </summary>
        [HideInInspector] public ObiNativeFloatList restBends = new ObiNativeFloatList();               

        /// <summary>
        /// two floats per constraint: max bending and compliance.
        /// </summary>
        [HideInInspector] public ObiNativeVector2List bendingStiffnesses = new ObiNativeVector2List();

        /// <summary>
        /// two floats per constraint: plastic yield and creep.
        /// </summary>
        [HideInInspector] public ObiNativeVector2List plasticity = new ObiNativeVector2List();

        public override int priority => 100;
        public override int workItemSize => 4;

        public int AddConstraint(int batchIndex, Vector3Int indices, float restBend)
        {
            int constraintIndex = RegisterConstraint(batchIndex, indices.x, indices.y, indices.z);

            restBends[constraintIndex] = restBend;
            bendingStiffnesses[constraintIndex] = Vector2.zero;
            plasticity[constraintIndex] = Vector2.zero;

            return constraintIndex;
        }

        protected override void ResizeDataBuffers(int constraintCount)
        {
            restBends.ResizeUninitialized(constraintCount);
            bendingStiffnesses.ResizeUninitialized(constraintCount);
            plasticity.ResizeUninitialized(constraintCount);
        }

        protected override void CopyConstraintData(ObiActor actor, ObiConstraints source, int srcIndex, int dstIndex, int count)
        {
            var batch = source as ObiBendConstraintData;
            var user = actor as IBendConstraintsUser;

            if (batch != null && user != null)
            {
                restBends.CopyFrom(batch.restBends, srcIndex, dstIndex, count);
                bendingStiffnesses.CopyReplicate(new Vector2(user.maxBending, user.bendCompliance), dstIndex, count);
                plasticity.CopyReplicate(new Vector2(user.plasticYield, user.plasticCreep), dstIndex, count);
            }
        }

        public override bool EnabledForActor(ObiActor actor)
        {
            var user = actor as IBendConstraintsUser;
            return user != null && user.bendConstraintsEnabled;
        }
        protected override void OnClear()
        {
            restBends.Clear();
            bendingStiffnesses.Clear();
            plasticity.Clear();
        }

        protected override void OnDispose()
        {
            restBends.Dispose();
            bendingStiffnesses.Dispose();
            plasticity.Dispose();
        }
    }
}

Concrete implementations for the burst/compute backends need to implement a method to retrieve constraint data, and a method to actually enforce the constraints (that gets called once per solver iteration of that constraint type). Burst bend constraints look like this:

Code:

using UnityEngine;
using Unity.Jobs;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Mathematics;
using Unity.Burst;
using System.Collections;

namespace Obi
{
    public class ObiBurstBendConstraints: ObiBurstConstraints
    {
        BendConstraintsBatchJob projectConstraints;

        public override void SetConstraintData()
        {
            ObiBendConstraintsData b = abstraction as ObiBendConstraintsData;

            this.lambdas = b.lambdas.AsNativeArray<float>();
            this.batches = b.batches.AsNativeArray<int4>();

            projectConstraints.workItems = b.workItems.AsNativeArray<int2>();
            projectConstraints.particleIndices = b.particleIndices.AsNativeArray<int>();
            projectConstraints.restBends = b.restBends.AsNativeArray<float>();
            projectConstraints.stiffnesses = b.bendingStiffnesses.AsNativeArray<float2>();
            projectConstraints.plasticity = b.plasticity.AsNativeArray<float2>();
            projectConstraints.lambdas = this.lambdas;
        }

        public override JobHandle SolvePositions(JobHandle inputDeps, float stepTime, float substepTime, int steps, float timeLeft)
        {
            projectConstraints.positions = solverImplementation.positions;
            projectConstraints.invMasses = solverImplementation.invMasses;
            projectConstraints.deltas = smooth ? solverImplementation.positionDeltas : solverImplementation.positions;
            projectConstraints.deltaTime = substepTime;
            projectConstraints.countConstraints = smooth ? 1 : 0;
            projectConstraints.workItemSize = workItemSize;

            for (int i = 0; i < batches.Length; ++i)
            {
                projectConstraints.offset = batches[i].x;
                inputDeps = projectConstraints.Schedule(batches[i].y, 8, inputDeps);
            }

            if (smooth)
                inputDeps = ApplyDeltas(inputDeps);

            return inputDeps;
        }

        [BurstCompile]
        public struct BendConstraintsBatchJob : IJobParallelFor
        {
            [ReadOnly] public NativeArray<int2> workItems;
            [ReadOnly] public NativeArray<int> particleIndices;

            [ReadOnly] public NativeArray<float4> positions;
            [ReadOnly] public NativeArray<float2> stiffnesses;
            [ReadOnly] public NativeArray<float2> plasticity; //plastic yield, creep
            [ReadOnly] public NativeArray<float> invMasses;

            [NativeDisableContainerSafetyRestriction] [NativeDisableParallelForRestriction] public NativeArray<float> restBends;
            [NativeDisableContainerSafetyRestriction] [NativeDisableParallelForRestriction] public NativeArray<float> lambdas;
            [NativeDisableContainerSafetyRestriction] [NativeDisableParallelForRestriction] public NativeArray<float4> deltas;

            [ReadOnly] public float deltaTime;
            [ReadOnly] public float sor;
            [ReadOnly] public int countConstraints;
            [ReadOnly] public uint workItemSize;
            [ReadOnly] public int offset;

            public void Execute(int w)
            {
                var wi = workItems[offset + w];

                for (int i = wi.x, j = 0; j < workItemSize; ++i, ++j)
                {
                    if ((wi.y & (1 << j)) == 0) continue;

                    int p1 = particleIndices[i * 3];
                    int p2 = particleIndices[i * 3 + 1];
                    int p3 = particleIndices[i * 3 + 2];

                    float w1 = invMasses[p1];
                    float w2 = invMasses[p2];
                    float w3 = invMasses[p3];

                    float wsum = w1 + w2 + 2 * w3;

                    float4 bendVector = positions[p3] - (positions[p1] + positions[p2] + positions[p3]) / 3.0f;
                    float bend = math.length(bendVector);

                    float constraint = bend - restBends[i];

                    constraint = math.max(0, constraint - stiffnesses[i].x) +
                                 math.min(0, constraint + stiffnesses[i].x);

                    // plasticity:
                    if (math.abs(constraint) > plasticity[i].x)
                        restBends[i] += constraint * plasticity[i].y * deltaTime;

                    // calculate time adjusted compliance
                    float compliance = stiffnesses[i].y / (deltaTime * deltaTime);

                    // since the third particle moves twice the amount of the other 2, the modulus of its gradient is 2:
                    float dlambda = (-constraint - compliance * lambdas[i]) / (wsum + compliance + BurstMath.epsilon);
                    float3 correction = 2 * dlambda * bendVector.xyz / (bend + BurstMath.epsilon);

                    lambdas[i] += dlambda;

                    deltas[p1] += new float4(-correction * w1, countConstraints);
                    deltas[p2] += new float4(-correction * w2, countConstraints);
                    deltas[p3] += new float4(correction * 2 * w3, countConstraints);
                }
            }
        }
    }
}

The Compute equivalent is very similar, but instead of a Job it uses a compute shader to so the same work.

Some constraints are not just static relationships between particles, but require colliders or particle neighbor information that changes over time. We're making sure to expose collider and particle neighbor data (gathered during collision detection) as well so that it can be used in custom constraints. You'll be able to, for example, dynamically create constraints between particles that come in the vicinity of each other, and destroy the constraints once they're under too much stress.

Along with this, we're rewritting constraint coloring/batching. Instead of batching individual constraints, we group them into tiny chunks of 2-8 constraints called work items, and batch the work items instead. The coloring algorithm used in blueprint generation has been also upgraded from greedy coloring to recursive largest-first. Both changes result in considerably less batches, and a speedup of around 10-30% depending on the scene.

So my hope is for Obi 8 to be simpler, faster and a lot more flexible all at once.

I'm also tinkering with the idea of allowing custom particle attributes (so that your particles can have position, mass, political affiliation and social security number if you so choose) and custom colliders. This is in very early stages so the API hasn't come together yet.

Let me know what you think, what you'd like to see in the future or ask me if a specific use case will be possible. This way the thing that ends up being shipped will meet your needs better! Any feedback is welcome.

kind regards