Sticking cloth on object in VR

[mbbmbbmm]

Hello, I have a few questions! I am trying to achieve the following for a VR project (I'm using HTC Vive and Leap Motion):
I want to grab a texture object that uses the Obi cloth simulation and place it on a mesh object. The mesh object optionally has been changed before that so that it is necessary to update its mesh collider once. After the object is placed it should be possible to slide the texture/cloth over the object while still having cloth and mesh object stick together. In the end the cloth object should be frozen and optionally a new texture/cloth comes in - taking the old one out of the calculations to save performance etc.

1 For now I have put two handles on the upper corners of the cloth to grab it and place it. Could I turn these handles off after being done with the rough placement? Also it would be much more convenient to be able to grab the cloth anywhere so I wouldn't need dedicated handles. Is there something like temporary handles that could react to the grabbing gesture of the leap motion? Can I directly influence the positions of particles that are in a certain range of the hands?

2 How can I recalculate the Obi collider of the object from its mesh collider? I think I saw something in the API using the ColliderGroup, but this doesn't seem to exist anymore.. At the moment I just add a new collider once the mesh is ready - but I am not sure if that is a good solution... also the cloth seems to jump each time I do that. It doesn't bother me too much, just want to make sure if this would be expected behaviour.

3 Regarding the surface sliding I think I would need to have an Obi collision material that is very sticky with a small stick distance (so it doesn't stick to quickly) and also a low friction value. Then use Friction Combine Minimum and Stickiness Combine Maximum. Does that sound about right?

4 How many iterations of the distance constraints would be OK for VR? Maybe 20 or 30? With the default the cloth enters the mesh a little too frequently..

5 And a final question: There will be a lot of different combinations of objects and cloth textures coming in one after another. But only one combination would be active at a time. What would be the best way to set this up regarding performance and memory? Can I pool all the mesh objects and texture/cloth objects and have them share one Solver? Or would it be better to instantiate them from prefabs and/or have a different solver for each cloth texture object?

Thank you!
Jan

[niZmo]

Well I can answer 1 and 2.

1) Here is an older implementation of how to interact with obi partciles in VR. It'll get you in the right direction anyway, never used leap. This snippet uses pins, where handles would be better since the actor will never physically effect the controllers. If for the range grab, you want the cloth to go to you, you'll need to lerp each particle somehow so it doesn't snap to your hand. Doing so the cloth will go crazy.

2) Yes, if you change the mesh you will need to reload the collider and obiCollider. Obi now uses a new collision system where you add an obiCollider to objects you want to collide with. ColliderGroup is deprecated.

[mbbmbbmm]

Thank you very much for your help! I will try it tomorrow.

[josemendez]

Hello, I have a few questions! I am trying to achieve the following for a VR project (I'm using HTC Vive and Leap Motion):
I want to grab a texture object that uses the Obi cloth simulation and place it on a mesh object. The mesh object optionally has been changed before that so that it is necessary to update its mesh collider once. After the object is placed it should be possible to slide the texture/cloth over the object while still having cloth and mesh object stick together. In the end the cloth object should be frozen and optionally a new texture/cloth comes in - taking the old one out of the calculations to save performance etc.

1 For now I have put two handles on the upper corners of the cloth to grab it and place it. Could I turn these handles off after being done with the rough placement? Also it would be much more convenient to be able to grab the cloth anywhere so I wouldn't need dedicated handles. Is there something like temporary handles that could react to the grabbing gesture of the leap motion? Can I directly influence the positions of particles that are in a certain range of the hands?

2 How can I recalculate the Obi collider of the object from its mesh collider? I think I saw something in the API using the ColliderGroup, but this doesn't seem to exist anymore.. At the moment I just add a new collider once the mesh is ready - but I am not sure if that is a good solution... also the cloth seems to jump each time I do that. It doesn't bother me too much, just want to make sure if this would be expected behaviour.

3 Regarding the surface sliding I think I would need to have an Obi collision material that is very sticky with a small stick distance (so it doesn't stick to quickly) and also a low friction value. Then use Friction Combine Minimum and Stickiness Combine Maximum. Does that sound about right?

4 How many iterations of the distance constraints would be OK for VR? Maybe 20 or 30? With the default the cloth enters the mesh a little too frequently..

5 And a final question: There will be a lot of different combinations of objects and cloth textures coming in one after another. But only one combination would be active at a time. What would be the best way to set this up regarding performance and memory? Can I pool all the mesh objects and texture/cloth objects and have them share one Solver? Or would it be better to instantiate them from prefabs and/or have a different solver for each cloth texture object?

Thank you!
Jan

2) Go to ObiColliderBase.cs and change the "tracker" variable (line 46) from protected to public. Then, on a ObiCollider w/ Meshcollider, you should be able to do:

Code:

(obiCollider.tracker as ObiMeshShapeTracker).UpdateMeshData();

3) Sounds ok. Keep in mind that the combine mode determines how to combine the values for both objects involved in each contact, and the defaults when no collision material is present is zero for both friction and stickiness. So if your cloth has a material with 0.2 friction and combine set to "min", but your colliders have no material, the resulting friction between them will be min (0.2,0) = 0.

4) Well, the amount of iterations largely depends on your particular scenario, your performance budget, and how much deviation from a "perfect" simulation you can tolerate. Higher iteration counts will yield better simulation (the definition of "better" depends on what the particular constraint type does: for distance constraints = less stretchy cloth, for bending constraints = more rigid cloth), etc.

My advice, regardless of your target platform (VR or not) is to always use the lowest iteration count that yields acceptable results in your scenario. 20 distance constraint iterations is not crazy for a cloth-centered app, but would almost always be too much for character cloth in a third-person hack'nslash. 

5) I'd go with prefabs, and a single solver. Make a prefab out of each cloth object, and upon instantiation, set their solver. Having a separate solver for each one is also ok, you can adjust the particle pool sizes (solver.maxParticles) a bit better, also use independent solver settings for each cloth, but having multiple solvers has a slight performance hit. Might outweight the benefits, might not, all you can do is try. 

Start with one prefab per object, each one with its own solver (it's easier to set up). Measure performance, if it's enough you're done. If not, try a single solver.

[mbbmbbmm]

Thanks so much for your help!
I nearly got it working. I tried to use the ObiParticleHandle class to AddParticle() and RemoveParticle() the closest particle, but while that kind of works for a short amount of time it gets really weird when the Leap Motion hands lose tracking / go out of the field of view (which happens very frequently). I thought I would use ObiParticleHandle.RemoveParticle(), but I think calling it from OnDisable might be to late? Gotta try that tomorrow.

As far as I understand, I can't create a reference to the particle itself, but have to use the indices. As long as there is only one actor / cloth - are the indices of the particles reliable or is it possible that they change some time after I have cached e.g. the index of the nearest particle?

@niZmo: That's super cool, I'm a big fan of VRTK! I looked at the code you posted, only could parse parts of it because it's so specifically for VRTK, and kind of complex ^^; ... but it helped me get a rough idea, thanks again.

[niZmo]

Thanks so much for your help!
I nearly got it working. I tried to use the ObiParticleHandle class to AddParticle() and RemoveParticle() the closest particle, but while that kind of works for a short amount of time it gets really weird when the Leap Motion hands lose tracking / go out of the field of view (which happens very frequently). I thought I would use ObiParticleHandle.RemoveParticle(), but I think calling it from OnDisable might be to late? Gotta try that tomorrow.

As far as I understand, I can't create a reference to the particle itself, but have to use the indices. As long as there is only one actor / cloth - are the indices of the particles reliable or is it possible that they change some time after I have cached e.g. the index of the nearest particle?

@niZmo: That's super cool, I'm a big fan of VRTK! I looked at the code you posted, only could parse parts of it because it's so specifically for VRTK, and kind of complex ^^; ... but it helped me get a rough idea, thanks again.

This might help you a little better. I’ve been workin on and interaction system for Obi and wish I was further along to have a stable release but I grabbed a snippet for someone wanting to attach things to a cloth. It’s essentially the same to interact with an actor. I am at work, but I do have a method to use ObiParticleHandle in a similar fashion. A handle is better since the particles can’t influence the controller/hands. Anyway here is the link. I am assuming the leap hand object has an ObiCollider on it and your using the ObiSolver collision event to get what particle you are hitting with your hand. So with this snippet all you have to do is tell it the particle and its colliding ObiCollider and it will attach to the actor. It has a dictionary that will pair an ObiCollider with its pinnedIndex which is need for removing the pin constrint later on. The pin constraints pin index is NOT the same as the particle index its pinned to. It starts at 0 and counts up for every pin that’s added wether in the editor or added through code. Anyway it’ll be a little before I can post up how to dynamically use ObiParticleHandles for what you want to do. Got work then work again 8hrs after that lol. I do have a leap motion and this intrigues me to try it out. I remember TheStoneFox talk about putting leap support in VRTK, I don’t know it’s status though.

As far as particle referencing. The particles do not change. The particle are pool based but they won’t change unless they are put back into the pool by removing the actor from its solver. That being said the particles aren’t always in chronological order. To make it easier I’ll use ObiRope as an example since the particle are in line. Say you have Rope A that’s 10 particles long with indices 0 to 9. Then you make Rope B that’s 10 particle long with indices 10 to 19. You remove Rope A from the solver, and make a Rope C 20 particles long. It’ll use those recycled particles. The Particles of Rope C will be 0-9, 20-29. If you tear the rope between 7-8 it’ll take the next particle from the pool with is 30 and add it to one side of the cut. So Rope C’s particle order will be 0-7,30,8,20-29. Cloth is obviously a little different but if you are using a tearable cloth the particles being pulled from the solver is the same.

[mbbmbbmm]

Heading off to work now too, but will definitely check that out later!
I am not checking for collision at the moment but once the GrabStrength of the hand is strong enough I find the nearest particle and check if the distance is below a certain threshhold for grabbing. Pretty much like this:

Code:

IEnumerator GrabParticle(ObiParticleHandle currentHandle)
{
   if (actor.InSolver)
   {
       actor.Solver.RequireRenderablePositions();
       yield return null;

       float currentSqrMagnitude = float.MaxValue;
       float smallestSqrMagnitude = float.MaxValue;
       float smallestMagnitude = float.MaxValue;
       int currentIndex = 0;

       for (int i = 0; i < actor.invMasses.Length; i++)
       {
           currentSqrMagnitude = (actor.GetParticlePosition(i) - currentHandle.transform.position).sqrMagnitude;
           if (currentSqrMagnitude < smallestSqrMagnitude)
           {
               smallestSqrMagnitude = currentSqrMagnitude;
               currentIndex = i;
           }
       }
       smallestMagnitude = Mathf.Sqrt(smallestSqrMagnitude);
       if (smallestMagnitude < user.clothManipulate.grabThreshold)
       {
           currentHandle.AddParticle(currentIndex, actor.GetParticlePosition(currentIndex), cachedInvMass);
       }

       actor.Solver.RelinquishRenderablePositions();
   }
}

This is done "once per grabbing" so should be ok I guess, but maybe using Colliders would be better...

[mbbmbbmm]

Well it seems that I got it working now
I had been tired and apparently forgot to add a ref keyword to my index parameter or return it so that it can be actually altered and cached. D'Oh! Calling the Release()-method from OnDisable() also seems to work fine which means no more weirdness when hands disappear. I'm very glad, thank both of you once again for your help!