Transparent fluid when using Meta quest passthrough

[413866]

To enable passthrough on any quest (1,2 and 3). you have to set the background to black in the camera, this is fine because the user does not see this. It can not be any other color then black as that would cause the user to see this color.

The problem arises when you have a transparent fluid in the foreground. It will turn a dark black color because it absorbs this black color from the "invisible" background.

[josemendez]

To enable passthrough on any quest (1,2 and 3). you have to set the background to black in the camera, this is fine because the user does not see this. It can not be any other color then black as that would cause the user to see this color.

Hi,

This shouldn't affect the fluid color whatsoever, unless the camera doesn't render anything else on top of the background. Fluids will refract whatever the camera has rendered after opaque objects have been rendered. If the camera just renders a black background, this will be what is seen trough the fluid.

if you want to see the headset's camera output trough the fluid, that's outside the scope of Obi's built-in rendering system and something you must implement yourself by using a custom fluid shader that samples the passthrough camera texture instead of the frame buffer after opaque rendering.


kind regards,

[413866]

Everything else that is transparent works fine, like the beakers and stuff. it seems that the issue is because the obi fluid renderer is on top of the camera itself rather then below it.

[josemendez]

Everything else that is transparent works fine, like the beakers and stuff. it seems that the issue is because the obi fluid renderer is on top of the camera itself rather then below it.

No, that's not why it happens. The cause has to do with how graphics works in general: transparent objects are rendered separately from opaque objects (after all opaque objects), and there's two kinds of transparent objects:

A) "regular" transparents (like the beakers) that perform no refraction: they show whatever is behind them undistorted. For these, just rendering them after all opaque objects and alpha blending them with the current contents framebuffer is enough.

B) refractive transparents (such as fluids) that distort whatever is behind them. These need to somehow arbitrarily sample the color of objects behind them at a position other than where they're being rendered, so regular alpha blending with the framebuffer contents doesn't work.

The way refractive objects distort what's behind them is by "saving" whatever it's currently rendered by the game camera up to that point into a texture, and then sampling and distorting the contents of that texture while rendering the transparent object. However when using passtrough, the game camera is not rendering anything in the passtrough areas but a black background, so the texture sampled by the fluid renderer is black. The input of the physical cameras in the headset is stored in a different texture, that the fluid renderer knows nothing about.

You need to write a custom shader that samples the texture where the passtrough image captured by the physical camera is stored, instead of the texture that contains what the game camera is rendering. You can use the included FluidShader (Obi/Resources/ObiMaterials/FluidShader.shader) as a starting point for this.

kind regards,

[413866]

I have tried to fix this issue for a couple of days now and I seriously cannot seem to fix it. could you please help me with how to do this

[413866]

No, that's not why it happens. The cause has to do with how graphics works in general: transparent objects are rendered separately from opaque objects (after all opaque objects), and there's two kinds of transparent objects:

A) "regular" transparents (like the beakers) that perform no refraction: they show whatever is behind them undistorted. For these, just rendering them after all opaque objects and alpha blending them with the current contents framebuffer is enough.

B) refractive transparents (such as fluids) that distort whatever is behind them. These need to somehow arbitrarily sample the color of objects behind them at a position other than where they're being rendered, so regular alpha blending with the framebuffer contents doesn't work.

The way refractive objects distort what's behind them is by "saving" whatever it's currently rendered by the game camera up to that point into a texture, and then sampling and distorting the contents of that texture while rendering the transparent object. However when using passtrough, the game camera is not rendering anything in the passtrough areas but a black background, so the texture sampled by the fluid renderer is black. The input of the physical cameras in the headset is stored in a different texture, that the fluid renderer knows nothing about.

You need to write a custom shader that samples the texture where the passtrough image captured by the physical camera is stored, instead of the texture that contains what the game camera is rendering. You can use the included FluidShader (Obi/Resources/ObiMaterials/FluidShader.shader) as a starting point for this.

kind regards,

It seems that the passthrough image texture is not available due to privacy reasons of the Quest. Would there be a way different way of doing it?

[josemendez]

It seems that the passthrough image texture is not available due to privacy reasons of the Quest. Would there be a way different way of doing it?

Unfortunately not, refractive objects such as fluid need an image of the thing being refracted. If Quest forbids access to the camera feed, there's no other way to do it.

You can use opaque fluids however, which don't perform any refraction. To do this, bring the "transparency" slider in the ObiFluidRenderer component all the way to zero.

kind regards,

[413866]

Unfortunately not, refractive objects such as fluid need an image of the thing being refracted. If Quest forbids access to the camera feed, there's no other way to do it.

You can use opaque fluids however, which don't perform any refraction. To do this, bring the "transparency" slider in the ObiFluidRenderer component all the way to zero.

kind regards,

Sliding down the reflection to zero makes a small difference but there is still a very dark colour instead of transparent

[josemendez]

Sliding down the reflection to zero makes a small difference but there is still a very dark colour instead of transparent

You should slide down transparency, not reflection. Reflection has no impact on fluid color whatsoever.

You can use opaque fluids however, which don't perform any refraction. To do this, bring the "transparency" slider in the ObiFluidRenderer component all the way to zero.

You can't have transparent fluids that refract the camera feed, since Oculus apparently doesn't give you access to it. So making the fluid opaque will allow you to have colored fluids, but without any transparency.

[413866]

You should slide down transparency, not reflection. Reflection has no impact on fluid color whatsoever.


You can't have transparent fluids that refract the camera feed, since Oculus apparently doesn't give you access to it. So making the fluid opaque will allow you to have colored fluids, but without any transparency.

Which makes it very weird to me is that we have transparent beakers which work perfectly but only the liquids have a problem.