5

I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator on float. With simple % there is a problem on Nvidia and on AMD (e.g. 7 % 7 = 7) so I have written modulus as:

float Mod( float x, float y )    
{       
     return x - floor(x / y) * y;    
}

This have fixed the problem on Nvidia but AMD still returns wrong values.

Any idea what is going on? And how to fixed it?

I know I can use:

float( int( x ) % int( y ) )

but I think it is slower and I wonder if there is any better way.

Derag
  • 596
  • 3
  • 13

1 Answers1

1

In HLSL, you can use the built-in fmod function for this.

I'm not sure what's going wrong with your implementation, though. It looks mathematically correct. It's conceivable you're encountering a precision issue with the subtraction, depending on the inputs.

BTW, an alternative implementation which avoids the subtraction is:

float Mod(float x, float y)
{
    return frac(x / y) * y;
}
Nathan Reed
  • 25,002
  • 2
  • 68
  • 107
  • But frac(x / y) * y is exactly what normal modulus operator compiles to (on GCN at least) and it gives incorrect results on both Nvidia and AMD. I'm not sure about fmod, though. It seems to give slightly different assembly code. I will check it, thanks. – Derag Apr 19 '18 at 19:45
  • So I have chacked it and frac(x / y) * y, operator % and fmod give incorrect results on Nvidia and AMD. x - floor(x / y) * y works correct on Nvidia but AMD not so much. – Derag Apr 20 '18 at 07:57