[Solved] Error with while loop in a hlsl shader

[porcus]

Hi,

I tried to convert a glsl shader to hlsl.
The most things are working but in one line a get a very strange error:

HLSL pixel shader compilation failed:
(54): error X3511: loop does not appear to terminate in a timely manner (1024 iterations)

Code: Select all

[...]
while (NB.r < height) {<-----this is line 54
height -= step;
offsetCoord += delta;
NB = tex2D(heightMap, offsetCoord);
}
[...]

The glsl version of this shader is working fine.
What I don't understand is: How should the compiler even know how many iterations will happen, because
step and delta are depending on what is set in the ShaderCallback (height is 1.0 at the beginning).

If I replace (NB.r < height) with (1.0 < 0.9) which is always false the error disappears, but of course
this would not be a acceptable solution (and the results are not right of course).

I hope anybody can help me.

[9YkKsvXM]

-

[porcus]

Yes, I am using shader model 3.0

[BlindSide]

You might wanna try replacing these 2 lines:

Code: Select all

while (NB.r < height) {<-----this is line 54
height -= step;

With

Code: Select all

height = min(NB.r, height);

Or perhaps

Code: Select all

height = fmod(height, NB.r);

Or maybe even

Code: Select all

height = min(NB.r, height) - fmod(NB.r, step);

Just pick whatever works (If any).

[porcus]

Nothing of that works. There's no way to do this without a loop.
I've read on some other sites that the hlsl compiler tries to unroll
the loop and that's why it don't works. Is there any way to tell the
compiler that he shall not try to unroll it ?

[Mel]

HLSL are compiled in the video card, outside the exectuable. In fact, anything in a shader SHOULD ALWAYS have an easy to track end point. I mean, it may happen that the shader works for a small part of the image, anything that doesn't take 1024 iterations to render a single pixel (that's crazy, by the way)

If you have to iterate through a while loop, you should make sure it is always under control, anyway, what are you trying to do that takes imperatively a while loop to render? Are you sure there are no other ways to do it?

[porcus]

This loop calculates the collision point between the ray from the eye and
a heightmap. And this is impossible without a loop.

[sio2]

Try something like this:

Code: Select all

int maxIterations = 32; // Change this to suit your precision needs
while (maxIterations && (NB.r < height))
{
    height -= step;
    offsetCoord += delta;
    NB = tex2D(heightMap, offsetCoord);
    --maxIterations;
}

[porcus]

Thanks for your reply, but I with this I get only a white
screen with the sand glass mouse pointer. The program crashes during the
hlsl compiler tries to compile it. I get no error messages.

[sio2]

If you can post the shader we may be able to help further. You should be able to see messages from the offline compiler though...

[porcus]

Hello,

I solved the problem with this code:

Code: Select all

int i = 0;
for(i=0;i<200;i++) {
if(NB.r < height){
height -= step;
offsetCoord += delta;
NB = tex2D(heightMap, offsetCoord);
}
}

It's a little bit slow (60 fps in hlsl instead of 300 in glsl) but it works so far.

[Darktib]

This loop calculates the collision point between the ray from the eye and
a heightmap

It reminds me of picking. You can do this without a loop in one function call, but not in the shader. And I think it can be much efficient.

See arras's Shifted Terrain Scene Node, which allow you to perform this operation.

[porcus]

No that has nothing to do with picking or terrains.

[Nadro]

Hello,

I solved the problem with this code:

Code: Select all

int i = 0;
for(i=0;i<200;i++) {
if(NB.r < height){
height -= step;
offsetCoord += delta;
NB = tex2D(heightMap, offsetCoord);
}
}

It's a little bit slow (60 fps in hlsl instead of 300 in glsl) but it works so far.

What is Your graphic card? What is performane in both modes without shader? Maybe You have got enable VSync for DX apps in drivers? Difference is too big, it isn't cause by "HLSL".

[sudi]

Hello,

I solved the problem with this code:

Code: Select all

int i = 0;
for(i=0;i<200;i++) {
if(NB.r < height){
height -= step;
offsetCoord += delta;
NB = tex2D(heightMap, offsetCoord);
}
}

It's a little bit slow (60 fps in hlsl instead of 300 in glsl) but it works so far.

Maybe try

Code: Select all

if(NB.r < height){
int data = height - NB.r;
int data2 = data/step;
height -= data2;
offsetCoord += delta*data2;
NB = tex2D(heightMap, offsetCoord);
}

without any loop.

The datatype of data and data2 should be adjusted to the stuff u use there. But i think this might work.