Understanding (GL) rendering performance with terrains...

[Kidinnu]

So I'm interested in rendering very large terrains, and was poking through CTerrainSceneNode.cpp to understand how it worked and figure out how well it matches up with the GeoMipmap paper. I noticed that during the pre-render phase it was regenerating its index list from scratch every frame.

This seemed odd, so I tried tweaking it so that every patch had its own list of indices, and only patches that changed their LOD recomputed their index list.

Frame rate dropped in half.

So, questions:

1) what kind of support is there for profiling inside Irrlicht that I'm not finding? If none, any suggestions for what to read? Is there any way to turn on timing for various calls (like checking the

2) this seems like a lot of performance loss for going from 1 render call to 64, although a 16k vertex buffer is being passed along; is there some way to hack Irrlicht at a lower level to make use of the fact that it's the same vertex buffer?

3) <the big WTF moment> vertexCount is only used for one thing inside the openGL implementation of

Code: Select all

drawIndexedTriangleList(const S3DVertexTangents* vertices, ...)

:

Code: Select all

	const S3DVertex2TCoords* p = vertices;
	ColorBuffer.set_used(vertexCount);
	for (s32 i=0; i<vertexCount; ++i)
	{
		ColorBuffer[i] = p->Color.toOpenGLColor();
		++p;
	}

... which is happening on every rendering pass? So if I set vertexCount to some reasonable small number, like 6, my code runs at hundreds of fps and is visually correct (since it's textured anyway).

Looks like a performance bug?

[mm765]

any suggestions for what to read?

yes, read your pms (see top of page)

[hybrid]

These forums are lately being filled with much conspiracy. What in the world could be so confidential that you wouldn't share it with us, mm765

Back to the questions of Kiddinu:
1) you should use general profiling tools to find bottlenecks. If you want to know where you used Irrlicht structures in an inappropriate way you should probably present some code.
2) more efficient render techniques can and will be added. Hardware buffers are already part of IrrSpintz. Some additional architectural decisions are to be made before, though, to add these methods without loosing the simplicity of Irrlicht's API. Any suggestions or proposals are welcome, though.
3)Hmm, never touched these functions. But IrrSpintz also has some improvements on these things.

[mm765]

These forums are lately being filled with much conspiracy. What in the world could be so confidential that you wouldn't share it with us, mm765

heya hybrid
it has nothing to do with conspiracy or confidentially.

i just wouldnt find it to be good style to mention a different engine on in this forum and so i dont do it.

since im personally interested in terrain-rendering i answered to his post yesterday with the suggestion to take a look at another engine that has a working paging-terrain and the more "aggressive" culling he was talking about and that i would be interested in different terrain-implementations and how he can contact me if he were interested in talking about it.

since he posted this message today i assumed he had overlooked the pm and wanted to make him aware of it.

i hope that this satisfies your curiosity

[gfxstyler]

are we talking about ... LIGHTFEATER ??

[hybrid]

You never had problems drawing attention to lightfeather before. And no one ever got upset on this. So this post really confused me.

[mm765]

im sorry but you are mistaken on that. the only times i mentioned LF were when someone had mentioned it before and i felt the need to clarify things or wanted to ask for details (exactly 3 posts alltogether).

[hybrid]

Sure, LF is not that old, it was NX or NX++ before. At least I found three posts on the first two (of 11) pages of posts by you where some words were "(XXX has something for that)" where XXX has to be replaced by either LF, NX or NX++. But I really don't care how often someone's talking about solutions in other engines. It's usually even a very good thing to take a look at other solutions. It was just this really curious reply I was wondering about, so never mind.

[mm765]

Sure, LF is not that old

a bit over a year now

so never mind.

i dont mind at all - you asked i answered

[Kidinnu]

Yep, mm765 had already pointed me at Lightfeather in a pm, but I wanted to finish at least one toy project with Irrlicht before looking at another engine, especially since Lightfeather seems to be Irrlicht-derived.

At first glance, though, that'll be my problem with it, rather than a nice carryover of what I've learned. I've probably been spoiled by spending the last five years in Java-land; Irrlicht seems annoyingly hard to extend. It's not just the language, since SGI did a lot better on that count with Inventor and Performer in the early 90s, and they were both C++.

As I am given to understand in a thread on the beginner help page, if I want to "extend" Irrlicht - by which I mean, say, write a new kind of terrain node, or a variant FPS camera node - I have a few choices:

*) reimplement terrain and the triangle terrain selector classes, or the camera and fps camera classes
*) write code inside the library
*) edit the library source to export symbols from the classes I want to reuse

{Is it a bug, or a feature, that Terrain uses CTriangleTerrainSelector internally rather than I?}

I suppose what I'm looking for in a library right now is extensibility, not ease-of-use; since it's Irrlicht-derived, I'm afraid Lightfeather will follow the same general path. Lightfeather has more functionality, but at first reading the code is under-commented and not as high a quality. Hopefully I'll have time for some performance testing and checking the algorithms tomorrow.

(Not that Irrlicht is perfect there, either - there's some dodgy math in the Geomipmaps article that the terrain implementation is based on, and as far as I can tell the terrain node in Irrlicht doesn't even try to use that math, but has a blatant hack that you can visibly see causing errors.)

[stodge]

You can submit patches. Irrlicht's design is fairly simplistic (IMH) so you may have reached its limits for your requirements. You may have to look elsewhere or get into the code and improve it.

[Electron]

I suppose what I'm looking for in a library right now is extensibility, not ease-of-use; since it's Irrlicht-derived, I'm afraid Lightfeather will follow the same general path. Lightfeather has more functionality, but at first reading the code is under-commented and not as high a quality.

I don't really want to get into a flame war here, so I'll try not to be at all offensive or provocative, but a couple things there seemed unlikely/surprising

Lightfeather is originally Irrlicht-derived however nearly everything except the core:: namespace is fairly heavily rewritten and the rendering/resources pipeline bears no relation to Irrlicht's.

I'm afraid I don't quite get the part about following the same general path. In both engines you can add new types of scene nodes, new types of cameras, new types of models, new types of node animators/controllers, new gui elements, etc without recompiling the engine. So yes, they are similar in that regard, but I fail to see a problem. I don't remember what the case is with Irrlicht, but LF exports all symbols from the DLL by default
. LF also has multiple developers and is very open to patches. Is there something more you expect/need?

The code in LF is under-commented I will admit, perhaps more so than Irrlicht's in some places. There is, however, documentation on the site and an active IRC channel where we are happy to discuss the code and answer questions.

As for the code quality, I'm highly curious what makes you think the quality is lower than that of Irrlicht (unless I misunderstood you). LF's rendering pipeline is (IMO) much cleaner than Irrlicht's. I do not claim that LF's code is 100% percent perfect, however. If you can point out any specific instances of poor-quality code in LF we would be happy to fix them. (Please note that memory cleanup in the GUI is currently very poor. We are aware of this and it will be fixed).

If you like Irrlicht better than LF that's perfectly fine. Different engines for different people. I'm just a little curious about the basis for some of your comments.

[Kidinnu]

Electron:

I was afraid I might offend with that; sorry.

Extensibility - the more I learn about how MS and C++ work the more I think my annoyance with Irrlicht is just that the symbols aren't exported from the DLL by default. I have an old abhorrence of touching anything in library code, even as minor as adding exports, since you then have to make all those edits every time the original author updates. If I correctly understood the people who answered a previous post of mine, I can't add new cameras in Irrlicht without recompiling the engine or duplicating several classes. (e.g. if I want to make a new camera that inherits from CCameraSceneNode, I'm out of luck, because it isn't exported - I must either recompile with exports or copy the code into my own project and rename it.)

I've had a little trouble with the linker when I have gone in and tweaked exports in Irrlicht, but I'll chalk that up to user error and assume it wouldn't happen in Lightfeather or that I'll learn to use the tools better someday.

On quality and comments: the necessary disclaimer is that my day job is as a professor. I'm fairly harsh on code that students turn in that isn't commented or looks like a rough draft, and the contact I've had with the real world suggests that the software shops that I think are doing good things are just as anal as I am on large shared codebases.

As mm765 figured out elsewhere, I started looking at Irrlicht/Lightfeather because I was thinking of a terrain-oriented project. The Lightfeather terrain classes hardly have any API docs at all, and CLargeTerrainModel (the only source I've poked through) is really sparsely commented internally, too - the comments there look useful, but I want more. Meanwhile there are a couple of things in the code that would be an automatic -5 points on the "quality" score if I were grading.

I probably got spoiled by starting this summer's playing around with the WildMagic engine, which has two (pretty decent) books about it that I've assigned as textbooks before. I'm also finding more and more flags raised by the Irrlicht terrain code, which is why I was hoping for more comments in Lightfeather: I thought I'd read a claim that Irrlicht's CTerrainSceneNode implements Geomipmaps, but when you closely read the code the algorithm is way, way off, and there's absolutely no clue why. There are also some things that look like glaring bugs, but without comments I can't be sure.

Work and family are keeping me from putting in as much time as I want; I'll stop talking about Lightfeather until I've been able to put in 8 or 10 hours with it, but that may be most of a week away. Being really old (an early mammal, if not a dinosaur) I don't have an IRC client installed - heh, maybe it's finally time to take the plunge. If I do end up working with Lightfeather/CLargeTerrainModel, I'll comment & refactor the code myself as I come to understand it.

N.B. While writing this post I looked at Lightfeather's CGeoMipMapTerrainModel. Decent comments, you've corrected the things I thought were bugs in Irrlicht, and the code's a bit cleaner. Nice.

[Baal Cadar]

If you were my professor and commenting this way on my work, I'd be knocking on the dean's door right now.

Too much fluff, too little crunch. I still don't know what makes Lightfeathers less quality than irrlicht. What are (a simple example suffices) the -5 points in lightfeather?

WildMagic is a nice example of an academic engine. Nice demonstration of concepts, but totally useless in practice. The Pascal program among the C/C++ programs. Fashion versus function ( http://www.ctrlaltdel-online.com/comic.php?d=20060208 )

[mm765]

Electron:
I was afraid I might offend with that; sorry.

you got that wrong - no one is offended - we are eager to make things better. your comments however bear too few details for us to know what exactly is wrong/could be done better. but this is certainly the wrong place for that discussion.

for the rest of my answer (as usual) see your pms