Multithreaded Irrlicht

[zenaku]

I've been looking at various ways to add multithreaded support to Irrlicht.


In the scene manager, I was thinking about adding a thread per node. It's overkill now, but it's more future proof than a more coarsely grained solution.

Each scene node would have a run() method that would listen for messages from the scene manager. The scene manager in turn would post the various render pass messages back to the nodes. Since each scene node would also be a thread, it could potentially render at any time, so you'd have to synchronize with the scene manager about which render pass each node was in. The rendering system is a serial in nature; you must render lights, then solids, then transparent, etc. In a multithreaded system you'd render all the light threads, then solid threads, etc.

One of the main problems with this solution is that some of the node lists must be sorted. They cannot be made to render in parallel if they must be sorted first. Transparent nodes have this issue. Solid nodes also have it, but according to vitek it's more of a a texture optimization and not really necessary for correct rendering. I commented out the solidnodelist.sort() in the scene managers drawAll() routine and it doesn't seem to make a difference when running the irrlicht examples, although I do have 512MB of texture ram on my machine.

Finally, their would be reentrancy issues with the graphics driver. It would have to have critical sections wrapped around the driver calls.


I dunno. What do you guys think? I could go the quick and easy route and just hardcode some extra threads into irrlicht to speed things up here and there on multicore machines, but having each node as a thread seems like such a more elegant solution.

I think another way would be to use multiple scene managers... opinions please

[CodeDog]

Instead of each node being a thread, make a thread node.
Then you can add scene nodes as children to different thread nodes.
This would give the programmer the maximum level of control over how much or how little he wishes to thread his program.

[vitek]

I can tell you right now this is will not help performance, it will hinder it. Using many threads is bad for performance.

If you want to do threading, you should seperate tasks that are unrelated, or nearly unrelated, into threads. Even better, you could create a thread pool that just executes arbitrary code. You put jobs onto a queue and the threads in the thread pool take those jobs and execute them.

Travis

[rogerborg]

It is my experience that threads are nearly always a bad idea. A very, very bad idea. In the general case, if a thread is busy waiting on an external resource, then yes, you may be able to progress another thread, but the developer time you will spend finding and fixing contention and deadlock issues far outweights the performance benefit. Your test app will run faster, but you will find a slew of deadlocks and artifacts in your real game two minutes after it goes gold.

In this specific case, there are only three significant render phases that take up most of the GPU time and bandwidth:

* Solid
* Shadow
* Translucent

Translucent must be done in order, or horrible artifacts will occur. Shadow, OK, that might potentially benefit slightly. Solid, no. If materials are rendered unsorted, then any increase in CPU utilisation will (IMO) be outweighed by context switching in the GPU.

I'm not saying that you shouldn't try this, just that it would be a significant change, with the potential to seriously screw up all existing applications - so you'd need a doozy of a test suite, and a crystal clear understanding of the process and the potential deadlocks - with no guarantee that we'd see a gain in performance at the end of it.

You may be able to tell that I am less than keen on this.

[buhatkj]

I read recently at the URL below that Valve has made a deep investment in optimizing the source engine to use threading, such that it may take advantage of the current dual core, and upcoming quad core CPU's. I dunno if I would be so quick to jump out and criticize the idea of putting some thought into adding some kind of threading to irrlicht, i doubt valve would spend so much time/money on it if there were no advantage to it.
http://www.bit-tech.net/gaming/2006/11/ ... gin/1.html

I don't think it can be so conceptually simple as a thread for each scene node though. i think you would need to pick apart the rendering process, and look at each step and see which parts make sense to run threaded, and which must be synchronous. for example, let's say we have a routine that just goes through the vertices in each mesh, and applies transformations to them for animation. that can probably be threaded. but as you say, the drawing of lights, transparent objects, etc, needs to happen in a certain order.
valve called this sort of thing hybrid threading in the article. like some things are threaded, some aren't.

[rogerborg]

I like the idea of putting some thought into it. Sorry if I came across as negative, but the plain fact is that Valve represent the top of our profession. We should aspire to emulate them, but not shoot ourselves in the shoot ourselves in the shoot ourselves in the sho-

Dammit

shoot ourselves in the foot trying to overreach.

[zenaku]

The idea of a thread per node is way overkill, like I said before. I'm well aware that it would only hurt performance.

The idea is that when Intel talks of 80+ core CPUs, at that point it would be advantagous to have each node a seperate thread. I'm looking at what is more scalable in the future than what is most immediately applicable. If we want to go what's most immediately applicable we should optimize for 4 cores, but IMO that's short sited.

Thanks for all the comments so far

"... but the developer time you will spend finding and fixing contention and deadlock issues far outweights the performance benefit."

When CPUs have 80+ cores, I'll have to disagree here.

[zenaku]

Instead of each node being a thread, make a thread node.
Then you can add scene nodes as children to different thread nodes.
This would give the programmer the maximum level of control over how much or how little he wishes to thread his program.

I like this idea. Then you can always define the # of threads. With a thread per node you don't get that option.

[hybrid]

NVidia and ATI have several paper on the use of multithreading in realtime 3d graphics. All target at special calculation tasks, animations, physics, etc. You simply cannot render from separate threads because gpu is not simultaneously accessible.
Unfortunately, this does not give a simple option to make some benefit of multiple threads. However, I/O is a very interesting task to parallelize. This would require to get a thread safe texture cache implementation or a decent separation of image creation and texture addition.
The ordering of the nodes should be preserved, the next release will give some improvements for OpenGL with less texture changes. It will definitely help more than using many threads.

[zenaku]

NVidia and ATI have several paper on the use of multithreading in realtime 3d graphics. All target at special calculation tasks, animations, physics, etc. You simply cannot render from separate threads because gpu is not simultaneously accessible.
Unfortunately, this does not give a simple option to make some benefit of multiple threads. However, I/O is a very interesting task to parallelize. This would require to get a thread safe texture cache implementation or a decent separation of image creation and texture addition.
The ordering of the nodes should be preserved, the next release will give some improvements for OpenGL with less texture changes. It will definitely help more than using many threads.

Yeah I think you guys are right. Thread per node is a bad idea. No matter how many cores the CPU has, if you have to have a critical section around the driver calls, it's pointless. They will all be spin locking waiting for their turn to call DrawIndexedTriangleList or whatever...

[Legion]

There is no easy solution to just "threading" code.
Alot of special care has to be taken when writing threaded programs.
As a rule of thumb, if parts of your program can be independant,
you can easily thread them.
Eg. running your soundengine from a separate thread from the 3dengine.
However, splitting a 3d engine into threads would be very hard.
Because alot of parts needs to be synchronized you'll end up with alot of
spinning.
However some clearly obvious things such as maploading can be threaded
quite easily.
This will be the case in the game im currently working on,
where maps will be loaded before you reach them. (seamless world)
So the thread will precache them and eventually drop maps from the cache.

[zenaku]

There is no easy solution to just "threading" code.
Alot of special care has to be taken when writing threaded programs.
As a rule of thumb, if parts of your program can be independant,
you can easily thread them.
Eg. running your soundengine from a separate thread from the 3dengine.
However, splitting a 3d engine into threads would be very hard.
Because alot of parts needs to be synchronized you'll end up with alot of
spinning.
However some clearly obvious things such as maploading can be threaded
quite easily.
This will be the case in the game im currently working on,
where maps will be loaded before you reach them. (seamless world)
So the thread will precache them and eventually drop maps from the cache.

I've written multiple large applications that are multithreaded. It's not difficult. Really, it's not!

What is difficult is converting old code designed to run in a serial manner to something designed to run parallel. If the design was always parallel, making it threaded is trivial. Unfortunately rendering is one of those serial problems. I'm very familiar with multithreaded programming, but not so much with 3d rendering. That's how this thread got started

My own conclusion is that as long as the driver calls are not thread safe, there is no point in making the rendering system threaded.

[CodeDog]

I write multi-threaded programs every day. Threading is not bad; bad threading is bad. The trick is choosing what to thread. You would not for example use 2 threads that write to the same texture at the same time, they would just end up taking turns. However, you might want a thread that reads huge files (while the main process continues the render loop at high fps) and when done notifies the main process thread that the "node" is ready to be used. Only once the “node” has finished being read from the disk is it handed off from the worker thread to the main process thread to be added to the scene and rendered with the other nodes.
This way you could start loading a zone when the player gets close to the boarder without the usual 10-30sec game freeze you see in non threaded games.
The first candidates I look at when picking what to thread.
Hard drive access.
Database queries.
Network access.

At the very least any modern program with a GUI (or 3D interface) should have the main process run the GUI and have one worker thread for doing everything else.

[rogerborg]

That sounds like a reasonable start. Can you lash up a patch for that so that we can regression test it and benchmark the performance gain on various system?

[CodeDog]

I have a worker thread for my game but the thread code is windows specific.
The real challenge here would be to make a cross platform threading model for Irrlicht. In windows I would normally use AfxBeginThread for trivial threads or derive a class from CThread if I want two way communication, but in unix I would need to rewrite the code to use fork() and in linux I would need to use vfork() unless we include a cross platform threading library like pthreads.
The other issue is inter-process communication. In windows we use sendmessage or postmessage whereas in unix/linux you would need to use sockets or pipes.