irrlicht on linux inside windows managers

[cheshirekow]

Hello Devs,

I'm aware the most users of the engine are creating games that are intended to run full screen. I'm in the camp that would like to use irrlicht to render a scene inside a gui. I'm also in the camp that targets linux. I'm aware the intersection of these two is actually quite small, but I haven't let that stop me.

I recently posted a solution for getting irrlicht to render inside a window in wxWidgets: http://irrlicht.sourceforge.net/phpBB2/ ... hp?t=39719. However this solution is somewhat fragile and there are a couple of things bothering me about it. I've finally had time to dig through the source and figure out exactly what was going on. I'd like to describe to devs what I've found and ask them for some feedback. In particular, is it within the interest of the devs to support irrlicht from within a window manager? i.e. is the windows gui support incidental? Or is this something you really wish for the engine in general? Hoping that it is the latter, I've identified a few problems with the current implementation of CIrrlichtDeviceLinux that are preventing this from being reliable on linux. I'm not sure about on Mac, but since mac comes with x11 I assume the ideas will extend to that OS. Here is what I've found (line numbers are from today's svn trunk, rev 3402):

1. Currently, when the library user passes in an X11 window ID through SDeviceCreationParameters, the implementation of CIrrlichtDeviceLinux makes a few assumptions:
   1. the X window does not have a glxWindow associated with it (CIrrDeviceLinux.cpp:694)
   2. the GL Context has not been created (CIrrDeviceLinux.cpp:698 & CirrDeviceLinux:719)
   3. irrlicht will be responsible for managing the X11 message pump (CIrrDeviceLinux.cpp:846)
2. Despite these assumptions, the COpenGLDriver class (COpenGLDriver.cpp:533) does not recieve the context and the display created by CIrrlichtDeviceLinux, but rather picks up the current display and context, assuming the device has set it up (this is why it actually works on linux right now)
3. The X11 message pump is handled by the window manager (Gnome, or presumably KDE, XFCE, etc)
4. GTK exposes the underlying X11 data structures, so they can be retrieved and passed into irrlicht
5. wxWidgets exposes the underlying GTK data structures, ...
6. Despite my expectations, when passing an "ordinary" X-window in to irrlicht, it cannot create the glx window and the context on it's own (this could me something of my own fault, but this is kind of a small point)
7. Both GTK and wxWidgets have OpenGL objects for managing openGL windows
8. Either GTK or wxWidgets does not reliably size the windows prior to drawing them (I'll explain why this is significant)

These facts have lead to a couple of roadblocks in trying to integrate irrlicht with wxWidgets on Linux:

• We can create an opengl window, and context from within wxWigets or GTK (wxCanvas or GtkGLExt), but we can't tell irrlicht that we've done so, so we get some error messages in the console and (what is a bigger concern) future versions of the engine may fail unnecessarily in the device construction
• The opengl driver can initialize correctly because the opengl context and related Drawable created by the window manager is still current
• Sometimes (isn't this cute) when the irrlicht device is created it will detect an X window size with the correct height but a width of 1px. This happens reliably on x86 Linux but infrequently on AMD64. I'm guessing it's the different versions of GTK or Gnome for the different platforms. And, since the window manager library is taking care of the X message loop, the irrlicht device never understands that the window size has changed. In fact, we have no way of passing messages from the window manager to irrlicht. In any case, we have IVideoDriver::onResize() but we don't have IrrlichtDevice::onResize(). As a result, the rendered scene is one pixel wide, stretched across the width of the actual window. Now, if a library user is also using a window manager, they should probably assume responsibility in their code integrating with the window manager to enable/disable rendering with irrlicht when the window is not visible, but I think we'll need a way to pass resize, minimize, restore, etc kinds of events from the window manager into irrlicht.

I'd like to get some feedback on these issues. In particular, are there ways with the current API to address them (that I have just missed)? If not, I'd like to make a formal feature request to enable integration with a window manager in linux, and for these things to be considered in future releases (so as not to make it any harder to integrate with the window manager)

[CuteAlien]

Thanks for the work. I had seen your wxWidgets post already and found it interesting. And yes - I'm very interested in running it within another GUI, I just never found the time so far to dig through that. I would even very much like to have an integration example one day in our examples. Unfortunately I'm right now a few days before a beta-release in a death-march project (with master release also following this month), so just putting your info on todo for now.

But I'm pretty sure others are also interested in this.

[cheshirekow]

That is good to know. I guess the important thing to note then is that it is a low priority for y'all. In the meantime, I have added to my wxWigets+Irrlicht+Linux post a patch that adds a new type of event (Window event) and changed the linux irrlicht device to override the null device and intercept these events. This adds the ability to resize the window and suits my purpose just fine, but I would prefer not to do a lot of work that is contrary to the direction the official sources will go. If there won't be any movement along this direction soon, I will continue to work with the modified source then.

[hybrid]

Well, simply do so. If it fits your needs, it's ok. If you want to prepare a patch which is likely to be added to the engine, you need to invest lots of work in preparing and altering the changes. You need to discuss things, and try to adhere to the Irrlicht way of doing things. And that's not a matter of a few days. At least not for such general things as the system/device event.

[cheshirekow]

Well, I wanted to get a sense of if there was anything in the works so I didn't replicate any effort, or end up with a source tree that was uncontrollably out of sync with the repository. Also, I was hoping to get some feedback on my understanding of the issues to avoid wasting time if I was overlooking something.

[hybrid]

I don't know if it's my or your limited caps in English, but this

If there won't be any movement along this direction soon, I will continue to work with the modified source then.

does not look to me as if you are interested in a proper discussion about the further development.
So here are some ideas about the further development: The resize problem will be a larger thing. Our idea is to add some kind of system event (just like mouse event, keyboard event, etc) which can be handled in the event receiver to do certain things. I'm not sure if this would cure your 1px window problem, as this seems to be a problem with the window handler and should be fixable with the current system already.
Another thing is the window id passing. It could be viable (but was not yet tested) to change to the ExposedVideoData instead of a plain window id in the creation struct. This would allow to pass on the opengl context and other things, maybe even merge in the device adapter number etc.
For both things, it would be good to have reasonable test code (in the feature request tracker at SF) and continue the discussion here to find out the necessary requirements for a proper solution.
Depending on the input this might be solved withtin the next weeks, but it might as well take until next year. No promises for any running project!

[cheshirekow]

Thanks for the reply.

does not look to me as if you are interested in a proper discussion about the further development.

Meh. I would be interested in participating in a proper discussion, if there is already a thread for this idea I'd be happy to give my input there, or if this thread should change it's title and become the place for that discussion... that sounds good to me too. Or if it's better just to post a test case on the issue tracker and let that be the place for the discussion, that's cool too.

I like the idea of a system event. That's basically what I did in my wx example. I created a "window" event so I can inform the device of the window size change. It's nice because the API doesn't require a change. The postEventFromUser already provides an entry point for these external events. This doesn't quite handle the 1px thing, but I think it has to do with the projection matrix remaining unchanged when the screen is resized. In other words, I think what I need to do is fix the camera to match the aspect ratio of the new window size so that the appearance is the same as for the smaller window. This is my first order of business for my wx example.

The exposed video data sounds like a promising idea. That would provide more options for whether or not a gl context is already created and whether or not glx has been notified of it's target x window. I have a few more things I want to try on my wx example, then I will be happy to put it up and follow the discussion on the issue tracker.

[hybrid]

The trackers are not good for discussions. Hence, just add a ticket with a short text and a link to the forum thread to the tracker. Please do this for each issue you need, and for each bug you find. And of course not just you, but everybody here
We can keep the discussions about the issues you mentioned in this thread.

The user even is not meant for such system events. There are real user events on both Windows and Linux, which are passed through via that mechanism. So we need to add a new event type. This would also require to think about possible events, which would go via the system event mechanism, to make sure how the internal structure should look like.

[cheshirekow]

I agree, user events and system events are different and should probably be treated that way.

I've discovered a few things recently and I would like to point out what I've learned for future reference. I was previously a bit puzzled why irrlicht worked when attached to a wxGLCanvas but not a normal wxWindow (or wxPanel, etc). As I mentioned in the first post, since the COpenGLDriver class queries for the drawable and the gl context by using XLib and GLX library calls, the failure of attaching to an X Window in CIrrlichtDevice is kind of moot when using a wxGLCanvas. However, this kind of a fragile solution. By making a small change to the source, I have managed to get the irrlicht device attached to a normal wxWindow without any errors.

What I have learned is that in CIrrlichtDeviceLinux::createWindow(), irrlicht attempts to create a glx window attached to the xwindow whose ID is passed in via SIrrlichtCreationParameters. The problem with this is that the window that was passed in might not have the same visual attributes that are implied by the other parameters in SIrrlichtCreationParameters. I do not believe it is reasonable to require the library user to provide an XWindow whose attributes satisfy the requirements of SIrrlichtCreationParameters because the GUI Toolkit will be the one who set's those up. The work-around I've used is to create a new XWindow, as a child of the window that the user passes in. The new window has the attributes necessary such that the new glx window can be attached to it. This change introduces an additional requirement, in that the device needs to handle a size-change event so that it can resize the new XWindow to match it's parent's size, whenever that parent's size is changed.

A huge advantage of this approach is that it is easy to make this thread-safe. I know multi-threading is a taboo topic on these forums, but when integrating a rather "heavy" irrlicht-based application with a native GUI, it would make for a poor user experience if the GUI was blocked for lengthy render calls. As an alternative, the irrlicht device can be created in a separate thread, and then render to it's own XWindow (still a child of the GUI's XWindow) without blocking the GUI thread... allowing all the GUI controls to remain snappy and responsive. This, of course, assumes that the GUI thread will need to communicate with the render thread relatively infrequently, but I see that as a common application scenario.

I will be uploading my new (working) wxIrrlicht code to the thread in CodeSnippets (at http://irrlicht.sourceforge.net/phpBB2/ ... hp?t=39719 ) which demonstrates integration with wxWidgets via a custom wxWindow class, and how to enable various options for

• rendering in the main thread (event based or timer based)
• rendering asynchronously in a separate thread
• rendering continuously in a separate thread

P.S. I added a feature request to the tracker, id: 3081163, https://sourceforge.net/tracker/?func=d ... tid=540679