Irrlicht 2.0 - What's in store for the future!

[Brkopac]

Hey All,

I haven't seen a thread dedicated to Irrlicht 2.0 and what is in store for the future. I'd love to broach the subject; I was wondering if the dev's have put up a road map or not yet? I'd love to hear what you'd love to see in irrlicht 2.0.

[kklouzal]

Yeah! I would love to see a roadmap, looking at that kind of stuff makes me feel like a kid in a candy shop :P
True Type Font support better be on it.

[ErUs]

Should befresh re-write using the following

• C++11 - Every platform can use this now (android, iOS)
• STL - No need to use non standard containers
• GLM - No need to re-write maths libraries
• SDL2 - This is the new standard, valve are using it too

also the new implementation should be written with the new graphics APIs (DX11, Mantle, Metal) in mind, the future will be stateless!

[Brkopac]

Should befresh re-write using the following

• C++11 - Every platform can use this now (android, iOS)
• SDL2 - This is the new standard, valve are using it too

also the new implementation should be written with the new graphics APIs (DX11, Mantle, Metal) in mind, the future will be stateless!

In regards to C++11 I agree to a certain degree. Unfortunately Visual Studio 2010 does NOT support all the C++11 features, so I think this would have to be scaled back; With that being said, I personally believe anything before Visual Studio 2010 is absolutely pointless to support and Irrlicht should be moving to more compliant C++11 containers.

In regards to SDL2, I fully agree. This should be something Irrlicht moves to.

[kklouzal]

The in-house containers need to be dropped in place of STL containers, that's a given.
C++11 features need to be adopted as such to reduce internal code complexity.
Adopt SDL2 for platform specific things.
Bring the minimum compiler version up to VS2010 and whatever the equivalent compilers are on mac/Linux.
The supplied collision and gui should be depreciated and/or removed. Alternate gui and physics libraries are simple enough to implement and anyone who will actually use Irrlicht in a completed project will eventually do this anyways.

[devsh]

SSSE3 vectors and matrices is a must

GLSL-style vector boolean operations/functions (implemented with SSE)

OpenGL-core or OpenGL 5.0 (the future API rewrite) driver implementation

Eliminate CPU-GPU stalls from the funking bad code, i.e. uploading VBOs just before using them in a draw, non-asynchronous texture uploads etc.

More GPU-side buffer upload, copy, re-cast support (i.e. VBO to FBO, FBO to VBO)

way better batching support (one VBO housing several meshes)

compute shaders and OpenCL interop hooks for OpenGL and DX11/12 (sharing textures etc.)

[CuteAlien]

My 2 cents...

Biggest move is probably getting rid of the fixed function pipeline. Nadro is working on that the most so far (which means so far he really does pretty much everything, I only hope I can be some support there some day).

TTF's - planned again as for every version *sigh*

STL - the obvious choice for a new engine. Harder for an existing engine where switching means breaking every app out there using the library. Not nice. Possible roadmap: 1. Get irr::core to work with <algorithm> (implement corresponding iterator interfaces). 2. Reimplement irr::core interfaces in STL. 3. Switch internally to STL. 4. Deprecate the old interface (preferably without ever removing them so we never have to break user code).

c++11 (and 14) ... don't know, just hadn't situations yet where it mattered inside the Irrlicht code. Probably as you guys mentioned - depends on the system we support minimally (VS 2010 lowest I guess).

GLM - I've only used it for ~2 months and don't like it yet (compared to working with Irrlicht). But same as STL, if we ever switch it has to be done without breaking interface preferably (and that always means we have _more_ code to maintain so it's not something I'm too keen on).

SDL2 - sure would be good. Simply no one volunteered to write it so far, otherwise really no reason against that.

Absolutely against removing the GUI. I'm using it constantly and have by the way finished projects with it. Same for collisions (I'd rather prefer to see some improvements there).

Can't say much about devsh's wishes as it's not my area. Everything in buffers is the way to go I've been told and not really having to do much with that stuff I'll just agree.

edit: Note this all is just my opinion, not an offical roadmap or anything!

[devsh]

the problem with irrlicht is, it gives you 6000fps on Hello World compared to lets say OGRE

but its so fragmented in using the GPU and soo simple in getting things to draw that it skips over complex GPU memory managment
I mean I'd love a setup where I could allocate buffers on GPU with irrlicht and just give meshes, transform feedback, textures etc. offsets and lengths in the buffer and manage the buffer myself... but thats an ideal
What is unacceptable, is the fact that if I want to use the irrlicht API I have to use a funking IImage as a proxy to updating a texture and lock() unlock() stalls the GPU.

so when you end up having loads of objects or varied geometry it becomes a nightmare of 3000 draw calls!
Worse yet if your geometry is dynamic! then you get a hell of micro-stuttering (Build a World stutters by 300ms on some systems when updating the minimap because Sodan coded the texture update using basic irrlicht texture functions while I was on holiday)
So then you wonder why is OGRE faster?

so you end up with shitton of drawcalls and resources being switched all the time, it doesnt matter what NVIDIA extensions get introduced... binding textures and resources is good as it lets the driver/GPU predict whats going to be accessed and fill up your caches more efficiently

And there is a host of things that gets defaulted to CPU, i.e.:
-particle animation (transform feedback?)
-billboarding (cmon!)
-skinning

it would be easy to provide opensource vertex and geometry shaders (with nice functions you can basically copy out into custom shaders if there is a need) for the above.

plus the thing we need most is more examples in the SDK, or a supplementary SDK of examples showing how to do effects EFFICIENTLY... cause a lot of n00bs (including me 5 years ago), dont understand why their very simple app is running at 20fps on a 680GTX and i7 while using Quake3 maps or models with polycount below 4k. Not many have a clue that a 2-pass textureOffset() gaussian blur is 100x faster than a naive implementation, or that in order to do HDR with a large glow radius you need to downscale 4 times, etc.

So we need examples of how to do basic things efficiently:
--Modern&Efficient HW Occlusion Culling in COMPLEX GEOMETRIES (talking about a complicated map) with multiple dynamic objects
--Modern Sun Shadowmapping Technique (such as CSM+VSM+PixelSwimmingPrevention or the newer logarithmic thing, didnt mention PSSM cause you cant reap the benefit of being camera rotation independent unlike CSM) NOT PCF
--Modern Deferred Rendering (tile based renderer, not naive approaches) with local shadow mapping manager
--Modern Instancing techniques
--Streaming resources (loading textures for materials in realtime under artificial VRAM limits)
--Modern ways of Order Independent Transparency
--GPU particle system


You can throw in a basic tutorial in Indirect Illumination via Voxel Cone Tracing and dynamic voxelization of RSMs in there

[kklouzal]

I don't think you should be so worried about breaking everyone's code. Simply yanking out Irrlicht's containers and having people use STL would mean they need to go through the code and basically fix a bunch of typos. It's not like your breaking the logic of their code, that would be mean.

As for more examples, yeah sure that would be nice, but I don't think the 'higher ups' need to be worried about creating examples, the ones that are there were plenty enough to get me familiar with how to get things going and onto the screen. If more examples are needed or if the ones that are there need improvement I think anyone has the ability to edit them since their on the wiki??

[hendu]

You can probably guess that I'd be far more conservative

- C++11 - there is no benefit and you break compilers people use. The few things that would bring benefit can be used conditionally, just like override is already being used (it enables devs to build with a recent compiler to have it check function overrides, the fixes to which then benefit everybody)
- GLM - it's big and nasty just like boost, eww
- I would not drop fixed-function support. One sw driver and the dx8 driver sure.
- any advanced techniques: with the same view. That is, optional, without breaking anything already existing.

[devsh]

If you're going to call something 2.0 then you have carte blanche to break APIs and drop stuff, its a wonderful thing to do a rewrite to support only post 2007/2008 systems... performance increase and confusion decrease

The biggest thing that should happen is that irrlicht should be threaded or at least THREAD-SAFE, because thats the way the GPU APIs are moving right now (even new OpenGL rewrite). I shouldnt have to use my time in my precious rendering thread to poll for input, and be forced to load assets (textures, models) at the expense of issuing rendering commands.

If you're not going to do dramatic changes/improvements then call it Irrlicht 1.10

You cant have a unified abstracted API for both dx8,dx9,dx10,dx11,dx12 or even dx8,dx11,dx12 because the ways we use the GPU have changed dramatically... you'll either end up holding back your dx11,dx12 implementation by not exposing codeflows impossible on dx8, or end up emulating dx11,dx12 features and not telling the user about it (FBOs in OpenGL driver and using the old RTT extension for one FBO (1x MRT) and using the new extensions/flow for more than one FBO (actual MRT) ).

As for mobiles, just do a good OpenGL ES 2.0,3.0,4.0 implementation and that will suffice for the other post 2007 GPUs

I dont see why we need the Software Renderer (which is completely bloody useless, singlethreaded, buggy and missing more than a fair share of even OpenGL 1.4 capabilities!) or dx8 (seriously?, what do you develop for - GeForce 4MX? - there are actually few GPUs that only support dx8!)

As this is a graphics engine, I have concerns over not emphasizing proper GPU usage and shying away from a GPU-centric SDK. As far as irrlicht goes, its only a very basic implementation of an Graphics API-agnostic (an attribute which I will dispute in a second) overlayer, a very satisfying and robust set of asset loaders (praise on this front guys), a simple scene graph (which needs to be more open and have an example on how to modify) and a nasty GUI.

The IVideoDriver and GPU related things are not API-agnostic, most stuff has been coded to dx9 first and the rest was retrofitted/emulated around it (using lock() and unlock() on textures and a very bad OpenGL-driver-stalling emulation of that call).

Just by code content you can see (when you open your IDE) that largest chunks of code are:
1) Asset Loaders (textures, models)
2) CPU side texture manipulation, software renderer etc. (which is not even SSE and uses wrong equations, lacks filtering etc.)
3) SceneGraph (Scene Manager and Scene Nodes)
4) GUI
5) Actual Graphics API code

So we can see that the thing that should be the ACTUAL FOCUS is the least beefy.

2D stuff and GUI is broken because it basically kills your FPS (issuing probably more drawcalls than your entire scene), lacks batching and Z-Depth/ordering. I dont know how text is done but I had to do my own implementation just to be sure.
[despite partial redesign, we still issue ~180 drawcalls from GUI due to original code structure]

P.S. from personal experience I can tell that Blending is VERY BROKEN and needs a redesign (dont want to rebind the funking FBO if I want to change blending method and coefficients, its NEEEDS to be a per material thing). Seriously guys, WTF

[Nadro]

This is my private opinion (only some points):
1. Drop fixed function pipeline stuff.
2. C++11 eg. std::thread.
3. Command Buffer.
4. STL instead of built-in containers.
5. Increase end user control over rendering process.
6. Drop collisions.

[devsh]

I wholeheartedly agree on collisions, Soren did some for hitting the blocks, ended up with hundreds of MB used to store collision meshes and 20% of render time spent finding intersection of one line... if you're not going to have hierarchical structures and acceleration data structure for collisions then just drop it, Bullet does a way better job of this.

My private opinion of (3) is that NVIDIA/AMD drivers already have 2 or 3 threads and obviously a mutexed or atomic command buffer between each, so adding another CPU buffer will just make the pipeline longer and more RAM hungry. I would definitely say no to a command buffer.

[CuteAlien]

Ehm, what would we gain by dropping collisions? It's pretty much an independent module - and one that is extremely useful for anyone doing a quick prototype. It even helped a few times already in finding bugs in other classes (like matrix classes) making the engine more stable in general. And I've used it several times already and always found it useful - it's so much less pain in a small project than adding another library if all you need is a simple traceline. And actually it has optimizations - you can use octrees. Those could be implemented nicer - like merging the 2 implementations, avoid some hacks, replacing c++ defines by variables and adding a few test. I'd like to see all that, but even without any changes it still is a very useful module. Not everyone writes games with huge levels ...

[Nadro]

@CuteAlien
There are many better libs dedicated for physics, but it's hard to not agree with you that many people need just basic collisions for simple games, so collisions system from Irrlicht is enough for those users.

@Devsh
I forgot to replay about blending. In trunk we support blending per material (both blend factor and operation), IVideoDriver::setRenderTarget method will change in future, so in this case blend improvements will be also available

Command buffer is really helpful from multithreading POV and in many cases it improves performance (please don't forget about other GPUs).