Feedback about STL and irr::core

[CuteAlien]

Irrlicht and STL is a question which pops up every few years. Right now Irrlicht has own implementations for the container classes core::array (similar to std::vector), core::list, core::map and also it's own core::string classes.
That made a lot of sense back at the time, but it's unlikely anyone writing a new engine would still do that. It has still some advantages, but also a lot of disadvantages. I'll summarize a few...

*** Why keep own classes ***

- We already use it. Staying with it means we stay compatible and don't break any applications using Irrlicht. We don't introduce new bugs moving to STL. Any kind of online info and tutorial will stay valid, while a lot of it would be outdated when switching. This is a pretty big break - Irrlicht with STL would kinda be a different library.

- More control, we decide what we need and implement exactly that, no more no less. When we need changes we can just code them.

- Independence of STL. No need to link to another external library. That makes distributing applications somewhat easier on some platforms. It can avoid license troubles. It ensures we always have core classes available on new platforms immediately (as long as c-lib exists). Irrlicht can even work on exotic platforms which might not have STL (do those still exist?).

- Switching to STL can make language wrappers harder to write. Maybe even impossible (might depend on the language). I'm not having much experience with that myself, so maybe people working on language wrappers can give me some feedback there.

- Visual Studio makes it hard mixing debug and release builds of libraries and apps using STL.

- STL is maybe (arguably) somewhat less intuitive for beginners, especially when arriving from other programming languages.

- It's rather easy to create beginner unfriendly code with STL. It's more suited towards c++ experts. Things with STL can look complicated and produce really ugly compiler messages. Part of it is that STL allows building more complex structures very easily - which can be a feature as well as a curse.

- Many STL containers are not suited well for games. Meaning there's a certain risk that Irrlicht programmers go wild with container classes without realizing the costs (I've done and seen that in the past enough in the game industry).

*** Why switch to STL ***

- It's the standard. Nearly everyone using c++ is (or should be) familiar with it.

- It's far more powerful. It has more features than Irrlicht. It's architecture is easier to extend (thanks to the clean iterator - algorithm separation). This matters a lot as missing features in Irrlicht core classes are regularly slowing down our progress. We have the choice of switching to STL or implementing even more core classes in Irrlicht. And we never seem to find the time to add those features. Some things we need are for example algorithms (more flexible sorting) and maybe thread support in the future (which is finally supported in STL since c++ 11).

- Stability. STL is better tested. Irrlicht core classes had a few bugs over the years, while that rarely happens in STL.

- We don't have to maintain STL ourselves - yay!

- STL performance is generally better than the performance of corresponding Irrlicht classes. I'm not even sure why that is the case and maybe we could improve Irrlicht. I wrote some speed-tests to compare most functions of the Irrlicht container classes and Irrlicht was never faster, while it was beaten by STL often and sometimes very clearly.


Obviously we wouldn't deprecate Irrlicht core classes immediately. The easiest change might be to use some STL and only internally at first. And trying to get Irrlicht core classes closer to STL (thought we wanted to do that for years without ever finding the time/manpower). We probably should prevent mixing STL and Irrlicht containers.
Or we break hard. Replace any internal use of Irrlicht containers with STL and also change the interfaces. Keep Irrlicht core classes around for a while, but no longer use them.

OK, enough info... give me some feedback where you stand on this. And post pro/contra arguments which I have missed. At this time there are no concrete plans yet - I just want to get some feeling on where the community is standing on this.

[AReichl]

look at this:
http://www.open-std.org/jtc1/sc22/wg21/ ... n2271.html

The G3D-Engine for example strictly avoids STL, because they claim their own
solution is optimised for gaming ( e.g. data alignment ).

Others even use BOOST ( the other extreme, hehe ).

I would say, if irr::core has SPECIFIC GAMING RELATED advantages over STL
( e.g. data alignment, SSE, ... ) then keep it. Maybe "upgrade" it to be superior
to STL or at least add some C++11 functionality ( e.g. "move" ).

In the end it's all about Irrlicht beeing "lightning fast".

[hendu]

This was one of the things that most pissed me off in Qt. They duplicate absolutely everything for no reason at all, and of course the API was all different to standards.

In comparison, irr didn't bother me much. Most methods were named the same, and irr doesn't force you to use the containers much, very rarely are they passed in the API.

I voted to switch to STL just for the smaller maintenance overhead. But do keep it working with all widely used compilers, not just new shiny - no C++11!

[CuteAlien]

C++11 is nearly half a decade old by now, shiny new would rather be C++14. Admittably the g++ compiler I use currently still needs a flag to compiled C++11 code. But Irrlicht 1.9 will not yet use C++11 and the next Irrlicht is probably going to take a while again. Are there still C++ compilers which don't support C++11? I mean besides the old Borland (Embarcadero) command line compiler (we gave up on that already).

The problem is that if we want to add things like thread safety we will otherwise implement wrappers from scratch again. Also C++11 has a few nice features. I love the variable initialization in the class declarations for example, that reduced so much stupid initializer list overhead code.

[Nadro]

I think that we should switch to both STL and C++11 after v1.9 release. AFAIK only old and exotic compilers may have problems with C++11, but C++11 has a lot of nice features, thats why I would like to use this standard during Irrlicht development.

BTW. Thanks CuteAlien for create this thread.

[Virion]

I'm all for C++11 for maybe irrlicht 2.0. It's time for irrlicht to move forward otherwise there is no reason to call it Irrlicht 2.0.

[hendu]

I regularly use gcc 4.2, 4.4, 4.7, 4.8, as well as 5.2. Only the newest of those supports C++11 properly.

If I want to make a portable binary, without bloating it by several megabytes with a static libstdc++, I have to build with an older gcc.

[CuteAlien]

I guess supporting gcc 4.8 won't be a problem, but yeah - anything older, especially before 4.7 would indeed be tricky. I'm mostly working with 4.7. Not sure how well the thread support in that is as it's still missing all the thread related memory protection stuff.

The question for even older gcc versions is - do you think you will still use all those by the time Irrlicht 2.0 will be released? Given our usual speed... that's not going to be this year.

I have to admit there's nothing in C++11 which I absolutely need for Irrlicht. It's just that I already work with that stuff in other projects and it's simply more fun to work with ...

[hendu]

Just use pthreads for threading? There's a library which windows users can install, and it's native on linux and mac.

[CuteAlien]

Using STL it would make sense to use the STL version of threads. Otherwise we have yet another library (ever lib causes linker troubles sooner or later, there's no exception to that).

[Foaly]

Sorry, but which STL are we talking about here?
I had never heard about it before, so I tried look it up, but it looks like it is used to refer to different things:
The C++ Standard Library
The Standard Template Library (of which multiple implementations exist.)

Would it mean that STL is included in the Irrlicht.dll, or does it come separately?

Generally, it is possible to wrap any C++ code for .NET.
If "STL" was used in Irrlicht, it would require to wrap at least all the STL-classes used by Irrlicht and every place where they are used.
Though, .NET does not support templates, only generics, which are compiled at run time, so only would not be possible for the user to create template-instances, only to use those provided by Irrlicht.


Since .NET has it's own container classes and interfaces, there are no advantages for us.
So I'm against it. I don't care whether you use it internally, if that improves the performance.

[CuteAlien]

The Standard Template Library is the C++ Standard Library. And yeah, there exist multiple implementations (but the same is true for OpenGL for example). It wouldn't be included with Irrlicht. There is no question that switching to it will make application linking and distribution somewhat harder, it's one of the disadvantages (doesn't matter too much to me as I always use STL in my apps anyway...).

[chronologicaldot]

http://stackoverflow.com/questions/5205 ... -all-about
Funny to see that come up... But we're just talking about irrlicht containers.

We could all switch to Ogre like everyone else. Just kidding.

if you don't have time for updating the methods, how are you going to have time for switching out all the containers?

[quote="CuteAlien"]- STL performance is generally better than the performance of corresponding Irrlicht classes. I'm not even sure why that is the case and maybe we could improve Irrlicht. I wrote some speed-tests to compare most functions of the Irrlicht container classes and Irrlicht was never faster, while it was beaten by STL often and sometimes very clearly.[quote]
I believe the STL is faster because its implementations are compiler-specific, so it can directly utilize built-in commands. e.g.

Code: Select all

_GLIBCXX_MOVE3(__position + 1, end(), __position);

Taken from the vector::erase() in vector.tcc ( https://gcc.gnu.org/onlinedocs/gcc-4.6. ... ource.html )

Code: Select all

 
00133   template<typename _Tp, typename _Alloc>
00134     typename vector<_Tp, _Alloc>::iterator
00135     vector<_Tp, _Alloc>::
00136     erase(iterator __position)
00137     {
00138       if (__position + 1 != end())
00139     _GLIBCXX_MOVE3(__position + 1, end(), __position);
00140       --this->_M_impl._M_finish;
00141       this->_M_impl.destroy(this->_M_impl._M_finish);
00142       return __position;
00143     }
 

Whereas our own implementations don't get to the nitty gritty assembly code. Our irrArray erase() even says "MAY BE SLOW" or something to that effect.
I made my own doubly-linked list, which was faster and more flexible than the built-in irrlicht containers (though more memory-consuming), but there is no common interface, so it's inconvenient to plug in my own.

A band-aid would be writing in typedefs like what was done for snprintf, and that way the transition could be alot easier, assuming that most irrlicht container functions are named the same as the STL functions. This is, of course, a temporary time-consuming fix, which seems like a waste of precious time.
I do understand the concern with not breaking the API - at least, not until after 1.9. I'm under the impression that, despite irrlicht being used by countless programmers, there seems to be a rather small active developer community, and we all can keep up with the changes, so if an all-out-change did occur, those of us who'd notice it would all be aware of it (though making an announcement on the main website about the change and compatibility issues would be a must).

I'm tired. Pardon me if I typed something without thinking.

[Nadro]

It looks like we should switch to C++11 + STL after v1.9. Good, I'll use those standards in the new FVF branch.

[thanhle]

I think STL is great. Once implemented, you don't need to provide much support for it.
The engine need changes. Thread safe and Multi threading is really needed.

-----
Don't talk about boost, it is too big and not needed in any 3D rendering or game engine.