Why doesn't Irrlicht use STL?

[vitek]

As has been mentioned, it is not that the containers are necessarily faster, but that they COULD BE faster

This is not the only advantage. The Standard C++ Library containers have well defined behavior and performance characteristics. Depending on the implementation they are usually very well tested. Unlike Irrlicht, they actually have test suites to verify behavior, performance and portability.

why not take the time it would take to convert the Irrlicht engine to using the STL and instead work on improving the existing implementation of Irrlicht's containers?

IMO this would be a bad idea. This would require more effort by the Irrlicht dev team. There is the time to do the initial implementation, then time to do maintenance. That doesn't include the time it would take to write a test suite.

It's been pointed out that they are multi-threaded (which the STL is generally not), and therefore they could be faster, given some more work.

It depends on what you mean by thread safe. There are two types of thread safety.

Typically a library will guarantee minimal [level 1] thread safety. This means that any access to global or static data is safe. This is true of most implementations of the Standard C++ Library implementations.

So far, the only real reason that I see you giving to use the STL is "it's standard!". My response is: so what? Why does that make it better? Writing in assembler was standard until C came along, it doesn't make it better

This standardization is quite important. It means that you can expect consistent behavior between implementations. As I mentioned above, there are performance guarantees, and there is a lot of documentation. The standard containers have been used in many applications for more than two decades.

Travis

[MasterGod]

I'm glad I started this thread as it seems people were keeping important opinions to themselves. (PLEASE don't quote this statement and fight over it)

While reading it all I began to wonder, what happens when C++0x comes to the table? Will the developers convert the Irrlicht containers to use the new C++ features? I guess not..
So I guess to keep Irrlicht for more then few years we will have to sacrifice some time and completely convert to STL.

And by the way, I think it was Dorth who asked it - I didn't use STL cause I really didn't knew that it even exists until some time ago.

P.S

I understand that Irrlicht's containers are fast and efficient but are they faster/more efficient then the STL's or is there another reason STL isn't used?

I guess I got my answer

[Dorth]

wyrmmage, I only picked parts of Halifax statement so that he could not say I did not answer each. Also, the final quote I use to say what he answered me is wrong is not from him, but from my initial challenge to him, to which he still hasn't rose, which was to find at least 15 applications using Irrlicht core containers UNIQUE features. And as a bonus, that none of them would rather switch to STL if it didn't affect compatibility with Irrlicht. I did not use search, btw Halifax, because the charge of the proof is always on the one making the claim. You claim there are HUNDREDS! perhaps THOUSANDS! of Irrlicht applications that would be broken by that. I am not even convinced there are more than 100 Irrlicht app in total. Even less so that would be broken by a conversion. And even less that couldn't be fixed with a quick typedef or parsing. That whole argument is bull. I'd be far more worried of how the inner workings of Irrlicht would react to the conversion, but apparently it,s been done often before, though that's another unproven claim


Compatibility is EXTREMELY important. This is a GRAPHIC library, which means it's nearly guaranteed to be used conjointly with other libraries. You will not, or nearly not, see other libraries using the core containers. You will, however, see many libraries using the STL. Now if i was self-contained within Irrlicht, it wouldn't be so bad. But if you use the core and the std namespace at the same time, you conflict on the list, which is ridiculous. And if it was self-contained, then you could change one with another without breaking compatibility with other Irrlicht apps. Unless you currently have a reason to change a standard, DON'T. If you get one later, change it then, and as locally as you can, unless it's to correct a mistake in it, then try and make your changes accepted by the committee. Why? Because that way a team of 200 persons can work together on one project. That way you can mix useful libraries. That way, the code is portable and adaptable.

[rogerborg]

But if you use the core and the std namespace at the same time, you conflict on the list, which is ridiculous.

It would be ridiculous, if that wasn't exactly the problem that namespaces are designed to solve.

So I'm clear, are you actually volunteering to do the work in moving Irrlicht over to STL, including comprehensive regression testing?

[Dorth]

Heh, I did, when I had more time. I will try my best. Though some claims to already have done so. I think that if they would share their work, it'd be wonderful ^^

And, well, using irr::core::s32 is really long for s32. But I agree. If it was a new feature, I'd say move them to irr::container:: or the like. Just that would be help a huge amount.

[Halifax]

Though some claims to already have done so. I think that if they would share their work, it'd be wonderful ^^

3Demon has it implemented already. All core:: containers are currently stripped and replaced with their STL counterparts. Check it out, download the source from the SVN, or talk to spintz about helping you. He has tested the STL containers side by side against their core:: counterparts as well, and it has been proven that they run faster. (In the 1 test that was made public.)

And I don't intend to arise to your challenge because it is a simple abject thing to cavil over.

[Dorth]

For someone who's dropping it... And don't you get that the only reason I made the challenge in the first place is because of the ridiculous claim you made? I didn't think it was feasible to succeed in it, nor do I think so right now. If you notice, I do not attack you, I attack your claims, or, if so happen, your actions. I have not been hostile to you on the forum, no matter what you think, praising you when you do a good move, acknowledging your good ideas. But I do not agree with you here in this topic. And because I've been very detailed about it, you take this as an attack, which is annoying. Ask Mastergod. I'm as helpful as I can to him all over the forum, yet recently he wrote one or two posts that reminded me more of the poster he was a few months ago than the poster he became. And I picked him on that. I've still helped him though. Same here. There was a claim that many did it, some actually pertaining in this discussion. So I ask them, if it's easy and done to submit a patch. No point in reinventing the wheel (part of my argument in this topic, too). But if they don, yes, I'll hunt for it, whether it be in 3demonz or elsewhere. I do not claim nor want supremacy. I just want a valid argument with a sound reasoning that leads to a good and proper decision.


Argh, too many lines, no breather. Like I said, bring arguments WITH proof and I'll take them and examine them for what they are worth. But up to now, the only downside I've seen on the stl side is the time that would need to be spend converting, which as many claims, is minimal.


I maintain that if no decision is taken by then, this would make an excellent subject in the Irrlicht conference to debate.


What are the opinions of Hybrid, bitplane, Luke and Niko on this, being the main contributors and creator of the engine?

[Halifax]

Uhm, I did drop the debate/discussion and simply tried to point you in the right direction of who has done it already.

EDIT:
I never stated that I felt as if you were attacking me on the forums because I realize you don't. I said that I felt as if you were attacking me in this specific thread.

[Dorth]

Then my apologies for the two mistaken interpretation, but, no, I'm not, well, mostly not, in this thread. Let's leave it there, until new arguments, players or a decision is brought in

[Halifax]

Then my apologies for the two mistaken interpretation, but, no, I'm not, well, mostly not, in this thread. Let's leave it there, until new arguments, players or a decision is brought in

I agree. I will apologize as well for my behaviour in this thread.

[wyrmmage]

So, since it seems most people are behind the STL argument (and I'm perfectly willing to use the STL instead of the Irrlicht containers, if that's what the majority of people want), would it not be possible to make functions that take STL containers instead of IRrlicht containers (which it seems has already been done), and then replace the existing functions that take Irrlicht containers to functions that just translate the Irrlicht container into a STL container and then calls the STL version of the function.
If this was done, it would not break compatibility with existing applications, and users could use STL if they wanted to. After many releases, we could start to fade out the Irrlicht containers, using the method that someone already proposed (I believe it was Dorth).

I hope that made sense :\
-wyrmmage

[CuteAlien]

I think the point isn't even so much about irr::core vs. STL. It's more about using STL within Irrlicht at all or still avoiding it completely and adding missing functionality to irr::core only when absolutely necessary.

Still lets do a small comparison first:

Currently the two libraries only have a few classes where they overlap anyway. That's
irr::core::array <-> std::vector
irr::core::list <-> std::list
irr::core::map <-> std::map
irr::core::string <-> std::string
irr::core::allocator <-> std::allocator.

Those 5 classes, or did I miss something?

Classes can't exactly be compared 1:1 as Irrlicht puts the algorithms in the classes while STL does separate that. Which means Irrlicht is a little easier to use for beginners, but harder to maintain in the long run (code duplication) and not offering the flexibility of STL here.

One thing that can be nearly completely ignored is speed. It is possible to be faster than typical STL implementations, but usually not by a large margin as the speed improvements are usually only linear by a small factor. I've seen up to a factor of about 2 in some custom implementations for similar base classes for many typical operations. Which can be important, but in my experience the few speed-critical parts in games tend to avoid such base classes completely. On the the other hand it is very easy to be way slower than STL if you don't care enough about algorithms on implementing.

When it comes to functionality array and string offer rather similar stuff than the stl counterparts. The part mostly missing in those classes are iterators, which certainly is a result of the different concept used in Irrlicht. The list is still missing a few features, but nothing that couldn't be added on need without any trouble (maybe someone has fun doing so - it's a simple exercise, just compare the documentations). The map does work so different that I'm not even sure it can be compared to the STL implementation. Also it does not really fit into the other irrlicht classes. It's using iterators, but not throughout, so for example iterating over the result of find() is not possible (which is bad btw). Also it's not using the same naming than other Irrlicht base classes (remove instead of erase, isEmpty instead of empty). Allocaters are very different and the Irrlicht allocaters are clearly easier to understand and use (I can't tell much about flexibility, as I avoided STL allocators so far).

A rather important point, at least for me, is how much you can trust the base classes. And so far STL wins that for me clearly. There are still sometimes bugs found in the Irrlicht base classes and I never managed to do that for STL even though I use that a lot more. There's simply no chance the irrlicht classes will ever get the amount of testing of STL.

A bonus for irrlicht classes is that they are easier to debug. I rarely did this for STL, but the few times I did it was rather difficult.

So all in all - for the trivial classes string, array and list it doesn't matter so much. A little more work and they would be a nice replacement. The allocaters are maybe a point for irrlicht. I read about some trouble with certain old compilers like VS6, but I don't know exactly what they are about, so I can't comment on that. The stuff about thread-safety looks like a rumor to me - I never did see Irrlicht caring about that.

The map on the other hand is an example of what happens when we continue to avoid STL. It was missing in Irrlicht and so someone implemented it and the maintainers where happy enough about that to just use it even though it didn't even fit into the other base classes. Also due to the missing iterator for find it's usability is currently rather reduced (though that could be fixed, or maybe it is already in svn?).

But there are a lot more features in STL which are useful and simply can't be used so far because no replacement exists. Comparable to map there would for example be the multimap and the set. Comparable to the array there would be the deque. We miss a replacement for streaming and all the locale support. And no algorithms. And the big question for me is should Irrlicht continue to work around that and add only classes on need?
+ It keeps Irrlicht small and under control
+ It might keep it easier to understand (I'd say STL is a little harder to read for beginners)
+ It might work on more compilers (but old ones)
+ Backward compatibility
- It needs to be maintained constantly
- It will probably always lack in features
- Programmers are lazy, so if you have the choice of using the irr::array when you should add an irr::set, what will you do?
- It does not seem to get that much love by the maintainers currently as other parts of the engine (just my feeling...)

A last word on backward compatibility. As much as I like that, I wouldn't overrate it. Keeping two base classes doing same stuff is some additional work for maintenance and I'm not sure if that is worth it (though it's maybe not that much work - 5 classes and most of them are not that complicated). Backward compatibility mostly matters for tools which have to stay close to current Irrlicht versions. For games I usually work with one Irrlicht version and stay to that. For my tools I would be happy spending a day on converting everything to STL. It would make my code a lot shorter as I could remove all the irr -> stl string conversions ;-)

The decision is certainly up to the maintainers and I can live with whatever they decide, but it would be somewhat interesting for me (and probably others) if completely avoiding STL is also a rule for the future. Completely independent of what is happening to existing irrlicht base classes.

[hybrid]

Ok, before someone thinks that we simply ignore this thread: I did vote against the integration of irr::map, because it's awkwardly designed. I did clean up some stuff already, but I won't bother with it more than necessary. That's not true for the other containers, though. I try to fix all problems with those, and enhance their interface while keeping the general use pattern.
However, since I use Irrlicht, the official rule was to avoid all parts of the STL, i.e. all containers, streams, algorithms, everything. Since there had been no other "official announcement", I'll keep it that way.

[rogerborg]

Thanks, hybrid, that's very clear.

[hybrid]

But there's no reason to stop the discussion because of that. It's just that IMHO there's no good chance to get this into the core right now. However, I also discussed an STL-based implementation of the Irrlicht containers with Dorth some time ago, which could be a good idea to lower the bug probabilities. Of course this would be an optional compile switch, but might lead to STL sneaking in somehow.