Irrlicht Engine

Hi folks,

In my opinion one of the biggest shortcommings of Irrlicht is lack of its own file format, which gives rise to a chain of other problems.
Apart from better support for material shadders, this is probably the single most important factor why Irrlicht engine may be rejected by a client for a serious game/application.

For instance, I work as a professional 3D developer for a multi-national company, and recently we had to choose Ogre over Irrlicht for a project just due to lack of own file-format in Irrlicht, and thus lack of DCC tools and limitations of skeletal/morph animation etc.

Is anyone working on this?

If not, I hereby propose to start a project to:

1. Define Irrlicht's own file format.
2. Write exporters for the format for all major DCC tools, or at the very least for 3dsMax and Blender.
3. Write the loaders for the format to integerate with Irrlicht core..
4. Add shader/material support as well as bone/morph animation support etc.

I can manage to dedciate some time regularly for the initial development, however, I personally lack any experinece of writing exporters for 3dsMax etc. so we need to build a team with at least 2-3 people.
Anyone interested?

the topic of an irrlicht mesh format has been covered

the problem is that people are too busy and the time people do spend on the engine most find more valuable to spend time on things other than an irrlicht 3d mesh format

i think it would take someone showing up saying "hey, here is a fully tested patch to include a brand new 3d format with ALL the code already written" :-p

I must admit I don't know how much can be done right now with the .irr format - if it is only scene or also includes mesh-information.
Personally I rather work on things like improving the Collada-support as it allows exchanging data with way more applications than any custom engine-format. And support for that already improved in 1.8 (a little for import, a lot for export) and will most likely get some more improvements as there's another patch in the pipeline.

And not sure if so many people actually want to write exporters for 3d tools - we're still not even having a new b3d exporter for Blender 2.5 (which I think is the most popular modeling tool in this community) and that is a rather popular format (also the Collada import in Blender is currently crashing when we use Collada exports from Irrlicht, something I already tried to debug, but gave up as building Blender with Collada support enabled is hell on earth).

We just need full support for .FBX and it would work just as good as an Irrlicht mesh format.

aanderse wrote:the topic of an irrlicht mesh format has been covered

yes? where?

aanderse wrote:the problem is that people are too busy and the time people do spend on the engine most find more valuable to spend time on things other than an irrlicht 3d mesh format

considering that a very large majority of Irrlicht users are students and hobbists, it makes sense because they can make do with limited and obsolete formats. Their requirements are not the same as that of a professional project being undertaken by a large company, where price is usually not a big issue and mostly industry-standard tools are used. The question to ask is, does the irrlicht community want to reach out to those AA users? Or is everyone happy keeping Irrlicht as a hobbyst engine?

aanderse wrote:i think it would take someone showing up saying "hey, here is a fully tested patch to include a brand new 3d format with ALL the code already written" :-p

I strongly doubt that would ever happen, as its not a one-man job Very different skill-sets are required to put all of this together. Of course, someone way more experienced than me is welcome to prove me wrong

CuteAlien wrote:Personally I rather work on things like improving the Collada-support as it allows exchanging data with way more applications than any custom engine-format. And support for that already improved in 1.8 (a little for import, a lot for export) and will most likely get some more improvements as there's another patch in the pipeline.

With all due respect, shoot the guy who told you collada is meant for real-time rendering. As for exchanging data with lots more applications, its mostly google-backed propaganda than anything and even if it was true that is not what a game really requires. If you were building a general-purpose world-editor or something similar then yes, you could utilize it. Let me ask you a straight question. Have You personally ever built an entire professional game that has been marketed and which is using collada as file format? I know you couldn't have, cause no one can Even if you ignore the fact that collada is only a data-exchange format and was never meant to be used as the actual render format, working with collada is hell on earth, and not just in blender. And thats because of the open nature of the fromat, meaning no one has a final say on how it should be implemented. Take 2 different collada exporters for the same DCC tool and export the same mesh, you will get two very different files. There are too many more arguments I could provide against collada, but I think you get the picture.

What Irrlicht really needs is its OWN set of file formats. An efficient binary mesh format. A material definition format, preferably xml based. And, depending on the strategy, perhaps also a seperate skeleton file format, also preferably xml based.
But this is not enough! Creating these file-formats would be easy. The real challange is to constantly keep all the exporters for all major DCC tools in the market up-to-date because trust me, 90% of the professional 3d artsits upgrade to the latest version of their favourite DCC tool as soon as its available and they expect your exporter to still work seamlessly.

Since this is a lot of work, a common strategy employed is to actually sell the exporters for commerical DCC tools for a reasonable price, lets say something like 150€ to 250€. Anyone using a commerical tool like Max or Maya would have no problem spending that amount of money on a ready-made fully functional exporter. Its peanuts. The revenue thus generated is used to support ongoing development and ensure that exporters are always up-to-date. Remember, your file-format is only as good and as usable as the exporters for it.

3DModelerMan wrote:We just need full support for .FBX and it would work just as good as an Irrlicht mesh format.

while I agree with you that full .FBX support would be a great thing, it does not resolve all the issues Irrlicht is facing. (see my post above)
also keep in mind that .FBX, like collada before it, is meant for efficient data exchange, in this case between different Autodesk DCC tools, and its not the format you would want your final game assets to be in (considering efficiency etc.)
you need a small efficient binary format. both collada and .FBX are large and clumsy.

xirtamatrix wrote:As for exchanging data with lots more applications, its mostly google-backed propaganda than anything and even if it was true that is not what a game really requires.

I worked on Collada export recently precisely because it allowed me exchanging scenes with other tools. So now I can for example export Irrlicht-scenes and render them in Max as well as in Mitsuba (and Blender will also work one day...). I'm not working on a game - last time I wrote a game I used an own scene-format loading a modified .obj for static meshes + .x for animated meshes - and we've also improved support for both of those formats in Irrlicht since then (and another patch for improving .X is also still in the pipeline, but can't be applied without further modifications as it currently breaks something).

xirtamatrix wrote: Creating these file-formats would be easy. The real challange is to constantly keep all the exporters for all major DCC tools in the market up-to-date because trust me, 90% of the professional 3d artsits upgrade to the latest version of their favourite DCC tool as soon as its available and they expect your exporter to still work seamlessly.

Not sure if I'd agree that creating a format is easy, but I agree that keeping exporters up-to-date will be some work. And tricky as I don't think any of the current Irrlicht developers even owns any of those tools (like Maya or Max 3D), but maybe I err there.

Not sure if I'd agree that creating a format is easy, but I agree that keeping exporters up-to-date will be some work. And tricky as I don't think any of the current Irrlicht developers even owns any of those tools (like Maya or Max 3D), but maybe I err there.

The difficulty of creating a format is directly related to what the format is supposed to be able to do, if it's just a matter of storing data in a predictable format then the process is quite straight-forward. If it should support storage of a wide range of data and datatypes, and maybe even user-defined data it will get somewhat more complicated

I agree with xirtamatrix on the fact that irrlicht should have its own set of formats; being able to develop and maintain these at the same pace as engine development can be a great benefit, together with the fact that you can allow for optimized importing and exporting
Creating conversion tools or importers/exporters for other tools for these formats would still be somewhat of an issue, but is much more manageable IMO and can be aided by tools such as assimp (which I'm using myself for resource conversion to my custom formats)
Also, I feel like the community would be interested more in creating conversion tools and importers/exporters for tools like max and blender rather than building custom resource loaders and exporters for the engine

I don't know what your (and the rest of the devs for that matter) opinion is on this topic, but these are just my 2 cents

CuteAlien wrote:I worked on Collada export recently precisely because it allowed me exchanging scenes with other tools. So now I can for example export Irrlicht-scenes and render them in Max as well as in Mitsuba (and Blender will also work one day...). I'm not working on a game - last time I wrote a game I used an own scene-format loading a modified .obj for static meshes + .x for animated meshes - and we've also improved support for both of those formats in Irrlicht since then (and another patch for improving .X is also still in the pipeline, but can't be applied without further modifications as it currently breaks something).

Indeed. Both .obj and .X were fairly good formats for their time. Problem is... times have changed =)) besides you must be aware of the .X problem of multiple weighted single vertices. And as microsoft abondoned it, its pretty much sitting obsolete out there. While a lot of university/school assignments can still successfully be completed with these formats, one would have to be very brave or simply stupid to use these today for a project.

CuteAlien wrote:Not sure if I'd agree that creating a format is easy, but I agree that keeping exporters up-to-date will be some work.

Surely, I meant "comparitively easy" though keeping the loaders updated with each Irrlicht upgrade would be no small task indeed.

CuteAlien wrote:And tricky as I don't think any of the current Irrlicht developers even owns any of those tools (like Maya or Max 3D), but maybe I err there.

Yes, I'm afraid this could turn out to be true. Work-arounds could be found for the ownership problem, but we sure do need someone with good knowledge of 3dsMax SDK and ideally some prior experince of writing a mesh exporter. I could write a simple scene-exporter with the 3dsMax SDK in about 30 minutes but thats as far as my knowledge and experinece with it goes.

So, anyone out there with knowledge/experience of 3dsMax SDK? Please shout =)

Radikalizm wrote:The difficulty of creating a format is directly related to what the format is supposed to be able to do, if it's just a matter of storing data in a predictable format then the process is quite straight-forward. If it should support storage of a wide range of data and datatypes, and maybe even user-defined data it will get somewhat more complicated

Indeed. If we proceed with this project, the first task would be to research what would actually be a suitable format for mesh files. This depends on what we want or dont want to be included. For instance, do we want to include morph data together with all the mesh information? I for one would stróngly support this. Don't underestimate the strength of morphing, its the only efficient way for developers of multiplayer games who wish to provide unique avatars for each player. And who doesn't? Thus, we need morphing that works on rigged meshes. Likewise, there are other issue that need to be considered and decided upon before we can finalize what is it exactly we want in our mesh file. This would then define how simple or complicated it is to implement.

Radikalizm wrote:I agree with xirtamatrix on the fact that irrlicht should have its own set of formats; being able to develop and maintain these at the same pace as engine development can be a great benefit, together with the fact that you can allow for optimized importing and exporting. Creating conversion tools or importers/exporters for other tools for these formats would still be somewhat of an issue, but is much more manageable IMO and can be aided by tools such as assimp (which I'm using myself for resource conversion to my custom formats)
Also, I feel like the community would be interested more in creating conversion tools and importers/exporters for tools like max and blender rather than building custom resource loaders and exporters for the engine

Quite agree with you there, and thanks for the support =)

Radikalizm wrote:I don't know what your (and the rest of the devs for that matter) opinion is on this topic, but these are just my 2 cents

Yes, I too would like to know if the core development team is working on this issue at all, or is there anything in the pipeline? If not, I hereby volunteer to run this project, so long as we can find a few skilled people from the community to take part.

If not, I hereby volunteer to run this project, so long as we can find a few skilled people from the community to take part.

I'd be willing to pitch in on this too, I can't say I'm an expert on the subject but I've had to design multiple file formats in the past so I think I can be of some value

Collada is not well supported by Blender. It even crashed when I import Collada model from Maya to Blender.

Radikalizm wrote:I'd be willing to pitch in on this too, I can't say I'm an expert on the subject but I've had to design multiple file formats in the past so I think I can be of some value

Awesome! Welcome to the team

CuteAlien wrote: And not sure if so many people actually want to write exporters for 3d tools - we're still not even having a new b3d exporter for Blender 2.5 (which I think is the most popular modeling tool in this community) and that is a rather popular format (also the Collada import in Blender is currently crashing when we use Collada exports from Irrlicht, something I already tried to debug, but gave up as building Blender with Collada support enabled is hell on earth).

Hey CuteAlien : actually we (SuperTuxKart folks) do have a B3D exporter that works with blender 2.5/2.6 it's in our SVN repository. Possibly it could be linked on your wiki or so

@xirtamatrix

I think what we need to do is to list down all features we need and then only the format can be structured properly to fit with the requirements before start coding it.