I think our best option would be in fact to assume that a material file is provided alongside the specified mesh, or that a developer can specify a material file to be used
Ok so you're favouring Option 2. Well, it seems to be more logical, until you picture a scenario where a huge number of meshes in your scene all using same material, thus each one of those meshes would be having a corresponding material file with exactly the same material definition. If we have a mechanism to re-use a material once it has been loaded, this would still be ok apart from the file clutter.
The serialization interface definitely will be important for a clean and portable implementation of the mentioned formats, but as stated it needs some work
Please educate me in few words if possible =) what would the serialization interface be used for?
The serialization interface definitely will be important for a clean and portable implementation of the mentioned formats, but as stated it needs some work
Please educate me in few words if possible =) what would the serialization interface be used for?
You need for each class a way to read/write the membervariables of it's objects - that process is called serialization. So it's likely that an easy solution for an Irrlicht format would be using that. And the serialization interface already can write all attributes - so that part could be used directly in a binary writer (as well as it's now used for the xml-format). The main trouble is that it does not support object creation inside the attribute serialization - so you can't create any child-objects so far (except with ugly hacks). We need a more complex serialization for that than we have now.
xirtamatrix wrote:You need for each class a way to read/write the membervariables of it's objects - that process is called serialization. So it's likely that an easy solution for an Irrlicht format would be using that. And the serialization interface already can write all attributes - so that part could be used directly in a binary writer (as well as it's now used for the xml-format). The main trouble is that it does not support object creation inside the attribute serialization - so you can't create any child-objects so far (except with ugly hacks). We need a more complex serialization for that than we have now.
I see. Thank You so much CuteAlien. Indeed I was not aware that we have such an interface in Irrlicht. The term "serialization" is also used in other context, such as file output to disk so I was getting confused. Actually, I've been doing the same thing manually in my project I shall take a look at it within a day or so to familiarize myself.
xirtamatrix wrote:You need for each class a way to read/write the membervariables of it's objects - that process is called serialization. So it's likely that an easy solution for an Irrlicht format would be using that. And the serialization interface already can write all attributes - so that part could be used directly in a binary writer (as well as it's now used for the xml-format). The main trouble is that it does not support object creation inside the attribute serialization - so you can't create any child-objects so far (except with ugly hacks). We need a more complex serialization for that than we have now.
I see. Thank You so much CuteAlien. Indeed I was not aware that we have such an interface in Irrlicht. The term "serialization" is also used in other context, such as file output to disk so I was getting confused. Actually, I've been doing the same thing manually in my project I shall take a look at it within a day or so to familiarize myself.
Ok, after some initial study and searching, it appears to me we might be talking of two different things. I have a question:
Isn't the whole IAttribute interface meant for serializing SCENE-NODES? For instance, to output scene files? But we're not talking about a scene format here, though admittedly that would come later. What we are talking is a MESH file format.
What is therefore needed is for external DCC tools to actually CREATE the mesh file, and Irrlicht to have a loader, similar to COgreMeshFileLoader or C3DSMeshFileLoader to actually load it.
Where does serialization of scene-nodes comes in then?
Last edited by xirtamatrix on Tue Nov 15, 2011 12:39 pm, edited 1 time in total.
It's an attribute system - it isn't restricted to scenenodes. A mesh is also just an array of values - and so you can serialize it with the array-attribute (I don' t know if that's the best solution - maybe it isn't).
RdR wrote:Whats the advantage to create another mesh format?
Why not improving the support for the current animated mesh formats
Have you actually read the full thread?
The mere idea of supporting a plethora of old obsolete formats IS the problem.
Why would we not publish and maintain our OWN efficient file format, and then focus to supporting it with all modren features, rather than having to constantly support x,y,z, file formats which, as I already pointed out, are mostly obsolete or suitable only for school/hobbyst projects?
CuteAlien wrote:It's an attribute system - it isn't restricted to scenenodes. A mesh is also just an array of values - and so you can serialize it with the array-attribute (I don' t know if that's the best solution - maybe it isn't).
I must admit that with my limited knowledge I'm not able to see the benefit, though if You and Radikalizm both support the idea of using the attribute system then I'd go with it.
The way I see it, we might just as well get the job done with simple data-streams. That would be fast, without any need to get into modifying the current attribute system to fit for this.
Once again, ALL we need to do is to LOAD the mesh file, quite the same as existing mesh loaders do, and I dont see them using the attribute system. Or am I mistaken here?
Regarding the answer to RdR's question I'd like to add that there has been no real proof that inventing and supporting a new format is less work at all. And even if there is a requirement that is not covered by any of the existing formats out there, it's also a question if this is a general requirement for the whole community, or just a corner case. I love file formats, so Irrlicht will surely support a new format that is tailored for the engine. Still, I will continue to support all the other formats and add other ones as well. And a new format will only work if there's enough support in all available modelling tools on the market - which is a tough job, especially keeping up with all tolls and releases.
@xirtamatrix:
The serialization system is just an easy way to get all data associated with any structure through a common and simple interface
You get information about the data value itself and what kind of data it is supposed to represent, and since not every type of architecture represents data in a similar fashion you can use it to build a portable format for describing data which provides enough information so that each supported architecture can load it correctly
If you were to work with raw datastreams you'd have to provide different binary files for different architectures. If you were to distribute a different package for each supported architecture, and if you were to have an automated build system for data compilation for each architecture this would be feasible, but it would still create an overhead in terms of pre-processing and memory although faster loading times might be a possibility
In the end it all comes down to proper agreements and a good design. It would be completely possible to design a file format which just dumps raw data from an x86 machine, and you could just tell any other machine that the data is provided as interpreted by an x86 machine, and that they should just deal with it. For some architectures this might work without complication, for others this could create a huge mess (not to mention the complications the programmer who has to write conversion mechanisms will have to face), so you can't really call this a good approach.
Since irrlicht is available on so many platforms we should try to keep portability a top priority, since you can't just exclude any platform from being able to use our formats
The case for Irrlicht's own format goes like this:
1. There are three user-groups in the market for a 3D engine. Students, Hobysts and Commercial companies.
2. By supporting a range of older file formats, Irrlicht caters well to the first two groups.
3. A commercial project is usually financed well and the cost of the 3D engine is usually a small percentage of the overall cost. But with its ease of use and vast user-base, Irrlicht still stands a good chance to be used as a 3D engine for a commerical project. The price, or lack of it, is irrelevant here.
4. Commerical projects do no work on 1-man-does-all paradigm. There is usually a team invloved. All ART is created totally seperately by the art team and to be honest, art IS actually 70% - 80% of a game. No commercial project will chose en engine which does not come with artist-friendly tools. A minimum required would be exporters that allow artists to export their meshes or entire scenes from top industry standard DCC tools. It is generally accepted in the market that such top-tier tools are 3dsMax, Maya, SoftImage and perhaps also Cinema4D. Despite all the hype, Belnder is still a hobysts tool or at best, an indie tool though it surely stands a chance to create its own nich market.
5. Artists are very hihgly paid and their time costs a lot. Companies are happy to spend money on any tools that help speed-up the work and imporve quality.
6. Currently, Irrlicht provides NO suitable tools for AA artists and the roots of the problem lie in the fact that Irrlicht does not support a coherent set of efficient file formats for which up-to-date tools would be available in the market.
In conclusion, to improve Irrlicht's adaption for commerical projects, it requires a set of most up-to-date tools, i.e. exporters for major DCC packages, ideally also providing viewers which integrate with the package for the artist to be able to view and modify their work in realtime.
For such exporters to produce files which are efficient, encode all data required by different engine systems and which fulfill the very high requirements of modren gamming, Irrlicht needs to specify its own file formats.
That's what we're attempting to do here.
It is not a mystry why Ogre gets chosen more often than Irrlicht for commercial projects. One only needs to understand the requirements of commerical clients and cater to them.
Ogre does that. Irrlicht does not... so far. Its time to change the game.
Last edited by xirtamatrix on Tue Nov 15, 2011 2:04 pm, edited 1 time in total.
You will face endianess problems with every binary format you use. Whether with or without serialization. After all, serialization is just a structured way to put structured data into a long list, storing the structure information in this list. And of course also providing a way to get the data back. Doing all this in a uniform way with a common API is serialization, otherwise it's just I/O. For Irrlicht based apps using the interface is simple. For the loaders and exportes of other apps you have to mimick this process. But that's true also for many other formats. But for this file format, performance will be a very high priority. Otherwise Collada or FBX would already do the job. So deserializing large arrays (of indices and vertices) will be a major task to optimize. Not sure if the current attribute system would help.
BTW: We support the ogre mesh format...
Hi, just got xirtamatrix's PM about this thread. I have been incredibly busy working away from home on contract, I'll be back for a couple of months as of Saturday 26th of November. I'll find some time for Irrlicht between DIY and Skyrim
Throwing some ideas out here:
We're talking about making this Irrlicht's native animation format (i.e. based on skinnedmesh), which all modern loaders will use. To be future-proof we'd need to review all current animation formats, catalog the features they support, and design an animation playback model that supports most of them. Some things will incompatible with Irrlicht's innards, so we need a roadmap of what would need to change in future (flexible vertex format, shader loaders, level of detail controls and so on), what we're not bothering to support (IK, any form of physics) and how to access the stuff that we don't support (hit-boxes, collision meshes and so on).
IMO the hard part is gathering the requirements (reading and absorbing a library's worth of documents), because if they're wrong then the design is wrong from the start. Everything else is just writing code, which is the fun part!
I shall take a look at it within a day or so to familiarize myself.
