programmatically assign materials based on node id.

[Cube_]

so I currently have a block system where I generate each block from a master list, then store a pointer in the chunk structure to that data (as the data is nonunique), this index of available blocks is built programmatically and thus manually assigning materials is not convenient.

I would like to assign the materials programmatically as well, something along the lines of having a large enumerator with material indices and then automatically applying material 1 to block 1, material 2 to block 2, material 3 to block 3 etc.

However I'm not entirely sure how I would do this, I know that assigning materials by pointer works, however afaik I can't store a pointer in an enumerator, potentially this could be solved by polling the master list, filling a vector with materials according to the current block count and then initializing them all through a function that pulls textures in order.
However this is not necessarily optimal (the initialization only has to run once but no one likes loading screens, least of all the developer who has to run a lot of tests, meaning a lot of loading screens)

so, how would you solve this?
The reason for asking is mainly because the block list is variable in length, with only the core blocks being predetermined, everything else is pulled from a list of block definitions and/or dynamically generated depending on the context of the specific block, hence I can't very easily hand craft such a list.

simplified: the goal is to take the list of blocks and then generate a material for each, filling them from start to finish by loading textures from atlases until either the last texture has been reached or the last material has been reached (setting all other subsequent materials to *empty, which will display a single missing texture texture)
potential solutions I've figured out: loop or iterate over a vector, with a pointer to a material in each slot, then assign materials based on index, block one gets assigned matVec.at(1) or some such methodology, this sounds suboptimal though.

I apologize if my request is unclear or if I repeated myself, I shouldn't be asking questions at 2 am, there's no guarnatee that I make sense that late.

[REDDemon]

I don't understand sorry XD. seems need something like and associative container?

[Cube_]

essentially, what I do is I source all block ID's from two lists, the core list (hardcoded) from an enumerator
example
enum BlockList
{
BT_dirt
BT_grass
BT_stone
};

then I load other blocks from file(s), loading them into a vector, now I wish to be able to dynamically generate materials based on a similar metric without having to manually do this for every material:

Code: Select all

    grass->setTexture( 0, driver->getTexture("res/texture.png") ); //should pack into an atlas and load from there
    grass->setFlag(video::EMF_BILINEAR_FILTER, false);
    grass->setFlag(video::EMF_WIREFRAME, false);

having to do that would force me to have hardcoded materials, or to use some ugly hack to get it to work which would be difficult to maintain and difficult to read.
Anywho, in worst case I'll just use the naive method I thought up in my original post.

[tuket]

You could use XML files and parse them at runtime.

For example.

Code: Select all

 
<blocks>
 
    <block>
        <id>0</id>
        <name>grass</name>
        <texture>grass.png</texture>
    </block>
    
    <block>
        <id>1</id>
        <name>stone</name>
        <texture>stone.png</texture>
    </block>
 
</blocks>
 

In this way you don't have to recompile the game each time you make a modification.
Also there XML files can be sent through the network easily. So for instance, if you want to make an automatic update system you only have to send the new XML file. If the block types were hardcoded the whole executable file would need to be replaced (and the exe is platform dependent).
There is an XML parser built in Irrlicht but I have never used it. I used RapidXML which I think is very good (and the fastest).

I hope this is what you were looking for.

[Cube_]

well, I have two lists, on is hardcoded and one is loaded from a text file (plaintext, one per line). XML seems a bit cleaner but doesn't really solve the issue of having to manually specify all materials, it does solve the issue of having to manually add them to the source code bloating it and increasing compile time however.

Really though this still means that modding support will be subpar, the reason I want to programmatically solve this is because I'll allow arbitrary amount of additional block addons, this in extension means that users would manually have to edit the xml or a bunch of entries would be missing, although this could be solved by having an xml with the same name as the mod file (say... moreGems.xml and moreGems.blkpk), then I just need to figure out how to efficiently scan directories for files, enumerate them all by name, check the contents, sort them by priority to build a dynamic blocklist and then assign materials based on the current number of blocks, not quite what I originally had in mind but I sorta like it.

Might not want to use XML though, seems a tad overkill. although then again the file is only read on startup so it's not a huge issue if it's slow or overkill, in fact I could have a master config that only updates if new files or file changes have been detected, that way I can just compare hashes which is faster than parsing large xml files, if a new or changed file is detected it should trigger a reparse of the relevant section of the master config, then write the new hash.

Thank you, you didn't directly solve the problem but you did get me thinking in what I think might be the right direction, I was going at it a completely different way which ultimately seems less elegant now.


EDIT: In extension the hardcoded list could then be used to generate a default xml and blkpk combo so a master config can be built, sorta like a failsafe if the user accidentally nukes the files, given that I already have all this data it's actually really simple to rebuild new files from them, also useful in case the user modifes the core list and wishes to revert (and allowing mods to the core list could be interesting, this allows all sorts of things like overriding grass with something else, change block priorities to change how they're dynamically assigned id's and all sorts of nifty things like that).


EDIT 2: And this system also solves the issue of allowing users to mess with the world generation very elegantly, this way I can add a simple knuth shuffle to the loaded block list after the initial sort to allow the users a random block generated world (which, I intend to do. I'll expose every world generation setting under an advanced world generation area to allow users to play with some really silly settings, for example setting the following would generate a staircase from world bottom to world top that intersects 0,0,0 (probably dependent on seed, but for the default seed I set it works)

low bound: 1.0f
high bound: 3.0f
algorithm: 1
octaves: 1.0f
persistence: 2.0f
scale: 0.002f

fun fact: I still don't understand exactly how the bounds work, probably because I can't visualize the 3D simplex noise in my head as the math is a bit above what my brain can process, but I know they normalize the values within a range... I have to experiment further when I have greedy meshing working but for now I can't because loading a lot of chunks gets unplayably slow too fast.
ah well, this ended up solving a few design problems, neat.

[tuket]

Really though this still means that modding support will be subpar, the reason I want to programmatically solve this is because I'll allow arbitrary amount of additional block addons, this in extension means that users would manually have to edit the xml or a bunch of entries would be missing, although this could be solved by having an xml with the same name as the mod file (say... moreGems.xml and moreGems.blkpk), then I just need to figure out how to efficiently scan directories for files, enumerate them all by name, check the contents, sort them by priority to build a dynamic blocklist and then assign materials based on the current number of blocks, not quite what I originally had in mind but I sorta like it.

Once I did something similar for a school project. You could have a folder "block_types" and inside it place all the addons of blocks. In order to detect files in a directory you could use Boost FileSytem.
Another approach would be to have one file per block type. This avoids loading a big file in to memory and instead you would load small chunks. Also, since in the file system you are not allowed to have files with the same name, you can take advantage of that to create blocks with unique names.

Might not want to use XML though, seems a tad overkill. although then again the file is only read on startup so it's not a huge issue if it's slow or overkill, in fact I could have a master config that only updates if new files or file changes have been detected, that way I can just compare hashes which is faster than parsing large xml files, if a new or changed file is detected it should trigger a reparse of the relevant section of the master config, then write the new hash.

Do you mean that you wouldn't need to parse xml files at startup if there are no changes? So you would have a large file that directly map into memory. Anyhow, I don't think xml parsing is very slow. It just depends in the parser you use. With rapidXml you can expect throughputs of 1BG/s http://rapidxml.sourceforge.net/manual. ... erformance.


In an other order of things, it seems that you are willing to make something similar to minecraft, isn't it?
I am just trying to give some tips from my short experience. Have you taken into account the float precision issue? When worlds are very big, float precision issue can araise. More info here: http://scottbilas.com/files/2003/gdc_sa ... _paper.pdf.

Have you considered using world streaming? In the same article there is info about world streaming.

Sorry if you already considered these points. I just wanted to make sure you knew about them.

[Cube_]

with streaming I assume you mean paging in chunks as the player moves, this has already been implemented.
to solve floating point errors I just move everything around the player instead of the player itself, it shouldn't be significantly slower, as for how I intend to parse the xml, you're indeed correct in that I won't read xml files that haven't been changed, it's faster to just read the master list instead of rebuilding it every time, however I wouldn't read the entire file into memory, that'd be insanely memory consuming, for large files I'd only stream in a bit at a time, extracting the relevant data and then close the file handle when done.

[AReichl]

> to solve floating point errors I just move everything around the player instead of the player itself

I have tried something like that but failed.
I would be VERY interested in an example how to do it.

Regarding XML reading: there are 2 methods - DOM and SAX. DOM loads all into memory (which you don't want) and SAX, which is more like "streaming".

[Cube_]

I figured I'd just roll my own system of parsing the XML line by line, well block by block; after all it's just a text file.

As for how to page chunks around the player and move them, that logic is very complicated and far too long, not to mention it currently has some kinks I wish to work out (such as chunks randomly unloading before they should, loading with subtle off by one positioning errors and whatnot, you know the usual for something like this) before I'd even consider sharing some code

[REDDemon]

> to solve floating point errors I just move everything around the player instead of the player itself

I have tried something like that but failed.
I would be VERY interested in an example how to do it.

Regarding XML reading: there are 2 methods - DOM and SAX. DOM loads all into memory (which you don't want) and SAX, which is more like "streaming".

Just move the player, when player quit a "origin centered cube" of size says 1024.f, then translate everything by +/-1024 so that player can enter the cube again. It is faster than moving everything around the player because requires seldom updates, while updating everything would be (slightly) slower (and less accurate).

You load chunks when they are within sight ( 2048 plus far plane), and you unload chunks when they are far enough (say 4000 + far plane).

Of course you have to check whetever the translation cause a loss of precision so

Code: Select all

 
auto newPosition = oldPosition + increment;
if( (newPosition - increment)!= oldPosition)
   // threat error in some way
 

and then decide if that loss is acceptable, in general you are constrained to place chunks at integer multiples of 1024. Everythin suffers a loss of precision (in that case you want to store original position and recompute translated position at every "world translate") wich is easy to do when you do that on your own chunks, but may be problematic when you translate stuff created by "other developers" that does not allow to access translation value directly

[Cube_]

I considered moving everything at certain intervals, the only issue I found was that this could cause microstutter, especially if the player moved far enough, triggered a move, then wanted to go back, triggering another move (at least if I use the simple scheme of just subtracting the difference from center, that would cause the player to relocate to the opposite side of the virtual constraint box and if they want to go backwards a bit they cause another translation which would cause potential for more microstutter.

Although it's of course an issue I have to solve anyway, if there are a large amount of entities around microstutter might happen anyway, although it's not really microstutter if it happens any frame the player moves. My solution for this was initially to just parent entities to their active chunk, that way they move in relation to their parent causing less move operations to be performed.
Although to be fair both of those are conjectures more or less pulled straight from my rear, I've yet to implement the entity parenting nor have I tried your suggested method, I should try it at the very least and compare performance.

Potentially I could solve the potential duplicate microstutter by translating everything at fixed intervals of a slower rate than the player can move, if we assume the cross section is 1000 meters and the player can move at 10 meters per second, then translating everything every 50 seconds would be enough to always keep the player at the center of the translation cube regardless of if it has moved or not, being able to prevent microstutter would also be nice but that might be a nonissue if it the operation is fast enough.

[REDDemon]

Translating the whole scenegraph is a relatively cheap operation. (it should not cause any stuttering, the only problems are of precision)

The stutter you describe might be caused by wrong scheduling of chunk loading. (in example translating cause 10 chunks to be loaded in the same frame, and translating back cause another 10 chunks to be loaded). If the chunk loading is based on relative distance between player and chunks (that distance don't change when you transalte the whole scengraph) it should give no longer any problem and if it does you need to mitigate it using a cache for recently loaded chunks(or chunks that are likely to be loaded soon), and possibly load chunks in a background thread.

[Cube_]

interesting, I'll have to try that (but first I'll have to read the API to figure out how to translate the scene graph).

As for how I check which chunks to load and unload, you're indeed correct that I compare chunk centers to player location (and if they're greater than a full chunk and a bit I page them in or out)
As for keeping a cache, I cache chunk data but not chunk mesh data (which is the bulk of build times), mainly due to working within memory limits (I intend to support platforms with 1GB of ram, thus far my memory architecture is well within bounds but I can't have a full buffer of just mesh data for quickly paging in new chunks; however that should not be a huge issue once I refactor mesh generation code to a separate thread)