How to make a really huge world?

[Ion Dune]

In my project I want to have a huge, continent size world with massive LOD, with only land that is beyond the horizon being culled. Obviously, to achieve this, I would have to significantly lower the detail of far away land, with terrain on the edge of the horizon being down to only a couple of triangles. I will post my ideas for such a system here, in the hopes that I could get help/advice on ways to manage it before I take the plunge and begin development.

First of all, I would like to say that I have looked at arras's tiled terrain scene node, but it is not what I am trying to create. Arras's terrain uses culling to maintain usability, but does not seem to use LOD. Indeed, I compiled his example with culling turned off and I got a too many primitives error (2^16, IIRC).

Also, if anyone has ever played Oblivion, this is pretty much the system I wish to make.

My idea is this:
-split the world up into sectors, and probably store the data for each sector (terrain heightmap, and objects in the sector: trees, rocks, etc.) in an XML file (one per sector).
-for the sector you are in and all adjacent ones, load upon entering and render the trees rocks and entire heightmap.
-for somewhat farther away sectors, load partial heightmap (skip every other?)
-as you get farther and farther away from the current sector, skip more and more until you are loading maybe 4 or 1 height per sector.
-make an array of terrain nodes (I'll probably copy and alter the current terrain scene node to better suit my needs), with farther away sectors merged into fewer terrains. This is what I mean by that (green lines are terrain nodes, grey lines are sectors):

-check collision only for current and adjacent sectors
-granted, there will be a (hopefully) slight loading time between sectors, as the LOD is changed. I also think that I will shift everything so that the player remains close to 0,0,0, because I have noticed that Irrlicht gets buggy at great distances. I'll also probably make a system through which I only have to alter sectors/terrains (add/subrtact vertices from current terrains) so that I don't have to reload everything.

So, thats my plan. Does anyone have any advice/any flaws to point out in my design? Has anyone ever tried/done this before, that I have maybe missed? Am I taking this in the right direction, or is there a better way to do something like this?

[MasterGod]

Theoretically I did it. Well, I've created a system that should answer those demands but I didn't actually created such a huge world to test it though it should work. I have it in my engine of course. You can read the manual and quick guide for further information. I call the "sections" regions by the way.
Look for Level Graph Management System (LGMS) - That's how I call this system. The manual explains how it can be used to create big worlds like GTA cities for example but you'll have to read it to understand what I mean.

P.S
I've been inspired by Irrlicht's scene graph API so lots of stuff would be similar but don't get confused.

Oh and even though the amount of docs (manual + quick guide) can sound scary (don't worry you need only the first 14-15 pages), just for the sake of education I'd recommend reading most of it. (actually, I recommend reading it all hehe..).

P.P.S
There is just to much to say about handling/creating such system so just read my docs and then come back here to discuss about what you need, it can give you a good perspective about what you're up against.

[Ion Dune]

I read (or at least skimmed and looked at the chart ) the part of your manual a while back, and this is not what I need to achieve. I'm going to have my buildings (and thus all included objects, like in your tree) in separate zones (just like in oblivion) so they are not an issue at all. Right now my biggest focus is terrain. I need to have around 4,700m (assuming my character is ~1.5m tall) of terrain visible until the horizon. I don't think I'm going to need any extra culling besides regular bounding box culling. My focus is making manageable LOD for far away terrain and making it easily updated as the player moves around in the world.

Also, if I've missed something huge in your management system, please point me out to it. I think I've gotten the gist of it, but I'm not sure.

PS: a slightly unrelated question: what is it in the Terrain Scene Node that makes it flip out when you try to use a larger (IIRC 512X512 was either the max or a trouble spot, but I'm not sure) height map?

[noals]

im a noob in programming and im just learning the basic stuff from now on but i want to do a terrain kinda like that.

thinking about it, i wonder if playing with the distance from the player to whatever instead of sector isnt better.

i mean something like
1 km around the player = full renderer
10 km around the player = 50% or whatever
50 km around the player = heightmap only
so having different drawing distance around the player.

trying stuffs with heightmap, i noticed than even if the heightmap is 256x256, you can have a terrain 100 time the size of it. ( the problem i encountered is the drawing distance ^^; )
so if your sector are big enough, why bother thinking about sector that are 3 or 4 sector away ?
even in oblivion, if i remember well, the drawing distance for details is kinda low, no ?.

anyway, like i said, im a noob so well, dont pay attention if i say stupid stuff, im discovering ^^

[Ion Dune]

Do you mean using actual distance calculations? The reason I'm using sectors is to avoid doing sqrt calculations (for distance) which can be expensive if you're doing a ton of them. Ideally, sectors will help remove at least some of the necessary calculations.

As far as I remember (its been a while), Oblivion had an adjustable view distance, and at full you could see pretty darn far.

It's something I want for my project for one to be able to see all the way to the horizon. To do this, I have to do mad LOD to make it manageable. If you have a 256*256 terrain, thats 2^16 vertices (if I understand it correctly), which is a ton. Turn that into a 1000*1000 array of terrain, and thats so many its not even funny. This means I have to extremely simplify far away terrains so that they are manageable.

Somewhat Unrelated: Spent some more time looking at the Terrain Scene Node. I see that it simply skips vertices when rendering to do the LOD. Though this won't work for me because 5km*5km of terrain is going to take up tons of memory. I'm thinking that I could have levels of distance defining how detailed the terrain is, and then do further LOD in the same way it is done for the terrain scene node. This way moving from one sector to another only results in a change of rendered LOD, and its not until you move say 5 sectors that the actual terrain detail is altered.

[MasterGod]

@Ion Dune: I don't know what you've missed but I'll point that out. If you set a region's idle state to true (which means it IS idle) all the scene nodes under it will not be rendered (nor processed). The level nodes won't be processed and the scene nodes under the regions won't be rendered.
This means that you can set all regions which are far (you define what is far) to idle==true and they won't be rendered.
Take an oblivion scenario - You're in the forest, the city region is idle and the forest region puts a skybox of the city in low detail so it looks far and when you cross some bridge or some place you flag as city entrance you load the city region (idle == false) and remove the skybox of the low-detailed city and place a skybox of a low detailed forest. (You can also take a screen shot of it so it would look more realistic instead of the same image all the time).

P.S
Don't forget we aren't professionals so don't expect too much.

[Ion Dune]

Hmm... I see. The thing is, I'm not even going to really touch on cities and stuff like that for now. Right now I just want to get a working system for land, because at great distances that is all you really need to see. When I go to add cities and other environmental things, I will definitely have a more in depth look at your system and code.

[MasterGod]

Then maybe a combination of arras's terrain scene node and my system is actually what you need.

[noals]

what do you think about false sector ?

i mean each sector have his double
you render always at 100% but far sector just show some sector(bis) with what you need to see at far distance without having to play with object, detail, resolution or whatever. (kinda like the screenshot method MasterGod talk about but with sector)

in dark and light, you can see pretty far but its a fact that the fps in dark and light really sucks.
that do a while i tested it so maybe they made their engine better but i doubt about it.




Unrelated:

Spent some more time looking at the Terrain Scene Node

i did a little, i noticed that the "ETerrainElements.h" is missing from the source for the E_TERRAIN_PATCH_SIZE stuff from addTerrainSceneNode ^^ but yeah, i just started yesterday...

[ultramedia]

I've got something on the way, it might take a week or two, but it is a rush order so it will be soon.
I just had to sort out a camera system I liked first (I'm pretty happy with it now) : http://irrlicht.sourceforge.net/phpBB2/ ... hp?t=27741
I'm 90% done on the seamless LOD tiling, when I get the heightmaps working I'll be starting a new thread in the Project announcements forum.

Oh yeah, and I had to learn c++ (fortunately, I've done a lot of coding in other languages)

P.S. I'm using cell groups of 4x4 instead of 3x3 as in your diagram, it lets me stagger the wrap around of the cells in each octave. With 3x3 the wrap around gets triggered on all the octaves at the same time. The downside of 4x4 is that I have to let each octave 'bite' a chunk out of the cells in the next octave up every second wrap around. (I know that sounds mental sorry, I'm on lunch and I don't have time to do a diagram).

[Lekane]

Oh hay, look who's doing the same thing Except i work on a sphere so not infinite hehe (though i could easily turn it into flat infinite once done )

The patch technique is pretty good, cpu usage is light (little peak when generating new set of patches but no headshot), uv calculation is dead easy since you work with squares.

if you want to see an early stage vid:
http://fr.youtube.com/watch?v=hqQAJHAuxTs

[ultramedia]

if you want to see an early stage vid:
http://fr.youtube.com/watch?v=hqQAJHAuxTs

Looks good. Only crit is the LOD popping - don't leave it too late to add in the geomorphing...

P.S. what mapping do you use for your heightmaps to fit them to your sphere? Cylindrical or cubic?

Oh and Ion Dune, make sure you check out libnoise if you haven't already...

[Lekane]

you're right for popping, i'll take care of it as soon as the splits work fine...

As for the mapping, well there is none hehe, i call the fractal for each new vertex i create, i'll probably store the results later on for texturing purpose... and it'll be cubic since the sphere is a cube at its simplest lod.

[Ion Dune]

@noals: huh? I'm not quite sure what you mean, though that pic looks kind of like what I'm trying to make.

@ultramedia: Great! At the absolute least, I will have a look at what you do for inspiration and ideas. As for using 4x4 instead of 3x3, I hadn't really thought about it. 3x3 is just the way it came up when I sketched it up in paint. Now that I think about it, I might go with 5x5 or something, to sacrifice render time for less loading time. I'm not quite sure what you mean by 'bite', though. As for libnoise, I'm probably going to be pre-making my heightmaps in Photoshop or similar, but I will keep track of that in case I change my mind.

@Lekane: Video looks nice, pretty much exactly what I want to do, be somewhat flat. My world is actually going to be a section of a planet, so its not infinite, but not quite spherical.

[Lekane]

in case you didn't read this:

Chunked LOD Paper: http://tu-testbed.svn.sourceforge.net/v ... -notes.pdf

Quadtree on gamedev: http://www.gamedev.net/reference/progra ... quadtrees/

edit: damn url tags