Input wanted for hardware buffers design

Raedwulf · Post by **Raedwulf** » Mon Dec 03, 2007 10:50 am

also some trouble may arise if multiple nodes share the same mesh and some are static and some are not

The 'static meshes' can be 'static to the hardware' and 'not fully static to irrlicht'. So if the mesh is static, it can still be modified, but internally, irrlicht's VBO implementation can be made to destroy/recreate the buffer. This may sound its slow but its not, its about as fast as a vertex array. Non static vertex buffers are only slightly faster than vertex arrays.

One hopes that animations will mostly be stored in a number of vbo's i.e. per frame. This would put the advantage of static VBOs in animations too.
Probably the reason why new graphics cards have massive amounts of memory these days... its not only for textures you know

Cheers.

Luke · Post by **Luke** » Mon Dec 03, 2007 11:34 am

One hopes that animations will mostly be stored in a number of vbo's i.e. per frame. This would put the advantage of static VBOs in animations too.

I'm not exactly sure what you mean there

BlindSide · Post by **BlindSide** » Mon Dec 03, 2007 2:18 pm

Raedwulf wrote:One hopes that animations will mostly be stored in a number of vbo's i.e. per frame. This would put the advantage of static VBOs in animations too.
Probably the reason why new graphics cards have massive amounts of memory these days... its not only for textures you know

Thats most definately NOT

the reason as most new graphics cards will be running applications that use bone-based animation formats and hardware skinning, which basically means only one static copy of the mesh is needed and all animations are done in the vertex shader.

Cheers

Saturn · Post by **Saturn** » Mon Dec 03, 2007 9:06 pm

What blindeside says, plus: vertex shading is almost free. Whether you interpolate between two buffers in a shader or multiply-add matrices is next to meaningless performancewise. More time is spent in fragment shading anyway.

Apart from the issues mentioned by blindside, there is another huge problem with your proposal: You can only really blend between two states, without making an ugly vertex mess. When you have more than one animation active on a mesh, then you cannot just blend the vertex positions of precalculated and then interpolated frame buffers. Not only will it look ugly, but also change the number of positions and normals as an input to the shader. This will actually making things slower instead of faster.

Klasker · Post by **Klasker** » Mon Dec 03, 2007 10:53 pm

I didn't read everyone's posts closely, but I'll just throw in some thoughts I have made about implementing VBO's while maintaining a "nice and clean" design. Nice and clean would include simplicity, thread-safety, loose coupling, and non-monopolization.

A hardware buffer can be regarded as a relation between the video driver and the mesh buffer, since it depends on both, and logically only one is necessary for each mesh/driver pair. We want the hardware buffer to be able to update itself according to changes in the mesh buffer (although ideally it should remain static).
To achieve this, suppose a mesh buffer has a Change ID, which is simply an integer with an initial value of 1. Every time the mesh is altered, it increases by 1. The HardwareBuffer object keeps a copy of the change ID it had last time the mesh was uploaded to the hardware. When the VBO is needed, it can quickly test if its mesh has been changed, and re-upload it if necessary, also updating its change ID.
This approach, which I personally call Lazy Observer, has the advantage of being thread-safe, and the mesh buffer has no coupling to the video driver or the hardware buffer whatsoever. A downside is, of course, that you need to call a method like "changed()" after a sequence of changes in a mesh buffer.

Creating a hardware mesh buffer should be doable using a simple method like:

Code: Select all

createHardwareMeshBuffer(IMeshBuffer* meshBuffer)

And maybe a second one that simply retrieves a hardware mesh buffer of the given IMeshBuffer if it exists (somewhat like getTexture):

Code: Select all

getHardwareMeshBuffer(IMeshBuffer* meshBuffer)

When a mesh buffer is rendered using the drawMeshBuffer method, maybe the video driver can use a table or something to look for a hardware buffer using that mesh buffer.

I know it's not the traditional Irrlicht-approach, but those were a few thoughts of mine. There's a bit more to it than I wrote here, but make of it what you will.

hybrid · Post by **hybrid** » Mon Dec 03, 2007 11:29 pm

Yeah, that's a nice implementation of some ideas already mentioned. The problem is that you cannot release the VBO when destroying the meshbuffer. Since you gave up the tight coupling with the video driver there is no way to inform the driver of its destruction.

BlindSide · Post by **BlindSide** » Tue Dec 04, 2007 4:05 am

How about something like:

Code: Select all

typedef struct
{
u32 ChangeID;
u32 MeshRef;
} SBufferRelation;

Init both to 0, increment MeshRef by 1 each time a meshbuffer that can share this hardware buffer is created, and decrement it each time one is destroyed. When the hardware buffer finds that MeshRef is 0, it destroys itself.

ChangeID can be used to notify it when to upload vertex data like Klasker suggested.

Though there are still some obvious problems:

1. Where are the pointers to these structures going to be stored globally for all new devices and drivers to access? Some kind of list somewhere?

2. How and when is the hardware buffer going to check its status, which driver/device will take care of updating it, and what if that driver/device gets destroyed and it is still in use, who will take care of it then?

Cheers

EDIT: Nevermind there is already grab() and drop() so I guess theres no point in having a seperate reference count. Maybe just a flag for when the meshbuffer gets destroyed?

Klasker · Post by **Klasker** » Tue Dec 04, 2007 7:22 am

hybrid wrote:Yeah, that's a nice implementation of some ideas already mentioned. The problem is that you cannot release the VBO when destroying the meshbuffer. Since you gave up the tight coupling with the video driver there is no way to inform the driver of its destruction.

EDIT: Nevermind there is already grab() and drop() so I guess theres no point in having a seperate reference count. Maybe just a flag for when the meshbuffer gets destroyed?

Since the HardwareBuffer object is going to grab() the mesh buffer, the mesh buffer isn't going to get destroyed. What we really need is way to detect that it's not being used. A remove() method setting a flag like BlindSide mentioned would solve that problem, but then everyone will have to track their mesh buffers themselves.

Last I checked, in Lightfeather they have "use()" and "unuse()" to do it for them, though I constantly found it confusing when to use grab() and when to use use(). The idea was that use() is a stronger reference than grab(), since it forces the object to stay alive, while grab() only stops the object from being destroyed entirely. It probably works wonders, but it will definitely serve to complicate the Irrlicht API - imagine all the people asking what's the difference between grab() and use().

The video driver should obviously keep a list of all its hardware buffers. We can let the HardwareBuffers evict themselves from the video memory after 10 frames of not being used. Simply add a "LastUsed" counter to each hardware buffer. That would make VBO's somewhat easier to use, but of course, there is still the problem of freeing the IMeshBuffer when it truly isn't being used anymore.

Perhaps hardware buffers could remain a complete intern thing. So they are enabled like this:

Code: Select all

enableHardwareBuffer(IMeshBuffer* buffer, bool enabled);

That way the driver can on its own accord evict and insert hardware buffers when it sees fit. It's not like the user is going to need the IHardwareBuffer object, anyway. This way, when a mesh buffer hasn't been drawn for 10 frames, the video driver kills the VBO, and drops the mesh buffer. If has any other references it stays alive, but otherwise it disappears like it should.

EDIT: On second thought that has some serious problems too. The more I think about it, the more sense I make of Lightfeather's approach. A kind of "phantom reference" would be useful here, but I still think it's a too massive change - and who knows what side-effects that will have

EDIT2: I just noticed that the whole use() and unuse() thing is basically what BlindSide suggested before his edit

Luke · Post by **Luke** » Tue Dec 04, 2007 7:40 am

Klasker, nice, the Lazy Observer and 10 frames ideas could probably work.

On second thought that has some serious problems too. The more I think about it, the more sense I make of Lightfeather's approach. A kind of "phantom reference" would be useful here, but I still think it's a too massive change - and who knows what side-effects that will have

hmm, what's problems?

Luke · Post by **Luke** » Tue Dec 04, 2007 9:27 am

the only problems I can see is dealing with a meshbuffer being deleted and a new one being created at the same address. I think the driver should grab the meshbuffer while it's using VBO on it.

and the LastUsed could be 200 to 10000, no problems

Luke · Post by **Luke** » Tue Dec 04, 2007 2:21 pm

design of it:

http://img220.imageshack.us/img220/1931/vbo8xy8.png

it seems nice, I cannot see any problems so far.

want do you think?

weigo · Post by **weigo** » Fri Dec 07, 2007 11:08 pm

Hi,

i was trying to figure out, if irrlicht currently supports VBOs. it seems it doesn't.

whats the result of your discussion?

whats the current implementation of mesh animation? are vertices transformed by the cpu and send to the graphics card in each time step? if so, is this done by the getMesh() of SAnimatedMesh?

on modern graphics cards there's no reason to not upload everything to a VBO. memory consumption is only a problem in specific cases. so the default behaviour of a MeshBuffer should be to mirror the system buffer to a VBO, except the user turns it off explicitly. (as Klasker wrote: "enableHardwareBuffer(IMeshBuffer* buffer, bool enabled);") This way the VBO is depending on IMeshBuffer and would be destroyed when the system buffer gets dropped. Uploading could be done lazy within drawMeshBuffer().

difficulties arise when VBOs get used to hold arbitrary data for shaders. IMO the only way to take care of this application dependent use is to simply wrap OpenGL's and Direct3D's apis.

just my thoughts.

hybrid · Post by **hybrid** » Fri Dec 07, 2007 11:26 pm

Luke finished the implementation for OpenGL already. It's working pretty good. You can set the mapping hint (static, dynamic, stream, none) to ask the gfx card to store the vertices and indices in some way on the GPU (or avoid it for sure).
The thing is that destroying a meshbuffer on the CPU does not automatically remove the hardware buffer. Hence Klasker's idea which is now implemented, which leads to automatic hwbuffer destruction if a meshbuffer was not drawn for some frames.
So if you want to try the new feature use the most recent SVN trunk and set the meshbuffers hwbuffer mapping hint to EHM_STATIC or something. And yes, animation uploads the meshbuffers each time the render method was called.

Spintz · Post by **Spintz** » Sun Dec 09, 2007 1:23 am

Couple things :

Dropping a hardware buffer after n renders is, IMO, terrible. If you must upload automatically, without user interaction, I guess doing it the first time you draw it works, but a method should be provided for manually dropping a hardware buffer and if that's not called, it stays in memory until the driver is dropped.
The interface needs to support static/dynamic vertex buffers and index buffers separately.

Luke · Post by **Luke** » Sun Dec 09, 2007 3:12 am

Dropping a hardware buffer after n renders is, IMO, terrible. If you must upload automatically, without user interaction, I guess doing it the first time you draw it works, but a method should be provided for manually dropping a hardware buffer and if that's not called, it stays in memory until the driver is dropped.

well that can easily be done, I'm just not sure what's so terrible about it

The interface needs to support static/dynamic vertex buffers and index buffers separately.

yep definitely, I was planning doing this with the meshbuffer changes.