replacing ITexture by "hardwarelinks"

[Nox]

hi,

i tried to merge ITexture and IImage and replace ITexture by a hardwarelink class. . It works like the hardwarebuffer-class for the vertices and indices. Moreover the loaded images are stored in the driver.

"How can this be useful?" - if the driver and the irrlichtdevice would be seperated, driver- and settingchanges on-the-fly wont be a big deal anymore because all driverspecific ressources can be destroyed and will be loaded automatically.

This patch is not really tested or finished. It is more a example how it can looks like and to get some feedback:

http://7bitfaster.de/downloads/texturel ... 2436.patch

[BlindSide]

I like this idea. Very much. Anything that would abstract textures away from a specific driver would make me very happy at this stage.

May need some more thinking and deliberation though. I'll have a quick look through your patch and come back with some suggestions/improvements (If any).

EDIT:

Ok so I see you doing things like:

Code: Select all

-			MipMap[0] = new CImage(BURNINGSHADER_COLOR_FORMAT, image);
+			MipMap[0] = new CImage(BURNINGSHADER_COLOR_FORMAT, image, "Burning_MipMap");

I did assume at first that every texture would need a unique identifier/name of some sort for this idea to work.

But in this case, would naming it "Burning_MipMap" mean that it has the same identifier as every other mip map created using the burning driver? or did you devise some system to differentiate between those.

Cheers

[Ulf]

I love this idea. I'd like to help too. I'll be limited in helping on this at the moment as I'm a noob, but I might be able to pick out little things until I get a broader view of everything.

if the driver and the irrlichtdevice would be seperated, driver- and settingchanges on-the-fly wont be a big deal anymore because all driverspecific ressources can be destroyed and will be loaded automatically.

Do you mean if you change driver, it will load all the resources from the old driver into the new driver? If so, that is cool.

Code: Select all

+	//! Allow the Driver to use Non-Power-2-Textures
+	/** BurningVideo can handle Non-Power-2 Textures in 2D (GUI), but not it 3D. */
+	ETCF_ALLOW_NON_POWER_2 = 0x00000040,

I'm still working on basic things for my 2D game engine and haven't really got to the blitting part.

Can irrlicht currently render non power of 2 textures? Because that's what I wanted.
For 2D, you shouldn't be limited to power of 2 textures.

Mostly when doing 2D (I mean 2D bitmap type graphics), you would save the images for sprites for example, in their final rendering format.
You don't want to resize on the fly, or make animation frames on the fly.
You would have a texture, either a strip, or block, that contains all the animation frames for the sprite.
When you blit, you just blit the appropriate frame by referencing the appropriate rectangle in the texture, and you have frame animation.

I'm not saying you'd never want to resize the image.

Quick question:

Code: Select all

-		virtual void draw2DImage(const video::ITexture* texture,
+		virtual void draw2DImage(video::ITexture* texture,
 			const core::position2d<s32>& destPos) =0;

Why the need to remove the const from the texture parameter?

[hybrid]

While merging IImage and ITexture might seem reasonable, there are some fundamental problems to solve first. Might help to see ITextureLink
So the main problem is, that IImage is not stored in the device or driver, while ITexture is. So it's a problem of still freely creating images, properly removing images while keeping the hw textures, etc.
Maybe it would help to characterise the properties of images and textures first, and then extracting the common part into a base class.
The idea of giving IImage mipmaps seems ok, and would also provide a way to pass them to the hw textures right now, without changing the classes that much. Any objections to this step?

[Nox]

Darn, okay you found something which should not there . For the new system IImage objects need sometimes a name (i.e. if it replace the ITexture). But this name is only interesting for storing and saving the images. I gave the burning a name for debugging only. I have removed them now.

EDIT:
This patch allows to store IImages in the device. Moreover the IImages are grabbed and dropped by the texturelinks. So creating a ITexturelink without storing the IImage is possible too. A ITexturelink of an IImage can be removed by removeTextureLink.
The thing with the mipmaps may be possible. But this is something for another patch.

P.S: @hybrid just take a look at the patch

[BlindSide]

I'm not concerned with combining IImage and ITexture. I just want to have driver-type independent textures.

We can either do this as a quick fix, or wait till we get around to sorting out the messy IImage business entirely, which may be a long time from now. (I'd prefer both)

May I ask Nox, what motivated you to make this change? I have my own personal reasons for wanting it, but I'm curious why you needed this.

[bitplane]

Just piping up in support of this, it's actually Hybrid's domain...

I like the idea as it opens lots of options in the future, including the ability to change video driver at runtime or change the display mode without re-loading all the textures. Two images could be shared across different video drivers, we could even have incremental texture loading, where getTexture starts with the highest mipmap and adds detail as the texture loads. We could save video ram by removing textures that have not been used in a while.

There are some issues though. The texture/image cache will need to belong to the device rather than the driver, the D3D managed pools will have to be replaced or we'll have 3 copies of each texture in memory. The software drivers will need a special setting; keeping two copies of textures in memory on embedded systems is a waste of RAM.

[Nox]

Well. The copycount is a problem. I can not remove the buffercopy of the OGLTexture because sometimes the colorformat and the size differs from the IImage (i.e. for the 16bit flag or rendertargets without fbo support).
Of course this patch is not optimised and the map is not balanced, so O(log n) is not garanteed.

@BlindSide well there are no more textures in the old way. There are only images for the userland. The motivation is that this feature is requested frequently and i said that this would be possible. So i had to proof it

[hybrid]

If you don't provide the ITextureLink file, this won't work at all. You also don't have to think about log(n) here, because we need to care for usability and memory usage first.
Moreover, storing the images in RAM and then providing a way to swap the drivers is not really helpful. Moreover, recreating the driver with different settings (but the same driver type) is much simpler than via this approach. So what are the real advantages here, and what are the cons of this patch?

[Nox]

http://7bitfaster.de/downloads/texturel ... 2436.patch

P.S: @hybrid just take a look at the patch Wink

The patch adds the ITextureLink file.

@Ulf sry, i missed your post. Well the const does not allow a grab of the Texture and in one place this const was removed by a const_cast. But i think "why keeping a const which will removed by a cast".

@hybrid well. If i dont miss anything irrlicht loads an image, creates a texture and stores the texture in an array. With this patch irrlicht loads an image, store it and creates a texture if needed. Of course is changing the driversettings possible without this patch but irrlicht cant do this in the current state, can it?

Here are some pros and cons im aware of:
+seperates hardwarebuffer from softwarebuffer
+merges IImage and ITexture
+hardwarebuffers are created automatically
+once loaded images are stored if wished
-stores the texture three times(!) because with the current settings the imageformat can differs from the textureformat (*)
-does not support automatic updates of the hardwarebuffer (*)
-need a lookup
-locking of textures is not possible(*)

(*): this problem can be solved by implementing adding some methods or changing the behavior. I.e. automatic updates needs an Image::setdirty. The locking of textures maybe need only a redirect of IImage::lock depending or a access to the texturelinks. The storage problem is harder to solve. One solution is to convert the image to the texturesettings or make only a buffercopy if needed. This way the copycount can be reduced to 2 for ogl and dx and for the software to 1.

[BlindSide]

I don't want to store IImages in system ram either. I was thinking about something like either retrieving the texture from the driver's video memory (Be that gpu or system depending on the driver type), and recreating that texture for the other driver. Or another way of doing this would be storing the unique filename of the texture (But this wouldn't be consistant with any modifications applied to the texture or even textures created at runtime).

What do you think about the first approach hybrid? Speed between gpu and system memory is not an issue as it would be used very infrequently (Changing driver types, etc).

This isn't directly related to the IImage problem, but it would be nice to create more seperation between IImage and ITexture. Maybe we can even make IImage a pure template type defined in the headers (Which may help with the different color formats issue).

Cheers

[hybrid]

http://7bitfaster.de/downloads/texturel ... 2436.patch

P.S: @hybrid just take a look at the patch Wink

The patch adds the ITextureLink file.

No, it does not. *Please* have a look yourself

@Ulf sry, i missed your post. Well the const does not allow a grab of the Texture and in one place this const was removed by a const_cast. But i think "why keeping a const which will removed by a cast".

grabbing and dropping is possible with const objects.

@hybrid well. If i dont miss anything irrlicht loads an image, creates a texture and stores the texture in an array. With this patch irrlicht loads an image, store it and creates a texture if needed. Of course is changing the driversettings possible without this patch but irrlicht cant do this in the current state, can it?

The thing is that ITexture does not necessarily hold memory for the texture, as it's kept in GPU memory. When storing the IImage instead it would always require system memory *and* GPU memory.
Since changing driver settings is one of the most important things this patch might be used for, it has to be checked if it's really the best solution to go for.

Here are some pros and cons im aware of:
+seperates hardwarebuffer from softwarebuffer
+merges IImage and ITexture
+hardwarebuffers are created automatically
+once loaded images are stored if wished
-stores the texture three times(!) because with the current settings the imageformat can differs from the textureformat (*)
-does not support automatic updates of the hardwarebuffer (*)
-need a lookup
-locking of textures is not possible(*)

(*): this problem can be solved by implementing adding some methods or changing the behavior. I.e. automatic updates needs an Image::setdirty. The locking of textures maybe need only a redirect of IImage::lock depending or a access to the texturelinks. The storage problem is harder to solve. One solution is to convert the image to the texturesettings or make only a buffercopy if needed. This way the copycount can be reduced to 2 for ogl and dx and for the software to 1.

HW and SW buffers are already separated into two classes (IImage and ITexture). Whether merging them is good or bad depends. Automatic creation of HW buffers is a no-go, as we also need pure SW images. The last pro is unclear, don't know what gets stored here (and is different to the current solution).
The loopup is quite ugly and should be put into some overloaded operator. This would require a pointer to the driver being stored, though. The copy count has to be 0(!) for hw accelerated drivers, at least it should be possible. 2 or 3 is intractable. Locking is also inevitable.

BlindSide's solution of gathering the data from the GPU upon request is a pretty good approach for driver changes. Adding more separation between IImage and ITexture seems like the opposite of what we try to achieve in this thread, though. For a leaner CImage implementation we'd need IImage manipulators. Those would remove the necessity for most CImage methods, and hence avoid this huge implementation overhead.

[BlindSide]

One of the biggest problems with sharing textures across different driver types without keeping a copy in system memory would be having consistency over both versions.

A *slow* approach would be that when doing a lock, the texture would be downloaded from an arbitarily selected driver that holds that texture to a temporary buffer (As they would essentially both hold the same information). Then upon unlock() that data would be uploaded to both drivers. Yes this is similar to what's happening now, but the buffer would be created and destroyed during this process, to accomidate the fact that most people wouldn't need to perform this operation.

Another approach would be to provide a pointer directly to the data (If any of the drivers that "share" this texture support this), and then propogate the changes to the other drivers using that driver's modified data upon unlock().

We could provide an alternative function that doesn't create a temp buffer or download any data and instead uploads a user provided buffer for better performance. This would be for things like streaming video data.

I'm still uncertain HOW to go about all this though. At this point I just want to establish a concensus on what exactly our goals are and what the interface should provide (If it needs to be any different than the current interface).

I'm not sure if anyone noticed but I personally want to provide the ability to e.g. share a texture between a hardware and software (Or another hardware) driver at the same time. If we are going to go down this path we may as well make that possible.

[Nox]

@hybrid im so sorry. I was not aware of it. Shame on me Uploaded it.

grabbing and dropping is possible with const objects.

i was not aware of the fact that a grab/drop const is possible. Again a new fact learned. But the software drivers need a non-const pointer of the IImage. So i can cast it or remove the const or let the copy of the image in the softwaredriver. I dont know which is the best solution and it depends on the creation behavior for different settings. I guess the best way would be to avoid the Texturelinks in the softwaredrivers.

The thing is that ITexture does not necessarily hold memory for the texture, as it's kept in GPU memory. When storing the IImage instead it would always require system memory *and* GPU memory.

DX9 creates the "normal" textures with D3DPOOL_MANAGED. At the best of my knowledge this means that the texture is stored twice: "D3DPOOL_MANAGED Resources are copied automatically to device-accessible memory as needed. Managed resources are backed by system memory and do not need to be recreated when a device is lost.".
The OGL textures creates a CImage as backup.
So the "normal" textures are already stored twice!

Since changing driver settings is one of the most important things this patch might be used for, it has to be checked if it's really the best solution to go for.

Well it is just one possible way. Maybe the other ideas are better.

Automatic creation of HW buffers is a no-go, as we also need pure SW images.

The HW buffer is created automatically if getTextureLink is called and there does no HW buffer exists for this image.

The loopup is quite ugly and should be put into some overloaded operator

Well it is similar to the vertex- and indexbuffers. There is a lookup too.

The main differences are:
-no difference between IImage and ITexture (should be dropped) anymore
-the copy can be moved from the hardwareside to the softwareside this way
-the sw buffers are completely independend from the hw buffers

[hybrid]

Having two different drivers in one device is not supported so far, so you'd have to provide for a way to create them on the fly (and in parallel?) and switch between them. Question is where this would be needed. Another thing is a recreation of the same driver, due to device lost or context switch. This is not really a user-space thing, so it might not require all these changes anyway.
For me, the merging of ITexture and IImage would be useful if we can provide things that ITexture has or gets (mipmap, 3d level, cubemap, ...) in IImage for a clean way from files to textures, as well as offering access to those properties from the user-app.
If you just need to copy data from one driver to the other you should just lock the textures you need, copy them into an IImage, drop your device, generate the new one, and load the textures from your IImages.