Feature request: premultiplied alpha

[Akabane87]

Hi there,

I tried to do a render target on a 32bits color texture initially cleared with fully transparent pixels. I ran into the following issue that the alpha blending rule when rendering onto this texture is not correct : the alpha result seems to take into consideration the color of the source texture (the render target) although its alpha is 0. The result pixels are then tint with the render target color and even the destination alpha is incorrect.

Here is a sample demonstrating the issue :

Code: Select all

 
int main()
{
    // ask user for driver
    video::E_DRIVER_TYPE driverType=driverChoiceConsole();
    if (driverType==video::EDT_COUNT)
        return 1;
 
    // create device
 
    core::dimension2d<u32> dim(800, 600);
    IrrlichtDevice *device = createDevice(driverType, dim, 32);
 
    if (device == 0)
        return 1; // could not create selected driver.
 
    device->setWindowCaption(L"Irrlicht Engine - 2D Graphics Demo");
 
    video::IVideoDriver* driver = device->getVideoDriver();
 
    // create 2 render target textures cleared with 2 different fully transparent colors : black and red,
    // and then draw on it a semi transparent white square on it 
    video::ITexture* renderTarget1 = driver->addRenderTargetTexture(dim, "RTT1", video::ECF_A8R8G8B8);
    if(renderTarget1)
    {
        driver->beginScene(true, true, video::SColor(255,0,0,0));
 
        // set render target texture
        driver->setRenderTarget(renderTarget1, true, true, video::SColor(0,0,0,0));
        driver->setTransform(video::ETS_VIEW, core::IdentityMatrix); //<- set view matrix to identity
 
        driver->draw2DRectangle(video::SColor(100,255,255,255),
            core::rect<s32>(100, 0, 200, 100));
 
        // set back old render target
        // The buffer might have been distorted, so clear it
        driver->setRenderTarget(0, true, true, video::SColor(0,0,0,0));
        driver->setTransform(video::ETS_VIEW, core::IdentityMatrix); //<- set view matrix to identity
 
        driver->endScene();
    }
    video::ITexture* renderTarget2 = driver->addRenderTargetTexture(dim, "RTT2", video::ECF_A8R8G8B8);
    if(renderTarget2)
    {
        driver->beginScene(true, true, video::SColor(255,0,0,0));
 
        // set render target texture
        driver->setRenderTarget(renderTarget2, true, true, video::SColor(0,255,0,0));
        driver->setTransform(video::ETS_VIEW, core::IdentityMatrix); //<- set view matrix to identity
 
        driver->draw2DRectangle(video::SColor(100,255,255,255),
            core::rect<s32>(200, 0, 300, 100));
 
        // set back old render target
        // The buffer might have been distorted, so clear it
        driver->setRenderTarget(0, true, true, video::SColor(0,0,0,0));
        driver->setTransform(video::ETS_VIEW, core::IdentityMatrix); //<- set view matrix to identity
 
        driver->endScene();
    }
 
    
    while(device->run())
    {
        if (device->isWindowActive())
        {
            u32 time = device->getTimer()->getTime();
 
            driver->beginScene(true, true, video::SColor(255,0,0,0));
 
            driver->draw2DRectangle(core::rect<s32>(0, 50, 400, 150), 
                video::SColor(255,0,0,255), video::SColor(255,0,0,255), 
                video::SColor(255,0,255,0), video::SColor(255,0,255,0));
 
            core::position2d<s32> m = device->getCursorControl()->getPosition();
            driver->draw2DRectangle(video::SColor(100,255,255,255),
                core::rect<s32>(0, 0, 100, 100));
 
            driver->draw2DImage(renderTarget1, core::position2d<s32>(0,0), 
                core::rect<s32>(core::position2d<s32>(0,0), renderTarget1->getSize()), 0, video::SColor(255,255,255,255), true);
 
            driver->draw2DImage(renderTarget2, core::position2d<s32>(0,0), 
                core::rect<s32>(core::position2d<s32>(0,0), renderTarget2->getSize()), 0, video::SColor(255,255,255,255), true);
 
            driver->endScene();
        }
    }
 
    device->drop();
 
    return 0;
}
 

And here is the result :

The 1st square on the left is directly rendered on the screen,
the second one is rendered through the 1st render target initially cleared with fully transparent black pixels,
and the 3rd one is rendered through rge 2nd render target initially cleared with fully transparent red pixels.

[Akabane87]

I found this interresting subject in the msdn website about alpha blending with render target with d3d9 : http://msdn.microsoft.com/en-us/library ... 85%29.aspx. I looked into the source code of irrlicht but none of the renderstate and the flags supposed to be used to make it work seem to be used.

[hendu]

There is no bug here.

When you render into those RTTs, you use an alpha value of 100. If you check the GL alpha blend docs, the common "alpha blend" mode is GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA.

This means that the destination alpha is not taken into account at all. White * alpha 100 + red * alpha 155 = what you see in RTT 2.


Note that if you do want to use the destination alpha, you can use it with a custom blend mode. I think you can't use the builtin 2d functions with that though.

[Akabane87]

Yes this is the common blend mode used on a frame buffer. But when someone generally uses a render target and want to render a semitransparent result, he certainly don't want to have his resulting texture tinted with the color of render target. For me it sounds logical to not use the common blend method when we use a render target.

In a second thought, I'm still searching for a manual method to achieve the type of blend I want.

Would you mean that I should use

Code: Select all

material.MaterialTypeParam = irr::video::pack_textureBlendFunc(irr::video::EBF_SRC_ALPHA, irr::video::EBF_DST_ALPHA, irr::video::EMFN_MODULATE_1X, irr::video::EAS_TEXTURE | irr::video::EAS_VERTEX_COLOR);

instead of

Code: Select all

material.MaterialTypeParam = irr::video::pack_textureBlendFunc(irr::video::EBF_SRC_ALPHA, irr::video::EBF_ONE_MINUS_SRC_ALPHA, irr::video::EMFN_MODULATE_1X, irr::video::EAS_TEXTURE | irr::video::EAS_VERTEX_COLOR);

[hendu]

Yes. But be aware that using that form may then give incorrect results when blending something where both have some alpha.

[Akabane87]

Sure it will ... Since I draw multiple semi transparent things on my render target, I will have issues for sure.

The correct calculation should be something like result = srcAlpha * srcColor + (1-srcAlpha)*dstAlpha*dstColor.
And we come back to the initial problem that we can't do this with the engine. Since (cf http://msdn.microsoft.com/en-us/library ... 85%29.aspx) there is a way to render correctly the colors on a transparent render target I assume this is a bug :p.

[hendu]

Uh, your page says the exact same thing as I do. Your formula is not available in DX either.

Ie, both DX and GL agree that the result is correct, it's your expectation that's incorrect

[Akabane87]

oO What I read in this page is that the driver is supposed to set D3DPMISCCAPS_SEPARATEALPHABLEND if we want to apply alpha channel independantly from color channel blending.

"When D3DRS_SEPARATEALPHABLENDENABLE is set to FALSE (which is the default condition), the render-target blending factors and operations applied to alpha are the same as those defined for blending color channels." When I watch the source code of irrlicht, all blending modes use the same calculation for color blending and alpha blending. I sincerely have no idea if my calculation is the correct one (actually it is not since the alpha should not fit with this formula). What is sure is that if someone wants to render something on a fully or semi transparent texture it will not do what you expect, and more likely you will be unable to do what you expect.

If you tell me it is impossible to do so either with opengl or dx then OK... but I'm a little bit suprised nobody ever complained about this...

edit : seems my formula were correct after all for the color blend only. Here is an interresting topic relating the same problem :
http://www.facepunch.com/showthread.php?t=1198195

[hendu]

Separate alpha blend is not what you think it is: it merely means a separate multiplier for the alpha channel and the rgb channels, instead of the same multiplier for all four. It only affects what ends up in the destination alpha channel.

[Akabane87]

Indeed and it is exactly what I think .

It would allow to have this formula for alpha channel only and any other formula for color blending :
"
(1.0 - alphaBelow) * alphaAbove + 1 * alphaBelow

aka

GL_ONE_MINUS_DST_ALPHA, GL_ONE
"

edit :
Here is the confirmation of what I guessed :
http://en.wikipedia.org/wiki/Alpha_compositing -> part "Alpha blending"
As we can see, compared to what irrlicht does, there is a missing dstAlpha factor multiplying the dstColor


edit 2 :
And a very nice article explaining the whole topic : http://home.comcast.net/~tom_forsyth/blog.wiki.html#[[Premultiplied%20alpha]]

[Akabane87]

I made this in practice by simply writing a function PremultiplyAlpha(ITexture* texture) that allow me premultiply all RGB channels with the A channel for all pixels of the texture. I use it after the loading of my texture.

Then I simply use this texture when drawing with

Code: Select all

 
if(usePremultipliedAlpha)
        material.MaterialTypeParam = irr::video::pack_textureBlendFunc(irr::video::EBF_ONE, irr::video::EBF_ONE_MINUS_SRC_ALPHA, irr::video::EMFN_MODULATE_1X, irr::video::EAS_TEXTURE | irr::video::EAS_VERTEX_COLOR);
    else
        material.MaterialTypeParam = irr::video::pack_textureBlendFunc(irr::video::EBF_SRC_ALPHA, irr::video::EBF_ONE_MINUS_SRC_ALPHA, irr::video::EMFN_MODULATE_1X, irr::video::EAS_TEXTURE | irr::video::EAS_VERTEX_COLOR);
 

in a 2D drawing function of mine... And all work perfectly with a real blending working correctly on any alpha-using destination.


So what I suggest to make irrlicht able to do this without having to premultiply alpha after loading or having to write a new loader is to simply add a bool paramater to the load function of any texture format allowing to use or not premultiplied-alpha pixels or not (by default on false to not break everything). Additionnaly to this, add a flag inside the texture to say if it uses a premultiplied alpha or not to let any 2D draw function (and any high level draw) what sourceFactor use amongst EBF_ONE and EBF_SRC_ALPHA. And that's all !

This is really a small change that could allow users to do multiple things in 2D or 3D as stated in the last article I provided previously commencing by the possibility to use a transparent render target.

[hendu]

OK, now your issue is clear. Might want to edit the title to say "Feature request: premultiplied alpha" or similar.

[Akabane87]

Yeah thanks to this conversation the cause of the problem is now clear. I edit the title to make it clearer.