Irrlicht unit tests

[wing64]

SVN: 4154
Log: http://pastebin.com/22fmttPQ
PC: i7-930, ATI Readeon HD4670 with Catalyst 12.3, Ram 6 Gb
OS: Win7 64bit

[Nadro]

I fixed an issues with an aero effects, which cause an errors in D3D tests. Now my platform passed 60/65 tests (in debug mode 61/65 -> loadScene doesn't work properly on my config in release mode)

Here is my results:
http://pastebin.com/izMQDRRR

[CuteAlien]

I tested again last week with latest changes, but forgot posting. On Windows VS32 there are 4 tests fixed since last run ('drawPixel', 'textureRenderStates', 'draw2DImage', 'twodmaterial') on Linux one more fails now ('drawRectOutline').

Code: Select all

 
SVN Revision                4162/VS32    4162/Linux32  4166/Linux64
'matrixOps'                                                    f (fixed now)
'fast_atof'                                                    f (fixed now)
'writeImageToFile'                                             f (fixed now)
'ioScene'                                                      f (fixed now)
'drawPixel'                     ok                f            f
'drawRectOutline'                                 f            f
'material'                      f                 f            f
'renderTargetTexture'           f                 f            f
'textureFeatures'               f                 f            f
'textureRenderStates'           ok                f            f
'transparentMaterials'          f                 f            f (sometimes)
'antiAliasing'                  f  (fixed)        f            f
'draw2DImage'                   ok                f            f
'lights'                        ok                f            f
'twodmaterial'                  ok                f            f
'viewPort'                      ok                f            f
'mrt'                           ok                f            f
'projectionMatrix'              ok                f            f
'lightMaps'                     ok                f            f
'triangleSelector'              ok                f            f
 

Linux 32-bit: http://ideone.com/kXduu
Windows VS32: http://ideone.com/a3dVC
For comparison this time a new Linux-64 system: http://ideone.com/CLIxQ

New Linux system data:
NVIDIA Driver: 295.40
Linux-x86_64 on Debian Squeeze
8 GB Memory
GeForce GTX 550 TI

[CuteAlien]

Most tests (at least on Linux64, probably also on Linux32) are because the screenshot function switches the colors red and blue. So changing GL_BGR to GL_RGB for ECF_R8G8B8 in createScreenShot in COpenGLDriver fixes 13 tests (yay!). Also fixes screenshots. Not sure why it works like this on Windows, but works there either way, so fixing this doesn't seem to break anything as far as I could see).

Reason I didn't fix it yet is that there seem to be more cases where it looks to me like rgb and bgr might be switched (just from looking at names). But that would probably need own tests before I mess up things there. Can you take a look Hybrid - you know more about OpenGL driver. Once this is done we're down to just 3-4 tests failing, getting there :-)

[Foaly]

63 of 69 tests passed.
http://pastebin.com/caTBThNQ

svn revision 4896
Windows 7 64 bit, Irrlicht 64 bit with unicode filesystem and without D3D8
16 GB RAM, 8 Cores 3.11 GHz

failing tests:
projectionMatrix
draw2DImage
antiAliasing
transparentMaterials
renderTargetTexture
drawVertexPrimitive

[kklouzal]

Windows 8.1 64-Bit; IntelHD-4000 & ATI Radeon-8700M; VS2013 Express; Build Target: Release-Fast FPU @32 Bits. Latest Irrlicht Trunk SVN @time of post. ATI-Catalyst Version 14.4
35 tests of 69 passed.
https://gist.github.com/kklouzal/e336adf010bb3bd04575

[CuteAlien]

Thanks, the different image sizes is interesting. I'll run it again when I'm back to Irrlicht next week.

[AReichl]

Found something in "testUtils.cpp":

#if defined(_MSC_VER)
should be
#if defined(_MSC_VER) || defined(_IRR_WINDOWS_API_)

and it should be placed BEHIND
#include "testUtils.h" // + Irrlicht.h with _IRR_WINDOWS_API_

And because
#define TESTING_ON_WINDOWS
is only used locally once, we could simplify everything to:

#include <assert.h>
#include <memory.h>
#include <stdarg.h>
#include <stdio.h>

#include "testUtils.h" // + Irrlicht.h with _IRR_WINDOWS_API_

#if defined(_MSC_VER) || defined(_IRR_WINDOWS_API_)
#define _CRT_SECURE_NO_WARNINGS 1
#include <windows.h> // For OutputDebugString()
#endif // _MSC_VER || _IRR_WINDOWS_API_

[CuteAlien]

Thanks looks better that way I guess. Checked in.

[Coolcoder360]

I dunno if this is still a thing to post test results here,
but I ran the tests and passed 67 out of 70 tests on Archlinux, with Intel HD 3000 on an i7, 6GB RAM, on svn version r5600.
It failed tests 51, 53, and 58.
the log is here: https://ideone.com/IHGBIV
Compared to the last test results I see, that's quite good, good job to the dev team.

[AReichl]

Last time i run the tests, i had a difference between DirectX and OpenGL in some of them.
Also one test was different when compiling with VisualStudio or MinGW ( all on Windows ).
I think i solved some differences by adding a compiler flag for MinGW, but i have to check again.
I could repeat them and see what has changed since.

[CuteAlien]

@Coolcoder360: Hm, have to run again here. This is with Irrlicht trunk? I'm a little confused because test-numbers seem to have changed (51 always fails on OpenGL, but it was test 50 in Irrlicht 1.8). I forgot to run them anyway after latest fixed for some OpenGL materials, might be some have to be adapted after fix.

@ AReichl: Never hurts to post them here once in a while. Even if it's just to see if something changed.

[Coolcoder360]

Yes, this was with the latest svn trunk.

And to clarify the numbers with names:
51 - drawVertexPrimitive
53 - renderTargetTexture
58 - antiAliasing

[juozas]

Some tests failing on my Ubuntu 20.04 LTS with GeForce 210 legacy card and nouveou driver. CPU: Pentium(R) Dual-Core E6300 @ 2.80GHz
Failing tests:
Test 53 'renderTargetTexture'
Test 58 'antiAliasing'
Ful log: https://gist.github.com/juozaspo/0cd745 ... 71644acb17

[CuteAlien]

Thanks! I should give Nouveou another chance at some point. Rather glad to see you get so much working with it by now (last time I tried Nouveou on my Ubuntu Laptop it froze before even getting to the login screen, but been a while).