How to decrease texture loading time?

Post your questions, suggestions and experiences regarding to Image manipulation, 3d modeling and level editing for the Irrlicht engine here.
ibax
Posts: 193
Joined: Thu Jun 21, 2007 8:56 am
Location: hungary
Contact:

Re: How to decrease texture loading time?

Post by ibax »

Multithreading: I have only one .obj file and one .mtl file, and of course the 100 texture files. The mesh loading and texturing isn't inside irrlicht, I just load the irr file in irrlicht.

DDS: I never heard about is... I would like to try this also, but is it possible to do this, if the texture loading is based on the .obj and .mtl file (basically the .irr file) ? Can you write here a short example, how to manage DDS file support? Or this is a built in function? Is this DIRECT3D compatible also?

(I already found a jpg2dds converter here: http://www.easy2convert.com/jpg2dds/ )
CuteAlien
Admin
Posts: 9734
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: How to decrease texture loading time?

Post by CuteAlien »

My point still stays the same - don't start optimizing until you know _what_ is causing the slowdown. This is the first step in every optimization. You can compile in release with debug-information (I think VS also allows that by now, maybe by setting the "Debug Information Format" - in any case you need to have some .pdb file besides the dll). I don't understand why you get wrong information when compiling Irrlicht in debug - sounds to me rather like maybe your application is not using the dll which you compiled - could it be it finds another Irrlicht.dll on your system instead?

As long as it shows that a function that's not even called takes the most time you are definitely not getting the correct information. Set a breakpoint in that function to ensure it's really not called.

Don't consider the time you spend now on figuring out profiling to be wasted - in the long run that's a skill you will need all your life as coder. It's nearly as essential as knowing how to compile stuff and how to use a debugger.
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
ibax
Posts: 193
Joined: Thu Jun 21, 2007 8:56 am
Location: hungary
Contact:

Re: How to decrease texture loading time?

Post by ibax »

in any case you need to have some .pdb file besides the dll

Yes, pdb file exists. What is the role of this file?

As long as it shows that a function that's not even called takes the most time you are definitely not getting the correct information. Set a breakpoint in that function to ensure it's really not called.

You mean a breakpoint to png_set_expand? This is a function of irrlicht? Or what do you mean exactly?

Don't consider the time you spend now on figuring out profiling to be wasted - in the long run that's a skill you will need all your life as coder. It's nearly as essential as knowing how to compile stuff and how to use a debugger.

This profiling sounds interesting, but I never worked with that? Is there some desciption, tutorial, how to manage this?
hybrid
Admin
Posts: 14143
Joined: Wed Apr 19, 2006 9:20 pm
Location: Oldenburg(Oldb), Germany
Contact:

Re: How to decrease texture loading time?

Post by hybrid »

ibax wrote:This profiling sounds interesting, but I never worked with that? Is there some desciption, tutorial, how to manage this?
This link may help
CuteAlien
Admin
Posts: 9734
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: How to decrease texture loading time?

Post by CuteAlien »

VerySleepy is one profiling tool. There are also others (for example from AMD and from Intel) but I had generally rather good experiences with VerySleepy so far. So not sure what is wrong in your case.

I think pdb files contain symbol information which support tools like debuggers and profilers.

And yes - I meant set a breakpoint to png_set_expand which is inside Irrlicht. You can do that either by including the Irrlicht project file for a while to your project solution (pretty much as easy as clicking "add" - "existing project" and then selecting the Irrlicht project file). Also some other ways, but just add it to the solution, it's the easiest way (and once you don't need it anymore just go over it and hit the delete button and it's gone again). Then compile it with debug-information (aka select debug as target and click compile) and then set a breakpoint (you can do a text-search over all files in your solution in VS by pressing ctrl+shift+f to find png_set_expand ).
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
ibax
Posts: 193
Joined: Thu Jun 21, 2007 8:56 am
Location: hungary
Contact:

Re: How to decrease texture loading time?

Post by ibax »

Hi,

I made some profiling...
Image

Does this mean, that png_set_expand ran approximately 10% overall?
CuteAlien
Admin
Posts: 9734
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: How to decrease texture loading time?

Post by CuteAlien »

It seems so. Have you set a breakpoint and checked where it's called from now that you don't have png's anymore?
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
ibax
Posts: 193
Joined: Thu Jun 21, 2007 8:56 am
Location: hungary
Contact:

Re: How to decrease texture loading time?

Post by ibax »

hello,

I made another profiling. FYI: I'm developing under a VMWARE virtualized Win XP machine, becuase my computer has a 64bit Win7, and afaik Irrlicht with the precompiled dlls supports only 32bit.

I compiled the app in debug mode. The profiling has quite different result

when I executed the app on the vmware machine
Image

just the thread
Image

or on the Win7 64bit machine
Image
just the thread
Image


So in Win7 64bit there is a png set expand function, which makes that high load, on the vmware machine it is the sscanf funcion...


I was trying to set breakpoints and execute my app from VS, but I got several error messages which does not appearing when I execute the exe file directly (why is that?).

Error message:

Code: Select all

Unhandled exception at 0x7c812afb in house.exe: Microsoft C++ exception: CppSQLiteException at memory location 0x0012df30..

Code: Select all

Unhandled exception at 0x673e6e67 in house.exe: 0xC0000005: Access violation reading location 0x00000068.
mongoose7
Posts: 1227
Joined: Wed Apr 06, 2011 12:13 pm

Re: How to decrease texture loading time?

Post by mongoose7 »

You can't have your application compiled in Debug mode and Irrlicht compiled in Release mode.

I don't understand why you can't man-up and recompile Irrlicht.

You are running a 64-bit VMware in order to run a 32-bit app? Or are you running a 32-bit VMware?
luthyr
Posts: 69
Joined: Wed Dec 30, 2009 5:47 pm

Re: How to decrease texture loading time?

Post by luthyr »

I think if Irrlicht is in release, the functions listed are about guaranteed to be wrong.
ibax
Posts: 193
Joined: Thu Jun 21, 2007 8:56 am
Location: hungary
Contact:

Re: How to decrease texture loading time?

Post by ibax »

Hi,

The irrlicht.dll, which can be downloaded from here: http://downloads.sourceforge.net/irrlic ... ht-1.8.zip is compiled in Debug or Release mode?
Currently I'm compiling everything in Release mode, because I use an own logging method for checking the functionalities, variables, etc ...

What can be the reason, that this sscanf function takes the most CPU during the texture loading when using the vmware winxp virtual machine? On my Win7 computer I have that png_set_expand function every time...
mongoose7
Posts: 1227
Joined: Wed Apr 06, 2011 12:13 pm

Re: How to decrease texture loading time?

Post by mongoose7 »

I can only repeat what CuteAlien said. It would appear that the function being reported is not the one that is using all the time. If you are using the release DLL (the one in the ZIP file) then the functions should be reported correctly. Maybe there is a static function nearby that is using all the time?

What CuteAlien suggested was to stop in the function that is being reported. In Win7, see if you can stop in png_set_expand. Presumably you can't, because it isn't being called? If you can get a map file, maybe look to see if there is a static function that is loaded after png_set_expand, or see if png_set_expand is the last PNG function that is loaded.

Come to think of it, there are no references to the driver. But this would be because the entry points are static and obtained through the dynamic library calls. So I don't think you will be able to find the function responsible in this way. Maybe you need to use hendu's method of putting timing calls into your own code.
CuteAlien
Admin
Posts: 9734
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany
Contact:

Re: How to decrease texture loading time?

Post by CuteAlien »

Please compile Irrlicht yourself. It also has build targets for 64-bit (I'm working with those myself most of the time by now). And it's not hard - Irrlicht already comes with project files - you can simply include those in your solution and then make sure you copy the resulting dll in the place used by your application. To be able to set breakpoints you have to do that anyway. You do _not_ have to use a debug-build, but what you need to have is building with debug information (or the correct pdb file maybe - I'm not 100% certain myself what verysleepy is using but I think at least the correct pdb is needed). Otherwise the results can be messed up (as they obviously are). For example one of your results as DirectSoundCreate on top - which seems rather confusing.

Basically - once debugging with the Irrlicht dll works correct the profiler should probably also work correct.

You get the Irrlicht version for this from svn. For example here the latest 1.8 version: svn checkout svn://svn.code.sf.net/p/irrlicht/code/tags/release-1.8/ irrlicht-1.8
If you have any questions about svn or compiling just ask again and we can help with that.
IRC: #irrlicht on irc.libera.chat
Code snippet repository: https://github.com/mzeilfelder/irr-playground-micha
Free racer made with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
ibax
Posts: 193
Joined: Thu Jun 21, 2007 8:56 am
Location: hungary
Contact:

Re: How to decrease texture loading time?

Post by ibax »

Thanks for helping me... Currently the irrlich.dll is recompiled in debug mode (the size of irrlicht.dll file is the same, but I have now a pdb file also), and I have very sleepy results...

Image

WaitForMultiple objects and sscanf requires the most resources.

On my Win7 x64 I could not recompile irrlicht.dll in debug mode, I received the following:

1>------ Build started: Project: Irrlicht, Configuration: Debug x64 ------
1>Error: The "ConfigurationGeneral" rule is missing from the project.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

I'm trying to find a solution for this, but currently I cannot push more effort to that set_png_expand issue, because I have no x64 debug irrlicht.dll... I'm trying...
Post Reply