createScreenShot speed

[Dave]

Hi all,

I'm experimenting with the 'createScreenShot' function. In my test program I use a loop that iterates 25 times a second (to reproduce 25fps). In that loop, I call the 'createScreenShot' and use the 'writeImageToFile' function to save all the screenshots on my hard drive.
The problem is that instead of the 25fps, I get around 5fps, which is not sufficient. Is there any way to get more speed out of this?

What I'm trying to capture do, is capture all animation in a scene to image files, one for each frame. Then afterwards I use them in an external program. For realistic results I need 25+ fps.

[hybrid]

You should create a custom render loop for this such that you can influence the timer. Simply advance 1/25s each render. Also make sure that your animations are not frame rate dependent.

[Dave]

I'm not sure what you mean by custom render loop. Right now I have a loop (it's infinit and uses 'sleep' to generate delay that simulates fps value) that is respondsable for the rendering and animation. I have a custom animation system which moves all items to their according location each frame. So I guess that my animation is frame dependent.

What I don't understand is, if I don't use the "export to image" functionality, everything works. I can set the fps value and I see that the rendering goes faster / more smoother if I increase it. When I add the export thing, I just see things going slow, which makes me believe that the createScreenShot function causes the delay.

I don't understand why my animation should be frame independant, because for every iteration in the loop, the animation gets applied to the objects, everything is rendered and the final image is saved. Nothing gets done parallel, everything is in the same thread. So that means that if things get slower, one of those 3 operations are causing the delay...i think.

Hmm, I might now see what you mean hybrid. I should influence (delay / speed up) the internal timer of Irrlicht according to my fps value (=25). Or call the 'Tick' function myself every 1/25s. But why should this work better than my "normal" loop?

Just for clearing things up, this is what I have (pseudo-code):

Code: Select all

while(device->run())
{
  sleep for 'x' milliseconds to maintain fps

  calculate and apply animation to all objects

  render everything

  create screenshot and save to disk
}

[vitek]

Unfortunately the createScreenShot() code was intended to be used to create a screenshot only at a user request, i.e. one screenshot when the user pressed the tab key

It is not cheap to read the back/front buffer from the gpu into an image. This this makes createScreenShot pretty expensive. Also, the amount of data is significant. At 1024x768x32bpp, you are capturing 3MB of data every time you call createScreenShot, and then you write it to disk. At 1280x1024x32bpp is is 5MB. So one way to improve things would be to reduce the resolution used for rendering.

In addition to the cost of getting the data, the raw image data is getting converted to the proper image file format which takes some time. For example the scanlines of a bmp image are flipped, and a jpg image is compressed using the jpeg library. This takes some time also. You could create your own binary image format and just write out the data to disk in the binary form. Once you're done recording, you could run a utility to convert the binary images to some usable format like jpg.

To answer your second question, by using frame rate independent animations, your animation will appear to have moved the same amount in the same amount of time regardless of how fast the rendering cycle is running. If you don't use frame rate independent animation, your animation will run slower when the frame rate is lower.

Travis

[Phant0m51]

I remember making a short film using Half Life, and what Half Life did to allow you to do it is it stopped the timer, created a screenshot, then animated for a frame and created another screenshot. It produced 30 fps films without any problems because it split 1 second of animation into 30 frames, so no matter how long it took to create the screenshot, it always got 30 fps with a smooth video. I believe that is what Hybrid is trying to say.

[vitek]

Ah, yes.

I guess that I'm saying is that the current createScreenShot() method was not intended to do screen capture at real-time frame rates. As mentioned above, you could let the system think that 1/25 sec had elapsed when really 1 sec had passed, but this would not be real-time anymore [it would be 1/25th of real-time]

Travis

[hybrid]

No, the idea is to use some predefined animation, such as the one from the demo start. This can be rendered to a real-time movie with the slowest screen shot method when using a manual timer. That's what I thought was intended and why I suggested the custom render loop with a manual timer.

[Dave]

Thanks everyone, for the quick reactions!

Because the previous code post didn't really show what I was doing, here is the more detailed version:

Code: Select all

DWORD WINAPI ProcessingThread(LPVOID lpParam)
{
	// Initialize variables used in this thread.
	int timeStart; // Start time before processing.
	int timeProc; // Time needed for processing.
	int timeEnd; // Wanted end time, used to maintain FPS value.

	while (device->run()) // Loop while Irrlicht device is running.
	{
		timeStart = timer->getRealTime();

		// Applies custom animation to all objects and also saves a screenshot to disk.
		if (processing)
			ProcessFrame();

		RenderFrame(); // Does nothing more than render 1 frame.

		timeProc = timer->getRealTime() - timeStart;
		timeEnd = 1000 / GetFPS();
		if (timeProc > timeEnd) // FPS value can't be maintained.
			;//break; // Error.
		Sleep(timeEnd - timeProc);
	}
	return 0;
}

For testing purposes I disabled the check if the FPS can be maintained, because it can't when saving the screenshots.

I understand that capturing the front buffer to disk might take some time. As you said, vitek, this is dependant of the image resolution and some other factors. I lowered the resolution from 800*600 to 640*480 and 320*240 and this made no difference; still got around 5fps. Also tried changing from jpg to bmp or png, but nothing changed. I also disabled my custom animation system, but still the images got captured/saved at 5 fps.

This makes me believe that for some reason the problem is not the actual saving of the screenshots, but perhaps my way of calling the function to save it. In my code above, I use a seperate thread to render, animate and output everything. Without the image capturing, this system works perfectly. It does however sync the rendering with both the animation and image output; everything runs at the same framerate. This is not a problem, I did this on purpose because there is no need to render quicker than the animation. You wouldn't really see the difference if the camera is not moving, which it isn't in my program. The framerate dependant animation is also not a problem right now, but thanks for noticing; perhaps I'll change that in the future.

Should I perhaps have to do something with the Irrlicht timer? I tried to change the speed but this made no visible effect. I know I'm just guessing here, but I really don't know.

[hybrid]

So you really want to do some real-time capturing? I don't think that this will work since grabbing the image data from the gfx card already takes a lot of time. Just check the time by adding the OpenGL call directly into the render loop (of course when using the OpenGL driver). This will simply dump the pixels to some user-allocated byte array. I guess this will already drop your frame rate below reasonable thresholds.

[bigfish]

Have you tried rendering everything to a texture? I'm really not sure how that works but since there are plenty of games where it looks like that can be done in real time, I would guess it's somehow less costly. Maybe you could render your whole scene to a texture, display said texture on the screen but also keep a copy of it on the heap. If you're still willing to render at a lower resolution and not try to capture a really long video, that would not only eliminate the screen-scraping process and the image formatting, but also save all the disk IO. This is a totally uneducated guess though - I don't know how big of an impact the disk IO actually has. Of course with 25 FPS you'll chew through your memory pretty quickly. If you could afford like 200 megs of heap space without any swapping going on, I think you could get around 10 seconds of 640x480x24 at 25 FPS. But then if the real performance hit is just the screen scraping and image formatting, maybe it would at least buy you enough time to write each frame to a raw format on the disk and convert it to a usable format later.

[vitek]

You haven't made clear if you require the screen capture to be done in real-time or not. If you don't require the simulation to run at real-time while the screen capture is running, you can force the time to only change by a small amount every tick.

Code: Select all

//
// disclaimer: this code is totally untested, and may not compile at all.
//
u32 timeForCapture = 0;
while (device->run()) // Loop while Irrlicht device is running. 
   { 
      // if we aren't processing or we haven't cached capture start time
      if (!processing || !timeForCapture)
      {
        timeStart = timer->getRealTime();

        // this has to be kept up to date while rendering normally 
        timeForCapture = timer->getTime();

        RenderFrame(); // Does nothing more than render 1 frame. 

        timeProc = timer->getRealTime() - timeStart; 
        timeEnd = 1000 / GetFPS(); 
        if (timeProc > timeEnd) // FPS value can't be maintained. 
           ;//break; // Error. 
        Sleep(timeEnd - timeProc); 
      }

      // if we are processing, we need to capture as fast as we can
      // but the capturing needs to get images as if 20fps is maintained
      else
      {
        // force time to a specified value
        timer->setTime(timeForCapture);

        // and advance the timer to pretend 50ms has elapsed
        timeForCapture += (1000 / 20); // 20fps

        // Applies custom animation to all objects and also saves a screenshot to disk. 
        ProcessFrame(); 

        RenderFrame(); // Does nothing more than render 1 frame. 
      }
   } 

Travis

[Dave]

Currently the screen capture needs to be real-time. So for that I don't think I'm going to get 25fps; it'll have to do with 5fps then. Perhaps with some more performance tweaks (resolution, driver change, render to texture) I'll get around 10-15.

Thank you all for your help and sorry for the somewhat late response