converting jpeg in memory buffer to bitmap format

[slavik262]

Hi all. I’m in a robotics program (FIRST Robotics) and my team’s robot has a webcam attached to it. Using the API provided by the FIRST organization that runs this world-wide competitive program, the robot sends packets over a TCP connection to a particular port on a laptop in the same network which includes the latest image from the webcam encoded as a jpeg. This is displayed by dashboard software (also provided by FIRST) on the laptop so that the drivers of the laptop can have a live video feed.

The thing is, I don’t like the FIRST dashboard for several reasons. I'm making a custom dashboard program, and I'm going to implement a video feed. I have the networking figured out, but the problem is the jpeg format of the incoming images. I'm using Irrlicht for drawing the dashboard, and as you know it can display images based off of a jpeg file, but it would be a horrendous waste of resources to continually save the jpeg data from the packets to a file and read it back in. Alternatively, Irrlicht can draw images straight from a memory buffer, but the image must be in a standard bitmap format (16-bit A1R5G5B5 or R5G6B5, 24-bit R8G8B8, etc.). I need a C or C++ library of some sort that can convert a jpeg data stored in a memory buffer to a bitmap format (I’d assume R8G8B8 would be the most common).

Does anybody have such a library? I’d really prefer not to create my own jpeg decoder, as I don’t really have the time to dedicate that much time to coding just a single aspect of our robot system.

[hybrid]

You can also put the jpeg image into memory, create a MemoryReadfile from it and load it via the usual Irrlicht loaders from there.

[vitek]

Honestly, I don't think that Irrlicht isn't particularly well suited for this.

The best way to do what you want is to get an IReadFile pointer that has the jpeg data, and then create an IImage for that. Then copy that data into a existing ITexture object. Something like this...

Code: Select all

// do the following stuff when new jpeg data comes in over the network
io::IFileSystem* fileSystem = device->getFileSystem();
asset(fileSystem);

io::IReadFile* file = fileSystem->createMemoryReadFile (jpg_data, jpg_data_len, "dummy.jpg");
assert(file);

video::IImage* image = driver->createImageFromFile(file);
if (NULL != image)
{
  // now you can use the image to get a texture. see following comments...

  image->drop();
}

Now that you have the image data, you need to get it into a texture. The options are to either create a new texture every frame, or to update en existing texture. Both of these options are very expensive.

Code: Select all

  // just stick this into the if above
  video::IVideoDriver* driver = device->getVideoDriver();
  assert(driver);

  // remove the texture from the cache
  driver->removeTexture(myDisplayTexture);

  // create a new texture, which gets inserted into the cache
  myDisplayTexture = driver->addTexture(image->getFileName(), image);

  // update the gui element to refer to the new texture
  myDisplayElement->setImage(myDisplayTexture);

You could avoid creating a new texture every call by copying the image data into the texture manually. This can be tricky, but it might be less expensive. You also wouldn't need to worry about the texture cache.

Code: Select all

video::IVideoDriver* driver = device->getVideoDriver();
assert(driver);

// sx, sy and colorFormat depend on the size and format of the incoming
// jpeg data. just want to do this once at program startup.
video::ITexture* texture = driver->addTexture (const core::dimension2d<u32>(sx, sy), "::dummy.jpg", colorFormat);

  // this stuff would go into the if statement above
  video::ITexture* texture = driver->getTexture("::dummy.jpg");
  assert (texture);

  c8* tex_data = texture->lock();
  if (NULL != tex_data)
  {
    // copy data from image. this code can be swiped from the engine. 

    texture->unlock();
  }

Travis

[slavik262]

Honestly, I don't think that Irrlicht isn't particularly well suited for this.

I chose Irrlicht because it draws quickly (since it's a graphics engine) and especially because its windowing/GUI system is really easy to work with. I need to develop this application really quickly, but tools/languages more suited for that (VB.NET, C#, GTK+, etc.) wouldn't give me the real-time performance I need.

To be frank, why shouldn't I use Irrlicht?

Also, I looked through the source and I'm still confused as to how to copy data directly into textures. Could you point me in a direction or give an example? I'm using Direct 3D 9

[vitek]

Look at the source for CImage::copyToScaling().

Travis

[slavik262]

Should I be able to use similar code to copy the image directly to the texture, or is there anything I need to do to reformat it first? What format should the image itself be in?

I apologize for asking so many questions. I've just never dealt directly with a DirectX texture before (or a texture from any other driver), so I don't know what special considerations have to be taken or what its memory layout is.

[hybrid]

Just take the version which writes to a raw pointer and uses informations such as size and color format. You can query those values directly from the texture. Then lock the texture, use the returned pointer to write to, and unlock it.

[slavik262]

Sounds wonderful. Thank you for all the help.

[dmer]

i want to update texture data real time ,anybody,do your know how to do that ,similer opengl, You can use glTexSubImage2D, as the name implies, to alter just a sub-region of the texel data.
thanks