Stream a texture over the internet?

[dorrax]

I have some interest in making a multiplayer game. I have no experience with server/client stuff, but I thought that the simplest way to set one up would simply to have the server render to texture and then send video and sound to the client, which sends back keyboard and mouse data. I think this would have several advantages and disadvantages.

ADVANTAGES:
Very simple client program
Almost impossible to hack the client to cheat
Client can run on weak computer
Client can connect to modded servers more easily

DISADVANTAGES
Server has to do a lot of work, rendering a different scene for each player
Very fast internet connection is needed
GPU has to send the texture back to CPU, which is probably not optimized

Thankfully I think that 320 by 240 is a decent resolution and I even find 160 by 120 acceptable, so that makes the textures small and makes this more feasible, but it still might be a challenge. Let's say we have a server with 10 clients connected. The server will render 10 views to 10 textures, 1 for each player. If each texture is 24 bits per pixel, and 320 by 240 pixels in size, that's 320 * 240 * 24 = 230kB per player or 2.3MB total for each frame. If the speed is 30 frames per second, that's 230kB per frame * 30 fps = 6.9MB per second. Most people don't have internet speeds that high, so some form of compression is probably needed, but that will add time for compression at the server, and time for decompression at the client end.

This is a challenging problem, but it's one I'm interested in. I think this is how OnLive does it's "Cloud Gaming", which I don't like the idea of, I just think it would be cool way for a user to set up a server for his friends, assuming he has a good enough computer.

HERE ARE SOME QUESTIONS:
How to get textures back from GPU as fast as possible? I've seen some suggestions about pixel buffer objects in openGL (I will be using OpenGL and linux), but no clue how to do this in irrlicht.

How to send / receive texture data across internet? I realize this is a more generalized question. I have done some (very basic beginning stuff) reading about sockets and C++, but I have no experience using them. Would TCP or UDP work better? Is a stream of textures a stream or is every frame a message?

If I manage to get anything working, I'll be sure to let you know, thank you.

[greenya]

I don't know how to get textures back from GPU but having 320x200 resolution you can simplify you client to a movie player, so you stream video 320x200 and only track user input and send back to the server. On server you only need to render to texture so you get frame and can send it over the internet. To make an IImage from ITexture you can use IVideoDriver::createImage(), then you can easy access to image data or save it to disk file.

P.S.: besides if you will stream video, you can easy tweak the bandwidth, like youtube has switching the quality. So you will not have to send 6.9MB per second, but much much smaller, since video encodes with loosing quality.

[hendu]

Onlive uses both special gpus and hardware H.264 video compression, straight from the gpu output, so it never even goes to the cpu.

[dorrax]

Onlive uses both special gpus and hardware H.264 video compression, straight from the gpu output, so it never even goes to the cpu.

Well that makes sense. Probably no way to get one of those GPUs. I guess it would need an ethernet port instead of any of the normal video outputs. Closest thing I found to that is nvidia grid, which isn't a GPU so much as a specialized rack of GPUs designed for streaming games, and you can't buy one, you can rent a server from them and run your software in a virtual machine. I don't like to see gaming moving in that direction.
I also know they try and use high resolutions, make it look like you'd expect the game to look as if you were running on your own machine. Since I want to run at very low resolution that begins to make this feasible for normal hardware, but it would be a stretch.

and Greenya, why would I need a 320 by 200 resolution for that? Is there some reason a 320 by 240 wouldn't work?

One other thing I thought could speed it up would be to reduce the bits per pixel from 24 to 12 or 6. It would greatly reduce the number of colors available, and I would probably need some form of dithering shader to make it look acceptable, but I could just call it retro and sell it to hipsters. But how would I use such an old color format on modern hardware? I might have to settle on an 8 bit per pixel approach and waste 2 bits or use a 3 bit red 3 bit green 2 bit blue.

Thanks

[Nadro]

Closest thing I found to that is nvidia grid, which isn't a GPU so much as a specialized rack of GPUs designed for streaming games, and you can't buy one, you can rent a server from them and run your software in a virtual machine.

AMD has more liberal offer in this case, you needn't buy whole server like in NV offer, but you can just buy simple GPU - AMD Radeon Sky (anyway you must prepare a software on your own hand, but it looks like this is it what you want).

[dorrax]

Thanks for telling me about that, the radeon sky looks interesting. Seems very new so I'll have to pay attention to when it becomes available to buy. According to what I read it can handle 6 users at a time with 720p streaming, so it might be able to handle more at 320 by 240. It's passively cooled though, which is fine in a server in a data center where everything is cooled down, but might require some extra work to cool it in a desktop, but a standard liquid cooling set up should be fine.

[eejin]

Unless you are going for a blocky feel or making a really small game window I would render at atleast 640×360. I can load a 720p youtube video within a second and then it's smooth all the way. And this is at my home, at school or at friend's houses. (Netherlands)

Another thing you could do is instead of sending a texture, you send the keyboard data and stuff to the server. The server calculates stuff and sends that back so the client only has to draw stuff.

[dorrax]

eejin you have deduced exactly what I want to do, I was amazed at how well ultima underworld and daggerfall looked for their day and thought it would be cool to try that, so yes, I want a blocky appearance with full screen 320 by 240 resolution. I got the Black Art of 3D Game Programming book from 1995 and used java and openGL to make my own crappy software 3D renderer using that resolution, I even wrote a z buffer that does depth sorting to get alpha composition right, but it barely manages 30 FPS running extremely simple scenes. I'm sure I could improve it a bit, but I think I should use a "professional" graphics engine instead.

Here is a video of what I accomplished in java, I had a problem with the camera code that made it tilt funny, it's fixed now.
http://www.youtube.com/watch?v=_XYj113Le58