Can the BSP loader be forced?

christianclavet · Post by **christianclavet** » Sun Apr 12, 2009 6:21 pm

Hi, I was just wondering if we could disable the format check for the BSP loader as I saw in the source:

This code in the source check for a very specific header info and drop loading if the header of the file is not matching.

Code: Select all

if (header.strID != 0x50534249 || header.version != 0x2e)
	{
		os::Printer::log("Could not load .bsp file, unknown header.", file->getFileName(), ELL_ERROR);
		return false;
	}

having a possibility to "force" loading of a BSP could allow us to check for compatibility with other BSP variants. (I think, but could be totally wrong here, still a noob) (We could try to load SourceSDK BSP, MOHAA BSP, OpenArena? , etc)

The current loader seem to load the geometry and the lightmaps(not sure for the lightmaps, but seem to).

The good thing about BSP geometry is everything is CONVEX and good for culling algorythms and physic calculation. Would be nice that we try if the loader would load more variants of BSP format.

loki1985 · Post by **loki1985** » Sun Apr 12, 2009 8:05 pm

well, i would say: just try it, and report how it works. in the worst case it will (probably) just crash the application.

tho i doubt SourceSDK BSPs will work, IIRC they changed a lot from the original id Software format.

Katsankat · Post by **Katsankat** » Mon Apr 13, 2009 8:25 pm

in the source SDK there are usable infos in bsp.h for sure,
but bsp.cpp ... is not there. And no info from Vavle unless ...

Good article about it
http://www.geocities.com/cofrdrbob/bspformat.html

Although point entities are directly usable for any bsp version,
detail brushes (brush based entities) need code.
I coded the func_train if you need Christan. Do you have IRC?

christianclavet · Post by **christianclavet** » Mon Apr 13, 2009 11:40 pm

Wow! Thanks Katsankat! I'll check that doc. Seem to talk about details of the format. I'm mostly interested by the headers now. You have redone the FUNC_TRAIN?

Loki. I was really unsure about modifying the source. But since your comment, questionned myself and done simple test with other things related to the source. It's not to bad. I don't know why I was so afraid.

Think I will try to tweak it to see if I can load some other types of BSP in there.

If some of you have other BSP files, it would be cool if I could have access to an example file to see If I can load it. I already have access to my maps produced with Hammer/Source SDK.
EDIT: Tried it with Source SDK .BSP: It doesnt work nothing is loaded and a crash occur. Probably that the "lumps" and headers are too different. Perhaps If I have more time, and the info is right, and I'm getting better to imitate a programmer...

EDIT2:

Here are some diffences I've found:

Headers:

Code: Select all

struct tBSPHeader
		{
			s32 strID;     // This should always be 'IBSP'
			s32 version;       // This should be 0x2e for Quake 3 files
		};

For SourceSDK:
strID='VBSP'
version=''17, 19 or 20 dec"

IRRlicht Lumps infos:

Code: Select all

enum eLumps
		{
			kEntities = 0,     // Stores player/object positions, etc...
			kTextures,         // Stores texture information
			kPlanes,           // Stores the splitting planes
			kNodes,            // Stores the BSP nodes
			kLeafs,            // Stores the leafs of the nodes
			kLeafFaces,        // Stores the leaf's indices into the faces
			kLeafBrushes,      // Stores the leaf's indices into the brushes
			kModels,           // Stores the info of world models
			kBrushes,          // Stores the brushes info (for collision)
			kBrushSides,       // Stores the brush surfaces info
			kVertices,         // Stores the level vertices
			kMeshVerts,        // Stores the model vertices offsets
			kShaders,          // Stores the shader files (blending, anims..)
			kFaces,            // Stores the faces for the level
			kLightmaps,        // Stores the lightmaps for the level
			kLightVolumes,     // Stores extra world lighting information
			kVisData,          // Stores PVS and cluster info (visibility)
			kMaxLumps          // A constant to store the number of lumps
		};

SourceSDK lumps: (From web page)

Code: Select all

Lump:       Name:                   Purpose:
0           Entities                Map entities
1           Planes                  Plane array
2           Texdata                 Index to texture names
3           Vertexes                Vertex array
4           Visibility              Compressed visibility bit arrays
5           Nodes                   BSP tree nodes  
6           Texinfo                 Face texture array
7           Faces                   Face array
8           Lighting                Lightmap samples
9           Occlusion               Occlusion data(?)
10          Leafs                   BSP tree leaf nodes
11          Unused
12          Edges                   Edge array
13          Surfedges               Index of edges
14          Models                  Brush models (geometry of brush entities)
15          Worldlights             Light entities
16          LeafFaces               Index to faces in each leaf
17          LeafBrushes             Index to brushes in each leaf
18          Brushes                 Brush array
19          Brushsides              Brushside array
20          Areas                   Area array
21          AreaPortals             Portals between areas
22          Portals                 Polygons defining the boundary between adjacent leaves(?)
23          Clusters                Leaves that are enterable by the player
24          PortalVerts             Vertices of portal polygons
25          Clusterportals          Polygons defining the boundary between adjacent clusters(?)
26          Dispinfo                Displacement surface array
27          OriginalFaces           Brush faces array before BSP splitting
28          Unused
29          PhysCollide             Physics collision data(?)
30          VertNormals             Vertex normals(?)
31          VertNormalIndices       Vertex normal index array(?)
32          DispLightmapAlphas      Displacement lightmap data(?)
33          DispVerts               Vertices of displacement surface meshes
34          DispLightmapSamplePos   Displacement lightmap data(?)
35          GameLump                Game-specific data lump
36          LeafWaterData           (?)
37          Primitives              Non-polygonal primatives(?)
38          PrimVerts               (?)
39          PrimIndices             (?)
40          Pakfile                 Embedded uncompressed-Zip format file
41          ClipPortalVerts         (?)
42          Cubemaps                Env_cubemap location array
43          TexdataStringData       Texture name data
44          TexdataStringTable      Index array into texdata string data
45          Overlays                Info_overlay array       
46          LeafMinDistToWater      (?)
47          FaceMacroTextureInfo    (?)
48          DispTris                Displacement surface triangles
49          PhysCollideSurface      Physics collision surface data(?)
50-52       Unused
53          LightingHDR             HDR related lighting data(?)
54          WorldlightsHDR          HDR related worldlight data(?)
55          LeaflightHDR1           HDR related leaf lighting data(?)
56          LeaflightHDR2           HDR related leaf lighting data(?)
57-63       Unused

Here found a list of lumps directly from source SDK:

Code: Select all

static char *s_LumpNames[] = {
	"LUMP_ENTITIES",						// 0
	"LUMP_PLANES",							// 1
	"LUMP_TEXDATA",							// 2
	"LUMP_VERTEXES",						// 3
	"LUMP_VISIBILITY",						// 4
	"LUMP_NODES",							// 5
	"LUMP_TEXINFO",							// 6
	"LUMP_FACES",							// 7
	"LUMP_LIGHTING",						// 8
	"LUMP_OCCLUSION",						// 9
	"LUMP_LEAFS",							// 10
	"LUMP_FACEIDS",							// 11
	"LUMP_EDGES",							// 12
	"LUMP_SURFEDGES",						// 13
	"LUMP_MODELS",							// 14
	"LUMP_WORLDLIGHTS",						// 15
	"LUMP_LEAFFACES",						// 16
	"LUMP_LEAFBRUSHES",						// 17
	"LUMP_BRUSHES",							// 18
	"LUMP_BRUSHSIDES",						// 19
	"LUMP_AREAS",							// 20
	"LUMP_AREAPORTALS",						// 21
	"LUMP_UNUSED0",							// 22
	"LUMP_UNUSED1",							// 23
	"LUMP_UNUSED2",							// 24
	"LUMP_UNUSED3",							// 25
	"LUMP_DISPINFO",						// 26
	"LUMP_ORIGINALFACES",					// 27
	"LUMP_PHYSDISP",						// 28
	"LUMP_PHYSCOLLIDE",						// 29
	"LUMP_VERTNORMALS",						// 30
	"LUMP_VERTNORMALINDICES",				// 31
	"LUMP_DISP_LIGHTMAP_ALPHAS",			// 32
	"LUMP_DISP_VERTS",						// 33
	"LUMP_DISP_LIGHTMAP_SAMPLE_POSITIONS",	// 34
	"LUMP_GAME_LUMP",						// 35
	"LUMP_LEAFWATERDATA",					// 36
	"LUMP_PRIMITIVES",						// 37
	"LUMP_PRIMVERTS",						// 38
	"LUMP_PRIMINDICES",						// 39
	"LUMP_PAKFILE",							// 40
	"LUMP_CLIPPORTALVERTS",					// 41
	"LUMP_CUBEMAPS",						// 42
	"LUMP_TEXDATA_STRING_DATA",				// 43
	"LUMP_TEXDATA_STRING_TABLE",			// 44
	"LUMP_OVERLAYS",						// 45
	"LUMP_LEAFMINDISTTOWATER",				// 46
	"LUMP_FACE_MACRO_TEXTURE_INFO",			// 47
	"LUMP_DISP_TRIS",						// 48
	"LUMP_PHYSCOLLIDESURFACE",				// 49
	"LUMP_WATEROVERLAYS",					// 50
	"LUMP_LEAF_AMBIENT_INDEX_HDR",			// 51
	"LUMP_LEAF_AMBIENT_INDEX",				// 52
	"LUMP_LIGHTING_HDR",					// 53
	"LUMP_WORLDLIGHTS_HDR",					// 54
	"LUMP_LEAF_AMBIENT_LIGHTING_HDR",		// 55
	"LUMP_LEAF_AMBIENT_LIGHTING",			// 56
	"LUMP_XZIPPAKFILE",						// 57
	"LUMP_FACES_HDR",						// 58
	"LUMP_MAP_FLAGS",						// 59
	"LUMP_OVERLAY_FADES",					// 60
};

Matching lumps are:
only the entities lump is matching. That explain why it was crashing.
That docs is really good. I'll check further (need a way to keep the old ENUM while switching to a new one if the header match a SOURCESDK one.

Still it seem encouraging that it could be possible for a noob like me to have the BSP loader compatible with other variants...

Code: Select all

Katsankat · Post by **Katsankat** » Tue Apr 14, 2009 4:30 pm

You should check botman's page http://hpb-bot.bots-united.com/bsp_tool.html

Katsankat · Post by **Katsankat** » Wed Apr 15, 2009 3:56 am

I managed to load a several HL bsp (and mods) version 30
using only the edges list, and the leaf list. That's the easy part.
... And the fatest to draw. FPS go beyond 1200 FPS in fullscreen.
Why, no mesh was created: the bsp edges list is simply used for draw3DLine().
Also, I didn't go further as textures are stored in wad files.
The PVS stuff in itself is not the hardest part either. It is easy to determine
in what leaf the cam is, and this is the first step to process visible leafs.

I just noticed also, most of the Q3 maps won't load properly. I tried to
copy CLevelMesh for some kind of HLLevelMesh but it didn't work because important
stuff is handled internally inside irrlicht

HL map with irrlicht, looks like a train station

netpipe · Post by **netpipe** » Wed Apr 15, 2009 3:19 pm

it would be cool to get quake 2 bsp loading there are tons of maps for it. the source code is available at http://www.alsprogrammingresource.com/q2view.zip

christianclavet · Post by **christianclavet** » Thu Apr 16, 2009 2:21 am

Cool, Katsankat

You done a great job! Being able to see something is a real feat. (Not near to accomplish yet)

For the textures, I was planning to retrieve the name and check for a equivalent name inside a texture folder. (I'm doing exactly the opposite when I bring something from 3DS MAX to Hammer).

The BSP contain more infos that we could need at first. Being able to load the face list and retrieve the internal lightmaps would be great.

For "props", I'm not planning to do it since we'll have to support the HL MDL format.

Katsankat · Post by **Katsankat** » Thu Apr 16, 2009 5:46 am

Using the map format does not force to use mdl format. A model is just a char array inside the bsp.
I did get rid of the maximum as you guess. I am writing a better token parser for entity list with brackets.

From Quake1 to HL2 there has been some road. Quake1 wal evolved into HL wad, then HL2 vmt shader material.
They took it, improved it, and kept licenses intact.
We gotta use textures formats understandable by editors though.

What was done in irrlicht Q3 namespace is that the whole map is turned into a mesh. At least it is lightmapped and textured

With vis, render only the faces you want. I wonder if both are compatible.

de_dust with spawnpoints

A real problem : all bsp maps are rotated +90 on X axis. It is because Quake is apparently using "a standard right-handed (X,Y,Z) coordinate system" (from reference for Quake maps)

If you look closely you will notice de_dust is mirrored. It might have something to do with ETS_WORLD in irrlicht. If anyone has an idea of what happens, any help appreciated!

hybrid · Post by **hybrid** » Thu Apr 16, 2009 7:42 am

Converting from right-handed to left-handed requires a flip about one or all three axes. This does flip the faces as well, though, so they also need to be flipped back. I guess this wasn't done for performance reasons (or maybe because some bsp formats use a left-handed system?) which leads to rotated/mirrored meshes.

Katsankat · Post by **Katsankat** » Thu Apr 16, 2009 8:41 am

3 axes in this case. Thanks for your confirmation. Now the flip() call at the very bottom of this page makes sense, in the render() method. http://gpwiki.org/index.php/Half-Life_BSP_viewer