make sharedlib fails on Linux

[bvanevery]

This is shader-pipeline branch. On Lubuntu 12.10 Linux with GCC 4.7.2, some kind of compile incompatibility in CBSPMeshFileLoader.cpp involving -fPIC.

bvanevery@nomad:~/devel/shader-pipeline/source/Irrlicht$ make -k sharedlib
g++ -I../../include -Izlib -Ijpeglib -Ilibpng -I/usr/X11R6/include -DIRRLICHT_EXPORTS=1 -Wall -pipe -fno-exceptions -fno-rtti -fstrict-aliasing -g -D_DEBUG -fPIC -shared -Wl,-soname,libIrrlicht.so.1.8 -o libIrrlicht.so.1.8.0 CBSPMeshFileLoader.o CMD2MeshFileLoader.o CMD3MeshFileLoader.o CMS3DMeshFileLoader.o CB3DMeshFileLoader.o C3DSMeshFileLoader.o COgreMeshFileLoader.o COBJMeshFileLoader.o CColladaFileLoader.o CCSMLoader.o CDMFLoader.o CLMTSMeshFileLoader.o CMY3DMeshFileLoader.o COCTLoader.o CXMeshFileLoader.o CIrrMeshFileLoader.o CSTLMeshFileLoader.o CLWOMeshFileLoader.o CPLYMeshFileLoader.o CSMFMeshFileLoader.o CColladaMeshWriter.o CIrrMeshWriter.o CSTLMeshWriter.o COBJMeshWriter.o CPLYMeshWriter.o CSkinnedMesh.o CBoneSceneNode.o CMeshSceneNode.o CAnimatedMeshSceneNode.o CAnimatedMeshMD2.o CAnimatedMeshMD3.o CQ3LevelMesh.o CQuake3ShaderSceneNode.o CAnimatedMeshHalfLife.o CBillboardSceneNode.o CCameraSceneNode.o CDummyTransformationSceneNode.o CEmptySceneNode.o CGeometryCreator.o CLightSceneNode.o CMeshManipulator.o CMetaTriangleSelector.o COctreeSceneNode.o COctreeTriangleSelector.o CSceneCollisionManager.o CSceneManager.o CShadowVolumeSceneNode.o CSkyBoxSceneNode.o CSkyDomeSceneNode.o CTerrainSceneNode.o CTerrainTriangleSelector.o CVolumeLightSceneNode.o CCubeSceneNode.o CSphereSceneNode.o CTextSceneNode.o CTriangleBBSelector.o CTriangleSelector.o CWaterSurfaceSceneNode.o CMeshCache.o CDefaultSceneNodeAnimatorFactory.o CDefaultSceneNodeFactory.o CSceneLoaderIrr.o CVertexDescriptor.o CParticleAnimatedMeshSceneNodeEmitter.o CParticleBoxEmitter.o CParticleCylinderEmitter.o CParticleMeshEmitter.o CParticlePointEmitter.o CParticleRingEmitter.o CParticleSphereEmitter.o CParticleAttractionAffector.o CParticleFadeOutAffector.o CParticleGravityAffector.o CParticleRotationAffector.o CParticleSystemSceneNode.o CParticleScaleAffector.o CSceneNodeAnimatorCameraFPS.o CSceneNodeAnimatorCameraMaya.o CSceneNodeAnimatorCollisionResponse.o CSceneNodeAnimatorDelete.o CSceneNodeAnimatorFlyCircle.o CSceneNodeAnimatorFlyStraight.o CSceneNodeAnimatorFollowSpline.o CSceneNodeAnimatorRotation.o CSceneNodeAnimatorTexture.o CVideoModeList.o CFPSCounter.o CNullDriver.o COpenGLDriver.o COpenGLNormalMapRenderer.o COpenGLParallaxMapRenderer.o COpenGLShaderMaterialRenderer.o COpenGLTexture.o COpenGLSLMaterialRenderer.o COpenGLExtensionHandler.o CD3D8Driver.o CD3D8NormalMapRenderer.o CD3D8ParallaxMapRenderer.o CD3D8ShaderMaterialRenderer.o CD3D8Texture.o CD3D9Driver.o CD3D9HLSLMaterialRenderer.o CD3D9NormalMapRenderer.o CD3D9ParallaxMapRenderer.o CD3D9ShaderMaterialRenderer.o CD3D9Texture.o CColorConverter.o CImage.o CImageLoaderBMP.o CImageLoaderDDS.o CImageLoaderJPG.o CImageLoaderPCX.o CImageLoaderPNG.o CImageLoaderPSD.o CImageLoaderTGA.o CImageLoaderPPM.o CImageLoaderWAL.o CImageLoaderRGB.o CImageWriterBMP.o CImageWriterJPG.o CImageWriterPCX.o CImageWriterPNG.o CImageWriterPPM.o CImageWriterPSD.o CImageWriterTGA.o CSoftwareDriver.o CSoftwareTexture.o CTRFlat.o CTRFlatWire.o CTRGouraud.o CTRGouraudWire.o CTRNormalMap.o CTRStencilShadow.o CTRTextureFlat.o CTRTextureFlatWire.o CTRTextureGouraud.o CTRTextureGouraudAdd.o CTRTextureGouraudNoZ.o CTRTextureGouraudWire.o CZBuffer.o CTRTextureGouraudVertexAlpha2.o CTRTextureGouraudNoZ2.o CTRTextureLightMap2_M2.o CTRTextureLightMap2_M4.o CTRTextureLightMap2_M1.o CSoftwareDriver2.o CSoftwareTexture2.o CTRTextureGouraud2.o CTRGouraud2.o CTRGouraudAlpha2.o CTRGouraudAlphaNoZ2.o CTRTextureDetailMap2.o CTRTextureGouraudAdd2.o CTRTextureGouraudAddNoZ2.o CTRTextureWire2.o CTRTextureLightMap2_Add.o CTRTextureLightMapGouraud2_M4.o IBurningShader.o CTRTextureBlend.o CTRTextureGouraudAlpha.o CTRTextureGouraudAlphaNoZ.o CDepthBuffer.o CBurningShader_Raster_Reference.o CFileList.o CFileSystem.o CLimitReadFile.o CMemoryFile.o CReadFile.o CWriteFile.o CXMLReader.o CXMLWriter.o CWADReader.o CZipReader.o CPakReader.o CNPKReader.o CTarReader.o CMountPointReader.o irrXML.o CAttributes.o lzma/LzmaDec.o CIrrDeviceSDL.o CIrrDeviceLinux.o CIrrDeviceConsole.o CIrrDeviceStub.o CIrrDeviceWin32.o CIrrDeviceFB.o CLogger.o COSOperator.o Irrlicht.o os.o leakHunter.o CGUIButton.o CGUICheckBox.o CGUIComboBox.o CGUIContextMenu.o CGUIEditBox.o CGUIEnvironment.o CGUIFileOpenDialog.o CGUIFont.o CGUIImage.o CGUIInOutFader.o CGUIListBox.o CGUIMenu.o CGUIMeshViewer.o CGUIMessageBox.o CGUIModalScreen.o CGUIScrollBar.o CGUISpinBox.o CGUISkin.o CGUIStaticText.o CGUITabControl.o CGUITable.o CGUIToolBar.o CGUIWindow.o CGUIColorSelectDialog.o CDefaultGUIElementFactory.o CGUISpriteBank.o CGUIImageList.o CGUITreeView.o zlib/adler32.o zlib/compress.o zlib/crc32.o zlib/deflate.o zlib/inffast.o zlib/inflate.o zlib/inftrees.o zlib/trees.o zlib/uncompr.o zlib/zutil.o jpeglib/jcapimin.o jpeglib/jcapistd.o jpeglib/jccoefct.o jpeglib/jccolor.o jpeglib/jcdctmgr.o jpeglib/jchuff.o jpeglib/jcinit.o jpeglib/jcmainct.o jpeglib/jcmarker.o jpeglib/jcmaster.o jpeglib/jcomapi.o jpeglib/jcparam.o jpeglib/jcprepct.o jpeglib/jcsample.o jpeglib/jctrans.o jpeglib/jdapimin.o jpeglib/jdapistd.o jpeglib/jdatadst.o jpeglib/jdatasrc.o jpeglib/jdcoefct.o jpeglib/jdcolor.o jpeglib/jddctmgr.o jpeglib/jdhuff.o jpeglib/jdinput.o jpeglib/jdmainct.o jpeglib/jdmarker.o jpeglib/jdmaster.o jpeglib/jdmerge.o jpeglib/jdpostct.o jpeglib/jdsample.o jpeglib/jdtrans.o jpeglib/jerror.o jpeglib/jfdctflt.o jpeglib/jfdctfst.o jpeglib/jfdctint.o jpeglib/jidctflt.o jpeglib/jidctfst.o jpeglib/jidctint.o jpeglib/jmemmgr.o jpeglib/jmemnobs.o jpeglib/jquant1.o jpeglib/jquant2.o jpeglib/jutils.o jpeglib/jcarith.o jpeglib/jdarith.o jpeglib/jaricom.o libpng/png.o libpng/pngerror.o libpng/pngget.o libpng/pngmem.o libpng/pngpread.o libpng/pngread.o libpng/pngrio.o libpng/pngrtran.o libpng/pngrutil.o libpng/pngset.o libpng/pngtrans.o libpng/pngwio.o libpng/pngwrite.o libpng/pngwtran.o libpng/pngwutil.o aesGladman/aescrypt.o aesGladman/aeskey.o aesGladman/aestab.o aesGladman/fileenc.o aesGladman/hmac.o aesGladman/prng.o aesGladman/pwd2key.o aesGladman/sha1.o aesGladman/sha2.o bzip2/blocksort.o bzip2/huffman.o bzip2/crctable.o bzip2/randtable.o bzip2/bzcompress.o bzip2/decompress.o bzip2/bzlib.o -L/usr/X11R6/lib -lGL -lXxf86vm
/usr/bin/ld: CBSPMeshFileLoader.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
CBSPMeshFileLoader.o: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
make: *** [sharedlib] Error 1

[hendu]

This happens if you did a static build, and then tried a shared build without cleaning in between - the .o file was compiled without -fPIC.

Please try after a make clean.

[bvanevery]

Aw gee whiz. I haven't seen a "plain makefile" build in so long, I'm surprised that this level of build primitiveness exists. Why wouldn't the makefile rules have different suffixes for static vs. shared components? It is typical to want to build both static and shared versions of a library, to have both available in a SDK installation, and to want to get the entire build done in one batch build. I realize the simplicity of plain makefiles has some advantages, but this does lack fit and finish. It also promotes less exercise of the build components, as users won't be routinely building both, but rather one or the other.

[CuteAlien]

Irrlicht always was compact in the sense of having very little dependencies on other stuff. And as I had my bunch of fights with stuff like wrong automake versions mingw or having to deal with the horror that is called scons I must admit I kinda start to like the simplicity of an easy to read Makefile - at least it's easy to work around problems. Thought we still might switch one day to cmake (or something similar as long as it's not scons) as all those manual files tend to get out of sync continuosly (especially the different VS versions).

[bvanevery]

Well, leaving the big build system planning horizon stuff aside, I think there are some more basic "best practices" for hand rolled Makefiles that could be done here. Like different suffixes for static and shared build rules. Or subdirectories for each, that's the other way to go typically.

Premake is the non-CMake build system that I've looked at to possibly do the same job. It's Lua-based so you'd get a better scripting language. I don't think it has built in query scripts for things like "find the OpenGL headers" and so forth. However, CMake's versions of those can go stale, so you end up having to write your own anyways. Premake probably also doesn't support as many IDE backends as CMake. Nothing is going to beat CMake in that respect, although I don't personally care about supporting lots of IDEs.

[hendu]

That's kinda missing the point here: right after reading "very little dependencies on other stuff", you want to add a build system that needs a language not used anywhere else in the project.

[bvanevery]

I was responding to CuteAlien's conjectures about doing CMake someday. Which is a component not presently in Irrlicht. Same as Premake would be. The Lua in Premake is embedded, it is not a separate component from Premake.

For present realities, again, shared and static libs could be built with different suffixes or subdirectories. These are the typical drills.

[CuteAlien]

I've only rather recently learned about premake and it did indeed look rather interesting. But haven't worked with it yet (while I did use cmake already in some projects).

[gerdb]

Like different suffixes for static and shared build rules. Or subdirectories for each, that's the other way to go typically.

That would be the best way, simply separate obj folders for shared and static version, dont understand the discussion about CMake.

[bvanevery]

Coughing up a CMake or Premake build is a much more labor intensive way to deal with build issues in general, of which this shared vs. static lib stuff is a specific instance. I don't think it's illogical to suggest such a course of action, it's just a lot more work.

[hendu]

They will also cause a lot of their own issues.

Anyway, looking forward to your patch to the build system

[CuteAlien]

The stuff about CMake, Premake is mostly because we have too many files right now which must be updated by hand. And then people using other IDE's might have it easier as well (thought one reason against CMake is that right now the project files generated by it will make it harder on C::B probably as last I checked it was only creating Makefiles for it and no project files). And when switching to that then it's probably a good time to rething folders etc. But certainly nothing speaks against improving the Makefile as well for now. Builds with different flags being in own folders would probably be nicer I guess.

[hendu]

Well, you know my stance on it: ignore people with IDEs. Practically no projects (as a % of all open source) include IDE project files with them, but only a standard build infra (make or autotools). Instantly only one place to update.

CMake, while a lot better than hipster build-systems-of-the-week like rake, scons, and *bake, still creates a huge number of issues compared to standard tools.

[CuteAlien]

Ah well... I work with VS all day by now so it's rather nice to have that ;-) (and actually pretty much every OS project I know has some VS support, although sometimes it's not good).

[bvanevery]

Well, you know my stance on it: ignore people with IDEs. Practically no projects (as a % of all open source) include IDE project files with them, but only a standard build infra (make or autotools). Instantly only one place to update.

In a CMake build, the "one place to update" is the CMakeLists.txt. All the various possible IDE files are generated from that one source. Same idea in a Premake build. Most IDEs have their own native format, but notably, KDevelop4 takes CMakeLists.txt files natively. My jury is out on what I think of KDevelop4 as I haven't been using it very long, but I have been able to browse and trace Ogre with it fairly painlessly. I believe Anjuta does magical things with Autoconf files, if that's one's culture, but it isn't mine so I haven't tried it out.

Anyways hendu I hope you don't have influence over builds in Irrlicht circles, because it sounds like with your attitude, even very basic "real world" things like supporting Visual Studio .sln files wouldn't have gotten done. Your attitude doesn't promote cross-platform growth. You sound like a "one true set of tools, and they're the Unix command line" kind of guy. Oddly enough it wouldn't bother me right now as Linux is my development platform, but it's a bad move for attracting open source developers. Better infrastructure yields more and better cross-platform programmers, and there aren't enough of those for every open source 3d engine to have them.

CMake, while a lot better than hipster build-systems-of-the-week like rake, scons, and *bake, still creates a huge number of issues compared to standard tools.

I'm a CMake veteran for 7 years now, and got paid to do it for money once, so I'm not buying that. Big builds have issues. A big CMake build has fewer issues than manually maintaining both Makefiles and Visual Studio files. Anyways go to school on how much trouble the Ogre crowd is or isn't having with their builds. Open Scene Graph is another serious 3d engine using CMake. They converted a very long time ago and are probably an example of best practices by now. Like Kitware itself, the purveyors of CMake, OSG is focused more on scientific visualization than games so they aren't your direct competition.