BHM File Format

[Homer]

That's a curious observation - you should attempt to determine where the bottleneck is for your shader implementation.
And how do these results look when the cpu is under extreme loading (by some other process with equal priority)?

[Scali]

That's a curious observation - you should attempt to determine where the bottleneck is for your shader implementation.
And how do these results look when the cpu is under extreme loading (by some other process with equal priority)?

Not that curious considering that I mentioned a while ago that switching the D3D9 version to software vertexprocessing also boosted the framerate from about 170 fps to 350ish.
The shaders on the Intel chip are just very much underpowered. By using CPU skinning instead of shaders, I suspect that it also uses the CPU for the legacy fixed function T&L afterwards (my skinning code only performs the actual skinning itself, so I set up the positions and normals, then let OpenGL process the projection and lighting as usual).
Either that, or the fixed function T&L is running on the hardware, but it is far more optimized than the code that the shader compiler generates.
I believe there are registry keys to force software T&L for a certain process. I'd have to look into that. I'm not sure if that works for OpenGL aswell as D3D, but I know they force certain games to software because it's much faster.

Here's some documents on that from Intel:
http://www.intel.com/assets/pdf/whitepaper/318888.pdf
And a list of software that they have determined to run better in software:
http://www.intel.com/support/graphics/sb/CS-028231.htm

[Scali]

Hum, I just realized that I never actually built a release package from the current codebase. And as a result, there still is no download link on the project page.
Remind me to do this tonight
I think the current codebase is mature enough for a first release. I've tested it on Windows, linux and FreeBSD... someone also reported some small bugs/issues, which I've fixed.
I still want to improve the lighting model at some point, but that's icing on the cake more than anything.
I should remember to update the README.txt first though, as I've added extra fallback paths for non-VBO and non-programmable hardware, so virtually all of the extensions I listed are now optional, rather than hard requirements.

[Scali]

D'oh, the Visual Studio projects weren't in quite as good a shape as I thought. I hadn't updated the VS2005 project in ages, and I need to fix up some of the paths, so it works with the newer project layout, else it won't compile out-of-the-box.
I also had some minor changes on my local copy... had to rethink some of the design of the renderer... Do I want materials to be part of a mesh, or part of an object? I've chosen mesh for now, that way is less error-prone with regard to different vertex formats. Especially in D3D, a shader doesn't exist in a vacuum, but needs to know the vertex format to work on. So the same shader may have to be compiled multiple times for different vertex formats in different meshes.
So I'll have some more work to do before I can make a proper release.

[Scali]

Hum, totally forgot to build that release. Perhaps I'll find the time to do that next week... assuming I don't forget about it again
I could update the code a bit as well, by merging some of the changes I've made for the Croissant 9 thing.

Oh yea, and I mustn't forget to add some gratuitous CPUInfo stuff to the example program, just to promote that project as well. You know, give it some of that whizz-bang factor that the *nix crowd is so fond of... printing lots of meaningless information about the CPU on startup and all that.

[Scali]

Sadly I've not had time to build a proper release yet, with all the new projects I've taken on a few weeks ago.
However, for the VJ tool which I spoke of here, we are planning to use BHM for storage and communication purposes.
The first result of this choice has already emerged: we now have a parser in C# as well, and a small tool which can load a BHM file and display its contents with a treeview and a list for the data chunks. It's a nice tool for inspecting and verifying the contents of a BHM file, and I'm planning to include it in the open source project.

[G-Spider]

Hi ,Scali
I download the BHMSample-20100424.zip file ,and open the skin.bhm data file  .
I want to know the relationships between the data and BHM structure .

data:
42484D00 64000000 BC040000
05000000 00000000 01000000
07000000
01000000 00110000 00000000
02000000 20110000 00000000
03000000 18110000 00000000
04000000 12110000 00000000
05000000 13110000 00000000
06000000 15110000 00000000
07000000 19110000 00000000
08000000 11110000 0A000000
09000000 00110000 00000000
0A000000 20110000 00000000
0B000000 01120000 00000000
0C000000 12110000 00000000
0D000000 13110000 00000000
0E000000 11110000 04000000
0F000000 00110000 00000000
10000000 20110000 00000000
.......
struct:
typedef struct {
   char description[4];
   unsigned int version;
   unsigned int treesize;
} FileInfo;

typedef struct {
   unsigned int numchildren;

   unsigned int chunkid;
   unsigned int type;
} TreeNodeInfo;

typedef struct {
   unsigned int id;
   unsigned int type;

   unsigned int datasize;
   unsigned int totalsize;

   unsigned int dataoffs;
} ChunkInfo;

[Scali]

Well, it starts with the FileInfo struct.
This contains "BHM" in the description.
Then the version and the treesize.
The treesize is number of bytes for TreeNodeInfo structs that follow (so you can find where the chunks start in the file).
They are stored as depth-first. So you can parse then recursively. There is always one root node. So you start by parsing the first node, then you just parse the number of children for each node recursively, until all nodes are parsed (meaning you have parsed all children of the root node, which should also be as many bytes as the FileInfo said).
Then you will find the chunks in sequential order. Each chunk is a ChunkInfo node, followed by a raw blob of data bytes, the size of which is in the ChunkInfo. The next chunk follows immediately after the last byte of the previous chunk data.

[G-Spider]

aha? Thx.
data:
42484D00 64000000 BC040000 ;//=101
;//101 TreeNodeInfo;
05000000 00000000 01000000
07000000 01000000 00110000
00000000 02000000 20110000
00000000 03000000 18110000
00000000 04000000 12110000
00000000 05000000 13110000
00000000 06000000 15110000
00000000 07000000 19110000
00000000 08000000 11110000
0A000000 09000000 00110000
00000000 0A000000 20110000
.......
00000000 63000000 15110000
00000000 64000000 11110000
;-------------------------
;//ChunkInfo
64000000 11110000 C0000000
C0000000 00000000 00000100
02000300 02000100 .......

63000000 15110000 C8000000
C8000000 00000000 00000000
0000403F 00000000 .......
......

[Scali]

I finally took some time to update the GLUX project.
Some small code updates, but more importantly, I've added a VS2010 solution, and a zip file download, including the source, docs and pre-built libraries for Windows:
http://sourceforge.net/projects/glux/

I will also update the BHM project itself shortly. I got stuck when I wanted to update it earlier, because I was trying to support 3 different versions of Visual Studio at the time. I've decided to give that up, and only support VS2010. My code works in the free Express version of Visual Studio, so people won't have to buy a new version of Visual Studio if they want to build my code, just a free download.

[Scali]

Right, the BHM file format project has been updated as well, and I have created a zip file for download:
http://sourceforge.net/projects/bhmfileformat/

Same as above, includes the source, makefiles, VS2010 solution, pre-built libraries and binaries for Windows, docs etc.

edit:
While I'm at it, I'll also make a list of things I want to do for a next release:
- Update the readme file... The code went ahead of the readme file and already contains a CPU fallback path for skinning etc on non-shader hardware, and a fallback for no VBO extensions. Basically this means there is only the ARB_multitexture requirement left (and I don't quite remember why, as I don't use multitexturing... but it might introduce some functionality related to textures and shaders that I use... but probably not when the CPU fallback is in effect). DONE
- Replace cml math code with my own math, to remove the dependency on cml. DONE
- Add CPUInfo functionality, so some basic info about the CPU/cache/memory is displayed... combined with a slightly fancier handling of FPS, it could serve as a simple benchmark program... first step towards an open source 3DMark.
- Create an object for the main class as well, rather than just a main.cpp that is procedural in nature.
- Try to use an OpenGL 4.x context, instead of running in legacy OpenGL mode. Would also be a nice exercise in cleaning up the code perhaps.
- Add viewing tool for BHM files, to compensate for the only(?) downside of BHM compared to XML: human readability.

[Scali]

By the way, any OS X users on these boards?
If so, could you please try to build the BHM3DSample from this project?
I've tested it on Windows, linux and FreeBSD. It should also work on OS X, but I've never been able to verify that. Found out that OS X doesn't come with gcc preinstalled, unlike most other *nix-like OSes. So I can't just test it on a random OS X machine, it really needs to be set up for development, with the proper Xcode and UNIX commandline tools installed.
I know the CPUInfo project works on OS X, someone tested that a while ago, so I'm reasonably sure that this code will work as well... There may just be some slight glitches (as with CPUInfo initially), which I'd like to iron out.

[SpooK]

Environment:

• Mac OS X 10.7 Developer GM (11A511)
• Xcode 4.1 Developer Preview for 10.7 (gcc 4.2.1)
• GLEW 1.6.0 - Compiled/Installed from source
• FreeImage - Compiled/Installed via Mac Ports
• CML 1.0.3 - Downloaded from SourceForge

Changes needed to compile:

• Mac Ports places files under /opt/local/ instead of /usr/local; add -I/opt/local/includes and -L/opt/local/lib to the Makefile, immediately following their /usr/local counterparts, for maximum compatbility
• Remove -lrt from Makefile LIBRARIES argument listing
• Apply workaround for unsupported POSIX clock_gettime() as follows...

Code: [Select]

/*
 BHM 3D Sample
 Author: Scali

 This code is released under the BSD license. See COPYRIGHT.txt for more information.
 */

#ifdef _WINDOWS
#include <windows.h>
#include <Mmsystem.h>
#elif defined(__APPLE__)
#include <sys/time.h>
#else
#include <time.h>
#endif
#include <GL/glew.h>
#include <GL/glut.h>
#include <stdio.h>

#include "GL/GLUX.h"
#include "GL/GLTypes.h"
#include "Geometry/Object3D.h"
#include "Geometry/BHMObject.h"
#include "State.h"

#define lengthof(x) (sizeof(x)/sizeof(x[0]))
#define WIDTH 640
#define HEIGHT 480

BHMObject* pBHM = NULL;

#ifndef _WINDOWS
unsigned int timeGetTime()
{
struct timespec time;

#if defined(_POSIX_TIMERS) && _POSIX_TIMERS > 0
clock_gettime(CLOCK_REALTIME, &time);
#else
struct timeval tv;
gettimeofday(&tv, NULL);
time.tv_sec = tv.tv_sec;
time.tv_nsec = (tv.tv_usec * 1000);
#endif

return (time.tv_sec*1000L) + (time.tv_nsec / 1000000L);
}
#endif

Compilation results are as follows:

Code: [Select]

c++ -Wall -I/usr/X11R6/include/ -I/usr/local/include -I/opt/local/include -I. -I../BHMFile -I../ -g -c -L/usr/X11R6/lib -L/usr/local/lib -L/opt/local/lib -L../ -L. GL/GLTypes.h GL/GLUX.cpp GL/GLUXMath.cpp
ar rv libglux.a GLUX.o GLUXMath.o
ar: creating archive libglux.a
a - GLUX.o
a - GLUXMath.o
ranlib libglux.a
c++ -Wall -I/usr/X11R6/include/ -I/usr/local/include -I/opt/local/include -I. -I../BHMFile -I../ -g -o BHM3DSample -L/usr/X11R6/lib -L/usr/local/lib -L/opt/local/lib -L../ -L. State.h Main.cpp State.cpp Geometry/Object3D.cpp Geometry/Mesh.cpp Geometry/BHMObject.cpp Geometry/VertexDescriptor.cpp Geometry/Buffer.cpp Geometry/VBOMesh.cpp Material/Material.cpp Material/FFMaterial.cpp Material/ShaderMaterial.cpp Material/SkinnedMaterial.cpp Material/ASMMaterial.cpp Material/Texture.cpp Material/ASMShader.cpp Material/Shader.cpp Material/Program.cpp Material/SkinnedASMMaterial.cpp Anim/Interpolate.cpp Anim/Movable.cpp -lX11 -lglut -lGL -lGLU -lGLEW -lm -lfreeimage -lbhmfile -lglux
Geometry/Mesh.h: In constructor ‘Mesh::Mesh()’:
Geometry/Mesh.h:74: warning: ‘Mesh::pSkinVertexBuffer’ will be initialized after
Geometry/Mesh.h:70: warning:   ‘Material* Mesh::pMaterial’
Geometry/Mesh.cpp:13: warning:   when initialized here
Geometry/Mesh.cpp: In destructor ‘virtual Mesh::~Mesh()’:
Geometry/Mesh.cpp:33: warning: deleting ‘GLvoid*’ is undefined
Geometry/Mesh.cpp:39: warning: deleting ‘GLvoid*’ is undefined
Geometry/Mesh.cpp: In member function ‘void Mesh::SetSkinVertex(VertexDescriptor*, GLsizei, GLsizei)’:
Geometry/Mesh.cpp:104: warning: deleting ‘GLvoid*’ is undefined
Geometry/Mesh.cpp: In member function ‘virtual void Mesh::CreateVertexBuffer(void*, unsigned int)’:
Geometry/Mesh.cpp:121: warning: deleting ‘GLvoid*’ is undefined
Material/ShaderMaterial.cpp: In static member function ‘static Shader* ShaderMaterial::LoadShader(const char*, GLenum)’:
Material/ShaderMaterial.cpp:48: warning: format not a string literal and no format arguments
Material/ShaderMaterial.cpp:48: warning: format not a string literal and no format arguments
Material/ShaderMaterial.cpp: In member function ‘virtual void ShaderMaterial::LinkShaders()’:
Material/ShaderMaterial.cpp:72: warning: format not a string literal and no format arguments
Material/ShaderMaterial.cpp:72: warning: format not a string literal and no format arguments
Material/ASMMaterial.cpp: In static member function ‘static ASMShader* ASMMaterial::LoadShader(const char*, GLenum)’:
Material/ASMMaterial.cpp:45: warning: format not a string literal and no format arguments
Material/ASMMaterial.cpp:45: warning: format not a string literal and no format arguments

The end result is a file (BHM3DSample) that starts under X11 and loops a claw-like hand grasping a ball/sphere as it emerges from a table/plane below.

[Scali]

Okay, thanks for testing. Nothing too big, apparently. No problems with OpenGL/GLUT on OS X at least, that was something I was not sure of.
I can't quite recall why -lrt was added to the libraries... perhaps that's the one where clock_gettime() resides? I'd have to see if it can be removed altogether. Else I'd have to make some kind of Mac-specific Makefile options.
A bit surprising to find that it is not supported on OS X, seeing as I wrote and this on FreeBSD, and OS X is supposed to be built on BSD.
Oh well, I will just use your workaround in timeGetTime(), might fix the same problem on other systems as well.
I'll also add the /opt paths. I've seen those on other OSes as well, such as Solaris and HP-UX. Won't hurt to have them in there.

And yea, your description of the animation sounds about right. I forgot to mention it, but I put a video of it on YouTube: http://www.youtube.com/watch?v=VB3pElOLFzo
That's what it should look like.

I was thinking of including an OS X binary together with the Windows binaries, but on second thought... Not sure how feasible that is. It's going to require X11, GLEW and FreeImage to be installed, and whatever other dependencies those pull in. Not easy to just include a binary and some libs in the zip file and have it run out-of-the-box, like the Windows version, I suppose?

Edit: I can remove -lrt from FreeBSD at any rate. I think I added it for linux compatibility, but I don't have a linux system to test on. I might have to fire up a live CD at some point and see what gives.
README.txt is updated at any rate, as is Main.cpp, to fix the timeGetTime() issues.

Edit2: Fired up an Ubuntu live CD, doesn't seem to mind when I remove -lrt from the Makefile, so I've now updated the Makefile.
Also made a small fix to Mesh.cpp. It used memcpy() without explicitly importing string.h... For some reason the Ubuntu gcc complained (perhaps it suppresses previously included headers somehow, so it compiles as if each file is built in a separate session, rather than everything using the same header cache), yet neither OS X nor FreeBSD did, apparently. Ubuntu is right though, so I've fixed that now.

[SpooK]

I was thinking of including an OS X binary together with the Windows binaries, but on second thought... Not sure how feasible that is. It's going to require X11, GLEW and FreeImage to be installed, and whatever other dependencies those pull in. Not easy to just include a binary and some libs in the zip file and have it run out-of-the-box, like the Windows version, I suppose?

That's basically what an installer or basic downloadable application (.app file) is for. The .app file is actually a folder of other contents/resources/dependencies, but the Mac OS X GUI will treat opening a .app file as opening the underlying executable. This obviously creates a dependency on using XCode, so yes, I would avoid that scenario.

Basic static linking should be achievable as well. From what I have experienced, as long as you target a common architecture (e.g. Intel/32-bit), the binary will pretty much be universally supported from 10.5 up, without having to deal directly with fat/universal binaries or the lipo tool. There are exceptions that break this simplicity, of course, based on which dependencies are included.

As for X11, I think it comes installed by default, as of 10.6. However, I have worked with OpenGL/GLUT and Quartz (native) directly, so I know it is possible with minimal effort. I will take a look at your example again and let you know what the process/result is for that.