DirectXGraphics:
Vertex Blending
Author:
Jack Hoxley
Written: 31st July 2002
Contact: [EMail]
Download: Vertex
Skinning.zip (58kb)
Contents
of this lesson
1. What is Vertex Blending?
2. How to set up simple vertex blending
3. Extending this to use all 4 matrices
4. Taking it all the way: Indexed vertex blending
1. What is Vertex Blending?
I'm
going to have to assume that by this point in the
series you have a reasonable understanding of the
Direct3D pipeline  if not, this tutorial won't
make a huge amount of sense to you.
As
you should be aware, when you send vertices to the
device for rendering (Device.Draw* calls) they go
through a number of stages before final
rasterization. In particular, there is the
transformation engine  this is near to the start
of the process, and deals with multiplying every
vertex in the data stream by the world matrix
(view and projection can be done later). The world
matrix will typically rotate scale and translate a
given object.
As
it currently stands, the world matrix transforms
every vertex in exactly the same way  the whole
mesh is rotated, or the whole mesh is translated
across world space. Using current techniques you
cannot apply different transformations to
different parts of the same mesh without rendering
each part separately. This works well in most
cases, but it's lacking a more fluid, dynamic and
artistic touch.
Take
a look at the following screenshot (from the
sample):
Nothing hugely exciting about this
 a simple textured octagonal prism. Given current
methods, and without altering the rawgeometry how
would you make the above mesh render like this:
Interesting
eh? Even if you were allowed to alter the actual
geometry being rendered, the above shape would
still be a little tricky to create (unless you
used a software emulation of the D3D technique!).
To
sum up, vertex blending allows us to specify up to
4 matrices per rendered mesh, and to specify how
the transformations are distributed across the
geometry. The above image was generated using 2
matrices, with the second matrix focusing mostly
on the bottom of the prism (hence why most of the
distortion is visible there).
Using
vertex blending with bones is a very common
technique in commercial games. Take a humanoid
mesh, and set up a simple skeleton (you may need
to read up further on skeletal animation),
typically a set of vertices are attached to each
node in the skeleton. As the skeleton is animated
it moves the (rendered) mesh around. Using vertex
blending you can let any vertex be influenced by
more than one matrix (where each matrix represents
a bone for example). This creates much more
realistic character animation  much smoother (you
can even make muscles flex) and fluid.
2. How to set up simple vertex blending
Vertex
blending is not a hard effect to master, the most
difficult aspect is deciding how you are going to
use it. You will need to determine a pattern or
algorithm to set blending values for each vertex,
and then to decide what transformation to store in
each slot.
The
first step is to check for hardware support of
vertex blending matrices. You can only ever apply
a maximum of 4 matrices per vertex whatever the
hardware.
Dim DevCaps
As D3DCAPS8
D3D.GetDeviceCaps D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, DevCaps
If DevCaps.MaxVertexBlendMatrices < 2
Then
Unload Me
End
End If 



The important part
is the MaxVertexBlendMatrices value, this will
either be 0,1,2,3 or 4. 0 indicating no support
for vertex blending (the majority of recent
hardware has at least minimal support). For the
sample code presented with this tutorial, a simple
2 matrix blending method is used, hence the above
code only checks for 2 or more.
The next step is
to alter the vertex format  we must specify a
blending 'weight' for each vertex you want to
render. This is a floating point value indicating
how much each matrix is going to contribute to the
overall result. Typically this will be left in the
range of 0.0 to 1.0, with 0.0 being disabled/no
effect, and 1.0 being fully effective. 1.0 doesn't
mean that it will be the only matrix that affects
the current vertex, it just means that the current
matrix is not scaled. The general formula being:
P_{final}
= P_{vertex} * W_{1}*M_{1}
* W_{2}*M_{2} * W_{3}*M_{3}
* (1.0W_{1}W_{2}W_{3})*M_{4}
where P_{final}
being the final worldspace coordinate, P_{vertex}
being the original modelspace vertex coordinate.
W_{1}, W_{2} and W_{3} are
the three floatingpoint blending weights. M_{1},
M_{2}, M_{3} and M_{4} are
the four matrix slots.
The vertex data
type, for this simple sample, is altered in two
ways: The actual data structure and the the FVF
descriptor.
Const FVF_BLENDEDVERTEX2 = (D3DFVF_XYZB1
Or D3DFVF_NORMAL
Or D3DFVF_TEX1)
Private Type
BLENDEDVERTEX2
P As D3DVECTOR
B1 As Single
N As D3DVECTOR
T As D3DVECTOR2
End Type 



The data structure
has an additional single, this MUST come between
the position data and the normal data  or
Direct3D will get confused. The FVF has changed so
that the first flag is now D3DFVF_XYZB1  the B1
indicating that we have 1 weight defined in the
vertex.
This sample only
uses the first two matrix slots (of the four
available), that is the D3DTS_WORLD and
D3DTS_WORLD1 slots defined in the
Direct3DDevice8.SetTransform( ) function. If
we plug this into the general equation, we can now
see the following:
P_{final}
= P_{vertex} * B1*D3DTS_WORLD *
(1.0B1)*D3DTS_WORLD1
that is, as you
might recognize, the standard linear interpolation
equation. As B1 tends towards 1.0 the vertex will
be more affected by the D3DTS_WORLD matrix, as it
tends towards 0.0 it will be affected by the
D3DTS_WORLD1 matrix. Anything in the middle will
be a mix of both!
One slight
problem remains. When we load geometry from a .x
file, it probably won't know that it's going to be
used for vertex blending purposes  and as such,
won't have the necessary blending weight parameter
in the vertex structure. We must therefore add
this member to the mesh's vertex format. When
you've loaded the mesh as per normal, we use this
following short piece of code to alter the vertex
format:
If Not
(TubeMesh.GetFVF = FVF_BLENDEDVERTEX2)
Then
Set TubeMesh = TubeMesh.CloneMeshFVF(D3DXMESH_MANAGED, FVF_BLENDEDVERTEX2, D3DDevice)
End If 



Obviously, it's not
necessary to apply this process to
"handmade" geometry, as you should have
programmed the algorithm/code to use blending
weights from the beginning!
CloneMeshFVF( )
will have now allocated space for the blending
weight, BUT it won't have put any data in  it
will be left as 0. Therefore the next step is the
all important  setting blending weights:
Dim MeshVerts()
As BLENDEDVERTEX2
ReDim MeshVerts(TubeMesh.GetNumVertices)
As BLENDEDVERTEX2
D3DXMeshVertexBuffer8GetData TubeMesh, 0, TubeMesh.GetNumVertices * Len(MeshVerts(0)), 0, MeshVerts(0)
'we can now process each vertex
'4a. I want to know the maximum Y value
For I = 0
To TubeMesh.GetNumVertices
If MeshVerts(I).P.Y > TubeMesh_YHeight
Then TubeMesh_YHeight = MeshVerts(I).P.Y
Next I
'4b. we can now calculate the blending weights
For I = 0
To TubeMesh.GetNumVertices
MeshVerts(I).B1 = Sin(((MeshVerts(I).P.Y  0) / TubeMesh_YHeight) * (PI / 2))
Next I
D3DXMeshVertexBuffer8SetData TubeMesh, 0, TubeMesh.GetNumVertices * Len(MeshVerts(0)), 0, MeshVerts(0) 



There isn't anything
new to the above code, it's essentially the same
code used by the keyframe animation tutorials (original,
extended). As
already mentioned, the method you use to determine
blending weights can be as simple or as
complicated as you like. Bare in mind that if you
have to regularly alter the blending weights that
calculating it for a 1000 vertex model (as in this
sample) can slow the application down quite a bit.
By this point
we've done all the initialization necessary to use
vertex blending in our application. There is one
final render state to observe before rendering the
mesh:
'D3DVBF_DISABLE
 render using the
standard 1matrix
transformation
pipeline
'D3DVBF_0WEIGHTS  use
first matrix only, but
no weights specified
in each vertex
'D3DVBF_1WEIGHT
 one weight value per
vertex, use first two
matrices
'D3DVBF_2WEIGHTS  two
weight values per
vertex, use first
three matrices
'D3DVBF_3WEIGHTS 
three weight values
per vertex, use all
four matrices
D3DDevice.SetRenderState D3DRS_VERTEXBLEND, D3DVBF_1WEIGHT
'standard
code: tell D3D what
data is being rendered
next...
D3DDevice.SetVertexShader FVF_BLENDEDVERTEX2 



As a side note,
D3DVBF_0WEIGHTS has the same outputting result as
D3DVBF_DISABLE, but doesn't actually disable the
extended transformation pipeline  so probably
isn't as efficient. If you're not going to use
blending weights for a while, it would be best to
use D3DVBF_DISABLE. Also, in the SDK help files,
D3DVBF_1WEIGHT is spelt wrong! _1WEIGHT is
correct, unlike the _1WEIGHTS in the SDK help
files.
That is all of
the vertex blending samples at the simple level.
As it currently stands, whatever type of matrices
are defined in D3DTS_WORLD and D3DTS_WORLD1 will
affect all rendered geometry. The sample code
makes use of rotating through two axis (Y and Z)
to get the results shown in original screenshots.
The code for this is not very important, and you
can find it in the downloadable archive.
3.
Extending this to use all 4 matrices
The
previous section dealt with using only the first
two matrix slots. Just to cover all the bases I'm
going to give a quick demonstration of how you
could use all 4 matrices  with these two examples
you should be able to work out how to do 3matrix
blending. The downloadable sample code does not
include any of the source from this point onwards,
but you should see how it will easily fit in...
For
using all 4 matrices we must change the FVF and
the vertex declarator again:
'increase
_XYZB1 to _XYZB3
Const FVF_BLENDEDVERTEX4 = (D3DFVF_XYZB3
Or D3DFVF_NORMAL
Or D3DFVF_TEX1)
'add
B2 and B3
Private Type
BLENDEDVERTEX4
P As D3DVECTOR
B1 As Single
B2 As Single
B3 As Single
N As D3DVECTOR
T As D3DVECTOR2
End Type 



By using this data
format, you need to put valid matrices in all four
slots: D3DTS_WORLD, D3DTS_WORLD1, D3DTS_WORLD2,
D3DTS_WORLD3. The weighting of all 4 matrices is
calculated as follows:
D3DTS_WORLD = B1
D3DTS_WORLD1 = B2
D3DTS_WORLD2 = B3
D3DTS_WORLD3 = (1.0  B1  B2  B3)
which follows the format of the general equation
stated earlier.
When you are
ready to render the geometry you must use the
following two lines of code:
D3DDevice.SetRenderState D3DRS_VERTEXBLEND,
D3DVBF_3WEIGHTS
D3DDevice.SetVertexShader
FVF_BLENDEDVERTEX4 



upon doing this you
will be ready to go!
4.
Taking it all the way: Indexed vertex blending
Vertex
blending has been available through Direct3D for
several versions now, the methods discussed thus
far are nothing new. However, there is one new
addition to vertex blending for Direct3D8 
indexed vertex blending.
This
still limits you to a maximum of 4 matrices per
vertex, BUT it allows you to specify up to 256
matrices for each call. How does this work? well,
it's actually very clever...
If
you're an experienced DirectX programmer, you may
well remember using/learning how 8bit palletized
textures work. Each pixel stores an 8bit number 
this indexes to to a palette of 256 different
colors, but each color in the palette is defined
as a 24 bit RGB triplet. A similar idea applies
here.
For
each vertex we still have the blending weights
(1,2 or 3 of them), but we also add a 4byte
packed index value. This 32bit long is 4 indices
packed into one variable. The idea being that you
can store the 3 weights as appropriate, but then
use the indices to select which 4 matrices they
apply to  from a list of up to 256.
Say
for example you had a list of 35 matrices setup,
and for a triangle of 3 vertices you wanted 11
different matrices (based on each vertices
distance from a bone node for example). This
wouldn't be possible using the traditional method.
For
example:
V0 = matrices 1, 3, 5, 7 weights of 0.2, 0.3, 0.4
V1 = matrices 1, 3, 6, 9 weights of 0.2, 0.6, 0.1
V2 = matrices 2, 4, 6, 8 weights of 0.1, 0.05, 0.3
to
use indexed vertex blending we must (again) alter
the vertex structure and the FVF:
Const FVF_INDEXBLENDEDVERTEX4 = (D3DFVF_XYZB3
Or D3DFVF_LASTBETA_UBYTE4
Or
_
D3DFVF_NORMAL
Or D3DFVF_TEX1)
Private Type
INDEXBLENDEDVERTEX4
P As D3DVECTOR
B1 As Single
B2 As Single
B3 As Single
IdxList
As Long
N As D3DVECTOR
T As D3DVECTOR
End Type




A useful trick for
setting the four indices is to use the
D3DColorARGB( ) function. Because VB doesn't have
an unsigned 32bit integer variable, when bit
shifting/packing the 4 indices we have to mess
around with the signed bit. Using D3DColorARGB( )
avoids this problem. For example:
'//based
on the data from the
example above.
V(0).IdxList =
D3DColorARGB(1, 3, 5,
7)
V(1).IdxList =
D3DColorARGB(1, 3, 6,
9)
V(2).IdxList =
D3DColorARGB(2, 4, 6,
8) 



If you do use the
above technique, but choose not to use 4 matrices
then the color components refer to:
A=D3DTS_WORLD3
R=D3DTS_WORLD2
G=D3DTS_WORLD1
B=D3DTS_WORLD
Before you
actually use indexed vertex blending you need to
check hardware support. Only the more recent
hardware will support any significant number of
matrices  my ATI Radeon8500 supports 58 matrices
(slots 0 to 57), and not the full 256 possible.
Dim DevCaps
As D3DCAPS8
D3D.GetDeviceCaps D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, DevCaps
DevCaps.MaxVertexBlendMatrixIndex
'this
stores the number you
mustn't exceed. 



When you actually
want to use indexed vertex blending there are 3
things you must do:
first: create and
apply each matrix. You can't apply more matrices
to the device than specified in the above
enumeration. The following code will allow you to
set a matrix in slot "I":
D3DDevice.SetTransform D3DTS_WORLD + I, matTheMatrix
'More
specifically,
D3DTS_WORLD = 256,
therefore, slots 256
to 511 are reserved
for
'the matrix palette. 



secondly, you must
enable indexed vertex blending:
D3DDevice.SetRenderState D3DRS_INDEXVERTEXBLENDENABLE, 1 



lastly, you must set
the FVF:
D3DDevice.SetVertexShader FVF_INDEXBLENDEDVERTEX4 



Having completed all
these steps, all geometry rendered from now on
will be probably blended based on the matrix
palette indices and the weighting values.
Configuring
Direct3D to let you use indexed vertex blending is
not a complicated task  however, writing an
algorithm that selects the correct weights and
matrix indices is going to be the far harder
challenge. It may well be worth writing a plugin/tool
for a 3D renderer that allows an artist to specify
weighting values.
One solution I
have thought of  assuming you were using this
technique in conjunction with skeletal animation
(I would!):
assume a reasonable skeleton with 58 bones (the
same number of matrices my Radeon8500 supports).
We can generate a matrix for each bone node (as a
trivial example  shoulder>elbow>wrist
nodes) quite easily. Then, we loop through every
vertex in the mesh/skin, and calculate a list of
distances from the vertex to each bonenode. Sort
this list so we can popout the 4 closest bones
(this isn't a very smart method, but it would
work). We can then work out as a proportional
value how close each bone is to the vertex and
generate 3 blending weights. Algorithm complete.
In using a
solution like that the artist need only specify a
skeleton animation and a meshskin, and the
resulting animation would be a very convincing and
fluid system.
So, that is this
tutorial completed. Uses for vertex blending can
be huge  but I'll have to let you do your own
research there.
You can (and I
strongly advise you do) download the sample code
from the top of the page, or from the DirectX8
Downloads page.
Enjoy!
