One of the biggest problems I have when writing games is in creating or acquiring appropriate 3D models to use. In exploring some open source projects for a ship strategy game I’ve been working on, I found this site: Danger from the Deep. I was, in particular, very impressed by the visuals in this game and the models that they use are available independently under a Creative Commons licence. I grabbed a copy of the these models from their download page and took a look at them.

Some of the older models were simple 3ds files, but in amongst these are some *really nice* models in a custom format created by Luis Barrancos, Thorsten Jorden, Martin Albertstadt & Marco Sarolo. In the \data\objects\ships\corvettes\FlowerClass directory, for example, is FlowerCorvette_RN.ddxml. As the file suffix would suggest, their file format is a nicely structured xml file. Since I like ships :) I thought that this would be a good opportunity to write a custom content importer to bring these ddxml files into my own XNA project.

Creating a Project

First we need to create a new project, which will become the viewer for the model that will result from our content importer.

Creating a Content Pipeline Extension Project

Next we need to right click the solution and a add a new project to the solution:

Select the Content Pipeline Extension Library template, give it a sensible name and click OK to add it to our solution.

By default the resulting project will have a ContentProcessor class already created. For the purposes of this Demo we don’t need this. so you can delete it. Instead we need to add a new ContentImporter class. Once again we can use a template to do this. Right-click the content importer project and select Add->New Item…

Give the importer a sensible name – in the Demo project provided I called this class DDXMLImporter.cs.

The DDXML Structure

If you take a look at the DDXML model file from Danger from the Deep, you’ll see it is a well formed XML file with nodes for material & mesh definitions. The material is principally defined by paths to texture images. For the purposes of this demo, we’re just going to pick up the diffuse file. The mesh is defined by a list of the X, Y, Z positions in all the vertices in a vertices node. The actual triangles are then defined by indices into this vertex list for each corner of the triangle, plus a list of texture coordinates & normals – each stored under an XML node. The snippet below gives you a flavour of it.

<dftd-model version="1.0">
  <material name="FlowerMatV1" id="0">
    <diffuse color="1 1 1" />
    <specular color="1 1 1" />
    <shininess exponent="65" />
        <map type="diffuse" filename="FlowerCorvette_RN_1942_WA_color.jpg" uscal="1" vscal="1" uoffset="0" voffset="0" angle="0" >
        <skin filename="flowercorvette_rnavy_camo2.jpg" layout="north_atlantic_royal_navy"/>
        <skin filename="flowercorvette_rnavy_camo3.jpg" layout="mediteranean_sea_all_navies"/>
        <skin filename="flowercorvette_rnavy_camo4.jpg" layout="canadian_northatlantic"/>
        <skin filename="flowercorvette_rnavy_camo5.jpg" layout="canadian_coast_na"/>
    </map>
    <map type="normal" filename="FlowerCorvette_RN_1942_WA_bump.jpg" uscal="1" vscal="1" uoffset="0" voffset="0" angle="0" >
        <skin filename="flowercorvette_rnavy_bump2.jpg" layout="canadian_coast_na"/>
    </map>
    <map type="specular" filename="FlowerCorvette_RN_1942_WA_spec.jpg" uscal="1" vscal="1" uoffset="0" voffset="0" angle="0" >
        <skin filename="flowercorvette_rnavy_spec2.jpg" layout="canadian_northatlantic"/>
        <skin filename="flowercorvette_rnavy_spec2.jpg" layout="canadian_coast_na"/>
    </map>
  </material>
  <mesh name="FlowerCorvetteV1Body" id="0" material="0">
    <vertices nr="7234">2.535352 27.580421 -3.336818 2.535352 27.580421 -3.336818 2.535352 27.580421 -3.336818 -2.535351 27.580421 -3.336818 -2.535351 27.580421 -3.336818 -2.535351 27.580421 -3.336818 -2.535351 27.580421 -3.336818 -4.302600 22.158279 -3.575129 -4.302600 22.158279 -3.575129 -4.302600 22.158279 -3.575129 -4.302600 22.158279 -3.575129 4.302601 22.158279 -3.575129 4.302601 22.158279 -3.575129 4.302601 22.158279 -3.575129 5.031166 18.364771 -3.787620 5.031166 18.364771 -3.787620 

Although I’m going to dive right into the importer in-situ within my project, I should point out that debugging the parsing code in this way is a bit a pain. When I wrote this initially, I actually honed the parsing code independently in simple console app first.

Writing the Importer

OK so if we take a look at the class stub that the template has created for us, we can see straight away a couple of quick edits. The first thing we need to do is let Game Studio know what we are wanting to end up with from our importer. In this example we want to end up with a data structure we can push into the standard model processor (custom model processing is something we can return to in a future posting) later in the content pipeline. The data structure we want to end up with is a NodeContent class so we can edit the initial using statement thus:

// TODO: replace this with the type you want to import.
using TImport = Microsoft.Xna.Framework.Content.Pipeline.Graphics.NodeContent;

We can also add some sensible descriptions:

namespace ContentImporter
{
    [ContentImporter(".ddxml", DisplayName = "DDXML - Danger from the Deep Model", DefaultProcessor = "ModelProcessor")]
    public class DDXMLImporter : ContentImporter<TImport>
    {

Now we can add the private members we are going to need:

// The root NodeContent of our model
private NodeContent m_rootNode;

// The current mesh being constructed
private MeshBuilder m_meshBuilder;

// Identity of current MTL file for reporting errors against
private ContentIdentity m_mtlFileIdentity;

// Current material being constructed
private BasicMaterialContent m_materialContent;

// All vertex data in the file
private List<Vector3> m_positions;
private List<Vector2> m_texCoords;
private List<Vector3> m_normals;
private List<int> m_indices;

// Mapping from mesh positions to the complete list of
// positions for the current mesh
private int[] m_positionMap;

// Indices of vertex channels for the current mesh
private int m_textureCoordinateDataIndex;
private int m_normalDataIndex;

private XmlReader m_reader;

The crucial ones here are the Meshbuilder and root NodeContent classes (provided by the XNA framework) plus the lists of positions, texture coordinates, normals that we will fill these classes with as we read the file. The mesh is an indexed triangle mesh (the vertex definitions are re-used in multiple triangle descriptions) so we will also need a list of indices into our vertex list and an array of mappings to the positions in the MeshBuilder later on.

Because the file is an XML format, I can use the standard XmlReader class and I won’t dwell too much on the actual parsing. It’s not really the interesting bit of this discussion and I probably haven’t implemented it anywhere near optimally!

The entry point for Content Importers is, unsurprisingly, the Import method:

public override TImport Import(string filename, ContentImporterContext context)
{
    m_rootNode = new NodeContent();
    m_meshBuilder = null;
    m_rootNode.Identity = new ContentIdentity(filename);

    m_reader = XmlReader.Create(filename, null);
    ParseXMLNodes(m_reader, "");

    return m_rootNode;
}

He we create our actual instance of the NodeContent class and give it an identity for debugging. The real work then begins in the ParseXMLNodes method which divvies up nodes & node content to the appropriate methods as it works through the file. For the purposes of this discussion the initially interesting method is StartMesh, which is called when the first Mesh node is found in the file:

private void StartMesh(string name)
{
    m_meshBuilder = MeshBuilder.StartMesh(name);
    m_positions = new List<Vector3>();
    m_texCoords = new List<Vector2>();
    m_normals = new List<Vector3>();
    m_indices = new List<int>();

    // Obj files need their winding orders swapped
    m_meshBuilder.SwapWindingOrder = true;

    // Add additional vertex channels for texture coordinates and normals
    m_textureCoordinateDataIndex = m_meshBuilder.CreateVertexChannel<Vector2>(
        VertexChannelNames.TextureCoordinate(0));
    m_normalDataIndex =
        m_meshBuilder.CreateVertexChannel<Vector3>(VertexChannelNames.Normal());
}

This initialises the main repositories for the incoming data. Like Obj files, I found by trial & error that I need to reverse the winding order of these files to get the triangles facing the right way. This problem can manifest itself visually as an excessively dark model, missing triangles or a sort of ‘inside-out’ appearance when you twiddle - depending on you rendering settings.

In our instance of the MeshBuilder object we also add the channels we are going to need for the texture coordinates & normals that form part of our vertex definition.

Once our mesh is underway, we can expect to start parsing the contents of the vertices (which are positions as far as the importer is concerned), indices, texture coordinates & normals. In the demo project I wrote a little helper class to split the text content of these nodes into vertex2 or vertex3’s and thus fill up our member lists, like so:

void ParseNodeContent(XmlReader reader, string node_name)
        {
            switch (node_name)
            {      
                case "vertices":
                    ContentPipelineHelper.ParseVector3List(ref m_positions, reader.Value);
                    break;

                case "indices":
                    ContentPipelineHelper.ParseIntList(ref m_indices, reader.Value);
                    break;

                case "texcoords":
                    ContentPipelineHelper.ParseVector2List(ref m_texCoords, reader.Value);
                    break;

                case "normals":
                    ContentPipelineHelper.ParseVector3List(ref m_normals, reader.Value);
                    break;
            }
        }

Extracting the filename of the diffuse colour map is handled in the parseMap method where we fill in our mtlFileIdentity object:

private void ParseMap(XmlReader reader)
{
    if (reader.GetAttribute("type") == "diffuse")
    {
        string filename = reader.GetAttribute("filename");
        m_mtlFileIdentity = new ContentIdentity(filename);
        m_materialContent.Identity = m_mtlFileIdentity;
        m_materialContent.Texture = new ExternalReference<TextureContent>(filename, m_mtlFileIdentity);
    }
}

Finally, when we get to the end of our mesh node we need to finish this mesh off ready to start the next one:

private void FinishMesh()
{
    m_positionMap = new int[m_positions.Count];
    for (int i = 0; i < m_positions.Count; i++)
    {
        // positionsMap redirects from the original positions in the order
        // they were read from file to indices returned from CreatePosition
        m_positionMap[i] = m_meshBuilder.CreatePosition(m_positions[i]);
    }

    // Set the last material we parsed - this should really come from the ID s
    m_meshBuilder.SetMaterial(m_materialContent);
    // Create the actual triangles in the mesh
    foreach (int index in m_indices)
    {
        // Set channel data for normal for the following vertex.
        // This must be done before calling AddTriangleVertex
        m_meshBuilder.SetVertexChannelData(m_textureCoordinateDataIndex, m_texCoords[index]);
        m_meshBuilder.SetVertexChannelData(m_normalDataIndex, m_normals[index]);
        m_meshBuilder.AddTriangleVertex(m_positionMap[index]);
    }

    MeshContent meshContent = m_meshBuilder.FinishMesh();
    m_rootNode.Children.Add(meshContent);
    m_meshBuilder = null;
}

This is perhaps the most complicated method.  Initially we need to fill the m_positionMap array with the index of each positions in the MeshBuilder as we add the positions to it. When we create the actual triangles in a minute, we’re going to need this array to correctly the reference the stored positions. Then we set the material for the mesh and store the actual triangles according to the index list. In each case we also set the appropriate texture coordinate & normal. Once this process is done we can finalise the mesh with FinishMesh and add the returned MeshContent object to our NodeContent object.

That is it. Hopefully I’ve been able to keep the parsing code to a minimum so that you can see the general structure of an importer fairly clearly.

Connecting the Importer to our Game Project

In order for our viewer to use the importer we have created, we must add a reference to the importer project under the Content->References node of the viewer project tree within the solution.

Under the Projects tab of the Add Reference dialog, select the content importer project we created above.

Adding the DDXML models to our project

Now we’re ready to add the actual models to our viewer project.

One of the simplest ways to add files to your solution is just to drag them using Explorer. Drag the FlowerCorvette_RN.ddxml file from \dangerdeep-data-0.3.0\data\objects\ships\corvettes\FlowerClass\ into the content node of the ContentImporterDemo or viewer project in your solution.

The DDXML file also defines the texture image that will be used as the diffuse colour texture map for the model (actually they define several, but for the purposes of this demo we’re just going to use the first one). We need to drag the FlowerCorvette_RN_1942_WA_color.jpg from the same folder into the content node to make a copy of it there too. Although this file will be needed indirectly when the content is compiled, we do not need the project to do anything with it directly, so once it has been copied into the correct location you can right-click and select Exclude From Project.

Setting the Correct Importer & Processor for a Model

With the DDXML file added to the content, we need to make sure our new DDXML Content Importer is used (you may need to first build your importer project to ensure that our new importer appears in the drop down list of options). The Content Processor will remain the standard Model – XNA Framework option provided by default. Both of these can be set using the Properties dialog. If this is not open, right-click the file in the project tree and select Properties….

Putting It All Together

In the demo project provided, I’ve used some components (detailed in my earlier posts) to provide *very* basic camera and twiddling controls (arrow keys - Z,X to zoom).

Example Models Rendered using XNA

Other Bits & Bobs

I’ve left this project in a pretty raw state because I didn’t want to distract attention from the core topic here, but I’m sure that many of you will spot lots of opportunities to improve this project. The immediately glaring issues are that the file format provides for multiple diffuse colour maps and a normal map – both of which are ignored in my implementation. In addition some of you may notice the main gun turret & propellers are missing from my import of the Corvette. These extra elements can be animated independently and so are structure in a slightly more sophisticated way than my current importer can handle – I leave this as an exercise for the reader to look at ;) please feel free to let us all know of your improvements or corrections to the code provided here.

Finally – thanks to the Danger from the Deep guys for providing such inspiring models. Please take a look at their game – it is an amazing piece of work.

Files

The demo project described in this post can be downloaded here.