Advanced Graphics 2

This project was created as part of my Advanced Graphics and Real-Time Rendering module in semester 2 of my third year. The task was to create a framework using the DirectX graphics API, which would demonstrate a range of even more advanced graphical techniques, like terrain rendering with appropriate real-time triangle subdivision to achieve LODS, as well as voxel rendering, and model animations. There was also a optional task to create a bezier curve system to allow for the creation of splines with the option for modifying their control points at runtime.

What Did I Learn?

While developing the terrain system, I learned a lot about the different types of terrain rendering techniques, such as diamond square, fault formations, and perlin noise to achieve the desired midpoint displacement. I also learned about the different types of LOD techniques, and how proper triangle subdivision is achieved through the use of hull and domain shaders, with the control point and patch constant phases respectively.

The purpose of the control point phase in the hull shader is to set up the control points for the patch constant phase in the domain shader, which is where the actual subdivision takes place. The domain shader is also where the tessellation factor is set, which is used to determine the number of subdivisions that will take place. The tessellation factor is determined by the distance between the camera and the centre of the patch, which is then used to determine the number of subdivisions that will take place. After this, the patch constant phase sets the edge and inside tessellation factors for each patch, where a higher tessellation factor results is more triangle subdivision.

I then learned about the purpose of the domain shader and that it returns a position for each vertex by first providing barycentric coordinates for each vertex, which are then used to interpolate the position of the vertex. It also returns the normal for each vertex, which is then used to calculate the lighting, and texture coordinates, which are then used to sample the texture for the vertex which are provided as UV coordinates. The purpose of the barycentric coordinates is to act as weights for each vertex so that they are positioned correctly within the patch. These vertex positions are then passed to the vertex shader, where they are projected to clip space, and then rasterized to the screen.

// Domain shader - tessellate terrain using barycentric coordinates
[domain( "quad" )]
DOMAIN_OUT DS( PatchTess patchTess, float2 uv : SV_DomainLocation, const OutputPatch quad )
{
    DOMAIN_OUT output;

    // Bilinear interpolation
    output.PosW = lerp(
		lerp( quad[0].PosW, quad[1].PosW, uv.x ),
		lerp( quad[2].PosW, quad[3].PosW, uv.x ), uv.y );

    output.Tex = lerp(
		lerp( quad[0].Tex, quad[1].Tex, uv.x ),
		lerp( quad[2].Tex, quad[3].Tex, uv.x ), uv.y );

    // Tile layer textures over terrain
    output.TiledTex = output.Tex * 50.0f;

    // Displacement mapping
    output.PosW.y = texHeightMap.SampleLevel( smpHeightMap, output.Tex, 0 ).r;

    // Project to homogeneous clip space
    output.PosH = mul( float4( output.PosW, 1.0f ), World );
    output.PosH = mul( output.PosH, View );
    output.PosH = mul( output.PosH, Projection );

    return output;
}

I also spent time implementing various ImGui extensions like ImGuizmo, and ImGuiFileDialog. ImGuizmo is a gizmo system that allows for the manipulation of objects in the scene, and ImGuiFileDialog is a file dialog system that allows for the selection of files and directories. I learned how both of these systems worked and implemented them into the application. ImGuizmo was used to spawn gizmos at the positions of entities in the scene to allow for simple translation, rotation, and scaling. The extension also provides the options of creating a view cube which modifies the view matrix of the main camera to target the position of the active gizmo in the scene.

The ImGuiFileDialog extension proved very useful for allowing the user to change the diffuse, normal, and height map textures for the objects in the scene including the saving terrain data back to the appropriate JSON files.

// ImGuizmo - spawn gizmo at the position of the selected entity
// Decompose/recompose matrix
float* worldPtr = (float*)&worldMat;
float matrixTranslation[3], matrixRotation[3], matrixScale[3];
ImGuizmo::DecomposeMatrixToComponents( worldPtr, matrixTranslation, matrixRotation, matrixScale );
ImGuizmo::RecomposeMatrixFromComponents( matrixTranslation, matrixRotation, matrixScale, worldPtr );

// Handle manipulation of the gizmo
XMFLOAT4X4 view = pCamera->GetView();
float* viewPtr = (float*)&view;
XMFLOAT4X4 projection = pCamera->GetProjection();
float* projectionPtr = (float*)&projection;

if ( ImGuizmo::Manipulate( viewPtr, projectionPtr, imguizmoData.CurrentGizmoOperation, imguizmoData.CurrentGizmoMode, worldPtr, NULL, imguizmoData.UseSnap ? &imguizmoData.SnapAmount.x : NULL ) )
{
    // Update object parameters
    XMFLOAT3 pos = XMFLOAT3( matrixTranslation[0], matrixTranslation[1], matrixTranslation[2] );
    pTransform->SetPosition( pos );

    XMFLOAT3 rot = XMFLOAT3( matrixRotation[0], matrixRotation[1], matrixRotation[2] );
    pTransform->SetRotation( rot );

    XMFLOAT3 scale = XMFLOAT3( matrixScale[0], matrixScale[1], matrixScale[2] );
    pTransform->SetScale( scale );
}

Along with this, I learned about the various skeletal animations and the different types of interpolation techniques, such as linear, cosine, and cubic, and how they are used to interpolate between keyframes to achieve smooth animations. I also learned about the different types of skeletal animation techniques, such as forward kinematics, inverse kinematics, and dual quaternion skinning, and how they are used to animate the bones of a skeleton. Further, I learned about the different types of skeletal animation blending techniques, such as additive, and how they are used to blend between different animations to achieve more complex animations.

Model Animation	Terrain Generation

ImGuizmo Manipulator	Voxel Terrain

Future Additions

In the future, I would like to add more features to the application, such as the ability to create new skeletal animations, and the ability to create new terrain patches. Along with this, I would like to add the ability to have the player walk along the terrain, which would be an example of inverse kinematics, and have the camera follow from a fixed distance behind the player, and then perhaps add the ability to jump and have the camera continue to follow.

I would like to spend more time on making the terrain procedural rather than generating large grids of fixed sizes and needing to seed new values along with rebuilding the terrain to add new patches. Other than this, I think it would be interesting to work on a shrub and tree system that would allow for the creation of trees and shrubs that would be placed on the terrain and would sway in the wind.

Feel free to drop a comment below.