I recently tried playing around with DirectX 11 tessellation feature in UDK. The results are rather interesting. I did run into some unusual problems when this feature is activated.
- When using static lighting there is no self shadowing from the displacement.
- Self shadowing occurs when using dynamic light
- When you switch to light only mode, tessellation disappears. This could be the reason why self shadowing with static lighting does not work because the light calculation is evaluated without tessellation. (I guessing here)
- Tessellation with displacement is degraded when the object is scaled. The image above shows how much degradation occurs by scaling the mesh to 3.
- Tessellation does not work if there are no smoothing groups, so meshes would need to be assigned a smoothing group in order for tessellation to work.
- Multiple smoothing groups would create seams and tearing as shown below