I started working on an ice shader recently and was trying to replicate a hard edged effect purely in the vertex shader, with a soft edged mesh. My idea that was that this would be good for two reasons:
- We could use the mesh that the ice was growing on and duplicate it, saving memory as we have one less unique mesh and saving modelling time.
- We could bypass not being able to have vertex manipulation in the shader as hard edges would break apart (being made up of additional non-connected vertices).
Hard Edges
To create hard edges using the vertex shader I used the ddx and ddy functions, which are partial difference derrivatives. What these do is look at a 2×2 block of pixels in screenspace and compare the current pixel the adjacent one (+ 1 in x or y depending on the function), giving us an idea of how different they are to one another. This is often used for things like calculating mip map usage based on screenspace covereage. For more info on derivative functions, see a clockwork berry.
What I’ve done here is use the vertice’s world position as a compartior. For each function, where the world space position in the axis changes, we output a different figure. To combine these two, we cross them, and then normalize in order to give us a value that equates to the normal of the vertex. This gives us a hard edged normal for each face.
float3 x = ddx(i.worldPos);
float3 y = ddy(i.worldPos);
float3 normal = normalize(cross(x, y));

Luminescence
I wanted to remove colour from the above so that I could use it as lighting for my shader.
To convert from colour values to grey, I took the dot product of the normal and (0.22, 0.707, 0.071) and then saturated it.

float3 lumi = dot(normal, float3(0.22, 0.707, 0.071));
float4 lighting = saturate(float4(lumi, 1));
return col * lighting * _Color;

Final result with this technique below. It was very interesting and I may use the above techniques in future, but I’m going to go in a different direction with this shader. I want a more triangular, fractal look and this just looks far too uniform.
Also, as I’d need to duplicate the mesh and render twice (different materials = different draw calls), the very small memory saving of duplicating the mesh and manipulating the verts just isn’t worth it compared to modelling a separate ice mesh that I have direct control over. The hard edges could be achieved through texture mapping.
