Firstly thanks for the positive comments on my last post – as any journaler knows it really does make a huge difference to morale to know people like what you do enough to bother posting some feedback. After the initial work on flora outlined last time, I decided to take a bit of a break from that and look at the landscape texturing. This was done using a 524288 x 524288 (512K^2) clipmap based texture but even at that resolution my 40 Km square island was looking a bit Nintendo-64 for my liking…the ground texture was blurry and lacking detail as can be seen in these screenshots:
(Note that I’ve turned off the grass effect here for clarity)
not only this but it took a long time to generate texture pages which made moving around and changing things a bit painfull. The original desire here had been that one big unique texture would allow maximum freedom as you can put anything anywhere without fear of tiling artifacts or running out of memory – the memory footprint is constant. Unfortunately though I came to the thinking that this benefit wasn’t actually worth the sacrifice in resolution and data generation time so I’ve been looking at replacing it with a more traditional tile based system. The art of course is to somehow do this without the nasty carpet-like tiling so often seen. I am still using a 128K x 128K clipmap based texture for the height and normal data but to this I have added blending weights for up to sixteen landscape tile textures which will represent the various grass, dirt, sand, rock, snow effects or whatever. Hopefully sixteen will be sufficient as even with this it’s taking 12 bytes per texel.
During rendering, the landscape vertex shader unpacks these weights from the clipmap texture and passes them on so the GPU will interpolate them at the per-pixel level and the pixel shader can use them to read from an array of landscape tile textures for all non-zero wieghts, blending the read texture colours together for the final result. All these are blended onto a base texture (currently grass) so in effect there are seventeen actual textures in use.
Below are some screenshots of the new system, both performance wise and memory wise it’s quite similar to the old one but it should be pretty obvious even to the casual observer that there is significantly more detail present – I’m working with a scale of 1 cm per texel for now:
The first version of this I had running used (512 x 512) landscape texture tiles which I had ‘borrowed’ from a released game, but even with my most careful fBm based blending between textures I found the tiling artifacts caused by the repeating texture detail most unsatisfactory. To improve this situation I employed the texture synthesis class I wrote to help produce unique texture detail for the old massive colour texture system I was replacing. Using this I produced new (2048 x 2048) tile textures using the original (512 x 512) ones as exemplars. Although this uses more memory (we’re talking about 40 MB for the texture array even with BC1 compression), for the main landscape rendering I am happy with this budget, especially as it’s not that different from the space used by the old clipmap system. The key benefits though are that it not only reduces the frequency of the tiling artifacts by a factor of four in each axis, it also reduces the visibility of the artifacts as the synthesised texture is by it’s nature more chaotic.
As an example, below is one of the rock texture that I’m working with. On the left is the (512 x 512) ‘borrowed’ original, in the middle is a (2048 x 2048) version produced by simply tiling the original while on the right is a (2048 x 2048) version produced by the texture synthesiser:
as you can see, the middle tiled image shows a strong repetitive pattern very obvious when used in situ. By comparison, the synthesised image on the right while not exactly the same as the exemplar exhibits pretty much the same features but in a completely chaotic manner eliminating the tiling artifacts. Another bonus is the synthesised texture maintains the tiling property of the original so can be tiled as needed across the landscape.
Synthesis is carried out as an offline process during program startup with the results being cached on disk for subsequent runs. A lengthy pre-processing pass is carried out on the examplar (takes about an hour currently for a 512×512 exemplar on a single core HT machine) which generates an exemplar reference file which can then be used with various randomness parameters to generate different larger synthesised texture in about five minutes. The algorithm was originally adapted from a SIGGRAPH paper where it ran on a GPU so it could be sped up significantly but I find it easier to understand and debug new algorithms on the CPU first.
As the blend weights for the different textures are stored with the height field samples, they can be changed every 30cm or so which I think will turn out to be adequately fine granularity for good looking landscapes, and the fact that all sixteen tiles can be blended with various weights should mean no hard edges where path meets grass for example.
The other less major change in this version is a new grass texture. The one shown here is from an old nVidia sample which I think is better than the crobby one I had knocked up myself – I’m still not 100% happy with it but at least it’s an improvement.
Anyway, I’m quite happy with the end result. The island now has much finer ground texture detail for about the same memory and processing footprint and while it’s lost slightly in flexibility it’s gained massively in visual appeal so I count that as a win.
(Originally posted Saturday, July 12, 2008)
It’s been a couple of weeks but in the odd hour I’ve managed to steal here and there I’ve been working on the Island’s flora rendering, namely trees and some grass. I’ve been working on a few systems, but the main ones are the leaves for the trees and some grass for on the ground, the leaves are working pretty well now but the grass while good from some viewpoints isn’t as good as I would like, still it’s better than nothing.
There are currently three different types of tree (Black Tupelo, Weeping Willow and CA Black Oak) but as they are defined using just a handful of parameters in an XML file I hope to add some more soon. Ultimately I want to do an interactive editor tool for tree creation to make the whole process easier and more fun. LODs are supported on the trees branches/trunks with most having between five and eight LODs depending on the tree definition while the leaves use a continuous LOD scheme so close up you get all the leaves rendering while at 800m or so distance you get a minimum of only 20% of the leaves. Individual leaves blend in quite smoothly as you approach the tree making the transitions fairly seamless. Both the trunk and the leaves sway in the wind using a noise texture and hash function in the shaders.
The grass is made up of about 8000 instanced blades created on the geometry shader and uses the height and normals from the clipmap to automatically follow the contours and light appropriately, they also blend in over distance and sway in the wind producing a reasonable effect. I think it would benefit massively from better grass textures than my art skills allow so I’ll need to go on the hunt for some I can steal I think
In line with the whole point of this project, I’ve been trying out some techniques I haven’t played around with before. First is instancing which could be done in DX9 but has received a bit of an overhaul in DX10 making it even more useful. Key to the DX10 improvements is that you can now access the index of the primitive and instance from within your shaders using system semantics, enabling some funky effects to be done entirely in shader code.
One example of this is the LOD system for the leaves on the trees – by passing in a number of leaves to render to the geometry shader it can use the index of each leaf’s primitive to set the alpha so leaves appear to fade in as they are added to the drawn list rather than popping.
It’s very cool that I can draw *all* the tree branches and trunks with just *one* draw call, all the leaves with another and all the grass blades with just one other – all that visual prettyness with just three draw calls. Smart.
Instancing is of course great for performance, offloading almost all the work from the CPU and really letting the GPU do it’s thing. I’m constantly amazed at just how fast these DX10 cards are (I’m using two different flavours of GeForce 8800 GTS, one 320MB and one 512MB) when given data like this. It was only when I added some metrics code that I realised I was throwing over two million tris at the thing and it just lapped it up. Amazing.
Another new feature for me is the use of alpha-as-coverage which is a technique where the alpha value of fragments is used as the coverage information for multi-sample anti-aliasing rather than for actual alpha blending itself. This gives you a sort of dithered alpha style effect which is inferior to actual blending but has the massive benefit of being draw order independent, so I can render my grass and my leaves in any order I like and still get fuzzy edges and fades.
The final new addition to the Island is a fairly simple Bloom effect to add a bit more atmosphere. This is an old-school low dynamic range effect that takes the brightest areas of the screen, blurs them out and then adds them back on top to make those areas glow. Originally I thought it would give nice sun glows through the trees and while it does add a little glow in this case, it’s far more noticeable on the more distant horizon where it produces a nice almost dream-sequence glowy haziness which reminds me a bit of the Oblivion landscapes. Not sure it’s the effect I’m after but it’s quite nice none the less.
Anyway, enough talk, check out the screenshots and videos. If you can spare a minute to let me know what you think that would be great too – comments as always are most welcome.
Taking a break from the wet stuff, I’ve decided to spend some time looking at adding some trees and other flora to the island. After a bit of Google research, I found what appears to be one of the definitive papers on this subject, “Creation and Rendering of Realistic Trees” by Jason Weber and Joseph Penn. I frequently struggle with such research papers as so often the authors give you just enough information to show how clever they are but not enough to duplicate or extend their work without having to pretty much do all the research again; so it was with a little trepidation that I approached this one, but fortunately Weber and Penn appear to have accomplished that rare feat of producing a paper that is clear and fairly easy to understand. I applaud them.
Having a procedural bark or leaf texture is a bit of a stretch at the moment so I have ‘borrowed’ some from the excellent SpeedTree package – hopefully they won’t mind as this isn’t a commercial project and eventually I hope to replace them with something that is actually procedurally generated or at least synthesised.
Anyway, I’ve uploaded some new screenshots to show the initial results of my efforts – three tree examples showing a CA Black Oak, Black Tupelo and Weeping Willow. The parameters are taken roughly from the Weber/Penn paper modified slightly as I am trying to adjust the algorithm as I go to make it more suitable for runtime use. I want to port some more tree types to give more variation and possible even experiment with pseudo-random trees to see what comes out. No need for the island to be too realistic now is there?
One thing that is immediately obvious is there are no leaves at the moment – now either I limit my island to being in the depths of winter the whole time or I need to address this. As you might expect, leaves are next on the list of things to look at. The plan for these is to use sprites rather than the geometric leaf implementation described in the paper simply because with the correct texture it should produce reasonable results with much simpler geometry. It also gives me a reason to delve into geometry shaders – an aspect of DirectX 10 that I haven’t yet looked at. The plan is to have a vertex buffer with positions, tints and direction vectors that is turned into quads by the geometry shader for rendering…doesn’t sound too hard but as I’ve never written a geometry shader before it will be a learning experience. I’ll put up some more images once I get it working.
Other bits and bobs that will need doing is some form of dynamic level of detail so the 30,000 triangle tree can be rendered with just a couple of hundred tris/lines at distance and ultimately just as a sprite – I want to have some fairly dense forests so effective LOD is essential. Some wind sway in the branches and leaves would also be good, some sort of noise or turbulence in the vertex shader will hopefully take care of this and should look great once the dynamic shadows go in. Finally some degree of pseudo self-shadowing with a dappling texture might be worth playing with.
Anyway, that’s enough for now, I’m going to go read up on geometry shaders.
(Originally Posted Tuesday, May 27, 2008)
All I’m going to talk about today is one little aspect of one part of the world but it’s one of those little features that you think you can just drop in but three weeks later realise there’s just a bit more to it than you thought.
I had the deep water effect in and working but due to the way it works all the waves move in the same general direction which I thought was a bit poor – wouldn’t it be great if there were waves that washed up against the shore a bit more like they do in real life? I had a brief play with distorting the primary waves to achieve this but the stretching and warping effects were truly horrible so I soon ditched that idea. Instead, I came up with storing a second channel on the water depth field texture that stores the distance from each point in the field to the closest shoreline. This distance could then be used in conjunction with an animated constant to produce waves in the water vertex and pixel shader that moved towards the shore regardless of the shore’s orientation.
Turns out that modifying the shaders to produce the waves was the easy part, the generation of the required “distance to shore” values on the other hand turned out to be a complete pain in the proverbial. It’s easy enough to search from a water point to find the closest non-water point representing the shore, and this does indeed work fine for large bodies of water, but if you simply do this on it’s own then places where the water area isn’t wide enough for the full wave width (such as in channels and smaller bays and coves) the waves spawn at less than maximum wave distance very noticibly popping into existance at the centre of the water before travelling towards the shore which looks really really stupid. Herein lies the rub.
I’ve spent my spare time over the last couple of weeks trying various methods to see how I could normalise the distance-to-shore values so even in narrow stretches of water the centre of the water was stored at maximum wave distance and points between there and the shore were scaled to fit so waves always spawned properly. Most of these methods involve creating a binary bitmap of the water area and applying various morphological image processing techniques such as erosion and shrinking to try to end up with a ’skeleton’ of points in the centre of each area of water. These points are then all stored at maximum wave distance and the points between them and the shore stored at scaled distances.
The theory sounds simple but in practice it’s not so much, neither the conventional erosion algorithm or the binary shrinking algorithm produced quite the type of shape skeleton I was after and in the end it’s a hybrid style approach that I found gave the best result. It’s not perfect but it’s pretty close to what I want and when in use there are only a couple of problem areas where the coastline is particularly “crinkly”. Anyway, it’s certainly good enough for now so I’m going to move on to other things.
I’ve captured a short video to show the effect in action – it’s a bit subtle and took much longer than I had hoped but I think it’s worthwhile and shore adaptive waves are not something I’ve seen in many water demos, the video also shows the dynamic time-of-day effect…I’ll probably post more details on how that works in the future if it’s of interest.
(Originally Written on Thursday, May 15, 2008)
As my current area of focus is the water effect, I thought a brief synopsis of how it works would be a good exercise; there are several inputs to the water effect, some read by just the pixel shader and some by both the vertex and pixel shader. There is no geometry shader involvement at the moment. The inputs are:
- Water Depth Texture: This is a 2048×2048 8bit texture that stores how far below the water plane each point on the terrain is. Zero indicates sea level (or above) while 255 indicates a depth of 200m or so below sealevel. Anything deeper than this is clamped as the effect does not change for deeper water – one experiment is to try reducing this to a 1024×1024 to see if the effect remains of acceptable quality.
- Surface Heightfield Texture: This is a 64×64 three channel floating point texture storing the height of the water waves along with X and Z peturbations to make them more ‘choppy’ – this texture is generated each frame on the CPU using a three pass fast fourier transform to move samples from a Phillips spectra from the frequency domain to the spacial one. The code is half based on the Tessendorf SIGGRAPH paper (which appears to be the defacto deep water rendering paper) but also pieced together from various demos I found on the net. I lack the maths skills to work entirely from the paper and besides, why re-invent the wheel. I would like a better understanding of the code though so I can optimise it further, possible even putting parts such as the FFT into a shader to get the GPU to help.
- Reflection Texture: Rendered at half the screen resolution, the reflection texture is rendered separately before the rest of the world using alternative shaders that mirror the geometry across the water plane. The decision to use separate shaders rather than building the mirror into the transform was so that cheaper versions of the shaders could be used for speed. Once it’s rendered, it’s then blurred into a separate 1/4 x 1/8 screen size texture using a pixel shader to make the reflections less pixelated and generally look better.
- Refraction Texture: Rather than render a separate refraction texture, a copy of the main back buffer is made after rendering the scene but before the water is rendered. Although this includes geometry above sea level which it shouldn’t really, in practice the effect works fine none the less. If this did become a problem it may be possible to employ the stencil buffer to mask off rendered pixels above the water line (I believe Crysis does this)
- Foam Texture: A 512×512 8bit greyscale texture generated procedurally during program startup using fractal brownian motion, the foam texture is blended in by the pixel shader to provide wave crest highlights and shore foam where the water meets the land.
- Z-Depth Texture: A screen-sized single channel floating point texture used as a secondary render target during rendering of the rest of the scene to store screen Z depth values. Although DX10 can use the Z buffer as an input, you cannot use it as both an input and an output at the same time limiting it’s usefulness in this mode. Instead I opted for a more conventional approach rendering a separate depth image that I felt could be of use for other effects later (depth of field? SSAO?).
The vertex shader reads from the surface heightfield texture to deform the vertices of a fairly simple radial mesh grid centred around the camera. The spacing of the grid vertices increases as the mesh extends away from the camera giving a more uniform triangle size when perspective is applied during rendering. The vertex shader also reads from the water depth texture to scale the wave amplitude so the closer to shore you are the smaller the waves. A potential experiment here is to also scale the frequency of the waves although this might introduce too many warping artifacts into the texture co-ordinates.
Most of the work is done in the fairly meaty pixel shader however, which also samples the 64×64 water surface texture but this time takes three samples of two octaves each so it can work out an approximate surface normal at that point. It also reads from the water depth texture to generate a basic water colour and to modulate alpha and uses the fresnel term to blend between the reflection and refraction textures. The sampling method is basically the same for both of these; for reflection the surface normal is projected onto a plane slightly above the sea and this displaced point projected back into screen space to get the point to read from the reflection texture while for refraction the plane is slightly below the sea and it’s the refraction ray (using a air->water refractive index of 1.33) that is projected.
The foam texture is blended in to the wave tops to help simulate the effect of breaking water and is also blended in where the water depth is shallow to produce shoals; the intensity read from the foam texture is animated to ping-pong back and forwards between zero and one to add more movement to the water.
The Z-depth texture is used for two purposes here, foam texture is blended in again but at a higher frequency where the water Z-depth is close to the previously rendered Z-depth to produce foam around the shoreline and it’s also used to vary the alpha of the water where it meets other geometry producing a far more satisfying soft edge to the water (similar to depth sprites) rather than the normal hard Z-buffer intersection. These are especially satisfying as they operate purely on the Z-depths so occur wherever anything penetrates the water not just at the shore so boats, floating debris, rocks or whatever all get foam and nice soft edges.
Finally diffuse lighting is applied and everything munged together to get the final result. One final ‘bodge’ is an artificial strengthening of the fresnel reflection in a cone pointing towards the sun; this is done to produce a reflection of the sun disc in the water replacing the need for normal specular – I found that using normal specular resulted in unsightly artifacts when the sun was low in the horizon that made it clear the specular was treating the sun as a point instead of an area. HDR would make it even better simulating glints on the water in the sun cone.
So now about the problems: The effect is best at the moment for deep water such as the ocean around the island, but it doesn’t work quite so well for shallow smaller areas such as ponds and smaller lakes as the waves just don’t look right for small water bodies. Either I need to play with the shaders and parameters to try to make them adjust their behaviour in these cases or I’m going to have to identify smaller bodies and render them separately using a different effect.
Other problems at the moment include a deterioration in effect quality when very close to the water plane and a complete breakdown when you go underwater – the latter at least is simply because I’ve not done any handling code for this; the plan is to bring the far clip plane in, increase fog, flip the reflection and refraction texture rendering to take account of being under the water and add some full screen ‘wibble’ effect. Another missing piece is the lack of caustics on the sea floor underwater – these could be done with an animated texture blended in based on water depth – beams of light underwater would also be cool.
Overall though I’m pretty happy with the effect so far, especially as it still adheres to my goal of producing entirely procedural effects – there are no textures loaded from disc, everything is generated.
(Originally Written on Wednesday, May 14, 2008)
I’ve started this journal to diarise some of the projects that I work on in my spare time to amuse myself and gain exposure to and experience of various game related technologies. Although I spend my days working as a software developer/architect, my day job rarely coincides with what I would regard as more ‘fun’ technologies so I have side projects I do in my own time where I can work on whatever I like without the pressure of actual deliverables.
So what is the first project? Well the first one I am floating here is ‘Isis’ who, according to the Book of the Dead is “She who gives birth to heaven and earth” – an apt title I feel for what I hope will grow into a fully functioning procedurally generated world…or in the spirit of walking before running at least a single Island.
The main technologies that I have experimented with in this project so far are:
- DirectX 10 – I had never used before but now have at least a working understanding of. Microsoft redesigned the API significantly between DX9 and this version to make it a better fit for modern hardware (a move I agree with entirely) but it does mean a little re-learning is involved in making it work. Relying on DX10 does mean that Isis will only run on a capable graphics card and only on Vista, but as it’s purely a research project and my machines both at home and at work meet this requirement it’s not a problem.
- Clip Maps – As an experiment I’ve decided to store both the height field and the primary diffuse texture for the landscape of my world using single very large textures and employ the ‘ClipMap’ algorithm to enable these large textures to be rendered. Clipmaps store a stack of textures arranged concentrically around a point where each level of the stack stores an area twice the size of the previous one but at half the detail. During rendering the pixel shader calculates which clip map level to read from for the pixel being rendered using a combination of distance and screen space texture derivatives. At the moment the height field is a 128K^2 texture and the diffuse a 512K^2 one.
- Noise and Fractals – Although I’ve worked with noise functions before, this project places more demands on fractals than any I’ve undertaken previously. I’m using both absolute and differential 3D noise functions, turbulence functions and fractal brownian motion (fBm) all of which should be useful. fBm in particular has many uses for natural phenomenon simulation such as clouds.
- Atmospheric Scattering – The underlying sky colour in Isis is based in part on the “Atmospheric Scattering” work presented by Sean O’Neil in the book “GPU Gems 2″. Part of the sky vertex and pixel shader code is adapted from his original source complemented by further processing of my own to add some simple clouds and horizon haze.
- Deep Water – An implementation of the apparently defacto Tessendorf paper, I use fast fourier transforms to drive the wave simulation to simulate deep water waves and combine in reflection, refraction, fresnel, depth attenuation, crest and shore foam to produce the final effect. The water effect is currently my focal area as I am working on adapting it into an effect that can scale smoothly from large deep water areas to smaller localised lakes and pools.
Anyway enough rambling, here are some screenshots from the current version so you can see what I’m on about. It’s a long way from where I ultimately want to be but I’m reasonably happy with the progress so far.
There are of course many directions I would like to progress Isis further, but some of the ones most interesting to me at the moment are:
- Rivers – Flowing water ranging from slow moving wide rivers to turbulent streams
- World management – not a graphical effect but a data structure whos efficiency will be absolutely key to achieving any decent level of performance as things are added to the world. Some form of octree most likely.
- Trees – Adding trees to the landscape will both increase the level of realism and add visual interest but also add a sense of scale to the hills and valleys as the inherent fractal nature of the height field makes it otherwise hard to tell actual size. Rather than use pre-built models I want to use procedural methods to create a number of tree types during startup which are then used to populate the world according to heuristics. The vertex shader will dynamically modify the vertices at render time to simulate wind.
- Volumetric Clouds – Although there is a base layer of clouds in the sky currently implemented as a simple texture, the plan is to add volumetric clouds rendered most probably via a system of lit sprites or per-frame generated imposters. Being able to fly through the clouds will be a requirement of this system.
- Fauna – animals, fish, birds, something to help bring the place to life.
- Civilisation – To truly add interest to the environment the plan is to add a ‘civilisation’ system that generates a whole network of communities and social infrastructure within and around the natural features of the landscape.
- Buildings – People of course need a place to live, so creating hamlets, villages, towns and even cities is desired. Initially these will be constructed from pre-loaded models but ultimately it would be interesting to see how far down this route I can get with just procedural methods. Sensible placement and sizing of the settlements is one of the challenges.
- Tracks and Roads – To move about on, tracks, paths and roads that connect settlements are desired.
- Bridges and Tunnels – Where tracks and roads cross valleys and rivers bridges and tunnels should be created. The burrowing of tunnels into the hillside is a significant challenge for the rendering and collision system.
- Weather – rain, snow, storms, the usual things.
- Vehicles – to move around the world in rather than just flying. A ‘walk’ mode would be a good start to help demonstrate how large the place is.
- HDR and Lighting – bloom, tone mapping, motion blur and other post processing effects to give the visuals some of that professional polish
- Better Texture Management – a more flexible ‘megatexture’ on-demand texture upload system rather than the limited clipmap algorithm
Okay enough for now, I’ll post more as I go.