In Milan on Wednesday 17

Even though I’ve not been active as a Blender developer in the last few months, I kept lurking the mailing lists in an effort to stay up-to-date on the latest developments — and 2.5 in particular.

On Wednesday 17, together with a bunch of fellow Italian blenderheads, we will be talking about our dear 3d suite as part of the warm-up meetings of the Italian hackmeeting 2009.

Plans for the Lightcuts branch

Recent efforts, especially those led by the awesome cuban coder nicknamed Farsthary, have made it clear that sooner or later (but probably sooner), true Global Illumination will be available to Blender Internal.

Apart from that, in a number of discussions on the mailing list, it become apparent that the problem with Blender Internal is that its amazingly broad feature-set lacks a bit of overall vision, so that its not always clear how any two features interact. It seems that the post-2.5 period will be dedicated to some sort of design refactoring of Blender Internal so that different render paths can be handled more smoothly, both on the coder’s and on the user’s side.

I think that now it makes sense to wait for that to happen before attempting the integration of the Lightcuts stuff.

Moreover, I’m also very glad to hear that André Susando Pinto a.k.a Jaguarandi, great developer and past gsoccer colleague of mine, is again a gsoccer this year working on performance improvements to the internal raytracer. He’s the right guy to do that, after the convincing job he did last year on the BVH stuff. His work for 2009 will turn out extremely beneficial for my Lightcuts branch, which relies on the raytracer pretty heavily.

So my next steps are: 1) finishing the “reconstruction cuts” part of the algorithm and 2) porting the branch over to 2.5 — but not necessarily in this order.

In the meanwhile, I would like to spend my small amount of spare time to help out on 2.5 which is coming along really well: I’m truly impressed! Congratulations to everyone that has been working on it so far.

My collection of geeky t-shirts just got a lot better!

Even though this much awaited t-shirt is now secured in my hands, I’m still committed to improve my project. Admittedly I’ve not committed much recently (sorry for the pun) but I’m still working on it, albeit at a slow pace. Stay tuned!

Anyway: in the last couple of days I took the time to implement some minor stuff I had been postponing for a while.

First of all, support for baking is now in. Just bake with the “Enable Lightcuts” toggle pressed: it should work now. I have to admit that this did not go through much testing, so please report any strange behavior.

A second addition was compatibility with the SSS feature from Blender Internal. Previously, it used to crash outright. Now, it gives pretty good and smooth results:

SSS + Lightcuts (actually 1024 lights, not 8k as stamped)

(Apologies to Maurice R., who asked for this a long time ago… and it was actually a pretty easy fix!)

The possibility to have meshlights was another frequently requested feature. I just committed some initial code to support it.

Suzanne is the only light in this scene

It works automatically for all materials having an Emit value > 0.0. Unfortunately the object itself gets a completely flat shading. This will probably require a different handling UI-wise in the future, avoiding the Emit value altogether.

Finally, I also implemented some algorithmic improvements recently described in a paper by the original authors of the Lightcuts algorithm. According to them, this should lead to faster tree building times and better quality trees, which have in turn a positive effect on rendering times. Actually the improvements are barely noticeable most of the time but at the very least it simplifies code.

When computing the contribution of a cluster, you need to select a representative point light, whose position will be used to compute visibility and whose color will be used to shade the pixel. The original Lightcuts paper selects this representative light at a global level. With multiple representatives you choose a different representative at each sample.

The current implementation required some code refactoring, so that I ended up being a bit cautious about correctness — read: a number of optimizations are still missing. A lot of testing is still required but, as always, I rely upon my friends at blenderartists who are doing a wonderful job stress testing the system and reporting their feedback.

Quick comparison of original algorithm vs. multiple representatives option; the latter is currently significantly slower but a number of optimizations are still missing

In different news, the insane amount of bug fixing that has been going on at the end of the Apricot project, led many developers to plan a new release before diving headfirst into the enormous 2.50 undertaking. As far as I understand, Blender 2.48 will include not only bug fixes but also new features, including some of the amazing work my fellow soccers did this summer.

As for my project, while the code itself is in a fairly acceptable shape, it will take more time to merge. Unfortunately, even though the code is not invasive at all, the concept of lightcuts is not that easy to integrate cleanly in the current Blender workflow. And we all know that Ton would rather miss a feature than pollute the user experience with an alien concept; this error has been made in the past and now core developers are stricter on this — and I don’t blame them at all. I’m waiting for the Blender Conference to have some input or even some enlightening discussions with them about this delicate part of the work.

Here, instead, I would like to detail my plans for the future of this project.

Development — Development will continue, but necessarily at a slower pace. At the end of October I’ll be attending the Blender Conference where I would like to show a couple of new features. Apart from that, though, what this project needs now is some “boring” refinement.

Blender Internal features — I’ll have to go through all the Blender Internal features and for each of them ponder if it is possible to support it. Unfortunately the very nature of the algorithm makes it impossible to support all features (some of which are pure hacks) but what’s in and what’s left out must be clearly stated to users.

Workflow design — In my opinion if you want to use Lightcuts, especially if you want to obtain indirect lighting, you have to plan ahead a bit; I can’t see, in general, the same light rigs being successful in both Blender “vanilla” and Blender Lightcuts. Matt Ebb, and many others for that matter, would be happier to see Lightcuts as a less invasive tool. Of course mine is a statement while Matt’s is a desire, so it’s easy to agree with both. Anyway, however this “fight” ends, I need to figure out how to integrate Lightcuts in Blender from a UI point of view. In this particular area I would like to hear opinions from users.

Merging — Some people are afraid that a project living in its own branch is at a greater risk of dying; for sure it’s at a greater risk of going out of sync. While it’s true that this project is fairly isolated code-wise, and that development could continue also on trunk, I feel more free to experiment in my own branch. As my confidence with the Blender codebase increases, I could also attempt some bolder refactoring of some rendering engine internals — and that would be better tested in a branch.

My idea is to merge after the Blender Conference, while the aim for the entire project is to be released in Blender 2.50.

Ok, enough words. Here’s a Sponza test, provided by Melon on blenderartists:

A Sponza rendering lit by a single area light, with 5 bounces of indirect lighting

The entire thread is worth reading in my opinion.

Ps. — Next week I won’t be working on this project, but this is a planned break as I resume my duties so don’t think I’m running away!

I actually had a hard time figuring out how to read surface colors, which is something I need to color indirect lights and obtain color bleeding in the end. In the end I came up with some very hackish code that for sure wouldn’t be accepted in trunk. On the other hand I can’t see a cleaner way to do that without some changes in the existing Blender codebase. I have to talk to some other developer and see what’s the best thing to do.

Anyway, right now I have the ability to generate 1st and 2nd bounce indirect lighting from area lights, taking colours into account. I was able to obtain a rendering like this one:

"Unfinished Match" with indirect lighting - Tree creation time: 00:04.64 - Lights: 19968 (0l + 0s + 19968o) (10000d + 9968i) - Error rate: 0.020 Max cut: 1000 - Average cut size: 362.88 - Shadow rays: 345.63 (1.73%)

The interesting thing is that this model was previously lit by a very complex lighting setup in order to fake indirect lighting. This rendering was obtained by placing a single area light behind the window.

Here are some more tests:

Here there's an area light near the wall, turned upwards at 45°.

Hangar test - Here I exaggerated indirect lighting through post-process.

It is now possible to have several tens of thousands point lights without incurring in terrible tree creation times. This is particularly important when indirect lighting is added to the mix.

A simple way to add indirect illumination to renderers is to place several small lights on surfaces hit by direct lighting. This principle is used by a family of algorithms called instant radiosity, originally developed by Alexander Keller. A lot of research has gone into reducing the number of resulting lights, in optimizing their placement in the scene (so that, for instance, lights not influencing the final rendering get trimmed), and in overcoming some of the shortcomings of the algorithm, since it tends to suffer in some corner cases.

The final part of my Summer of Code project is indeed about creating a toy implementation of instant radiosity, in order to show a potential “killer application” for Lightcuts in Blender. I think it’s also important to have it in order to have a clearer idea on how to integrate Lightcuts from a UI point of view.

Ideally it would be nice to have just a button to press, without having to rework all the lighting differently, but I’m afraid that the user must have some awareness of the strengths and weaknesses of the algorithm in order to take full advantage of it. More on this in a later post.

Here are some very preliminary results from my initial experiments with virtual point light placement.

Textured area light with indirect lighting - Tree creation time: 00:00.54 - Number of (point) lights: 5966 (0l + 0s + 5966o) - Average cut size: 155.22 - Shadow rays: 153.77 (2.58%)

Textured area light with direct lighting alone (despite the stamp) - Tree creation time: 00:00.26 - Number of (point) lights: 3102 (0l + 0s + 3102o) - Average cut size: 134.83 - Shadow rays: 132.35 (4.27%)

Test scene - with indirect lighting - Tree creation time: 00:02.00 - Number of (point) lights: 18091 (0l + 0s + 18091o) (4900d + 13191i) - Requested error rate: 0.018 - Average cut size: 587.30 - Shadow rays: 582.20 (3.22%)

Test scene - direct only - Tree creation time: 00:00.43 - Number of (point) lights: 4900 (0l + 0s + 4900o) - Requested error rate: 0.020 - Average cut size: 567.16 - Shadow rays: 562.23 (11.47%)

(Yes, I am aware of the banding problems; it’s one of the issues I have to address next week before the Gsoc is over.)

This week’s work was indeed far less exciting: I worked on supporting more material features, as so far only lambertian diffuse and phong specular where supported. Now, all diffuse shaders are supported.

Every single shader must be specifically supported as we need to be able to answer this question: what’s the maximum value that a specific pixel, shaded this way, may assume over a bounding box of point lights? You don’t really need the exact maximum value, so a ridiculously large number would always be a “viable” answer. On the other hand, you want the bounding value to be as tight as possible, because if the error turns out to be very low, you may decide to skip all the lights further down in the light tree, thus saving rendering time.

I also added support for all falloff types (except custom curve falloffs). The default falloff type for Blender lamps is “inverse linear”, while only “inverse square” was supported so far: this could lead to very incorrect renderings, especially when testing existing scenes.

Now, for instance, this hangar scene (featuring a lot of dupliverted “inverse linear” point lights plus a couple of additional area lights) looks a bit better:

Tree creation time: 00:02.27 Number of (point) lights: 3841 (1476l + 0s + 2365o) Requested error rate: 0.020 Average cut size: 239.44 Shadow rays: 233.08 (6.07%)

Hangar (False colour)

(Thanks Gillan for pointing me to this model).

◊ ◊ ◊

On the textured area light side, Italian Blenderhead Lusque has performed an interesting experiment. If you bake direct lighting coming from a light source (e.g. an area light) onto the walls of a Cornell-like box, and then you use the baked textures as textured area lights, you basically have a simple indirect lighting pass.

Here you can see a test, where the indirect lighting has been exaggerated on purpose:

I’ll post Lusque’s blend file here as soon as we sort the remaining issues out.

(Please note that this is just an experiment and that it has nothing to do with the “Instant Radiosity” scheme I would like to implement; I am also perfectly aware that it has limited applicability, but it’s still very clever in my opinion).

In Blender you can already link a texture to a light, but the outcome is a sort of texture projection, which is useful if you want to fake effects like light passing through a tree without actually computing the visibility: that comes of course very handy for cutting rendering time, to have artistic control on shadowing, etc.

What I mean here with textured area lights, on the other hand, is actually modulating the color and intensity of the light throughout its area. This way you can obtain the effect of lighting coming from tv screens, large windows with varying lighting conditions, and even area lights with custom shapes. Here are some examples.

In the lightcuts paper you had also the added contribution of indirect light. In this case I added some additional lighting using a faint environment map. The interesting point to note is that putting additional lighting has a negligible impact on rendering time, if not a beneficial effect! This is because completely occluded areas evaluate far more lights than bright areas, as a consequence of using a proportional error metric; thus, having those parts brighter allows for less evaluations.

Here’s a false colour rendering where the green areas show where the environment lighting is more influential and the bluish areas show where the area light is more influential. The red channel counts the number of lights used with respect to the maximum, so the image is white where it’s most occluded, as expected.

By the way, as a SoCer I got to visit Google’s London office today. I was glad to have my Blender t-shirt handy for the circumstance. During the tour I’m pretty sure I saw a couple of guys editing a video using Blender on an enormous screen. Overall the visit was very inspiring and, you know, a free lunch is always welcome!