CS8803: Computer Graphics in the AI Era

Note: This entry is still a draft. For now I’ve included screenshots and videos of my assignments.

Final Project

The summer semester concluded with a final project. We had the choice of going the technical research track or creative track, and I chose to do the creative track in a group of 3 (one of them being my groupmate for a previous course in Video Game Design!).

Below is the final render of the scene we affectionately named “Isle of the Lantern Keeper”.

We used a mix of the computational methods we learned during the course:

1. Signed Distance Functions (SDFs) for the nature elements, firefly, hut, and lantern.
2. Volumetric Rendering for the aurora borealis, chimney smoke, and clouds.
3. Physics for the swing.
4. Noise for waterfall and island texture.
5. Fresnel reflection for the waterfall to reflect the sky.

Assignments

For each module, we needed to implement concepts in WebGL. For ML-related topics, we implemented algorithms in Jupyter notebooks that the course has provided the boilerplate for and trained neural networks to produce outputs we rendered in WebGL.

Each assignment had a default scene, and a creative expression part - which was fun because we get to choose our own topic!

Below are what I submitted as creative expressions.

Assignment 1 - Raytracing

Having taken CS6491 Computer Graphics last year, this assignment was mostly a refresher - although this time it was nice to implement it in WebGL instead of Java.

Assignment 2a - Signed Distance Fields (SDFs)

Procedurally positioned cloudberry drupelets. SDF-based repetition.

Assignment 2b - Neural SDFs

I really like plants and fungi and the first thing that came to mind are Mycorrhizal networks, a vast and dense network of fungi living in plant root systems, which plants make use of to transmit signals.

There has been research studying a form of intelligence in the network, where fungi explore numerous new paths and choose only viable, long-term ones, based on resource availability.

The density, appearance, and properties of this network share a lot of similarity to neural networks. So I decided to train my network on this 3D model of mushrooms.

Assignment 3a - Volumetric Rendering

I wanted to try rendering the Crab Nebula and searched for inspiration on Shadertoy. I came across Duke’s Supernova shader and implemented my version, tweaking the transparency near the cube edges, and making the center lighter and edges reddish. Kinda reminds me of the Protomolecule from The Expanse.

Assignment 3b - Neural Radiance Fields

We had a set of 3d models to choose from, and I picked the tiny_chair dataset to train the neural network on.

Assignment 4 - Gaussian Splatting

I trained on my Space Girl illustration but had to resize it to 256px x 256px because of memory issues. I tried between 150 and 200 number of gaussians, but they resulted in extra splats around the neck that added more noise. Going down to the default 100 seemed to produce a more accurate and less blobby result.

Assignment 5 - Position-based Physics Simulation

I like to explore subjects with radial symmetry in nature (such as flowers, orbiting planets, etc) so for this assignment, I chose a spider web. I generated 8 spokes (like a wheel’s spokes) and 3 “rings”. The points in the outer ring and center are fixed to keep the web shape intact, while leaving everything else bouncy. I removed the ground constraint and increased the web’s stiffness so the spring remains static unless dragged around by the mouse.

Initially, the points form a “web” but through some gravity and damping, falls into a spiral rose-like pattern.

Assignment 6 - Diffusion Models

I trained on the sketch of my website logo to see how well rough edges can be picked up. Training with the default parameters resulted in a very blobby ring and was missing the inner concentric details from the original image. So I increased the batch size to 1024 and increased the learning rate progressively.

Thoughts

The summer term being more condensed gave us about roughly 2 weeks to watch video lectures and do the assignments. Unfortunately, I was sick for half of the term, so I was sifting through the lectures just to absorb enough material to accomplish each assignment and not fail the midterm exam.

The quality of the lectures, the tons of supplementary readings, and the attention of the staff in building an active and engaging community and giving timely answers to questions were top-notch. The topics were highly interesting, and I came out with a better understanding and appreciation of the latest Computer Graphics methods. I’ll be sure to refer back to the materials in my future CG projects.

Despite the hurdles, the grading rubric was very clear and generous, and I managed to get an A (99.5% - and only because I lost 0.5 points in the midterm LOL).