Freshly Squeezed Progress Report: Profilin’ and Optimizin’

In my previous report, I got into the details of how I fixed the silly way I was storing data in my game so that it was more efficient and allowed the dungeon rendering to be faster in The Dungeon Under My House, my second Freshly Squeezed Entertainment project.

This past week, I continued optimizing my dungeon rendering code, with a goal of getting the animations to be 60 frames per second, plus figuring out why one door in the dungeon is so well lit compared to what it should be.

Sprint 2024-3: Pre-production and initialization

Unplanned and complete:

• Defect: Door visual floats in middle of view when open and seen from the side; should appear to block view to right instead

Planned and incomplete:

• Render dungeon based on light levels

In my last report, I said I was done with the rendering of the dungeon based on light levels, but I wasn’t satisfied with the performance, even after my changes from last week, so I decided to continue optimizing.

Even though the game isn’t meant to be played in real-time, I think the game feels a lot better when it runs at 60 FPS, and the slowness of the dungeon rendering was getting in the way of making the entire game feel crisp and smooth.

But first, this door was getting on my nerves.

That image was from a few weeks ago, when I hadn’t started working on the floor and ceiling rendering code yet. I was wandering through my small test dungeon, checking how the walls got darker the farther away you were from the light source, when I discovered this very bright door and doorway.

What was going on there?

I double-checked, and I didn’t set any ambient light on that cell, and in fact, the walls next to it were darker. It was just the door and its associated doorway that was rendering so bright despite the fact that the cell it belonged to was rendering much darker.

So I kept digging and logging, and eventually I decided to set the default color of a cell to something extreme. And there was my my first clue: apparently the door wasn’t taking its color from the cell it belongs to but from an adjacent cell, and in this case, one that was out of bounds and so wasn’t paying attention to the lighting code.

After digging into the code, I discovered that the way I wrote the code and the way the door is represented means that the wrong cell’s lighting data was being used. Basically, the hinge edge and the far edge of the door is stored, and I used the hinge edge’s location to identify the cell’s position. In some orientations, this worked fine, but in others, it grabbed the next cell.

And for things like the doorway, well, those could be arbitrary dimensions. How do I generically identify which cell it should belong to?

Well, I decided that I would just store the door’s current cell as data in the door itself, so instead of trying to derive the cell based on the door’s data, I could just get the cell itself.

Anyway, I also had a different problem with doorways. If the door is open, and you are facing the entryway, the door will be seen on the side.

But if you are facing away from the entryway (that is, the entryway is behind you), the door should be on the other side.

And it is…except it kind of floats out in the middle of the viewport.

This is ugly, and I wanted to fix it, too. If the door is to the side, it should block your view from that side, right?

Since this is raycasted code, the best idea I had for what was going on was that the rays were going around it.

So I think intuitively the rays would hit the door behind the camera plane, too, and should show that area as blocked, but the point of the camera plane is to not draw behind it, and the point of the rays being projected out was to avoid fisheye lens effects.

So, what could I do? I tried changing the field of view. I did find that if I change the FOV from 1.0 to 0.5 that the door seemed to block the far side of the view as I wanted, but the downside was that the player’s view seemed too constrained. I wanted the player to be able to stand next to a hallway and see that it exists, but too small a FOV prevents that.

But anything larger would show a floating door in certain orientations.

Now I loved my door and entryway, especially how it was centered in the tile, but I decided that the most straightforward thing to do is to make the door larger so that the hinge is at the far end of the tile rather than in the center.

This way, when the player is facing out of the entryway, the door is seen at the far edge of the viewport rather than have the door protrude out of the middle of the camera.

I’m not happy that these kinds of doors now take up the entire cell, but it seemed the least bad option. And on the plus side, I think I found that having a FOV of 0.66667 looks great.

Anyway, back to optimizing. Now that walls, doors, ladders, ceilings, and floors of the dungeon are rendered based on lighting, I found that there was a definite difference between how smoothly the game seems to be responding when stationary versus when turning or moving forward or backward.

If I remove the ceiling and floor rendering code, the walls and “things” render super quickly. The game feels very smooth.

But if I remove the walls and “things” rendering code, the ceilings and floors render just a little too slowly. It’s fast enough in terms of functionality, but it feels jarring and off.

I had changed my game’s dungeon representation from an std::map to an std::vector, using fast indexing math to quickly access the data I need in constant time, but it was still slow.

And using the “poor man’s profiling” technique of interrupting the debugger and seeing what code it is sitting in most often got me pretty far before, I decided to try using a real profiler to see what I can learn.

So I love valgrind, and I used the command “valgrind –tool=callgrind” on my game’s binary file. While the dungeon rendering was much, much slower when running with a profiler, I was able to collect some good data.

Then I used kcachegrind to pull up some fun visualizations.

What they told me was that my drawCeilings() and drawFloors() functions weren’t slow due to any one particular thing. There were lots of different things going on, each of which was taking up a chunk of the processing.

What this means is that while I could tackle any number of optimization opportunities that I see here, and I did gain small improvements when I did do so, there was something bigger happening.

What’s very interesting is that the drawWalls() and drawThings() functions take up so little processing compared to drawCeilings/drawFloors(). My main bottleneck with the latter wasn’t the various functions it called or objects it created. It was that it was working pixel by pixel across half a million plus pixels.

Basically, no matter how fast I get the code to render a pixel, it still has to draw a lot of pixels, and the main way I am going to speed things up is to draw fewer pixels.

I didn’t get the chance to try it before the end of the week, but my next major optimization attempt will be to draw the walls and “things” first, then scan pixel by pixel to find what hasn’t been drawn yet and draw the ceilings and floors at those points.

It should cut down significantly on pixels drawn, especially since most of those pixels were getting drawn on top of anyway with my current approach, so I anticipate that this new approach will mean most of the pixels won’t be so expensive to render.

But we’ll see.

Thanks for reading!

—

Want to learn when I release The Dungeon Under My House, or about future Freshly Squeezed games I am creating? Sign up for the GBGames Curiosities newsletter, and download the full color Player’s Guides to my existing and future games for free!