3D Design
Jul 2023
Gregory Markov

Complete Guide to the 3D Pipeline #4: Baking

This article is for beginners looking to become 3D artists. By the time you finish reading, you will know how to bypass the laws of physics, create a shadow on a plain surface, and bake better than your grandmother. A careful study of the text will take about 10 minutes.

This is the fourth part of the AAA pipeline series. Here are links to all five; choose the one you need:

  1. Introduction to AAA pipeline
  2. Blocking-out
  3. Mesh optimization
  4. UV unwrapping
  5. Baking (You are here)
  6. Texturing

What is Baking?

Baking is transferring details from a High poly model to a Low poly model with special textures. The goal is to make a surface look much more detailed than it really is. Ultimately, your game model seems as good as the High poly but affects performance less.

How does it work?

When people talk about texturing in 3D, they often think about painting a model. Though it’s true, and you will dive into it next time, textures can contain not only colors. There are ten different types of textures in Physically Based Rendering, and only one of them is responsible for dyeing.

This texture is flat, but thanks to PBR, it looks super detailed. Applied texture maps from left to right: Albedo, Height, Roughness, Normal, Ambient Occlusion.

How do you know that tree bark is not flat but has deep furrows? You understand it because sunlight bounces off the tree in different directions, creating vivid and shadowed places on the stem. While in the real world, it requires irregularities on the surface, in computer graphics, you can force the light to act as you wish.

Without shadows, this bark would look like a flat picture.

Thus textures can provide lighting, heights, opacity, and other characteristics. However, while baking, you will likely create normal maps and Ambient Occlusion maps since they allow for an illusion of a detailed surface.

⦿ Normal map is a texture where each pixel signifies the deviation in the perceived surface orientation compared to its original, unaltered surface normal. Simply put, it determines how the light will bounce from a surface.

Red channel indicates how the model should bounce light when lit from the right side. Green is responsible for bouncing light when lit from below. Blue specifies how the model should bounce light when lit from the front.
Some game engines can calculate the blue channel according to the other two. It reduces memory usage but slightly increases processing usage.

Normal maps are a really complicated topic, so we just peeled it a little. Check out Carlos Lemos’s normal maps tutorial to know more about its insides.

Without a normal map, the light will bounce according to the laws of physics.
After applying a normal map on that angle, the light will bounce as if it were rounded.

⦿ Ambient Occlusion(AO) map is a texture simulating how ambient light is scattered or occluded by objects in a scene. It represents areas of a model that would naturally be in shadow or have less light reaching them.

While the difference between normal maps and AO maps might be bewildering, the first one specifies how the light bounces off the model, while the second determines how much light or other particles can reach the model.

⦿ Color ID map is a technical texture that colors each material or element within a model. This helps to work on a model faster during the texturing stage. For example, you can assign a unique color to each material. Through this, you will be able to select and modify individual elements without affecting the rest of the model.

Model of the outfit organized with the Color ID map, so the artist can quickly select elements with the same material.

What to do before baking?

Get the Low poly ready

⦿ Configure smoothing groups. Use hard edges with separate groups for angles of 90° or less. For angles over 90°, use soft edges within the same group. Maintain consistency with hard edges in a loop to prevent artifacts.

Smoothing groups on the left applied incorrectly, so the right-angled elements look as if they were rounded.

⦿ Eliminate gaps between geometry. Ensure there are no gaps between the geometry. In such cases, slightly push the geometry inside the other.

⦿ Use triangulation. If you don't triangulate the model before baking, it will appear differently in various software.

The model may not be greatly optimized, but it definitely has been triangulated.

Prepare the High Poly

Take a look at this model, it will serve as an example in the second half of the article.

⦿ Refine geometry for better shading. If the High poly appears unattractive, adjust its geometry to transfer the details to the low-poly model accurately.

⦿ Avoid overly thin bevels, which can cause baking issues. Here are examples of good and thin bevels:

⦿ Create chamfered angles instead of 90-degree angles, as baking algorithms work better with chamfers. Geometry without chamfers doesn't bake well. Here's an example of how to chamfer geometry and enhance expressiveness:

The angle has a chamfer since it is harder for the baking algorithm to work with right-angled shapes.

⦿ Create the floating geometry. Place cylinders or complex shapes on top of the model instead of embedding them. As so-called floaters are not connected to the model, they can be baked onto normal maps without causing artifacts or issues with the geometry.

Geometry is floating in the air, making it easier to model and bake on a normal map.

After these steps, ensure that High poly and Low poly models align as closely as possible. To identify areas that do not match, apply different colors to the High poly and the Low poly. Then place one model inside another. Additionally, verify if the bevels align by assigning a reflective material to the Low poly model and making it transparent.

Prepare yourself

⦿ Remove unused elements from the scene. These include extra layers, materials, textures, objects, or empty groups. Over time, they can clutter the workspace and slow down the software, making navigating and managing your project more difficult.

⦿ Create a backup. During the baking, you might need to modify geometry, such as adding bevels or supports. This can cause chaos if you do not save the original model.

How to bake?

You can bake textures in any 3D modeling software. However, the easiest way to do it is to use Marmoset. Here is a step-by-step guide:

Step #1: Configure mapping

Set the texture resolution twice as high as the final model's resolution. This technique is commonly used in production since a 2K image downscaled to 1K looks better than one created directly in 1K.

Indicate which maps to bake, such as Normal, Ambient Occlusion, and Color ID. Select the folder where the program will save the baked maps.

Step #2: Create baking groups

To prevent artifacts during baking, you can explode the model into separate elements avoiding geometric intersections. You may want to use the backup to quickly reassemble the exploded parts after. However, the better way is to utilize baking groups.

Baking group is a cluster of elements set together for baking texture maps. For example, if a model comprises multiple materials, you can create separate baking groups for each and configure them separately.

Another use of baking groups is to prevent artifacts caused by overlapping geometries. Organize your model into sets of non-intersecting parts, and use corresponding names for low poly and high poly models to simplify navigation.

Two other baking groups organized with the same principle.

Step #3: Adjust the сage

Cage is a semi-transparent mesh that wraps around the selected group's elements. It defines the space in which the details from the High poly grid project on the Low poly surface.

You can adjust the cage size, and everything inside it will be baked. Vise versa, anything outside the cage won't be baked. So, remember these points:

  • Artifacts occur when the cage areas overlap even if polygons don't intersect
  • Reducing and enlarging cage size may eliminate artifacts

Step #4: Bake!

Finally, click the "Bake" button to generate PSD files. In the next pipeline stage, you will work with these files in texturing software like Substance Painter.


In Marmoset

Even if you perfectly followed the guide, artifacts may still appear. This is due to the complexity of baking and the two different normal map algorithms: geometry normal and surface normal.

Each algorithm has its strengths and weaknesses. Thus geometry normal bakes bevels and seams well but may stretch circles, while surface normal preserves circle shapes but struggles with edges.

In Marmoset, you can utilize the strengths of both algorithms by using the Paint Skew function. Suppose a particular area of your model is distorted after baking. In that case, you can select the zone and use the Paint Skew function to recalculate the normal map with a different algorithm.

Paint skew works just like any other brush. You can highlight the distorted area and recalculate it with another algorithm.

In Photoshop

Sometimes, you cannot fix the model's artifacts in Marmoset. In these cases, you can rebake a part of the model and repair it in Photoshop. To do this:

  1. Export the problematic element in your 3D modeling software
  2. Rebake that element
  3. Fix the normal map in Photoshop using brushes, masks, Clone Stamp, and Healing Brush tools.

If this can’t help, you have to redo the model.

To the texturing!

You finally nailed the technical stages of the pipeline. Now, when you have the Low poly model along with UV, Normal maps, AO maps, and Color ID, it’s time to finish the work.

Doubt no more and move to the last stop of our comprehensive guide:

  1. Introduction to AAA pipeline
  2. Blocking-out
  3. Mesh optimization
  4. UV unwrapping
  5. Baking (You are here)
  6. Texturing


Partially based on the material by XYZ Network

No items found.
No items found.

You might also like

Join our social networks

Stay in the loop with all the latest happenings in the game development world?

Subscribe to our newsletter

Receive fresh news and useful articles about game dev directly to your inbox! Stay up-to-date with the latest trends and events in the industry, and receive exclusive tips

Wrong e-mail adress
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Subscribe to our newsletter

Receive fresh news and useful articles about game dev directly to your inbox! Stay up-to-date with the latest trends and events in the industry, and receive exclusive tips

Wrong e-mail adress
One more thing

Follow us on Instagram and we will send you invite to the course

Follow Instagram

Gift for registering a person a list of 10 tips for those who want to change their profession.

Oops! Something went wrong while submitting the form.