3D Art Pipeline, for Games

As I move more into the area of web development, I run into a lot of people who are interested in making games, but have little idea on how they are made, especially the art. Hopefully this post will give a general idea of how art goes down. Specifically of an animated character, from concept to engine.

As a side note, every studio, project, and artist are different. This is meant to be an overview of the general flow, but most projects will have more steps, omit some, combine others, use different software. I

The idea: All things start in with an idea. Maybe it’s several well designed pages, with description and purpose. Other times it’s a napkin that someone scrawled on during lunch, complete with mustard stains .3D Art Pipeline_1

Concept: This is then kicked off to the concept artist. Their job is to turn the words into pictures. This usually starts as many thumbnails which then gets refined into color comps and eventually finalized.

3D Art Pipeline_2

Besides the design document, they usually have a style guide, they follow to help ensure that the character will fit into the rest of the world, and speak the same visual language. An amazing example of this is DOTA 2’s style guide

3D Art Pipeline_3

Their finished product is often a character turn around, with maybe a couple extra action / emotive poses.  This gets passed off to the modeler

3D Art Pipeline_4

The Model: There are many methods for making 3D models, my preferred method is the ‘Box Modeling Method’ in which you start with a box, subdivide it, move the vertices to best fit the shape you want, then rinse and repeat until you have the level of detail you are looking for.

3D Art Pipeline_5

Lets take  a second to describe the anatomy of a model, and polygons. All models are made up of 2D shapes, laid end to end. As you see in the left image, there are a bunch of rectangles (Which all consist of 2 triangles.) While they give the illusion of volume, they are all completely flat. With points that define the area. Then these polygons have color placed on them, through shaders. In fact, in order to see them they have to have 2 different types shaders on them, but lets not worry about that yet.

3D Art Pipeline_6

In order to get the color onto these models, you must lay them out flat, so that you can overlay an image onto them. This is called ‘unwrapping’. You can think of taking an orange rind, and chopping it up into little pieces so it will lay flat.

3D Art Pipeline_7

Once it’s unwrapped, the character artist has to actually make the images (maps) that will be laid onto the model. There are also a handful of maps that go into defining the material of a model.

These are usually done in either a traditional raster image program, like Photoshop, or in a sculpting program like ZBrush or Mudbox. Often mix and matched.

3D Art Pipeline_9

Inside the sculpting programs, the artist doesn’t have to worry about things like polygon limits. They get to just get in there and make all the detail they want, molding millions of polygons. Then at the end, they tell the program to make an image that will tell the low poly starting model, look like the high res beautiful one. That is an over simplification, but the basic idea.

3D Art Pipeline_10

The character artist now has the textures they made from either sculpts or hand painting / photo chopping. The next step is to compile these textures together into a material. A material is an instance of a shader. A shader tells the computer how to display the polygons. Light affecting a model? The shader told it to. Does the character have any color, outline, or transparency? Shader.  Did it move? That could be the shader, depends on how it’s implemented.

It’s good to note there are 2 types of shaders, vertex and fragment/pixel. Where artists are concerned, they are messing with pixel shaders. I’ll get into the difference in a bit.

So, the shader tells the computer how things should look, or rather, the rules to follow when determining what a thing should look like. Such as ‘look at a textureA, multiply it by lighting for the color, look at textureB and modify it by valueC for how shiny this spot is.” The material is then the specific instance, the place where the artist gets to plug in specific textures and values into those slots for textureA, textureB and valueC.

3D Art Pipeline_11

From left to right:
The same rhino with 3 different materials (probably different shaders too.)
A material view, showing primitive objects with different materials applied
A material editor view in 3DS Max

This is a pretty solid high level overview of the how a mesh gets turned into pixels

3D Art Pipeline_12

So, the artist has a finished model, it’s all textured and made pretty. The next step is to make it move, right? Whoa, slow down there buddy. We have to create the system which will allow it to move. This is called rigging. Just like the character artist had to layout the mesh flat, and assign the 3D vertices to a 2D Space, we must again bind each vertex to an object so we can easily move them over time.

This is the job of the rigger. The rigger creates a system of joints (or bones) and other voodoo to define how the character can move. Then they bind these joints to control objects. Further abstracting and distilling down the control, and exposed API (options) that the animator has access to. Once the controls are set up, the rigger then binds the mesh to the joints, in a process called skinning.

3D Art Pipeline_13

Then it can handed over to the animator, to begin, well, animating.

Animation is the process of creating a set of images that will be seen sequentially, over time, to give the illusion of motion. Usually the animator tries to convey things through this motion. “This is a big and heavy robot, moving forward in time.”, “This person just got hit, really hard.”, “This person is sad, and really just wants someone to acknowledge their existence.” The last one doesn’t happen as much in games.

In 3D, the animator does this by setting keys, on frames. Coming from the traditional animation term of ‘keyframe’. Keys define when an object should be at a specific place. It looks something like this

3D Art Pipeline_14

After that it gets exported and is ready for use in the game engine.

A few notes:
If the process is done well, most of these steps can be done in parallel, which is really neat. Not so much the first part (drawing.) But you can totally have a proxy model of roughly the right proportions get rigged with basic controls and passed onto the animator so they can all work in unison. Then update just get passed downstream.

Characters often have effects associated with them. Dust when they walk, fire when fly with their jetpack, lazer eyes, whatever it may be. This is done by a ‘technical artist’ and is a whole ‘nother beast we’ll get into at a later date. Maybe.

Anyways, I hope you found some good info in this high level overview, and will be kind to your artist in the future. They work very hard.

Advertisements

R6: Post Mortem

I recently completed working on a game with 3 other people titled R6. Check it out.

Tech: Socket.io and Babylon.js were the heavy lifters in this project.

it is a 3D multiplayer in browser robot racing game. I wanted to take a moment to go over what worked, what didn’t, any odds and ends of observations.

The Good:

  • Scope: It was surprisingly well scoped. Even though none of us had done a multiplayer game before, and I was the only one who had worked with 3D before (definitely not in browser, or with javascript before.)
  • Research: We broke off into 2 teams at the start. One for front end (Babylon research) and the other for back end (Socket) at the end of the second day we had game logic tied to 3D representations that could be integrated with an event driven back end.
  • Iteration: We hit numerous bugs and hiccups (camera jitter, frame rate issues, rotation / scale.) But because we always worked to have something playable at all times, they never built up. Frame rate was also something that we all ended up improving over time as we spotted things could be improved, because we all worked the whole code.
  • Naive first: While we definitely spent time architecting, especially the core game logic, often times we’d hit areas we just didn’t have enough experience in to make educated guesses. We’d talk briefly, decide on a plan of action, make a note of the probable down falls, and take action. More often than not these solutions worked well enough.
  • Sound: One of our programmers was a musician as wel

The Bad

  • Art Pipeline: art is a large and sometimes unwieldy beast. in order to be efficient we used a bitmap as collision detection server side, which meant there was no actual relation between what the client saw, and what the server was doing other than, I lined them up in 3DS Max. Due to time constraints this was a very manual process, and prone to error. But worse than mild collision inconsistencies, was it was time consuming. While it worked for this project, the next order of business would be building art tools.
  • Art Seclusion: I’ve been an artist on a project where I can’t get my own art in the game, because it has to go through an engineer. This is a terrible experience, I don’t get immediate feedback, as well, in order to tweak and polish requires me to eat up engineer time. This project had a reverse, nobody else could really touch the art / design in any way because of how tied up into my 3D applications it became. Which meant that I was very alone with problems, and if they wanted to input on it, they could not directly do so.
  • More Design Time: This always happens with shorter projects, but by the time it really starts coming to shape and you can polish the levels (or re-make them) your project over. This game could be quite good with another week or so.

I actually really enjoyed doing double duty as a programmer (about 70% of time) and an artist (remaining 30%.) It felt really good to flex a full range of ‘muscles.’

Vector Math: For Fun and Profit

Part 1: Hamsters, Scales, and Addition

Lets talk about vectors for a bit. I was taught briefly about them (along with sin and cos) waaaaay back when, in trig and geometry back in high school. They were just this thing I had to learn and they didn’t tie into reality. After those classes they just gathered dust in the corner. I’d occasionally see it out of the corner of my eye in the form for ‘vector art’ or ‘vertices’  for 3D models, but still, not really relevant. This all changed when I started programming games. Maybe I changed, or maybe they did (it was me, vectors haven’t changed much.) But now we get along great. Vectors are amazing, powerful tools and the best part is that they are dead simple.

I’m going to keep this light and full of pictures, so if you want to dive in deeper, Kahn Academy has you covered.

First, what is a vector? It’s a few numbers that are related, usually describing something to do with space (position, speed, force, etc).  A vector 2 looks like this [2,3] – because there are 2 numbers. Vector 3 [3,2,5]. There is one number for each plane you are talking about. In fact I’m pretty sure you could have a vector 1 if you felt like it and these observations would still apply.

Lets look at a vector in 3D space, talk about a few properties, then we’ll get into actual use, games!

vectors_1

This is vector 3 would be written down as [3,5,2].

Right off the bat we can see that it’s describing a point in space. It’s over a few units to the right (x : 3), it’s up in the air some (y : 5) and it comes at us a bit (z : 2).

Another thing it describes, is a direction. All spaces (a place where vectors live) have something called an origin. When I say that vector is 3 units over to the right, what is it over to the right from? Well, the origin. An origin is (0,0,0) in whatever space you are talking about. What is really cool about physics, or spaces in general is you can always pick your own origin, and align the axis in whatever way makes your life easiest. Though more discussion on that is out of the scope of this post.

vectors_2

Lets pretend that point represents a ball in the air, if you were to stand on the origin and look at the ball, you are looking in it’s direction. So if you know a point in space, you know a direction (always relative to the origin.)

The last thing it describes is a distance, or simply, the length of the line segment that connects the origin to our point. In this case it is a length (sometimes referred to as magnitude) of 6.1644. We got this by using the good ‘ol Pythagorean Theorem. A^2 + B^2 = C^2, or in our case X^2 + Y^2 + Z^2 = L^2 (L, being length.) It doesn’t matter that we have 3 planes instead of 2, the math transfers. In fact it works in 1 dimension as well. 3^2 = 3^2 after all.

vectors_3

I’m getting antsy, lets jump into talking about games. We can pick up operations as we go.

First thing, lets have a character, how about a hamster? That’s what you were thinking as well? Excellent.

vectors_4

Alright, we have our hamster, lets name her Stella. Since this is a game we have to have some mechanics. The first one that comes to mind, is movement. Lets say if the player presses W, Stella will move forward one unit in the X axis. As a vector that looks like [1,0,0]

vectors_5

There are a couple things going on here, I drew just the [1,0,0] vector coming off of the origin. I also drew it coming off of Stella, as that is how vector addition works. You just line them up, and do simple arithmetic.
[1,0,0] +
[2,0,5] =
[3,0,5]

Just like in normal addition, order doesn’t matter. In fact you could break Stella’s new position up into 3 separate vectors and add them together to get her new position
[3,0,0]
[0,0,0]
[0,0,5]

vectors_6

Excellent, Stella can now move forward. If we extend this logic a bit, we can have her move back with D. Instead of adding [1,0,0] we will add [-1,0,0]. We can also subtract [1,0,0] from her position, either way. Just like with arithmetic, it doesn’t matter which way you prefer. We can also giver her side to side by adding and subtracting on the Z axis when the player presses D and A, respectively.

One observation I wanted to point out about the above drawing, you can see that when we break up the vectors, it looks like we are drawing a box. To some extent we are, with 2 points you can define the minimum and maximum points of a box, and if we use the origin as the other point, we totally have 2 points.

vectors_7

You can see I went ahead and decided to draw a triangle that the vector can describe as well. If you think about it a bit more, that box actually describes 3 different triangles. Plus it describes 3 sets of 2 rectangles (each of the faces.) Those faces can all be made up of triangles. I’m getting ahead of myself, but the more relationships you start to see, the more you can have vectors do work for you. Though the opportunities really build up when we start to add sin and cos.

The final thing we will do is make it so that you don’t have to keep tapping the key to move down, lets let you hold the key.

For the sake of argument lets say we are running the game at 30 frames per second. Lets also say that we want Stella to move 5 units per second in the direction we are holding.

After some unit conversion

vectors_8

We come up with 0.16666 as the distance we want Stella to travel, every frame, in whatever direction the player is pressing.

How do we use this number? Well, when are dealing with vectors, numbers that just live by themselves are often referred to as scalars, or scalar value. If you recall, we encountered one of these earlier when were talking about the magnitude of a vector.

Multiplying a vector, by a scalar value (which is what we want to do) is simple. It’s noting more than distribution. In our case, it’s the even simpler because our starting vector looks like [0,0,-1]. So, [0,0,-1] * 0.166 looks like [0 * 0.166, 0 * 0.166, 1*0.166] which becomes [0,0,1.66]

Starting at [0,0,0], if the player holds S for 3 seconds, Stella would then be at [0,0,-15] :   30 (frames per second ) * 3 (number of seconds) * 0.166 (units per second) * [0,0,-1] (our vector)

If the player then taps W for 3 frames, and they’d end up at [0.5,0,-15].

Next time we will look into acceleration, unit vectors / normals. We will do this by putting Stella in spaaaaace.