Cleanup

Summary: A guide to managing memory and cleaning up Three.js resources (geometries, textures, materials, loaded files) by implementing a ResourceTracker class to automate disposal.

Cleanup

Three.js apps often use lots of memory. A 3D model might be 1 to 20 meg memory for all of its vertices. A model might use many textures that even if they are compressed into jpg files they have to be expanded to their uncompressed form to use. Each 1024x1024 texture takes 4 to 6meg of memory.

Most three.js apps load resources at init time and then use those resources forever until the page is closed. But, what if you want to load and change resources over time?

Unlike most JavaScript, three.js can not automatically clean these resources up. The browser will clean them up if you switch pages but otherwise it's up to you to manage them. This is an issue of how WebGL is designed and so three.js has no recourse but to pass on the responsibility to free resources back to you.

You free three.js resource this by calling the `dispose` function on [textures](textures.html), [geometries](primitives.html), and [materials](materials.html).

You could do this manually. At the start you might create some of these resources

```
const boxGeometry = new THREE.BoxGeometry(...);
const boxTexture = textureLoader.load(...);
const boxMaterial = new THREE.MeshPhongMaterial({map: texture});
```

and then when you're done with them you'd free them

```
boxGeometry.dispose();
boxTexture.dispose();
boxMaterial.dispose();
```

As you use more and more resources that would get more and more tedious.

To help remove some of the tedium let's make a class to track the resources. We'll then ask that class to do the cleanup for us.

Here's a first pass at such a class

```
class ResourceTracker {
  constructor() {
    this.resources = new Set();
  }
  track(resource) {
    if (resource.dispose) {
      this.resources.add(resource);
    }
    return resource;
  }
  untrack(resource) {
    this.resources.delete(resource);
  }
  dispose() {
    for (const resource of this.resources) {
      resource.dispose();
    }
    this.resources.clear();
  }
}
```

Let's use this class with the first example from [the article on textures](textures.html). We can create an instance of this class

```
const resTracker = new ResourceTracker();
```

and then just to make it easier to use let's create a bound function for the `track` method

```
const resTracker = new ResourceTracker();
+const track = resTracker.track.bind(resTracker);
```

Now to use it we just need to call `track` with for each geometry, texture, and material we create

```
const boxWidth = 1;
const boxHeight = 1;
const boxDepth = 1;
-const geometry = new THREE.BoxGeometry(boxWidth, boxHeight, boxDepth);
+const geometry = track(new THREE.BoxGeometry(boxWidth, boxHeight, boxDepth));

const cubes = [];  // an array we can use to rotate the cubes
const loader = new THREE.TextureLoader();

-const material = new THREE.MeshBasicMaterial({
-  map: loader.load('resources/images/wall.jpg'),
-});
+const material = track(new THREE.MeshBasicMaterial({
+  map: track(loader.load('resources/images/wall.jpg')),
+}));
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);
cubes.push(cube);  // add to our list of cubes to rotate
```

And then to free them we'd want to remove the cubes from the scene and then call `resTracker.dispose`

```
for (const cube of cubes) {
  scene.remove(cube);
}
cubes.length = 0;  // clears the cubes array
resTracker.dispose();
```

That would work but I find having to remove the cubes from the scene kind of tedious. Let's add that functionality to the `ResourceTracker`.

```
class ResourceTracker {
  constructor() {
    this.resources = new Set();
  }
  track(resource) {
-    if (resource.dispose) {
+    if (resource.dispose || resource instanceof THREE.Object3D) {
      this.resources.add(resource);
    }
    return resource;
  }
  untrack(resource) {
    this.resources.delete(resource);
  }
  dispose() {
    for (const resource of this.resources) {
-      resource.dispose();
+      if (resource instanceof THREE.Object3D) {
+        if (resource.parent) {
+          resource.parent.remove(resource);
+        }
+      }
+      if (resource.dispose) {
+        resource.dispose();
+      }
+    }
    this.resources.clear();
  }
}
```

And now we can track the cubes

```
const material = track(new THREE.MeshBasicMaterial({
  map: track(loader.load('resources/images/wall.jpg')),
}));
const cube = track(new THREE.Mesh(geometry, material));
scene.add(cube);
cubes.push(cube);  // add to our list of cubes to rotate
```

We no longer need the code to remove the cubes from the scene.

```
-for (const cube of cubes) {
-  scene.remove(cube);
-}
cubes.length = 0;  // clears the cube array
resTracker.dispose();
```

Let's arrange this code so that we can re-add the cube, texture, and material.

```
const scene = new THREE.Scene();
*const cubes = [];  // just an array we can use to rotate the cubes

+function addStuffToScene() {
  const resTracker = new ResourceTracker();
  const track = resTracker.track.bind(resTracker);

  const boxWidth = 1;
  const boxHeight = 1;
  const boxDepth = 1;
  const geometry = track(new THREE.BoxGeometry(boxWidth, boxHeight, boxDepth));

  const loader = new THREE.TextureLoader();

  const material = track(new THREE.MeshBasicMaterial({
    map: track(loader.load('resources/images/wall.jpg')),
  }));
  const cube = track(new THREE.Mesh(geometry, material));
  scene.add(cube);
  cubes.push(cube);  // add to our list of cubes to rotate
+  return resTracker;
+}
```

And then let's write some code to add and remove things over time.

```
function waitSeconds(seconds = 0) {
  return new Promise(resolve => setTimeout(resolve, seconds * 1000));
}

async function process() {
  for (;;) {
    const resTracker = addStuffToScene();
    await wait(2);
    cubes.length = 0;  // remove the cubes
    resTracker.dispose();
    await wait(1);
  }
}
process();
```

This code will create the cube, texture and material, wait for 2 seconds, then dispose of them and wait for 1 second and repeat.

[click here to open in a separate window](/manual/examples/cleanup-simple.html)

So that seems to work.

For a loaded file though it's a little more work. Most loaders only return an [`Object3D`](/docs/#api/en/core/Object3D) as a root of the hierarchy of objects they load so we need to discover what all the resources are.

Let's update our `ResourceTracker` to try to do that.

First we'll check if the object is an [`Object3D`](/docs/#api/en/core/Object3D) then track its geometry, material, and children

```
class ResourceTracker {
  constructor() {
    this.resources = new Set();
  }
  track(resource) {
    if (resource.dispose || resource instanceof THREE.Object3D) {
      this.resources.add(resource);
    }
+    if (resource instanceof THREE.Object3D) {
+      this.track(resource.geometry);
+      this.track(resource.material);
+      this.track(resource.children);
+    }
    return resource;
  }
  ...
}
```

Now, because any of `resource.geometry`, `resource.material`, and `resource.children` might be null or undefined we'll check at the top of `track`.

```
class ResourceTracker {
  constructor() {
    this.resources = new Set();
  }
  track(resource) {
+    if (!resource) {
+      return resource;
+    }

    if (resource.dispose || resource instanceof THREE.Object3D) {
      this.resources.add(resource);
    }
    if (resource instanceof THREE.Object3D) {
      this.track(resource.geometry);
      this.track(resource.material);
      this.track(resource.children);
    }
    return resource;
  }
  ...
}
```

Also because `resource.children` is an array and because `resource.material` can be an array let's check for arrays

```
class ResourceTracker {
  constructor() {
    this.resources = new Set();
  }
  track(resource) {
    if (!resource) {
      return resource;
    }

+    // handle children and when material is an array of materials.
+    if (Array.isArray(resource)) {
+      resource.forEach(resource => this.track(resource));
+      return resource;
+    }

    if (resource.dispose || resource instanceof THREE.Object3D) {
      this.resources.add(resource);
    }
    if (resource instanceof THREE.Object3D) {
      this.track(resource.geometry);
      this.track(resource.material);
      this.track(resource.children);
    }
    return resource;
  }
  ...
}
```

And finally we need to walk the properties and uniforms of a material looking for textures.

```
class ResourceTracker {
  constructor() {
    this.resources = new Set();
  }
  track(resource) {
    if (!resource) {
      return resource;
    }

*    // handle children and when material is an array of materials or
*    // uniform is array of textures
    if (Array.isArray(resource)) {
      resource.forEach(resource => this.track(resource));
      return resource;
    }

    if (resource.dispose || resource instanceof THREE.Object3D) {
      this.resources.add(resource);
    }
    if (resource instanceof THREE.Object3D) {
      this.track(resource.geometry);
      this.track(resource.material);
      this.track(resource.children);
-    }
+    } else if (resource instanceof THREE.Material) {
+      // We have to check if there are any textures on the material
+      for (const value of Object.values(resource)) {
+        if (value instanceof THREE.Texture) {
+          this.track(value);
+        }
+      }
+      // We also have to check if any uniforms reference textures or arrays of textures
+      if (resource.uniforms) {
+        for (const value of Object.values(resource.uniforms)) {
+          if (value) {
+            const uniformValue = value.value;
+            if (uniformValue instanceof THREE.Texture ||
+                Array.isArray(uniformValue)) {
+              this.track(uniformValue);
+            }
+          }
+        }
+      }
+    }
    return resource;
  }
  ...
}
```

And with that let's take an example from [the article on loading gltf files](load-gltf.html) and make it load and free files.

```
const gltfLoader = new GLTFLoader();
function loadGLTF(url) {
  return new Promise((resolve, reject) => {
    gltfLoader.load(url, resolve, undefined, reject);
  });
}

function waitSeconds(seconds = 0) {
  return new Promise(resolve => setTimeout(resolve, seconds * 1000));
}

const fileURLs = [
  'resources/models/cartoon_lowpoly_small_city_free_pack/scene.gltf',
  'resources/models/3dbustchallange_submission/scene.gltf',
  'resources/models/mountain_landscape/scene.gltf',
  'resources/models/simple_house_scene/scene.gltf',
];

async function loadFiles() {
  for (;;) {
    for (const url of fileURLs) {
      const resMgr = new ResourceTracker();
      const track = resMgr.track.bind(resMgr);
      const gltf = await loadGLTF(url);
      const root = track(gltf.scene);
      scene.add(root);

      // compute the box that contains all the stuff
      // from root and below
      const box = new THREE.Box3().setFromObject(root);

      const boxSize = box.getSize(new THREE.Vector3()).length();
      const boxCenter = box.getCenter(new THREE.Vector3());

      // set the camera to frame the box
      frameArea(boxSize * 1.1, boxSize, boxCenter, camera);

      await waitSeconds(2);
      renderer.render(scene, camera);

      resMgr.dispose();

      await waitSeconds(1);

    }
  }
}
loadFiles();
```

and we get

[click here to open in a separate window](/manual/examples/cleanup-loaded-files.html)

Some notes about the code.

If we wanted to load 2 or more files at once and free them at anytime we would use one `ResourceTracker` per file.

Above we are only tracking `gltf.scene` right after loading. Based on our current implementation of `ResourceTracker` that will track all the resources just loaded. If we added more things to the scene we need to decide whether or not to track them.

For example let's say after we loaded a character we put a tool in their hand by making the tool a child of their hand. As it is that tool will not be freed. I'm guessing more often than not this is what we want.

That brings up a point. Originally when I first wrote the `ResourceTracker` above I walked through everything inside the `dispose` method instead of `track`. It was only later as I thought about the tool as a child of hand case above that it became clear that tracking exactly what to free in `track` was more flexible and arguably more correct since we could then track what was loaded from the file rather than just freeing the state of the scene graph later.

I honestly am not 100% happy with `ResourceTracker`. Doing things this way is not common in 3D engines. We shouldn't have to guess what resources were loaded, we should know. It would be nice if three.js changed so that all file loaders returned some standard object with references to all the resources loaded. At least at the moment, three.js doesn't give us any more info when loading a scene so this solution seems to work.

I hope you find this example useful or at least a good reference for what is required to free resources in three.js


Voxel(Minecraft Like) Geometry

Summary: A tutorial on how to efficiently generate voxel-based geometry in Three.js, similar to Minecraft, by avoiding the common mistake of creating one mesh per voxel and instead building custom BufferGeometry with face culling, cells, textures, and editing support.

Introduction

I've seen this topic come up more than once in various places. That is basically, "How do I make a voxel display like Minecraft".

Most people first attempt this by making a cube geometry and then making a mesh at each voxel position. Just for fun I tried this. I made a 16777216 element `Uint8Array` to represent a 256x256x256 cube of voxels.

const cellSize = 256;
const cell = new Uint8Array(cellSize * cellSize * cellSize);

Filling Voxels with Hills

I then made a single layer with a kind of hills of sine waves like this

for (let y = 0; y < cellSize; ++y) {
  for (let z = 0; z < cellSize; ++z) {
    for (let x = 0; x < cellSize; ++x) {
      const height = (Math.sin(x / cellSize * Math.PI * 4) + Math.sin(z / cellSize * Math.PI * 6)) * 20 + cellSize / 2;
      if (height > y && height < y + 1) {
        const offset = y * cellSize * cellSize +
                       z * cellSize +
                       x;
        cell[offset] = 1;
      }
    }
  }
}

Creating a Mesh per Voxel

I then walked through all the cells and if they were not 0 I created a mesh with a cube.

const geometry = new THREE.BoxGeometry(1, 1, 1);
const material = new THREE.MeshPhongMaterial({color: 'green'});

for (let y = 0; y < cellSize; ++y) {
  for (let z = 0; z < cellSize; ++z) {
    for (let x = 0; x < cellSize; ++x) {
      const offset = y * cellSize * cellSize +
                     z * cellSize +
                     x;
      const block = cell[offset];
      const mesh = new THREE.Mesh(geometry, material);
      mesh.position.set(x, y, z);
      scene.add(mesh);
    }
  }
}

Performance Issues

The rest of the code is based on the example from the article on rendering on demand.

[click here to open in a separate window](/manual/examples/voxel-geometry-separate-cubes.html)

It takes a while to start and if you try to move the camera it's likely too slow. Like the article on how to optimize lots of objects the problem is there are just way too many objects. 256x256 is 65536 boxes!

Using the technique of merging the geometry will fix the issue for this example but what if instead of just making a single layer we filled in everything below the ground with voxel. In other words change the loop filling in the voxels to this

for (let y = 0; y < cellSize; ++y) {
  for (let z = 0; z < cellSize; ++z) {
    for (let x = 0; x < cellSize; ++x) {
      const height = (Math.sin(x / cellSize * Math.PI * 4) + Math.sin(z / cellSize * Math.PI * 6)) * 20 + cellSize / 2;
-      if (height > y && height < y + 1) {
+      if (height < y + 1) {
        const offset = y * cellSize * cellSize +
                       z * cellSize +
                       x;
        cell[offset] = 1;
      }
    }
  }
}

Memory and Face Culling

I tried it once just to see the results. It churned for about a minute and then crashed with *out of memory* 😅

There are several issues but the biggest issue is we're making all these faces inside the cubes that we can actually never see.

In other words lets say we have a box of voxels 3x2x2. By merging cubes we're getting this

but we really want this

In the top box there are faces between the voxels. Faces that are a waste since they can't be seen. It's not just one face between each voxel, there are 2 faces, one for each voxel facing its neighbor that are a waste. All these extra faces, especially for a large volume of voxels will kill performance.

It should be clear that we can't just merge geometry. We need to build it ourselves, taking into account that if a voxel has an adjacent neighbor it doesn't need the face facing that neighbor.

The next issue is that 256x256x256 is just too big. 16meg is a lot of memory and if nothing else in much of the space nothing is there so that's a lot of wasted memory. It's also a huge number of voxels, 16 million! That's too much to consider at once.

A solution is to divide the area into smaller areas. Any area that has nothing in it needs no storage. Let's use 32x32x32 areas (that's 32k) and only create an area if something is in it. We'll call one of these larger 32x32x32 areas a "cell".

Let's break this into pieces. First let's make a class to manage the voxel data.

class VoxelWorld {
  constructor(cellSize) {
    this.cellSize = cellSize;
  }
}

Generating Geometry Data for a Cell

Let's make the function that makes geometry for a cell. Let's assume you pass in a cell position. In other words if you want the geometry for the cell that covers voxels (0-31x, 0-31y, 0-31z) then you'd pass in 0,0,0. For the cell that covers voxels (32-63x, 0-31y, 0-31z) you'd pass in 1,0,0.

We need to be able to check the neighboring voxels so let's assume our class has a function `getVoxel` that given a voxel position returns the value of the voxel there. In other words if you pass it 35,0,0 and the cellSize is 32 it's going to look at cell 1,0,0 and in that cell it will look at voxel 3,0,0. Using this function we can look at a voxel's neighboring voxels even if they happen to be in neighboring cells.

class VoxelWorld {
  constructor(cellSize) {
    this.cellSize = cellSize;
  }
+  generateGeometryDataForCell(cellX, cellY, cellZ) {
+    const {cellSize} = this;
+    const startX = cellX * cellSize;
+    const startY = cellY * cellSize;
+    const startZ = cellZ * cellSize;
+
+    for (let y = 0; y < cellSize; ++y) {
+      const voxelY = startY + y;
+      for (let z = 0; z < cellSize; ++z) {
+        const voxelZ = startZ + z;
+        for (let x = 0; x < cellSize; ++x) {
+          const voxelX = startX + x;
+          const voxel = this.getVoxel(voxelX, voxelY, voxelZ);
+          if (voxel) {
+            for (const {dir} of VoxelWorld.faces) {
+              const neighbor = this.getVoxel(
+                  voxelX + dir[0],
+                  voxelY + dir[1],
+                  voxelZ + dir[2]);
+              if (!neighbor) {
+                // this voxel has no neighbor in this direction so we need a face
+                // here.
+              }
+            }
+          }
+        }
+      }
+    }
+  }
}

+VoxelWorld.faces = [
+  { // left
+    dir: [ -1,  0,  0, ],
+  },
+  { // right
+    dir: [  1,  0,  0, ],
+  },
+  { // bottom
+    dir: [  0, -1,  0, ],
+  },
+  { // top
+    dir: [  0,  1,  0, ],
+  },
+  { // back
+    dir: [  0,  0, -1, ],
+  },
+  { // front
+    dir: [  0,  0,  1, ],
+  },
+];

Building the Faces with Positions, Normals, and Indices

So using the code above we know when we need a face. Let's generate the faces.

class VoxelWorld {
  constructor(cellSize) {
    this.cellSize = cellSize;
  }
  generateGeometryDataForCell(cellX, cellY, cellZ) {
    const {cellSize} = this;
+    const positions = [];
+    const normals = [];
+    const indices = [];
    const startX = cellX * cellSize;
    const startY = cellY * cellSize;
    const startZ = cellZ * cellSize;

    for (let y = 0; y < cellSize; ++y) {
      const voxelY = startY + y;
      for (let z = 0; z < cellSize; ++z) {
        const voxelZ = startZ + z;
        for (let x = 0; x < cellSize; ++x) {
          const voxelX = startX + x;
          const voxel = this.getVoxel(voxelX, voxelY, voxelZ);
          if (voxel) {
-            for (const {dir} of VoxelWorld.faces) {
+            for (const {dir, corners} of VoxelWorld.faces) {
              const neighbor = this.getVoxel(
                  voxelX + dir[0],
                  voxelY + dir[1],
                  voxelZ + dir[2]);
              if (!neighbor) {
                // this voxel has no neighbor in this direction so we need a face.
+                const ndx = positions.length / 3;
+                for (const pos of corners) {
+                  positions.push(pos[0] + x, pos[1] + y, pos[2] + z);
+                  normals.push(...dir);
+                }
+                indices.push(
+                  ndx, ndx + 1, ndx + 2,
+                  ndx + 2, ndx + 1, ndx + 3,
+                );
              }
            }
          }
        }
      }
    }
+    return {
+      positions,
+      normals,
+      indices,
    };
  }
}

VoxelWorld.faces = [
  { // left
    dir: [ -1,  0,  0, ],
+    corners: [
+      [ 0, 1, 0 ],
+      [ 0, 0, 0 ],
+      [ 0, 1, 1 ],
+      [ 0, 0, 1 ],
+    ],
  },
  { // right
    dir: [  1,  0,  0, ],
+    corners: [
+      [ 1, 1, 1 ],
+      [ 1, 0, 1 ],
+      [ 1, 1, 0 ],
+      [ 1, 0, 0 ],
+    ],
  },
  { // bottom
    dir: [  0, -1,  0, ],
+    corners: [
+      [ 1, 0, 1 ],
+      [ 0, 0, 1 ],
+      [ 1, 0, 0 ],
+      [ 0, 0, 0 ],
+    ],
  },
  { // top
    dir: [  0,  1,  0, ],
+    corners: [
+      [ 0, 1, 1 ],
+      [ 1, 1, 1 ],
+      [ 0, 1, 0 ],
+      [ 1, 1, 0 ],
+    ],
  },
  { // back
    dir: [  0,  0, -1, ],
+    corners: [
+      [ 1, 0, 0 ],
+      [ 0, 0, 0 ],
+      [ 1, 1, 0 ],
+      [ 0, 1, 0 ],
+    ],
  },
  { // front
    dir: [  0,  0,  1, ],
+    corners: [
+      [ 0, 0, 1 ],
+      [ 1, 0, 1 ],
+      [ 0, 1, 1 ],
+      [ 1, 1, 1 ],
+    ],
  },
];

Supplying the getVoxel Function

The code above would make basic geometry data for us. We just need to supply the `getVoxel` function. Let's start with just one hard coded cell.

class VoxelWorld {
  constructor(cellSize) {
    this.cellSize = cellSize;
+    this.cell = new Uint8Array(cellSize * cellSize * cellSize);
  }
+  getCellForVoxel(x, y, z) {
+    const {cellSize} = this;
+    const cellX = Math.floor(x / cellSize);
+    const cellY = Math.floor(y / cellSize);
+    const cellZ = Math.floor(z / cellSize);
+    if (cellX !== 0 || cellY !== 0 || cellZ !== 0) {
+      return null
+    }
+    return this.cell;
+  }
+  getVoxel(x, y, z) {
+    const cell = this.getCellForVoxel(x, y, z);
+    if (!cell) {
+      return 0;
+    }
+    const {cellSize} = this;
+    const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
+    const voxelY = THREE.MathUtils.euclideanModulo(y, cellSize) | 0;
+    const voxelZ = THREE.MathUtils.euclideanModulo(z, cellSize) | 0;
+    const voxelOffset = voxelY * cellSize * cellSize +
+                        voxelZ * cellSize +
+                        voxelX;
+    return cell[voxelOffset];
+  }
  generateGeometryDataForCell(cellX, cellY, cellZ) {

  ...
}

Adding setVoxel

This seems like it would work. Let's make a `setVoxel` function so we can set some data.

class VoxelWorld {
  constructor(cellSize) {
    this.cellSize = cellSize;
    this.cell = new Uint8Array(cellSize * cellSize * cellSize);
  }
  getCellForVoxel(x, y, z) {
    const {cellSize} = this;
    const cellX = Math.floor(x / cellSize);
    const cellY = Math.floor(y / cellSize);
    const cellZ = Math.floor(z / cellSize);
    if (cellX !== 0 || cellY !== 0 || cellZ !== 0) {
      return null
    }
    return this.cell;
  }
+  setVoxel(x, y, z, v) {
+    let cell = this.getCellForVoxel(x, y, z);
+    if (!cell) {
+      return;  // TODO: add a new cell?
+    }
+    const {cellSize} = this;
+    const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
+    const voxelY = THREE.MathUtils.euclideanModulo(y, cellSize) | 0;
+    const voxelZ = THREE.MathUtils.euclideanModulo(z, cellSize) | 0;
+    const voxelOffset = voxelY * cellSize * cellSize +
+                        voxelZ * cellSize +
+                        voxelX;
+    cell[voxelOffset] = v;
+  }
  getVoxel(x, y, z) {
    const cell = this.getCellForVoxel(x, y, z);
    if (!cell) {
      return 0;
    }
    const {cellSize} = this;
    const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
    const voxelY = THREE.MathUtils.euclideanModulo(y, cellSize) | 0;
    const voxelZ = THREE.MathUtils.euclideanModulo(z, cellSize) | 0;
    const voxelOffset = voxelY * cellSize * cellSize +
                        voxelZ * cellSize +
                        voxelX;
    return cell[voxelOffset];
  }
  generateGeometryDataForCell(cellX, cellY, cellZ) {

  ...
}

Refactoring with computeVoxelOffset

Hmmm, I see a lot of repeated code. Let's fix that up

class VoxelWorld {
  constructor(cellSize) {
    this.cellSize = cellSize;
+    this.cellSliceSize = cellSize * cellSize;
    this.cell = new Uint8Array(cellSize * cellSize * cellSize);
  }
  getCellForVoxel(x, y, z) {
    const {cellSize} = this;
    const cellX = Math.floor(x / cellSize);
    const cellY = Math.floor(y / cellSize);
    const cellZ = Math.floor(z / cellSize);
    if (cellX !== 0 || cellY !== 0 || cellZ !== 0) {
      return null;
    }
    return this.cell;
  }
+  computeVoxelOffset(x, y, z) {
+    const {cellSize, cellSliceSize} = this;
+    const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
+    const voxelY = THREE.MathUtils.euclideanModulo(y, cellSize) | 0;
+    const voxelZ = THREE.MathUtils.euclideanModulo(z, cellSize) | 0;
+    return voxelY * cellSliceSize +
+           voxelZ * cellSize +
+           voxelX;
+  }
  setVoxel(x, y, z, v) {
    const cell = this.getCellForVoxel(x, y, z);
    if (!cell) {
      return;  // TODO: add a new cell?
    }
-    const {cellSize} = this;
-    const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
-    const voxelY = THREE.MathUtils.euclideanModulo(y, cellSize) | 0;
-    const voxelZ = THREE.MathUtils.euclideanModulo(z, cellSize) | 0;
-    const voxelOffset = voxelY * cellSize * cellSize +
-                        voxelZ * cellSize +
-                        voxelX;
+    const voxelOffset = this.computeVoxelOffset(x, y, z);
    cell[voxelOffset] = v;
  }
  getVoxel(x, y, z) {
    const cell = this.getCellForVoxel(x, y, z);
    if (!cell) {
      return 0;
    }
-    const {cellSize} = this;
-    const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
-    const voxelY = THREE.MathUtils.euclideanModulo(y, cellSize) | 0;
-    const voxelZ = THREE.MathUtils.euclideanModulo(z, cellSize) | 0;
-    const voxelOffset = voxelY * cellSize * cellSize +
-                        voxelZ * cellSize +
-                        voxelX;
+    const voxelOffset = this.computeVoxelOffset(x, y, z);
    return cell[voxelOffset];
  }
  generateGeometryDataForCell(cellX, cellY, cellZ) {

  ...
}

Filling the First Cell

Now let's make some code to fill out the first cell with voxels.

const cellSize = 32;

const world = new VoxelWorld(cellSize);

for (let y = 0; y < cellSize; ++y) {
  for (let z = 0; z < cellSize; ++z) {
    for (let x = 0; x < cellSize; ++x) {
      const height = (Math.sin(x / cellSize * Math.PI * 2) + Math.sin(z / cellSize * Math.PI * 3)) * (cellSize / 6) + (cellSize / 2);
      if (y < height) {
        world.setVoxel(x, y, z, 1);
      }
    }
  }
}

Generating BufferGeometry from Voxel Data

and some code to actually generate geometry like we covered in the article on custom BufferGeometry.

const {positions, normals, indices} = world.generateGeometryDataForCell(0, 0, 0);
const geometry = new THREE.BufferGeometry();
const material = new THREE.MeshLambertMaterial({color: 'green'});

const positionNumComponents = 3;
const normalNumComponents = 3;
geometry.setAttribute(
    'position',
    new THREE.BufferAttribute(new Float32Array(positions), positionNumComponents));
geometry.setAttribute(
    'normal',
    new THREE.BufferAttribute(new Float32Array(normals), normalNumComponents));
geometry.setIndex(indices);
const mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);

Adding Textures

let's try it

[click here to open in a separate window](/manual/examples/voxel-geometry-culled-faces.html)

That seems to be working! Okay, let's add in textures.

Searching on the net I found [this set](https://www.minecraftforum.net/forums/mapping-and-modding-java-edition/resource-packs/1245961-16x-1-7-4-wip-flourish) of [CC-BY-NC-SA](https://creativecommons.org/licenses/by-nc-sa/4.0/) licensed minecraft textures by [Joshtimus](https://www.minecraftforum.net/members/Joshtimus). I picked a few at random and built this [texture atlas](https://www.google.com/search?q=texture+atlas).

<img src="../examples/resources/images/minecraft/flourish-cc-by-nc-sa.png">

To make things simple they are arranged a voxel type per column where the top row is the side of a voxel. The 2nd row is the top of voxel, and the 3rd row is the bottom of the voxel.

Knowing that we can add info to our `VoxelWorld.faces` data to specify for each face which row to use and the UVs to use for that face.

Updating Geometry Generation for UVs

And we can update the code to use that data. We need to know the size of a tile in the texture atlas and the dimensions of the texture.

class VoxelWorld {
-  constructor(cellSize) {
-    this.cellSize = cellSize;
+  constructor(options) {
+    this.cellSize = options.cellSize;
+    this.tileSize = options.tileSize;
+    this.tileTextureWidth = options.tileTextureWidth;
+    this.tileTextureHeight = options.tileTextureHeight;
+    const {cellSize} = this;
+    this.cellSliceSize = cellSize * cellSize;
+    this.cell = new Uint8Array(cellSize * cellSize * cellSize);
  }

  ...

  generateGeometryDataForCell(cellX, cellY, cellZ) {
-    const {cellSize} = this;
+    const {cellSize, tileSize, tileTextureWidth, tileTextureHeight} = this;
    const positions = [];
    const normals = [];
+    const uvs = [];
    const indices = [];
    const startX = cellX * cellSize;
    const startY = cellY * cellSize;
    const startZ = cellZ * cellSize;

    for (let y = 0; y < cellSize; ++y) {
      const voxelY = startY + y;
      for (let z = 0; z < cellSize; ++z) {
        const voxelZ = startZ + z;
        for (let x = 0; x < cellSize; ++x) {
          const voxelX = startX + x;
          const voxel = this.getVoxel(voxelX, voxelY, voxelZ);
          if (voxel) {
            const uvVoxel = voxel - 1;  // voxel 0 is sky so for UVs we start at 0
            // There is a voxel here but do we need faces for it?
-            for (const {dir, corners} of VoxelWorld.faces) {
+            for (const {dir, corners, uvRow} of VoxelWorld.faces) {
              const neighbor = this.getVoxel(
                  voxelX + dir[0],
                  voxelY + dir[1],
                  voxelZ + dir[2]);
              if (!neighbor) {
                // this voxel has no neighbor in this direction so we need a face.
                const ndx = positions.length / 3;
-                for (const pos of corners) {
+                for (const {pos, uv} of corners) {
                  positions.push(pos[0] + x, pos[1] + y, pos[2] + z);
                  normals.push(...dir);
+                  uvs.push(
+                        (uvVoxel +   uv[0]) * tileSize / tileTextureWidth,
+                    1 - (uvRow + 1 - uv[1]) * tileSize / tileTextureHeight);
                }
                indices.push(
                  ndx, ndx + 1, ndx + 2,
                  ndx + 2, ndx + 1, ndx + 3,
                );
              }
            }
          }
        }
      }
    }

    return {
      positions,
      normals,
      uvs,
      indices,
    };
  }
}

Loading and Configuring the Texture

We then need to load the texture

const loader = new THREE.TextureLoader();
const texture = loader.load('resources/images/minecraft/flourish-cc-by-nc-sa.png', render);
texture.magFilter = THREE.NearestFilter;
texture.minFilter = THREE.NearestFilter;
texture.colorSpace = THREE.SRGBColorSpace;

Passing Settings to VoxelWorld

and pass the settings to the `VoxelWorld` class

+const tileSize = 16;
+const tileTextureWidth = 256;
+const tileTextureHeight = 64;
-const world = new VoxelWorld(cellSize);
+const world = new VoxelWorld({
+  cellSize,
+  tileSize,
+  tileTextureWidth,
+  tileTextureHeight,
+});

Using UVs and Texture in the Material

Let's actually use the UVs when we create the geometry and the texture when we make the material

-const {positions, normals, indices} = world.generateGeometryDataForCell(0, 0, 0);
+const {positions, normals, uvs, indices} = world.generateGeometryDataForCell(0, 0, 0);
const geometry = new THREE.BufferGeometry();
-const material = new THREE.MeshLambertMaterial({color: 'green'});
+const material = new THREE.MeshLambertMaterial({
+  map: texture,
+  side: THREE.DoubleSide,
+  alphaTest: 0.1,
+  transparent: true,
+});

const positionNumComponents = 3;
const normalNumComponents = 3;
+const uvNumComponents = 2;
geometry.setAttribute(
    'position',
    new THREE.BufferAttribute(new Float32Array(positions), positionNumComponents));
geometry.setAttribute(
    'normal',
    new THREE.BufferAttribute(new Float32Array(normals), normalNumComponents));
+geometry.setAttribute(
+    'uv',
+    new THREE.BufferAttribute(new Float32Array(uvs), uvNumComponents));
geometry.setIndex(indices);
const mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);

Using Multiple Voxel Types

One last thing, we actually need to set some voxels to use different textures.

for (let y = 0; y < cellSize; ++y) {
  for (let z = 0; z < cellSize; ++z) {
    for (let x = 0; x < cellSize; ++x) {
      const height = (Math.sin(x / cellSize * Math.PI * 2) + Math.sin(z / cellSize * Math.PI * 3)) * (cellSize / 6) + (cellSize / 2);
      if (y < height) {
-        world.setVoxel(x, y, z, 1);
+        world.setVoxel(x, y, z, randInt(1, 17));
      }
    }
  }
}

+function randInt(min, max) {
+  return Math.floor(Math.random() * (max - min) + min);
+}

Result with Textures

and with that we get textures!

[click here to open in a separate window](/manual/examples/voxel-geometry-culled-faces-with-textures.html)

Supporting Multiple Cells

Let's make it support more than one cell.

To do this lets store cells in an object using cell ids. A cell id will just be a cell's coordinates separated by a comma. In other words if we ask for voxel 35,0,0 that is in cell 1,0,0 so its id is `"1,0,0"`.

class VoxelWorld {
  constructor(options) {
    this.cellSize = options.cellSize;
    this.tileSize = options.tileSize;
    this.tileTextureWidth = options.tileTextureWidth;
    this.tileTextureHeight = options.tileTextureHeight;
    const {cellSize} = this;
    this.cellSliceSize = cellSize * cellSize;
-    this.cell = new Uint8Array(cellSize * cellSize * cellSize);
+    this.cells = {};
  }
+  computeCellId(x, y, z) {
+    const {cellSize} = this;
+    const cellX = Math.floor(x / cellSize);
+    const cellY = Math.floor(y / cellSize);
+    const cellZ = Math.floor(z / cellSize);
+    return ${cellX},${cellY},${cellZ};
+  }
+  getCellForVoxel(x, y, z) {
-    const cellX = Math.floor(x / cellSize);
-    const cellY = Math.floor(y / cellSize);
-    const cellZ = Math.floor(z / cellSize);
-    if (cellX !== 0 || cellY !== 0 || cellZ !== 0) {
-      return null;
-    }
-    return this.cell;
+    return this.cells[this.computeCellId(x, y, z)];
  }

   ...
}

Adding New Cells Automatically

and now we can make `setVoxel` add new cells if we try to set a voxel in a cell that does not yet exist

setVoxel(x, y, z, v) {
-    const cell = this.getCellForVoxel(x, y, z);
+    let cell = this.getCellForVoxel(x, y, z);
    if (!cell) {
-      return 0;
+      cell = this.addCellForVoxel(x, y, z);
    }
    const voxelOffset = this.computeVoxelOffset(x, y, z);
    cell[voxelOffset] = v;
  }
+  addCellForVoxel(x, y, z) {
+    const cellId = this.computeCellId(x, y, z);
+    let cell = this.cells[cellId];
+    if (!cell) {
+      const {cellSize} = this;
+      cell = new Uint8Array(cellSize * cellSize * cellSize);
+      this.cells[cellId] = cell;
+    }
+    return cell;
+  }

Adding the UI

Let's make this editable.

First we`ll add a UI. Using radio buttons we can make an 8x2 array of tiles

  <canvas id="c"></canvas>
+

+

+      <input type="radio" name="voxel" id="voxel1" value="1">
+      <input type="radio" name="voxel" id="voxel2" value="2">
+      <input type="radio" name="voxel" id="voxel3" value="3">
+      <input type="radio" name="voxel" id="voxel4" value="4">
+      <input type="radio" name="voxel" id="voxel5" value="5">
+      <input type="radio" name="voxel" id="voxel6" value="6">
+      <input type="radio" name="voxel" id="voxel7" value="7">
+      <input type="radio" name="voxel" id="voxel8" value="8">
+

+

+      <input type="radio" name="voxel" id="voxel9"  value="9" >
+      <input type="radio" name="voxel" id="voxel10" value="10">
+      <input type="radio" name="voxel" id="voxel11" value="11">
+      <input type="radio" name="voxel" id="voxel12" value="12">
+      <input type="radio" name="voxel" id="voxel13" value="13">
+      <input type="radio" name="voxel" id="voxel14" value="14">
+      <input type="radio" name="voxel" id="voxel15" value="15">
+      <input type="radio" name="voxel" id="voxel16" value="16">
+

+

CSS for the UI

And add some CSS to style it, display the tiles and highlight the current selection

body {
    margin: 0;
}
#c {
    width: 100%;
    height: 100%;
    display: block;
}
+#ui {
+    position: absolute;
+    left: 10px;
+    top: 10px;
+    background: rgba(0, 0, 0, 0.8);
+    padding: 5px;
+}
+#ui input[type=radio] {
+  width: 0;
+  height: 0;
+  display: none;
+}
+#ui input[type=radio] + label {
+  background-image: url('resources/images/minecraft/flourish-cc-by-nc-sa.png');
+  background-size: 1600% 400%;
+  image-rendering: pixelated;
+  width: 64px;
+  height: 64px;
+  display: inline-block;
+}
+#ui input[type=radio]:checked + label {
+  outline: 3px solid red;
+}
+@media (max-width: 600px), (max-height: 600px) {
+  #ui input[type=radio] + label {
+    width: 32px;
+    height: 32px;
+  }
+}

UX Behavior

The UX will be as follows. If no tile is selected and you click a voxel that voxel will be erased or if you click a voxel and are holding the shift key it will be erased. Otherwise if a tiles is selected it will be added. You can deselect the selected tile type by clicking it again.

This code will let the user unselect the highlighted radio button.

let currentVoxel = 0;
let currentId;

document.querySelectorAll('#ui .tiles input[type=radio][name=voxel]').forEach((elem) => {
  elem.addEventListener('click', allowUncheck);
});

function allowUncheck() {
  if (this.id === currentId) {
    this.checked = false;
    currentId = undefined;
    currentVoxel = 0;
  } else {
    currentId = this.id;
    currentVoxel = parseInt(this.value);
  }
}

Placing Voxels on Click

And this below code will let us set a voxel based on where the user clicks. It uses code similar to the code we made in the article on picking but it's not using the built in `RayCaster`. Instead it's using `VoxelWorld.intersectRay` which returns the position of intersection and the normal of the face hit.

function getCanvasRelativePosition(event) {
  const rect = canvas.getBoundingClientRect();
  return {
    x: (event.clientX - rect.left) * canvas.width  / rect.width,
    y: (event.clientY - rect.top ) * canvas.height / rect.height,
  };
}

function placeVoxel(event) {
  const pos = getCanvasRelativePosition(event);
  const x = (pos.x / canvas.width ) *  2 - 1;
  const y = (pos.y / canvas.height) * -2 + 1;  // note we flip Y

  const start = new THREE.Vector3();
  const end = new THREE.Vector3();
  start.setFromMatrixPosition(camera.matrixWorld);
  end.set(x, y, 1).unproject(camera);

  const intersection = world.intersectRay(start, end);
  if (intersection) {
    const voxelId = event.shiftKey ? 0 : currentVoxel;
    // the intersection point is on the face. That means
    // the math imprecision could put us on either side of the face.
    // so go half a normal into the voxel if removing (currentVoxel = 0)
    // our out of the voxel if adding (currentVoxel  > 0)
    const pos = intersection.position.map((v, ndx) => {
      return v + intersection.normal[ndx] * (voxelId > 0 ? 0.5 : -0.5);
    });
    world.setVoxel(...pos, voxelId);
    updateVoxelGeometry(...pos);
    requestRenderIfNotRequested();
  }
}

const mouse = {
  x: 0,
  y: 0,
};

function recordStartPosition(event) {
  mouse.x = event.clientX;
  mouse.y = event.clientY;
  mouse.moveX = 0;
  mouse.moveY = 0;
}
function recordMovement(event) {
  mouse.moveX += Math.abs(mouse.x - event.clientX);
  mouse.moveY += Math.abs(mouse.y - event.clientY);
}
function placeVoxelIfNoMovement(event) {
  if (mouse.moveX < 5 && mouse.moveY < 5) {
    placeVoxel(event);
  }
  window.removeEventListener('pointermove', recordMovement);
  window.removeEventListener('pointerup', placeVoxelIfNoMovement);
}
canvas.addEventListener('pointerdown', (event) => {
  event.preventDefault();
  recordStartPosition(event);
  window.addEventListener('pointermove', recordMovement);
  window.addEventListener('pointerup', placeVoxelIfNoMovement);
}, {passive: false});
canvas.addEventListener('touchstart', (event) => {
  // stop scrolling
  event.preventDefault();
}, {passive: false});

Mouse Behavior Explanation

There's a lot going on in the code above. Basically the mouse has a dual purpose. One is to move the camera. The other is to edit the world. Placing/Erasing a voxel happen when you let off the mouse but only if you have not moved the mouse since you first pressed down. This is just a guess that if you did move the mouse you were trying to move the camera, not place a block. `moveX` and `moveY` are in absolute movement so if you move to the left 10 and then back to the right 10 you'll have moved 20 units. In that case the user likely was just rotating the model back and forth and does not want to place a block. I didn't do any testing to see if `5` is a good range or not.

In the code we call `world.setVoxel` to set a voxel and then `updateVoxelGeometry` to update the three.js geometry based on what's changed.

Let's make that now. If the user clicks a voxel on the edge of a cell then the geometry for the voxel in the adjacent cell might need new geometry. This means we need to check the cell for the voxel we just edited as well as in all 6 directions from that cell.

const neighborOffsets = [
  [ 0,  0,  0], // self
  [-1,  0,  0], // left
  [ 1,  0,  0], // right
  [ 0, -1,  0], // down
  [ 0,  1,  0], // up
  [ 0,  0, -1], // back
  [ 0,  0,  1], // front
];
function updateVoxelGeometry(x, y, z) {
  const updatedCellIds = {};
  for (const offset of neighborOffsets) {
    const ox = x + offset[0];
    const oy = y + offset[1];
    const oz = z + offset[2];
    const cellId = world.computeCellId(ox, oy, oz);
    if (!updatedCellIds[cellId]) {
      updatedCellIds[cellId] = true;
      updateCellGeometry(ox, oy, oz);
    }
  }
}

Simplifying Adjacent Cell Updates

I thought about checking for adjacent cells like

```
const voxelX = THREE.MathUtils.euclideanModulo(x, cellSize) | 0;
if (voxelX === 0) {
  // update cell to the left
} else if (voxelX === cellSize - 1) {
  // update cell to the right
}
```

and there would be 4 more checks for the other 4 directions but it occurred to me the code would be much simpler with just an array of offsets and saving off the cell ids of the cells we already updated. If the updated voxel is not on the edge of a cell then the test will quickly reject updating the same cell.

For `updateCellGeometry` we're just going to take the code we had before that was generating the geometry for one cell and make it handle multiple cells.

const cellIdToMesh = {};
function updateCellGeometry(x, y, z) {
  const cellX = Math.floor(x / cellSize);
  const cellY = Math.floor(y / cellSize);
  const cellZ = Math.floor(z / cellSize);
  const cellId = world.computeCellId(x, y, z);
  let mesh = cellIdToMesh[cellId];
  const geometry = mesh ? mesh.geometry : new THREE.BufferGeometry();

  const {positions, normals, uvs, indices} = world.generateGeometryDataForCell(cellX, cellY, cellZ);
  const positionNumComponents = 3;
  geometry.setAttribute('position', new THREE.BufferAttribute(new Float32Array(positions), positionNumComponents));
  const normalNumComponents = 3;
  geometry.setAttribute('normal', new THREE.BufferAttribute(new Float32Array(normals), normalNumComponents));
  const uvNumComponents = 2;
  geometry.setAttribute('uv', new THREE.BufferAttribute(new Float32Array(uvs), uvNumComponents));
  geometry.setIndex(indices);
  geometry.computeBoundingSphere();

  if (!mesh) {
    mesh = new THREE.Mesh(geometry, material);
    mesh.name = cellId;
    cellIdToMesh[cellId] = mesh;
    scene.add(mesh);
    mesh.position.set(cellX * cellSize, cellY * cellSize, cellZ * cellSize);
  }
}

Final Result

The code above checks a map of cell ids to meshes. If we ask for a cell that doesn't exist a new [`Mesh`](/docs/#api/en/objects/Mesh) is made and added to the correct place in world space. At the end we update the attributes and indices with the new data.

[click here to open in a separate window](/manual/examples/voxel-geometry-culled-faces-ui.html)

Notes and Further Considerations

Some notes:

`RayCaster` might have worked just fine. I didn't try it. Instead I found [a voxel specific raycaster](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.42.3443&rep=rep1&type=pdf). that is optimized for voxels.

I made `intersectRay` part of VoxelWorld because it seemed like if it gets too slow we could raycast against cells before raycasting on voxels as a simple speed up if it becomes too slow.

You might want to change the length of the raycast as currently it's all the way to Z-far. I expect if the user clicks something too far way they don't really want to be placing blocks on the other side of the world that are 1 or 2 pixel large.

Calling `geometry.computeBoundingSphere` might be slow. We could just manually set the bounding sphere to the fit the entire cell.

Do we want remove cells if all voxels in that cell are 0? That would probably be reasonable change if we wanted to ship this.

Thinking about how this works it's clear the absolute worst case is a checkerboard of on and off voxels. I don't know off the top of my head what other strategies to use if things get too slow. Maybe getting too slow would just encourage the user not to make giant checkerboard areas.

To keep it simple the texture atlas is just 1 column per voxel type. It would be better to make something more flexible where we have a table of voxel types and each type can specify where its face textures are in the atlas. As it is lots of space is wasted.

Looking at real minecraft there are tiles that are not voxels, not cubes. Like a fence tile or flowers. To do that we'd again need some table of voxel types and for each voxel whether it's a cube or some other geometry. If it's not a cube the neighbor check when generating the geometry would also need to change. A flower voxel next to another voxel should not remove the faces between them.

If you want to make some minecraft like thing using three.js I hope this has given you some ideas where to start and how to generate some what efficient geometry.


Making a Game

Summary: A comprehensive guide on building a game with three.js, covering model loading with LoadingManager, animation playback, Entity Component System architecture, input systems, camera frustum checks, and a Finite State Machine for AI behavior.

Introduction

Many people want to write games using three.js. This article will hopefully give you some ideas on how to start.

At least at the time I'm writing this article it's probably going to be the longest article on this site. It's possible the code here is massively over engineered but as I wrote each new feature I'd run into a problem that needed a solution I'm used to from other games I've written. In other words each new solution seemed important so I'll try to show why. Of course the smaller your game the less you might need some of the solutions shown here but this is a pretty small game and yet with the complexities of 3D characters many things take more organization than they might with 2D characters.

As an example if you're making PacMan in 2D, when PacMan turns a corner that happens instantly at 90 degrees. There is no in-between step. But in a 3D game often we need the character to rotate over several frames. That simple change can add a bunch of complexity and require different solutions.

The majority of the code here will not really be three.js and that's important to note, three.js is not a game engine. Three.js is a 3D library. It provides a scene graph and features for displaying 3D objects added to that scene graph but it does not provide all the other things needed to make a game. No collisions, no physics, no input systems, no path finding, etc, etc... So, we'll have to provide those things ourselves.

I ended up writing quite a bit of code to make this simple unfinished game like thing and again, it's certainly possible I over engineered and there are simpler solutions but I feel like I actually didn't write enough code and hopefully I can explain what I think is missing.

Many of the ideas here are heavily influenced by Unity. If you're not familiar with Unity that probably does not matter. I only bring it up as 10s of 1000s of games have shipped using these ideas.

Let's start with the three.js parts. We need to load models for our game.

At opengameart.org I found this animated knight model by quaternius

quaternius also made these animated animals.

These seem like good models to start with so the first thing we need to do is load them.

Loading Models with LoadingManager

We covered loading glTF files before. The difference this time is we need to load multiple models and we can't start the game until all the models are loaded.

Fortunately three.js provides the LoadingManager just for this purpose. We create a LoadingManager and pass it to the other loaders. The LoadingManager provides both onProgress and onLoad properties we can attach callbacks to. The onLoad callback will be called when all files have been loaded. The onProgress callback as called after each individual file arrives to give as a chance to show loading progress.

Starting with the code from loading a glTF file I removed all the code related to framing the scene and added this code to load all models.

This code will load all the models above and the LoadingManager will call init when done. We'll use the models object later to let us access the loaded models so the GLTFLoader callback for each individual model attaches the loaded data to that model's info.

const manager = new THREE.LoadingManager();
manager.onLoad = init;
const models = {
  pig:    { url: 'resources/models/animals/Pig.gltf' },
  cow:    { url: 'resources/models/animals/Cow.gltf' },
  llama:  { url: 'resources/models/animals/Llama.gltf' },
  pug:    { url: 'resources/models/animals/Pug.gltf' },
  sheep:  { url: 'resources/models/animals/Sheep.gltf' },
  zebra:  { url: 'resources/models/animals/Zebra.gltf' },
  horse:  { url: 'resources/models/animals/Horse.gltf' },
  knight: { url: 'resources/models/knight/KnightCharacter.gltf' },
};
{
  const gltfLoader = new GLTFLoader(manager);
  for (const model of Object.values(models)) {
    gltfLoader.load(model.url, (gltf) => {
      model.gltf = gltf;
    });
  }
}

function init() {
  // TBD
}

Adding a Progress Bar

All the models with all their animation are currently about 6.6meg. That's a pretty big download. Assuming your server supports compression (the server this site runs on does) it's able to compress them to around 1.4meg. That's definitely better than 6.6meg bit it's still not a tiny amount of data. It would probably be good if we added a progress bar so the user has some idea how much longer they have to wait.

So, let's add an onProgress callback. It will be called with 3 arguments, the url of the last loaded object and then the number of items loaded so far as well as the total number of items.

Let's setup some HTML for a loading bar

We'll look up the #progressbar div and we can set the width from 0% to 100% to show our progress. All we need to do is set that in our callback.

We already setup init to be called when all the models are loaded so we can turn off the progress bar by hiding the #loading element.

Here's a bunch of CSS for styling the bar. The CSS makes the #loading div the full size of the page and centers its children. The CSS makes a .progress area to contain the progress bar. The CSS also gives the progress bar a CSS animation of diagonal stripes.

  <canvas id="c"></canvas>
+

+

+
...loading...

+

+

+

const manager = new THREE.LoadingManager();
manager.onLoad = init;

+const progressbarElem = document.querySelector('#progressbar');
+manager.onProgress = (url, itemsLoaded, itemsTotal) => {
+  progressbarElem.style.width = ${itemsLoaded / itemsTotal * 100 | 0}%;
+};

function init() {
+  // hide the loading bar
+  const loadingElem = document.querySelector('#loading');
+  loadingElem.style.display = 'none';
}

#loading {
  position: absolute;
  left: 0;
  top: 0;
  width: 100%;
  height: 100%;
  display: flex;
  align-items: center;
  justify-content: center;
  text-align: center;
  font-size: xx-large;
  font-family: sans-serif;
}
#loading>div>div {
  padding: 2px;
}
.progress {
  width: 50vw;
  border: 1px solid black;
}
#progressbar {
  width: 0;
  transition: width ease-out .5s;
  height: 1em;
  background-color: #888;
  background-image: linear-gradient(
    -45deg,
    rgba(255, 255, 255, .5) 25%,
    transparent 25%,
    transparent 50%,
    rgba(255, 255, 255, .5) 50%,
    rgba(255, 255, 255, .5) 75%,
    transparent 75%,
    transparent
  );
  background-size: 50px 50px;
  animation: progressanim 2s linear infinite;
}

@keyframes progressanim {
  0% {
    background-position: 50px 50px;
  }
  100% {
    background-position: 0 0;
  }
}

Prepping Models and Animations

Now that we have a progress bar let's deal with the models. These models have animations and we want to be able to access those animations. Animations are stored in an array by default be we'd like to be able to easily access them by name so let's setup an animations property for each model to do that. Note of course this means animations must have unique names.

+function prepModelsAndAnimations() {
+  Object.values(models).forEach(model => {
+    const animsByName = {};
+    model.gltf.animations.forEach((clip) => {
+      animsByName[clip.name] = clip;
+    });
+    model.animations = animsByName;
+  });
+}

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

+  prepModelsAndAnimations();
}

Displaying Animated Models with SkeletonUtils.clone

Let's display the animated models.

Unlike the previous example of loading a glTF file This time we probably want to be able to display more than one instance of each model. To do this, instead of adding the loaded gltf scene directly like we did in the article on loading a glTF, we instead want to clone the scene and in particular we want to clone it for skinned animated characters. Fortunately there's a utility function, SkeletonUtils.clone we can use to do this. So, first we need to include the utils.

Then we can clone the models we just loaded

Above, for each model, we clone the gltf.scene we loaded and we parent that to a new Object3D. We need to parent it to another object because when we play animations the animation will apply animated positions to the nodes in the loaded scene which means we won't have control over those positions.

import * as THREE from 'three';
import {OrbitControls} from 'three/addons/controls/OrbitControls.js';
import {GLTFLoader} from 'three/addons/loaders/GLTFLoader.js';
+import * as SkeletonUtils from 'three/addons/utils/SkeletonUtils.js';

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

  prepModelsAndAnimations();

+  Object.values(models).forEach((model, ndx) => {
+    const clonedScene = SkeletonUtils.clone(model.gltf.scene);
+    const root = new THREE.Object3D();
+    root.add(clonedScene);
+    scene.add(root);
+    root.position.x = (ndx - 3) * 3;
+  });
}

Playing Animations with AnimationMixer

To play the animations each model we clone needs an AnimationMixer. An AnimationMixer contains 1 or more AnimationActions. An AnimationAction references an AnimationClip. AnimationActions have all kinds of settings for playing then chaining to another action or cross fading between actions. Let's just get the first AnimationClip and create an action for it. The default is for an action to play its clip in a loop forever.

We called play to start the action and stored off all the AnimationMixers in an array called mixers. Finally we need to update each AnimationMixer in our render loop by computing the time since the last frame and passing that to AnimationMixer.update.

And with that we should get each model loaded and playing its first animation.

+const mixers = [];

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

  prepModelsAndAnimations();

  Object.values(models).forEach((model, ndx) => {
    const clonedScene = SkeletonUtils.clone(model.gltf.scene);
    const root = new THREE.Object3D();
    root.add(clonedScene);
    scene.add(root);
    root.position.x = (ndx - 3) * 3;

+    const mixer = new THREE.AnimationMixer(clonedScene);
+    const firstClip = Object.values(model.animations)[0];
+    const action = mixer.clipAction(firstClip);
+    action.play();
+    mixers.push(mixer);
  });
}

+let then = 0;
function render(now) {
+  now *= 0.001;  // convert to seconds
+  const deltaTime = now - then;
+  then = now;

  if (resizeRendererToDisplaySize(renderer)) {
    const canvas = renderer.domElement;
    camera.aspect = canvas.clientWidth / canvas.clientHeight;
    camera.updateProjectionMatrix();
  }

+  for (const mixer of mixers) {
+    mixer.update(deltaTime);
+  }

  renderer.render(scene, camera);

  requestAnimationFrame(render);
}

Cycling Through Animations

Let's make it so we can check all of the animations. We'll add all of the clips as actions and then enable just one at a time.

The code above makes an array of AnimationActions, one for each AnimationClip. It makes an array of objects, mixerInfos, with references to the AnimationMixer and all the AnimationActions for each model. It then calls playNextAction which sets enabled on all but one action for that mixer.

We need to update the render loop for the new array

Let's make it so pressing a key 1 to 8 will play the next animation for each model

Now you should be able to click on the example and then press keys 1 through 8 to cycle each of the models through their available animations.

-const mixers = [];
+const mixerInfos = [];

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

  prepModelsAndAnimations();

  Object.values(models).forEach((model, ndx) => {
    const clonedScene = SkeletonUtils.clone(model.gltf.scene);
    const root = new THREE.Object3D();
    root.add(clonedScene);
    scene.add(root);
    root.position.x = (ndx - 3) * 3;

    const mixer = new THREE.AnimationMixer(clonedScene);
-    const firstClip = Object.values(model.animations)[0];
-    const action = mixer.clipAction(firstClip);
-    action.play();
-    mixers.push(mixer);
+    const actions = Object.values(model.animations).map((clip) => {
+      return mixer.clipAction(clip);
+    });
+    const mixerInfo = {
+      mixer,
+      actions,
+      actionNdx: -1,
+    };
+    mixerInfos.push(mixerInfo);
+    playNextAction(mixerInfo);
  });
}

+function playNextAction(mixerInfo) {
+  const {actions, actionNdx} = mixerInfo;
+  const nextActionNdx = (actionNdx + 1) % actions.length;
+  mixerInfo.actionNdx = nextActionNdx;
+  actions.forEach((action, ndx) => {
+    const enabled = ndx === nextActionNdx;
+    action.enabled = enabled;
+    if (enabled) {
+      action.play();
+    }
+  });
+}

-for (const mixer of mixers) {
+for (const {mixer} of mixerInfos) {
  mixer.update(deltaTime);
}

window.addEventListener('keydown', (e) => {
  const mixerInfo = mixerInfos[e.keyCode - 49];
  if (!mixerInfo) {
    return;
  }
  playNextAction(mixerInfo);
});

Entity Component System: GameObject and Component

So that is arguably the sum-total of the three.js portion of this article. We covered loading multiple files, cloning skinned models, and playing animations on them. In a real game you'd have to do a ton more manipulation of AnimationAction objects.

Let's start making a game infrastructure

A common pattern for making a modern game is to use an Entity Component System. In an Entity Component System an object in a game is called an entity that consists of a bunch of components. You build up entities by deciding which components to attach to them. So, let's make an Entity Component System.

We'll call our entities GameObject. It's effectively just a collection of components and a three.js Object3D.

Calling GameObject.update calls update on all the components.

I included a name only to help in debugging so if I look at a GameObject in the debugger I can see a name to help identify it.

Some things that might seem a little strange:

GameObject.addComponent is used to create components. Whether or not this a good idea or a bad idea I'm not sure. My thinking was it makes no sense for a component to exist outside of a gameobject so I thought it might be good if creating a component automatically added that component to the gameobject and passed the gameobject to the component's constructor. In other words to add a component you do this

If I didn't do it this way you'd instead do something like this

Is it better that the first way is shorter and more automated or is it worse because it looks out of the ordinary? I don't know.

GameObject.getComponent looks up components by type. That has the implication that you can not have 2 components of the same type on a single game object or at least if you do you can only look up the first one without adding some other API.

It's common for one component to look up another and when looking them up they have to match by type otherwise you might get the wrong one. We could instead give each component a name and you could look them up by name. That would be more flexible in that you could have more than one component of the same type but it would also be more tedious. Again, I'm not sure which is better.

On to the components themselves. Here is their base class.

Do components need a base class? JavaScript is not like most strictly typed languages so effectively we could have no base class and just leave it up to each component to do whatever it wants in its constructor knowing that the first argument is always the component's gameobject. If it doesn't care about gameobject it wouldn't store it. I kind of feel like this common base is good though. It means if you have a reference to a component you know you can find its parent gameobject always and from its parent you can easily look up other components as well as look at its transform.

function removeArrayElement(array, element) {
  const ndx = array.indexOf(element);
  if (ndx >= 0) {
    array.splice(ndx, 1);
  }
}

class GameObject {
  constructor(parent, name) {
    this.name = name;
    this.components = [];
    this.transform = new THREE.Object3D();
    parent.add(this.transform);
  }
  addComponent(ComponentType, ...args) {
    const component = new ComponentType(this, ...args);
    this.components.push(component);
    return component;
  }
  removeComponent(component) {
    removeArrayElement(this.components, component);
  }
  getComponent(ComponentType) {
    return this.components.find(c => c instanceof ComponentType);
  }
  update() {
    for (const component of this.components) {
      component.update();
    }
  }
}

const gameObject = new GameObject(scene, 'foo');
gameObject.addComponent(TypeOfComponent);

const gameObject = new GameObject(scene, 'foo');
const component = new TypeOfComponent(gameObject);
gameObject.addComponent(component);

// Base for all components
class Component {
  constructor(gameObject) {
    this.gameObject = gameObject;
  }
  update() {
  }
}

GameObjectManager with SafeArray

To manage the gameobjects we probably need some kind of gameobject manager. You might think we could just keep an array of gameobjects but in a real game the components of a gameobject might add and remove other gameobjects at runtime. For example a gun gameobject might add a bullet gameobject every time the gun fires. A monster gameobject might remove itself if it has been killed. We then would have an issue that we might have code like this

The loop above would fail or do un-expected things if gameobjects are added or removed from globalArrayOfGameObjects in the middle of the loop in some component's update function.

To try to prevent that problem we need something a little safer. Here's one attempt.

The class above lets you add or remove elements from the SafeArray but won't mess with the array itself while it's being iterated over. Instead new elements get added to addQueue and removed elements to the removeQueue and then added or removed outside of the loop.

Using that here is our class to manage gameobjects.

for (const gameObject of globalArrayOfGameObjects) {
  gameObject.update();
}

class SafeArray {
  constructor() {
    this.array = [];
    this.addQueue = [];
    this.removeQueue = new Set();
  }
  get isEmpty() {
    return this.addQueue.length + this.array.length > 0;
  }
  add(element) {
    this.addQueue.push(element);
  }
  remove(element) {
    this.removeQueue.add(element);
  }
  forEach(fn) {
    this._addQueued();
    this._removeQueued();
    for (const element of this.array) {
      if (this.removeQueue.has(element)) {
        continue;
      }
      fn(element);
    }
    this._removeQueued();
  }
  _addQueued() {
    if (this.addQueue.length) {
      this.array.splice(this.array.length, 0, ...this.addQueue);
      this.addQueue = [];
    }
  }
  _removeQueued() {
    if (this.removeQueue.size) {
      this.array = this.array.filter(element => !this.removeQueue.has(element));
      this.removeQueue.clear();
    }
  }
}

class GameObjectManager {
  constructor() {
    this.gameObjects = new SafeArray();
  }
  createGameObject(parent, name) {
    const gameObject = new GameObject(parent, name);
    this.gameObjects.add(gameObject);
    return gameObject;
  }
  removeGameObject(gameObject) {
    this.gameObjects.remove(gameObject);
  }
  update() {
    this.gameObjects.forEach(gameObject => gameObject.update());
  }
}

SkinInstance Component

With all that now let's make our first component. This component will just manage a skinned three.js object like the ones we just created. To keep it simple it will just have one method, setAnimation that takes the name of the animation to play and plays it.

You can see it's basically the code we had before that clones the scene we loaded, then sets up an AnimationMixer. setAnimation adds an AnimationAction for a particular AnimationClip if one does not already exist and disables all existing actions.

The code references globals.deltaTime. Let's make a globals object

And update it in the render loop

The check above for making sure deltaTime is not more than 1/20th of a second is because otherwise we'd get a huge value for deltaTime if we hide the tab. We might hide it for seconds or minutes and then when our tab was brought to the front deltaTime would be huge and might teleport characters across our game world if we had code like

By limiting the maximum deltaTime that issue is prevented.

class SkinInstance extends Component {
  constructor(gameObject, model) {
    super(gameObject);
    this.model = model;
    this.animRoot = SkeletonUtils.clone(this.model.gltf.scene);
    this.mixer = new THREE.AnimationMixer(this.animRoot);
    gameObject.transform.add(this.animRoot);
    this.actions = {};
  }
  setAnimation(animName) {
    const clip = this.model.animations[animName];
    // turn off all current actions
    for (const action of Object.values(this.actions)) {
      action.enabled = false;
    }
    // get or create existing action for clip
    const action = this.mixer.clipAction(clip);
    action.enabled = true;
    action.reset();
    action.play();
    this.actions[animName] = action;
  }
  update() {
    this.mixer.update(globals.deltaTime);
  }
}

Player Component and Animation Names

Now let's make a component for the player.

The player calls setAnimation with 'Run'. To know which animations are available I modified our previous example to print out the names of the animations

And running it got this list in the JavaScript console.

Fortunately the names of the animations for all the animals match which will come in handy later. For now we only care the that the player has an animation called Run.

Let's use these components. Here's the updated init function. All it does is create a GameObject and add a Player component to it.

And we need to call gameObjectManager.update in our render loop

and if we run that we get a single player.

That was a lot of code just for an entity component system but it's infrastructure that most games need.

class Player extends Component {
  constructor(gameObject) {
    super(gameObject);
    const model = models.knight;
    this.skinInstance = gameObject.addComponent(SkinInstance, model);
    this.skinInstance.setAnimation('Run');
  }
}

function prepModelsAndAnimations() {
  Object.values(models).forEach(model => {
+    console.log('------->:', model.url);
    const animsByName = {};
    model.gltf.animations.forEach((clip) => {
      animsByName[clip.name] = clip;
+      console.log('  ', clip.name);
    });
    model.animations = animsByName;
  });
}

------->:  resources/models/animals/Pig.gltf
    Idle
    Death
    WalkSlow
    Jump
    Walk
 ------->:  resources/models/animals/Cow.gltf
    Walk
    Jump
    WalkSlow
    Death
    Idle
 ------->:  resources/models/animals/Llama.gltf
    Jump
    Idle
    Walk
    Death
    WalkSlow
 ------->:  resources/models/animals/Pug.gltf
    Jump
    Walk
    Idle
    WalkSlow
    Death
 ------->:  resources/models/animals/Sheep.gltf
    WalkSlow
    Death
    Jump
    Walk
    Idle
 ------->:  resources/models/animals/Zebra.gltf
    Jump
    Walk
    Death
    WalkSlow
    Idle
 ------->:  resources/models/animals/Horse.gltf
    Jump
    WalkSlow
    Death
    Walk
    Idle
 ------->:  resources/models/knight/KnightCharacter.gltf
    Run_swordRight
    Run
    Idle_swordLeft
    Roll_sword
    Idle
    Run_swordAttack

const globals = {
  time: 0,
  deltaTime: 0,
};
+const gameObjectManager = new GameObjectManager();

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

  prepModelsAndAnimations();

+  {
+    const gameObject = gameObjectManager.createGameObject(scene, 'player');
+    gameObject.addComponent(Player);
+  }
}

let then = 0;
function render(now) {
  // convert to seconds
  globals.time = now * 0.001;
  // make sure delta time isn't too big.
  globals.deltaTime = Math.min(globals.time - then, 1 / 20);
  then = globals.time;

  if (resizeRendererToDisplaySize(renderer)) {
    const canvas = renderer.domElement;
    camera.aspect = canvas.clientWidth / canvas.clientHeight;
    camera.updateProjectionMatrix();
  }

-  for (const {mixer} of mixerInfos) {
-    mixer.update(deltaTime);
-  }
+  gameObjectManager.update();

  renderer.render(scene, camera);

  requestAnimationFrame(render);
}

Input System

Let's add an input system. Rather than read keys directly we'll make a class that other parts of the code can check left or right. That way we can assign multiple ways to input left or right etc.. We'll start with just keys

The code above tracks whether keys are up or down and you can check if a key is currently pressed by checking for example inputManager.keys.left.down. It also has a justPressed property for each key so that you can check the user just pressed the key. For example a jump key you don't want to know if the button is being held down, you want to know did the user press it now.

Let's create an instance of InputManager

and update it in our render loop

It needs to be called after gameObjectManager.update otherwise justPressed would never be true inside a component's update function.

Let's use it in the Player component

The code above uses Object3D.transformOnAxis to move the player forward. Object3D.transformOnAxis works in local space so it only works if the object in question is at the root of the scene, not if it's parented to something else.

We also added a global moveSpeed and based a turnSpeed on the move speed. The turn speed is based on the move speed to try to make sure a character can turn sharply enough to meet its target. If turnSpeed so too small a character will turn around and around circling its target but never hitting it. I didn't bother to do the math to calculate the required turn speed for a given move speed. I just guessed.

// Keeps the state of keys/buttons
//
// You can check
//
//   inputManager.keys.left.down
//
// to see if the left key is currently held down
// and you can check
//
//   inputManager.keys.left.justPressed
//
// To see if the left key was pressed this frame
//
// Keys are 'left', 'right', 'a', 'b', 'up', 'down'
class InputManager {
  constructor() {
    this.keys = {};
    const keyMap = new Map();

    const setKey = (keyName, pressed) => {
      const keyState = this.keys[keyName];
      keyState.justPressed = pressed && !keyState.down;
      keyState.down = pressed;
    };

    const addKey = (keyCode, name) => {
      this.keys[name] = { down: false, justPressed: false };
      keyMap.set(keyCode, name);
    };

    const setKeyFromKeyCode = (keyCode, pressed) => {
      const keyName = keyMap.get(keyCode);
      if (!keyName) {
        return;
      }
      setKey(keyName, pressed);
    };

    addKey(37, 'left');
    addKey(39, 'right');
    addKey(38, 'up');
    addKey(40, 'down');
    addKey(90, 'a');
    addKey(88, 'b');

    window.addEventListener('keydown', (e) => {
      setKeyFromKeyCode(e.keyCode, true);
    });
    window.addEventListener('keyup', (e) => {
      setKeyFromKeyCode(e.keyCode, false);
    });
  }
  update() {
    for (const keyState of Object.values(this.keys)) {
      if (keyState.justPressed) {
        keyState.justPressed = false;
      }
    }
  }
}

const globals = {
  time: 0,
  deltaTime: 0,
};
const gameObjectManager = new GameObjectManager();
+const inputManager = new InputManager();

function render(now) {

  ...

  gameObjectManager.update();
+  inputManager.update();

  ...
}

+const kForward = new THREE.Vector3(0, 0, 1);
const globals = {
  time: 0,
  deltaTime: 0,
+  moveSpeed: 16,
};

class Player extends Component {
  constructor(gameObject) {
    super(gameObject);
    const model = models.knight;
    this.skinInstance = gameObject.addComponent(SkinInstance, model);
    this.skinInstance.setAnimation('Run');
+    this.turnSpeed = globals.moveSpeed / 4;
  }
+  update() {
+    const {deltaTime, moveSpeed} = globals;
+    const {transform} = this.gameObject;
+    const delta = (inputManager.keys.left.down  ?  1 : 0) +
+                  (inputManager.keys.right.down ? -1 : 0);
+    transform.rotation.y += this.turnSpeed * delta * deltaTime;
+    transform.translateOnAxis(kForward, moveSpeed * deltaTime);
+  }
}

Camera Frustum Check (Offscreen Detection)

The code so far would work but if the player runs off the screen there's no way to find out where they are. Let's make it so if they are offscreen for more than a certain time they get teleported back to the origin. We can do that by using the three.js Frustum class to check if a point is inside the camera's view frustum.

We need to build a frustum from the camera. We could do this in the Player component but other objects might want to use this too so let's add another gameobject with a component to manage a frustum.

Then let's setup another gameobject at init time.

and now we can use it in the Player component.

class CameraInfo extends Component {
  constructor(gameObject) {
    super(gameObject);
    this.projScreenMatrix = new THREE.Matrix4();
    this.frustum = new THREE.Frustum();
  }
  update() {
    const {camera} = globals;
    this.projScreenMatrix.multiplyMatrices(
        camera.projectionMatrix,
        camera.matrixWorldInverse);
    this.frustum.setFromProjectionMatrix(this.projScreenMatrix);
  }
}

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

  prepModelsAndAnimations();

+  {
+    const gameObject = gameObjectManager.createGameObject(camera, 'camera');
+    globals.cameraInfo = gameObject.addComponent(CameraInfo);
+  }

  {
    const gameObject = gameObjectManager.createGameObject(scene, 'player');
    gameObject.addComponent(Player);
  }
}

class Player extends Component {
  constructor(gameObject) {
    super(gameObject);
    const model = models.knight;
    this.skinInstance = gameObject.addComponent(SkinInstance, model);
    this.skinInstance.setAnimation('Run');
    this.turnSpeed = globals.moveSpeed / 4;
+    this.offscreenTimer = 0;
+    this.maxTimeOffScreen = 3;
  }
  update() {
-    const {deltaTime, moveSpeed} = globals;
+    const {deltaTime, moveSpeed, cameraInfo} = globals;
    const {transform} = this.gameObject;
    const delta = (inputManager.keys.left.down  ?  1 : 0) +
                  (inputManager.keys.right.down ? -1 : 0);
    transform.rotation.y += this.turnSpeed * delta * deltaTime;
    transform.translateOnAxis(kForward, moveSpeed * deltaTime);

+    const {frustum} = cameraInfo;
+    if (frustum.containsPoint(transform.position)) {
+      this.offscreenTimer = 0;
+    } else {
+      this.offscreenTimer += deltaTime;
+      if (this.offscreenTimer >= this.maxTimeOffScreen) {
+        transform.position.set(0, 0, 0);
+      }
+    }
  }
}

Touchscreen Support

One more thing before we try it out, let's add touchscreen support for mobile. First let's add some HTML to touch

and some CSS to style it

The idea here is there is one div, #ui, that covers the entire page. Inside will be 2 divs, #left and #right both of which are almost half the page wide and the entire screen tall. In between there is a 40px separator. If the user slides their finger over the left or right side then we need up update keys.left and keys.right in the InputManager. This makes the entire screen sensitive to being touched which seemed better than just small arrows.

And now we should be able to control the character with the left and right cursor keys or with our fingers on a touchscreen.

Ideally we'd do something else if the player went off the screen like move the camera or maybe offscreen = death but this article is already going to be too long so for now teleporting to the middle was the simplest thing.

  <canvas id="c"></canvas>
+

+

+

+

+

...loading...

#ui {
  position: absolute;
  left: 0;
  top: 0;
  width: 100%;
  height: 100%;
  display: flex;
  justify-items: center;
  align-content: stretch;
}
#ui>div {
  display: flex;
  align-items: flex-end;
  flex: 1 1 auto;
}
.bright {
  filter: brightness(2);
}
#left {
  justify-content: flex-end;
}
#right {
  justify-content: flex-start;
}
#ui img {
  padding: 10px;
  width: 80px;
  height: 80px;
  display: block;
}

class InputManager {
  constructor() {
    this.keys = {};
    const keyMap = new Map();

    const setKey = (keyName, pressed) => {
      const keyState = this.keys[keyName];
      keyState.justPressed = pressed && !keyState.down;
      keyState.down = pressed;
    };

    const addKey = (keyCode, name) => {
      this.keys[name] = { down: false, justPressed: false };
      keyMap.set(keyCode, name);
    };

    const setKeyFromKeyCode = (keyCode, pressed) => {
      const keyName = keyMap.get(keyCode);
      if (!keyName) {
        return;
      }
      setKey(keyName, pressed);
    };

    addKey(37, 'left');
    addKey(39, 'right');
    addKey(38, 'up');
    addKey(40, 'down');
    addKey(90, 'a');
    addKey(88, 'b');

    window.addEventListener('keydown', (e) => {
      setKeyFromKeyCode(e.keyCode, true);
    });
    window.addEventListener('keyup', (e) => {
      setKeyFromKeyCode(e.keyCode, false);
    });

+    const sides = [
+      { elem: document.querySelector('#left'),  key: 'left'  },
+      { elem: document.querySelector('#right'), key: 'right' },
+    ];
+
+    const clearKeys = () => {
+      for (const {key} of sides) {
+          setKey(key, false);
+      }
+    };
+
+    const handleMouseMove = (e) => {
+      e.preventDefault();
+      // this is needed because we call preventDefault();
+      // we also gave the canvas a tabindex so it can
+      // become the focus
+      canvas.focus();
+      window.addEventListener('pointermove', handleMouseMove);
+      window.addEventListener('pointerup', handleMouseUp);
+
+      for (const {elem, key} of sides) {
+        let pressed = false;
+        const rect = elem.getBoundingClientRect();
+        const x = e.clientX;
+        const y = e.clientY;
+        const inRect = x >= rect.left && x < rect.right &&
+                       y >= rect.top && y < rect.bottom;
+        if (inRect) {
+          pressed = true;
+        }
+        setKey(key, pressed);
+      }
+    };
+
+    function handleMouseUp() {
+      clearKeys();
+      window.removeEventListener('pointermove', handleMouseMove, {passive: false});
+      window.removeEventListener('pointerup', handleMouseUp);
+    }
+
+    const uiElem = document.querySelector('#ui');
+    uiElem.addEventListener('pointerdown', handleMouseMove, {passive: false});
+
+    uiElem.addEventListener('touchstart', (e) => {
+      // prevent scrolling
+      e.preventDefault();
+    }, {passive: false});
  }
  update() {
    for (const keyState of Object.values(this.keys)) {
      if (keyState.justPressed) {
        keyState.justPressed = false;
      }
    }
  }
}

Animal Component Setup

Lets add some animals. We can start it off similar to the Player by making an Animal component.

The code above sets the AnimationMixer.timeScale to set the playback speed of the animations relative to the move speed. This way if we adjust the move speed the animation will speed up or slow down as well.

To start we could setup one of each type of animal

And that would get us animals standing on the screen but we want them to do something.

class Animal extends Component {
  constructor(gameObject, model) {
    super(gameObject);
    const skinInstance = gameObject.addComponent(SkinInstance, model);
    skinInstance.mixer.timeScale = globals.moveSpeed / 4;
    skinInstance.setAnimation('Idle');
  }
}

function init() {
  // hide the loading bar
  const loadingElem = document.querySelector('#loading');
  loadingElem.style.display = 'none';

  prepModelsAndAnimations();
  {
    const gameObject = gameObjectManager.createGameObject(camera, 'camera');
    globals.cameraInfo = gameObject.addComponent(CameraInfo);
  }

  {
    const gameObject = gameObjectManager.createGameObject(scene, 'player');
    globals.player = gameObject.addComponent(Player);
    globals.congaLine = [gameObject];
  }

+  const animalModelNames = [
+    'pig',
+    'cow',
+    'llama',
+    'pug',
+    'sheep',
+    'zebra',
+    'horse',
+  ];
+  animalModelNames.forEach((name, ndx) => {
+    const gameObject = gameObjectManager.createGameObject(scene, name);
+    gameObject.addComponent(Animal, models[name]);
+    gameObject.transform.position.x = (ndx + 1) * 5;
+  });
}

Animal AI States (Conga Line)

Let's make them follow the player in a conga line but only if the player gets near enough. To do this we need several states.

- Idle: Animal is waiting for player to get close
- Wait for End of Line: Animal was tagged by player but now needs to wait for the animal at the end of the line to come by so they can join the end of the line.
- Go to Last: Animal needs to walk to where the animal they are following was, at the same time recording a history of where the animal they are following is currently.
- Follow: Animal needs to keep recording a history of where the animal they are following is while moving to where the animal they are following was before.

There are many ways to handle different states like this. A common one is to use a Finite State Machine and to build some class to help us manage the state.

So, let's do that.

Here's a simple class. We pass it an object with a bunch of states. Each state as 3 optional functions, enter, update, and exit. To switch states we call FiniteStateMachine.transition and pass it the name of the new state. If the current state has an exit function it's called. Then if the new state has an enter function it's called. Finally each frame FiniteStateMachine.update calls the update function of the current state.

Let's use it to manage the states of the animals.

That was big chunk of code but it does what was described above. Hopefully of you walk through each state it will be clear.

A few things we need to add. We need the player to add itself to the globals so the animals can find it and we need to start the conga line with the player's GameObject.

We also need to compute a size for each model

And we need the player to record their size

Thinking about it now it would probably have been smarter for the animals to just target the head of the conga line instead of the player specifically. Maybe I'll come back and change that later.

When I first started this I used just one radius for all animals but of course that was no good as the pug is much smaller than the horse. So I added the difference sizes but I wanted to be able to visualize things. To do that I made a StateDisplayHelper component.

I uses a PolarGridHelper to draw a circle around each character and it uses html elements to let each character show some status using the techniques covered in the article on aligning html elements to 3D.

First we need to add some HTML to host these elements

And add some CSS for them

class FiniteStateMachine {
  constructor(states, initialState) {
    this.states = states;
    this.transition(initialState);
  }
  get state() {
    return this.currentState;
  }
  transition(state) {
    const oldState = this.states[this.currentState];
    if (oldState && oldState.exit) {
      oldState.exit.call(this);
    }
    this.currentState = state;
    const newState = this.states[state];
    if (newState.enter) {
      newState.enter.call(this);
    }
  }
  update() {
    const state = this.states[this.currentState];
    if (state.update) {
      state.update.call(this);
    }
  }
}

// Returns true of obj1 and obj2 are close
function isClose(obj1, obj1Radius, obj2, obj2Radius) {
  const minDist = obj1Radius + obj2Radius;
  const dist = obj1.position.distanceTo(obj2.position);
  return dist < minDist;
}

// keeps v between -min and +min
function minMagnitude(v, min) {
  return Math.abs(v) > min
      ? min * Math.sign(v)
      : v;
}

const aimTowardAndGetDistance = function() {
  const delta = new THREE.Vector3();

  return function aimTowardAndGetDistance(source, targetPos, maxTurn) {
    delta.subVectors(targetPos, source.position);
    // compute the direction we want to be facing
    const targetRot = Math.atan2(delta.x, delta.z) + Math.PI * 1.5;
    // rotate in the shortest direction
    const deltaRot = (targetRot - source.rotation.y + Math.PI * 1.5) % (Math.PI * 2) - Math.PI;
    // make sure we don't turn faster than maxTurn
    const deltaRotation = minMagnitude(deltaRot, maxTurn);
    // keep rotation between 0 and Math.PI * 2
    source.rotation.y = THREE.MathUtils.euclideanModulo(
        source.rotation.y + deltaRotation, Math.PI * 2);
    // return the distance to the target
    return delta.length();
  };
}();

class Animal extends Component {
  constructor(gameObject, model) {
    super(gameObject);
+    const hitRadius = model.size / 2;
    const skinInstance = gameObject.addComponent(SkinInstance, model);
    skinInstance.mixer.timeScale = globals.moveSpeed / 4;
+    const transform = gameObject.transform;
+    const playerTransform = globals.player.gameObject.transform;
+    const maxTurnSpeed = Math.PI * (globals.moveSpeed / 4);
+    const targetHistory = [];
+    let targetNdx = 0;
+
+    function addHistory() {
+      const targetGO = globals.congaLine[targetNdx];
+      const newTargetPos = new THREE.Vector3();
+      newTargetPos.copy(targetGO.transform.position);
+      targetHistory.push(newTargetPos);
+    }
+
+    this.fsm = new FiniteStateMachine({
+      idle: {
+        enter: () => {
+          skinInstance.setAnimation('Idle');
+        },
+        update: () => {
+          // check if player is near
+          if (isClose(transform, hitRadius, playerTransform, globals.playerRadius)) {
+            this.fsm.transition('waitForEnd');
+          }
+        },
+      },
+      waitForEnd: {
+        enter: () => {
+          skinInstance.setAnimation('Jump');
+        },
+        update: () => {
+          // get the gameObject at the end of the conga line
+          const lastGO = globals.congaLine[globals.congaLine.length - 1];
+          const deltaTurnSpeed = maxTurnSpeed * globals.deltaTime;
+          const targetPos = lastGO.transform.position;
+          aimTowardAndGetDistance(transform, targetPos, deltaTurnSpeed);
+          // check if last thing in conga line is near
+          if (isClose(transform, hitRadius, lastGO.transform, globals.playerRadius)) {
+            this.fsm.transition('goToLast');
+          }
+        },
+      },
+      goToLast: {
+        enter: () => {
+          // remember who we're following
+          targetNdx = globals.congaLine.length - 1;
+          // add ourselves to the conga line
+          globals.congaLine.push(gameObject);
+          skinInstance.setAnimation('Walk');
+        },
+        update: () => {
+          addHistory();
+          // walk to the oldest point in the history
+          const targetPos = targetHistory[0];
+          const maxVelocity = globals.moveSpeed * globals.deltaTime;
+          const deltaTurnSpeed = maxTurnSpeed * globals.deltaTime;
+          const distance = aimTowardAndGetDistance(transform, targetPos, deltaTurnSpeed);
+          const velocity = distance;
+          transform.translateOnAxis(kForward, Math.min(velocity, maxVelocity));
+          if (distance <= maxVelocity) {
+            this.fsm.transition('follow');
+          }
+        },
+      },
+      follow: {
+        update: () => {
+          addHistory();
+          // remove the oldest history and just put ourselves there.
+          const targetPos = targetHistory.shift();
+          transform.position.copy(targetPos);
+          const deltaTurnSpeed = maxTurnSpeed * globals.deltaTime;
+          aimTowardAndGetDistance(transform, targetHistory[0], deltaTurnSpeed);
+        },
+      },
+    }, 'idle');
+  }
+  update() {
+    this.fsm.update();
+  }
}

function init() {

  ...

  {
    const gameObject = gameObjectManager.createGameObject(scene, 'player');
+    globals.player = gameObject.addComponent(Player);
+    globals.congaLine = [gameObject];
  }

}

function prepModelsAndAnimations() {
+  const box = new THREE.Box3();
+  const size = new THREE.Vector3();
  Object.values(models).forEach(model => {
+    box.setFromObject(model.gltf.scene);
+    box.getSize(size);
+    model.size = size.length();
    const animsByName = {};
    model.gltf.animations.forEach((clip) => {
      animsByName[clip.name] = clip;
      // Should really fix this in .blend file
      if (clip.name === 'Walk') {
        clip.duration /= 2;
      }
    });
    model.animations = animsByName;
  });
}

class Player extends Component {
  constructor(gameObject) {
    super(gameObject);
    const model = models.knight;
+    globals.playerRadius = model.size / 2;

  <canvas id="c"></canvas>

...loading...

+


WebGPURenderer

Summary: An overview of three.js's next-generation WebGPURenderer, which uses WebGPU by default with a WebGL 2 fallback, including usage instructions, code examples, and migration considerations from WebGLRenderer.

WebGPURenderer

The new `WebGPURenderer` is the next-generation renderer for three.js. This article provides a short overview about the new renderer and basic guidelines about the usage.

Overview

`WebGPURenderer` is designed to be the modern alternative to the long-standing `WebGLRenderer`. Its primary goal is to use WebGPU, which is a modern, high-performance graphics and compute 3D API. However, it's built to be a universal renderer. If a device/browser doesn't support WebGPU, the renderer can automatically fall back to using a WebGL 2 backend.

Providing a WebGL 2 backend as a fallback is a crucial design decision since it allows applications to benefit from WebGPU but without sacrificing the support for devices which only support WebGL 2.

Apart from the fact that `WebGPURenderer` enables access to WebGPU, it offers an exciting feature set:

- `WebGPURenderer` comes with a new node-based material system which allows to develop custom materials with greater flexibility and more robustness.
- The renderer supports TSL, the three.js shading language. With TSL, developers can write shader code with JavaScript in a platform-independent manner. Shader code written in TSL can be transpiled to WGSL or GLSL depending on the available backend.
- `WebGPURenderer` comes with a new post-processing stack with built-in Multiple Render Targets (MRT) support and automatic pass combination thanks to the node material.

Let's find out how to integrate `WebGPURenderer` in three.js applications.

Usage

`WebGPURenderer` has different build files so the way you import three.js changes:

If you are using an import map, it's recommended to change it to the following (the paths differ depending on your setup):

You can create an instance of the renderer just like with `WebGLRenderer`:

It's important to understand that WebGPU is initialized in an asynchronous fashion. Hence, it is recommended to use `setAnimationLoop()` to define the animation loop of your app since this approach will automatically ensure the renderer is initialized when rendering the first frame. If you prefer to manage your animation loop via `window.requestAnimationFrame()` or if you have to use the renderer in your init routine, you need an additional line in the above code section.

Most common methods known from `WebGLRenderer` like `clear()`, `setRenderTarget()` or `dispose()` are also present in `WebGPURenderer`. Please have a look at the API documentation for a full overview of the renderer's public interface.

Like mentioned in the initial part of the guide, `WebGPURenderer` uses a WebGPU backend by default and a WebGL 2 backend as a fallback. If you want to force the usage of WebGL 2 for testing purposes or if you want to exclude the usage of WebGPU for certain reasons, you can make use of the `forceWebGL` parameter.

-  import * as THREE from 'three';
+  import * as THREE from 'three/webgpu';

<script type="importmap">
  {
    "imports": {
      "three": "../build/three.webgpu.js",
      "three/webgpu": "../build/three.webgpu.js",
      "three/tsl": "../build/three.tsl.js",
      "three/addons/": "./jsm/"
    }
  }
</script>

const renderer = new THREE.WebGPURenderer( { antialias: true } );
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.setAnimationLoop( render );
document.body.appendChild( renderer.domElement );

+  await renderer.init();

-  const renderer = new THREE.WebGPURenderer( { antialias: true } );
+  const renderer = new THREE.WebGPURenderer( { antialias: true, forceWebGL: true } );

Migration

If you want to give `WebGPURenderer` a try, you have to be aware of the following.

- Custom materials based on `ShaderMaterial`, `RawShaderMaterial` and modifications of built-in materials via `onBeforeCompile()` are not supported in `WebGPURenderer`. This part of your application must be ported to node materials and TSL.
- `EffectComposer` with its effect passes are not supported because `WebGPURenderer` comes with a new, more modern post-processing stack. Similar to materials, post-processing effects are now written in TSL and the effect chain is expressed as a node composition. All common effects have already been ported to `WebGPURenderer` and exist in a more performant version as a node class. We have also added new effects like SSGI, SSS or a better DoF exclusively for the new renderer. Check out the official examples to get an overview of the current supported effects.
- The renderer itself is still in an experimental state although its maturity level has been greatly improved in the last years. Still, depending on your application and scene setup, you will encounter missing features or a better performance with `WebGLRenderer`. Feel free to file an issue at GitHub so we are aware of open tasks. We are improving `WebGPURenderer` with each release so it's recommended to upgrade to the latest version whenever possible.

State of WebGLRenderer

Although in the meanwhile a lot of work happens in context of `WebGPURenderer`, the node material and TSL, `WebGLRenderer` is still maintained and the recommended choice for pure WebGL 2 applications. However, keep in mind that there are no plans to add larger new features to the renderer since the project's focus is now on `WebGPURenderer` which you can easily see at the latest release notes. That said, we are currently investigating the possibility to add limited node material support to `WebGLRenderer` in order to make the transition to `WebGPURenderer` easier for certain projects.