artificial-intelligence

From a 2D Concept Image to a Three.js Scene: An AI-Assisted 3D Asset Workflow

Stackademic
0 comments

Learn how to turn a 2D concept image into a textured 3D GLB asset with AI, then load, frame, optimize, and display it in a responsive Three.js scene.

To turn a 2D concept image into a Three.js scene, generate a GLB with an image-to-3D tool, inspect and optimise the asset, then load it with Three.js GLTFLoader.

Creating a 3D asset for a web project traditionally begins in modelling software. An artist builds the geometry, creates UVs, paints textures, configures materials, and exports the result before a developer can load it into a Three.js scene.

That process gives artists complete control, but it can be slow when a developer only needs a prototype, background prop, product mock-up, or placeholder asset. AI-assisted 3D generation offers another starting point: provide a clean concept image, generate a textured model, export it as GLB, and test it directly in the browser.

This tutorial walks through that workflow from beginning to end. We will prepare an image, generate the asset, create a Vite project, load the GLB file with Three.js, scale and frame it automatically, and review the optimisation work required before production.

What We Are Building

By the end of the tutorial, we will have a simple browser-based 3D viewer with:

  • A textured GLB model generated from a concept image
  • A perspective camera
  • Basic studio-style lighting
  • Orbit, zoom, and pan controls
  • Automatic model centring
  • Automatic scaling
  • Responsive rendering
  • Loading progress
  • Basic error handling
  • Real-time shadows

The workflow is useful for rapid prototypes, indie games, product previews, portfolio projects, and early design reviews.

Prerequisites

You should have:

  • A recent version of Node.js
  • Basic JavaScript knowledge
  • A code editor
  • A PNG, JPG, or WebP concept image
  • A modern browser with WebGL support

You do not need previous 3D modelling experience, although familiarity with meshes, materials, textures, and coordinate systems will help.

Why Export the Model as GLB?

Three.js can load several model formats, but GLB is particularly practical for web projects.

GLB is the binary container format for glTF 2.0, packaging geometry, materials, and textures into a single file. It can also contain scene hierarchy and animation data. The Khronos Group’s official glTF overview describes glTF as a royalty-free specification designed for efficient transmission and loading of 3D scenes and models.

A single-file asset is easier to place in a public directory, upload to object storage, cache through a CDN, and reference from application code. It also avoids many of the broken texture-path problems developers encounter with multi-file formats.

GLB does not automatically make an asset optimised. A generated model can still contain too many polygons or oversized textures. It simply gives us a convenient delivery format for the web.

Step 1: Prepare a Suitable Concept Image

The quality of the source image has a major effect on the generated geometry. A visually impressive illustration is not always the best reconstruction input.

For this tutorial, choose a single object such as:

  • A backpack
  • A treasure chest
  • A potion bottle
  • A helmet
  • A chair
  • A lamp
  • A stylised vehicle
  • A simple game prop

Avoid beginning with a complex character. Hair, fingers, layered clothing, transparent surfaces, and extreme poses introduce more reconstruction ambiguity.

Use a clear silhouette

The object should be easy to distinguish from the background. A plain or transparent background is usually more useful than a detailed environment.

Avoid heavy perspective distortion

A dramatic wide-angle view may look interesting, but it can make hidden dimensions harder to infer. A three-quarter view often provides a good balance between the front and side of the object.

Keep the lighting readable

Strong shadows can be interpreted as part of the surface. Use soft, even lighting when possible.

Show one object

Multiple overlapping objects can produce merged or incomplete geometry. Crop the image so the intended subject is obvious.

Use multiple views for asymmetric objects

A single image leaves the rear and hidden sides undefined. If the object is asymmetrical or structurally complex, front, back, and side references can improve the reconstruction. Multi-view input is particularly useful when the unseen side cannot be inferred from symmetry.

Step 2: Generate and Export the 3D Model

For this walkthrough, I used Meshy AI to convert a clean concept image into a textured model. The browser-based workflow accepts formats including PNG, JPG, and WebP, supports both single-image and multi-view inputs, and can export the selected result as GLB.

Upload the image and generate several variations if the available credits allow it. Do not immediately select the result that looks best from the front. Rotate each candidate and inspect:

  • The rear of the object
  • The underside
  • Thin components
  • Openings and holes
  • Symmetry
  • Texture seams
  • Floating geometry
  • Distorted edges
  • Unexpected surface bumps

If the source image shows only one side, expect the system to infer the hidden geometry. That inferred section may need manual correction before production.

When exporting, choose GLB and reduce the polygon count if the original model is unnecessarily dense. Keep a higher-resolution version separately in case additional editing is required later.

The platform offers free credits, but generation and export availability depend on the current plan and remaining credits. It is more accurate to treat the free allowance as a way to test the workflow than as unlimited production capacity.

Step 3: Create a Vite Project

Open a terminal and create a new Vite project:

npm create vite@latest ai-3d-threejs -- --template vanilla

cd ai-3d-threejs

npm install

npm install three

 

Start the development server:

npm run dev

 

Vite will print a local URL, usually similar to:

http://localhost:5173

 

Open it in the browser to confirm that the starter project is working.

Step 4: Add the GLB Asset

Inside the project, create a model directory:

ai-3d-threejs/

├── public/

│   └── models/

│       └── concept-asset.glb

├── src/

│   ├── main.js

│   └── style.css

├── index.html

└── package.json

 

Rename the exported file to concept-asset.glb and place it inside public/models.

Files in Vite’s public directory are served from the root URL, so the model will be available at:

/models/concept-asset.glb

 

Avoid placing spaces in production asset names. Predictable lower-case names simplify deployment and reduce path mistakes.

Step 5: Create the HTML Structure

Replace the contents of index.html with:

<!doctype html>

<html lang="en">

  <head>

    <meta charset="UTF-8" />

    <meta

      name="viewport"

      content="width=device-width, initial-scale=1.0"

    />

    <title>AI-Assisted 3D Asset Viewer</title>

  </head>

  <body>

    <div id="app">

      <canvas id="scene"></canvas>

 

      <div class="overlay">

        <h2>Concept Asset Viewer</h2>

        <p id="status">Loading model...</p>

      </div>

    </div>

 

    <script type="module" src="/src/main.js"></script>

  </body>

</html>

 

The canvas will contain the Three.js renderer. The overlay gives us a place to display loading and error messages.

Step 6: Add the Page Styles

Replace src/style.css with:

* {

  box-sizing: border-box;

}

 

html,

body,

#app {

  width: 100%;

  height: 100%;

  margin: 0;

}

 

body {

  overflow: hidden;

  background: #111827;

  color: #f9fafb;

  font-family:

    Inter, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI",

    sans-serif;

}

 

#scene {

  display: block;

  width: 100%;

  height: 100%;

}

 

.overlay {

  position: fixed;

  top: 1rem;

  left: 1rem;

  padding: 0.9rem 1rem;

  border: 1px solid rgba(255, 255, 255, 0.14);

  border-radius: 0.75rem;

  background: rgba(17, 24, 39, 0.72);

  backdrop-filter: blur(10px);

  pointer-events: none;

}

 

.overlay h1 {

  margin: 0 0 0.35rem;

  font-size: 1rem;

}

 

.overlay p {

  margin: 0;

  color: #cbd5e1;

  font-size: 0.875rem;

}

 

The canvas occupies the full viewport, while the overlay remains readable without blocking camera controls.

Step 7: Build the Three.js Scene

Replace src/main.js with the following code:

import "./style.css";

import * as THREE from "three";

import { GLTFLoader } from "three/addons/loaders/GLTFLoader.js";

import { OrbitControls } from "three/addons/controls/OrbitControls.js";

 

const canvas = document.querySelector("#scene");

const statusElement = document.querySelector("#status");

 

const scene = new THREE.Scene();

scene.background = new THREE.Color(0x111827);

 

const camera = new THREE.PerspectiveCamera(

  45,

  window.innerWidth / window.innerHeight,

  0.01,

  1000

);

 

camera.position.set(3, 2, 5);

 

const renderer = new THREE.WebGLRenderer({

  canvas,

  antialias: true,

});

 

renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));

renderer.setSize(window.innerWidth, window.innerHeight);

renderer.outputColorSpace = THREE.SRGBColorSpace;

renderer.toneMapping = THREE.ACESFilmicToneMapping;

renderer.toneMappingExposure = 1;

renderer.shadowMap.enabled = true;

renderer.shadowMap.type = THREE.PCFSoftShadowMap;

 

const controls = new OrbitControls(camera, renderer.domElement);

controls.enableDamping = true;

controls.dampingFactor = 0.06;

controls.target.set(0, 0.8, 0);

 

const hemisphereLight = new THREE.HemisphereLight(

  0xffffff,

  0x334155,

  2.5

);

 

scene.add(hemisphereLight);

 

const keyLight = new THREE.DirectionalLight(0xffffff, 4);

keyLight.position.set(4, 6, 5);

keyLight.castShadow = true;

keyLight.shadow.mapSize.set(2048, 2048);

keyLight.shadow.camera.near = 0.1;

keyLight.shadow.camera.far = 30;

keyLight.shadow.camera.left = -8;

keyLight.shadow.camera.right = 8;

keyLight.shadow.camera.top = 8;

keyLight.shadow.camera.bottom = -8;

scene.add(keyLight);

 

const fillLight = new THREE.DirectionalLight(0x93c5fd, 2);

fillLight.position.set(-4, 3, 2);

scene.add(fillLight);

 

const floor = new THREE.Mesh(

  new THREE.CircleGeometry(4, 64),

  new THREE.MeshStandardMaterial({

    color: 0x1f2937,

    roughness: 0.9,

    metalness: 0,

  })

);

 

floor.rotation.x = -Math.PI / 2;

floor.position.y = -0.01;

floor.receiveShadow = true;

scene.add(floor);

 

function frameModel(model) {

  const initialBox = new THREE.Box3().setFromObject(model);

  const initialSize = initialBox.getSize(new THREE.Vector3());

 

  const largestDimension = Math.max(

    initialSize.x,

    initialSize.y,

    initialSize.z

  );

 

  if (!Number.isFinite(largestDimension) || largestDimension <= 0) {

    console.warn("Could not determine a valid model size.");

    return;

  }

 

  const targetSize = 2;

  const scale = targetSize / largestDimension;

 

  model.scale.multiplyScalar(scale);

 

  const scaledBox = new THREE.Box3().setFromObject(model);

  const scaledCenter = scaledBox.getCenter(new THREE.Vector3());

 

  model.position.x -= scaledCenter.x;

  model.position.y -= scaledBox.min.y;

  model.position.z -= scaledCenter.z;

 

  const finalBox = new THREE.Box3().setFromObject(model);

  const finalSize = finalBox.getSize(new THREE.Vector3());

 

  controls.target.set(0, finalSize.y * 0.45, 0);

 

  camera.position.set(

    finalSize.x * 1.7 + 1,

    finalSize.y * 1.1 + 0.8,

    finalSize.z * 1.7 + 2

  );

 

  camera.near = 0.01;

  camera.far = 100;

  camera.updateProjectionMatrix();

 

  controls.update();

}

 

const loader = new GLTFLoader();

 

loader.load(

  "/models/concept-asset.glb",

 

  (gltf) => {

    const model = gltf.scene;

 

    model.traverse((child) => {

      if (!child.isMesh) return;

 

      child.castShadow = true;

      child.receiveShadow = true;

 

      if (child.material) {

        child.material.needsUpdate = true;

      }

    });

 

    scene.add(model);

    frameModel(model);

 

    statusElement.textContent = "Model loaded";

  },

 

  (event) => {

    if (!event.total) {

      statusElement.textContent = "Loading model...";

      return;

    }

 

    const percent = Math.round((event.loaded / event.total) * 100);

    statusElement.textContent = `Loading model: ${percent}%`;

  },

 

  (error) => {

    console.error("Failed to load GLB model:", error);

    statusElement.textContent = "Could not load the model";

  }

);

 

function resizeRenderer() {

  const width = window.innerWidth;

  const height = window.innerHeight;

 

  camera.aspect = width / height;

  camera.updateProjectionMatrix();

 

  renderer.setSize(width, height);

  renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));

}

 

window.addEventListener("resize", resizeRenderer);

 

function animate() {

  controls.update();

  renderer.render(scene, camera);

}

 

renderer.setAnimationLoop(animate);

Save the file and return to the browser. The generated model should appear above the circular floor, with a shadow cast by the key light.

The official Three.js GLTFLoader documentation lists the glTF 2.0 features and extensions supported by the loader.

Understanding the Important Parts

Renderer colour management

This line sets the output colour space:

renderer.outputColorSpace = THREE.SRGBColorSpace;

 

Without appropriate colour management, textures may look darker or less saturated than expected.

The ACES filmic tone mapping setting helps produce smoother highlight behaviour:

renderer.toneMapping = THREE.ACESFilmicToneMapping;

 

Tone mapping does not fix incorrect materials, but it gives the basic viewer a more controlled appearance.

Shadows

Setting castShadow and receiveShadow on objects is not enough to produce shadows. The renderer must also enable its shadow map:

renderer.shadowMap.enabled = true;

At least one light must cast shadows:

keyLight.castShadow = true;

The floor receives the shadow:

floor.receiveShadow = true;

The model meshes cast and receive shadows through the traversal callback. Shadow-map resolution and camera bounds should be adjusted for the scene. Excessively large shadow maps can increase memory use and reduce performance, especially on mobile devices.

OrbitControls

OrbitControls allows the user to rotate, zoom, and pan around the asset:

const controls = new OrbitControls(

  camera,

  renderer.domElement

);

 

Damping produces smoother movement, but it requires controls.update() inside the animation loop.

Automatic centring and scaling

Generated assets may use different origins and scales. Some appear far above the floor, while others load outside the camera view.

The order of operations in frameModel() is important. It first calculates the original bounding box and applies the required scale. It then recalculates the bounding box after scaling, uses the updated centre to move the model to the scene origin, and moves the scaled lowest point to the floor.

If the model is centred before scaling, the later scale operation can move it away from the origin again. Recalculating the bounding box after scaling prevents that problem.

The final bounding box is used to position the camera and controls. This is useful for viewers that must accept different assets. For a final game or application, you may prefer a consistent asset standard instead of correcting every model at runtime.

Step 8: Check the Generated Asset

A model that loads without an error is not automatically production-ready.

Rotate it and review the following areas.

Hidden geometry

The back and underside are usually the least reliable areas when the model was generated from one image. Look for stretched surfaces, accidental holes, and duplicated components.

Material consistency

Check whether roughness and metallic surfaces respond sensibly to light. If every part is equally glossy, the material setup may need further work.

Texture resolution

A large texture can dominate the download size. A 4K texture is rarely necessary for a small background prop on a mobile screen.

Polygon density

AI-generated geometry may contain more triangles than the visible shape requires. A static prop does not need dense topology across flat surfaces.

Orientation

Three.js uses a Y-up coordinate system. If the asset appears rotated, correct it temporarily with:

model.rotation.x = -Math.PI / 2;

 

The exact adjustment depends on the export. It is better to fix orientation in the asset pipeline when possible rather than applying unexplained rotations throughout the codebase.

Step 9: Optimise Before Production

The prototype proves that the workflow works. Production requires stricter limits.

Reduce polygon count

Use the export controls or a dedicated 3D application to simplify the model. Inspect the silhouette after reduction, especially around curved edges and small components.

Resize textures

Choose texture dimensions based on how large the asset appears on screen. Common web targets include:

  • 512×512 for small props
  • 1024×1024 for medium assets
  • 2048×2048 for prominent assets

The correct choice depends on camera distance, screen resolution, and artistic requirements.

Compress textures

Modern GPU texture formats can reduce memory use and transfer size, but they require an additional conversion pipeline. Start with a measurable performance problem rather than adding every compression technique immediately.

Consider Draco carefully

Draco compression can significantly reduce mesh transfer size. Three.js supports Draco-compressed glTF through DRACOLoader, but the browser must download and run the decoder.

Compression therefore trades network size for decoding work. Test on representative mobile devices before adopting it globally.

Use lazy loading

Do not load every 3D asset during the initial page request. Load models when the associated scene, product, or level becomes relevant.

Dispose of unused resources

Removing a mesh from a scene does not automatically release every GPU resource. Long-running applications should dispose of geometry, materials, and textures when assets are no longer needed.

Common Problems

The model returns a 404 error

Confirm that the file is inside:

public/models/concept-asset.glb

 

and that the loader uses:

"/models/concept-asset.glb"

 

Remember that the public directory itself is not included in the URL.

The model is black

Check the scene lights and material compatibility. A standard material needs lighting, while an unlit material does not.

Shadows are not visible

Confirm all three parts of the shadow setup:

renderer.shadowMap.enabled = true;

keyLight.castShadow = true;

floor.receiveShadow = true;

The model mesh must also have castShadow = true. If shadows remain difficult to see, check the light’s shadow-camera bounds, the floor position, and the direction of the light.

Textures are missing

Export the asset as a self-contained GLB. If you use glTF with external textures, confirm that every referenced file is deployed with the correct relative path.

The model is too large or invisible

Use a bounding box to inspect its dimensions. The automatic scaling function in this tutorial handles many inconsistent exports, but malformed geometry can still produce an extreme bounding box.

The page is slow on mobile

Reduce the polygon count, texture dimensions, device pixel ratio, shadow-map resolution, and number of lights. The tutorial already caps the pixel ratio:

renderer.setPixelRatio(

  Math.min(window.devicePixelRatio, 2)

);

 

For low-powered devices, a cap of 1.5 or 1 may be more appropriate. Shadows can also be disabled on lower-end devices if they do not justify their performance cost.

Where AI Helps—and Where It Does Not

AI-assisted generation is valuable during exploration. It can turn concept art into a testable asset before a team commits time to manual modelling.

It works particularly well for:

  • Early prototypes
  • Background props
  • Design exploration
  • Placeholder content
  • Pitch demos
  • Simple product visualisations

It is less reliable when the asset requires:

  • Exact dimensions
  • Clean deformation topology
  • Controlled UV layouts
  • Consistent modular parts
  • Mechanical accuracy
  • Strict polygon budgets
  • Production-ready character rigging

The generated result should be treated as an asset candidate, not an unquestionable final output. Developers still need to inspect performance, while artists may need to correct geometry, materials, or textures.

Final Thoughts

An AI-assisted image-to-model workflow can shorten the distance between an idea and a working Three.js prototype. A clean reference image becomes a textured GLB, and the GLB can be loaded into a browser scene with only a small amount of code.

The speed is useful, but the engineering responsibilities remain unchanged. Assets still need predictable naming, correct orientation, reasonable polygon counts, efficient textures, error handling, and testing across target devices.

Use AI to accelerate the first version. Use measurement, validation, and deliberate asset standards to make that version ready for production.

Comments

Loading comments…