Command Palette

Search for a command to run...

Hectal
Course outline
Topic 8.1Intermediate ~22 min · step 1 of 6

Raycasting and Picking

In one line

A click gives two pixel numbers. A ray from the camera through that pixel finds the first object it touches. You convert the pixel with the canvas's own box, read the hit list, and meet the cases that surprise people: empty space, glass, hidden hit-boxes and a canvas that is not in the page's corner.

0/6 · 0%

Think of it like this

Stand at a window and point a laser pen at a wall of posters across the street. The beam leaves your eye, goes through the glass at the spot you chose, and touches the first poster in its way. That poster is the one you pointed at. Picking in 3D is the same beam, from the camera, through the pixel you clicked.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

ray
A start point and a direction, like a laser beam that never ends. Picking sends one ray from the camera through the clicked pixel.
normalised device coordinates (NDC)
Screen positions scaled to −1 … 1 in both directions, with y pointing up. The middle of the canvas is (0, 0).
hit
One place a ray touched an object. A hit holds the object, its distance from the ray's start, the 3D point and, for meshes, the face.
layer
One of 32 numbered groups an object can belong to. A raycaster only tests objects that share a layer with it.

Step by step

01Get the click in page pixels

A pointerdown or click event has clientX and clientY. These are the same numbers whatever the page layout, which makes them the right starting point.

02Subtract where the canvas is

getBoundingClientRect() gives the canvas's left, top, width and height in the same pixel space. Subtract left and top, divide by the size, and you have a position inside the canvas from 0 to 1.

03Scale to NDC and flip y

Multiply by 2 and subtract 1 to reach −1 … 1. Pixel y counts down and NDC y counts up, so the y value gets a minus sign. Forgetting the flip picks the mirror image: a click near the top finds what is near the bottom.

04Shoot and read the first hit

raycaster.setFromCamera(new THREE.Vector2(x, y), camera) aims the ray. intersectObjects(scene.children, true) returns the hits. hits[0] is the nearest. Climb from hits[0].object to the part you named, with parent, until you reach the object you care about.

Shoot and read the first hitdiagram
Rendering diagram…

Try it yourself

  1. 1

    Make the canvas fill the page

    In the first example change the CSS rule to #box { position: absolute; inset: 0; }. The canvas now fills the frame, and rect prints 0 0 and the frame size. The window way and the canvas way both say cube, because with a full-page canvas at the corner the two formulas are the same. The picture looks stretched, since the camera's aspect is still 2 (Topic 3.3 fixes that).

  2. 2

    Make the glass see-through

    In the hit-list example add glass.material.opacity = 0; before const raycaster. Nothing changes: glass@3.9 is still the first hit. The ray does not know what you can see. To click through glass, skip it with find(...) as the example does, or move it to another layer (and let the camera draw that layer too).

Code & diagrams

A canvas that is not in the corner New tab

The canvas is a small box 10 px from the top-left. The click is in the middle of the cube. The window way (innerWidth) misses; the canvas way hits. Open the status bar: one draw call.

idle

Console

Run the scene to see its console.

Expected console output

canvas rect: 10 10 140 70
click at: 80 45
window way: nothing
canvas way: cube
empty corner: nothing
The hit list: glass, hit-boxes, layers New tab

Three boxes in a row along the ray. The first hit is glass, even though you can see through it. Each line changes one thing.

idle

Console

Run the scene to see its console.

Expected console output

before the first render:      nothing
scene.children, not recursive: nothing
scene.children, recursive:     glass@3.9, hitbox@4.4, cube@4.9
hitbox.visible = false:        glass@3.9, hitbox@4.4, cube@4.9
first hit that is not glass:   hitbox
glass on layer 1:              hitbox@4.4, cube@4.9
raycaster sees layers 0 and 1: glass@3.9, hitbox@4.4, cube@4.9
What a hit carries, and clicking an instance New tab

Three tyres are one InstancedMesh: one draw call, and `instanceId` says which one. Click a tyre in the scene: the console prints "you clicked: tyre N".

idle

Console

Run the scene to see its console.

Expected console output

object:     tyres
instanceId: 2
distance:   5.21 metres from the camera
point:      1.47, 0.52, 0.09
normal:     -0.03, 0.26, 0.97
faceIndex:  144
uv:         0.26, 0.21
middle tyre: 1

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Pick before anything has been drawn

In the hit-list example, delete the line "renderer.render(scene, camera); // this updates every matrixWorld, which the ray needs".

terminal
$ Run
── what you'll see ──
before the first render: nothing
scene.children, not recursive: nothing
scene.children, recursive: nothing
hitbox.visible = false: nothing
Uncaught TypeError: Cannot read properties of undefined (reading 'object')
Every pick says nothing, and the "first hit that is not glass" line then fails because there is no first hit.

Myth vs fact

Myth

The ray only hits things you can see.

Fact

It tests geometry, not pixels. Invisible meshes, fully transparent materials and objects hidden behind glass are all hit. If something must not be clickable, move it to another layer or leave it out of the list.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

  • ▸

    Raycaster reads camera.matrixWorld, so a camera that has just been moved needs camera.updateMatrixWorld() before a pick in the same frame. intersectObject also takes a third argument, an array to fill, so a hot path can reuse one array instead of allocating a new one per pick.

  • ▸

    A box face is two triangles. A ray through their shared diagonal can report two hits on the same object, so compare hit.object and never count hits as different things.

Remember this

  1. 1

    The problem this solves: a click tells you two numbers, pixels on the screen. The scene has objects in 3D space. You need to turn "the user clicked at x = 400, y = 200" into "the user clicked the handlebar".

  2. 2

    The ray. A ray is a start point and a direction. Raycaster.setFromCamera(coords, camera) builds the ray that starts at the camera and passes through a point on the screen. The point must be in normalised device coordinates (NDC): x runs from −1 at the left edge to 1 at the right edge, y from −1 at the bottom to 1 at the top. The middle of the canvas is (0, 0).

  3. 3

    From pixels to NDC. A pointer event gives clientX and clientY, measured from the top-left of the page window. Ask the canvas where it is: rect = canvas.getBoundingClientRect(), then x = ((clientX − rect.left) / rect.width) * 2 − 1 and y = −((clientY − rect.top) / rect.height) * 2 + 1. Using innerWidth and innerHeight instead is only right when the canvas fills the whole window and sits at its corner. All these numbers are CSS pixels, so a pixel ratio of 2 or 3 changes nothing here (never multiply by devicePixelRatio), and a canvas scaled by CSS is handled because the box is the scaled one.

  4. 4

    The hit list. raycaster.intersectObjects(list, recursive) returns an array of hits, nearest first. An empty array means the user clicked empty space, and every handler must deal with that. recursive defaults to true; with false, only the objects in list are tested, not their children, so a Group hits nothing.

  5. 5

    What the ray ignores. It does not look at visible, material.visible, opacity or transparent. A glass panel with opacity 0.25 is hit first, and an invisible mesh is hit as well. It does respect layers and a material's side: with the default FrontSide, the back of a surface is not hit.

Explain it without notes

01

A click at clientX = 400, clientY = 200 on a canvas whose box is left 100, top 50, width 600, height 300 happens in the middle of the canvas. What are the NDC values, and what would the innerWidth formula give on a 1000 × 800 window?

02

A customer clicks a part that has a transparent glass cover in front of it. What does intersectObjects return, and how do you pick the part anyway?

Practice

01

Write ndcFromEvent(event, canvas) that returns a THREE.Vector2 in NDC from a pointer event, using the canvas's box.

02

Write firstPart(hits, names) that returns the first hit whose object, or any ancestor, has a name in names, or null.

Done when you can

  • Convert a pointer event to NDC with the canvas's own bounding box

  • Read hits[0] and handle an empty list

  • Say what a ray ignores (visible, opacity) and what it respects (layers, side)

  • Climb from a hit mesh to the named part