= Mouse Picking Demystified =
April 5, 2005
There comes a time in every 3D game where the user needs to click on something in the scene. Maybe he needs to select a unit in an RTS, or open a door in an RPG, or delete some geometry in a level editing tool. This conceptually simple task is easy to screw up since there are so many little steps that can go wrong.
The problem is this: ''given the mouse's position in window coordinates, how can I determine what object in the scene the user has selected with a mouse click?''
One method is to generate a ray using the mouse's location and then intersect it with the world geometry, finding the object nearest to the viewer. Alternatively we can determine the actual 3-D location that the user has clicked on by sampling the depth buffer (giving us (''x,y,z'') in viewport space) and performing an inverse transformation. Technically there is a third approach, using a selection or object ID buffer, but this has numerous limitations that makes it impractical for widespread use.
This article describes using the ''inverse transformation'' to derive world space coordinates from the mouse's position on screen.
Before we worry about the inverse transformation, we need to establish how the standard forward transformation works in a typical graphics pipeline.
This article is under construction!
{{{
#!latex-math-hook
\begin{eqnarray*}
&Model \\ &\downarrow \\
&World \\ &\downarrow \\
&View \\ & \downarrow \\
&Clip \\ & \downarrow \\
&Normalized Device\\ &\downarrow \\
&Viewport
\end{eqnarray*}
}}}
Some text:
{{{
#!latex-math-hook
{\boldsymbol M}{\boldsymbol p}&=&{\boldsymbol v}
}}}
More text:
{{{
#!latex-math-hook
\label{eqn:vp to ndc}
{\boldsymbol n} = \begin{pmatrix}
\frac{ 2{\boldsymbol v}_x }w - 1\\
\frac{ 2{\boldsymbol v}_y }{h}\\
2{\boldsymbol v}_z - 1\\
1
\end{pmatrix}
}}}