Selecting Objects in OpenGL

Continuing my exploits of building a 3D engine from scratch for no reason, I'm now moving on to object selection. From previous posts, I can now fly around basic 3D geometry but I want to be able to pick an object that's visible in the viewport using the mouse. This is made more difficult as OpenGL doesn't really have a concept of an 'Object' - it's just bothered with drawing triangles really fast - so asking it what cube is currently hovering under the mouse pointer is like watching snooker on a black and white TV. It's just not designed for it.

However, there are a few ways of getting this job done:

1) Use the OpenGL selection mode, redraw the small part of the screen around the mouse pointer and see what pixels are directly underneath

2) Do a depth test on the pixels

3) Redraw the scene using different colours for the objects and checking the pixel colour

For reasons of speed, the 3rd option is generally used, however, it hurt my head too much when I tried to get it too work. The second option is also very fast but can only really be used in particular circumstances, e.g. if you have a regular selection of objects that can be classified by their position like a chess board.
This left the first option - OpenGL's selection mode. This is actually thought to be the slowest but no-one seems to know why. However, for my needs, it worked and didn't bring the application to it's knees.

So, how to go about selecting objects in OpenGL? It basically just involves setting OpenGL to draw in a particular way and then calling your usual drawing code. To set up the drawing mode, the following code should work:

    // set up the selection buffer to store possible hits

    GLuint buffer[512];

    glSelectBuffer(512, buffer);

    // Get the viewport values

    GLint viewport[4];

    glGetIntegerv(GL_VIEWPORT, viewport);

    // Go into Selection Mode when rendering

    glRenderMode(GL_SELECT);

    glInitNames();  // Initializes The Name Stack

    glPushName(-1);  // Push at least one entry

    // go to projection matrix and limit the area around x,y to be 'drawn'

    glMatrixMode(GL_PROJECTION);

    glPushMatrix();  // Push The Projection Matrix

    glLoadIdentity(); // Resets The Matrix

    // set the matrix to only view around x,y (inverting y in this case)

    gluPickMatrix((GLdouble) x, (GLdouble) (viewport[3]-y), 1.0f, 1.0f, viewport);

    // also set the perspective to ensure the new aspect ratio is correct

    gluPerspective(45.0f, (GLfloat) (viewport[2]-viewport[0])/(GLfloat) (viewport[3]-viewport[1]), 0.1f, 1500.0f);

    // now paint the objects

    glMatrixMode(GL_MODELVIEW);

    parentView_->drawNodes();

This code:

• sets up a selection buffer to store the hits, 
• sets the render mode to GL_SELECT (the OpenGL selection mode) and initialises the name stack. 
• Stores, then resets the Projection matrix before limiting it to 1 pixel around x,y position
• Finally, the objects are drawn as usual

The only additional code needed when drawing your objects is to add:

glLoadName( i );

where i is an integer that you can use to identify the object being drawn as this is what will be returned from the selection tool.

Finally, we can see what has been 'drawn' in our 1x1 pixel box:

    // switch everything back to where it was before we started messing
    glMatrixMode(GL_PROJECTION);
    glPopMatrix();
    glMatrixMode(GL_MODELVIEW);
    // check for hits by switching render mode
    GLint hits=glRenderMode(GL_RENDER);
    // do we have any?
    if (hits > 0)
    {
        // note: selection buffer has 4 values per hit: # of hits at time, min depth, max depth, name
        // start by picking the first hit
        int choose = buffer[3];
        int depth = buffer[1];

        // now loop over the rest
        for (int loop = 1; loop < hits; loop++)
        {
            // is this object closer? Note - need to offset for the 4 values per hit
            if (buffer[loop*4+1] < GLuint(depth))
            {
                choose = buffer[loop*4+3];
                depth = buffer[loop*4+1];
            }
        }
        return choose;
    }
    return -1;


And that's it! As I say, there is a lot of talk saying that this is quite slow and I've no doubt that's the case. However, for a basic selection routine this is easy to code and not too tricky to understand. If I discover this is causing me problems I'll have to move on to the more complicated 'colour selection' version but for now, this works for me!

Find the code (getObjectAtScreenPos) at:

https://github.com/doc-sparks/Interface/blob/v0.6/oglwidget.cpp