Selecting Objects in OpenGL

Continuing my exploits of building a 3D engine from scratch for no reason, I'm now moving on to object selection. From previous posts, I can now fly around basic 3D geometry but I want to be able to pick an object that's visible in the viewport using the mouse. This is made more difficult as OpenGL doesn't really have a concept of an 'Object' - it's just bothered with drawing triangles really fast - so asking it what cube is currently hovering under the mouse pointer is like watching snooker on a black and white TV. It's just not designed for it.

However, there are a few ways of getting this job done:

1) Use the OpenGL selection mode, redraw the small part of the screen around the mouse pointer and see what pixels are directly underneath

2) Do a depth test on the pixels

3) Redraw the scene using different colours for the objects and checking the pixel colour

For reasons of speed, the 3rd option is generally used, however, it hurt my head too much when I tried to get it too work. The second option is also very fast but can only really be used in particular circumstances, e.g. if you have a regular selection of objects that can be classified by their position like a chess board.
This left the first option - OpenGL's selection mode. This is actually thought to be the slowest but no-one seems to know why. However, for my needs, it worked and didn't bring the application to it's knees.

So, how to go about selecting objects in OpenGL? It basically just involves setting OpenGL to draw in a particular way and then calling your usual drawing code. To set up the drawing mode, the following code should work:

    // set up the selection buffer to store possible hits

    GLuint buffer[512];

    glSelectBuffer(512, buffer);

    // Get the viewport values

    GLint viewport[4];

    glGetIntegerv(GL_VIEWPORT, viewport);

    // Go into Selection Mode when rendering

    glRenderMode(GL_SELECT);

    glInitNames();  // Initializes The Name Stack

    glPushName(-1);  // Push at least one entry

    // go to projection matrix and limit the area around x,y to be 'drawn'

    glMatrixMode(GL_PROJECTION);

    glPushMatrix();  // Push The Projection Matrix

    glLoadIdentity(); // Resets The Matrix

    // set the matrix to only view around x,y (inverting y in this case)

    gluPickMatrix((GLdouble) x, (GLdouble) (viewport[3]-y), 1.0f, 1.0f, viewport);

    // also set the perspective to ensure the new aspect ratio is correct

    gluPerspective(45.0f, (GLfloat) (viewport[2]-viewport[0])/(GLfloat) (viewport[3]-viewport[1]), 0.1f, 1500.0f);

    // now paint the objects

    glMatrixMode(GL_MODELVIEW);

    parentView_->drawNodes();

This code:

• sets up a selection buffer to store the hits, 
• sets the render mode to GL_SELECT (the OpenGL selection mode) and initialises the name stack. 
• Stores, then resets the Projection matrix before limiting it to 1 pixel around x,y position
• Finally, the objects are drawn as usual

The only additional code needed when drawing your objects is to add:

glLoadName( i );

where i is an integer that you can use to identify the object being drawn as this is what will be returned from the selection tool.

Finally, we can see what has been 'drawn' in our 1x1 pixel box:

    // switch everything back to where it was before we started messing
    glMatrixMode(GL_PROJECTION);
    glPopMatrix();
    glMatrixMode(GL_MODELVIEW);
    // check for hits by switching render mode
    GLint hits=glRenderMode(GL_RENDER);
    // do we have any?
    if (hits > 0)
    {
        // note: selection buffer has 4 values per hit: # of hits at time, min depth, max depth, name
        // start by picking the first hit
        int choose = buffer[3];
        int depth = buffer[1];

        // now loop over the rest
        for (int loop = 1; loop < hits; loop++)
        {
            // is this object closer? Note - need to offset for the 4 values per hit
            if (buffer[loop*4+1] < GLuint(depth))
            {
                choose = buffer[loop*4+3];
                depth = buffer[loop*4+1];
            }
        }
        return choose;
    }
    return -1;


And that's it! As I say, there is a lot of talk saying that this is quite slow and I've no doubt that's the case. However, for a basic selection routine this is easy to code and not too tricky to understand. If I discover this is causing me problems I'll have to move on to the more complicated 'colour selection' version but for now, this works for me!

Find the code (getObjectAtScreenPos) at:

https://github.com/doc-sparks/Interface/blob/v0.6/oglwidget.cpp

Using Basic Lighting in OpenGL

Let there be light. And let it be phong shaded.

Returning to my basic 3D cube from previous posts, I wanted to stop it looking quite so much like Jason's cube-of-many-colours and just have one colour that could then be changed depending on the status of what it represented. The problem here is that just setting the colour for each face to the same results in what is essentially shadow puppetry. There's no variation in the colours due to the environment and so they are all solid.

How do we get around this as easily as possible? Use the basic lighting supplied by OpenGL. Note that I'm not trying to do photo-realistic shading of a cube here, all I want is for the colour of the cube's faces to vary depending on how much light they are seeing. What this amounts to is finding angle between the light coming in and the 'normal' of the face - i.e. the direction perpendicular to the plane that the face is on. To do more complicated lighting you'd want to use shaders but that's serious overkill for what I'm looking for.

Anyway, there are several things that need to be done to get this working as intended - enable lighting in the OpenGL, position the light, set the normals and set the materials. The first part basically boils down to the following code:

GLfloat white_light[]= { 1.0f, 1.0f, 1.0f, 1.0f };    // set values for a white light
glLightfv(GL_LIGHT1, GL_DIFFUSE, white_light);        // set the DIFFUSE colour of LIGHT1 to white
glLightfv(GL_LIGHT2, GL_DIFFUSE, white_light);        // set the DIFFUSE colour of LIGHT2 to white

glEnable(GL_LIGHT1);                                  // enable the lights
glEnable(GL_LIGHT2);
glEnable(GL_LIGHTING);                                // enable lighting in general

All that's happening here is I'm setting the values for the two lights I'm going to use and turning them on. You generally have access to 8 lights (I think) depending the implementation. I'm setting the DIFFUSE (light reflected everywhere) value to white so the surfaces it hits will just reflect the colour that I set the material to. I'm not bothering with any SPECULAR (light reflected like a mirror) or AMBIENT (general light) as it's not needed for this at present (and I *think* Ambient is set by default).

Next we need to position the light:

// set the light position
GLfloat light_pos1[]= { 1.0f, 1.0f, 3.0f, 0.0f };
glLightfv(GL_LIGHT1, GL_POSITION, light_pos1);
GLfloat light_pos2[]= { 1.0f, -1.0f, -3.0f, 0.0f };
glLightfv(GL_LIGHT2, GL_POSITION, light_pos2);

We need to be careful about where this code it. It needs to be after all 'camera' translations/rotations but before drawing any objects. If you put it in the wrong place, you'll get weird effects like the light changing when you move the camera. Note that I'm also using two lights as if the face is pointing away from the light, you get no diffuse light at all.

Next on the list is the normal vectors. For a cube this is fairly trivial to work out but for more complicated geometry, these would usually be loaded with the object or calculated when on loading. For my cube, something similar to the following is needed:

glNormal3d(0, 0, 1);
glVertex3f(  0.5, -0.5, 0.5 );
glVertex3f(  0.5,  0.5, 0.5 );
glVertex3f( -0.5,  0.5, 0.5 );
glVertex3f( -0.5, -0.5, 0.5 );

Obviously the important bit is the glNormal3d command which describes the vector of the normal for this face. Similar calls are made for all of the cube faces.

And finally, the last bit setting the material of the cube. When using lighting, you can no longer just use glColor and must use glMaterial instead. This allows you to set the various aspects of how the object reacts to light, namely how the AMBIENT, DIFFUSE and SPECULAR light is reflected back. As I said above, I really care most about the DIFFUSE light in this case which will just colour the faces depending on their direction relative to the light:

GLfloat red[] = {0.8f, .2f, .2f, 1.f};
glMaterialfv(GL_FRONT, GL_AMBIENT_AND_DIFFUSE, red);

Note that this should be placed before calling the display lists/glVertex comands. I'm also setting the AMBIENT a little as well as there is some of this by default.

With all these elements in place, you should get cubes that are a single colour but are shaded appropriately given the angle of the light! Next problem to overcome - overpainting!

References:
http://www.cse.msu.edu/~cse872/tutorial3.html
http://nehe.gamedev.net/tutorial/texture_filters_lighting__keyboard_control/15002/

Code:
https://github.com/doc-sparks/Interface/tree/v0.5

(Note that the tag includes the changes for overpainting as well which basically means some init code has been switched to the paintEvent function)

Moving Around a 3D cube with Mouse and Keyboard (Part 2)

3D rotations broke my brain

So in the first part of this post, I got the cube to respond to mouse movements in so you can rotate around it. Inspiring stuff. Next, I want to add keyboard control using good old WASD movement. The movement I want to recreate is your typical RTS style mouse movement of having the camera rotate around a point (done) and then move that point over a plane using the keyboard (definitely not done).

First things first: How do we check for keyboard input? This is actually not as quite as simple as just triggering on a key press event as if you hold down a key as these don't fire often enough. It is also very difficult (maybe impossible) to poll the actual keyboard hardware in an OS-independent way. However, we can use the Qt-provided functions keyPressEvent and keyReleaseEvent to track the state of the keyboard and act accordingly. To do this, just override these functions in the MainWindow object (this is what gets the keyboard events by default) and update a QMap to the status of each key. The actual code I've added is quite simple and is shown below:

header:

public:    
    // check key status
    bool isKeyDown(int key);

private:
    // keyboard map for deciding key presses
    QMap keyboardMap_;

protected:
    void keyPressEvent(QKeyEvent *event);
    void keyReleaseEvent(QKeyEvent *event);

cpp file:

void MainWindow::keyPressEvent(QKeyEvent *e)
{
    // if we're quitting, then fine
    if (e->key() == Qt::Key_Escape)
    {
        close();
        return;
    }

    // otherwise update the keyboard map
    keyboardMap_[ e->key() ] = true;
}

void MainWindow::keyReleaseEvent(QKeyEvent *e)
{
    // update the keyboard map
    keyboardMap_[ e->key() ] = false;
}

bool MainWindow::isKeyDown(int key)
{
    // check in the map to see if the key is down
    if (keyboardMap_.contains(key))
        return keyboardMap_[ key ];
    else
        return false;
}

So we grab any key press or release events and then simply update the QMap with state for this key code. After that, all we need is an access function to allow the widget to query the key state and move the view accordingly.

Now, to actually move the view accordingly takes a little bit of thought. We have to be a little careful as to where we put the translation given by the keyboard movement in order to give the RTS style rotate-around-a-point camera we're looking for. At present, we have:

    // move into the screen
    glTranslatef(0.0f, 0.0f, -6.0f);

    // rotate the cube by the rotation value
    glRotatef(rotValue_.y(), 1.0f, 0.0f, 0.0f);
    glRotatef(rotValue_.x(), 0.0f, 1.0f, 0.0f);

To apply the lateral movement, we need to think about which order to perform these translations and movements in to get the affect we want, remembering that we are transforming the world relative to the camera. This turns out to be:

1. Translate back by the zoom factor 
2. Rotate the coordinate system around the origin (equivalent to rotating the camera)
3. Translate the view to the current focus point position

Applying these gives the following code:

    // reset the view to the identity
    glLoadIdentity();

    // move everything back by the zoom factor
    glTranslatef(0.0f, 0.0f, -zoomValue_);

    // rotate everything
    glRotatef(rotValue_.y(), 1.0f, 0.0f, 0.0f);
    glRotatef(rotValue_.x(), 0.0f, 1.0f, 0.0f);

    // finally offset by the current viewing point
    glTranslatef(posValue_.x(), posValue_.y(), 0.0f);

This almost gives us the keyboard control we were looking for. However, as it stands, if you just polled the key status and increased or decreased the x and y values, you would always be moving on those axes. What we really want is to move relative to the direction we're facing. Unfortunately, this is where we can't avoid some trigonometry as we need to take the movement speed and angle of rotation around the vertical axis to give the change in x and y values needed. Long story short, this code in a new 'mainLoop' function does the job:

    // check for keyboard movement
    if (parentWin_->isKeyDown(65))  // A
    {
        posValue_.setY( posValue_.y() + (0.05 * sin( PI * rotValue_.x() / 180.0) ) );
        posValue_.setX( posValue_.x() + (0.05 * cos( PI * rotValue_.x() / 180.0) ) );
    }

    if (parentWin_->isKeyDown(68))  // D
    {
        posValue_.setY( posValue_.y() - (0.05 * sin( PI * rotValue_.x() / 180.0) ) );
        posValue_.setX( posValue_.x() - (0.05 * cos( PI * rotValue_.x() / 180.0) ) );
    }

    if (parentWin_->isKeyDown(87)) // W
    {
        posValue_.setY( posValue_.y() + (0.05 * cos( PI * rotValue_.x() / 180.0) ) );
        posValue_.setX( posValue_.x() - (0.05 * sin( PI * rotValue_.x() / 180.0) ) );
    }

    if (parentWin_->isKeyDown(83))  // S
    {
        posValue_.setY( posValue_.y() - (0.05 * cos( PI * rotValue_.x() / 180.0) ) );
        posValue_.setX( posValue_.x() + (0.05 * sin( PI * rotValue_.x() / 180.0) ) );
    }

Note the conversion from degrees (as accepted by glRotatef) and radians (as accepted by sin/cos).

Things to note:

• I've added mouse wheel zoom by overloading mouseWheelEvent and clamping the zoom value.
• In order to poll the keyboard state and a fast enough rate, I've added a mainLoop slot function that is attached to the timer and then calls the updateGL function.
• In order to call into the parent window's keyboard map, you need to make the widget aware of it and I personally prefer to store this pointer in a member variable through the constructor rather than having a global variable or static singleton type framework.
• The rotation/translation order can be difficult to get your head around - try to remember that the camera is static and the transformations apply to the coordinate system!

And we're done! We now have a mouse and keyboard controlled scene to zoom around.

Find the code at:

https://github.com/doc-sparks/Interface/tree/v0.3

Moving Around a 3D cube with Mouse and Keyboard (Part 1)

I never got the hang of XBox controllers

So moving on from a painfully coloured 3D spinning cube created in OpenGL and Qt (see here), I now want to add user input via a mouse and keyboard. The first part of this will be fairy easy and just result in being able to rotate the cube when ever the middle mouse button is pressed - 'mouse look' for those in the know.

So, first up, we need to put an additional function in for checking mouse movement. Qt makes this very easy (not surprisingly) as all widgets can catch mouse events by overriding the mouseMoveEvent. So after adding the following:

header file: 

protected:
   void mouseMoveEvent(QMouseEvent *event);

cpp file:

void OGLWidget::mouseMoveEvent(QMouseEvent *event)
{

}

We now have a function that will catch any mouse movement. Or mostly, anyway. This will only work with movement when a button is pressed (dragging basically) which won't quite work for this. You therefore should add the following to the constructor of the widget:

setMouseTracking(true);

This will set the widget to track ALL mouse movements.

So now, how do we ensure the cube moves around with the mouse and stops when we release the button? We need to keep two 'temporary' (but member) variables that record the rotation of the cube and mouse coords. The new rotation is then calculated based on the difference between this mouse position and the current one, which is then added to the stored rotation when the mouse button was pressed. As with pictures, code is often worth a thousand words so all of that can probably more easily be understood by viewing the following:

void OGLWidget::mouseMoveEvent(QMouseEvent *event)
{
    // is the middle mouse button down?
    if (event->buttons() == Qt::MidButton)
    {
        // was it already down? If not, store the current coords
        if (!mouseLook_)
        {
            tmpMousePos_ = event->pos();
            tmpRotValue_ = rotValue_;
            mouseLook_ = true;
        }

        // update the rotation values depending on the relative mouse position
        rotValue_.setX( tmpRotValue_.x() + (tmpMousePos_.x() - event->pos().x()) * 0.2 );
        rotValue_.setY( tmpRotValue_.y() + (tmpMousePos_.y() - event->pos().y()) * -0.2 );
    }
    else
    {
        // turn off mouse look
        mouseLook_ = false;
    }
}

Things to note:

• I've got an additional flag showing whether mouse look is on or not - I could have set one of the other tmp variables to a special value but this is almost never a good idea and variables are (generally) cheap.
• I've applied a factor of 0.2 to each mouse movement. This is basically the mouse speed and should be configurable an ideal world
• To incorporate two axis rotation, I've changed rotValue_ to a QPoint type where x stores the y-axis rotation and y stores the x axis rotation.

So you can now rotate the cube when holding down the middle mouse button. Next time, we try to actually move the camera using the WASD keys. This will require a few more changes to the rendering code.

Find the code at:

https://github.com/doc-sparks/Interface/tree/v0.2

A Spinning Cube with OpenGL and Qt

At one point, even Crysis looked like this

As I mentioned before, I've always liked 3D art and games. I guess it comes from seeing the transition from basic 2D platformers to the 3D awesomeness that was Doom first hand (and if you actually need to click on that link to know what Doom is, you should be ashamed). I still have (vague) ideas of doing my own at some point but that is obviously now a lot easier with things like Unity and the Unreal Engine. There is very little need to code your own engine these days unless you're a major game studio (in which case you're probably not reading this).


Having said all that, I think it's always good to go over some of the basics of these technologies so you have a vague idea how they work and are coded. Plus, there are many situations where the pre-packaged engines aren't useful for what you're trying to do (as in this case here - but more on that in another post some time down the line!). To that end, I decided to come up with the minimal startup to running an OpenGL program within the Qt framework. This would provide me with a good basis for going forward in anything 3D related in the future and also help me understand the basic requirements of an OpenGL program.

Note: There are many good tutorials on the web for this (I personally  use Neon Helium). I'm putting this here (as with all my posts) to record my own personal experience and to help me remember just what I need to know!


To start with,  create a New Project in Qt: New Project -> Qt Widget Project -> Qt GUI Application. This sets you up with a basic main window and main cpp file. Now, add in a new widget that will be your main OpenGL widget (Right Click project -> Add New... ->C++, C++ Class and make sure you set the base class as QGLWidget and Type as QObject).


This sets up the widget class for you to add to. Next thing is to make this appear (and take over) the main window. So add the following to the Main Window Constructor:

 
MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent)
{
    // Show the interface fullscreen
    showFullScreen();

    // create and add the openGL Widget
    OGLWidget *w = new OGLWidget();
    setCentralWidget(w);
}

You may also want to override the protected keyPressEvent to allow you to quit out:

 
void MainWindow::keyPressEvent(QKeyEvent *e)
{
    if (e->key() == Qt::Key_Escape)
        close();
    else
        QWidget::keyPressEvent(e);
}

This has setup the basics for Qt, now comes the OpenGL bit. There are 3 methods you need to override in OGLWidget (or whatever your widget is called) to get your program to actually show anything (note the includes required - put these at the top of your implementation file):

initializeGL

 
#include "oglwidget.h"
#include <GL/glu.h>
#include <QDebug>
#include <QTimer>
#include <QMouseEvent>

void OGLWidget::initializeGL()
{
    // enable depth testing - required to stop back faces showing through (back face culling)
    glEnable(GL_DEPTH_TEST);

    // set up the timer for a 50Hz view and connect to the update routine
    refreshTimer_ = new QTimer(this);
    connect(refreshTimer_, SIGNAL(timeout()), this, SLOT(updateGL()));
    refreshTimer_->start(20);
}

• Called (not surprisingly) just before the first call to resizeGL or paintGL
• The only OpenGL thing done here is to set the depth Test through glEnable(GL_DEPTH_TEST); Not doing this can make back faces show through.
• Other than that, I just setup a timer to repaint the screen

resizeGL

 
void OGLWidget::resizeGL(int width, int height)
{
    // Set the viewport given the resize event
    glViewport(0, 0, width, height);

    // Reset the Projection matrix
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();

    // Calculate The Aspect Ratio Of The Window and set the perspective
    gluPerspective(45.0f,(GLfloat)width/(GLfloat)height,0.1f,100.0f);

    // Reset the Model View matrix
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();
}

• Called on the resize of the widget with the width and height of the widget passed through
•  This is where the viewport is setup and the basic matrices are reset to the identity
• glViewport - Set up where in the widget to show the 3d view
• glMatrixMode - Set which matrix to use. Of interest here is the ModelView matrix (from local object coords to eye or camera view) and Projection matrix (how the eye coords are projected and clipped to the screen). Both are set to the identity here.
• gluPerspective - A GLUT routine that sets a nice viewing frustrum with a z clipping plane

paintGL

 
void OGLWidget::paintGL()
{
    // cler the screen and depth buffer
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    // reset the view to the identity
    glLoadIdentity();

    // move into the screen
    glTranslatef(0.0f, 0.0f, -6.0f);

    // rotate the cube by the rotation value
    glRotatef(rotValue_, 0.0f, 1.0f, 0.0f);

    // construct the cube
    glBegin(GL_QUADS);

    glColor3f(   1.0,  1.0, 1.0 );
    glVertex3f(  0.5, -0.5, 0.5 );
    glVertex3f(  0.5,  0.5, 0.5 );
    glVertex3f( -0.5,  0.5, 0.5 );
    glVertex3f( -0.5, -0.5, 0.5 );

    // And 5 others like this...

    glEnd();

    // finally, update the rotation
    rotValue_ += 0.2f;
}

• Called on any redraw event
• Here the actual polygons are drawn
• glClear - used to clear both the color buffer (basically clearing the screen to a colour set using glClearColor) and the depth buffer as well
• Reset the MODLVIEW matrix (the default here) to the identity
• Apply both a translation and rotation to the current (modelView) matrix
• Finally, setup to draw quads and set the colour and vertex positions for all faces of the cube

And there we have it! This produces a basic spinning cube in front of the camera while using a full screen Qt window to display it. Next, mouse control...

Update: The code for this can be found in my github repo:

https://github.com/doc-sparks/Interface/tree/v0.1