Lego Mouse – Programming an Air Mouse made of Lego

Note: .NET Framework 4.0 or above is required to run the PC binary program. Visual studio 2012 is required to open the PC source files.
Sources – http://hostr.co/bLF6GYxKOl08
Binaries – http://hostr.co/4esWCQuqVaXo

Introduction

An air mouse (or more precisely, a rotational mouse) is a mouse which makes use of an accelerometer instead of traditional means to detect motion. Usually, the rotation of the user’s wrist is what an air mouse picks up.

Today, we’ll be looking at how to actually develop an air mouse using the all-so-wonderful-and-universal Lego Kit. This post will consist of multiple parts explaining the various steps of developing and using this Lego air mouse.

The “Mouse”

The mouse was built using Lego elements. It is a minimalistic design comprising of the following elements:

The accelerometer sensor (on port 2) will be responsible for picking up motions by the user, while the touch sensors will be responsible for picking up “left click” (touch sensor on port 1) or “right click” (touch sensor on port 3) events. The NXT brick will communicate with the laptop (via bluetooth) and will be sending data from these sensors to it. Of course, the laptop will respond accordingly to the data sent to it.

The mouse will be programmed using the NXT Version of C, NXC (Not eXactly C). NXC is basically just C with a few modifications made specifically for the NXT Brick, hence the name “Not eXactly C”.

The Concept (Translating tilts to cursor movements)

The accelerometer used is a 3-axis accelerometer. This means that it can measure the acceleration along 3 distinct, orthogonal axes, namely the x-axis, y-axis and z-axis as denoted in the image above.

Gravity (or more precisely, acceleration due to gravity), coupled with the technique of vector resolution, is the main thing that helps us to determine the tilt of the mouse.
Mathematically speaking, the resolute (or projection) of a vector m along another vector n is given by the dot product of m with n, which is equivalent to $|\textbf{m}||\textbf{n}|\cos\theta$, where $\theta$ is the angle between m and n.
The sign of the resolute of gravity along any axis (or, the component of acceleration due to gravity along any axis) gives us a crucial information about that axis :

• If the resolute has no sign (that is, the resolute = 0), the axis is parallel to the ground. (I.E. at rest)
• If the resolute is positive, the axis is pointing towards the ground.
• If the resolute is negative, the axis is pointing away from the ground.

Which means to say, by knowing the sign of the acceleration along any axis (which is really just the resolute of gravity along that axis), one can determine the nature of the axis:

• If the acceleration is zero, the axis is parallel to the ground.
• If the acceleration is positive, the axis is pointing towards the ground.
• If the acceleration is negative, the axis is pointing away from the ground.

Because of this property, we can easily determine the tilt of the mouse along the x-axis and y-axis. Intuitively, we can then translate the tilts along these two axes into a movement of the cursor on the PC, as the tilts tells us how we can increment or decrement the x and y coordinates of the cursor.

Imperfections of the accelerometer

In the concept outline, we said that the acceleration along any axis will be 0 if it is parallel to the ground.  Unfortunately, what we perceive as a parallel rest position of the mouse with respect to the ground will never be perfectly parallel to the ground.

This means that we can not assume that by tilting the mouse such that it looks “parallel” to the ground, the acceleration measured along the x-axis and the y-axis will turn out to be 0.
As such, we have to define a position as the rest position of the mouse, and we must take all measurements with respect to this rest position.

Making room for shaky hands

No one can hold his or hand out still without any support. That is unless, of course, you are dead. Because of this, we would want our mouse to accommodate for the unintended shaking of hands. This is done by setting a threshold for which the magnitude of the resolute of gravity along the x-axis or y-axis must exceed before it can be registered as an intended tilt.

Enhancements – Incremental cursor speed

Initially, I felt that the cursor was moving too slowly and  I tried changing the speed. That is, I increased the constant that when multiplied with the sign of the accelerometer reading along an axis, gives the corresponding change in the cursor position on the PC. However, because of the increase in speed, the precision deproved tremendously.

In order to accommodate for both speed and precision, I’ve decided to make it such that the constant (mentioned before) is multiplied with the accelerometer reading directly instead of the sign of the accelerometer. That is, the cursor speed is now proportionate to the accelerometer’s tilt. By doing this, the mouse will be precise at low tilt angles, and fast at high tilt angles.

Connecting NXT Brick to laptop via Bluetooth

(This is assuming your NXT Brick has been paired with your laptop) You will need to know the incoming COM Port (NOT the outgoing COM port) of your NXT Brick on your laptop. It can be found under: Bluetooth Devices (the icon with a bluetooth icon in your system tray) > Open settings > COM Port. A COM Port looks something like “COM12”.

Afterwards, a Serial Port can be set up using the incoming COM Port, which can then be used to listen to incoming data from the NXT Brick to the laptop. The laptop can then move on the process the data sent and to move the cursor or to click any mouse buttons accordingly. All of these will be done with C# using the .NET Framework.

Of course, the NXT Brick must be connected to the PC first before any transaction can take place. This connection can be done under the Bluetooth menu of the NXT Brick. Note that you should connect to the PC through line 1 from the NXT.

Pairing and connecting are two separate procedures, so do check that you have done both if you find any problems.

Running the programs

The programs should be run in the following order for the mouse to work.

1. Run “Lego Mouse Server” on the PC
2. Enter the correct incoming COM Port and click the start button in the “Lego Mouse Server”
3. Connect the NXT Brick to the laptop
4. Run the program “Lego Mouse” on the NXT

NOTE : The mouse will calibrate itself once you run the “Lego Mouse” program. Calibration here means to set the rest position of the mouse which all future measurements will be taken with respect to. So make sure you have properly positioned the mouse in the correct position before running the program on the NXT.