Its the Final Countdown

Quantitative Testing Session Results

Another testing session bites the dust and its time to look at the results of the latest iteration. In this post I will look at the survey results and how the relate to the goals for this prototype and breakdown the responses from my five victims. This will be looked at the quantitative data recorded with a separate entry for session observations and quantitative results.

Of the testers 4/5 had prior experience with the Street Survivor prototypes which was important for testing the effectiveness of the visual feedback relative to the physical method used in prototype two. Of the four that had previous experience 75% said that they found that the locks were easier to open with this prototype.

Interesting though a significant number of users still had trouble rotating the dial in the correct direction, despite the visual feedback.

Results for Question 3.

This trouble was also reflected in question four with most two users reporting they had repeated troubles. Rotating the dial in the wrong direction more than five times.

Results for Question 4

This is an interesting development as again it seems that users are focusing their attention entirely on rotating the lock rather than the supplementary information provided. Even a result of 3-4 times could be considered a failure for smooth operation of the controls. From this result further refinements would be to invert the control scheme or alternatively rebuilt a high quality physical system.

Despite their troubles users still reported that the visual aids were helpful to their playing of the game.

Results for Question 8

Again this seems counter intuitive, but in hindsight an important design flaw may be behind this result. Both the password progress system and the visual indicator were classified as 'visual feedback' systems in both the survey and testing session wording. These will need to be separated in future testing sessions and potentially even having the code modified to include only one of the features.

Results for Question 7

The results from question seven make it clear that the password progress box was helpful in improving users experience and may have been the primary reason for users finding the locks less difficult to unlock. Overall the difficulty seems to have stabilized to an adequate level where it is sufficiently challenging as the results from question six suggest.

Results for Question 6

From these results it seems clear that the added features have not lead to an improvement in users ability to rotate the dial in the correct direction. Further work will need to be done in this space to ensure that future builds are more usable. It also shows that the locks are easier to open and potentially reduces the reward felt by users when opening a lock.