Time to Evaluate

Final Survey Questions

The qualitative feedback section of my Survey contained only two questions:

• Finish this sentence. After opening the prototype lock i felt...
• Please write a short sentence describing your experience rotating the dial

I got much better feedback from these questions than I did on my original survey. I think by providing clearer instructions of what I type of feedback I was hoping to achieve I ended up with a much better result.

Question 1 Results

Responses for the first question were largely as I had hoped, evoking nostalgia, satisfaction and a sense of achievement and even vindication in users. This sense of accomplishment was exactly what I was setting out to achieve when designing the lock and what I was trying to capture with this question (its nice when things work out).

Answers to this question also brought up some further points for improvement, suggesting a change in unlocking sound to create a more outlaw style theme and to ensure the audio feedback would play as the person moved to the key point. Not as they landed on it, to avoid them from passing over it too many times and having to restart. 

Also confirmed my initial thoughts that the lock controls were a little awkward and raised an important point about the reliance on audio. If this channel is unavailable or if there is significant background noise the game in its current state would be unplayable.

Question 2 Results

This is where most of testers vented their frustration while some testers remarked that the controls were smooth and that the game was fun, there was a lot feedback on how to improve the system. Again the main take away was that the audio clicks needed to play sooner to ensure users would not overshoot the lock points and subsequently have to restart.

Observation Results

Tester 1: Observations

• Tester starts with Physical lock 9:10am
• Unlocked in 1min 30secs
• User only took one attempt before moving onto digital lock
• Tester starts with Digital Lock 9:14am
• 4 Locks completed in the 5min time frame
• 1st lock took the longest to complete
• Subsequent locks were opened in around 30seconds
• "Damn it went past it again"
• More support for changing the audio timings
• User was very engaged with the prototype, there were signs of concentration
• User used the spin method on the first lock
• subsequent locks used the tap method

Tester 2: Observations

• Tester starts with Physical lock 9:27am
• Unlocked in 1min with no real issues
• Checked password between moves
• Again only one attempt before moving onto digital
• Tester starts with Digital lock 9:30am
• 8 Locks completed in the 5min time frame
• 1st lock too the longest
• 5th and 6th lock only 15 seconds for completion
• 7th and 8th took beetween 20-30 seconds
• Clear pattern of learning after the first two locks, from locks 3-6 seemed everything really clicked for the tester. (excuse the pun)
• Used tap method throughout the test

Tester 3: Observations

• Tester starts with Physical lock 9:41am
• Unlocked in 1min 20secs
• One attempt before moving onto digital lock
• Tester starts with Digital lock 9:43am
• 3 locks completed in 4mins
• 1st lock took 30seconds
• 2nd lock took 45seconds
• Used tap method

Tester 4: Observations

• Tester starts wtih Physical lock 9:49am
• Reading through password before attempting to fix the lock
• One attempt before moving onto digital lock
• Tester starts with Digital lock 9:51am
• 3 locks completed in 5min time frame
• Some technical issues 2nd lock only took 30seconds
• Used the tap method
• experimented with spin on the first lock

Tester 5: Observations

• Tester starts with Physical lock 10:54am
• Was unable to unlock and opted to move to digital version
• Tester starts with Digital lock 10:56am
• 5 locks unlocked in 5min time frame
• User had issues with directions and keyboard mapping
• Settled on the tap method of controlling
• used the spin method for the first lock

Tester 6: Observations

• Tester starts with Digital lock 11:17am
• 2 locks completed in the 5min time frame
• Lots of moaning and groaning
• User was explaining that they kept just missing the point or just going too far
• Used a combination of spin and tap methods

Evaluations

Based on the observations and survey results I am very happy with the difficulty level of my prototype. While it was reviewed and was observed to be quite difficult while learning the sense of achievement that was reported was what I was hoping for with this game concept. I will be keeping the combination lock system as it gives a real safe cracking theme and the current code promotes precision. 

From the observation and the feedback it is also clear that the lock code needs to be further tested to ensure that it is stable and working after resets. But also that the sound clips need to be moved to ensure that they play while moving to the point rather than once the correct point has actually been selected.

Overall I feel like this was a much more successful testing session and am looking forward to implementing these changes into the design while also building a physical controller.