IP3 - Testing and Evaluation

Today was the very last testing session for this course. Overall I feel it went pretty well. For the 3rd prototype of Hop to Who, I didn't make any huge changes, mostly it was a refinement of the game based on last session's feedback.

I had a total of 6 participants play the game. Three of whom had already played in previous testing sessions so were pretty familiar already with the game and the remaining three testers playing for the very first time. I thought it was really beneficial to observe the difference in the participant's behaviours and responses due to this.

Overall the response was very positive and testers responded very positively to the questions about their general understanding of the game and their enjoyment levels. All said they understood the goal of the game, and all rated it a 4 or 5 on the 5 point scale of enjoyment of the game.




In this prototype I decided to rely less on myself giving verbal instructions, and added How to Play instructions to the splash screen to be read on start up. I think this decision resulted in less of an immediate understanding of the game rules, and more of a learning curve. I noticed most players didn't spend a lot of time reading the instructions and were keen to play the game as fast as possible. Perhaps for a future change a more interactive tutorial could be integrated so players do not have to read a long written explanation about how to play, rather learn by doing.
Despite this all participants still quickly grasped the game controls and game mechanics within 1 or 2 times in playing the game. So I feel confident that the majority of players could quite easily start playing the game of the first time without the need for in-person facilitation, and learn fairly quickly how the game works.




Including an easy level definitely helped ease players into the game before it got too challenging. From my observation, 5 out of 6 testers started at the easy level and worked their way up to the hard level. Every player actually got a chance to win the game at least once, which is a vast improvement from the last testing session. The outlier participant (who actually was a first time player) didn't attempt easy at all, he jumped straight into the medium level for 1 turn then tried the hard level 2 times - he managed to win a lot and picked up the game play very quickly. The survey results were a little mixed, most rated it an average of 3 (where 1 is too easy and 5 too difficult), 2 rated it a 4, and one person rated it a 1 (too easy - the outlier tester). It was interesting to note the varying skill levels of the participants, especially the one who was very skilled on the first time playing, compared to another player who had previously tried the game before in the last session, but still struggled a little on the easy level. Seeing these difference in levels, I feel supports my decision in including the different difficulty levels, so everyone regardless of skill, can still enjoy the game. In the future I could implement even more challenging levels to cater for those who find the game too easy.


As for the physical controls, they worked a lot better than in the last round of testing, although still could use a bit of reworking to ensure the tiles don't bounce as much. If I had more time I would move the position of the contacts so even if the tile did bounce, it wouldn't affect the Makey Makey from registering input.

Another feedback suggestion I'd implement is to remind the player to reset all the tiles before replaying. If this isn't done at the correct time before pressing the replay button, can disrupt the game as the program thinks those tiles have already been eliminated.

Survey link:
http://goo.gl/forms/mIRqJAnZBz

View the summary of responses here:
https://docs.google.com/forms/d/1XMu8zuWiq71SCMVKwqRIt1wVrZILvw3hIjOje1wk6QU/viewanalytics

IP3 Concept

Hop to Who?! combines the games Guess Who and Frogger. There are two main aims of the game, firstly to safely guide the frog across the road, and second to guess the identity of the frog’s mystery date waiting on the other side of the road. This game is aimed at those aged 5 years and up. It’s purpose is to mix elements from both games in order to create a game that is both challenging and fun.

A physical Guess Who tile board is the physical input for the game. Each “tile” shows a different and unique character. As the game progresses, clues about the chosen single mystery date will be shown on screen, allowing the player to eliminate incorrect characters by dropping a tile face down. Each time a tile is put down, this controls the forward movement of the main frog on screen.

The Story of "Hop to Who?!":

The Frog needs to cross the road to meet his blind date. As he crosses, the game will display clues about the characteristics of the blind date. The player needs to both get across the road and match the mystery date with the final remaining character tile.

Frogger elements:

• The game objective of getting the frog safely across the road.
• Timing the moves to avoid getting hit by moving traffic.
• The frog character and the level design.

Guess Who elements:

• Eliminating all the characters displayed so only 1 remains
• Giving descriptive clues to help the player eliminate the character tiles.
• Physical interaction of flipping down tiles.

How to play:

• The on-screen interface will display the game interface, there also will be a physical Guess Who style board with 10 character tiles shown face up.
• There are 2 objectives, both equally important. Get the frog across the road to safety and eliminate all but one of the characters correctly.
• The onscreen clues will give the player hints as to which frog tile to drop down.
• Each time a tile drops down, the frog on screen will move forward one space
• As the game progresses, further clues will be given to help eliminate frogs.
• The challenge is to flip the tiles down in sync with moving the frog to avoid collisions. This requires 2 mental processes to happen at once, strategic elimination and quick response.
• If the player gets the frog to safety, then there is the added reveal to check if the player has selected the right character.
• If the remaining tile matches the mystery date, the frog will live happily ever after. Otherwise sadly the frog goes home alone.
• There are 3 levels to choose from, Easy, Medium or Hard

Game Updates:

From the last prototype released, there have been a few iterations integrated in to the latest version:

• The game board itself has been updated. As the tiles bounced when dropped down, I used conductive foam, placed where the tile's contacts meet to help prevent this, and reduce noise. 
• A splash screen has been added, to give a short overview of the game (so I don't have to give a detailed explanation to participants every time they play). 
• Three difficulty levels have been implemented, based on the speed of the traffic.
• Updated some graphics, such as the trucks
• Removed the number text at the top (which previously indicated which tile numbers had already been eliminated, as past testers paid no or little attention to these).

Post-testing survey link: http://goo.gl/forms/rXBebqb2jx

IP2 - Testing and Evaluation

Today's testing session went pretty well. I had a total of 10 users play the game and complete a survey.

This latest prototype generally achieved it's purpose, and all of the users testing it had little problem grasping the concept of the game and how the physical controls operated. 100% of users responded that they understood the game's concept. 8 out of 10 understood it well from my instructions given, 2 users would've preferred some more explanation. Especially for those who had never played Guess Who, I could've explained the elimination concept more clearly in detail, as those who never played had trouble choosing the correct character tiles to eliminate at first.

The game board was successful and I received positive feedback on it's construction. There were a few technical issues, where 2 of the tiles weren't able to make an adequate contact with the earth points. As the tiles are pretty thick and heavy, the steel wool helped to cushion the impact when the tile drops, but for one tile (#6) it broke off, so when it was dropped, sometimes the metal to metal contact caused the tile to bounce and not register a move - usually causing the frog character to get hit by a truck. Another tile's hinge was a bit too tight (#4), so it didn't drop down easily, also causing problems with the game. At times this resulted in the user losing the game, when they otherwise would've won. 3 users noted in the survey to improve the tech breakdowns on the game board as an issue, so this is something I will definitely improve in the next iteration.

Most participants grasped the guessing/elimination concept quickly. 4 of the 10 users had a little trouble matching the clues to the eliminate the correct frogs - for example some users had trouble eliminating the right tiles after receiving a clue that used a negative style description, "I am not wearing a hat" - users would put down tiles of the frog's not wearing a hat, rather than eliminating the opposite frogs with hats. However, no one had trouble with the opposite clue, "I'm wearing a hat", users all correctly eliminated frogs not wearing hats.

I personally don't think I'd change this type of negative clue, as it does require a bit more mental challenge, and over playing multiple times, users tended to realise this mistake and self-correct to eliminate the right character tiles.

I received positive feedback regarding enjoyment and fun users had, and about the uniqueness of the controls. 7 users rated the game 5 out of a 5 point scale (5 being the best score), and the remaining 3 rated it a 4. As for uniqueness, 7 rated it 5 out of 5, 2 users rated it a 4, and 1 user rated it a 3 (5 being the best score).

As for the difficulty, there was a wider spread of replies, this may be due to the fact that the earlier sessions, the speed of the trucks was faster and the earlier users found it a bit too hard, so I compensated later by slowing the speed down. Half of the users found the difficulty suitable, rating it an average of 3 (challenging, but not impossible), and an the rest evenly spread on either side of this rating. Some user suggestions included creating levels to help ease players in and teach them how to play, gradually making the levels more difficult. Another was to be able to choose a difficulty level. This is a definite consideration for the next prototype.

  

   

Unfortunately I didn't capture any recording of wins - here's some recording of players almost winning, one user would've won if my game board didn't fail :(

Survey Link:
http://goo.gl/forms/ayAlSBwWgu

Summary of responses:
https://docs.google.com/forms/d/1JfgVLDcQDcQhkIdgUM1F2P1qE-_GWyKw7wCLIwAiEFk/viewanalytics

IP2 - Concept

Hop to Who?! combines the games Guess Who and Frogger. There are 2 main aims of the game, firstly to safely guide the frog across the road, and second to guess the identity of the frog’s mystery date waiting on the other side of the road. This game is aimed at those aged 5 years and up. It’s purpose is to mix elements from both games in order to create a game that is both challenging and fun.

A physical Guess Who tile board is the physical input for the game. Each “tile” shows a different and unique character. As the game progresses, clues about the chosen single mystery date will be shown on screen, allowing the player to eliminate incorrect characters by dropping a tile face down. Each time a tile is put down, this controls the forward movement of the main frog on screen.

The Story of "Hop to Who?!":

The Frog needs to cross the road to meet his blind date. As he crosses, the game will display clues about the characteristics of the blind date. The player needs to both get across the road and match the mystery date with the final remaining character tile.

Frogger elements:

• The game objective of getting the frog safely across the road.
• Timing the moves to avoid getting hit by moving traffic.
• The frog character and the level design.

Guess Who elements:

• Eliminating all the characters displayed so only 1 remains
• Giving descriptive clues to help the player eliminate the character tiles.
• Physical interaction of flipping down tiles.

How to play:

• The on-screen interface will display the game interface, there also will be a physical Guess Who style board with 10 character tiles shown face up.
• There are 2 objectives, both equally important. Get the frog across the road to safety and eliminate all but one of the characters correctly.
• The onscreen clues will give the player hints as to which frog tile to drop down.
• Each time a tile drops down, the frog on screen will move forward one space
• As the game progresses, further clues will be given to help eliminate frogs.
• The challenge is to flip the tiles down in sync with moving the frog to avoid collisions. This requires 2 mental processes to happen at once, strategic elimination and quick response.
• If the player gets the frog to safety, then there is the added reveal to check if the player has selected the right character.
• If the remaining tile matches the mystery date, the frog will live happily ever after. Otherwise sadly the frog goes home alone.

Making IP2

For the 2nd interactive prototype, not a lot changed concept wise, but I made a lot a changes to the game's interaction and code.

In this iteration, the Guess Who element and the physical controls were added. In order to account for the guessing/elimination features I had to add in new object (character.as) for the extra mystery frog date characters , which held attributes representing their characteristics:

• colour: red, blue, green
• wearing a hat: true/false
• wearing glasses: true/false

As well as adjusting the existing code to support the new feature, I also added a new function to show the clues that help describe the mystery frog (chosen randomly each time the game restarts)

One annoying problem was with one of the red trucks appear at the top of the screen, above the clue box, even though the stacking order seemed to be correct. I tried deleting the last truck added to the truck array, but this just forced the next last truck to do the same thing. I ended up working around this and cheating by adding an extra truck to the stage, but positioning it below the visible stage area, so it stayed hidden.

Working on the new code

The final interface, with the new clue box sidebar added to display the clues one at a time

The game board is made from a piece of 15mm thick MDF, and the 10 tiles are also cut from this material. Each tile has an illustration of a different frog character (each one has a unique description, and physical attributes are evenly spread out). The top of each tile has a metal screw at the top with a wire soldered through to the back connected to a Makey Makey input (the letters WASDFG and the 4 directional arrow keys). The tile is attached to the bottom board with a hinge, so when it is dropped down, it hits another metal screw which is wired to the earth.

The game board

The back of the board

Close up of the game board and Makey Makey

After initial testing the game, I found that the tiles sometimes bounced when dropped down. This caused the controls to malfunction, either not registering at all, or conversely too many times, meaning the movement of the frog character on screen didn't move at all or move too many times. To combat this, I added some aluminium foil to cover the bottom contact. This seemed to work at first, but didn't fix the issue completely. I then tried taping some pieces of steel wool on instead, this worked a lot better and also made it less noisier. I then decided to solder the steel wool in place so it stayed in place.

I also decided to add in a physical replay button to replace the spacebar key - it's just a wire attached to the space key, with a paper "button" taped on top which when pressed touches a screw connected to the ground.

Overall I'm pretty happy with how it all turned out. I was pretty lucky to have my partner help me construct the board in a way that kept it all nice and neatly organised and taught me how to solder the wires in place. If it were up to me there would've been a mass confusion of wires running all over the place. The steel wool and screw contacts are probably not the most ideal solution for ensuring the best contact between the tile and the board, as it does tend to move a bit. But for this stage it works the majority of the time and is easy to adjust and manoeuvre. The size and weight of the board also are a bit too much, but it was a cheap and easy material to use appropriate for a prototype at this early stage.

Testing the game

Adding the steel wool to the bottom metal contacts

Week 9: Experience Prototyping: The Restaurant experience

Restaurant Dining Experience 

In class we discussed experience prototypes. We were asked to consider how this form of prototyping could be incorporated into the process of designing new technology for restaurants. 3 different stakeholders were considered: the customer, the waiter, and the chef.

• What is the existing experience? 
• What external/internal factors impact on the experience?
• What aspects of the existing experience could be enhanced/augmented/supported with technology?
• How would introducing technology into this context change the experience?
• What experience scenarios might you test with the technology?

Customer:
What is the existing experience?
The service they receive from entering until leaving the restaurant: Waiting for a table; being seated; reading the menu; ordering; waiting to receive their order; eating; paying the bill. Pre and post experiences could also be considered: booking a table; checking out a restaurant's website, review or social media; reviewing the restaurant after visiting.

What external/internal factors impact on the experience?
Recommendations and reviews, quality of service, quality of food/drink, price, speed of service, interior decoration, atmosphere, number of diners, queues, noise, customer's expectations, who they are dining with, other patrons.

What aspects of the existing experience could be enhanced/augmented/supported with technology?
The ordering experience, the menu itself is an obvious one. Another could be when waiting in a queue for a table at a busy restaurant, a system to see when a table is free, or get feedback on how long your waiting time will be, you could even start ordering before being seated to speed up service time.

How would introducing technology in to this context change the experience?
It might allow for less frustration when waiting for a table to open - customers could leave and do something else if they know how long to wait and comeback later. They also would be able to eat earlier if they ordered food earlier, and wouldn't have to wait as long while hungry.

What experience scenarios might you test with the technology?
Get a small group of "diners"in a waiting area to simulate the experience. Give them allocated slots which are display on screen, give them the option to wait, order and/or come back later at the estimated time. 

IP1 - Testing and Evaluation

In this week's A Prac Class it was time to test my Interactive Prototype 1.

I firstly explained the goal of the game, to move the frog across the road while avoiding the moving trucks (red blocks). Most testers were already familiar with the Frogger game, so this was easily understood by the majority. After this, I explained how to control the frog character (using number keys and arrow keys).

As the game was quite quick to get through, I instructed testers to play as many times as they liked, until they won, or got sick of playing. The first few users all only got to move once, and died upon hitting the first truck. This was mostly because they didn't understand the number controls (each number could only be pressed one). After the first try they managed to progress further in the game.

The first 3 testers tried around 3-4 times, they all progressively improved however none made it to the end, so I adjusted the speed of the trucks to make the game easier. I also did a quick demo first, so show how the controls worked, rather than just give the instructions verbally. After doing this , all the testers managed to win at least once. It still seemed quite challenging (didn't win all the rounds) but could get through after 2 or 3 tries.

I measured by observation. I counted and noted the exact point where the tester's character died, so I had a quantitative measure of their progress over the span of their attempts. I also noted how they used the controls, and where their gaze was (either on screen or on the keyboard).

Many testers had some trouble using the number keys, especially in the beginning. However I am not so concern with this problem, as the next prototype's controls will be completely different. Plus the element of elimination of keys (or in he final prototype, character tiles) is an integral part of the gameplay, and is supposed to offer a challenging element - plus the game is quick, so it's fast and easy to restart the game if needed.

The testers didn't really notice the numbers on screen (showing which numbers were eliminated). Testers had their gaze on the frog character only, and occasionally looked at the keyboard. So I discerned from this, that visually showing what numbers/character tiles on screen probably wouldn't be necessary for the next round. The results from the survey as back up this - where 3 out of 8 finding the game a bit too difficult,  where most marked this as suitable.

As for the testing session itself, I probably didn't give enough explanation to the initial testers who had more difficulty grasping the controls, so giving a demo may have contributed to the improvement of the later testers.

For the next testing session, I'll still pay attention to the explanation of the controls and the difficulty levels, as these will be different again. The added concept the the guessing may also need to be paid attention to, as it adds another level of complexity to the Frogger elements and hasn't been able to be tested in this session.


Detail of results:

I had a total of 8 testers play my game and complete the survey.

Observation and process:

User 1: 

• Died straightway on 1st truck
• Died on 4th truck
• Died on 7th (last truck)
• Died on 7th (last truck)
• User didn't understand that keys could only be pressed once at first. Kept pressing random keys in no particular order, so found it difficult to keep track of which keys were already pressed. Gave up after 4 times trying and not winning

User 2:

• Died on 1st truck
• Died on 3rd truck
• Died on 6th
• Died on 7th (last truck)
• As the previous user, took a while to grasp the controls. Gave up after 4 times

At this point I made the game difficulty easier by reducing the speed of some of the trucks

User 3:

• Died on 1st truck
• Died on 6th
• Died on 7th (last)
• Died on 7th

At this point I made the game a bit more easier again. And I changed my method of explanation at the beginning, by demonstrating the game quickly first.

User 4:

• Died on 7th (last)
• Died on 5th
• Win!
• This user was the first one to make it across the road. The new difficulty level seemed to be improving the chance of winning

User 5:

• Died on 7th (last)
• Win!

User 6:

• Win!
• Died on 6th
• Win!

User 7

• Died on 7th (last)
• Died on 5th
• Win!
• Died on 4th
• Died on 4th
• Win!
• Win!
• Win!
• This user seemed to enjoy playing and kept playing the game after they already won.

User 8

• Died on 7th (last)
• Win

Survey:

View a summary of the responses here: 
https://docs.google.com/forms/d/1ydUyNWBemM7w0qzX1dCX0t2DamKjKIZwSK29fL5FMfM/viewanalytics#start=publishanalytics