WP3 Deployment and test


Experience prototype workshops

The experience prototype workshops were based on seven3-hour workshops each conducted in Sweden, Denmark and Finland. One additional workshop in Sweden was conducted together with personnel from the municipality. The aim of the workshopswas to find pros and cons with three concepts, Spirited network, Mirror mail and Do you remember? The participants also created wish lists with wants and needs in relation to the concepts and identified new ideas and suggestions for development of the concepts. These wish lists and new ideas were used to further develop the concepts, but also give indications about which ones to focus on.

There were some cultural differences identified, primarily between Swedes and Finns. The Finns really liked Mirror mail, but the Swedes were reluctant to adopt the concept due to integrity issues. The Swedes on the other hand really liked Spirited network. The Danish users were more in between the different concepts. All in all Do you remember?was chosen for several different reasons, for example that it was considered interesting in all the workshops.

Evaluation of the demo

The demo evaluation was based on six workshops that lasted 3 hours each, conducted in Denmark, Finland and Sweden. The evaluation was done in a group of seniors that were not familiar with each other playing the game together, and in a family setting where two or three generations were playing together.
 


The objective of the workshop was to identify possible impact of the games, strengths and weaknesses of the demos, and to have the participants suggest modifications. Additionally, we also identified the strengths and weaknesses of the gaming platform, primarily from the perspective of playability, but also from technical viewpoints. For example better support for the flow of the games was neededto keep up interest and motivation. This were connected to both technical features as well as game features.

In addition to the workshops, an online questionnaire with 220 respondents from Sweden, Denmark and Finland was used to gather insights about the use of technology and Internet as well as to ask about gaming and storytelling.This provided deepened knowledge about the preferences and current experiences of technology and of gaming and storytelling. For example what triggers storytelling, what people enjoyed talking about and listen to were presented in the results from the study.

Senior focus group and gameplay workshop

This workshop with five seniors (average age 70) was conducted at Halmstad University with the aim of evaluating game mechanics and content for Photobluff. After a short introduction, the seniors played the game with two researchers observing them. The main themes that guided the observations were 1) gameplay, 2) coolness/entertainment, 3) humor/emotional immersion, and 4) game feedback. These themes had 15 sub categories, into which the data was categorized.

The results were about everything from traditional usability issues to issues related to game play and game mechanics.For example, how to support the interaction in the games and provide sufficient feedback concerning the gameplay (e.g. award of points) in the interface were presented in the report.

Field trial – Demonstrator test

During May and June 2012, a two-week field trial was conducted in Sweden and Finland. All three games, Picture Pong, Photobluff and PicMyChoice, were tested. Two trials were running in two different contexts; family settings where seniors and their families played together, and service centers’ settings where seniors and personnel where testing the games. 16 seniors evenly distributed between genders and nationalities were involved in the family setting. Twelve seniors and threeemployees were involved in the service centre setting.

The main aim of the field trials was to provide the development team with feedback for game design as well as to learn about: playability, usability, sociability, willingness to pay, and support for social interaction. The trial was started with a background questionnaire that was handed out to the participants. During the testing, the seniors involved filled in two weekly questionnaires. The trials were ended with a concluding interview as well as a final questionnaire. Additionally, general feedback was gathered via an app on the iPads as well as through a Facebook.

In Sweden, the seniors were positive, interested and curious about the games. Since this kind of technology, from their point of view, usually belongs to adolescents, they found it very interesting from the start. In Finland, the seniors were mostly skeptical. They thought the games were annoying, boring or hard to understand. Most Finnish players showed no interest in giving suggestions for improvement. A majority of the Finnish group did not think that the games would have any impact on their wellbeing and that interest would disappear or they wouldn’t play again.

Finnish players answered that if they would play in any situations it would be with grandchildren or at dinner party with friends. Most of them would not play at all. A few Finnish seniors mentioned that the aspect that was most fun with the games was to try the new iPad.

Cognitive walkthrough

A cognitive walkthrough was conducted with usability experts from Halmstad University. The cognitive walkthrough started with a task analysis that specified the sequence of steps that was required by a user to accomplish a task. In this analysis, the system responses to those steps were also listed. The experts then “walked through” the steps as a group, asking themselves a set of four questions at each step. By answering the questions for each subtask, data was gathered during the walkthrough and usability problems were documented.

All in all 28 comments were made, ranging from non-critical to critical usability issues such as lack of feedback or problems adding players to the game. Unfortunately, an old version of the prototype was assessed, so some of the comments were not relevant anymore.

Photo content workshop

In a photo content workshop with five seniors with an average age of 70 (three men and two women), the game Reminds mewas evaluated as a paper prototype. The aim with the workshop was to evaluate one game based on content and to identify strengthsand weaknesses in gameplay, pictures and game questions as well as to get modification suggestions from the focus group of seniors.

The results in the shape of recommendations about photo content and game questions from Halmstad University were discussed with the development team via mail and Skype, and used in the next version of the games.

Usability evaluation

Before the final field trial started, Halmstad University conducted a small usability evaluation in a laboratory setting to assess the games, trying to identify any remaining usability issues. As usability flaws had proven to highly influence the seniors’ experience of the games, this was deemed essential before the launch of the field trials.

Eight users, primarily male players, between the ages of 35-55 tested the game in four game sessions. A task-oriented evaluation was conducted with a focus on the following tasks: add and modify players, handle back and forth navigation, test menus and help, and play through the games. All in all 17 issues, primarily non-critical ones, were identified.These issues were handled by the design team before the release of the final version of the prototype to be used in the tests.

Final field trials

The final field trials were conducted in Sweden and Finland during the autumn of 2012 and were running for a period of three weeks in a real life setting. The trials were set in two different contexts, family setting where seniors and their families played together and an activity centre setting where seniors played with seniors.

The main objectives for the test and evaluation activities in the final field trials were to provide input for the impact evaluation and for the commercialization work package. Furthermore the trials aimed at providing input for the development team pertinent to gameplay, content, and context (setting, pictures, and game questions). A total of 117 seniors and family members to seniors were involved in the field trials.

The overall conclusions for the tests regarding commercialization were:

The overall conclusions for the tests regarding impact were:

The overall conclusions for the tests regarding content were: