This week, we needed to pitch our ideas to the studio. Therefore, we really needed to solidify our concept and idea. Prior to this, it was obvious in our group discussions that each individual had their own interpretation of the theme. Therefore, I suggested for the team to all go and further design the theme concept space and report back to the team.

As a result, below was my own interpretation of the theme space:

Enhance Mundane Spaces

Definitions

Enhance: intensify, increase, or further improve the quality, value, or extent of.

Mundane: lacking interest or excitement; dull.

Spaces: position (two or more items) at a distance from one another, be or become euphoric or unaware of one's surroundings, especially from taking drugs

To Enhance

• You empower

To be Mundane

• Mundane tasks

o Use your curiosity to enjoy activities such as washing dishes and organizing.

o vacuuming, doing dishes, or dusting the house

o activity you dislike

• It is not a choice, rather it is a must

• Something you dislike

• Something you find discomfort

To NOT be Mundane

• Spark curiosity (https://www.psychologytoday.com/au/blog/what-matters-most/201705/how-deal-boring-or-mundane-tasks)

• Challenging/complex task

• Strength of creativity

• Prudence strength to organise

• Stay focused, using self-regulation strength

• Appreciation of beauty

Inspirations:

* https://www.researchgate.net/publication/263594631MundaneMobilitiesPerformancesandSpacesof_Tourism

• https://www.psychologytoday.com/au/blog/what-matters-most/201705/how-deal-boring-or-mundane-tasks

Reflection on this exercise

This was a great concept as it allowed all group members to share their interpretation to ensure that we captured a holistic interpretation of the theme space. By employing this method, the team was able to freely devise and formulate their own ideas which were shared and further refined as a group without the fear of subconscious judgement

Due to the Coronavirus, all of our courses moved to online. Hence, from this week, we need to attend class through Zoom.

In this week's class, we presented as a group, and chatted with classmates, tutors, and teachers after the presentation, and collected a lot of feedback.

About our team's concept

Our team's concept is Hasaki which is a hearing-centric swordman somatic game. Our theme is Ability-centric Interactions, so under this theme we thought of designing a game for the blind to enhance their hearing. From the perspective of human value, we hope that this design can meet the needs of improving listening, perceiving the world, and entertainment.

Individual work

Made a simple sketch for this game for illustration:

The use of the entire game is as follows:

1. The user stands in the circle.
2. An aggressive sound is played.
3. The user judges the direction and then fights back with the weapon:

    If right — 4.Moving on to the next level.

    If wrong — Minus one health If right — 4.Moving on to the next level.

                                                        If wrong — Minus one health If right — 4.Moving on to the next level.

                                                                                                         If wrong — Lost all three health — game over!

Note: The user has three chances totally in each level.

Interactions

Game storie

• Design various stories to fit different levels

Game mode / difficulty

• Add ambient sound and touch (rain / wind…)-Simulate game scene
• Provide-provide interference
• Accelerate attack frequency

Action detection response

• Feedback is provided to the user in response to the action (via prompt sound effects)

The main feedback that we have collected

• considering looking into examples of vertical slices-I believe these will reduce the scope of what you'll need to do, so you'd only need to work on a specific section, allowing it to show off just the highlights kinda
• To enhance the intensity of interactions, and you can also consider the multi-player mode.
• Consider could be collaborative? Could be a wearable technology? For example, friends can wear the device to fight with each other.
• If there is only four speakers, this is too easy even for the blind people. Put more.
• On a technical note you can get full 3d audio just on a pair of headphones. The examples you showed had the people using their hearing to move getting more information from the surrounding environment than just enemy direction. Having people listen for quieter sounds and giving the player the ability to move around the room would add interesting complexity. Other control methods could also be added by using a cane as well for the user to be able to feel their environment.
• Safety is something that needs to be taken into consideration since waving around a sword of some kind leaves risks in it self. Another thing is that the target group (blind people) is a very sensitive target group and you guys should therefore be careful when making something for this group.
• Check the game “seeing” produced by Tencent to get more inspiration. More interaction could be considering what would happen when they fail to cut the laser, what would happen when they cut enough amount of the laser, how they can stop the game and what would be the gesture to start the game.
• It would be real interesting if the audio was moving instead of static sound at each speaker.
• First, attack methods can be more diverse, such as using multiple weapons. In addition, monsters can be varied to increase the difficulty of the game. The type of monster can be perceived by different sounds. Bseides, the sound of the hit can also be changed. For example, if the user cuts the monster, one kind of sound is produced; when player miss, play another kind of sound; when player hits the key, the third sounds can be played.
• Could be interesting to use sound to navigate around in a dark room / maze.
• Great idea, but instead of just 4 direction of speakers will you conside make it 360 around the player.
• At the moment it seems quite simple and could get boring quickly so I think you need to add another element / story / object to engage users.
• To add some new skills for the player? Or design some unique attack way for the enemy and make it a VR SEKIRO, or maybe you can combine it with fitness game cause repeat is acceptable in fitness.

Future Development

• Add a plot to this game.
• Research more technology backups.
• Try voice and gesture recognition system.
• Rethink the game mode-adding mutiple-player mode.
• Consider the safety issue for the player.
• Rich interactive experience。
• Detail the game contents.
• Consider wearables - headphone - 360-degree stereo surround

Reflect of this week

This week is the first week of online courses. There is a difference between online courses and on-campus courses, and it is difficult to adjust for a while. On the one hand, the time spent on the road is saved. But on the other hand, the class in zoom lacks interaction, especially among classmates. Of course, this may also be caused by the pitch content of this week. But this format did make me felt more tired.

Back to the pitch of our group, our group had planned to work together on a single project. We thought it was an interesting concept without thinking deeply. It did overlap with some existing products including games using Kinect and Leap Motion. We haven’t explored more possibilities for using the body to control some other things. Feedback from Alison and Clay mentions that we may be able to use some unusual control methods or get rid of gesture control, or even bring sign language and make controlling meaningful. Lorna ’s feedback mentions we might move away from the screen and couch, and we think it is very instructive. But I have to admit that we still feel very bottlenecked. We have two other group meetings this week after class. But in fact, the progress is not very smooth. Everyone suffered some blows after pitch and wanted to turn to personal directions. In a way, this will make our job easier for the rest of the semester. But in fact, most of us have not found a clear direction to go on, which kind of worries us.

What could have been done better

In fact, at this stage, each of us is still confused about what we are doing next. Ideation is the most difficult part of every project when I started my every semester. When we just formed the team, we all seemed a little too taken for granted, thinking that the idea of the gesture plane game we came up with was interesting. Perhaps we are too limited by such an idea that I want to make a game, so I have not explored existing products and papers. This led us to frustration and lost direction during the pitch phase. So, maybe, every time before presenting the idea to others, we should first get opinions from some potential users or experts, and not just addicted to our ideas may make us do better.

Work to do

Next week we want to confirm the feasibility of the direction we are currently proposing. Because after the course structure changed, we are not sure if this meets the requirements. Of course, we will start our proposal writing. We see the umbrella mode mentioned in the announcement email might be most suitable for our group, but we don't know how to integrate this model into the proposal because it asks us to talk about details of the project, such as integrated experience and how to engage users.

Inspired

One idea that I particularly like in pitch is Hasaki, an idea that uses sound to guide the player’s movement. This makes us rethink the output of our project and does not necessarily need to use the screen as a result. For example, the sound is a direction worth exploring, of course, it may be more than that. We are now considering providing users with a healthier and more dynamic control way that keep them from using screens too much.

Fire safety using audiometrics

Cannot touch and use the real concrte tools via phone, since there could be a gap when it comes to reality from the virtual environment. It could be more interesting if it combines with the VR or AR, which can reconstrct the realy world.

Housework

Should avoid the potential dangers. Competion can be a motivation for housework. Projector seems can be placed by VR for more interesting contents.

Emotional birds

I was thinking people's emotion at one time can be complex, for example i can be angry, regretful and overwhelmed at one time. So a mixed input and output can be considered.

Zoomista

People could have different feeling at one color, and it could annoy people when making sound in the public.

Brotherhood

A sudden sound in the public could annoy the others, and the aggressive words might offend people.

Garfunkel

The interaction seems single and simple. And it would be annoying when everything can make a sound.

Twisted

A penalty can be added when the users have a wrong try. And how about use different auditory for feedback.

Lome

Are the 9 years old chiled the best target users? Should research about educational background to find out what kind of contens suit for which group best. Also it could be more resaonable if there is a feedback for the wrong try.

Project Process

Storyboard

Technologies

• Recording equipment
• Playback device
• Infrared sensor
• Motion sensor
• LED lights

Recorders and speakers are used to collect and play music.

The infrared sensor is used to measure the distance between the dropper and the test tube. When the dropper is lower than the test tube, and the user presses the rubber tip, the sound can be transmitted from the dropper to the test tube.

The motion sensor is used to identify whether the user shakes the flask or the test tube to determine whether the sound should be mixed or played. The number of shakings also illustrates the different needs of users.

In addition, we also need LED lights that can change different colors. It may be installed in all equipment except the trash can to indicate whether there are sound clips in the container.

Feedback from contact

Through classmate feedback, several questions and suggestions were repeatedly mentioned, such as mixing problem, interaction, and visual feedback.

1. How music/sound is added. Importantly, many people have doubts about the synthesis of music. They want to know how to synthesize music so that the synthesized music pieces do not become noise. We didn't think about it at first. After the reminders and suggestions from classmates and the group discussion on Friday, we decided to set some background music or music frames or beats in advance. In this way, based on the original, and then create, then the music obtained can be relatively artistic.
2. More ways to interact. Classmates and tutors gave more innovative ways to interact. For example, heating/cooling can speed up / slow down the speed of music playback and some ways to change the sound effect (the type of liquid/chemical properties can represent the combination of music). Of course, after this group discussion, we will decide whether to add these effects according to the specific project process. Because in fact, our concepts are relatively complete and difficult to complete, we are not sure that some of the functions we set up can be completed within the prescribed time. If we are in the process of production, the completion of the facility is high, and we have not encountered a particularly difficult problem, we may add more interesting interactive methods.
3. More visual feedback. Ben gave us a suggestion to add smoke effects in the process of mixing music. This suggestion made our members surprise. And according to Ben, this effect is not difficult to complete, just buy a device that can release smoke. So we will add this suggestion to our concept. In addition, another student suggested that the volume of music can be indicated by the brightness of the light. After discussion, we should be able to achieve this effect, and in our imagination, this effect is not difficult to achieve, and it will not add special difficulty to our project.
4. There is another problem that made our team members realize that it is a serious problem. Sound Lab is a relatively complex and process-intensive device, which means that users may feel confused and not know what to do next when using it. So some students suggest adding some tips or tutorials to guide users on how to operate. From the perspective of user experience, this is necessary, but I don't think it is a good way to write the usage process on paper, because based on the experience of previous trade shows (once made instructions), few people read the instruction before participating in the game. So we are still discussing whether there is a way to understand the operation process without text reading, or how to further simplify the process. Although we have no idea yet, this is an important issue that we need to resolve before submitting a proposal report.

Work to do

Instead of turning teamwork into individual work, we applied to Lorna to work together on a project after class. The next step is to focus on writing the report. The report will need some information about problem space and identification, so I will have a deeper understanding of some problems in this field. In addition, there are some points that are not considered in our concept, such as the way of combining and collecting music (the order of two pieces of music or sound / the environment and natural sound will be added when collecting the sound, how to reduce The effect of noise on music). In view of these points, we need to have a more detailed discussion to determine that the music produced is not unpleasant. Of course, we will also discuss the novel interaction methods mentioned in the feedback and evaluate their difficulty (may be screened out a few easier ones, and if time permits, design them in detail to use as a backup options).

Because of the one week break, all the study activities have been pulled off. This week, people became anxious as we don’t know what would this course be like and our team could change a lot after the break. But in the meanwhile, this break provides time for us to consider the theme.

Project discussion

First of all, via communication through Facebook, we decided to come up with ideas individually first and then together, discuss the ideas and set down to one of them.

Based on what we have discussed in the class, I made something new

House pong game

This idea is developed on the interactive floor. Doing housework is boring and we want this activity could become a connection between the family members. So I think, why not make it a cooperation game. Because it is a game, it can be playful and attractive, encouraging all the family members to join in the housework. The virtual ball would come from one person to another, they can communicate through the game.

Pigeon

This idea is developed on the monster mailbox. I thought sending and receiving messages in the office could be boring. Combine with my working experience, the fact is that it is hard for me to keep sitting as I need to get or send some things to other people. So I came up with this idea. Pigeon can send messages and items, also can get them back when needed.

Timetable cube

Still in the office, as every week, every day, there are many meetings and other schedules, I came up with the idea to make a timetable cube wall. The cubes are placed on the wall. Each color represents for on place or room when people select a color and press the cube, the cube would recognize the fingerprint and put this person’s photo on the cube with the background color as selected. So the wall would be like a place where people can see all the people's timetables. The wall would be connected to computers and photos, so people can edit it through these devices.

Discusstion

After the discussion, our team members have some suggestions on the ideas.

For the pigeons, it might decrease the working effectiveness an it is hard to control the amount and how it can be used in the office so people might not accept it.

The timetable is more visualized but not interesting enough. Because of the time and place are fixed, it is hard for people to play with the wall. So it becomes an enlarged timetable and not playful enough. It did not change the fact that office is mundane.

We hesitated between steamer and pong game after the discussion and asked advice from Lorna. Finally, settle down to the house pong game.

Prepare for the presentation

Not like before that each person would responsible for one part of the content, this time we decided to separate the work to video, audio, sketch and edit. I am response to the audio part. To make sure that we are working on the same content, on the one hand, we need to communicate frequently on Facebook, on the other hand, I decided to use a doc to communicate the working detail. So our team members can search for what they need to do.

https://docs.google.com/document/d/1dEUoKnp5DWaSzl5QfDLAPMVsqkumpLDlw8IKw1ltIik/edit?usp=sharing

About proposal

This week, we finished the first proposal of the physical device, which including the video and pitch. The name of our system is Hasaki, which is a hearing-centric swordman somatic game.

Individual task-Scenario

I created the scenario of the proposal. The storyline is as follows:

• Jim lost his eyesight because of an accident. The sudden change made him very uncomfortable. The color world now has only empty darkness, and he feels as if he has lost all contact with this world.
• The world has changed since he knew Hasaki from a friend.
• When he first played the game, he was very unskilled. He is insensitive to sound. He often lost the game because he couldn't feel the sound.
• With practice, he gradually became more comfortable, and he could adapt to more challenging games.
• The monotonous life before was fun because of a game designed for him. He was braver to re-perceive the world with his voice.

Individual task -Target users

I created two target users with this work. One personal is for common people, and another is for blindmen.

Comments of presentation

Bata skwad

The problem of the trust of the phone is always dull people. The theme forced on Personality is a great way to reconsider the issue. How its work process is clear to show in the presentation. Trying to rethink about the machine exterior which suitable to ergonomics.

Fitlody

The overall work is pretty good. The combination of fitness and music to design somatosensory games is an exciting starting point. The sketch can let users know this device more clearly. Persona is designed to be very concise so that the audience can clearly understand the target user. Perhaps the scenario could be further improved, but the current is not very detailed. Body storming is very interesting, and from the video, we will know how this device works.

Negative Nancies

This idea is fascinating, the introduction and storyboard of this device are very detailed. However, The sketch maybe needs to improve. Tring to make the sketch more detailed. Target users possibly need to add motivation and user detail.

EMS - Housework

The design of the personas is excellent. I could easily find the characteristics of the user. Consider the detailed storyboard.

Zoomista

The project has a useful review of the literature. In the presentation, each part has a bright showing. Using emotional color is an enchanting design idea.

Botherhood

Brotherhood could solve the problem when people are waiting for a long time—considering make the video voice louder because sometimes the dubbing of the video is not particularly precise. In addition, Showing technique solutions are significant parts because it will be showing how it works.

Lome

Using flower is interesting in the design idea. The scenarios are clear. On the other hand, Trying to find a way to encourage users to keep a good mood.

Idea selection

After we formed the team, the first step was to determine the basic final direction would carry on. First, each of us shared our personal pitch and then looked through other peers' ideas in this domain. For each idea, we talked about three aspects: the relavance to our domain; the feasibility to realise; is it interesting or meaningful. Some ideas seem have been achieved by existing technology,some are irrelevant to this domain. During the discusssion, we misunderstanded this domain at first, but after talking to Lorna, we clearly knew what"Ability-centric interactions" stands for. We're supposed to create some specifc and exclusive ability related things for its specifc target, for example, we can think about what ability or demand the deaf people need.That's just similar to the idea in my pitch. At last we agreed to carry on my swordman hearing game.

Team meeting

We had the first meeting on the Zoom. I led this meeting as the creator for the original idea. Based on what we got now, i listed three main questions for idea extension:

1. What other forms can it be?
2. What interactions can be added?
3. What technology can we use for each functions?

Through these question,we then came out lots of great ideas:

1. Infrared beam to detect the movement;
2. Photorersistance to detect the movement;
3. Add plot to this game;
4. Use the disturbing sound to increase game difficult level, like the wind and rain. We can even simulate the rain and wind by water spraying and fan.

Below is the note of team meeting.

Online pitch and critique on others’ concepts:

During this weeks’ contacts sessions, we conducted the online pitch activities via Zoom and gave and received feedback via Slack. Although we have switched to the online alternatives, our online presentations have gone well and smoothly. By watching other groups' presentations, we were provided the chance to get a better understanding their initial ideas and concepts. For two days contacts, some novel and engaging ideas were presented! We also gave our critiques to other team’s concepts by trying to give our constructive and specific suggestions.

Critiques on our concept:

After we presented our idea through the pre-recorded video, the teaching team and our peers have given us a lot of valuable and actionable feedback, which was very useful for us to refine our concept. After received feedback from Slack thread, we also organised our team’s zoom meeting to invite Lorna, tutors and our peers to discuss around our concept, so as to provide our team with deeper clarification on what we have and haven’t done well for the initial concept and what can the team concept be further refined. Through this session and these online platforms, and thanks for all the teaching staff and our peers, we gathered a lot of feedback, and detailed summary will be listed below.

Feedback summary:

Some positive ones:

1. Using human’s body as controller makes interactions engaging and intuitive.
2. Making the interaction process more acceptable and operational.
3. Providing users with freedom of control and movement.
4. Enhancing people’s interests in playing games, especially for those who hate using some physical tokens as controllers.

Some actionable and constructive ones:

1. Avoiding making users feel fatigued.
2. Going beyond the screen-based output.
3. Translating into a non-game space will be more worth exploring.
4. Exploring other parts of body as the controller.
5. Considering combining that will educational content.
6. Combining different gestures to everyday objects and day-to-day life.
7. Reducing users’ cognitive and memory loads when learn how to interact.
8. Giving users freedom to select specific body parts to control specific interactions.
9. Considering physical differences among different user groups.
10. Exploring to apply same tech and interactions into different contexts.

All the feedback from teaching staffs and our peers have provided our team a clearer way to explore our domain and the problem space. Following the given feedbacks, our team decided to do some desk research around existing solutions and then conducted user research to get to know more about our target users.

Some existing/similar technology in this field we can check out:

1. Leap motion
2. Nintendo Wii
3. Switch Gyro
4. Xbox Kinect
5. PlayStation Move
6. PlayStation Eyetoy
7. Microsoft Adaptive controller (also gather some ideas around ‘disability support’)

Team progress

In this weekend, our team had 2 meetings to discuss around the feedback we have received and how can we refine our concept as the response to these valuable feedbacks. Based on the feedback we received, we had further discussion about specifying our team’s problem space. After that, we decided to move our focus from only ‘hand gestures’ to different parts of our bodies; to go beyond only focusing on game space; to put our concept into a more day-to-day environment; to explore how to apply this technology into various contexts, to reduce the reliance of system on the screen, etc.

As for our team’s future plan, we suppose to first conduct more research around the related work in our chosen domain, and then go deeper around our team’s domain and individual focus by explore the different solutions under the umbrella of team’s domain and problem space. We also break the team section of the proposal report into different small parts and assigned them to each team member.

Individual progress

Since our team have received a lot of valuable feedback, I spent a lot of time on going through each of them in detail, and discussing them with my team, so as to decide how to elevate our team’s concept to the next level. I organised these valuable feedbacks and summarised them to some implications that we can draw on and take into consideration further.

Apart from that, I also done some research around the existing and similar solutions in our theme. By doing that, I can get a better understanding of our chosen domain, and some ideas about how to make our concept more unique.

References

img 1. retrieved from https://www.ultraleap.com/tracking/

img 2. retrieved from https://techotv.com/nintendo-wii-u-games-price-release-date-specs-reviews/

img 3. retrieved from https://www.htxt.co.za/2019/10/30/risk-of-rain-2-adds-gyro-aiming-on-switch-plus-other-console-updates/

img 4. retrieved from https://www.google.com/imgres?imgurl=https%3A%2F%2Fwww.lifewire.com%2Fthmb%2FCvta8BuIe9jpARgjXxjbmJDcUtk%3D%2F1024x685%2

Improvement

1. Compressing the image file size for better display in journal post.
2. Adding the alt text description of each image

Last Saturday we had some discussions online and identified that the 'Hasaki' is the project we want to complete in the near future.

Theme of Team

Our theme is ability-centric interactions, which require us to design interactions from an ability-specific point-of-view for some specific targets. For our project, we are going to create hearing-somatosensory games, which focus on improving hearing ability, especially for blind people.

Original Idea

The initial intention was from a YouTuber, Bertolt Meyer, who's born without an arm. However, he made himself a mechanical synthetic arm to satisfy all his needs, including his love of music. I was touched and inspired by him and wanted to design something for the disabled to make their life more convenient or bring them more happiness. As hearing is one of the most essential sensory to feel the world for the blind people, and the level of hearing ability determines how much they fell the real world. Therefore, our human value hopes to provide them with exciting entertainment, which can help improve hearing ability at the same time.

What is this project/ installation looks like?

How it works

Scenario

Some Warning

Safety is something that needs to be taken into consideration since waving around a sword of some kind leaves risks in it self. Another thing is that the target group (blind people) is a very sensitive target group and you guys should therefore be careful when making something for this group. What mean by that was just be aware that this is a protected group when it comes to research so keep that in mind.

However, at the moment it seems quite simple and could get boring quickly so I think we need to add another element/story/object to engage users.

More function

maybe come up a way that present visual information using sound, rather than just sound comes from different directions

On a technical note we can get full 3d audio just on a pair of headphones. The examples we showed had the people using their hearing to move getting more information from the surrounding environment than just enemy direction. Having people listen for quieter sounds and giving the player the ability to move around the room would add interesting complexity. Other control methods could also be added by using a cane as well for the user to be able to feel their environment.

So in one word, We need to think a lot about how can we make this installation more playful and meaningful. Since our potential is the people we need protect them avoid the potential risks.

General Feedback:

positive ideas:

1. Incentivising housework is an interesting space
2. Interesting to see how children can be motivated to clean by using games
3. Wouldn't want people waving around a knife
4. trying to solve an actual problem in our daily life.
5. A clear description of the concept including persona, storyboard, prototype

Constructive ideas:

1. make the interaction between multiple users more continuous
2. using a projector in a well-lit room or several rooms may prove a challenge.
3. each type of chore have it’s own challenge/expertise
4. sound could be used to help with understanding ball movement around the house
5. punishment vs reward
6. track who catches the ball and capture the work done
7. change the ball projection into something else

Team progress

Work completed

1. Determined the overall plan
2. Determined what to improve
3. After the group was downsized, there was a careful division of labor for each person
4. Collected and organized everyone's comments on the presentation
5. Start making a paper prototype

Reflection of completed tasks:

1. Speed up communication efficiency, and respond to Facebook in time
2. Google drive shared, put all shared files in one place
3. Google drive upload pictures need to be named and sorted for teammates to use
4. Help other teammates while completing tasks
5. Communicate task progress in a timely manner
6. When getting stuck, ask teammates for inspiration instead of thinking alone

Next step

1. Determine the number of chores
2. Determining the characteristics of each type of household chore
3. Determine the sound effect during a collision
4. Determine the evaluation method
5. Complete paper prototype
6. First paper test with a paper prototype

Individual process:

Next step:

1. collect the material and start making the prototype
2. prepare Arduino and the collider
3. investigate the code that can make the ball slowly pop out of the water
4. research family games to find ways to increase interaction and improve every way of chores

Reflection of completed tasks:

1. Drawing speed is too slow, don't pay too much attention to beauty, the main purpose is to express meaning.
2. Pr is unskilled and cannot help teammates edit special effects to speed up efficiency.
3. The connection between Arduino and the projector has not been figured out, and we have to find a way to solve the related code problems.

As I mentioned before in my previous post, I was interested in exploring natural sounds to help students learn at school, more specifically in primary schools. Therefore, the first steps I did was to start investigating what the purpose of the idea will be in terms of what it could help teach. How I did this was to first ask someone who is a teacher's aide. I asked them about what students were struggling with regarding their education that could be helped using music. They responded with multiplication and how students are finding it difficult to learn and memorize. This surprised and interested me as I enjoyed learning the times table when I was a kid. However, I wasn’t inspired enough to start coming up with an idea. Sometimes when this happens, I like to look or walk the space to try and get some inspiration. Three dices were what got my attention and started to make me think about how I could use them to help students learn multiplication.

The Original Idea:

• To use three blocks or die as the main object. The focus was on teaching students to multiply by 2 numbers. One block or dice would represent one number and the third would represent the second number. The second block would represent multiplication; however, apart from that, there are no other users for this object.

Original Interactions

• The user would roll the first and third die to choose the objects they would multiply.
• Each number on the dice would then be assigned a natural sound, the user records the sound that was produced when they interacted with an object that they had on them. These would most likely be school supplies/stationery as they would use this product at school and home.
• The user would need to create the natural sound that represented the answer to answer the question

Issue with Idea

• The second dice needed more uses.

Changes to the idea incorporated the object helping students with simple mathematical problems such as adding, subtracting, dividing and multiplying, where the user rolls the second object to choose an operation. The other two die are still choosing the numbers involved in simple math problems. Two sides of the second dice will be blank, the user can decide which operation they will do. Each operation will be assigned a mini percussion tune by the user.

Issues with the idea:

• Times table include 1 to 12, however, the dice is 1 – 6
• Focus on 4 operations instead of one.

Current Iteration on the idea

• Use 1 x 12-sided dice.
• Allow the dice to record natural sounds from the user’s objects.
• Each number, from 1 – 12, is assigned a natural sound.
• Check if the natural sound being recorded is the same as the recording that is stored for the answer.
• Focus is on multiplying two numbers; however, users can multiply using multiple numbers.

Interactions:

• Throwing - this will allow the dice to know that the next sound represents the answers and needs to be checked.
• Pushing down on the specific point of the dice will allow the natural sound to be recorded for a certain number.
• Rolling the dice to get a random number.
• Interactions with the object to get it to produce sounds.

Issues:

• Answers are higher than 12. A way to solve this is to have each digit, from 0 – 9, have a particularly natural sound. Therefore, 12 would have the sounds for digit 1 and digit 2. A 16-sided dice will need to be used with 3 blank sides where the user can decide.

Other Ideas

• Instead of using dice, an object such as a rubber, pencil or flexible ruler could be used. However, this might have been difficult to get children to understand the correlation between multiplication and interacting with these objects.
• Have the dice made up of 12 smaller objects that fit together, however, the problem with this is that it will have a larger amount of equipment and mess, as well as more potential to lose parts.
• Another idea was to have shapes within shapes to represent the number and to have the user learn and explore the natural sounds using the objects.

Meeting

We have had two meetings; one on Tuesday where we discussed what parts of the concepts we could explore, such as the different contexts and individual objects. We also decided that we would continue to collaborate as a team by picking similar areas rather than going in very different directions. The team also read the comments and summaries and grouped information such as the positives, negatives and questions to have a clearer idea of similar critiques and feedback that stood out. From there we decided to have a meeting on Friday night where we needed to have read the ECP and come up with ideas about what individual paths we wanted to take, contexts and specific users we want to focus on.

After Friday’s meeting, we came up with different concepts such as:

• School
• Public Transport
• Bus stops and Train stations
• Home – different rooms focused on specifically
• Visually Impaired

We narrowed it down to school and visually impaired as we were interested in creating a product that would help teach a part of a subject other than music that could be accessible for those who are visually impaired. We decided that we would have another meeting on Sunday to discuss the research we have done, such as:

• Research more about the subjects that students struggle with.
• Can music help students learn other subjects?
• Existing products.

We have the first online presentation on Tuesday and Wednesday. I got some inspirations from them and commented on them. We received some feedback from classmates on the presentation.

Reflection of the comments on our team

About the feedback, I made some conclusions. Firstly, I think we need to add each type of chore expertise. What we want now is that when washing clothes, the ball is sprayed from the basin by pressure from squeezing water. A prompt tone should be added for each collision to give sufficient time for the next person to receive the ball.

In addition, we need to add a reward system to stimulate people's interest in long-term play. I think it is possible to set a rank medal for each person.

For example： Bronze-Silver-Gold/Novice-Apprentice-Journeyman-Master

For safety reasons， I think we need to drop to use the kitchen knife to catch the ball. The three suitable general categories are: washing clothes, cleaning the table, mopping the floor.

Besides, regarding the problem of the ball passing through the wall, I think it can be reminded by sound, but the wall will definitely block the light of the projector, so we decided that it is more practical to do this housework in the spacious living room so that everyone can have face-to-face interaction.

The general comments on other ideas

Since I have commented on all the projects on slack, I will just comment on the whole here. I think the most problematic part of your thoughts is not enough interaction. This is also reflected in our thinking, that is, the interaction of each different housework does not have its own characteristics.

The idea wich interests me most is GlobalZoo. I did a similar project with Arduino last semester. Generally speaking, it is to cast small cubes to change the state of the earth. So, I hope to get some new inspirations, such as how people's carbon footprint can be more intuitively expressed through interactive devices. I think the use of water in the project is a good idea, and it can also be connected with the changes of the earth. After all, it says that "water is the source of life", but I think that we can continue to go deeper and make more interesting interactions.

In addition, I think the way the earth changes states can also be enriched, not just change color.

---

Individual progress

Work completed

I was responsible for the persona and the drafts(storyboard+scene) in our team. About the persona, I choose two typical people who have totally different personalities and lifestyles. One of them is a full-time housewife and another one is an office worker. The main pain point of the full-time housewife is having fun with her family members and the other's is accompanying with her friends. Based on the persona, I think we need to make these links more cooperative next.

About the drafts, I drew a general effect and more than a dozen single-action diagrams. I set each action as a separate artboard so that the teammate who edits the video only needs to play these pictures in order when editing. Also, the teammate who shot the video and the one who responsible for the special effects also played very important roles. Although some pictures were not used because of editing, the effect presented in the final video was roughly in line with my requirements.

The past two weeks have been absolutely crazy. They have definitely tested our limits, very early in the semester, and introduced us into the online life. UQ has moved all courses online and this changed how Physical Computing will run. Due to the current social distancing rules and government advice, we are unable to meet up in person. We have made multiple calls on Facebook to discuss our thoughts and our plans for the group work. After making sure we were on the same thought process, we were able to delegate tasks over Facebook and then rejoin after we had finished our work to finalise our presentation.

These two weeks I worked on developing our idea with my team and then designing a sketch for our product, "Emily". I focused on writing up our intended experience and how it fits into the domain. The sketch was a very basic design of what we were envisioning if we created Emily as a ball. After we received feedback and were able to discuss it amongst each other and the teaching team, we were able to look into different forms and how we could all even incorporate separate ones as our individual Emily. We received some mixed feedback from the studio (as many concepts will) but the general consensus was well. We have discussed the bigger idea of Emily and are happy with what we have created so far. After listening to my peers' presentations, I could understand the input and outputs that others have created and gain more inspiration.

In regards to our concept, our team still needs to nitpick at our idea and refine what we aim to create. We need to decide how we want to delegate work for our final form. I need to conduct research into negative reinforcement and how people respond to objects in their house. This should help our team find a direction and solidify our concept. This will then help us in writing our proposal, which is the next assessment piece due.

Week3 Zoom meeting

In week4, Lorna had an online meeting with us to give answers to our confusions about the changes of our physical computing teaching plan. Our confusions mainly focus on how can we build our project individually at home without tools like saws, laser cutter, etc and what scale our individual project should be compared with group project. Lorna gave us accurate and patient response, which can help us make a plan for the rest time of this semester.

Week3 Group meeting

After the Zoom meeting, we conduct a group meeting focus on our current idea and prepared for the presentation of next week. In the previous meeting we conducted last week, we have already decide our idea, which is a creative learning game for teenagers to help them learn coding in a more enthusiastic way. So in order to help the audience understand our idea in the presentation next week, we communicated about the structure of our slides and divided the work.

After a long time discussion, we decided to make our slides according to the structure below:

Introduction of project and team in general

• Team’s name:BEE BEAR LION
• Background:Problems solving
• Inspirations:Chessboard at CES 2019
• Purpose
• Target Audience
• Concept
• Storyboard
• Plan
• Prototype/Body-storming
• The body storming part was assigned to Bowen, and the other three group members's task is making the presentation slides. The part I should make in the slide is about our concept and story board. Finally I have to make a video to combine all our works into one video and upload it to the youtube list.

We also prepared three questions at the last page of our slides, these three questions are all about technical problems:

• How to make the character move and turn right/left?
• How to make the camera detect the QR code (Marker) and translate it into code in our computer?
• Which programming language can help us control the movement of robots? Can we use Javascript?

Week4 Contact

In the two contact of this week, we present our idea on the class. the contact class was conducted online via zoom. After our presentation, we got a lot of feedback from our classmates, and we summarized the following points from the feedback:

• Explore the gap between plaint-language and coding language
• Change the target audience from age 9-12 to the people who never learn coding language (the age is not enough specific for our project, cuz there may a lot of children already had the foundation of coding)
• Maze — Box moving (From Zebing Yao, Clay 'I think keeping the robot in the maze is limiting')
• This would allow for the robot to modify its own code being able to correct your code. (From Clay)

So in the next session, the first change of our concept is that we are going to change the idea of the maze into a push box game, which let the player control the robot to push the box to the specified position by coding.

As for the robot, we have several ideas to achieve the function.

• The first idea is using the Nintendo Labo robot as the character, this robot can move forward, backward or turn left and right by the vibration of the joycon. But this robot is not programmable, so this idea was abandoned in later discussions
• The second idea is using Anki vector smart robot, which is programmable and is very suitable for our current concept.
• The last idea is using Arduino to build our own robot.

Critique conclusion & Idea improvement

---

Through the critique from the Slack, our team discussed and concluded several points as the future potential improvements.

1. Focus more on exploring the gap between plaint-language and coding language (From Lorna)
2. Change the target audience from age 9-12 to the people who never learn the coding language before. (the age is not enough specific for our project, cuz there may a lot of children already had the foundation of coding)
3. Maze — Box moving (From Zebing Yao, Clay 'I think keeping the robot in the maze is limiting')
4. This would allow for the robot to modify its own code being able to correct your code. (From Clay)
5. It is hard for our target audience, which needs an effective and attractive tutorial for them.
6. Consider more about how can users learn from failures (when they use the wrong coding logic or the wrong order of the codes blocks).

What our team have done so far

After reading through the critique, our team has conducted some research about the existed coding-learning robots, for example, the Anki Vector and Cozmo. Anki Vector is an AI robot that can express emotion and accomplish simple problems. Furthermore, the Vector supports audio-based commands input and gives access to users to re-program every function of the robots. That is perfectly suitable for our concepts and solve the technical problems, as we have no idea about how can we build our own intelligent character. However, with the help of Anki Vector, our team can revise the initial idea and make the interaction process easily implement.

Impacts on our concept

The original workflow of the Ninja Run is to use code blocks to compose a functional program and recognized by the camera. Then, the camera transmits the data to our laptop. Finally, the laptop output the transcribed data to our intelligent character to manipulate its movement. Credit to Owen's magnificent works in the proposal slide, the image below demonstrates the basic workflow.

Although the workflow is clear, the mechanism and technology are still not workable at that time.

The new concept of our project is that using the Anki Vector as our character, in terms of its functions, we can use voice input to send commands to it. We only need to pre-program some functions with it and use voice input to trigger those functions. Besides, based on the ability of the robot, our team modified the content of the game, as the maze is only involved in moving but we want our users to learn more coding technic.

Therefore, the new game will be moving the box to the target place(still not the final decision). It has some benefits, like with the different goals, the game can shift the difficulty to improve the usability. We still keep the coding box because of its tangible features that can enhance the understanding of users.

About the camera part, we want to build an app by unity with the AR technology. When it detects the coding blocks, the app will output the matched audio to trigger the robots.

The plan for the next weeks

The allocation of every team member for this week is to explore the gap between plaint-language and coding language and get through some literature reviews to find out the playable solutions. Besides, put the concept into practice, use the unity to build the app and try to re-program the Anki robots to meet our demands. Furthermore, the concept and interaction flow are still not fully decided yet.

Critiques about Team Garfunkel’s Idea

The overall feedback, about how the idea of exploring sounds through everyday objects, was positive; however, people were confused with our concept.

These included:

• What are natural sounds?
• Will natural sounds be used or instrumental notes, when a user interacts with the object?

These questions stemmed from a lack of clarity with parts of the video. I had edited it to play piano notes when the user interacted with the object. At the time I was thinking that the piano notes would be used as an example for when and what type of sound will be produced. I should have also explained that when a person first interacts with the object, the natural sound the object produces will be used and manipulated, using other interactions, to change it into an instrumental note that is similar to the natural sound. From there, the person can create a tune. This was not effectively communicated and there were still errors, such as not being able to record the natural sound and use it later to create a tune.

All the feedback received has been useful in being able to build upon and improve the idea. They also allowed me to figure out what the next steps will be. The feedback given gives a clearer understanding of how people viewed the concept. This gives you different perspectives, to keep in mind when you’re continuing to expand and explore the idea.

The majority of the feedback focussed on making the context and target audience more specific as the initial idea was too broad and general. Originally, the focus was on allowing anyone, who was not a musician, to be able to create music anywhere. The main objects would be those that could be found or placed on a table. This made it hard to come up with a concept that was broad enough, because of the potential that users wouldn't share enough similarities to create a successful product, therefore, narrowing the user base will provide a clear idea of what important functions and purposes should be focused on when creating the concept.

Issues that people found with the broad concept were:

• The concept could become bothersome after a while;
• Making every object musical will take away the novelty;
• Possibly irritating for those around the user.

These issues will be taken into consideration when further exploring my pathway as I want to create a product that users will continue to use, find playful, enjoyable, interesting and motivates them to use it without becoming bored and irritated by the sounds that are produced or even the interactions themselves.

People did give suggestions about what type of context to focus on and whether to use natural sounds or tunes. The ones that stood out to me were about using the concept for educational purposes. This got me thinking about schools and what area do students have trouble with and where the exploration of natural sounds could be useful. I wanted to have a way to incorporate these sounds as a lot of people were interested, particularly in regard to the natural sounds of objects and how the interactions and size would affect the sound.

Critiques about the Presentations

It was a different experience from what I had done before when viewing presentations and critiquing them. I was curious about how this would be accomplished before the contacts; fortunately, the technology and tools used made critiquing easier than I thought it would be because of how we got to watch the video together, hear discussion on Zoom and write the critiques in slack. I have always found it hard to write critiques, whether this is on paper or online, as I normally prefer a face-to-face conversation because then you can use facial and body expression to determine how the person is taking the critique and whether you need to clarify. This is a skill I need to improve on, as well as working on techniques such as writing quick notes on what to include in the critique or the idea itself, rather than long sentences or paragraphs, as my critiques start to become generic rather than specific and useful over time because I miss or forget information. Overall, I found the presentations had aspects in all of them that would be interesting to explore and investigate whether this is teaching people certain tasks through simulation. Other interesting aspects were interactions with materials, objects, familiar concepts or spaces in a new way. Teams could explore their idea in a new context, situation or place. They can also look at specific features of the idea or different focus groups.

Teamwork

This week as a group we worked on our concept, iterating through the problem space and how the concept would solve this problem, then creating a storyboard and detailed explanation of the product, all of which was used in the presentation on Tuesday.

Problem Space

In the group my main focus was on developing the problem space, the main issue we feel 'Helping Hand' Will solve is a form of social distancing but devolved through wanting to be polite and not having to tell others what distance you wish to keep depending on your mood or what we called 'Social Activity' Listed below are three examples and thus the social distance they wish to keep from others. These might be a bit extreme however they get the general idea across. This problem space made it quite easy for us as a group to then develop the concept to better solve this problem, through gestures and motions used to keep others away (rude) or inviting them to come closer (nice).

Post Presetation

After the presentation there was numerous feedback, however the main concern was in the form of the concept, creating a robotic hand that creates complex gestures might be too hard to do, too heavy or creative that it attracts attention when the user wants to be left alone. This is where the group split into individual workers, to develop their own form they would like to work on, and what role that fulfills, personally I really like the emotion and 'sass' that body gestures can make, so why stop at a hand, I want to make a small moving mannequin that conveys these gestures of rudeness or being inviting, while being simple with maybe one joint per limb, making it easy to pull off, while the whole body can convey emotion better.

Critiquing

With all the changes going on, it was really interesting to use the platform of Zoom to participate in classes for the first time. Using multiple platforms such as Zoom, combined with Watch2gether and Slack was a creative way of watching all the teams concepts and then providing feedback to them, while engaging in class.

Energy Saving Emily appeared to be quite an intriguing concept to a lot of our peers! We received some great feedback from everyone, giving us paths to explore that we previously didn't think about. I compiled all the relevant feedback that was given.

We did find a few comments about how implementing the aspect of discomfort could be deterring to people wanting to interact with it, however our topic is Change Through Discomfort, so that sort of defeated the purpose of our whole idea.

Potential Future Implementation Aspects

Aspects of feedback that have really interested me include implementing an emotional connection between the user and Emily, Emily providing further discomfort by potentially making the water or aircon too cold when using the shower or AC for too long, how Emily responds to several bad habits happening at a time, or the movement of Emily being annoying, whether it be the user having to chase Emily or Emily chasing after the user. A comment that definitely gave us something to think about is having some sort of reward system or end reward once the behaviour has been corrected.

Concerns Going Forward

Reflecting upon these ideas, I would love to explore more around the element of restricting the users access to their energy supply if they are not being energy efficient. I think this is a playful and annoying element that could be applied to the concept. In terms of adding an emotional connection to Emily, ideas that I'm currently thinking surround speech between the user and Emily. I'm unsure how I would implement this though, so further exploration around technologies would be needed. These ideas sound really exciting however I am definitely concerned about finding ways to actually incorporate some of these ideas. I've never really taken on a task like this individually so am hoping that the aspects that I do incorporate into my final concept will be functional and suitable to the project!

Things To Look Forward To

We have luckily gained ourselves a new team member! It will be really interesting to see the ideas and skills that she brings to the table. I'm looking forward to exploring my own skills and thinking about how to get a start on Emily, especially in the situation we are facing when a lockdown could happen at any point in time. It's quite important that I have access to as many resources that are necessary, should anything drastic happen! I'm also intrigued about the paths that my team members decide to take, to see what kind of form they give Emily and what habits they decide to explore. It will definitely be a challenge but is something that I'm getting more excited about as the semester progresses and our idea comes along.

Challenging the first week to have the studio online, especially had presentations and critiques, but it actually worked! Using WatchTogether makes the PowerPoint clearer to watch and most of the presentations easy to follow but I have to say I prefer to have a discussion face to face rather than online.

Most of the teams worked well and had interesting ideas/concepts and all of us still have some space to work on with.

Critique Reflection

Our team received 38 comments to help us improve our idea.

We organized them all and sorted them into several aspects below.

Physical Interactions

There were several comments point that our design is "too smart" to have physical interactions.

Motivate Users to Learn Emotion Management

Our design should let users learn to express themselves in a healthy way. We should design a function for users to actively express their dissatisfaction and learn to manage their emotions while being aware of their own emotions. Variations Reactions to Different Emotions and Words are needed.

Expression

We should not prevent users from expressing their emotions, and we should respect the way people express their emotions. Encourage them to vent their bad feelings instead of hiding them.

Reflection

Our original design is based on speech detection. All the technologies are around the sound sensor which provides less physical interactions between users and our design.

In this case, we discussed making some changes. We discussed focusing on adding more physical interactions to provide users opportunities to interact with the flower rather than only let the flower detect what it heard. And this justification may prompt users to express themselves while being aware of their own emotions. Users are less likely to passively accept detection and judge their emotions, and more likely to actively and correctly express their emotions.

Moreover, Change the original state of death of the flower, and make the user realize that there may be a problem in the way of emotional expression at the moment in a better way, so as not to make the already negative emotions of the user more negative.