Plan for next two weeks

This week I'm starting to check anything about voice control and sound effects. Because these are two essential parts of our project, voice control can maximize the use of visually impaired people; they do not need to use any physical remote control, only need voice commands to complete the relevant operations. Sound effects can directly improve the user experience of the device. The central part of the safety alarm has been completed, so this time, I want to discuss these two aspects mainly.

voice control

Regarding voice control, I considered using Siri as a terminal to receive user voice commands. Shortcuts in iOS 12 let get things done with our apps, with just a tap or by asking Siri. In addition to running shortcuts available on your iOS device, we can use the Shortcuts app to create custom shortcuts, simplifying everyday tasks by combining steps across multiple apps.

I have successfully used shortcuts to control my desk lamp (not a smart desk lamp) to turn on or off. So my current idea is to use software that can remotely control Arduino as our remote control. Then through a series of measures, the app is connected with shortcuts. In this way, you can bind Siri's voice commands to the Arduino's buttons. Thus, a series of operations can be completed through voice commands. So based on this theoretical basis, what I want to do now is to find a suitable app that can connect to the Arduino Bluetooth module. Then I need to write some code for the Bluetooth module to make it work. When these tasks are completed, our voice control should be ready to use.

Sound effects

In the initial plan, it was considered that people needed to wear eye masks for testing. So we don't want our users to wear headphones. Because the human ear is also an important organ to distinguish directions. So we still want to use sound to play sound as much as possible. As it happens, Bonnie and I both have a HomePod, which is a speaker consisting of 8 speakers and a series of sensors. It can detect the surrounding environment based on where it is placed. With spatial awareness, it automatically analyses the acoustics and adjusts the sound based on its location. Direct sound, including the main vocals and instruments, is beamed to the middle of the room, while ambient sound is diffused into the left and right channels and bounced off the wall. So the entire space is filled with rich, well-defined tones. (From https://www.apple.com/au/homepod/)

In other words, we may only need these two speakers to create a very realistic sound effect. So what we need to do is to use Adobe Audition to make some analog surround sound. As can be seen from the following figure, in the audition, we can set the left and right channels of a music clip, and through some adjustments, a single channel of music can become multi-channel, or from the left channel to Right channel. In this way, to the user, they can actually hear a sound from left to right, just like someone running in front of them. And audition can also analyze the location of the sound. We can carefully adjust some parameters by viewing the analysis results to make the final sound more realistic.

So based on these theories, I will try to complete a sample in the next two weeks. If successful, I will invite some students to do a user test.

Work Done

This week, other course assessments have been focused on due to them having closer due dates. Thus, not a lot of work has been done with the concept and the prototype. The balance between the different courses has been something I am struggling with as I tend to find that I spend the day focusing on only one project because I find it hard to leave it and go onto other work. This has meant that I don't spend equal amounts of time on courses. Possible solutions that could remedy this is by having a plan for how I am going to tackle the busy weeks ahead. I find it hard to come up with a plan because I never stick to it and, therefore, it only serves as a reminder of the work I didn't complete as I overestimate what I can accomplish. Thus, I need to work on improving this skill. Some of the work, I have managed to do, focused on further exploring the concept and problem space to find ways to improve the aspects of teaching children multiplication in primary school. How this was achieved was through online research into strategies to teach multiplication and what the recommended order is or different techniques and skills that make learning easier. Other research involved finding resources that children would have access to in the class and could use with the prototype. Some questions were asked to a teacher aide to see how children are taught multiplication. One of the ways they kept mentioning was repetition, to keep the students repeating, saying or writing in worksheets over and over again. This was also supported by online research. Additional Work done was exploring and using the accelerometer and gyroscope. I did have some issues with the sensor and the Arduino as the inputs were not changing or not appearing, even when the sensor was being moved and rotated. This was then found to be due to the lack of connection between the breadboard and sensor. As a result, it will need to be soldered.

Ideas for the Concept

The idea for the concept was to have the user roll random numbers and then multiply them together. Not enough research and reflection on previous research had been done, however, on how students start learning multiplication as multiplying random numbers is not an effective method for memorisation. The methods were not focused on further, because the purpose was to help students with learning multiplication by having it be flexible and work alongside content or methods the teacher is using to educate them in the classrooms.

From previous research conducted, a specific order for how the times' tables should be taught was found; nevertheless, through further exploration, the same resource was found to be updated and the order was slightly different. Therefore, the current ideas are to facilitate learning multiplication one times table at a time through repetitive learning and aide teaching through the concept. For example, the two’s times tables was suggested as the first one that children should learn as they are at this stage of learning familiar with doubling numbers because they learn addition before multiplication in primary school. How the user would then use the concept is by manually rolling the dice until it landed on a two. They would then do an interaction which has yet to be decided, however, shaking is currently being thought of. This selects the two and thus, all number will be multiplied by two until this process is repeated. The user would then manually roll the number 0, throw the dice and then produce the sound. The process is then repeated until they have completed the 2 times tables and become familiar with it over a reasonable period by continuing to repeat this process and getting the answer correct, as well as understanding why the answer is that number through asking them word questions, as well as using addition resources that will not be incorporated into the dice. The child would then randomly roll the numbers that will be multiplied by two until they are familiar and comfortable with this process. The user can then move on to the next times' tables.

Plans for Next Week

Further research and user testing will be conducted to understand what changes need to be made to the concept in terms of improving how music is incorporated with the prototype so far. Other work includes getting the sensor working with the photon, soldering it and ensuring the sensor can determine which way is up.

Hardware

This week I tried to improve the prototype. Based on previous user feedback, I plan to move the Arduino and battery compartment to the waist and connect it to the helmet with a jumper to reduce the weight on head. This is a difficult decision, because my prototype has tended to be completed for the hardware part before then, and it is mentally difficult to disassemble and remake it. So, I first tested the feeling of connecting the jumper to the waist and the possibility of fixing the Arduino at the waist.

My idea is to place the Arduino in it with a sports belt. I first experimented with the battery compartment. In fact, it will still shake when it is in it, which is not very stable, and there are many jumpers on the Arduino, which is difficult to fix further. Another problem is that when the jumper extends from the head to the waist, there is actually a feeling that the head is constrained, which is not convenient for exercise. So, I finally decided to abandon this change.

Another direction of improvement is that in fact, the major source of the weight on head is the battery set composed of 6 AA batteries, so I bought a 9V battery as a replacement. But there is no 9V battery connecter at the moment, so I only used a jumper to test it, and it worked fine so far. I plan to buy a connector at Jaycar next week.

Name of concept

This week I finally thought of a gorgeous name for my project called Breath I/O.

I/O represents both breath in / out and input / output for interaction, which can represent the concept of this project better. A topic this week is to prepare a pitch for about 1 minute to introduce your concept to the audience. So here it is:

People breathing every day, but nobody pays attention to it. But if you spend 10 seconds with me on your breathing… 1.. 2.. 10… have you felt your breathing rhythm? Do you know that keeping a good breathing rhythm is benefit to your exercise? Many beginners don’t or find it very hard to keep because of distraction. But with this wearable device, it can detect your breath and help you to breath with a good pace. All you need to do is put it on your head and enjoy your jogging and the small screen will notify you when your breath doesn’t follow the rhythm.

Keeping it short but attractive.

# Website

The preliminary work of the portfolio started this week. At least we have a banner, hooray, and more content is still being added. Stay tuned ;).

# Future work

In the next week, I will continue to improve the programming and UI. Based on the feedback, I’m trying using graphics to replace most of the text, and add a tutorial at the first time, so that users can understand the way of interaction more clearly.

Design Process

Simple code

Since there are some people suggesting that we should make the simple make more sense for users from the last appraisal, I just wondered whether the current version of the simple code is easy to understand for my target users or not. And I tried to find some simple code design principles to help me make it more understandable, but it seems that there is no such type of material. So I just planned to use the card sorting to test target users which simple code is can be easier to understand. So, I used the online card-sorting websites trello.com creating four cards with a random order, then I introduced the basic functions to them, and let them do the card sorting.

I found that all the participants sort it like these two results:

So in the next stage, I prefer to use this version of simple code, since it seems to make more sense for users.

Map

As mentioned in the last journal, two of appraisals thought that they feel confused about the difficulty of the tasks, they do not think that the difficulty is different, so this week I was drawing some tasks for user testing, to let the user rate for the difficulty, then I will adjust the difficulty to make it not too hard for all the users to motivate users to use that, but some kind of difficult to keep them engaging and keep challenging for some tasks. The current maps I have thought about this week is:

Plan

Since my partner, Bowen is working on combining the two parts of our work together, so next week it will have the full functions, at that time we will do some user testing to choose the tasks for final outcome, and set the difficulty. Besides, I will write the instructions for the users that can make sure they understand the purpose and basic rules of the concept.

Work Done

This week I started looking at ways to implement the ultrasonic sensor and the microphone into Spud.

I started watching videos of how to use ultrasonic sensors and tried it myself. It was quite straightforward and easy to work on. For voice recognition, I found a way of using the microphone on the laptop instead of the Arduino microphone module using C# (they used the Visual Studio IDE) from this website tutorial here. Hopefully it would not be that hard to implement these in Spud as it might be tricky to do this with two different modes, the alert and friendly mode, and having to send information from the visual studio IDE to Arduino.

I also did user testing to determine how far the distance of the person walking towards the user should be before Spud reacts. Results:

• >1.3m away from user = normal
• <1.3m away from user = Spud turns angry as a warning
• <0.8m away from user = Spud shakes its head/ waves to the person.

Work to be done

For Spud, I have to start implementing the ultrasonic sensor and the voice recognition as fast as possible. At the same time, I also have to work on my website as that might take a long time to make it nice and presentable. I'll first make the form of the website and set up the CSS and JavaScript code. Then, I will write down content about my work with Spud and decide how to display it without making it just seem like a document.

Work Done

In this Tuesday, we took a meeting to discuss how to show our project in the final exhibition. We thought we have to show our project in a room on the campus. For each of us doesn't have enough place for our project in our homes. Also, our internet can't support this live.

In Friday's workshop, we took our materials to the campus. We were trying to fix our materials together. However, there were still some problems.

Firstly, because our transparent board was too fragile to support an adult exercising on it, we had to put some supports under the board. However, we only bought 4 supports and it was really difficult to meet both practicality and aesthetics. But, if we buy more supports, there isn't enough space for the sensors and the LED strips under the board.

Secondly, we still can't find the suitable pressure sensor. We found that the pressure sensor on the market can only detect the pressure of 10kg at most, and cannot detect the weight of an adult.

However, Clay told us that maybe a Wii fit board could solve the problems. For it can connect with the computer by bluetooth. Unfortunatly, the seller from EB game told me that the wii fit board is out of stock in whole Brisbane. Finally, I find a second hand one from facebook market and we are planning to pick it in next Tuesday.

The plan for next week

After we picking the wii fit board, we are going to conbine it with our original materials together. And the most important thing is that we have to make sure how to transport the data from the wii fit board to the computer and use it in Arduino. This is a big challenge for us, because none of ue has used such thing before.

Work Done & How it Relates

The start of this week was pretty quiet for me with this course. After the Wednesday studio that included the explanation of the next deliverable and my questions answered, I began focusing on building my website. I'm not sure if it's the best task to try to tackle next but it's definitely one that scares me. I have been trying to implement the base of the website and get a rough version of the landing page so that I can focus on my product as well as producing and inserting the content.

I envisioned my landing page to be very minimal with just the word 'Emily' and a light bulb that could be turned on and off (ie. saving energy just like Emily is designed to do). I have made a still photo of what I have envisioned with the "lights on". There is definitely more I could play with but my focus is definitely needed elsewhere.

Whilst the design is quite basic and blunt, this is the style that I am intending. The product is in the form of the snowman which is typically portrayed as bright and snowy and fun. I hope to move this into the rest of my website.

Work To Do

With only 17 days until the Exhibition, there is a huge list of things I need to complete. I'm trying to break it down into tasks that I can complete over a few days. This helps me remain calm and not panic about the craziness of the ending of the semester! I will still need to build the rest of the website and write the content I want to go into it.

Emily still needs a lot of work before I am happy with the final form. I need to fix the technology including the temperature comparison and the display of lights. I also hope to finish the additional parts of Emily (such as arms and a hat) and combine it all together including spray painting.

This week started with the Wednesday studio sessions recap in which everyone expressed their pros on isolation as well as their current progress. Followed by this I investigated my prototype feedback and where/how I can improve my concept ( but also not change the concept). This includes making the system easier and quicker to use such as making the system hands-friends. Furthermore, I need to do investigate more formal research towards my user audience and the interaction.

I have also investigated creating my portfolio which will be created over a period, where more and more will be added. My team has also had a chat about the final report and what that entails for us.

Over the next few weeks, I am starting to adjust the prototype so that it is ready for the final exhibit. This includes adding in a working multi-user experience and a more hands-free experience.

What I did?

In my last prototype, I can use the punch card reader to the punch cards and insert the data into the web based IDE, so this week, I redesigned the interface of the website and start to build the web-based game. This was built by react and redux, now I can use keyboard to control the movement of the character and I also made a map for this game, as it shown in the picture above, I added some rocks and trees in the game window, and the character cannot walk across these objects. what users should do is to control the character to get to the destination. I also added a console window under the game window, so when users run the code, if there is any error occur, the error information will be directly shown in the console window, which will help users to debug by themselves.

What I will do next?

The game function still need a function to trigger game victory, and I still need to link the IDE with the game, in order to let the users control the game by punch cards. as for the physical part, I want to add some physical buttons to trigger the function of the buttons on the browser, so that users can focus on the physical devices instead of clicking the buttons on the screen.

Team Appraisal

In this week's studio session, we have conducted the team appraisal session. In this session, we are required to work as a team to critique the work done by the group members from 3 groups. In the task assignment phase, we were arranged to comment on the work of team X, team Hedgehog and team E.M.S. I have gained a lot at this stage. Due to the impact of the epidemic, user testing is difficult for the entire project. Although we have thought of many methods such as using questionnaires and zoom meetings to obtain user data for project iteration, the obtained data is still not enough. I can be explained the whole concept in the form of video demonstration and document supplements to allow classmates to understand the whole picture of the project and obtain more user data so as to gain more inspirations for project refinement and reach the user-centered design. At the same time, I can also reflect on my own projects by watching the work of other students, so as to get inspiration for improving different feedback.

The following reflections were drawn when I watched the works of other groups

Team X

The team completes the entire project in the form of teamwork, and they share the same domain as ours, which is music metric. They chose to combine music and sports to make sports less boring. One of the highlights in their design is not only to provide users with sound feedback but also a series of visual feedback (such as the use of light bars to present different visual effects according to different pressures), which is something we can learn from. In the project, less consideration was given to visual design, and I put more emphasis on the realization of functions, so although the whole project realized some of the functions that I wanted to achieve, the work was not so beautiful. In addition, the combination of sound and cool visual effects allows users to have a better immersive experience, thereby interacting with music better to achieve a user-centric design. By watching the idea of the group's projects, I decided to work hard on the visual effects in future improvements and think about how to present daily sounds more interestingly.

Team Hedgehog

This team also works collaboratively to complete a whole project and I really like their design idea of helping visually impaired people improve their hearing through games. Because the domain of the project is not the same as ours, we have little inspiration for our project in terms of inspiration and technology, but the team members among them gave me some inspiration for the construction of the 3d music scene. Our project is also highly related to music, and I am currently looking for suitable methods to make the entire product more playable and interactive. In the part related to music playback, I can consider using Unity to create a visual-pleasing music playback environment, so that users can get a more immersive experience.

Team E.M.S

This team works for the domain of enhancing mundane space. Their aim is to turn the boring housework activities into a project that the whole family is happy to participate together in the form of hitting games. This not only increases the playfulness of housework activities but also ensures the collaborative experience between the family members. This project makes me think of the collaborative experience in our project. At the first prototype proposal, we received some feedback of whether the product can be operated by multiple people. Since we haven't thought of a suitable method to make users interact with our sound lab with multiple people, this part of the idea is a bit stagnant. Their idea was to enable more people to participate in the form of a batting game, which also gave me some inspiration. In addition to the sound collection stage, which allows multiple people to use a dropper to collect sounds, whether it is also possible to design some corresponding interactive games during the sound mixing stage to make this product more interactive. This is the focus of work I am looking forward to working with in the following week.

Feedback & reflection

I was responsible for designing the test tube part in our group, and I received the comments of three groups in the team appraisal part related to my part. After reflecting on their comments, I summarized the following points to provide ideas for the iteration of the future project.

Visual aspect: LED lights are not very obvious

The critique from team X noted that:"the effect of LED lights is not very obvious. From the screen, we can't tell whether the LED light in the test tube is lit." Since I chose plastic test tube toys to make the test tube part, the material is transparent, so there is a disadvantage that it can not hide the sensor used in the test tube for the interaction effect, also, because the plastic is thicker, the brightness of the LED lights is not that obvious and user can not judge the status of the device by watching the brightness of the LED. In this part, I will choose to replace the test tube production materials. In the first aspect, I will consider choosing a larger test tube to install the required sensor. In the second aspect, I will find a way to make the visual effect more obvious (such as changing the color of the test tube in order to make the LED light more obvious). In order to match the parts of my other team members, we will make a discussion on the selection of materials and the overall design through the form of group meetings to make the overall product look more consistent.

Fix the sensor

As one critique reads that: "We are worried that when the user shakes the test tube or puts the sound in the test tube into another container, this process will affect the stability of the connection in the test tube." In the design process, because the jumper cable will indeed have poor contact, sometimes it will also cause disconnection by the different actions of the user. Therefore, it will take a long time to investigate the cause of the connection, which will not only spend a lot of time, in the final exhibition, this problem will cause a poor user experience. Originally, we welded some sensors, but it was still not stable enough. After using, moving and other operations, it will still cause disconnection. We discussed and decided to use electrical tape to fix the sensor and cable wire. Although this method is a bit time-consuming, it does play a fixed role and also makes a simple integration of the messy cable wire to a certain extent.

The form of the presentation of music needs to be considered twice

I have also received relevant suggestions regarding the presentation of music playback. For example, one group from the critique showed that "rather than lighting an LED, I think you need to display the presence of sound in a more obvious way." Combined with the reflection gained from critiquing other groups‘ work, I think I can add some visual effects when the users playback the music. I also need to do some research on how to combine sound and visual effects together, which can help me better serve the project. At present, I intend to use Unity to design a scene for the music playback as visual feedback for users playing this device.

Project Progress

This week we work collaboratively in school and integrate the parts we have completed separately together. At present, the team members have realized the main functions of their respective parts. The main task of our team this week is the integration of code and debugging of the entire project. On the arduino code, we have classified and commented according to different parts of each different section to facilitate debugging in the future

At this stage, we have only completed some main functions. Ideally, the device consists of a dropper, three test tubes, two jars, and a beaker. But at present, we just installed a dropper, a test tube, a jar, and a beaker and almost used up the analog and digital interfaces on the Arduino Uno development board.

We searched for the relevant development board and calculated the number of interfaces to be used for the final product. We decided to purchase Arduino mega as our development board for subsequent design.

The following picture is the overall presentation after integrating the work of all team members.

Future work

1. Organize the feedback obtained and discuss it with the group to determine the overall improvement plan for the next round iteration.
2. Thinking about how to add visual effects to make users more immersed in the interaction with sound.
3. Improve the overall product design, because for the current product, the nakedness of many cable wires will make the whole product look messy.
4. Start thinking about how to design my own portfolio.
5. Buy sensors with strong interaction effects and larger development boards with more interfaces.

Team meeting

This week my team and started with having a meeting to go through the feedback we had gotten and discuss what the next phase of our development would be. The feedback we had received was mainly positive and gave us confident that our concept and how we displayed it in the video was good and that people thought it was exciting.

Everyone of us had asked questions in our videos in addition to a series of team questions that asked around the overall concept. For my prototype I got confirmation that the trowing metaphor was appropriate for its functionality. In addition, many expressed that a way for the users to either replay the message and/or delete was something we should include so this will be taken into further consideration.

At the team meeting we decided that we would have a physical meeting on Friday to see if we were to put the prototype together as one.

Friday Meeting

The team concluded that we would try to assemble all of the prototypes into one so that each and every ball could perform the entire interaction flow. While Thomas and Sigurd managed to put their prototypes together fairly easily, me and Tuva met with some additional issues. Firstly, we hadn't used the same libraries on our prototypes so we had to decide on which one to use. Tuva tried to install the one I used, but for some reason this didn't work and so, together with a tutor, we decided to scarp it all together. A new library that was available for both me and Tuva was found so we decided to use this one. Since most of my code had been based on the last library I had used meant that I had to rewrite most of my code. It took my a while but I managed to make it make it work finally. Secondly I was tasked with merging mine and Tuvas code. However, as we expected the accelerometer couldn't distinguish between a shake and a throw so it was decided to add a squeeze in-between to "lock" the colour and starting the throwing state of the ball. This worked perfectly and so now the interaction is as follows:

The inside of the ball looks currently as a hot mess but everything works perfectly. However, because of the new pressure sensor the schematic was even more difficult to look decent. Hopefully people can make something out of it.

Going forward

Based on the previous user testing the team also discussed some additional feature that we could add or try separately. For my part, most of my users wanted a way to delete the message instead of sending it. I was thinking of adding some code which registered when a user dropped the ball instead of throwing it upwards and that this could be a good metaphor for deleting and resetting the ball. This is however something I will have to look into closer at a later point.

Recap

This week started off with the regular recap that we gave to the class about our progress and isolation. It was nice to hear what other classmates thought were the positive aspects of being isolated. There were many answers that ranged from saving time on commutes to saving money on food (ducking in to McDonalds on the way home) and not having to get up early and stay late for some annoying class times.

In terms of work this week, it was pretty relaxing not having something directly due but it was also good to just have a look at whats coming especially in this course with the final exhibit. I also had a look at the reflection and got that all formatted, headings etc so I can knock it out when it comes time to do that.

Over the next week I need to start making the adjustments to my prototype to ensure its ready for the final exhibit. This will include adding a new input device which may be using the bluetooth depending on if this is possible for me to achieve. I also want to make an instructions menu screen outside the elevator because inside you have a limited amount of time to play the game and get to where you need to go.

This week started off having a short break from the project to look at other courses. During Tuesday’s studio I expressed my concern about the form of the prototype. Lorna had left a few suggestions on Miro, and my mind was constantly going back to feedback I’d gotten for the previous assessment about looking at something more tactile such as a guitar. I’ve brainstormed plenty of alternatives to the current piano form. The guitar would involve a very physical form of interaction, with buttons to register the number, and an area to strum to actually input the pressed number. After mentioning that in the studio, Lorna made suggestions which I interpreted as something along the lines of exploring a form that wasn’t resemblant of a musical instrument, or even any current existing form. She mentioned on Miro how my current form is rather specific. I chose the classic piano form as it is something that’s familiar for most users - whether it’s an actual piano, computer keyboards, or other instruments similar to a piano that has progressive keys representing notes. An alternative design I’d considered was something more flexible that could be adjusted to fit the preferred form. Users can curve it for their comfort and allow minimal movement. The keys would be attached to each other in a chain-like structure, to allow for pivotal adjustment. I do have the concern that this may not work if I wanted to contain the Arduino and breadboard within it, so I will have to ponder this idea further.

Form aside, I had an issue with numbers being registered with simple touch, as this could be done by accident. I realised I could solve it by having the jumper wires connected to copper tape below the key, which would be connected to the touch input when the key was pressed down. This makes the action of pressing down the key, not just an additional form of feedback, but a necessity to register input. Due to the cardboard structure being rather unstable, I have been looking for stronger materials to use to achieve this effect. Metal wires are definitely currently the most desirable and accessible material to use to create this pressing down motion, whether it is by using it as a hinge, or placing springs beneath each key to have it spring back. This would allow for each of them to go down without bringing the keys around it down with it, which is what the previous structure had.

I’ve illustrated what this might look like, to better understand how it might work since visual diagrams are a bit easier for me to wrap my head around. The desaturated pink represents the copper tape, and the components have been labeled appropriately. The key is similarly shaped to the current keys, however the bottom area has a small area that flattens out to ensure contact with the copper tape below it. On the right, I’ve also included a sketch of the previous design of the keys, which I have completely scrapped now.

In regards to the detail of the keys themselves, I’ve chosen to scrap the idea of braille for now, and using physical cues such as the markers on the f and j on the English computer keyboard, so users are able to understand which key they are currently at, and can quickly adjust their finger positions for ease of use. This also would work with the current idea of input being registered with the pressing down of a button rather than simply tapping the copper tape area. I’d previously had it as an idea, thought it wouldn’t be enough, but after feedback it’s something I will definitely return to.

At this point in the semester, I definitely feel I’ve fallen back, and it’s difficult to regain traction. I’m hesitant to access materials at uni since the travel is long, so I may need to stick with the materials and tools I can access here. I am somewhat confident I can create something feasible and of good quality with what I have, though it may not be the wood and metal I initially desired. My main motivator is to meet my personal expectations that I set from the beginning of the semester, and any that those around me may have for me. Especially since my sister had success with this course. This semester has had quite the effect on my mental health, especially right at a time where I was starting to seek help for it, but have now been placed in another new situation I need to adjust to greatly. Not really an excuse as everyone is dealing with similar things, however I have lowered some personal expectations to push myself to at least achieve my initial goals without overdoing it and feeling incredibly disappointed in myself for not achieving the highest possible grade. In the end I want to have had fun and learnt something, whether that's regarding the elements of my concept and research, or the skills I've been developing, or just about myself.

Going back to my concept, I feel I have focused too much on the form of the structure, and not enough on the actual learning aspect of the concept, so I have backtracked a little, and have been attempting to research more about how mathematics practise can be aided with music. One idea appeared to me in a dream (poetic, yes, but I have stress dreams every night so it’s not as poetic as I wish it was h a ha), to have music with questions tied in. Instead of assigning a note to a number, users are required to input the correct answer to the questions to continue the song. This is similar to learning the alphabet song, periodic table song, or practising multiplications like I did growing up to the tune of something. Some thought still needs to be put in, especially since this will require a lot of changes to code, which may be too demanding at this point in time. However, I will continue to research this since I believe it may have the potential to be more promising in allowing students to practise mathematics.

Recap This Week

This week I cannot use the zoom to take the studio since there was something wrong with that and I am still working on that, I hope it can be used next week. Also, the zoom is not the only thing not to work well, my slack also had problems, and I couldn't reply or send messages on it. The good news is that Lorna and I figured out a solution to make sure we can still communicate well. We decided to do the report back via email if the slack and the zoom still not work in the next studio.

Report-back

What is one good thing you can say about lockdown?

Cannot go to the campus makes me feel bad but I have to say, during this lockdown period, I do start to have organized days. I want to try to keep doing more and more things, because it's not easy for me to keep doing one thing, whether it's important or not. This time I spend more time with myself, which makes me want to live a more organized life. I also start small to manage myself. Whether it was from eating breakfast every day or exercising every day to making a daily plan for myself, these changes made me realize how much time I had wasted in the past. So during this lockdown, I think the most rewarding thing is to get along well with myself.

Based on feedback to date & what you have left to complete: What are your key priorities?

One of the most parts of my project that I have left is the physical shape building. My teammate and I have already ordered the materials that we are going to use, but it still has not been delivered yet. And I am still working on some coding areas to let my project work better. In this case, my key priorities must be the physical shape handmaking.

What concerns/questions do you have? (doesn't have to be directly project related)

Seriously, it's a really long time to stay at home, I am a little bit concerned about communication with other people. Of course, I talk with my friends and even staff in the shopping mall, but not like on the campus. Less social activities make me feel a little bit nervous about social someday.

In addition, during the completion of the project, I found that using a pressure sensor can hardly make all six petals receive pressure sensing. I'm thinking about turning the squeezable part of LOME onto the stamen so that the location is more concentrated, and there is enough space for the pressure sensor.

Recap & Refresh

I cannot believe it is already week 11 now, personally I am still not that into full study mood compare to the days we can still go to campus. But now since it hasn’t got too much time left for this semester, I need to refresh my mind and be ready for the final delivery. At the end of last week, I talked with Alison about my concerns about the following works in this course, and especially the final product we will present at the end of this semester. I feel a bit disappointed with the work I've done so far, although I do believe I have tried my best to make the prototype as much as I can. But I'm still not satisfied with the work. That’s because I have been to the exhibition last year to see other teams' final product and the way how they presented their work. Compared to their work, I fill my individual work is too shallow, and childish couldn’t reach the standard of this 4 unit course. The other reason is my last semester of doing the Master of Interaction Design course, I really want the outcomes of this course to be the highlight in my portfolio. Alison helped me to go through all the situations we are facing now and give me some faith to carry on the work from now on. Here is the reflection of what I did for the previous weeks (from week 7 to week 10) throughout the prototype development process:

The objective conditions are:

• team-based project switched into an individual project
• the workload to accomplish a whole project increased significantly without a functional team
• the difficulty of conduct user research especially with the user group of children (although the other team member made the commitment to conduct the interview with their family members, for some unknown reasons it didn’t happen)
• the limitation of materials and tools since we cannot get to the workshop on campus

My personal issues are:

• bad time management when work at home, especially procrastinate the documentation and demo video making works to the last three days of the deadline
• suffering demotivation during the prototype making progress, and being passive about the final product result
• feel helpless without working with peers in the same physical space sometimes, because peer pressure is one of my energy sources

In order to prevent myself fall into the loop being self-doubt and passive about the following works and challenges. I attempted to separate things I'm currently worrying about into two big categories, 1) the things I can do better or make changes from now on; 2)the things I cannot control or fix with my own capability and knowledge. By doing this, I have a clearer goal of what I want to achieve and improve the skill throughout, and by the end of this course.

Since there are two weeks left before the final delivery and the online exhibit, and four weeks before my thesis delivery. I would like to allocate my time into three parts:

1. the prototype minor functional changing and details improvement, to make sure the prototype could be prepared in its best state for the online exhibition
2. annotated portfolio website design and making
3. process documentation and try to include a bit more academic aspect into the project, keep journal writing(especially add more details into week 9 & 10 journals) and verbal material for the portfolio website as well

Studio & Contact Session

During this week's studio session, Lorna took us to have a quick look at the rundown for the final delivery. After the class I went through the brief again carefully by my self, and made a plan for what I need to in the next two following weeks. Here is my response to the activities in class.

Q1: What is one good thing you can say about lockdown?

To be honest I’m not able to think about anything good for my course and thesis. Since they have been affected a lot from many aspects, and those things directly resulted in unsatisfied outcomes. But for my personal life, there’s something I really appreciate and feel lucky about. Which is I could spend much more time with my boyfriend and his family at a farm. Which is a brand new experience for me.

Q2: Based on feedback to date & what you have left to complete; What are your key priorities?

I'm not really sure what I can do more for my prototype since I almost finish all of the parts of my prototype. But for sure, I will do some minor adjustments and improvements for my prototype to get prepared for the final delivery and exhibition. But at this stage, I don’t think I can do much more about my prototype under the current situation.

Based on two team appraisals I've got on Miro so far, both teams brought up the major problem that the input might cause confusion for the user. Since it became a good/bad choice question, that throw garbage into the recycling bin is good, and throw garbage into the general waste bin is bad. And they suggested me to add garbage distinguish or classify function to my prototype by using camera and machine learning skills.

For the function of garbage distinguish/classify, suggestions from Ben is to utilizs the colour tracking and detection feature. However, I believe it is kind of challenging and difficult for me to implement this feature from the technical aspect. Also, I need to consider my budget spending on this project, I don’t have much left to complete this as part of the final delivery, to be honest. I hope some of my teammates will have the chance to implement this interesting function ion their work. And I'm really looking forward to seeing the final results of their final product as well. Here is the source: webcam colour detection and processing (what that video is using) to Arduino

Finally, I believe the key priority for my own project is to put more focus on communication tools preparation and practice for the online exhibition rather than the prototype. The communication document for the last time was not finished properly and with pretty low quality from my own aspect. Therefore, in order to have better communication with the audience to understand and interact with the prototype online, I will try my best to do a better job of demonstrating the final product.

Q3: What concerns/questions do you have? (doesn't have to be directly project related)

• Again, the biggest concern from the beginning of this project and still hasn’t been solved is the lack of children's engagement throughout the whole design process. As a team, we only got one children's interview so far. And I couldn’t be able to get any child to watch the demo video or actually interact with my prototype. I will not continue trying to tackle this issue from now on, because this is beyond my capacity. No matter how hard I try, it still cannot be solved. And it is taken too much energy from me as well.

Alison gave me the suggestion on last Friday that I could add some supplementary information to complement this drawback. The first thing I can do is add more academic research including theoretical frame, principle of children education, and school and family settings. For instance, positive reinforcement to help children learn better and teach/train children in patience. The second thing I can have a look at is the children’s teaching television programme. Seeing how the TV shows teach/introduce concepts to children is also a good way to know more about children’s mindsets under the creative learning theme.

• The final team report is another thing I'm worried about at the moment because it is really hard to work as a team based on previous experiences. It is even difficult to get all the team members together to talk about the assignment.
• Time management for both thesis and the rest delivery of this course is another thing I'm worried about. Since the deadline for the critical reflection essay is the same day with my thesis, I'm a bit worry about if I could finish it with high quality. So I asked Lorna if it is possible to get an extension for the last assignment given this certain situation.

Feedback from the individual prototype evaluation

At the end of this week, I received three feedback from the online individual prototype evaluation. Not as I expected, I didn't manage to get five users to participate. And there's only one user who has teaching experience with children in third grade at a public school in the US. For the general experience, all of the three participants could understand the intended experience the concept is trying to deliver. Some of them showed strong empathy through the changes especially when the “tragedy” happening on the globe. And they are" looking forward to seeing the enhanced version of The Earth Globe in the future. Here are the valuable constructive suggestions from the individual participants.

For the visual aspect, the user hopes the globe could show the environmental changes be more vivid, more like an immersive experience. They would like to see if the best result and the worst result that happens to the globe become more dramatic. This will need to include an extra LED screen or projector to show more visual changes. Although this is a good suggestion, it will not be considered to be added in the next stage since the initial concept is not to use a digital screen in the physical installation. But the issue that the user is having a bit of a hard time telling which was supposed to be the bad bin and which is good has been solved. The big signs for the rubbish bins have been made and added to them to show the user clearer distinguish between them. In terms of hearing, the melody is chosen for the prototype to meet the baseline that understandable for the users. However, they are a bit ambiguous, not clearer enough for some users to distinguish the positive and negative notice by the globe. The exploration and improvement of the sounds used in the final product will be carrying on in the next developing process. Finally for the touch, according to one of the users, the buzz and shaking function will be reconsidered and make adjustments in the next step. One serious issue is that it did not noticeable enough compare to the dropping down and lifting up features. It is cannot be fixed, then this feature will emerge with the other one together.

The Sight Function

The visual effect of LED lights colour changing and shifting received all positive responses from the testers. The dropping down and lifting up the function of the globe also draws enough attention from the user. They did feel an emotional connection with the globe and be reminded that every action counts when it comes to environmental protection.

One of the users says:

“With the music, color, and the lifting and dropping of the globe, it is a very straightforward way of showing whether the earth is doing well. When the globe is “dropping his head”, it feels heartbroken for me.” - Participant 02

The other user also states that:

“The colour change on the globe reminds me of global warming. Since it was triggered by an action, I would associate the action done to the outcome displayed on the globe. I felt like I was curious as to how much “worse” the globe could get and also how much “better” it could get depending on what I do. I think it’s good that eventually, the whole thing turns red indicating an irreversible kind of damage.” - Participant 04

The Hearing Function

The melody be chosen for the prototype meet the base line that understandable for the users. However, they are a bit ambiguous, not clearer enough for some user to distinguish the positive and negative notice by the globe. The exploration and improvement for the sounds used in the final product will be carrying on in the next developing process.

One said:

“Accompanying music would be better if it’s a sigh voice from an older man gradually… you might do some research about how music prompt behaviours change.”- Participant 01

The other user also mentioned:

“I want it to be more fancy! Like a magic ball can see the future or it can talk to me or it can sing a song to me not just music… but with lyrics and easy for me to remember and sing to other kids.” - Participant 03

The Touch function

The buzz and shaking function will be reconsidered and make adjustments in the next step. One serious issue is that it did not noticeable enough compare to the dropping down and lifting up features. It is cannot be fixed, then this feature will emerge with the other one together.

One of the user expresses her concern that:

“…Can make it more smooth, it shakes so horrible…”- Participant 03

One-minute pitch (including the name of your concept):

Version for parents:

Are you having trouble to teach your kids about how to do a better job in recycling the garbage? Do you find it is hard to keep their interest for a longer period so they can develop a new habit to protect the environment? Either text-based or oral introduction is not attractive enough for children to keep them concentrating on the topic. Therefore, we present an alternative solution for teaching children about environmental protection through their everyday life at school. The interactive globe earth will give children a multiple digital sensation interactive experience when they throw away the garbage. Next, the children will get feedback from the earth by how the globe changing in real-time. In this way, children will understand that every decision they made in daily life will have an actual impact on the planet we all living together.

Meeting Reflection

Based on the critiques, Solomon and I conducted a zoom meeting this week to list all the considerable suggestions and allocate the tasks. The main problems are the unclear target audience and no edified pseudocode. Therefore, to get a further understanding of the requirements from users, we plan to have a card sorting to evaluate which kind of pseudocode suits the users most. The brief procedure is that invite the people, who are pro in programming, to sort the cards based on the tasks we designed. If the current pseudocode doesn't match their conceptual model, they can create one for us. From this activity, we can learn the common combination of the blocks and, especially, know which kinds of pseudocode we should apply to the blocks.

Besides, additional research will be located on keeping identifying the age groups of our users. After the meeting, we concluded that although cover all the users with the same programming skill and problems, which they both have negative impressions on learning programming, is effective, the passion for our project is different. For instance, senior people may merely accept robot-based learning. And which age groups have the highest demands of learning programming is one of evaluating standards. Here is our mind map for the future iteration:

Prototype

This week, we tried to combine our prototype but still reminding issues to fix, like the different signal types and input mode varying that need extra works on refining our codes. And we are still struggling with how to give feedback on users' errors. Plan 1 is that as we already knew the precise route, we can detect every input from users and check whether it is correct or false so that the robot can give instant feedback. Whereas this plan breaches our initial wish, we hope users learn the ability of locating the errors themselves based on the visualized codes, which is the process of robot performance. Plan 2 is that the robot has the mark-recognize system, we can place an object with a special mark. Thereby, after the robot finishes all the input commands, it will start the position detection. The problem with this one is the system can not give detailed feedback on the errors. So, maybe combining plan 1 and plan 2, which detects input codes via both physical and code-based checking and record all the specific errors as the feedback. For example, if users tend to place redundant codes but still arrive at the target spots, the system will detect those useless codes and give feedback like there are redundant commands on the X steps.

The map of the system

In our system, the map also represents the different levels of programming. There are some limitations to the function of robots, the system only can realize simple programming syntax like if statement and loop. Therefore, the map can not be over complex and as the robot needs use the camera to find its charger, the map can not be 3 dimensions.

Work Done

This week I started working on getting the text-to-speech functionality working. I was previously working on my windows computer downstairs but wanted to try to work on my mac as it was a lot more convenient. However, I had a lot of difficulties getting all the settings right and wasted a lot of time trying to get it talking to the board. I ended up giving up and just working downstairs. When I moved to working downstairs again, because my connections between all the sensors and the board were very dodgy, my program stopped working. I decided it would be more beneficial to focus this week on getting all the components soldered so that I could be sure that any functionalities that weren't working was the code instead of not knowing what was going wrong.

This week I also started going through some of the feedback I received on my prototype. It was mostly really positive and everyone seems to enjoy Emily's singing as much as I do! A common theme among all feedback was about the logic used to determine the appropriate temperatures. I have been putting off doing this properly as I'm not quite sure how to categorise the temperatures by just comparing the indoor and outdoor temperature with no other information. I think I should add buttons to simulate turning on and off the air-conditioner and heater so that Emily will have more information. This way I can have different logic for when both are off, when the heater is on, and when the air conditioner is on. I think this will make a lot more sense when I am explaining the logic too. One of the teams also linked a research paper about using smart homes that share data between all the devices to take control of the home, based on this information. I thought this was a really interesting perspective, however, I think in order to encourage the long term behaviour change I want, Emily needs to alert the user at the time of the bad behaviour rather than just make the changes herself, even if that is more efficient. Although it may be interesting to think about using this type of idea to allow Emily to take control if the user is out of the house and accidentally left an appliance on.

Work to do

Now that I have moved to the soldered veroboard, I can focus on getting the text-to-speech working. I have bought the amplifier and speaker so I just need to get it all talking together.

Related Work

Paper about smart homes discussed above: K.-H. N. Bui, I. E. Agbehadji, R. Millham, D. Camacho and J. J. Jung, “Distributed artificial bee colony approach for connected appliances in smart home energy management system,” National Research Foundation of Korea, 2020.

Report back

In this week's studio, we talked about the following questions.

What is one good thing you can say about lockdown?

I think lockdown gave us a chance to get used to those tools that we barely used before. Few months ago I would never think about using Zoom and Miro to engage in this course and cooperate with my teammates, so it is good to have a new form of course delivery. Also, it pushes us to adapt to complex cases, find alternative approaches while encountering difficulties. Overall it is a memorable experience during this semester.

Based on feedback to date & what you have left to complete: What are your key priorities?

The key priority is to combine each teammate's components in the following weeks into a single installation, as we are working collaboratively as a team. Also, we are considering to add more features to existing prototypes, based on the useful feedback.

What concerns/questions do you have? (doesn't have to be directly project related)

At this moment, because the pressure sensor I have is RP-C18.3-ST Thin Film Pressure Sensor. It is a 0.4mm thin flexible sensor with low response time (<10ms) that can be contained in various physical things. The only concern is that the pressure measuring range is from 20g to 6kg. So that the pressure sensor can not hold the weight of an adult. We need to figure out some alternative approaches in the final design, either using other ways to detect pressure value or use some structural designs to place the sensor properly so that it won't hold the entire weight of an adult.

Workshop

About the pressure sensor's concern, we got some useful advice from our tutor. One is using bathroom scale to track weight and send the data back to computer, we got some tutorials online for consideration. But lack of coding examples to get started. The other is to get a second-hand Wii Balance Board, transfer the data via Bluetooth, the scale really only captures your weight and sends it up to InitialState for tracking and feedback. This sounds more easy for us so we are looking to purchase one.

Also, we brought the materials we purchased from bunnings to uni and start constructing a panel that can allow people to stand on, the supportive structure is still needed to be considered, and the proper way to combine the Wii board. That's our next week's focus point.

Contact & Workshop

For the contact session, Lorna first explained to us what the final delivery was going to be and what should we prepare for the final exhibition. Then we started the report-back to express our thoughts and concerns in terms of the lockdown and how to complete the final delivery, my responses were:

1. What is one good thing you can say about lockdown?

For one advantage about the lockdown, I’d say it saves time. I don’t need to spend time on commuting, I can set aside the time to do other things, like working on my projects, taking a nap or a short break.

2. Based on feedback to date & what you have left to complete:

The feedback I received so far is not bad since peers and evaluation participants have given positive feedback in terms of the physical form, interaction means, and functionality. There are also some suggestions for further improvement. For example, considering using the LED strip or adding the faded effect to the single LED light to give users the continuous visual feedback, rather than lighting up a LED every 2 seconds. Also, using the laptop speaker to give auditory feedback will be a good alternative, because it can provide more polished audio. In addition, adding decorations to the ‘breathing tree’ will give a better experience of my prototype.

3. What concerns/questions do you have? (doesn't have to be directly project related)

One concern is about the final exhibition, I’m just wondering how to take my ‘breathing tree’ (the clothes stand) to campus since I am planning to do the exhibition on campus for ensuring a better quality of the live stream. Also, I am concerned about how the virtual exhibition is going to be, something like others talk with me first, and then I show the functionality and interaction behaviours of my concept?

In the workshop session, Clay and Alison guided us to practice the one-minute pitch for the final exhibition. The key feedback here was that when preparing the pitch to communicate our ideas, we should consider how to quickly and compellingly introduce our ideas to clients. My response was:

One-minute pitch (including the name of your concept):

Are you still struggling with the lack of breath when singing? Let’s try the ‘Breathing Tree’! It’s a tree-shaped device can help youngsters practice the breathing technique used for singing. It aims to help young singing-lovers practice abdominal breathing technique in an intuitive and playful way, so the primary goal is guiding users to practice through multi-sensory engaging feedback. It will provide both auditory and visual feedback by detecting users’ abdomen movement and their breathing length respectively. By using this device, you will get rid of practicing the breathing technique in the tedious and less intuitive way, instead, it will give you playful instructions to practice that in the correct and interesting manner.

Team progress

This week, our team shared our opinions on the final delivery and how to better design for our web-based portfolio. After discussing with team members, one useful framework I may consider using is jQuery. Besides, since almost all of us are using the microphone sensor for our prototypes, Paula ordered 3 extra ones as the spares for some unexpected damage of the microphone sensors. So, Wally and I also picked up our sensors this week from Paula.

Individual process:

This week, I mainly worked on improving and building my prototype according to the previous feedback, and purchasing materials.

First, I went to the workshop for getting technical help around making the microphone of my concept from Clay on Friday. We first got the microphone disassembled, so that to put the microphone sensor in it. Clay also soldered one long wire for me to connect the microphone and Arduino.

After a few times attempts, the microphone part worked for controlling the gradually lighting up effect of LEDs.

For the ‘Belt’ part of my concept, the solder of the force sensor was broken, so Clay soldered that again and added a protective cover to the connection between the sensor and wires. Much appreciated for Clay’s help!

Previously, users thought adding some decorations to the ‘Breathing Tree’ part would make the prototype look better, so I bought some plant strips for decoration to give users better visual experience.

Plan for the next week:

1. Completing the audio part of the ‘Belt’ part
2. Adding decorations to the ‘Breathing Tree’ part
3. Finishing the wiring part of the ‘Breathing Tree’ part

Reflections:

In terms of preparing the pitch for ‘selling’ my concept in the final exhibition, it’s important to grab the audience’s attention and give them a panoramic view of my concept within a very short time. Therefore, I think the whole pitch can start with describing the ‘problem’ I’m trying to solve (what), followed by giving the rationale of my design (why) and then talking about what’s unique of my concept (selling point). During the pitch, it’s important to let the audience know the USP (unique selling proposition)so that they will be more attracted by the content.

For the visual part of my concept, using some light effect is effective to give users continuous feedback. But in order to give users more obvious feedback and to make my prototype more applicable in different scenarios (like in daytime), using the LED strip to give brighter feedback would be a good direction for exploring.

Nǐn hǎo,

This week I started making version 2 of the prototype, which contained a few added quirks to the robot's face. The new changes were the features I touched on last week. An eyebrow and new smiling implementation were used alongside potentiometer to control robot's expressions. It is just a simple build that tested out the new neopixel strips I acquired last week.

As can be seen, a potentiometer acts as volume control for robot. Turning the potentiometer in either direction controls the mood of robot which is reflected through smile and eyebrow.

In addition to simply implementing previous functionality, I have also added the leds turning on one by one. The purpose here was to support the transition from happy smile to frown as user lower's the volume.

In Arduino, very simple code is used, which reads the potentiometer and depending upon it's range, lights up the led. Something to implement in future would be a variable which holds the anger of robot and increases once a transition have been made from smile to frown.

To do -

The team is going to meet on Tuesday and we are going to try and put the prototype together. Before that I would try to include other sensors as well. Additionally, I would try to include code for each part so that every part works together.