Exhibition

This Wednesday is the time of the online exhibition. We booked a library room with a projector as the display location in advance. We entered the site for one hour in advance to debug the project to ensure that everything worked properly. It was our advance inspection that found some unstable connections and made corresponding adjustments. The following pictures are the photos in the exhibition.

The entire exhibition lasted for three hours, during which we accepted the review of the classmates and the teaching team. We also communicated with other groups through discord during this period. On the whole, the online demo was smooth, and the idea was well communicated.

Live video demonstration

Overall reflection

Overall, I am satisfied with the final result of our group. Despite being affected by the epidemic, learning and life have become less convenient than usual, our team still completed the project as much as possible and carried out the final improvement before the final exhibition, and in the final online exhibition, the students and teachers who watched our products in our live broadcast room also gave positive comments. I will complete my reflection on the following aspects.

'Actual' to'Ideal'

Our team develops the project in the form of teamwork. Although we have realized the main functions, the actual product still deviates from the expected effect. Most of the deviations are mainly due to technical reasons, as well as the emergence of uncontrollable factors caused by the impact of the epidemic. Through three main iterations, we have designed a device that allows users to better interact with daily sounds, and also completed the realization of the main functions. Reviewing the overall design ideas, our products still have the following defects:

1. In our original plan, the dropper was used as a radio device. It is wire-free, and users can take it to record their favorite sounds anywhere. Although we have connected an extension cord, it can be used in a limited range, but it still cannot provide users with a more intelligent experience. Similarly, due to the installation of many sensors, there are still many wires connected to the rest of the project. Although we tried many ways to hide the wires, it was still unavoidable that the nakedness caused the visual unsightly. How to beautify products to a greater extent, as well as intelligent products will be the goal and improvement direction of future research.
2. Unity has not successfully connected with Arduino and python. In the original plan, we planned to create visual effects for the sound playback, but unfortunately, we have no way to connect it with python and Arduino. Although the generated visual effects are very cool, at present, it is only possible to display the visual effects by manually inputting the path. Overcoming this technical barrier is also the direction of future efforts.
3. I did not find a suitable sensor to make the sound deletion function. In our original plan, there was the concept of a trash can. As soon as the instrument is dumped over the trash can, the sound will be deleted. At present, this part does not think of a suitable technical implementation method but uses other methods to achieve the deletion effect of music.
4. The product does not completely get rid of the computer and achieves a highly intelligent effect. In our original plan to use the raspberry pi to replace the computer, and because of time constraints and limited materials on hand, we still use the computer to complete the relevant functions.

Relevance to course

The course theme is mainly about novel physical interactions. Our products are highly consistent with the course theme. Unlike traditional interaction methods (such as keyboard and mouse interaction), our products simulate the operation of the chemical laboratory, providing users with a variety of novel interaction forms such as squeezing, touching, dumping, shaking, etc. In order to give users a better operating experience.

Relevance to studio

Studio theme refers to an interesting, highly playable, innovative feature that can produce music for daily life. Our products meet the above requirements.

First, the combination of music synthesis and chemistry experiments is a very new concept. The user also showed his curiosity when he saw the device. The design idea is very attractive.

Second, our project is very close to life. In an ideal situation, people can take a dropper to collect various sounds in daily life, and take them home to interact with daily sounds. The place where the sound is mixed can also be more commonplace in daily life such as homes, companies, etc. The visual effects and auditory effects produced by playing sounds can play a good role in soothing emotions.

Third, the product is highly playable and interactive, and also gives users a high degree of freedom. Users can not only record any sound they like but also mix their favorite music clips. When using the device, the user can not only interact with the device but also enjoy a collaborative experience with friends.

Reflect on Human value

As an interactive device for daily sound processing, the "Sound lab" can make music more into people's lives. Users can alleviate the tension and anxiety of life through the interaction with daily sounds. The product is simple to operate and has a high degree of freedom. Users can "mix" their own DIY music according to their favorite, which adds a lot of fun to the daily boring life. From the feedback of user tests, they believe that the device can greatly reduce the stress of life, and they are more attentive to the voice of life.

Relect on User experience

In the stage of final product display, users can quickly know the concept of our project through the introduction, and can quickly get started. They think that the method of using a combination of visual effects and sound is very novel, and it is very unique to use daily sound as an entry point. Users think that in this way, everyone will pay more attention to the surrounding life and also play a good role in relieving stress. In addition, users can complete the mixing operation without instructions. They also appreciated the different visual effects we designed. I can think of our products as suitable for user experience.

Thanks

Due to the impact of the epidemic, this semester is not easy. Thanks to the teaching team for our entire project.

This is my last journal. If I graduate successfully, the completion of this project will also draw a perfect end to my student career. Although there are doubts, uncertainties, and stagnation in the completion process, thank you for my team members and everyone, we finally completed our original idea!

Exhibition

This Wednesday is the time of the online exhibition. We booked a library room with a projector as the display location in advance. We entered the site for one hour in advance to debug the project to ensure that everything worked properly. It was our advance inspection that found some unstable connections and made corresponding adjustments. The following pictures are the photos in the exhibition.

The entire exhibition lasted for three hours, during which we accepted the review of the classmates and the teaching team. We also communicated with other groups through discord during this period. On the whole, the online demo was smooth, and the idea was well communicated.

Live video demonstration

Overall reflection

Overall, I am satisfied with the final result of our group. Despite being affected by the epidemic, learning and life have become less convenient than usual, our team still completed the project as much as possible and carried out the final improvement before the final exhibition, and in the final online exhibition, the students and teachers who watched our products in our live broadcast room also gave positive comments. I will complete my reflection on the following aspects.

'Actual' to'Ideal'

Our team develops the project in the form of teamwork. Although we have realized the main functions, the actual product still deviates from the expected effect. Most of the deviations are mainly due to technical reasons, as well as the emergence of uncontrollable factors caused by the impact of the epidemic. Through three main iterations, we have designed a device that allows users to better interact with daily sounds, and also completed the realization of the main functions. Reviewing the overall design ideas, our products still have the following defects:

1. In our original plan, the dropper was used as a radio device. It is wire-free, and users can take it to record their favorite sounds anywhere. Although we have connected an extension cord, it can be used in a limited range, but it still cannot provide users with a more intelligent experience. Similarly, due to the installation of many sensors, there are still many wires connected to the rest of the project. Although we tried many ways to hide the wires, it was still unavoidable that the nakedness caused the visual unsightly. How to beautify products to a greater extent, as well as intelligent products will be the goal and improvement direction of future research.
2. Unity has not successfully connected with Arduino and python. In the original plan, we planned to create visual effects for the sound playback, but unfortunately, we have no way to connect it with python and Arduino. Although the generated visual effects are very cool, at present, it is only possible to display the visual effects by manually inputting the path. Overcoming this technical barrier is also the direction of future efforts.
3. I did not find a suitable sensor to make the sound deletion function. In our original plan, there was the concept of a trash can. As soon as the instrument is dumped over the trash can, the sound will be deleted. At present, this part does not think of a suitable technical implementation method but uses other methods to achieve the deletion effect of music.
4. The product does not completely get rid of the computer and achieves a highly intelligent effect. In our original plan to use the raspberry pi to replace the computer, and because of time constraints and limited materials on hand, we still use the computer to complete the relevant functions.

Relevance to course

The course theme is mainly about novel physical interactions. Our products are highly consistent with the course theme. Unlike traditional interaction methods (such as keyboard and mouse interaction), our products simulate the operation of the chemical laboratory, providing users with a variety of novel interaction forms such as squeezing, touching, dumping, shaking, etc. In order to give users a better operating experience.

Relevance to studio

Studio theme refers to an interesting, highly playable, innovative feature that can produce music for daily life. Our products meet the above requirements.

First, the combination of music synthesis and chemistry experiments is a very new concept. The user also showed his curiosity when he saw the device. The design idea is very attractive.

Second, our project is very close to life. In an ideal situation, people can take a dropper to collect various sounds in daily life, and take them home to interact with daily sounds. The place where the sound is mixed can also be more commonplace in daily life such as homes, companies, etc. The visual effects and auditory effects produced by playing sounds can play a good role in soothing emotions.

Third, the product is highly playable and interactive, and also gives users a high degree of freedom. Users can not only record any sound they like but also mix their favorite music clips. When using the device, the user can not only interact with the device but also enjoy a collaborative experience with friends.

Reflect on Human value

As an interactive device for daily sound processing, the "Sound lab" can make music more into people's lives. Users can alleviate the tension and anxiety of life through the interaction with daily sounds. The product is simple to operate and has a high degree of freedom. Users can "mix" their own DIY music according to their favorite, which adds a lot of fun to the daily boring life. From the feedback of user tests, they believe that the device can greatly reduce the stress of life, and they are more attentive to the voice of life.

Relect on User experience

In the stage of final product display, users can quickly know the concept of our project through the introduction, and can quickly get started. They think that the method of using a combination of visual effects and sound is very novel, and it is very unique to use daily sound as an entry point. Users think that in this way, everyone will pay more attention to the surrounding life and also play a good role in relieving stress. In addition, users can complete the mixing operation without instructions. They also appreciated the different visual effects we designed. I can think of our products as suitable for user experience.

Thanks

Due to the impact of the epidemic, this semester is not easy. Thanks to the teaching team for our entire project.

This is my last journal. If I graduate successfully, the completion of this project will also draw a perfect end to my student career. Although there are doubts, uncertainties, and stagnation in the completion process, thank you for my team members and everyone, we finally completed our original idea!

Exhibition

This Wednesday is the time of the online exhibition. We booked a library room with a projector as the display location in advance. We entered the site for one hour in advance to debug the project to ensure that everything worked properly. It was our advance inspection that found some unstable connections and made corresponding adjustments. The following pictures are the photos in the exhibition.

The entire exhibition lasted for three hours, during which we accepted the review of the classmates and the teaching team. We also communicated with other groups through discord during this period. On the whole, the online demo was smooth, and the idea was well communicated.

Live video demonstration

Overall reflection

Overall, I am satisfied with the final result of our group. Despite being affected by the epidemic, learning and life have become less convenient than usual, our team still completed the project as much as possible and carried out the final improvement before the final exhibition, and in the final online exhibition, the students and teachers who watched our products in our live broadcast room also gave positive comments. I will complete my reflection on the following aspects.

'Actual' to'Ideal'

Our team develops the project in the form of teamwork. Although we have realized the main functions, the actual product still deviates from the expected effect. Most of the deviations are mainly due to technical reasons, as well as the emergence of uncontrollable factors caused by the impact of the epidemic. Through three main iterations, we have designed a device that allows users to better interact with daily sounds, and also completed the realization of the main functions. Reviewing the overall design ideas, our products still have the following defects:

1. In our original plan, the dropper was used as a radio device. It is wire-free, and users can take it to record their favorite sounds anywhere. Although we have connected an extension cord, it can be used in a limited range, but it still cannot provide users with a more intelligent experience. Similarly, due to the installation of many sensors, there are still many wires connected to the rest of the project. Although we tried many ways to hide the wires, it was still unavoidable that the nakedness caused the visual unsightly. How to beautify products to a greater extent, as well as intelligent products will be the goal and improvement direction of future research.
2. Unity has not successfully connected with Arduino and python. In the original plan, we planned to create visual effects for the sound playback, but unfortunately, we have no way to connect it with python and Arduino. Although the generated visual effects are very cool, at present, it is only possible to display the visual effects by manually inputting the path. Overcoming this technical barrier is also the direction of future efforts.
3. I did not find a suitable sensor to make the sound deletion function. In our original plan, there was the concept of a trash can. As soon as the instrument is dumped over the trash can, the sound will be deleted. At present, this part does not think of a suitable technical implementation method but uses other methods to achieve the deletion effect of music.
4. The product does not completely get rid of the computer and achieves a highly intelligent effect. In our original plan to use the raspberry pi to replace the computer, and because of time constraints and limited materials on hand, we still use the computer to complete the relevant functions.

Relevance to course

The course theme is mainly about novel physical interactions. Our products are highly consistent with the course theme. Unlike traditional interaction methods (such as keyboard and mouse interaction), our products simulate the operation of the chemical laboratory, providing users with a variety of novel interaction forms such as squeezing, touching, dumping, shaking, etc. In order to give users a better operating experience.

Relevance to studio

Studio theme refers to an interesting, highly playable, innovative feature that can produce music for daily life. Our products meet the above requirements.

First, the combination of music synthesis and chemistry experiments is a very new concept. The user also showed his curiosity when he saw the device. The design idea is very attractive.

Second, our project is very close to life. In an ideal situation, people can take a dropper to collect various sounds in daily life, and take them home to interact with daily sounds. The place where the sound is mixed can also be more commonplace in daily life such as homes, companies, etc. The visual effects and auditory effects produced by playing sounds can play a good role in soothing emotions.

Third, the product is highly playable and interactive, and also gives users a high degree of freedom. Users can not only record any sound they like but also mix their favorite music clips. When using the device, the user can not only interact with the device but also enjoy a collaborative experience with friends.

Reflect on Human value

As an interactive device for daily sound processing, the "Sound lab" can make music more into people's lives. Users can alleviate the tension and anxiety of life through the interaction with daily sounds. The product is simple to operate and has a high degree of freedom. Users can "mix" their own DIY music according to their favorite, which adds a lot of fun to the daily boring life. From the feedback of user tests, they believe that the device can greatly reduce the stress of life, and they are more attentive to the voice of life.

Relect on User experience

In the stage of final product display, users can quickly know the concept of our project through the introduction, and can quickly get started. They think that the method of using a combination of visual effects and sound is very novel, and it is very unique to use daily sound as an entry point. Users think that in this way, everyone will pay more attention to the surrounding life and also play a good role in relieving stress. In addition, users can complete the mixing operation without instructions. They also appreciated the different visual effects we designed. I can think of our products as suitable for user experience.

Thanks

Due to the impact of the epidemic, this semester is not easy. Thanks to the teaching team for our entire project.

This is my last journal. If I graduate successfully, the completion of this project will also draw a perfect end to my student career. Although there are doubts, uncertainties, and stagnation in the completion process, thank you for my team members and everyone, we finally completed our original idea!

studio session

This week is the last week of the studio session, and the time in this semester seems to be passing by very fast, and I have only one week left to work on our project and show it online. The last week of the studio focused more on the reflective of our own projects and plans for online exhibitions. In the report back session, I shared with my classmates about the recent changes in life caused by the relaxation of the ban. Due to the relaxation of the current social ban, my team members and I will go to the UQ innovate to do some group activities about soldering and project assembly.

In the report back stage, we also discussed how to change the project if I have magic. This can't help me to reflect on the whole project of our team. The most needed change in our group project is to make the entire device wire-free. The wire-free here not only refers to removing the messy jumper wire connecting each sensor, but the most important thing is also the design of the dropper part, our ideal is to use the dropper to record any sound users like, that is to say, users can take the dropper to everywhere for sound collection. Although we used extension cords in the design of the project to expand the range of recorded sounds, the sound recording will still be limited because the extension cords are fixed. If there is no wire restriction, the product will be more popular with users. Another thing I hope to solve with magic is the visual effects part of my design. Despite trying many methods, but still can not connect Arduino with python and unity. This caused that in the future exhibition, we need to manually enter the path to complete the display of the visual part.

Since the online exhibition is relatively unknown, our group conducted a group meeting on how to present the project after the report back. Our main concern is how to better communicate our design ideas through online platforms and how to allow users to interact with our products. For this reason, I also seek help from the teaching team in class, based on the discussion results, we decided to act as a ‘robot’ in the online exhibition, that is, we only give a brief introduction to the project, all other operations will be given by the user, maximize the distance between users and products. We all looking forward to next week's online exhibition...

project progress

This week is very busy because it is at the end of the semester, not only do I have to prepare for the exhibition of the next week, but also I will have 4000-word report to submit. This makes me feel overloaded. Fortunately, the main functions of our entire project have been basically completed. Our main work this week is to integrate various parts and think about how to achieve the visual-pleasing design.

Since the chemical equipment toys we bought for the project before are very small in size, we can't even have spaces for the installation of all sensors, especially vibration sensors, so we purchased large glass laboratory equipment instead. However, although the large glass-based experimental equipment can put all the sensors inside, because these types of equipment are transparent, the sensors installed in the equipment can be seen clearly, which is not visually beautiful, so I tried to use white acrylic to paint on the equipment. On the one hand, let all equipment reduce transparency to hide the sensors and wires that were originally exposed. On the other hand, make the lighting effect on the equipment more obvious. The colored equipment can well hide sensors and messy jumper wires and can better display the led inside, but we found that the colored equipment is a bit different from the chemical equipment. To this end, we use a marker to draw a symbolic mark, so that it is visually more like a set of laboratory equipment.

At the same time, I made a test tube rack according to the size of the test tube. I chose a larger cardboard box in order to hide the development board and breadboard in order to prevent users from seeing the messy wires to achieve a good visual effect. The first two pictures are the status when we have just integrated our respective parts together. Due to the installation of many sensors, the line is particularly messy. We use electric tape to wrap up the messy wires of different parts and hide the excess wires in the carton. The whole work becomes neat.

Major changes in my group members' respective parts

Iris is responsible for designing the dropper part in our whole project. The main change in her part of the final delivery project is the design of the dropper. In order to better hide the switch and achieve a good "squeeze" interactive effect, we are replacing the material of the dropper. The picture below is what the final dropper looks like.

The role of dropper: sound recording

The function currently implemented: squeeze the plastic head and place the dropper close to the sound source to record.

Rhea is responsible for designing the sound mixing part in the group. The main change in her part of the final delivery project is the design of the visual effect of the sound mixing as well as adding the function to interact with friends. The interactive device in the picture below is composed of two handles. The user can invite his friends to comment on the remix he made. Shaking the smiling face means like, the system will automatically play a clapping sound, shaking the crying face means dislike, the system will play a sad sound effect and then the mix will be deleted automatically. In the process of mixing, close the lid and playback will stop automatically....

Overall Product after integration

Portfolio

At present, I have completed the text part and updated the corresponding pictures and text to my portfolio. My portfolio can be found at UQ Interaction Design Exhibit 2020. URL is as follows:

Sophia Jin's portfolio link

prepare for the exhibition

Next Wednesday is the online exhibition, we will conduct another project integration and confirm the presentation form next week.

We are going to shoot live video and conduct a group test to confirm that the project can be carried out smoothly.

This week I mainly improved my part of work (test-tube part) based on user feedback from last week combined with my research on sound visualization. At the same time, I set a framework for my portfolio and started to conceive the components of the portfolio.

Studio session

In the online studio stage, Lorna gave us a brief introduction to the requirements of the portfolio, this inspires me in terms of the design of the concept section of my portfolio in the future. During the studio session, I realized that there are only two weeks away from the exhibition. The time is relatively tight. I must arrange the time reasonably in the next two weeks in order to have a good user experience in the online exhibition.

At the moment I working on using unity to make visual effects of playing sounds. In the previous week of the exhibition, our team plan to work together to beautify the previous prototype, and reorganize the parts to ensure that each part can be used normally. Although the time is a bit tight, we still need to work hard to guarantee the project to be completed step by step as planned.

Due to the impact of the epidemic, this year's exhibition was changed to online, and we will use discord as a tool to interact with other groups and visitors online. In this week's studio, we are also familiar with the use of this software to prepare for the subsequent online exhibition.

Project progress

This week our team decided to improve our own parts, and we will carry out unified project integration next week.

This week, I referred to the tutorial video I found last week about using unity to visualize the sound and made a small program to increase the visual effect of the sound playback part of the group project.

Through this week's production, the unity program can generate different waveform diagrams according to different audios of different sounds.

The visual effect achieved so far is shown below:

Visual effects when no sound is playing:

Visual effects when playing low-frequency music:

Visual effects when playing high-frequency music:

This week I tried to connect this unity project with our overall project (that is link unity, Arduino and python together) to add this visual effect to the delivery of the entire project, but due to technical limitation, the complete connection has not been achieved. Through the discussion with the teammates, we decided to retain the main function of the overall project, consider whether it is possible to present this visual effect from other manual methods. As an alternative to displaying this part, I designed an input box on the interface for manually inputting the path of the sound to be played. If the technical barriers are not overcome in the next few weeks, we will present this visual effect separately by manually filling in the audio path.

I will also briefly describe the implementation process here:

This effect uses a plug-in: audiovisualizer, which allows users to produce different visual effects based on different sound files.

The input box of this interface is used to accept user input, store the input string as a path, click the button to call its bound event, play audio, and generate visual effects. The input detection uses the inoutfile component to monitor the editing events, and when the input is completed, the string is obtained and stored in the path.

The following figures are some screenshots for the code parts for the unity project.

Portfolio

Through this week's introduction in the studio session, I plan to start thinking about the components of my portfolio and the parts I want to cover in my portfolio. I plan to refer to excellent open-source portfolio templates and make changes according to the needs of my content.

My intention is to make a one-page responsive portfolio that can adjust the different ratios of content according to the different platforms visited. When I browsed through last year’s portfolio, I found that the pleasing portfolio is often using a combination of pictures, text, and video to introduce the product and convey the ideas. Although pure text can express the design ideas clearly, with the help of pictures, videos can make the content more intuitive and won't feel boring while. This week I confirmed the sections displayed in my portfolio and made a determination of the overall framework.

Next, I need to fill the content and finally make a small modification about the overall layout according to the content and can complete the production of the personal portfolio.

The logo

Future Plan

1. Work as a team to connect the entire project together
2. Focus on considerations related to aesthetic design
3. Start with the text part of the portfolio

Report back

In this week's studio session, Lorna gave us a brief introduction to the final delivery. Through the introduction, I am pretty clear about what needs to be done in the following weeks. After that, we started on the report back session on the following questions:

What is one good thing you can say about lockdown?

Personally speaking, I am not used to the life of lockdown and don't like to stay at home for a long time. For the learning environment, I still feel that the school's academic atmosphere can help me improve my learning efficiency. Through face-to-face communication with the tutor, I think that I can get the answers I am confused about faster... However, the lockdown does help me to save some time and money, cuz I don't need to catch the bus to school, instead, I just need to attend the online zoom meeting to get access to the class and get in-time help from the online environment.

Based on feedback to date & what you have left to complete?

From the team appraisal, I have gained lots of useful feedback related to my personal work. Our team worked in the form of cooperation, and my personal focus is completing the playback and transmission function(that is the test tube part) in the overall project. In last week's journal, I've made a reflection on the feedback from other teams and decided to make improvements in the following areas.

In the aesthetic part, 1) the brightness and darkness of the LED lights are not obvious enough to play a role in telling the current status of the system, I would like to use RGB light strip to take the place of the LED light installed in the test tube. 2) Because there are lots of sensors installed on the test tube and will use lots of wires to link them together. Although in the previous week, I find out a way to use the built-in resistance of the Arduino development board (internal pull-up resistor)to get rid of the resistors, the overall design of the test tube is still a little bit messy and I am still working for how to hide these wires as much as possible to achieve visual simplicity.

For the functional aspect, the current work is not fully functioned and I will complete the coding part in the backend as well as complete the installation of all the work performed by my group members to group together and complete the overall product.

At present, we use python (pyaudio module) to complete the recording and playback of the project. The realization of the mixing function of our project is achieved by playing the selected music files at the same time. In order to ensure that the mixing time is the same, we have fixed the recording time to 10s, that is, whenever the user starts recording, this recording time is the fixed 10s, and there is still no better way to make the mixing part more user-friendly. In the following week, I would like to conduct more user research about the time for recording and gain some insights for the completion of the overall functions.

The key priority for the project is to make the project become a fully-functioned and visually pleasing product.

What concerns/questions do you have?

My major concern is about time management, there are still plenty of works need to be need before the final exhibition. Also, at the end of the semester, when other courses have reached the busiest time, how to use the time to complete the project to ensure the quality of the completion is an important issue that I need to consider.

For the presentation of the exhibition, due to the particularity of the period, we will open online exhibitions. It will have some challenges in conveying concepts to the participants without letting them try our product by themselves. How to organize the final exhibition is also a problem need to consider.

For the technical concerns, I have mentioned in the previous question, that is a major concern is the realization of the mixing function, how can this function become more user-friendly and how to make the process more smooth.

One-minute pitch (including the name of your concept)

Have you noticed the various voices in daily life? For example, the life-like sound, such as the sound of vehicles passing by, the sound of crowds, the rising and falling of birds in the morning, the delicate rhythm of the wind blowing the leaves, or the crisp sound of the rain on the ground in rainy days and etc. When you start to pay attention to the sounds around you, you will find that there are many beautiful sounds in everyday life waiting for us to discover. The sound lab is a sound processing interactive device that can help people to interact with everyday sounds. By using this device, people can create their own music by the sound collected in daily life.

Project Progress

This week we went to jaycar to buy an Arduino board(Mega) with more interfaces, as well as purchased multiple vibration sensors (used to achieve the interactive effect of my test tube part). We also bought a lot of jumper cables for extension cables.

Since jaycar does not have the vibration sensor we planned to buy, we have selected sensors in the store similar to those previously planned.(Linker Vibration Sensor Module for Arduino)

The shock sensor is similar to the DFRobot digital vibration sensor, and is a digital plug-and-play sensor module. It can detect the vibration signal, and then output the switch signal to the Arduino. The module uses the PH2.0 interface, which can be easily connected to the sensor expansion board using a digital cable. It can perceive weak vibration signals and can realize interactive works related to vibration. I used this sensor to realize the interactive effect of starting the vibration test tube playback. The following figure is the connection diagram of the sensor.

The following picture is the code I used to test whether the sensor can detect vibration.

Test equipment: a vibration sensor, a led light. The led is connected to the digital signal port 13 and the vibration sensor is connected to the digital signal port 3.

Test result: According to the connection in the figure above, after plugging in the LED small lamp, the small lamp will light up briefly every time the digital vibration sensor is vibrated. So this sensor can meet the interactive effect I want to achieve, I will put it inside the test tube.

In addition, I made a related investigation on the playback part of the project. According to the results of previous user feedback, they generally believe that I can combine the sound with cool visual effects so as to make people more immersive in everyday sound. To this end, I conducted targeted research in this area.

I have conducted in-depth research on the concept of music visualization, and feel that it can be used in our project.

Music visualization is based on the core of vision, music as the carrier, and the public as the object of the appeal. With the help of a variety of new media technologies and other communication media, the music content is interpreted through pictures and images, and a combination of audio-visual mass communication. It is a technology that can provide an intuitive visual presentation for understanding, analyzing, and comparing the expressiveness and internal and external structures of musical art forms. With the development of new media art and the continuous improvement of technology, music visualization provides many possibilities for people to enjoy music and understand music and can give people a strong visual impact.

With this background knowledge as a backing, I feel that the combination of daily sound and visual elements can make the product more interactive and better meet user needs to achieve the purpose of user-centered design.

I intend to use the waveform diagram as an inspiration for visual design. In the playback part of the sound, I plan to add the following design:

I plan to add visual effects to the sound generation part of the design of the entire project, that is, when the user uses this device, whenever there is sound, the waveform on the screen will present different visual effects according to the frequency of the audio. And I found Unity can help me realize this function, there is a plugin in unity called"audiovisualizer". It can convert audio into waveforms to achieve the desired effect.

I followed the instructional video below to make my unity project:

Future work

1. Continue to improve the work based on feedback
2. Using Unity to combine sound with cool visual effects
3. Planning the website for the project(plan the structure of the portfolio)
4. Start documenting the related context for project description.

Team Appraisal

In this week's studio session, we have conducted the team appraisal session. In this session, we are required to work as a team to critique the work done by the group members from 3 groups. In the task assignment phase, we were arranged to comment on the work of team X, team Hedgehog and team E.M.S. I have gained a lot at this stage. Due to the impact of the epidemic, user testing is difficult for the entire project. Although we have thought of many methods such as using questionnaires and zoom meetings to obtain user data for project iteration, the obtained data is still not enough. I can be explained the whole concept in the form of video demonstration and document supplements to allow classmates to understand the whole picture of the project and obtain more user data so as to gain more inspirations for project refinement and reach the user-centered design. At the same time, I can also reflect on my own projects by watching the work of other students, so as to get inspiration for improving different feedback.

The following reflections were drawn when I watched the works of other groups

Team X

The team completes the entire project in the form of teamwork, and they share the same domain as ours, which is music metric. They chose to combine music and sports to make sports less boring. One of the highlights in their design is not only to provide users with sound feedback but also a series of visual feedback (such as the use of light bars to present different visual effects according to different pressures), which is something we can learn from. In the project, less consideration was given to visual design, and I put more emphasis on the realization of functions, so although the whole project realized some of the functions that I wanted to achieve, the work was not so beautiful. In addition, the combination of sound and cool visual effects allows users to have a better immersive experience, thereby interacting with music better to achieve a user-centric design. By watching the idea of the group's projects, I decided to work hard on the visual effects in future improvements and think about how to present daily sounds more interestingly.

Team Hedgehog

This team also works collaboratively to complete a whole project and I really like their design idea of helping visually impaired people improve their hearing through games. Because the domain of the project is not the same as ours, we have little inspiration for our project in terms of inspiration and technology, but the team members among them gave me some inspiration for the construction of the 3d music scene. Our project is also highly related to music, and I am currently looking for suitable methods to make the entire product more playable and interactive. In the part related to music playback, I can consider using Unity to create a visual-pleasing music playback environment, so that users can get a more immersive experience.

Team E.M.S

This team works for the domain of enhancing mundane space. Their aim is to turn the boring housework activities into a project that the whole family is happy to participate together in the form of hitting games. This not only increases the playfulness of housework activities but also ensures the collaborative experience between the family members. This project makes me think of the collaborative experience in our project. At the first prototype proposal, we received some feedback of whether the product can be operated by multiple people. Since we haven't thought of a suitable method to make users interact with our sound lab with multiple people, this part of the idea is a bit stagnant. Their idea was to enable more people to participate in the form of a batting game, which also gave me some inspiration. In addition to the sound collection stage, which allows multiple people to use a dropper to collect sounds, whether it is also possible to design some corresponding interactive games during the sound mixing stage to make this product more interactive. This is the focus of work I am looking forward to working with in the following week.

Feedback & reflection

I was responsible for designing the test tube part in our group, and I received the comments of three groups in the team appraisal part related to my part. After reflecting on their comments, I summarized the following points to provide ideas for the iteration of the future project.

Visual aspect: LED lights are not very obvious

The critique from team X noted that:"the effect of LED lights is not very obvious. From the screen, we can't tell whether the LED light in the test tube is lit." Since I chose plastic test tube toys to make the test tube part, the material is transparent, so there is a disadvantage that it can not hide the sensor used in the test tube for the interaction effect, also, because the plastic is thicker, the brightness of the LED lights is not that obvious and user can not judge the status of the device by watching the brightness of the LED. In this part, I will choose to replace the test tube production materials. In the first aspect, I will consider choosing a larger test tube to install the required sensor. In the second aspect, I will find a way to make the visual effect more obvious (such as changing the color of the test tube in order to make the LED light more obvious). In order to match the parts of my other team members, we will make a discussion on the selection of materials and the overall design through the form of group meetings to make the overall product look more consistent.

Fix the sensor

As one critique reads that: "We are worried that when the user shakes the test tube or puts the sound in the test tube into another container, this process will affect the stability of the connection in the test tube." In the design process, because the jumper cable will indeed have poor contact, sometimes it will also cause disconnection by the different actions of the user. Therefore, it will take a long time to investigate the cause of the connection, which will not only spend a lot of time, in the final exhibition, this problem will cause a poor user experience. Originally, we welded some sensors, but it was still not stable enough. After using, moving and other operations, it will still cause disconnection. We discussed and decided to use electrical tape to fix the sensor and cable wire. Although this method is a bit time-consuming, it does play a fixed role and also makes a simple integration of the messy cable wire to a certain extent.

The form of the presentation of music needs to be considered twice

I have also received relevant suggestions regarding the presentation of music playback. For example, one group from the critique showed that "rather than lighting an LED, I think you need to display the presence of sound in a more obvious way." Combined with the reflection gained from critiquing other groups‘ work, I think I can add some visual effects when the users playback the music. I also need to do some research on how to combine sound and visual effects together, which can help me better serve the project. At present, I intend to use Unity to design a scene for the music playback as visual feedback for users playing this device.

Project Progress

This week we work collaboratively in school and integrate the parts we have completed separately together. At present, the team members have realized the main functions of their respective parts. The main task of our team this week is the integration of code and debugging of the entire project. On the arduino code, we have classified and commented according to different parts of each different section to facilitate debugging in the future

At this stage, we have only completed some main functions. Ideally, the device consists of a dropper, three test tubes, two jars, and a beaker. But at present, we just installed a dropper, a test tube, a jar, and a beaker and almost used up the analog and digital interfaces on the Arduino Uno development board.

We searched for the relevant development board and calculated the number of interfaces to be used for the final product. We decided to purchase Arduino mega as our development board for subsequent design.

The following picture is the overall presentation after integrating the work of all team members.

Future work

1. Organize the feedback obtained and discuss it with the group to determine the overall improvement plan for the next round iteration.
2. Thinking about how to add visual effects to make users more immersed in the interaction with sound.
3. Improve the overall product design, because for the current product, the nakedness of many cable wires will make the whole product look messy.
4. Start thinking about how to design my own portfolio.
5. Buy sensors with strong interaction effects and larger development boards with more interfaces.

We completed the design of the main functions and the optimization of the lines this week. The main function of Dropper is recording. The composition of Dropper is relatively simple. It consists of a button and a LED light. When the user squeezes the upper end of the dropper, the button will be triggered, and the LED light will turn on to indicate that the dropper is recording. Due to the simple structure of the components, we followed the example of the following circuit diagram to connect on the breadboard. As shown in the above figure, we used two resistors. At one end of the LED, we used a resistance of 220Ω, and at the end of the button, we used a resistance of 10K. Working principle: When the button is not pressed, the input voltage detected by pin 2 is low; when the button is pressed, the pin 2 and VCC will be turned on, and the input voltage detected by pin 2 is high. By judging whether the button is pressed, to control the LED on and off. The implementation of the code is also shown in the figure below.

Similar to the principle of dropper, we started to install test-tube. The main function of Test-tube is sound storage and playback. In our interaction plan, when the dropper touches the inner wall of the test tube, the music transmission will be completed (that is, the music collected in the dropper will be transferred), and when there is music in the test tube, the LED in the test tube will light up . Only when the user dumps the test tube into the beaker, the content stored in the test tube will be transferred, and the light will also turn off to indicate that the test tube is empty. Through the research on the existing sensors, we believe that the pressure sensor and tilt switch can accomplish the effect we want. By testing the different ranges given by the sensor with different forces and controlling the sensitivity of the sensor by adjusting the resistance, we simulate the effect of the dropper triggering the inner wall of the test tube by limiting the pressure value. The tilt switch is also a digital signal, and the connection method is no different from ordinary buttons. By judging that the value of digital read is 1/0, it can be judged whether the state of the test tube is tilted or flat. The connection of the pressure sensor is shown below:

In our plan, another function of the test tube is to shake the test tube to play back the stored sound. Due to the limited sensor on hand, there is no related sensor on hand to achieve this interaction effect, so we currently press the test tube hard (that is, use the pressure sensor) to trigger this effect.

Unlike the storage contents of test tubes, the jars store fixed melody that cannot be changed, so the LED light in the jar will always be on. These melodies are music fragments that can help people relieve stress such as piano music and violin music. These melodies can help users who have no musical foundation to create more harmonious music. Similar to the interaction form of the test tube, users can listen to melody by shaking the jar (currently using press sensor) and do the music transmission by dumping (tilt switch) the jar on the beaker.

Beaker is the sound mixing device. It is equipped with RGB lights, tilt switch and pressure sensor. The step to do the sound mixing: First select a main melody stored in the jar, and select the daily sounds stored in the test tube that you want to mix together and then pour into the beaker in sequence. The user can mix the music by shaking the beaker (using press sensor to simulate this function), and the mixed sound will be played while shaking. The lights in the beaker change according to the different audio being poured.

The chemical sets for kids we purchased have been delivered. After the experiment on the breadboard was successful, we began the installation and debugging of the sensor. The following figure is the effect diagram of the connection.

Personal contribution (follow-up)

This week, in addition to the technical implementation, I focused on recording videos of personal parts and writing documents.

In the team, I was responsible for the design of the test tube part and responsible for connecting the different parts completed by the members of the team to complete our group. The following video can be used as a display of my personal contribution and the overall work of the group.

Test tube can be regarded as a device for sound storage, playback and transmission. The following interaction plan can better understand how to interact with the product. I have marked my personal part (the test tube part).

The functions that the test tube part can achieve are as follows:

When the sound-collecting dropper touches the inner wall of the test tube, the sound stored in the dropper will be transferred to the test tube.

The LED light will be on when there is sound storage in the test tube.

When shaking the test tube, the stored sound will be played back.

When the test tube is picked up and poured into the beaker, the sound in the test tube will be transferred to the beaker, and the led light will be dimmed to indicate that the test tube is empty.

The test tube in the beaker corresponding to the test tube will light to indicate that the sound has been transmitted.

I also made corresponding research on the choice of sensor. Since the press sensor can accurately read the force acting on it, I think it can be used to make a touch effect. The tilt switch can be used as a digital signal to detect the tilt state of the object. I think it can be used to achieve partial tilting of the test tube. At present, due to the limited purchase of sensors, I find it difficult to realize the function of shaking to trigger sound playback with the sensor at hand. To this end, I asked tutor, he provided us with an idea is to use the tilt switch to determine whether the test tube vibrates by detecting the number of ball touches in the tilt switch within one second. The code is as follows:

The code is a counter and is updated after 1000 milliseconds, by judging the size of the counter value in 1000 milliseconds, it can be used to detect whether vibration has occurred. However, we still cannot use this method because the tilt switch will be used to control whether the test tube is tilted, and the balls in the tilt switch will also collide when tilted. I have n’t thought of a reasonable solution to control two effects with one sensor, because when shaking the test tube, the tilt effect will be triggered as soon as the ball touches, and the two effects will interfere with each other, which will affect the realization of the entire function.After checking the related sensors, I feel that the vibration module can meet our needs, and I will consider using this sensor when the next project is improved.

Another major improvement this week is the design of the lines. Because there are many accessories installed on the test tube, and in order to extend the life of each sensor, we connect a corresponding resistance in series, so there will be a lot of wires plugged into the breadboard. Through integration with the work of other team members, the wiring of the entire project is very chaotic. If there is a disconnection in the work, it is difficult to check. With the help of tutor, I learned that every arduino development board has a built-in resistor, and you can optimize the circuit by using the built-in resistor instead of the existing resistor. This discovery made my test tube part more tidy from an aesthetic point of view. During debugging, the entire project still does not work properly. After investigation, it is found that the method of built-in resistance is only suitable for the connection of digital signals, and the built-in resistance is not suitable for analog signals, because it will cause signal interference. As before, I chose to use built-in resistors to connect digital signals (led lights, tilt switches) and analog signals (press sensor) to the corresponding resistors.

Future Plan

1. Conduct user testing sessions for the overall project and conclude the key findings.
2. Modify the conceptual model and do some corresponding change in the design of the project
3. Continue working on the back-end part of our project.

Work done and reflection

This week, our process is a little bit stagnant, and because of the unfamiliarity with the circuit diagram, we have encountered some troubles in designing the circuit. Specifically, the projects we can complete are relatively fragmented, and only a few simple changes can be made through the tutorial to complete part of our project without understanding the true working principle of Arduino, so it is difficult to assemble scattered functions. To this end, we conducted a group meeting to streamline our interaction process and then unify the background knowledge to ensure the project can proceed smoothly.

Through discussion, we also iterated on the design idea in our design. In our original plan, the role of the Erlenmeyer flask was to mix the selected music clips, and in order to ensure the sweetness of the mixed music, we also decided to design the concept of music frame in the Erlenmeyer flask, that is, to provide the users with fixed background music to allow them to mix music and create music on this basis. The specific operation method is to tap the Erlenmeyer flask, the background music will start playing, and by shaking the flask, the selected music clips can be mixed. For this function, we comprehensively consider from the aspect of user-centered design and the realization of technology. In terms of user-centered design, due to the particularity of the period, we demonstrated this function through bodystorming and obtained user feedback through online interviews. By analyzing the interview data, we found out that he functions included on the Erlenmeyer flask are too complicated. The interactive form of mixing the sound by shaking the Erlenmeyer flask and tapping the flask to change the preset sound often makes the user forget the steps he has operated, causing cognitive overload. It refers to users should memorize the actions represented by each interaction form during the use of the Erlenmeyer flask and this will increase the user's inconvenience and reduce usability of this device. From a technical perspective, at present we want to use a tilt switch to complete the different commands by limiting the output value of the serial port to detect the analog signal. However, it is found through experiments that the sensitivity of the tilt switch is not very high, and it is difficult to distinguish the two behaviors of shaking and tapping from the output value of the analog signal, so it is difficult to limit the output value of the tilt switch to control the different interactive effects of this behavior. Based on this, we have modified the original design and introduced the concept of jars, the purpose of which is to strip the function of "music frame" from the many functions contained in the flask. To be specific, jars are generally regarded as containers for reagents in people’s mind, in the project, we use jars to store fixed music clips, and users can play the stored preset music clips by shaking the jars. Similar to the test tube interaction, users can transfer music by pouring jars. The difference is that after the music transfer is completed, the preset music in the jar will not be deleted.

Hereunder is the overall plan for completing the project, including how to use python to trigger each movement.

But in the discussion, it was found that it is difficult to use Arduino to call the trigger of the keyboard. To this end, we conducted technical-oriented research on how to use python to complete the back-end program writing and how to connect Arduino with the program. By unified serial port, the baud rate can link them together. In the last week, we have achieved the recording and playback of sound through import the pyaudio module. This week we have finished coding the main function in our project with python. (recording, playback, and mixing). The mixing function is achieved by playing the selected clips simultaneously.

Plans for next week:

1. Clarify the interaction plan (that is, specify the interaction, instructions, and back-end operations)
2. View the corresponding tutorials, master the connection methods of different components, and clear the materials needed for the project and build the project.
3. Connect the major part of the code with Arduino.
4. Improve the prototype and record explainer video.
5. Start writing detailed introduction documents.

What is your concept / individual focus / individual responsibility for the project?

Our team works in the form of cooperation and our design idea is not much different from the previous proposal stage, that is, to create a playful interactive device for sound processing. To be more specific, “sound lab” is a sound processing device. The shape of the product will be the same as the commonly used tools in the chemical laboratory, that is, test tubes, droppers, flasks, trash cans, and the interactive form of the product will also simulate the way chemical experiments are usually carried out. It has five main functions, they are collecting and recording sound, playing sound, storing sound, mixing sound, and deleting sound. Within the group, I am mainly responsible for the design of sound storage and playback devices and will be looking at the aspects of visual design, physical interaction, technical involved, playful concerns, user-centered design to complete my part. I am aiming to design a radio and playback device, and at the end of the semester, assemble with several components from other team members to complete the final product.

What is the ideal finished product?

Each tool in the entire design has its own purpose and function. The trash can is used to delete sounds, the dropper is used to record and collect sounds, the test tube can store different sounds, the jar can store preset melody, and the flask can mix and play sounds. The complete operation process is as follows:

1. The user finds his favorite sound in daily life (such as the sound of tap water, the singing of birds, etc.), he can move the dropper to a position close to the sound source, squeeze the plastic tip of the dropper, record the sound, and the sound will be stored in Dropper.
2. Hang the dropper over the test tube and squeeze the tip of the dropper, the music data in the dropper will be transferred to the test tube and the led light in the test tube will begin to glow to indicate that there is sound stored in the test tube.
3. The stored music can be played simply by shaking the test tube.
4. In order to ensure the harmony of music, we provide a lot of preset music in different jars.
5. Pour the test tube over the flask, then the sound from the test tube will be upload to the flask, and the selected music will be mixed together if the user shake the flask. There will be smoke during the mixing process, which represents the sound mixing by simulating the visual effects of chemical experiments.
6. When test tube, flask touch the trash can, the stored sound will be deleted

Progress

During the break and this week, my main research is focusing on the realization of technology. Since I am new to Arduino, I spent some time to get a rough understanding of Arduino and consult related tutorials about sensor selection and code implementation. In the process of research, I found that the Arduino only has 32k of storage and cannot complete the sound storage simply using the Arduino. How to do the sound storage become the tricky part of our project delivery at the present. And we have tried different ways in order to complete this function. Through investigation, I found that there are ways to expand the storage.

Voice Recording Module

Limitation: supports recording and playback within 10s, cannot support the long time recording function.

Raspberry Pi: Can be used Microcomputer, not sure how to connect to Arduino. (Need more investigation)

SD card module: can store a large number of audio files and we also find some tutorials about how to decoding the sound and store it into the SD card.

The tutorial might be useful for building

Make your own spy bug Arduino Voice recorder. (https://www.instructables.com/id/Make-Your-Own-Spy-Bug-Arduino-Voice-Recorder/)

Technical issues: how to call audio files in SD card, how to transfer the soundtrack in the SD card to the computer...

Since we are unable to think of a suitable solution, our project was stalled for several days due to technical obstacles. Thanks to the tutors for providing us a new direction in solving the problem, that is using python to call the recording function in the PC and link it with the Arduino

https://realpython.com/playing-and-recording-sound-python
https://pythonforundergradengineers.com/python-arduino-LED.html

https://pythonprogramming.altervista.org/record-with-python/?doingwpcron=1587691974.7996830940246582031250

Tutorial video:

Code:

https://python-sounddevice.readthedocs.io/en/0.3.14/index.html

At present, our plan is to use Python to complete the coding part and then make connection to the Arduino.

In-class activity

In the Tuesday studio, we conducted a regular report-back session. In this week, I take most of my time on the team report and some individual related work, I will have a detailed description of this part in the personal progress part of this journal. My major questions in this phase are more related to technology:

How to complete the sound transaction (transfer the recording in the dropper to the test tube), and how to delete the sound in the dropper.

After the session, we use Miro as the sharing platform to conduct the activities related to Methods for discovery.

I think Miro is a very suitable platform for teamwork. Due to the particularity of the period, we cannot carry out face-to-face group activities to discuss the direction of future work. In the previous time, we have doubts about how to share ideas effectively and timely, but with the help of this platform, our various team members can post their own data points on our own channels in time, and it is also very convenient to integrate ideas.

The activity we took in class is the observation task. This activity helps me in terms of finding a new way to collect various data. The method I chose for the task (conduct the observation on the train) is finding the relevant video on youtube. Hereunder are my findings during this activity.

The second in-class activity is the break down of the progress, this activity helps us got more inspirations and have an in-depth consideration of our group project. The picture below is the procedure of completing this activity, followed by this we have created our own project.

In the Thursday studio, it focused more on the Arduino things. Through this session, I have a general concept of what is Arduino and what ability I need to equipped with in order to use the kit. At this stage, I also have a general understanding of the various parts of Arduino and their selling prices. Through the sharing of tutors, I have a general understanding of the direction of future learning and the budget for completing the project.

Personal Progress

This week we will focus more on writing group reports. Apart from this, our group have also divided everyone's work and have a discussion on how we could work as a team to finish the final product.

I am responsible for the sound capture & record aspect of the sound lab and will be looking at the aspects of visual design, physical interaction, technical involved, playful concerns, user-centered design to complete my part. For my part, I designed an online questionnaire to investigate user needs. Questions are around people's attitude towards DIY music and how they would like to interact with music. Originally I planned to launch a workshop to collect more opinions, but due to the particularity of the current period, I can only use the form of online questionnaires, so the conclusions I may get are somewhat limited. But through this form, I still extracted a lot of useful data points and sorted out a lot of insights for the subsequent improvement of my part.

(The link for the questionnaire: https://docs.google.com/forms/d/e/1FAIpQLSdqE2l268cdMDaGNhXsWAlsf57Y43RBeRFauACt8yA1Kg/viewform?usp=sf_link)

In this activity, I have received 30 participants and the results of the questionnaire are as follow:

Key Insights gained

• Most people are interested in DIY music, however, most of their music creation only stays on the surface stage(e.g humming some random melodies come into their minds.) A small number of people will be exposed to professional equipment or musical instruments to perform a complete DIY music process. Music creation is relatively unfriendly to non-professionals

Inspiration: In order to facilitate people's music creation, we must find a kind of music that people can touch at any time. (Everyday Sound)

• Musical disharmony will affect users‘ creative passion.

Inspiration: Control the quality of the collected sound. (Possible solution: Install a decibel detector in the radio equipment(the dropper). When the collected sound exceeds the decibel of the noise, it will not be recorded.)

Inspired by 10 top sensors in Arduino, I will be working on deciding what sensor I am going to use in my project.

Future plan

Next week is the Easter break and there are no classes. In order to promote the progress of the project, I decided to use the holiday to do some research on the technology aspect of our project.

• Conduct more background research into the relationship between music and pressure.
• Research on possible techniques involved.

Because I am new to Arduino, for the completion of the final work I have to watch some online tutorials to get some technical support kit.

• Research on recording and radio equipment, Find the right component and buy components that are not provided in the kit.
• Decide on the task flow of the prototype and start prototyping

Progress

In this weeks' online studio session, each of us has a short stand-up session and reporting to our classmates as well as the teaching team about the progress (team & individual approach), concerns for the coming week, plans for the coming week.

In consideration of the 3 options given by Lorna to move forward for this work, we have a group discussion about sharing our ideas in response to these options. We reached an agreement that is to work together to complete a project. Our decision is based on the following concerns:

• The complexity and integrity of the concept. As there are multiple interaction methods contained in our concept, if we made it into the individual approach, the workload will increase dramatically. In order to finish the complete work, we would have to cut back on some features and change some of the initial interaction methods, which will affect the integrity of the entire idea.
• The technology and material preparation of the concept. When I got the Arduino kit, I found some tutorial videos about some common components and sensors and thought about what needs to be used in subsequent projects. According to the understanding at this stage, I think we may need to use the following components (the specific detail still needs to consult tutors to confirm the feasibility of the application): Recording equipment, Playback device, Infrared sensor, Motion sensor, LED lights. At present, the Arduino provided by the school does not meet our needs and we still need to purchase any other equipment, tools, and props, such as test tubes, recording equipment, players, etc. We think it is better to complete a complete work than to buy three similar equipment to complete a simple version of the idea.
• Collaborative experience & safety concerns. We have cooperation places, and will also take corresponding protective measures. Collaborate on a project to promote intra-group communication.

In the following project, we are more based on the group to complete the improvement of the concept. Due to the particularity of this semester, we still have the main direction of everyone. I will cover it in the following content.

Team

This week, our team focuses more on the assessment part that is the group proposal and individual statement which due next Thursday. Many of our discussion is about understanding the brief and conduct the relevant research for the purpose of putting forward a well-considered proposal from a more comprehensive perspective.

For the group report part, we reach the following agreements:

• Team domain

Relieve life stress. As we found in the background research, fast-paced life makes people more and more impetuous, and people will also generate more and more pressure because of social factors. Based on this, we would like to find a relevant method that will link to the relief of the stress of people.

Through the discussion, our understanding of the "context of use" is different. With questions, we asked the tutor about the explanation of this part in the group meeting ended in the studio session and understand this refers to explaining to the readers about in what circumstances will the user use our device.

• Feedback

After the presentation, we have received plenty of feedback on slack. We have discussed our insights for the feedback and thinking of how we could better our idea.

We concluded the following insights that are worth working further into after the group discussion:

• Multi-person participation.

However, we think the original plan can achieve multi-person participation to some extent, for example, people can collect sounds separately, and arrange music together. (We think that why users comment on this point may because we did not mention this part in detail when introducing the idea in the video presentation part, which caused some misunderstandings, but this also made us notice the importance of multi-person participation.)

• How to reduce the noise to guarantee the quality of the sound.

One possible refinement: Detect the number of decibels, decibels that exceed the noise range will not be recorded)

• Provide the pre-set melody.

Refinement: provide some pre-set melody in the flask, the melody can be changed by knocking the flask, similar to air pod playing the next song

• Using temperature element to control the speed of music playback

We will rank it as a less important part and would make little refinement according to this point, as there is no direct relationship between temperature and speed of the music.

• Edit/control of a certain sound.

Although this feedback is meaningful, it is contrary to our target audience (ordinary people with little music-related knowledge and this feedback seems too complicated for ordinary users and works better with those professionals)

• Collect multiple pieces of music. Users found out people will take lots of droppers with them when they are attracted by multiple voices which will be inconvenient to do so.

Possible refinement: three buttons on the test tube, each button represents a tone, or use an eight-segment digital tube, click once to record the first section, press twice, record the second section.

• Add smoke, lighting and other visual effects
• Use the brightness of the light to indicate the length of the recording time

At the same time, we assigned the part of the team report. I am responsible for research on the target audience and intended experience. In consideration of the feedback and the in-depth research related to our idea, I have a more precise description of the target audience and I will describe this part in the individual part.

During the group meeting, what confuses our most is that we are not sure what data is intended to collect at this stage, how to filter feedback because there are more feedbacks collected, some feedback views are contradictory but all sound reasonable, how to integrate feedback and determine which needs improvement is also a problem we need to consider.

Based on this, we consulted the tutor and he gave us some methods that we can choose to collect feedback, like research, workshops, prototypes (paper, body storming & physical) and etc. Review the prerequisite courses we have studied before, we would like to combine HCI theory in order to collect the data related to "useful" "desirable" aspects and the method we would like to use for testing the video prototype would be: TAM(Technology Acceptance Model), we would like to design the questionnaire in 4 dimensions according to TAM(perceived usefulness (PU), perceived ease of use (PEOU), attitude (ATT), intention to use (ITO)). We sent out the questionnaire and have received several feedback, my teammate Iris will be responsible for analyzing the feedback collected in the testing session.

Individual

In the cooperation stage of the group, I am responsible for researching potential users, in our original idea, we set the target audience as: Music lovers and producers; those who are interested in daily sound; those who are sensitive to sound. But for feedback research on video prototype, I realized that this equipment seems unnecessary for the music producer, they have already got the professional recording equipment and excellent music producing software. Based on this, I suggest narrowing our target audience to ordinary users who love music and want to create music. I would like to remove the professionalism and do some relevant modification on our persona and storyboard.

As for the individual approach, I am responsible for research on the function: Play sound and delete stored sound. At present, I am thinking of how I can contribute to the team. The aspects I will be focusing on might be: How to play the sound regularly, how to control the playing time, how to choose to play tracks, etc and how it can relate to visual/physical interaction, design, technology, playful concerns.

Concerns

The current problems are mainly concentrated on two aspects:

The first is technical difficulties. According to the current project, we are not clear about the realization of the following functions according to the existing knowledge reserve.

1. How to transfer the recording in the dropper to the test tube, and how to delete the sound in the dropper. (The possibility of using Makey Makey to realize the function)
2. How to remove the sound from the flask by pouring the flask to the trash
3. How to judge the length of time, and light up according to the length of time

The second aspect is the online research. As for the testing session, we need the real users to test the interaction part contained in our work and do the refinement according to the feedback. Due to the particularity of the situation, the prototype testing session becomes difficult to achieve.

Future Work

1. Working on the group report and think more about the individual approach.
2. Taking visual/physical interaction, design, technology, playful concerns into consideration when conducting personal work
3. Find technology-related information (such as coding, sensors required, how to design the circuit...) that better work for the completion of our work
4. Conduct more academic research and find more relevant work for inspiration.
5. Have a general understanding of the kit on hand and think about how we could do with this tool.

Online Studio

From this week, all of our studio sessions are changed to online mode. In the studio on Tuesday and Wednesday, we mainly listen to different groups present their original design ideas through the form of Zoom meeting. At the same time, we can give them our own feedback and critique according to their presentation, also, we can better our idea based on the content of others' presentation as our inspiration. In this activity, we got a lot of thoughtful suggestions, which will be very helpful for our future refinement of idea. We can also gain experience from presentations of different groups, such as novel perspectives on thinking problems, interesting interaction methods, novel presentation forms of domains, and implementation methods for similar functions and etc. Reviews for each group can be found in the slack presentation channel. After the presentation session, I have summarized the following points that inspired me in terms of the betterment of our project.

• The use of color.

The idea the emotional bird intends to use color to represent different motion which makes me think of our idea also contains the element color. However, we just use color to distinguish between different tracks which is less innovative. How to use the color wisely might be a tricky part, at present, I am thinking of using color to represent the soundtrack of different frequencies...

• Collaborative experience

When critiquing the Hedgehog, I wondered how multiple users can join in this game and I realized this is also a thought-provoking question that need us to think about. Our idea is to facilitate the collection of interesting sounds around us and try to use them to create a unique and interesting melody, however, we only think how single users can interact with everyday sound and haven't thought about how multiple users can get involved. How can people get collaborative experience will be a tricky part that is worth considering in the future discussion.

Feedback and reflection

In exchanges with tutors and the comments left by our classmates on slack, we have gained a lot of meaningful feedback. We are happy to see that many of our classmates like our idea to play with everyday sound and post on their opinions on how we can better our concept.

Within the in-group discussion, we concluded the following points that are worth thinking over:

• Set some fixed music frames

For someone who is afraid of the mixture of the sound would not that pleasing, they can play the preset audio and then find a way to mix the sound into the track may make the producing smoother.

• Interaction form

This is the question we would like to get answers at the end of the presentation. We are wondering whether the exiting interaction we designed to meet the need of the potential users and what interaction can be added in terms of better meet the users' needs. Based on the feedback we have so far, there are some intriguing suggestion that is worth taking into consideration.

Utilizing liquids of different densities to mix together (for example, oil in water and foam created by soap in water) to represent the combination of different sounds.

Add smoke effects in the clip blending session. Adding this effect will make this procedure more like conducting a chemical experiment, really playful suggestion.

• The way to store music

If a user is interested in multiple pieces of music when he goes out, he needs to take a lot of droppers with him. For the refinement, we need to think of in what way can multiple clips can be recorded in a more user-friendly way.

Future Work

In the next week, we intend to:

1. Conduct in-depth research on feedback, assess the severity of the problem and do the following refinement of the concept.
2. Conduct more background research of the theme - music metric as well as the related technology.
3. Read topic-related literature and find some existing product which is similar to our domain.
4. Design discussions for different target audiences.
5. Start working on the team report.

Affected by the epidemic, the school changed all courses to online courses which will make this a little challenging for the first teaching week of the change. Fortunately, although some unexpected things still happen, generally speaking, this week has progressed smoothly. Taking advantage of the one-week suspension period last week, our group conducted an online meeting to determine the overall idea and discuss some details according to this idea. At the same time, based on the existing ideas, we made a paper prototype and took some clips as the material for the video presentation of week 4.

Idea Description

Combined with the results of the discussion on the theme of musical metrics in the world cafe event, we are very interested in working with the theme of everyday sound. Inspired by the everyday sound hidden in Billie Eilish's song "Bury a Friend" and "Bad Guy", she uses the sound dentist drilling her teeth and the sound produced by the traffic light in Australia street as a soundtrack and mix them with different instrument and make a wonderful song.

This makes use think of we have neglected various voices in daily life, for example, the life-like sound（such as the sound of vehicles passing by, the sound of crowds, the rising and falling of birds in the morning, the delicate rhythm of the wind blowing the leaves, or the crisp sound of the rain on the ground in rainy days and etc. If we start to pay attention to the everyday sounds, we will find that there are many beautiful sounds in everyday life waiting for us to discover. During the group discussion last week we explored the playful interaction mode that can be achieved for everyday sound, that is, using chemistry experiments to simulate the sound processing process to enable people to interact with everyday sounds. The following picture is the draft of our concept.

The whole device can be regarded as a daily sound processor. The principle of the whole device is to collect sounds, recombine them, and play them. We use the props that often appear in the chemical laboratory as a presentation form and simulate the process of doing experiments to act as the way for the users to interact with everyday sounds.

Presentation

In week 4 we mainly focusing on making preparations for the group presentation. Within the group, I was responsible for making the ppt for the presentation, designing the persona, discussing what needs to be included in the presentation as well as recording my part of the research which needs to be included in the video presentation part. Our video conference this week is basically to discuss some details such as identifying the target audience and designing persona and storyboard, discussing the technologies that may be used in future implementations, and the feedback we hope to get in the presentation.

During the group discussion, we took a closer look at the target users this device can serve better with that is:

1. Music lovers and producers.
2. Those who are interested in daily sound.
3. Those who are sensitive to sound

(This refers to those who are good at discovering the sounds around them as well as those who are associative to the sound of a particular segment)

The following pictures are the personas I created.

The first persona Amy can be used as a normal user. She is a student and good at discovering the beauty in life. she would like to collect interesting sound happen in daily life and she will play it back in her free time to feel the beauty that certain moment brings to her.

Our second persona Dennis can be used as those who have musical backgrounds. He is a music producer, and he often stores some daily sounds as materials for music production as his inspiration. Also, he often tries to mix different soundtracks. Sometimes, he may come up with some surprising effects.

At first, we only had a preliminary idea about the general direction of this idea, but less technical considerations. Based on the concept and a preliminary understanding of the Arduino components at this stage, I think we may use the following device.

Recording equipment (For sound collection)

Playback device (Producing sound)

Infrared sensor(Detect the position of the dropper. When the position of the dropper is within the range of the sensor, the function of storing music will be triggered.)

Motion sensor (Trigger music playback when the flask is shaken by someone)

LED lights(Using different light to represent different sound)

In order to get feedback and better improve our concept, we invite the students who have listened to the lecture to give valuable opinions from the complexity of the idea and the playfulness of interaction.

Taken all the listed questions into consideration, we have produced the following video to present our idea in detail.

Please refer to the following link to have a look at our video presentation.

This video contains a detailed description of the background, inspiration, prototype(explaining how to use it by bodystorming), target audience, personas, storyboard, technology involved, questions involved.

Group formation and idea generation

After discussion and ideation process of different topics in world cafe, we selected three topics that we were interested in and groups are allocated randomly according to our preferences. I am very happy that I have chosen the first voluntary theme--music metrics, and also very happy to see three members, Rhea, Iris and Haley, who have the same preferences as me. Since I have worked with some of the team members before, so we just conduct a brief self-introduction then we start preparing for the presentation related work for next week.

We started a preliminary discussion based on our own insights on the topic -- music metrics and try to try to get some inspirations from the ideas generated by our classmates in the world cafe process. Merging and integrating existing ideas, for this topic we can conduct in-depth research in the following four aspects.

Everyday sound collection(This is the theme we decided to work on)

All of us were attracted by the idea about the everyday music collection glasses. Because we realised there are a lot of voice exisiting in the nature that is always neglected by us because of the fast pace of city life. How to make use of these pieces of everyday music and interact with it become a major concern for us. During the discussion, we found out that the way to present it in the VR environment seems not that novel and playful. Inspired by the chemical experiment happens, I come up with an idea called music lab. It consists of 4 major part, voice collection device, voice storage device, voice operating device and voice emptying device.

Eyedropper: Press the tip of the eyedropper to draw the everyday sound you find interesting

Test tube: can be seen as a sound reservoir. Store various collected sounds.

Erlenmeyer flask: Sound generator. Dump your interested test tube to this bottle store and shake the bottle to mix the poured sound into a unique melody. The speed of the melody will be determined by the speed of the rotation.

Rubbish bin: Responsible for processing undesired sounds. Once the test tube / conical flask touches the trash can, the existing sound will be emptied. These empty devices will be able to store new sound clips.

Music + exercise

This theme is inspired by the ideas of utilising different fitness equipment to generate music. When thinking of the interaction mode which will make it more playful and unique to others, we have some different options like punching, riding, body movement... However, these ideas seem to be similar to the ideas of some existing interactive installations. Regarding how to better combine sports elements and music, although our group thinks it is an area worth exploring, it is difficult to determine a clear direction on the current form of interaction and presentation.

Color + music

Shadow + music

Ideation and presentation

Ideation process

I used music as my theme and started looking for inspirations. The reason I choose this theme is because I think music has become an essential part of everyday life and music plays a big role in regulating people's emotions. There are also many interesting interactive devices designed around the theme of music.

Based on this topic, I watched many examples that inspired me. I will list some ideas that inspired me most.

Interactive sound exhibition--Line

https://www.youtube.com/watch?v=hP36xoPXDnM

The designer use lines to represent novel music instruments. People can interact with it by touching lines on the wall or hanging on the ceiling, stepping the lines on the floor.

Advantages: Easy operation and novel interaction mode

Limitation: Not very friendly to users without music theory, it is less likely to create pleasant songs due to unfamiliarity with musical instruments. People will lose interest in it because of uncoordinated melody created.

Inspiration: In my idea, I can consider the innovations that are user-friendly to the zero base users.

ARASHI Live Tour 2014 THE DIGITALIAN

https://www.bilibili.com/video/av5176824?from=search&seid=3752956218064935948

This is a clip of a concert, it shocked me a lot. The accompaniment of the song in the video was done through the singer's clothes. This coat can create different melody according to the state of the user's muscles and different vibration frequencies. When performing this song, the singer needs to adjust his posture to cooperate with the members to complete the unique accompaniment.

Advantages: Very novel way to interact with music. Every rhythm is a unique tune.

Limitation: Requires constant experimentation for team collaboration music creation.

Inspiration: Is there an interactive form that is easy to operate for more people to join in the melody creation?

Given the above inspiration I have come up with my idea which name is Melo Catcher aiming at create simple ways for users to diy their own melody. Detailed description of my idea can be found on the previous journal entry.

In-class presentation and reflection

In class, I conducted a 2-minute idea presentation session. I got inspiration from many interesting ideas in this activity. I also got a short evaluation after the presentation, which inspired me a lot.

Personal pitch feedback and summary are reflected in the following aspects:

• The concept is a little off the brief. The interaction mode is simple and the rhythm performed by each handle is fixed which make the interaction less novel and playful.
• People are not provided with the unlimited choice. As the music performed by each handle is fixed, users can only adjust the speed of the music which will make the machine less playful.

Inspiration from other work

Punching Bag Fountain

A really eye-catching idea to bring music and punching bag together. The installation share the same theme with me, that is, music. The interaction mode is novel and allows multiple people to come around to perform the unique piece of music. The usage of fountain to display the rhythm being created is romantic and eye-catching. If the installation exists I am very willing to have a try.

Also there are some concerns, the size of this installation, how users can control the quality of the cooperation, I mean, how to ensure that the music played by people working together is pleasant, because the discordant rhythm will reduce the user's favourability.

Works to do

In week3, I will make changes to my ideas based on feedback from the teaching team and inspiration from other students' presentations.

Also, I will do some background research on this theme to clarify what need to be done, what can be done, how can it be improved.

Soldering things

This week we learned how to perform soldering on circuit boards based on circuit diagrams. Here under are the works I have created in the class.

I was exposed to do some soldering things in high school interest classes, so this event is not too complicated for me. Through this tutorial session, I found some key points in conducting the soldering process.

• Safety-related problems

When the soldering rod starts to work, the temperature is very high. The correct use of the instrument when soldering seems extremely important. Always wear goggle and covered shoes to prevent possible damage.

• Well-designed circuit

Make sure the circuit design is reasonable before soldering, as it is difficult to change once soldering is completed.

• Think over before soldering

During the tutorial, the soldered circuit board could not work because of my wrong reading of the circuit diagram. The soldered work is difficult to change, and it took me a long time to correct it into the correct circuit.

• The right amount of tin wire

Too little tin wire may cause disconnection of the circuit, on the contrary, too much amount will cause unnecessary connections and cause the circuit to malfunction.

Melo Catcher – A music box that can DIY your own melody

Music is long be regarded as a way to relieve stress and make people happy (Myriam, 2013). Nowadays, many designers try to integrate the element of music into daily life using various interesting forms of interaction.

Inspiration

Interactive sound exhibition

Playing three novel music instruments by touching lines on the wall, floor, or hanging on the ceiling.

ARASHI Live Tour 2014 THE DIGITALIAN

A specially designed coat that can produce different sound according to different body movement.

What is Melo Catcher:

Melo Catcher is an interactive music box that provides users with a new form of musical interaction. Users can enjoy the beauty of music and DIY their own melody regardless of their musical experiences.

How to use it:

Each handle represents a different piece of music, people can start generating their own melody simply by rotating different handles on the box. The speed of the music will change in accordance with the speed of the rotation. Once the device is triggered, the screen will generate a different wave pattern based on the selected music and shaking speed. The music will stop when people stop rotating the handles.

Who I am designing for:

Anyone who loves music and enjoys creating.

The sparkling point:

• Easy to use

Inspired by the design of a music box, the interactive device looks more like a large music box with many switches, users are more likely to be comfortable with this device.

• No background restriction.

People don't need to be proficient with a certain instrument, anyone who is interested in music can use this installation for music creation.

• Unique experience for musical interaction.

Users can feel different rhythms by shaking different handles and trying different speeds, at the same time, they can also gather multiple people to create unique melody.

Reference:

[1] Thoma, M. V., Marca, R. L., Brönnimann, R., Finkel, L., Ehlert, U., & Nater, U. M. (2013). The Effect of Music on the Human Stress Response. PLoS ONE, 8(8).

[2] LINES - an Interactive Sound Art Exhibition. (2016). Retrieved 2 March 2020, from https://www.youtube.com/watch?v=hP36xoPXDnM

[3] Music box. (2020). Retrieved 2 March 2020, from https://en.wikipedia.org/wiki/Music_box

Poster

About me

Hello! My name is Jianing Jin, you can also call me Sophia. I am always curious about new things and am happy to use my imagination to create something interesting.

I am a master student major in interaction design, and this is my last semester. Through the previous course learning and knowledge storage, I have a more comprehensive understanding of interaction design from various perspectives. At the same time, the study of courses such as graphic design, web design, design thinking, human-computer interaction and etc. laid the foundation for the smooth progress of this course.

Expectations

For this course, I am looking forward to it, but I will inevitably have some concerns. I participated in the exhibitions held in the past year, and I like the finished products created by each group. I also hope to apply what I have learned in the past 3 semester and complete a satisfactory and creative work through the cooperation with the other group members. This week, I had a for a systematic study of the machines that I might use in UQ innovate, and I am looking forward to doing some handmade products.

As for the concerns, I am not very confident in my coding skills and for the completion of the whole process, coding should be a mandatory skill that cannot be neglected. I hope to learn from the skilled tutors and group members about it during the semester. Also, I am worried about the workload. As it is a 4-credit course and the whole process lasts only one semester, how to manage the time wisely and how to ensure the idea our group want to build can catch up the final deadline is a big challenge. Moreover, I am a little worried that the gap between the final product and our expectation due to limited capacity.

I hope to complete a truly meaningful and innovative work through this course, which will complete a perfect ending for my master study.

Week1 catch up

The following is the supplementary content according to the comment on the first entry.

Works done

The first contact session of the course is more about the course description. During the session we conduct a brief ice-breaking within the team and communicate with concerns, rumours, exceptions...about this course.

Here under are what our group came up with during the session.

How relates

Work to do