Documentation & Reflection

Self Introduction

Xue Xia - Mon 2 March 2020, 10:15 pm
Modified: Mon 2 March 2020, 10:15 pm

I am a master student whose major is interaction design. I have learned a lot of skills in the previous study such as presentation, visual design, software programming, front-end web development, user research method, and evaluation method. However, I still think my ability to coding is poor. In addition, I haven’t learned any hardware programming, back-end web development, physical construction, sensors, and basic electronic skills. Thus, I am a little worried about the course. This is my final semester, I really need to pass the course to graduate on time. Thus, I will do self-learning to strengthen my skills in software programming and front-end web development. I hope to learn some skills of physical construction, sensors and basic electronics in the course. I think I may have the ability to join physical construction work in the team project after learning those skills. I am familiar with the presentation, visual design, user research method and evaluation method, which I will practice in the course. However, I think I have no talent of hardware programming and back-end web development, I have no confidence of working on it in the team project even though I can learn it from the tutor in this course. Overall, I want to practice skills that I have learned before and learned skills relevant to physical construction in this course.

Project Inspiration

Jessica Jenkinson - Mon 2 March 2020, 10:14 pm


Stress is an issue that is affecting society more than ever. 72% of Australians admitted that stress affects their physical health and 64% admitted that it affects their mental health. The interactive stress ball is a concept developed to combat this growing issue. The concept involves technology that can detect the stress level of a person in its vicinity. The idea was inspired through research that uncovered the growing prevalence of stress as well as various technologies that focus on stress relief such as "calm technologies". I was also inspired by a current installation called TeamLab that uses sound and light to create a "body immersive" experience.

The stress ball is to be positioned on a surface close to the user and when triggered, through increased user stress levels, it will glow red and vibrate until the user holds the ball. The user can then use the ball as a normal stress ball due to the squishy exterior of the device. On top of this, the ball will emit pulses, noises and light at a certain tempo which the user will have to match their movements and breathing patterns to. Through the stress detecting feature, as the user interacts with the tempo the ball will eventually decide when the users stress is at an acceptable level and return to its normal state. This will continue every time the users stress reaches a certain level determined to be too high.

Stress and Australians - WayAhead. (2020). Retrieved 2 March 2020, from

teamLab Planets TOKYO | Toyosu, Tokyo. (2020). Retrieved 2 March 2020, from

This Interactive Stress Ball Is Better Than Valium. (2020). Retrieved 2 March 2020, from

Written by Steve Veltkamp on September 4, 2. (2020). [Video] Designing Calm Technology in IoT. Retrieved 2 March 2020, from

Floor Colouring

Nick Huang - Mon 2 March 2020, 10:12 pm
Modified: Mon 2 March 2020, 10:23 pm

What is it?

Floor colouring is an interactive device, mainly composed of a mobile projector and a bracelet that allow one or more users to interact with the floor in novel ways. This mobile device can project a painting onto the floor, and then the user can adjust the colour of the paint through the bracelet, and then colour the different parts of the painting by mopping the floor.


So when users interact with this device, there are three main parts, paintings, bracelets and the projector. For paintings, users can choose paintings with different difficulty levels through the projector, that means paintings with low difficulty level have fewer colour blocks that need to be filled in, while paintings with high difficulty have more complicated blocks need to be filled in. In addition, users can draw some of their own content in advance and upload it to the projector, so they can customise their paintings. The size of the painting can be adjusted according to the size of the floor to be cleaned, such as zoom in or zoom out. For the bracelet, the user can use the bracelet to select different colours when colouring the painting. The device also allows one or more users to interact and collaborate with each other to complete the colouring process. In addition, when a user finishes the colouring of the painting, they can share their filled paintings on some social media, so that to enable users to socially connect with others. The projector is mobile, so it can be moved around their rooms. It can also show the coloured content onto the floor in real time. In addition, users can also use the records in the projector to print their own work through the printer.

Context of use:

The device can also be used in different scenarios, such as at home or in some public places, like parks. By using it at home, it can increase users’ interest in cleaning, and enable them to cooperate with their families to record the process of cleaning each time and enhance their intimate relationship with family members. In some public places, it can enhance people's awareness of public health and can also serve as a new way of social interaction.


colouring cleaning housework collaborative interaction customisation

Explanatory Post

Jiajia Liang - Mon 2 March 2020, 10:03 pm
Modified: Mon 2 March 2020, 10:07 pm

Dance with Light


My concept named Dance with Light.

This installation is consists of three parts. A huge screen, some blocks of different shapes, and the movable light.


People freely assemble blocks together and adjust the light source. By irradiation, a shadow will be formed on the screen. When the shadow's outline match some certain patterns, the system will be triggered and the black shadow will change to the colorful animation to inform the players that they have succeeded. For example, two blocks that look completely unrelated to each other, through constantly rotating and trying at multiple angles, end up with the silhouette of a fish. Then the system is triggered, the fish swells its tail and the screen becomes the ocean.


This installation can be played by multiple players at the same time and also has a competitive mode to make it more interesting. This game needs the player's imagination and creativity, promotes the coordination of hands and brains, and improves the ability to control spatial and planar graphics.

And here is my A3 concept sketch


This installation was inspired by a game I used to play as a child: step on the shadow. We chased each other in order to step on each other's shadow. And hand shadow: use our hands in different gestures to get many different shadows, usually of animals such as butterflies, spiders and wolves.

Imgur Imgur

Therefore I set out to design my concept. At first, I just thought of a table with blocks on it and a light source suspended in the center above the table (figure 1). Then I thought, why not hang up all the building blocks as well (figure 2), however, another problem arose. What if the light source could not be distinguished? If the light source is fixed in one place, the user will be very inconvenient. So I designed multiple light sources and made them mobile through the track(figure 3). Finally, it occurred to me that the rectangular table was not convenient for the user to put the two blocks together. If the table/screen is designed to circle shape, it will be more reasonable, and also more convenient when the player is in competitive mode(figure 4).

Other resource for the future work

After completing my concept I have done some research and found that one game that really coincides with my idea, called ‘shadowmatic’. Here are some screenshots of the game.

This game is very simple and delicate, I believe it will provide some help to my future work(eg. in the design of scenes and objects).

Imgur Imgur Imgur Imgur

shadow #graphic #creativity #imagination

Project Inspo: Pillow Pal

Piyumi Pathirana - Mon 2 March 2020, 10:01 pm
Modified: Mon 2 March 2020, 10:06 pm

Pillow Pal

Concept Sketch


Concept Description

With constant pressures being placed on our shoulders everyday and an increase in demanding situations that people feel the need to succumb to, people are dealing with stressors and unwanted negative emotions all the time. These can be very hard to overcome when feeling surrounded by people or being placed in unfamiliar situations, as people tend to not even realise the distressing feelings that they are experiencing.

I myself get quite stressed and upset when things don't go according to plan and don't even notice how I'm feeling! Once those feelings come, I find it very hard to restore myself back to a calm state right away.

So, to combat these emotions, that's where the Pillow Pal can help. The Pillow Pal is pillow that senses the negative emotions a user is undergoing (potentially through heartbeat and body warmth). It uses a meditative approach to create a calming atmosphere.

Based on the particular emotion a user feels, the Pillow Pal releases a scent and song to relax the user, as well as changing colour to make the user aware of the type of emotion they might be undergoing. Initially, the song that is played will try to resonate with how the user is feeling, before gently transitioning to a more relaxed song. Once the Pillow Pal has registered that the user is calm, it changes its colour to a warm comforting white, while maintaining its soothing scent and calming song.

The Pillow Pal hopes to bring some light to those darker days that almost everyone experiences on a day-to-day basis, and to bring about awareness of recognising these negative emotions.



#sensory #emotions #relaxation


Zhijiang Wang - Mon 2 March 2020, 9:42 pm
Modified: Tue 3 March 2020, 4:49 pm



For thousands of years, mankind has been looking to nature for inspiration, motivation and relaxation. Connecting with nature allows man to disconnect from the internet, and connect with the Inner-net. On the path of self experience, nature supports a man's wellbeing by providing a meditative experience. The aim of this immersive experience is to allow human beings to naturally align themselves with the elements of nature, the experience of which can be enhanced through the activity of meditation.


"Elements" allows visitors to do meditation surrounded by different natural elements (eg: thunder, snow, dust, rain, and etc) without get touched in the space. Motion sensors can detect visitors' movements as they navigate through this space and stop to pouring down stuff around the participants.

There are two way to interact with this installation.The breath can be detected to adjust the speed of elements, the more heavier breath will cause heavier effects(eg:heavier raining, bigger snowing). Another way is change the mode of elements by holding the breath for 5 seconds(eg change raining mode to windy mode).

Interaction mode: breath

Context: Home, public installation, gallery




[rain house]


immersive experience #natural elements #meditation

Project Inspiration

Heng Jiang - Mon 2 March 2020, 9:40 pm


I was inspired by my friend. He told me that he had a lot of clothes, some of them only worn once or twice. Sometimes it was difficult to choose for different places. I think there should be many people like him who don’t know how to choose clothes. Even wear inappropriate clothes for some specific occasions. So I have this idea. It will collect your personal information and combine your color and style according to your occupation, identity, frequent occasions, personal preferences, etc. to meet your occasion needs.


This is a wardrobe that automatically helps users choose clothes. In front of the wardrobe, there is an electronic screen. It is used to collect user information and can interact with users.When the user uses it for the first time, they need to enter their basic information, and then enter information according to prompts, such as destination, what to do today



auto choose clothes

The Human Sea-mulator

Kelsey Quinn - Mon 2 March 2020, 9:32 pm
Modified: Mon 2 March 2020, 9:38 pm


This device is designed to create awareness about the impact of human activity, climate change and rising temperatures on the ocean. It consists of a large tank of water with sensors for various changes such as temperature and salinity, extra jugs and boxes filled with various things that can be added to the tank, and a laser projection system to project images onto the water surface.

Users add contents from the jugs and boxes to the large water tank, and the laser system projects images onto the water surface depicting various changes in the ecosystem of the ‘ocean’ based on what users added. For example, users adding warmer water increases the overall temperature, and the projection changes to depict bleached coral, fewer animals, wilder storms, and other effects linked to rising ocean temperatures. Users can also add contents in attempts to restore the ‘ocean’ to its original state, such as colder water. Other potential interactions include adding salt or fresh water, adding and removing human ‘rubbish’ such as plastics, adding and ‘cleaning up’ liquids that represent oil and other chemicals, and adding and removing ‘boats’ that represent fishing and other activities that remove animals and resources from the ocean.

This is intended to be installed in a park or other public place where people often walk past, and can stop and engage with it as long as they like. It is also intended that multiple people can interact with the tank at the same time, so that the effects of one person adding, for example, ‘rubbish’ items while another removes ‘boats’ can be depicted simultaneously.


Inspiration article:

Impacts of rising ocean temperature:

Lasers and water:

environment laserprojection

Week 1 Poster

Qinhan Yang - Mon 2 March 2020, 9:30 pm

My creative inspiration comes from life. People always emphasize an efficient life, but since entering university, countless homework and living abroad have left us exhausted, often forgetting our deadlines, forgetting to organize our life, space, and study.

Some mobile apps are actively working on this issue, but tedious text and tables can even make you feel more tired. At this time, I hope to have a kind of real-life tasks to make our tasks visual and make the interaction more interesting.


This is an efficient room. Especially improve people's work efficiency. Cumulative boxes represent tasks that should be completed. The boxes are getting higher every day and getting closer to the deadline. Part of the task is to get an ax and cut the growing box to prevent it from running out of time. Once you cross the alert line, the room will attack people, forcing them to start working.


Explanatory Post - S4460633

Edward Carroll - Mon 2 March 2020, 9:21 pm

When constructing my idea, I didn’t just sit down and think of new brilliant ideas to change the world. I just went about my day like normal but knew I would have to come up with an idea before my presentation. I drew my inspiration from everyday tasks and how I could reinvent some of these tasks to make my day to day life easier. My idea is to reinvent the toothbrush and create a new ‘Smart Toothbrush’. The idea is that you buy a new toothbrush and stand that is linked wirelessly to your phone or computer. You brush your teeth every day like usual however the toothbrush maps the inside of your mouth and will be able to tell you the areas of your mouth that you miss as you brush.

I would also have the capabilities to track the health of your teeth and alert you should you need to go see a dentist about fillings, cracked or chipped teeth, and other common dental issues. If the toothbrush detects anything wrong with your teeth it will alert your phone or computer and send a copy of the scans to your dentist to aid in the repair of your teeth.

everyday #health #daily life

Explanatory Post - S4460633

Edward Carroll - Mon 2 March 2020, 9:21 pm
Modified: Mon 2 March 2020, 9:22 pm

When constructing my idea, I didn’t just sit down and think of new brilliant ideas to change the world. I just went about my day like normal but knew I would have to come up with an idea before my presentation. I drew my inspiration from everyday tasks and how I could reinvent some of these tasks to make my day to day life easier. My idea is to reinvent the toothbrush and create a new ‘Smart Toothbrush’. The idea is that you buy a new toothbrush and stand that is linked wirelessly to your phone or computer. You brush your teeth every day like usual however the toothbrush maps the inside of your mouth and will be able to tell you the areas of your mouth that you miss as you brush.

I would also have the capabilities to track the health of your teeth and alert you should you need to go see a dentist about fillings, cracked or chipped teeth, and other common dental issues. If the toothbrush detects anything wrong with your teeth it will alert your phone or computer and send a copy of the scans to your dentist to aid in the repair of your teeth.

everyday #health #daily life

Week 2 - Artificial Conscience

Lucy Davidson - Mon 2 March 2020, 9:08 pm
Modified: Mon 2 March 2020, 9:14 pm

Idea Explanation


Climate change is the biggest problem we are facing today and we really need to be making immediate changes to our lifestyle. It can feel very powerless seeing the big companies and governments not really doing much to help this issue, so by having a fun and playful interaction like Artificial Conscience, it can motivate everyday people to make small changes for a bigger cause.

I am very passionate about minimising the effects of climate change but still find it difficult to find ways to do this in my everyday life. It has been found that one of the quickest ways to change behaviour is through disapproval ( Now not everyone has a blunt friend who will tell you when your makeup looks bad or when you’re using too much electricity, so that’s where Artificial Conscience comes in. Artificial Conscience is a digital figurine that responds to the environment around it. It can be used in many different settings but the most appropriate and common use would be in a public place in households where all members of the family can interact with it (e.g. living room). It is used to encourage conscious effort in reducing energy and water use in households by reminding the user when they are ineffectively doing so.

It can respond to its environment through its rotating arms, colour (string of LED lights inside frosted casing), facial expressions on screen. For example, if the user has a 20 minute shower, they will then come into the living room to be publicly shamed by the drowning plant on the coffee table (see figure below).


I think its a very interesting interaction as instead of just a meter showing you that you have used 22L of water today, it gives an emotional context to situation. By personifying the environment in this way it allows people to have an emotional response to their inefficient use of resources, utilising the human values of compassion and need for approval.


This idea was inspired by lua, a digital plant pot that visually alerts the user of what the plant needs.

Digital Plant Pot -

#climatechange #sensors #sustainability

Week 1 Project Inspiration

Wei Cui - Mon 2 March 2020, 9:04 pm

visualization of health

My concept is a portable device which visualizes health status to develop people's awareness of their health condition.

The device has ball-shaped outer shell, the digital plant inside the ball represents people's health. When people interact with it, the digital plant will show different appearance of growing. As health is abstract and may be neglected in daily life, it helps people to know their body state in an intuitive way.

Mainly, the Interaction mode is Taking the ball with you and carry out daily activities. The more energy you gained, the better the plane grows. The plant will show how your activities turn into nutrients. It's like your energy has been stored in the ball, you can see directly from the digital plant that the energy makes positive effects on your health. If you have a fairly sedentary life, the plant will grow slowly. On the contrary, if you get enough exercise, the plant will be lush and green.

The other Interaction model is when you put your hands on the ball, the sensor will detect your body temperature. If the temperature is lower than the normal level, you will see the digital plant wither away. As your body is not in a good condition. There will also be a voice reminder like "you have been in the air-conditioned room for too long, stand up and stretch your legs." Similarly, if it shows that the body temperature is a little bit higher than the normal level, the leaves of the plant will weaken and become yellow. And the voice will remind that maybe you need to have a rest.



visualization of health


Hao Yan - Mon 2 March 2020, 8:57 pm
Modified: Tue 3 March 2020, 12:44 am


umbrellas have found their way into many areas

protecting somebody from sun, rain, snow, wind, sand…

I was inspired by the small umbrella on the cocktail,It’s small but fully-equipped. Same as other umbrella,It consists of rib、canopy and a handle.

Its simple mechanical structure can control canopy closing and spreading.

Seen from the top of the umbrella. If canopy is regarded as a circle, then the circle becomes larger in the process of the umbrella from the closed state to the fully extended. So I think this can be used as the display part of a device.

I was thinking, if each umbrella is fixed on a servo motor, and it is ultimately controlled by Arduino. In this way, dozens or even hundreds of such umbrellas can be used as a screen display. And, in a course last semester, I tried to use Arduino to make an interactive device, which uses Ultrasonic Sensor, which is a sensor that can calculate the distance between objects. In this way, if people approach or move away from the device, the displayed picture will be different. If this device is hung on the wall, then this is a subversive decoration.


closed state

12531583150923<em>.pic</em>hd.jpg 屏幕快照 2020-03-02 下午7.11.17.png

spread state

12541583150936<em>.pic</em>hd.jpg 屏幕快照 2020-03-02 下午7.11.23.png


屏幕快照 2020-03-02 下午7.12.30.png 屏幕快照 2020-03-02 下午7.12.41.png


YouTube - Need the video id from the URL - the bit after watch?v=

Daniel Rozin, Artist and Professor, Interactive Telecommunications Program, NYU, makes mechanical "mirrors" out of uncommon objects that mimic the viewer's movements and form

Imitating this art form, I can express the mirror in the video as an umbrella, and make some changes to the way of interaction. I think this work can greatly attract people's curiosity.

display #umbrella #interactive #calculate

Week 1 Reflection

John Cheung - Mon 2 March 2020, 8:56 pm
Modified: Mon 2 March 2020, 8:58 pm


Today we have discussed the fear and expectation of this course


1, I have never taken any 4 unit course until this semester. I am not able to predict the workload and the amount of time I need to spend on this course

2, Some students said this course required a finished product at the end of the semester, I am not confident with my building skill.

3, Team coordination/ Time organisation are always the big problems in a group assignment.


1, Learning the new technology, such as the laser cutter, 3D printing machine... and applying them on this project through the innovation lab tutorial.

2, Coming up extraordinary ideas with my team members

3, Using different tools and equipments to cope with the difficulties I faced during the prototype making, learn from the mistakes and safely use all the tools.


The HCI grand challenge topic is learning and creativity


The way how people learn has changed rapidly. The use of technology in the contexts of education and creativity is very common nowadays.The new generation of learners prefers a different learning style comparing to the past learners, but these learning style has a lot of challenges to deal with. For example, mobile learning is a big trend, students can acquire knowledge anywhere anytime they want, but the effectiveness is something educators and researchers need to consider.

The Serious games for learning was the main topic that I focused on. I believed that keeping a balance between seriousness and gaming was the biggest challenge in creative learning. When we are making the course content more attraction, gamification elements are used frequently, for example, bringing excitement ad fun to the learning journey. But too much fun would distract students from learning. I suggested that educator should use the technology to add gamification elements to the game instead of merging the gaming content with the course content. For example, they can use an AR glasses to guide student to finish a specific task.


♠ = Design to / for

♣ = Place

♥ = Interaction Mode

♦ = Interaction Quality

In our group, we have selected an idea which is "Using smell to guide cleaner to clean up a specific part of the hospital." For example, cleaner A is responsible for the apple smell, any location smell like apple will be included in his cleaning task. It is a very interesting ideation method, our group has come up with a lot of interesting ideas and it help brainstorm the project topic.

Project Ideas & Inspirations

Shawn Xue - Mon 2 March 2020, 8:53 pm
Modified: Tue 3 March 2020, 4:03 am



How it works

The display shows a virtual pet. The hand tracker detects your hand movements and mimics them on the screen. You can then stroke or play with the virtual pet using your own hands.


  • Stress
  • People who can't have pets



Leap Motion(Ultraleap)'s controlers are able to precisely track hand movements and gestures which provides opportunity for various creative projects.

Project Concept-Skyscraper

Jiexiang Xu - Mon 2 March 2020, 8:41 pm
Modified: Tue 3 March 2020, 2:25 pm



What is Skyscraper

Skyscraper is a desktop lamp. It looks like a tall building with lots of windows. Behind each window is an LED bulb. Skyscraper is able to record gestures made by users and convert these gestures into electronic form, and then present them on the surface of the desk lamp. In addition, the user can also control the state of the light emitted by each window, such as the brightness, color, flicker frequency and chronological order of the light grid. After the user completes the innovation, Skyscraper can connect to another one via the network, and can project the produced pattern on the second device. Since the device is mainly based on gesture interaction, the innovative form has a strong appeal. Moreover, for users, the patterns of each work are unique and will not be repeated.

In essence, Skyscraper is an interesting table lamp. In addition to the basic functions (lighting), the desk lamp can customize and transmit information to help users express their emotions and convey love, thereby enhancing the relationship between people.

Target audience:

Any people with intimate relationships (parent-child / couple / close friends) can use Skyscraper as long as they want to express their emotions in a special and romantic way. In addition, People who are shy at accepting and expressing love can also use it to relieve anxiety, improve empathy, and increase their sensitivity to other people's emotional responses.


  • There are words on the tall buildings in Shanghai
  • "Human body printer"


skyscraper gesture intimate relationship lamp emotion lights

Explanatory Post

Huiyu Jia - Mon 2 March 2020, 8:29 pm
Modified: Mon 2 March 2020, 10:46 pm

Shadow Band



About the idea

Many people have a dream about a rock band, but for most people, it is difficult to learn to play an instrument and form a band. Their lives are full of work and trivial matters, they don't have time to learn to play an instrument, or they don't have spare money to buy an instrument. When they see the band playing passionately on TV, they will remember their inner pursuit of music. If my idea can solve a "problem", I think it is to help people make up for their regrets about their music dreams.

I used to play a virtual piano on my mobile phone. For a moment I even felt that I was a pianist, but the realism of 2D games was not enough. I played some somatosensory games. They can capture my actions and give feedback. So I think instrument playing can also be a somatosensory game, and this is my original inspiration.

You don't have to pay for real instruments, and you don't have to worry about playing skills. These instruments will be projected on the wall, and there will be your shadow on the wall. The sensor that captures your figure will connect your movement with the instrument. When you make a playing action, your shadow will be the same, then the instrument will sound. You can choose to be a guitar player, your friends can play drums and saxophone, etc. Then the shadow band is set up and starts playing.

As for the context of use, this is a highly interactive installation and has artistic value, so it can appear in interactive technology exhibitions, as well as in the modern art museum.

Of course, it would be very interesting if it can be developed as a somatosensory game for people to play at home. You can form a band with your parents or friends. This is an entertaining way to promote good communication.

Source of inspiration

collaborativeaction #somatosensory #music

Project Inspiration - SKY REACHER

Liony Lumombo - Mon 2 March 2020, 8:16 pm


The more rapid development of technology, the development of children to use it also increases simultaneously. Children become addicted to smartphones and tablet devices to play, not to learn. And that makes parents worried. Some developers develop educational games to insert education while children play. Educational games are built on media with screens (smartphones, tablets, computers) and increase to Augmented Reality and Virtual Reality.

A large company, LEGO, followed this development. Although they make plastic bricks, recently they made AR support their products. Users can see animations or other forms of shapes they build using bricks. This idea is the main inspiration for Sky Reacher. It will integrate Bricks and AR into one product, not separate products.

However, in Sky Reacher, the form used is not bricks, but blocks. Blocks have basic shapes that are usually played by pre-school children. The advantage of this game is that it teaches problem-solving and imaginative. This game can be played alone or by the group.

The main feature of Sky Reacher is an animated game that can be seen directly on blocks without using a smartphone. So children can enjoy the game instantly without holding their device. The goal is made to motivate them to build the highest blocks possible. And challenges are inserted in a random time count on random blocks as well like the game in general, which has goals and challenges.

The process that occurs starts with the reading of markers on the blocks, then emits visual animation and games on the same block. Each block added, the visual will change. The essence of this product is a game that uses no screen at all but still has dynamic visuals.

See this:
  • Genc, Z. (2014). Parents’ Perceptions about the Mobile Technology Use of Preschool Aged Children. Procedia - Social and Behavioral Sciences, 146(C), 55-60. doi:10.1016/j.sbspro.2014.08.086
  • Koralek, D. (2015). Ten Things Children Learn From Block Play.
  • License Global. (2018). LEGO Brings its Bricks to Life with AR App.

children toys game blocks ar

Project inspiration - Musician

Zhuoran Li - Mon 2 March 2020, 8:10 pm
Modified: Sun 8 March 2020, 9:40 pm


Music is always connected with waves. Either in a music editor or music player, there are always waves or different height squares to show the music's key.

For instance, Microsoft media player was once the most popular player and it used many different colorful patterns to display the music in a visual way.

Imgur Imgur

In the short film "Secret chord", the character is assessed by playing music. On the screen behind him, the music is displayed as waves. Considering the support from materials science, what if the waves can be out of the screen and tangible? What if people can actually touch the waves, play with it or create music by moving the waves?


This is how I came up with the idea of "Musician". People can just pick up a string or a wire to play music. Ideally, the material should be unlimited.


There are not many limitations of "Musician". People pick up a string, fix it on something, maybe the wall or the table. Then twist it to different shapes.

The shape of the string represents the melody of the music.

The fixture can be the beginning or end of the music, or it can represent a stop in the music.

different colors of the string represent different instruments. For example, the black string has the sound of the piano and the blue string sounds like a cello.

To interact with the string, there are three modes.

  • play:

When people touch the string, it makes sound and the volume depends on how hard people press it, the speed depends on how fast people move their fingers.

  • create:

Pick up a string, twist it to any shape, generate a piece of unique music by using the string.

  • explore:

Users are encouraged to try different colors, different combinations, different patterns. They can even try to change the status of the string, for example, pour some water onto it or put the string in different height. There should be an endless possibility.



To use it

Imagine some children are enjoying a birthday party and they want some music. So the grandma gives them some balls of wool. The children begin to play with the string on the floor. They use pins to fix the wool. It might be dangerous, so they decide to use stickers. One of the children fixed the beginning of the wool. Two of the children are trying to play the sound together. Another child is creating new music by using two strings, thinking about how music can be happy and enjoyable. The last child is bringing in a new color of wool. As the different color gives the sound of different instruments, the child is thinking where is the best place to let the sound come in.


Imagine another scenario, a person use the string to reproduce a piece of famous music for people who cannot hear. Through the shape of the string and the vibration, those people can feel how the music been played and they can even play by themselves. This can be a whole new experience.


To develop it

  • As the string can generate vibration, there is a possibility that it can be linked to the artificial cochlea and help deaf people play or listen to music.
  • What's more, music can invoke memories, so the "Musician" can also help people getting back special memories.
  • There are many "waves" in nature, mountains, ocean, and horizon. The "Musician" can freeze the beautiful scene and shows the music generate by nature.
  • As the color represents for the instrument, and color can also show the emotion, the emotion and unique sound of the instruments can be connected somehow.




music wave:


Secret Chord:

music visualization music without limitation children friendly encourage exploration