Documentation & Reflection

Exhibit

During the exhibition, I have watched many interesting and meaningful concept and our team have demonstrated our concepts by using three teddy bears, each of us were working on different learning content and these contents focus on incorporating multi-sensory learning that encourages children to use more than 1 senses to engage with learning. Each prototype utilizes the sense of touch, hearing and vision to assist with children’s learning experience. We got many positive feedbacks and many audiences thought our concepts can effectively help kids learning in a funny and interactive environment, and develop their motor skills, and they thought it was an interesting way to present our concept. We also got many suggestions on how we can incorporate our concept together. Some of them also mentioned the tone of the audio is not friendly enough and the use of the word is too complex for the target age group.One of them has mentioned I need to deal with the incorrect response and no response situation. I think I need to consider these issues to make the prototype more friendly to the kids.

This week I was working on the demo video of the final product and develop the portfolio website.

Demo Video

Product Test

I have tested the usability of the final product with my friends, they mentioned that they can easily understand overall interaction flow and the prototype is quite interactive. However, some of them said the colors were not enough for children to learn color, it will be better to add more colors. Furthermore, the size of the toy can be bigger so that there can be more holes in the toys and the math question can be harder in different levels of the game. And the backpack is a little bit big compare with the size of the teddy bear, one of them suggests I can make a hedgehog toy instead so that users can directly insert the sticks on the back of the hedgehog instead of inserting sticks inside the backpack. Some of them also mentioned the current game process is simple and there can be more interactions. I think it will be a better user experience when combining four of our team members' features together and ITZY will be able to teach children different learning topic through more inputs and outputs

Reflection

ITZY aims to allow users to learn at a more comfortable pace with more than one interactive input and this allows for a synchronized development of the brain. Targeting both the creative and analytic part of the brain. My final project allows users to learn math and colors with multiple inputs and outputs (Insert stick input, shake hand input, audio output, led light outputs) and create an interactive environment for children aged 3-5 years old to learn math and colors. Instead of using traditional math learning methods (eg.teacher-centered instruction), I let kids interact with the toy and learn math through a visual approach. It also helps them develop motor skills by inserting sticks inside different color regions.

This week, I was working on assembling the final prototype for the exhibition. Our group decided to let each of us use the same toy during the exhibition. We bought the same teddy bears and work on different learning topics.

I made a backpack which is attached to the toy and users can insert sticks inside different area of the backpack. I have drilled holes on a box and painted colors on the different areas of the box. The speaker and pressure are put inside the bear, users can shake the bear's hand to get auditory feedback.

I was focusing on teaching children math and color. When the game start, users can insert sticks in different areas of the bear's backpack based on the given auditory instructions. The sticks will trigger the button inside the box and give further instructions depends on users' responses. Users will be given math questions based on the number of sticks in the backpack. This process help children learn math through a visual approach.

Visual approach

I was also working on building the frame of the website which will be used in the exhibition.

Next week, I will be working on taking videos of my prototype and put them in my website presentation, also further develop the website and make users easily understand my concept, prototype mechanism, and prototype interaction.

Final circuit diagram

Work done

This week, I was working on improving the quality of the audio output, I have changed a speaker and use DFplayer to store the mp3 file, and play the audio through it. I also use the expanded board to put more buttons and let the button to trigger the audio output.

Plan

Next week, I will work on assembling each component and making an actual toy based on them. There will be a teddy bear toy or turtle toy and the board will be put inside the toy. The speaker will be on the head part of the toy and the pressure sensor will be placed on the hand part of the toy. The toy will be divided into different regions with different colors. Users can put blocks into different color regions and they can learn math and color by following the auditory instructions.

Research

Impact of utilizing multimedia technologies on enhancing the effectiveness of teaching children at early stages.

Multimedia technologies

Learn math through visuals approach can enhance students’ understanding of mathematical concepts.

Visuals approach

Interview link:

Interview

Critique

Our group has done critique of other group's prototype. We have watched the video and discuss the strengths and weaknesses of each prototype. There are some interesting concepts such as the group Hedgehog, the idea of developing video games through auditory output is good, the game is engaging and funny. However, some of them have not presented much research which is related to the problem space.

Link

Work done

I previously make the audio output by encoding audio data, but the Arduino IDE doesn't support the file size of encoded audio data and the audio quality is not good.

To improve the quality of the audio output, I used DFplayer module to store the audio file, and it loads the audio data to the speaker module, it helps increase the storage and the audio quality is improved.

Work done

This week I have combined all my features into one board and test the functionality. After that, I put the Arduino board inside a box and designed the box into different colors. The box can be seen as my first prototype which will be used for testing in the future.

Interaction

When the prototype is activated, the speaker will give auditory instructions to the users and let them put blocks into different color regions of the prototype. User needs to put correct numbers of cubes in correct position. If users respond incorrectly or do not respond, the speaker will give further instructions.

After that, users will be given a simple math question based on the number of cubes in the prototype, they can easily answer the questions by counting the number of blocks and they need to do correct responses to answer the questions based on the auditory instruction. In this case, users need to press the pressure sensor, and then the speaker and LED light will give both auditory and light feedback if user responds correctly.

Next step

I found that the auditory output is not very clear during testing, and the Arduino IDE does not support a large amount of encoded audio code, In the next sprint, I will improve the audio quality by putting the audio file in an SD card and using DFplay component to play the audio. I will also make the prototype more playable by changing the appearance and adding more inputs based on feedback and research.

This week I was working on different features of my project using Arduino. I have built the multiple buttons input last week that let users count the number by inserting the stick.

To make the output, I want the kids to hold the toy bear's hand, so I tried to use the pressure sensor to detect users' motion.

In order to make the prototype more interactive, I decided to add audio output rather than display the instructions and questions on the screen. I came to the Jaycar and bought a mini speaker and I tried to encode my audio file an get the code that can be used in Arduino IDE. However, auditory feedback is not very clear.

Next week I will work on combining these features together on the same board and make a box to contain the whole circuit which makes the prototype more playable.

This week I was building my prototype inputs. My individual focus was to help kids learn math. I came to a toy shop to find some inspirations and I decided to start working on the multiple buttons inputs. Since I am going to let kids inserting the sticks to learn number fist, so I can use buttons as the insert inputs. Once one button is pushed, the value on serial monitor will count up.

Next week, I will start work on the voice input and output if the auxiliary kit arrives. It will be used to give auditory feedback to the kids and let them know the questions and instructions easily.

Individual exploration update

Last week, I was thinking about building a turtle toy as my individual prototype, and that aims to teach pre-schoolers to learn some easy math.

This week, our group has decided to let each of us work on one project and everyone focuses on different learning goals. I think it was a great idea to compare with everyone works on individual prototype. Since there will be more communication among us and we can share our research or code. Furthermore, it will be better that our users can learn different topics on one prototype since users may feel bored when they are leaning a single topic for a long time, ITSY allow them to learn color, math and shape at the same time and also improve their motor skills

I aim to create an interactive learning environment for children to learn math. Instead of giving questions and letting children answer it, ITSY will give instructions to kids by using voice output, let them count the number of items with different shapes and colours that they have collected before and ask them easy math questions based on that. For example, users need to count the number of items with round shape and items with red colour, there will be a math question which is related to these two numbers such as ‘shake the bear’s hand if ‘3+4’ equals 7’. The question will be voice output as well. Children can easily answer the question because they can see how many items they have just counted. So that the toy provides visual support for children who start learning math. Users need to do correct responses like shaking the toy bear’s hand, then the bear will give light feedback if users do correct responses.

Meeting

Our team was working on building our Miro board based on our concept. We have explored our concept by finding the strengths and weaknesses of our ideas and analyze different context of use for different users.

we also drawed another mind map to explore the functionality of our prototype, help everyone understand what our prototype will be look like.

Workshop

During workshop session, tutors showed us how to build simple circuit and connect it with Arduino. It was quite useful since I am going to use Arduino to build my prototype such as using the motion sensor to make the shake bear hand function and use speakers to implement the voice output.

Zoom meeting

Our team has analyzed the feedback we got during the pitch and find some ways to improve our concept. We decide not to use the website to display questions since it does not involve many interactions with the kids and some kids are too young to read text from the website. We are going to improve our input and output based on our research on HCI for children. Furthermore, we need to further specify our target audiences and learning topics.

We have come up with some personas for different users and do more research on teaching and learning methods.

Individual focus

The target users that I will be focus on are pre-schoolers who start learning some easy math. I am going to make a turtle toy and let children improve their motor skills, also learn math and color at the same time. The turtle toy will mainly consist of three parts, a turtle shell with different color regions that let users insert sticks in it, turtle feet that let users to shake and turtle’ s head with a light and speaker on it.

There will be a display screen to tell users how to play with the toy, for example, the screen will display some icons that tells user put 3 sticks in the red area and 4 sticks in the green area, the speaker on turtle's head will also produce auditory feedback about the instructions. I choose to use icons and voice output since users can easily understand what they should do and will be more interactive. Users need to insert certain amount of sticks into different area of the turtle shell and that help them learn numbers and colors, after that, the screen will show some easy math questions like ‘shake the turtle’s green feet if 3+4 equal to 7’, children can easily answer the question by seeing the number of sticks that inserted in the turtle shell. So that the toy provides visual support for children who start learning math. The light on the turtles’ head will turn green if users do correct responses.

Research on domain

Children with autism

Special case: Interaction in children with autism

HCI for Children with Disabilities

This book provided a set of methodologies, architectural models, examples of interaction with children with disabilities, case studies, and usability considerations in the context of disability that focuses on providing the reader with the elements required to build human–computer interaction systems thinking in children.

Research on learning method

Multisensory instruction

Multisensory instruction is a way of teaching that engages more than one sense at a time. User for problems like: Dyslexia, Dyscalculia, Dysgraphia, ADHD

Universal Design for Learning (UDL) Strategies

UDL is a type of teaching that gives all students flexible ways to learn and succeed. UDL strategies allow kids to access materials, engage with them and show what they know in different ways.

Meaningful play

1. Gives the child a choice about what he or she wants to do
2. Feels fun and enjoyable for the child
3. Evolves spontaneously, rather than giving kids a script to follow
4. Is driven by intrinsic motivation about what the child wants to do
5. Creates a risk-free environment where kids can experiment and try new ideas.

Good-quality apps or games for preschoolers

1. encourage creativity
2. encourage problem-solving
3. develop communication skills
4. develop social skills
5. build on interests
6. have positive messages about relationships, family and life

10 Fun Activities to Do with Your 3-5 Years Old

1. Trace letters together
2. Do puzzles
3. Throwing games
4. Matching shapes
5. Play bingo
6. Create a tin can phone
7. Do a color scavenger hunt
8. Try an information game
9. Play with sensory toys
10. Notice the seasons

Concept

After discussion and research on synchronized learning and motor skills, our group decided to create an interactive learning experience for children with learning disabilities.

Because of electronic devices, children are incorporating less motor skills to learn and play. We also found that traditional learning techniques normally engage just two senses – hearing and vision, we want to allow users with more than one interactive inputs and learn by using their left and right brains synchronously, targeting both creative and analytic part of the brain.

We are going to design a toy with multiple inputs and outputs. For example, users can pull, twist, push and shake the toy to respond to the questions displayed on a website and the toy will give sound and light responses.

For the mechanism of the toy, buttons will be attached to the toy and that lets the user press, there will be motion sensors to detect users' actions like twist pull and shake. Lights will give visual responses to the user whereas the voice module will play sounds and give auditory feedback.

Critique

During the zoom session, I have watched lots of interesting concepts and made some critiques on these.

link of critique

I have learnt a lot from other groups, such as do enough research on users and specify the target users in a smaller range. Some of them did a good job of finding users' needs and came up with many novel interactions.

Feedback

After presenting, We have received lots of feedback. Some audiences mentioned our interactions (pull, twist, push and shake) may be too violent for children.

We also need to determine the age range of our target audience so that we can decide what kind of knowledge exactly the toy is trying to teach. Some students also said the idea of using a website to display questions is not very good, it will be better to make the idea more moveable, moreover, some kids can not read the text on the website.

In order to improve our concept, we think we need to determine the age range of our target users such as children aged 3-5 years old and choose a specific topic aim to teach kids. We also need to improve the way of interaction based on target users, for example, let children hug the bear instead of pushing or pulling it.

Synchronized Learning research:

Since our group decided to work on synchronized learning, I have done some research on that.

brain regions

The human mind can rapidly absorb and analyze new information as it flits from thought to thought. These quickly changing brain states may be encoded by synchronization of brain waves across different brain regions.

There are millions of neurons in the brain, each producing its own electrical signals. These combined signals generate oscillations known as brain waves, which can be measured by electroencephalography (EEG). The research team focused on EEG patterns from the prefrontal cortex —the seat of the brain’s executive control system — and the striatum, which controls habit formation.

Left & right brain exercises(video):

Left & right brain exercises video 1

Left & right brain exercises video 2

Neither side of brain can function independently, both brains work and communicate with another.

how brain work

Later research has shown that the brain is not nearly as dichotomous as once thought. For example, research has shown that abilities in subjects such as math are strongest when both halves of the brain work together. Today, neuroscientists know that the two sides of the brain collaborate to perform a broad variety of tasks and that the two hemispheres

"The pop psychology notion of a left brain and a right brain doesn’t capture their intimate working relationship. The left hemisphere specializes in picking out the sounds that form words and working out the syntax of the phrase, for example, but it does not have a monopoly on language processing. The right hemisphere is more sensitive to the emotional features of language, tuning in to the slow rhythms of speech that carry intonation and stress."

brain collaboration

Temporal synchronization is believed to be the main mechanism of information binding, (i.e., segregation and integration) in the brain. Neurons in different regions of the brain get into synchrony, and as a consequence, information is processed in the brain. Synchronization phenomenon has a key role in various cognitive functions of the brain, and various brain disorders have been linked to abnormal levels of synchronization. For example, Alzheimer’s disease is often associated with loss of local and global synchronous activity between brain regions. Schizophrenia—as another cognitive impairment—is linked to altered synchronization levels in the brain. Autistic patients have also shown lower synchronization values across various brain regions compared to healthy controls.

various cognitive functions of the brain

Game-based learning:

introduction of Game-based learning video

Video:

Bash told us about his experience and gave us much advice on how to get a job, such as building network and finding recruiters. We also had to write 50 words about ourselves，find the uniqueness of ourselves and finally choose one word to summarize. It was a quite meaningful activity, I haven' t thought about my uniqueness before and I end up with using "optimistic".

Team:

After the video chat session, we formed our group. I was excited to be a member of "half ice no sugar". We were going to work on the theme of "creative learning". We started to think about our concept after we know each other. We were interested in the concept "Morse code" and we had explored this concept, thinking about making a language learning platform. Fardeen also introduced a game called 'Bop it' to us. It is a game designed for kids with multiple inputs. And we were going to design a game which also has multiple inputs like 'Bop it', for example, use pull, twist, push, flick, smash as inputs which allow kids to develop their synchronized learning ability.

In class activity:

Our project ideas had separated into different themes, and we were divided into different groups to further explore these themes. And I had been working on sassy tech, bothersome tech, creative learning, and the body as controller. These are all very interesting themes, we had explored the target users for these themes and developed more new concepts based on that.

This activity not only helped me get familiar with different themes and concepts but also let us get to know other students. After that, I decided to choose creative learning as my first preference and bothersome tech as my second preference, since I thought people are willing to learning things in an interactive way. And bothersome tech was a quite interesting area, my friends and I have come up with many interesting ideas on that.

Presentation:

The week, we were having our 2 min pitch during the contact. It was good to see others' concepts. Some of these are really great and simple concepts that match all criteria very well. We also wrote down critiques to others' concepts which help us to improve our concepts in the future.

link to the critiques sheets.

I also got feedback from Lorna. She gave me some advice about the interaction of my game. Not only project shadow on the screen, there can be more ways of interaction such as sound or heat effect, users need to squat down when they hear the sound or feel the heat. The game can also motivate players to do other exercises such as jumping. One of my friends also mentioned my concept is not novel since she had played a game with a similar technique. But I thought the purpose and interaction of my game are different.

Soldering:

I haven't done any soldering before and I have limit experience with building the circuit. But this session was running pretty well because tutors gave enough help. I think soldering is very useful for us to design an interactive project if we are going to implement sound or light effects.

Idea:

'Squat deep' is an interactive game for people to do strength body exercise easily and effectively

Some people feel painful and tired when they are doing exercises in the gym and sometimes they may feel bored. This game aims to motivate people to do squats effectively and create an interactive environment for people to do exercises.

The game interface and players' shadow will be projected on a screen. There will be different balls move from one side to another. Players need to move their bodies to control the position of their shadow on the screen, use their shadow to catch the black ball on the screen and they can get one score. Players also need to squat down to avoid touching the fireball, if the player touches the fireball, they will lose one health point. By doing that, users can do squats while they are playing the game, the more scores they got, the more effective squat exercise they have done.

The game can be set up in the gym for customers to play, and the game can be multiplayer, for example, customers in the gym can play the game using different screen and see who gets higher scores.

The game can also be set up at home and allows people to do exercise easily at home rather than going to the gym.

Poster link

Inspirations:

1. One of the inspirations comes from my own experience. I always feel bored when I was doing exercises in the gym and I want to do strength body exercise in a funny way and feel motivated.
2. A previous DECO7385 project- Street Spirit. This project also uses projectors to project human's shape on the screen and users can interact with each other.

Video link

Intro:

I'm Zihan, and I'm the third-year Bachelor of IT student, majoring in user experience design.

I have limited experience with physical computing so that I hope to learn more about hardware implementation during this semester and I aim to build a highly interactively installation at the end of the semester.

7 HCI grand challenges:

Key finding:

Have to find balance between being a helpful addition to somebody's life without completely taking their ability to have control over their own life, and find a medium where interaction is intensive but not invasive.

Example: Black Mirror: Social Credit System

Poker Activity:

Design to conflict kindergarten using air with the quality disgust.

1. Someone bring smelly food to the kindergarten.
2. Kindergarten class takes flight to travel. Children feel disgusted because the plane vibrates because of the strong airflow.
3. Smoke comes into the kindergarten. Kids thought somewhere on fire and struggle coming out of the room.