What I have done?

This week, I still focused on the game function of my project, I created four functions to control the movement of the character in the game: moveWest( ), moveEast( ), moveSouth( ) and moveSouth( ). I assigned these four functions to four different punch cards respectively, so now I can control the movement of the character by swiping different command cards and insert the function name in to the IDE. I can also wrap the commands with for loop and if else sentences to achieve different tasks.

I also start building my portfolio website, and I also made some illustrations for it. these illustrations are used as the title pictures.

What to do next?

In the next stage, I still have some tasks to do:

• Set a victory trigger for the game
• continue making my portfolio
• start writing the final team report.

What I did?

In my last prototype, I can use the punch card reader to the punch cards and insert the data into the web based IDE, so this week, I redesigned the interface of the website and start to build the web-based game. This was built by react and redux, now I can use keyboard to control the movement of the character and I also made a map for this game, as it shown in the picture above, I added some rocks and trees in the game window, and the character cannot walk across these objects. what users should do is to control the character to get to the destination. I also added a console window under the game window, so when users run the code, if there is any error occur, the error information will be directly shown in the console window, which will help users to debug by themselves.

What I will do next?

The game function still need a function to trigger game victory, and I still need to link the IDE with the game, in order to let the users control the game by punch cards. as for the physical part, I want to add some physical buttons to trigger the function of the buttons on the browser, so that users can focus on the physical devices instead of clicking the buttons on the screen.

Contact

On this week's contact, we made group critique on other group's videos and report. As for my individual concept, I also get some feedback from other teams.

feedback1: "We feel that this concept is rather complex for a beginner coder, and teaches users how to code via trial and error, rather than understanding the actual effect the functions they are using have. "

reflection: the demo I showed in my prototype video is not the final version of my work, actually the purpose of my work is to help people understand the logic of programming. maybe in the next stage, I can add a normal mode, so that the user can use the keyboard to tap the common code to control the game characters, so that users can experience another programming learning method, and they can learn how to code by trial and error.

feedback2: "you’ve incorporated binary into it, which allows for effective translation between the punch card holes, and the code that it inputs. However, similarly to the previous point, this may also be difficult for beginner coders, who are unlikely to have basic binary knowledge, and may present a rather steep learning curve as they are learning both unfamiliar code and binary. "

reflection: I didn't explain him clearly in the video, actually the binary code I mentioned in my video is just used to explain how the punch card reader works, users can easily operate the punch card reader without the knowledge of binary numbers. In my final work, all punch cards will be printed with card information, which can help users identify each card.

feedback3: "in the experience process, these things we make ourselves can more intuitively express what we want to express. For example, we can let children make their own punch card, and use the card they made to let the device complete some different operations."

reflection: This is a good suggestion, let the children make punch cards by themselves to increase the fun of the works. But this function is not very helpful for programming learning, it can only be used as a supplement to the fun of the work.

feedback4: "Is it possible to add some other functions. Because I know that Arduino kit contains many small parts. If you add something on the device such as touch switch-control the device to turn on and off. 8x8 led display- used to display some simple expressions. After all, this is a device made for children, and making the device look more interesting in this way can arouse their curiosity."

reflection: It is a good idea, maybe I can make two physical buttons to replace the two virtual buttons on the browser.

Contact

On this week's contact, we reported back what we have done. As for me, in week9, I added a LCD screen to show the data of the card we swiped to help users check the card content more conveniently. I also added a LED strip on the punch card reader, and each time users swipe the card, LED lights will flash different colors according to the content of the card, which improved the interactivity of the device.

What to do?

I will prepare for my prototype pitch and finish my report.

After that, I will start to finish the game function inside the browser and link it with these physical devices.

What I did?

After I got the punch card reader idea last week, the main task I did this week is searching for the useful informations to achieve the function. My original idea is using the punch card reader to read the programming statement fragment of each card, and then these code will be sent to phpMyAdmin, and the front-end can fetch the data and show them on the browser. But after I searched for a lot of tutorials, I found that linking the Arduino with database cannot be achieved without ethernet shield, so I borrowed a ethernet shield and ESP 32 from Lorna, and I hope the function could work.

I tried to link the ethernet shield with my arduino board, but some errors occurred when I upload the sketch to my arduino. (avrdude: stk500_recv(): programmer is not responding) I have no idea why it happened. So I searched for the reasons of this problem, and I found that the problem may have occurred on pins 0 and 1 of the ethernet shield board, and a short circuit may have occurred, but I did not find damage on the board, so this problem has not been solved. I asked tutor about this question but it has not been answered yet.

So, I have to find another solution to solve it. My friend Wally gave me an idea that with serialport and express library of Node.js, I can read the Arduino data and send it to the browser and I did it, the data wiwas transfer to the terminal, and I can use express to send it to the localhost api, and use axios to fetch the data to the browser.

The front-end interface is like the second picture(have not done yet, the right part is game window...to be continued), when the user swipe the punch card, the code will show on the web-based IDE, and they can press the 'Run' button to use the code to control the game on the right part of screen.

Although the transfer data function is almost done, the punch card reader is not connect stable, so I plan to use the machine in UQ Innovate to build a better one. So on Friday, I made a reservation for the UQ Innovate, and built a wooden punch card reader. The first picture is the previous version. The material used is mainly cardboard, so it is not very sturdy and the connection is unstable. The second picture is a punch card reader made of wood, the recognition accuracy has been improved

What I will do next?

In the next session, I will start to build the web-based game, and connect the software part and hardware part together and I also need to prepare for the prototype video and report on 11th May.

Contact - Report Back

this week, we still have a report back session in our contact class. What we need to report is what we did during the mid-break holiday. As for me, my concept is to make an interactive coding game with Arduino and react. My idea is to connect the browser and Arduino, the game interface will appear on the browser and there is a coding Ide on the left of the screen with some coding instructions, and on the right of the screen is a game window, and you need to use code to control the character. The motor of Arduino will be connected to the computer. I will use the motor to control the movement of a card character. When each time you pass through a level in the browser, the Arduino paper character will move forward.

The picture below is the plan of my individual project:

Now I have completed the first milestone, so during the mid-break, I have did some research on the arduino and figure out how to connect the arduino with web browser. The way I followed to connect these two things is a nodejs library called socket.io and I can now control the motor by clicking the buttons on the browser.

After the report back I got some feedback from tutors, Clay thought my concept is not physical enough, because I just use the arduino as a feedback of the game, the main interaction is inside of the screen. So I have to change another ideas.

Thursday Workshop

This week's workshop is used to Q&A, we can ask any questions about our project and we can get some useful answers from Ben and Steven. Because I have to change another physical interaction in my project, so I just asked steven for any suggestions. Steven mentioned an interesting idea called 'punch card', which is a way to enter information physically, people can insert a small card with holes into the card reader, the card reader can recognize the content of the card and enter it into the computer. The 'punch card' video is shown below.

I tried to make a punch card reader with paper and some paper clips.

Although the recognition is a bit insensitive, the data can be read.

Report Back

On this week's contact session, we report back our progress of our current idea. As for me, because in our group, we separate our group project to individual project, so I just mentioned about my individual ideas and my next plan for my new idea. My new idea will focus on how to guide the children to learn the programming, and I want to combine the coding learning with a manual or user book, which is inspired by the coding game Shenzhen I/O. I also mentioned my progress of learning Arduino.

Methods for discovery

The other activity we did is to fill a Miro board to define our concept thoroughly, by making the tree diagram, we can make sure we all agree on every details of our concept. In this chart, we explored our concept from two aspects : The Thing and The Context. The thing mainly focus on the form and function and the context mainly focus on the target audience, time and place.

Arduino Tutorial

On Thursday, Ben and Steven taught us a lot about Arduino via zoom. Although I used the Arduino before, but to be honest, I still confused about how to link the circuit and interact with other software. I will keep learning on it and try to find out how to achieve the main function of my concept.

Week5 Contact

On this week's contact class, we have to report back the works we have done and what improvement we added to our new concept based on the feedback of the last week's critique. Each of out group members attended the report back session, and after we introduced our current situation, we got a lot of feedback and suggestions from tutors and our classmates.

Critiques and solutions

Last week we have got a lot of feedback from our classmate.

critique one: we need to explore the gap between plaint-language and coding language.

solution: In our first idea, we want to use the simplified coding language to help the users understand the logic of programming like if..else, do.. while, etc. According to the feedback, we will choose a specific programming language like python or java and let the users practise this specific language in an interesting way.

critique two: 'I think keeping the robot in the maze is limiting'

solution: we admit that the maze idea is limited and not interesting enough, so we want to change the game to a box pushing game, which allow the users to control the robot by coding and push the box to the correct place. That is what we thought currently, maybe after our deeply communication, we can find more interesting ideas.

critique three: This would allow for the robot to modify its own code being able to correct your code.

solution: Actually, we have not find out a good way to solve this problem, but I think we just want to make our game an interesting way to help them learning, we don't need to let the robot modify their coding errors, we can find another way to let the users find out the problems by themselves.

About my new ideas

For this new idea, it is still a multi-player game, the two robots was placed at two different sides, and the destination is in the middle. We will let the users to fill the blanks in the coding questions to help them understand the syntax of the programming language, absolutely the difficulty of this game will gradually increase from easy to hard, so you don't have to worry about it. When the player answer the question correctively, the robot will move forward and if they didn't complete the question, the robot will go backward, so the player who get to the destination first will win the game.

Work to do

This week I have got the Arduino kit send by Lorna, and I have started to do some research on it. so the next task for me in the next week is keep learning arduino and find a way to solve the technical problems.

Week3 Zoom meeting

In week4, Lorna had an online meeting with us to give answers to our confusions about the changes of our physical computing teaching plan. Our confusions mainly focus on how can we build our project individually at home without tools like saws, laser cutter, etc and what scale our individual project should be compared with group project. Lorna gave us accurate and patient response, which can help us make a plan for the rest time of this semester.

Week3 Group meeting

After the Zoom meeting, we conduct a group meeting focus on our current idea and prepared for the presentation of next week. In the previous meeting we conducted last week, we have already decide our idea, which is a creative learning game for teenagers to help them learn coding in a more enthusiastic way. So in order to help the audience understand our idea in the presentation next week, we communicated about the structure of our slides and divided the work.

After a long time discussion, we decided to make our slides according to the structure below:

Introduction of project and team in general

• Team’s name:BEE BEAR LION
• Background:Problems solving
• Inspirations:Chessboard at CES 2019
• Purpose
• Target Audience
• Concept
• Storyboard
• Plan
• Prototype/Body-storming
• The body storming part was assigned to Bowen, and the other three group members's task is making the presentation slides. The part I should make in the slide is about our concept and story board. Finally I have to make a video to combine all our works into one video and upload it to the youtube list.

We also prepared three questions at the last page of our slides, these three questions are all about technical problems:

• How to make the character move and turn right/left?
• How to make the camera detect the QR code (Marker) and translate it into code in our computer?
• Which programming language can help us control the movement of robots? Can we use Javascript?

Week4 Contact

In the two contact of this week, we present our idea on the class. the contact class was conducted online via zoom. After our presentation, we got a lot of feedback from our classmates, and we summarized the following points from the feedback:

• Explore the gap between plaint-language and coding language
• Change the target audience from age 9-12 to the people who never learn coding language (the age is not enough specific for our project, cuz there may a lot of children already had the foundation of coding)
• Maze — Box moving (From Zebing Yao, Clay 'I think keeping the robot in the maze is limiting')
• This would allow for the robot to modify its own code being able to correct your code. (From Clay)

So in the next session, the first change of our concept is that we are going to change the idea of the maze into a push box game, which let the player control the robot to push the box to the specified position by coding.

As for the robot, we have several ideas to achieve the function.

• The first idea is using the Nintendo Labo robot as the character, this robot can move forward, backward or turn left and right by the vibration of the joycon. But this robot is not programmable, so this idea was abandoned in later discussions
• The second idea is using Anki vector smart robot, which is programmable and is very suitable for our current concept.
• The last idea is using Arduino to build our own robot.

Week4 Zoom meeting

In week4, Lorna had an online meeting with us to give answers to our confusions about the changes of our physical computing teaching plan. Our confusions mainly focus on how can we build our project individually at home without tools like saws, laser cutter, etc and what scale our individual project should be compared with group project. Lorna gave us accurate and patient response, which can help us make a plan for the rest time of this semester.

Week4 Group meeting

After the Zoom meeting, we conduct a group meeting focus on our current idea and prepared for the presentation of next week. In the previous meeting we conducted last week, we have already decide our idea, which is a creative learning game for teenagers to help them learn coding in a more enthusiastic way. So in order to help the audience understand our idea in the presentation next week, we communicated about the structure of our slides and divided the work.

After a long time discussion, we decided to make our slides according to the structure below:

• Introduction of project and team in general
• Team’s name:BEE BEAR LION
• Background:Problems solving
• Inspirations:Chessboard at CES 2019
• Purpose
• Target Audience
• Concept
• Storyboard
• Plan
• Prototype/Bodystorming

The body storming part was assigned to Bowen, and the other three group members's task is making the presentation slides. The part I should make in the slide is about our concept and story board. Finally I have to make a video to combine all our works into one video and upload it to the youtube list.

We also prepared three questions at the last page of our slides, these three questions are all about technical problems:

• How to make the character move and turn right/left?
• How to make the camera detect the QR code (Marker) and translate it into code in our computer?
• Which programming language can help us control the movement of robots? Can we use Javascript?

Week5 Contact

In the two contact of this week, we present our idea on the class. the contact class was conducted online via zoom.

First Team Meeting

This Friday, we held our first group meeting, and before the meeting, each team member prepared at least three ideas, and by discussing together, we finally come up with an idea related with programming learning.

Final concept:

• Board Game with Block (Contains Script/command)
• Characters on board need to move Based on the script
• Players need to finish the script first before the character can be played. Players need to press the start button.
• This is a race game(multi-player)

Purpose:

• help people learn basic programming
• improve people’s logic ability
• cultivate the coding interests

Audience:

• Teenagers
• People have interest with programming

Description

Our final idea is to help teens and coding enthusiasts learn coding in a creative way. We will make a maze from cardboard or wood, and players can control the robot to walk in the maze. The control method of the robot is to arrange and combine different instruction blocks. Depending on the order of the instruction blocks, the robot will give different feedback.

Innovation

This week, I take part in the UQ innovation tutorial, which mainly help us learn how to use different tools to create items. In this process, we will get in touch with many kinds of tools like saws, drills and laser cutter. The instructor give us many advice like when we push the wood close to the saws, we have to be careful with our hands and remember to clean the dust on the floor when we sanding the wood. At the first half part of the innovation course, the l can easily understand how to operate the machine and the safety requirements, but when we start to learn laser cutter, I have a lot of confusions. The laser cutter is like a printer, and the acrylic materials are like papers. When you finish drawing a picture with Adobe Illustrator or AutoCAD, we can change the outline colour of the shapes, and each colour represent different depth the materials will be cut. The steps of operate the laser cutter is really complex and I tried my best to remember all the steps, but I'm still not sure whether I can operate with it by myself next time.

Tuesday Contact

On the contact this week, we did the world cafe, which is a good ideation method for people to brainstorm. In world cafe, each table will have four to five students communicate on one topic, and every five minutes, one person should stay at the table as the host and the other people should find another table and take part in the second round communication. We discussed about different ideas from context and target audience aspects, and give them some refine suggestions.

Wednesday Contact

On the contact of Wednesday, we take part in a guest lecture about how to find a job after graduate in Australia and I learned a lot about preparing resumes and some useful working experience. The other thing we did on this contact class is that we formed our team based on the theme we choose. The topic allocated to our group is about creative learning, and we come up with some interesting related ideas.

• Coding education
• Cooking
• Instrument
• learn circuit
• Teach dog how to use toilet
• Code Battle Similar to 7 Billion Human
• Reservoir Leak (Based on Plumber Game in Android)
• Battle of Element (Based on Little Alchemy and Card Game Like Owen Said)
• Combine AR with circuit components card to help people learn circuit
• Learn how to draw a picture
• Learn natural elements by the AR and cards

Work to do

In the next week, we will have presentation about our final idea.

Work Done

Soldering

On Tuesday this week we took part in the soldering practise, which taught us about how to using the electric iron to solder the Arduino components (like the bulb, switch and battery) on circuit board. At first, the tutor provided us with a circuit diagram which is about how to use the switch control the on and off the light. Then we started trying to connect the circuit on the Arduino board, although I come across some problems at the beginning of the class, but with the help of our tutors, I gradually remembered how to connect the circuit and the bulb is lighted successfully. Then we have to move all the components to the circuit board and soldering them together. The whole process is interesting and I learnt a lot.

Presentation

On Wednesday we had a presentation session on our contact class, every student bring their ideas and shared it with the other students. Each of us has two minutes to present our idea and answer the question. My concept is a kind of shooting game but you can interact with it by speaking. This idea allows users to turn their words into bullets to shoot the enemies.

Work To Do

• In the next week we will have a world cafe to communicate our ideas and vote for the top three themes. Based on the theme we vote, we can find our group mates in the next session.
• Next week, I will also have an innovative and induction course.

Speaking shooting game

---

A3 Poster

Design Concept

This device allows users to turn their words into bullets to fight with enemies. It consists of two important components, one is a sound sensor and the other one is a 360-degree curved screen. The sound sensor can detect the player's sound source angle to ensure that the word bullet can accurately hit the enemy. The player can stand at the center of the curved screen, so that the enemies will come from all directions, you need to be alert.

Description

At the beginning, you will find the enemy come from different directions, You can turn words into bricks or bullets to shoot enemies in the distance by speaking. The direction you speak is a factor that determines whether you can hit the enemies. When the number of enemies is higher or the life is higher in the late stage of the game, you may need to defeat them by rap or fast talking.

Target Audience

• people who feel stressful
• game player

Inspiration

This device is inspired by comic books. In comics, there are often many sound effects words, which is a manifestation of comics. I want to visualise the words and use them to achieve some interesting interactions with people's voices.

#game #sound interaction #relieve stress

Work Done

As for the first week of the physical computing course, unfortunately, I missed the contact class of Tuesday and Wednesday because of the travel ban of corona virus. After the class, I have read through the course slides and understand about what we should do and our current tasks. In these days, I have read the chapter of Challenge 3: Ethics, Privacy & Security, and I also did some ideation and come up with some useful and creative ideas.

7 HCI Challenges

The chapter I read is about ethics, privacy and security, in this chapter, the author mainly talked about the ethic and privacy challenges from six different aspects:

• Online Social Networks (OSNs)
• Healthcare technologies
• Virtual reality
• IoT and big data
• Intelligent environments
• Cybersecurity

Ideation

First idea: Water harp

About me 👋

---

Hi, my name is Wentai Ouyang, and you can just call me Owen. I'm a postgraduate student major in master of interaction design and this my last semester of my study. As for the aspect learning area I am interested in, the first thing is front-end web developing and the other one is UX design. I think the Physical computing course is both interesting and challenging, and I will try my best and I will invest enough time and energy to get the project done.

Expectations 🔭

---

About this course, what I feel is excited. I think in the last semester, this course can be considered as a good chance to examine the abilities we have learned in the previous three semesters, and I think with the accumulation of the last few semesters we can handle with the problems we faced with successfully. There are also somethings I have to concern about, as a 4-credit course, not like any other course we have learned, this course need us to involve more time and efforts, so what I worried about is whether I can allocate working time and complete tasks in a planned way.