Last week, there were four main tasks left in our project.

1. Voice control (There are some problems that we cannot solve because Google voice service can only be used in the United States)
2. Difficulty setting (we originally wanted to use different sound effects or storyline to distinguish different difficulty)
3. Integrate all the current work into a whole, which can be used by people.
4. Play sound effects (Sd card, or use unity)

Let me talk about the fourth question. I have tried many methods before. I even want to solder some wires to the sd card with an electric soldering iron, and then connect the Arduino to read the information. But there are many problems involved, and through communication with the tutor, I finally gave up this idea. Then our thinking turned to unity. We have established communication between Arduino and unity. In other words, we can directly use unity as a platform to receive Arduino signals through unity and give feedback. The feedback includes various sound effects and prompt sounds.

Voice control update

I am currently mainly responsible for voice control by using Bluetooth or wifi. The previous idea was to use an Android phone as an input terminal (because we failed to buy an Arduino speech recognition module). There are two benefits to using an Android phone. First, the Android phone, as a flexible smart platform, allows users to connect Bluetooth headsets (the headset has a mic). Second, we have the Bluetooth communication module JDY-16. This module will enable us to establish a communication connection between the mobile phone and the Arduino in a faster way. With this theoretical foundation, we try to use Android phones as part of our voice control. We downloaded the Arduino Voice Control app, which allows us to connect our Bluetooth communication module using a mobile phone. After we configured the Bluetooth module, the phone was able to connect successfully. But there is one problem that needs to be solved: Arduino Voice Control needs to cooperate with Google voice service to work.

New plan of Voice control

However, Google Voice is currently unavailable in Australia, so we have encountered some problems in this part of our voice control. Fortunately, we found an app that is very suitable for our use. Blinker is an IoT access solution designed to allow people to DIY their own IoT devices. ios, android Both support, local and remote support, Bluetooth, and WiFi support, you can drag and drop layout device control interface and easily create your IoT device. In other words, this is an app that can support people to connect their own smart devices. Because before this, I have written some code related to Bluetooth. And I'm sure our Bluetooth module is working correctly. So our first test was very smooth. Blinker was able to connect to Arduino successfully and received the signal from the mobile phone on the computer. By looking at the blinker's user manual, we found that the buttons on blinker support customization and users can use voice commands to complete the operation of a button. So we only need to bind a specific function to a button. Then add voice commands to this button. We can complete the voice control.

We have not much work left. Although there are some problems, in theory, we can find a way to solve them. The most important thing now is to integrate all the work of the four people (now we have started to do this work). We hope to complete this part of the work by Friday. Also, we need to make a shelf with some wood. We need to place some sensors on these shelves, such as laser sensors and ultrasonic sensors. Let these sensors and shelves form a monitoring system, which can always ensure that our users will not be injured due to some unexpected situations.