One of my work is to research for the related work. All the work below was completed during our project proposal, but i haven't described in my journal.They include literature reviews about similar exploration, guidelines for design and some theoretical support, as well as some published games for the blind.
Lahav, H. Gedalevitz, S.Battersby, D. Brown, L. Evett and Patrick MerrittComputing and Technology Team: Virtual environment navigation with look-around mode to explore new real spaces by people who are blind. Nottingham Trent University, Nottingham, UK.Pages 1072-1084 | Received 17 Aug 2016, Accepted 20 Jan 2017, Published online: 10 Feb 2017
The original intention of searching this article is because after our presentation pitch, in feedback some people mentioned a game for the blind people that uses sound and haptic feedback hints to explore the unknown space, such like exploring the maze.I also believed that this mode will bring a interesting game experience, suitable for the theme of our hearing ability-centric theme as well. However, i found that this concept has already been explored. In the article, in a virtual cane system simulating a room similar to the reality, the authors presented a new look-around method which enables the user to explore the unknown by the auditory information without moving in addition to the traditional walk-around way in which the user has to explore while walking. Besides, the authors selected the Nintendo Wii device as the visual and movement orientation controlller due to its widespread, easy and novel manipulation. In the experiment, the participants explored the unknown space and the obstacles information in it by these two methods. By comparing the accuracy and efficiency of the drawing mental maps, the authors verified the effectiveness of the look-around method.
Although the new direction of our project theme we wanted to work on has already been done by others, we also learned a lot from others work. First, the effectiveness and the ability to exercise spatial skill of this look-around method has been verified, which means we could apply this pattern in our design to help users perceive the gamified scene. Second, the Nintendo Wii controller can be a option or we can use the settings of his button to determine what necessary functions are available in our own controller. Third, we planned to add a voice prompt system to describe the virtual environment to our users, which has been introduced in this article as well. The authors provided their participants with obstacle information by different kinds of the auditory and haptic effects. Last, it presented us a method of verifying the validity of the design through comparing the mental map and map model, which could be taken to test the connection in the design of our auditory system and game scene.
Kristjánsson Á1, Moldoveanu A2, Jóhannesson ÓI1, Balan O2, Spagnol S3, Valgeirsdóttir VV1, Unnthorsson R3. Restor Neurol Neurosci: Designing sensory-substitution devices: Principles, pitfalls and potential. 2016 Sep 21;34(5):769-87.
In our future design, there is a great possibility to use voice and haptic system as input and output. Due to the loss of visual sensory, it is an inevitable challenge to convey information to them completely and accurately. Since when conveying information, different personal understanding, unclear expression and sensory caused by redundant information etc. are all the potential risks for unexpected results. Therefore, we have to follow certain rules when designing sensory substitution for the blind. In this article, the authors highlight some main points in the sensory substitution devices
- Only convey critical information.
- Focus on task.
- Need research in basic psychophysical representation.
- Must not interfere other perceptual function.
- Feedback modality can be various.
- Need consider the spatiotemporal continuity for different senses.
Although the sensory substitution system is slightly different from our design concept, because this system is designed to pass the information of the real environment to the blind people in the way of auditory and haptic, and our game environment will be a virtual scene. The design guidelines proposed by the author are also of great reference significance to us, because their common ground is to accurately convey information to users in the same sensory function.
On this basis, we have to mention the Voice-User Interface(VUI), a voice platform that makes human interact with computer. Nielsen also mentioned that the VUI is very suitable for the cases that the users can’t see because of the disabilities or other issues. It also has been widely used in products, such as the Microsoft Windows, Car Navigation, Android and OS etc. There are some practical tips should be informed for our design as well:
- Simple and conversational communication.
- Confirm of the finished task.
- A personality of the VUI.
Dunai, Larisa ; Lengua, Ismael ; Peris-Fajarnés, Guillermo ; Brusola, Fernando: Virtual Sound Localization by Blind People, Archives of Acoustics, 2015, Vol.40(4), pp.561-567
In addition to designing a game with a new experience for the blind, the original intention of our project is more important to help them train their hearing ability. The hearing ability is such important that it enables them perceive the environment, predict the danger and localizing, etc.The level of this sensory function largely determines the quality of their lives. Therefore, we wish our game could contribute to improve this ability.However, at that stage, we have not yet sought theoretical support for our will. For instance, can hearing ability be improved through training? Will our game settings have a positive effect on this? Fortunately, all our concerns have been confirmed in this article. At first, they demonstrated the blind people have a better performance on localizing sound than sighted people. In addition, through the experiment, the authors concluded that the blind can interpret the sound in the real environment more effectively after practice with the headphone. Sound sources in real life are often confusing because they are mixed with noise and the sound is mobile. However, after training in an ideal environment with earphones, the blind people's recognition of the sound source in the interference environment has also improved. In short, all kinds of theoretical support and experimental verification ensure the theoretical support and significant meaning of our project.
At present, there are not many games specially developed for the blind, the American Foundation for the Blind website(https://www.afb.org/node/16207/games-and-activities) listed some typical games, which mainly consists of two types: the haptic physical game, and the auditory electronic games. For the former, most of them are card and chess board games, players mainly distinguish game objects through different touch senses. For the latter, the game mode is mainly to let users make direction and simple decision-making promptly changes through voice or vibration information. A more complicated video street fighter game (https://www.youtube.com/watch?v=aX0oPwQPo9A) provides the player a training mode to remember the attack sounds of different characters, in this way the player can imagine what his opponent is doing. It inspires us a lot that we can apply a similar training model to help the player get familiar with the different sound effects as well. In addition, a game with a storyline published by Tencent(https://www.sixthtone.com/news/1003921/new-tencent-games-highlight-challenges-of-visual-impairment) provide us reference to set up game flow and voice cues.