This week I did some more work on my annotated portfolio. It’s been a while since I did any web design so it took me a wee bit of time and coding to get the hang of it again. I’ve chosen not to spend too much time on it, since it’s not as important as my actual prototype. I’ve mostly set up the visual aesthetic of it, and did some sketches before coding it in HTML and CSS to make the process easier. Some tweaks need to be made, but I’m more concerned about the actual content. When it comes to anything visual, I tend to overdo it and try to go out of my comfort zone. However in this case, I think it’s appropriate that I stay in my comfort zone and don’t get too creative with my annotated portfolio - keeping visual aesthetics to a minimum. Simple colours, simple code. Complex but understandable content.

To be honest, as personal as it may be, I have been dealing with mental health (as most people are) and the professional help I've been getting has just made things different for me - better in some ways, and worse in others. I'm truthfully very behind in my project, and I'm aware of the fact I have been avoiding doing the difficult things. I realised this last week, so I attempted to make a timetable to sort out the next few weeks - it's helped for the most part but the workload has still been overwhelming. I've gone through worse before, so I know I can get through it, but building the motivation is still a bit difficult, especially with the reminder that I failed to get motivated earlier in the semester. At this point I at least want something I can be proud of. I know my sister did well in this course, and I want to follow her in that path. I may not get the outcome I'd dreamed at the start of the year, but I just want something to be proud of, even if it just scrapes past the pass line.

I've got a new set up that will hopefully improve efficiency in all of my courses and commitments (with tutoring and an ongoing project with a client on the side, it has not been easy). I have seen an improvement in my productivity, and I only want to keep improving, even though it is near the end of the semester. I am still passionate about my project, and honestly if this all goes down the drain, I'd want to keep developing on it in my own time over the break.

This week started off having a short break from the project to look at other courses. During Tuesday’s studio I expressed my concern about the form of the prototype. Lorna had left a few suggestions on Miro, and my mind was constantly going back to feedback I’d gotten for the previous assessment about looking at something more tactile such as a guitar. I’ve brainstormed plenty of alternatives to the current piano form. The guitar would involve a very physical form of interaction, with buttons to register the number, and an area to strum to actually input the pressed number. After mentioning that in the studio, Lorna made suggestions which I interpreted as something along the lines of exploring a form that wasn’t resemblant of a musical instrument, or even any current existing form. She mentioned on Miro how my current form is rather specific. I chose the classic piano form as it is something that’s familiar for most users - whether it’s an actual piano, computer keyboards, or other instruments similar to a piano that has progressive keys representing notes. An alternative design I’d considered was something more flexible that could be adjusted to fit the preferred form. Users can curve it for their comfort and allow minimal movement. The keys would be attached to each other in a chain-like structure, to allow for pivotal adjustment. I do have the concern that this may not work if I wanted to contain the Arduino and breadboard within it, so I will have to ponder this idea further.

Form aside, I had an issue with numbers being registered with simple touch, as this could be done by accident. I realised I could solve it by having the jumper wires connected to copper tape below the key, which would be connected to the touch input when the key was pressed down. This makes the action of pressing down the key, not just an additional form of feedback, but a necessity to register input. Due to the cardboard structure being rather unstable, I have been looking for stronger materials to use to achieve this effect. Metal wires are definitely currently the most desirable and accessible material to use to create this pressing down motion, whether it is by using it as a hinge, or placing springs beneath each key to have it spring back. This would allow for each of them to go down without bringing the keys around it down with it, which is what the previous structure had.

I’ve illustrated what this might look like, to better understand how it might work since visual diagrams are a bit easier for me to wrap my head around. The desaturated pink represents the copper tape, and the components have been labeled appropriately. The key is similarly shaped to the current keys, however the bottom area has a small area that flattens out to ensure contact with the copper tape below it. On the right, I’ve also included a sketch of the previous design of the keys, which I have completely scrapped now.

In regards to the detail of the keys themselves, I’ve chosen to scrap the idea of braille for now, and using physical cues such as the markers on the f and j on the English computer keyboard, so users are able to understand which key they are currently at, and can quickly adjust their finger positions for ease of use. This also would work with the current idea of input being registered with the pressing down of a button rather than simply tapping the copper tape area. I’d previously had it as an idea, thought it wouldn’t be enough, but after feedback it’s something I will definitely return to.

At this point in the semester, I definitely feel I’ve fallen back, and it’s difficult to regain traction. I’m hesitant to access materials at uni since the travel is long, so I may need to stick with the materials and tools I can access here. I am somewhat confident I can create something feasible and of good quality with what I have, though it may not be the wood and metal I initially desired. My main motivator is to meet my personal expectations that I set from the beginning of the semester, and any that those around me may have for me. Especially since my sister had success with this course. This semester has had quite the effect on my mental health, especially right at a time where I was starting to seek help for it, but have now been placed in another new situation I need to adjust to greatly. Not really an excuse as everyone is dealing with similar things, however I have lowered some personal expectations to push myself to at least achieve my initial goals without overdoing it and feeling incredibly disappointed in myself for not achieving the highest possible grade. In the end I want to have had fun and learnt something, whether that's regarding the elements of my concept and research, or the skills I've been developing, or just about myself.

Going back to my concept, I feel I have focused too much on the form of the structure, and not enough on the actual learning aspect of the concept, so I have backtracked a little, and have been attempting to research more about how mathematics practise can be aided with music. One idea appeared to me in a dream (poetic, yes, but I have stress dreams every night so it’s not as poetic as I wish it was h a ha), to have music with questions tied in. Instead of assigning a note to a number, users are required to input the correct answer to the questions to continue the song. This is similar to learning the alphabet song, periodic table song, or practising multiplications like I did growing up to the tune of something. Some thought still needs to be put in, especially since this will require a lot of changes to code, which may be too demanding at this point in time. However, I will continue to research this since I believe it may have the potential to be more promising in allowing students to practise mathematics.

I realised I hadn’t considered the instrument as much as I could’ve. The xylophone was a bit of a “in-the-moment” idea. In my feedback for the previous report, the tutor suggested I looked at doing a guitar which is more body-controlled, and it put me into quite a bit of panic considering I had to potentially change my idea completely. After calming down a bit, I chose to simply do research on what instruments were good for visually impaired children, since I had not considered that at all, unfortunately. I learnt that while pianos, guitars, and violins were great for visually impaired children, instruments like a xylophone is terrible, as one source stated - due to the requirement of a stick to use it. I realised that the entire time I had really been creating an electronic piano, rather than a xylophone, however chose to conduct a bit further research into the instruments, since taking the form of a guitar is something that may feel a bit more comfortable to the user, and just from some brief thinking, having something similar to guitar hero could be interesting and rather novel. If users could press a number (simulating the holding of a string/chord), and then strum to input that number, it means that there doesn’t have to be an awkward placement of the braille on the key of the xylophone.

For now, I've stuck with a piano and will consider it for the next prototype. With the piano idea, the keys go down similarly to a real piano, which is a physical indicator that a key has been pressed for users. This is what a big part of my "xylophone" idea was causing me to think too closely about the instrument and trying to make it work, rather than finding an instrument that worked for visually impaired people, and making that work with my prototype.

I began building my digital interface in Unity. I set up a basic UI following this tutorial, since it’s been a while since I last used it. As a rundown of this past week, I’ve been jumping a bit all over the place. I mentioned previously that I was still struggling mentally at grasping things, but I think my anxiety built up to a point where I had to kick things off. I started by quickly setting up a capacitive touch circuit on my Arduino, and code I had used in DECO2300. It was easy to set up, but I quickly ran into strange problems. I thought my GND port was a bit dodgy, or if my wires were janky since it would not recognise the touch unless I held the wire in a certain way. I chose that I would work around it since I didn’t realise it was as bad as I initially thought it was - which I discovered a few days later. I was trying to work with the circuit again, switching out different wires, trying different GND ports, and I still kept having the same problem. I switched the code a bit so the LED would be on, and would dim if touch was recognised. I thought this might make it easier for me since I knew power was running through it, and any changes I make would be more evident to me. I then realised that it wasn’t detecting my touch at all, even when I held the wire the specific way that had worked previously. It was only when I asked my boyfriend to hold the wires for me so I could take photos of the build, that I made the discovery, that for some odd reason, the capacitive touch would only work if I was also touching the USB connection port with my other hand. It all made sense to me - when I was holding the GND wire in the specific position, I was resting my pinky lightly on the mental cable port, which is how it would sometimes work. I felt pretty stupid to be honest, and to make matters worse, I’d encountered a new issue that I perceived as potentially being dangerous. I sent pictures to a tutor for advice, and continued to look at different circuits that catered to multiple inputs.

Before the week started, I thought I would focus on the digital interface, however I think presenting the values isn’t currently as important as getting the visible components working. Thinking back now, I think I sidetracked a bit since I ended up trying to get audio working on the Arduino, and wasn't simultaneously working on the Unity audio output.

After my in-class report back, Alison suggested a glue gun, which I had been considering, and I didn’t realise how affordable they were until recently. I managed to grab the last one at Burleigh Officeworks, and extra glue sticks at Jaycar when I went to buy 3.3 Mega Ohm resistors for a new circuit. I'm yet to start using it. I also looked into the standard for Braille, and found a rather extensive document detailing the specifics of how Braille is to be done. It's much smaller than I had anticipated, but I plan to experiment with clay and glue, as I was unable to find small gems as Alison had also suggested.

I watched a video about a Youtuber called Molly Burke, who had night blindness and photophobia as a child, and whose conditioned worsened with age. She mentioned “pre-braille training” that involved feeling and identifying fabric materials and textures, and practising distinguishing them to ensure the person has enough finger sensitivity. Because of this, I considered perhaps applying some form of physical texture to the keys, to assist in identifying which key is being touched.

I found a tutorial on how to set up a touch piano, and I got the main set up built following this diagram.

I got a bit stuck on connecting the piezo buzzer since the one on the diagram had two pins, but mine had three. It took a bit of pondering, but I remembered encountering a similar situation in DECO2300 with a soil moisture sensor, and was able to connect it to the 3.3V port to power it. To my disappointment, however, the code returned an incredibly unpleasant constant beeping sound from the piezo. I've since found a different code and set up to try out. When doing this current setup, however, I didn't have enough blue wires to have a consistent colour code, even with the aux kit, so I used some extensions I had previously bought, and created a nice little rainbow, which I might match to the corresponding keys.

My concept is an electronic xylophone connected to a digital interface.

Each key numbered 0-9 has a distinct colour and tune, with each numerical operator also having a unique musical tune. All audio also includes a reading of the name of the element as well (e.g. “one”). Each key has (non-conductive) braille of the numerical value, allowing the user to feel what number they are about to touch. There are two modes of the interface - ask, and answer. Users can either ask for the answer of specific equations, or play a game-like format where they can practise answering basic mathematical equations. Audio will be outputted through the computer, and the music aims to enhance the memory of the user, while making an enjoyable and auditory experience for those with visual impairment, in something that is rather visual.

Our team focus is looking at musically-enhanced learning, and my individual approach is for visually impaired children. I’ve considered taking a simple approach, since it’s been difficult enough to source children at all in our current situation, however sometimes the easiest ways to go isn’t the best way to go, and it’d go against my goal of wanting to provide a better and more inclusive learning experience for visually impaired children. We’re all responsible for building our individual projects, which you would hopefully find in the same classroom.

I’ve still been struggling with self discipline and motivation, but I watched a Simone Giertz video and got to work on OneNote to figure out my sensors and the actual build. She created a “proud parent machine” using items she found around her workshop (which has many more items and tools than my minimally furnished apartment). She used the arm of an old lamp as the arm of her “proud parent” and used a motor to move it in such a way that resembled a parent patting her on the shoulder. She’s a great source of inspiration for me, with her interesting projects, and her interesting approaches to accomplishing them.

https://www.youtube.com/watch?v=M1B3gATS0GE

In my scribbles, I looked at what materials I would make my xylophone keys with - colour is an important factor, so I wanted to make sure my keys were colourful, however i don’t have many colourful conductive materials, so for now I’ve chosen to settle on giving the key a coloured outline, drawing inspiration from the electric field touchpad. I brainstormed a few different materials for the braille - carboard was the easiest, but I remembered I had clay and thought that could be an interesting approach. However, it is rather delicate and it gets shockingly messy - I also have a tendency to get sidetracked and make random objects out of the clay - like these ducks I made once upon a time.

I also had a look at some other materials, and I’m yet to settle on just one. It doesn’t need to be conductive, but I want to make sure it’s easy to handle, move, and work with other components to really ease the process. Planning is essential in ensuring no tedious processes are required further down the line for things I hadn’t carefully considered.

In our workshop, our tutors also helped us sort out what technology we may need. I have a number of options - from the electric field Lorna posted in slack, or modular touch and pressure sensors, and simple capacitive touch that I set up in DECO2300. In my OneNote scribbles, I wrote down any concerns, any ideas that popped up into my head, and again, any concerns about those ideas. I’d discussed with my team about our approaches, and how each of us may need to create just the simplest version of our prototype, thus I thought that maybe I’d begin with a single key of the xylophone, and hopefully go up to a 4-key xylophone, and gradually to a 10-key xylophone like my original plan, if all goes well. This straightforward approach also makes starting it a bit less scary, since I’m not overwhelming myself with incredibly high expectations, which I’m unlikely to achieve in my current circumstances - physically and mentally (and also a bit financially).

If I had the materials and time I would create a piece that could be used without the connection of a computer; this means it would have a physical interface, and be a portable device children can move around the classroom, with the internal components soldered in. I would also explore having the xylophone interactive either with touch or with a physical object (similarly to a real xylophone/glockenspiel). I really looked forward to using the physical workshops at uni, so if I had access to there, I would really go off with creating a metal glockenspiel or wooden xylophone that’s been carefully crafted, and a large piece of work that isn’t just limited to a single key, like my current stage of the prototype. Currently, I’m a bit scared to venture into the unknown since I could hit a dead end where materials aren’t accessible, but I’m also worried about that restricting me, which is why I’m currently working with what I have and what I know.

In week 6 we completed our report. It was quite a bit of effort that required a lot of individual and collaborative work. I had trouble beginning my individual section, but I managed to easily summarise our response to feedback, which was the part of the team section that I was responsible for writing. The lead up to the deadline was a bit nerve-wracking, but we managed to submit it 2 hours ahead of time, which was excellent! We set multiple objectives and deadlines ourselves, and maintained a system of having a Zoom call open while we did our work, to simulate a public studying space. It has a similar effect to working in a library, where you have the pressure of studying like everyone else, and not procrastinating to avoid judgment. I found this to be incredibly effective, for me personally, and my team members seemed to enjoy it too. I had some trouble trying to refine my concept, but I had a look at different sources and examples to get inspiration from. In my personal introduction, I think I wrote a bit too much about myself. I went into depth about my preferred methods, and a bit about what my team had discussed, which contributed to the great size of the paragraphs. In hindsight, I should’ve written significantly less, but when I revised my writing and tried cutting it down, I still ended up with rather large paragraphs.

In class, we used Miro to look at alternative methods of conducting field research.

We were required to observe train passengers, without actually being on a train ourselves. To be honest, I found this activity to be a bit difficult, since I used to take the train everyday, and I’ve made observations on my four years of taking trains to uni. It was fun exploring other means; and those that I thought would provide insight on passenger behaviour didn’t provide as much as I believed they would. This mostly applied to Youtube videos of passengers on trains, as a significant amount of them were vlogs, and would primarily contain the vlogger themselves, who generally try not to show others’ faces. Anyhow, I still managed to make some observations through the occasional shot of passengers. This also led me to look at other social media, primarily Instagram, where I've come across either photos of or artists’ illustrations of train passengers (my third method of observation). Instagram users had taken pictures of their trips, whether it was a casual trip into the CBD, or to a sports event. The different kinds of context were evident within the image.

While the artists’ illustrations depict their perception from within the train, and they have the option to emphasise specific people or scenes, I feel they draw a lot of attention and detail in the presence of the person. One artist drew many people with their head down, whether that was due to them being asleep, reading a book, looking at their phone, or simply just bored and looking around. These things were observed by the artist as being important or interesting factors. I enjoyed using the Miro board; I could see what other students had written as their means of research, and one that piqued my interest in particular was news footage. This made sense to me, since news footage generally captures natural movement of people in public spaces, and I believe some people respond less negatively to professional cameras, than to a vlogger with an intrusive selfie stick.

We continued to use Miro in our teams, and created a separate one for our group. We looked at refining our context, and breaking it down into the parts advised by the teaching team. This helped us clarify our context and concepts, and ensure we were all on the same page, which is incredibly important when doing a group project. I also used it to clarify my individual direction for the project, as it outlined the key information I was required to establish to claim that I had a clear understanding of where I wanted to go.

In our workshop, we also looked at the Arduino. We went through some basics of the set up, and what sensors I should be expecting in the auxiliary kit. I plan to do some further work and testing in the mid sem break.

In terms of my team and the assessment, we lost two team members, and gained a new one. It was expected, and I had the slightest consideration to defer the course, however my current course plan would not allow it, if I wanted to graduate this year. I've already delayed my program a year, and it's not ideal for me mentally and financially to extend my studies (those GoCard fees are no joke). Anyhow, this quarantine situation has certainly brought about a lot of change, and when I get adapted to one thing, something else changes. In this case, that was my team. It was sad to see my team members leave, especially since I was interested in seeing where they would go with their concepts, and I felt we had gotten along well. We bid them farewell, and I wished them all the best with their studies. It sucks for all the students and the teaching staff that so many unprecedented changes had to happen. I just hope we all make it in the end, and it's as much fun as I'd hoped it would be going into the course. Our new team member, Jen, is a familiar face, so there was not much need to warm up to each other. I think the first few weeks of the course where we talked to different members of the course is to thank for, since normally I would not know many people outside of my friend circle. Jen's team was also looking at a concept rather similar to our team's, so we had a discussion on how we would want to approach the project. We decided it fit nicely within our scope, as long as we described it as something along the lines of "Musically-enhanced learning for primary school children", since our concepts weren't based on the current school curriculum, but still aimed to provide an educational value.

In terms of the report, we broke it down into its sections, and designated different parts to each team member. We established that many parts were dependent on other sections and that a lot of it would be collaborative, however the sections written by individual team members. This meant that we'd have to write some parts before others; I'm responsible for writing the response to feedback, so I have had to use blog posts and individual sections in the report to get an understanding of what and how my team members responded to, and the commonalities of our team as a whole. For the most part, we approached things differently, especially since we each had diverse interests in primary school education, which is why I believed it was necessary to make some mention of individual responses.

There's certainly a lot of work to be done, but with the pressure leading up, and my desire to have a mid sem break that I feel I've actually deserved, I'm trying to do the work I can. I could definitely be a lot more productive, and the productivity tips from Lorna and the rest of the cohort have been somewhat effective, but I'm learning it all comes down to my self discipline. In all honesty, I felt I was losing myself the past week, but after some time in the sun I've regained myself a little, and in my current state I'm pumped to do my best until Friday evening, rest a little, then complete the work necessary over the mid sem break (while also making time for some games with friends, if I feel I deserve it).

After the presentations, we went away and did our individual research to explore what other areas we could look into, especially in response to the feedback we received. We realised we had to narrow down our context and give our concept purpose, which it previously didn’t quite have.

I took quite an interest into visual impairment, since I felt a lot could be explored to cater to the needs of those who are visually impaired. Upon researching I discovered that was true, and often even the basic necessities are not provided to those who need it, and I’m incredibly passionate in ensuring all students are on a level playing field when it comes to learning.

After some basic research, I learnt that there were three kinds of visual impairment (I didn’t know how to put it into words before). This is handy in having an educated and professional approach to visual impairment, rather than one that’s purely based on personal interest. As someone who is short sighted, I do get a sense of the struggles as I can’t fathom living without sight, and I’m fortunate enough to have the necessities provided to be able to see rather well. However there are those who either cannot afford or it is near impossible for them to have perfect vision.

I used a colour coded format to keep track of my dot points, and what sources I got them from. I was inspired by a format my friends used in ancient history in high school. It helped me keep my sources organised and thought my research was broke up, it helped me personally. I think I’ll continue using this format.

In terms of my personal idea, I went through different stages of ideation. I had several meetings with my team during and around class, about the research we’d conducted, any ideas we each came up with. We each seemed to be interested in different areas of primary school education. While I was looking primarily at visual impairment and enhancement of learning tools for children who are visually impaired, my team mates each looked at mathematics, the autism spectrum, and a deeper focus on music-enhanced learning. This encouraged a broad range of different research, and we each raised different points of focus, which we all benefited from, and I think this was a great example of how different minds can really bring different perceptions from our personal experiences, to create diverse collective research.

My Idea

Initially, I wanted to create something that allowed visually impaired users to create drawings. My rough idea dump was as follows:

Ideas

• Basic algebra/arithmetic - values represented by music rhythms
• Braille karaoke
• pin board where when they paint, a sound is created - associated with a colour and how they brush the canvas
• Pins are raised where they move the “brush”
• Physical and musical element to art
• Something (material?? velvet??) that changes direction
• Sand pit? Draw lines
• Heat
• Liquid

The pinboard in mind looked like this

where the different pins are movable. I wrote down different ways users could draw physical lines, however I decided that it wasn’t a very feasible idea. Thus, I went back to the drawing board and believed something with numbers could be a better approach. This also meant that I could support my team’s goal in creating something similar yet different enough to the other concepts.

During my research, I was thinking different kinds of tools that one can find in a primary school classroom. Previously, I was attached to the idea of the abacus since it involves very physical interactions, and good use of colour. However, I failed to think of how I might incorporate music. Thus I ditched the idea. I then thought of a basic calculator that children use in school, and how it could be a foot pad. This was a random idea that popped into my head, since when I sit down and try to think, I don't feel I get anywhere. The footpads would be combined with a screen that displayed equations, with each number having a corresponding tune, however I realised it didn't quite cater to the needs of those who were visually impaired, since I was picturing it to be something similar to Dance Dance Revolution, but instead Dance Dance Maths (lame, I know). I also realised it would exclude those who may have any mobility issues, and I didn't enjoy the thought of that, so I considered making it into a smaller keypad with braille numbers so that kids can just press the different keys. However I realised it was essentially the keypad on computer keyboards with braille, to my disappointment. Thus, I chose to go back to a previous xylophone/piano idea.

It still needs a bit of clarifying, but I have considered the feasibility. No matter what idea I end up with, I do expect there will be a lot of changes done along the way. I've been having a lot of issues with ideation, and it does make me upset, since I manage to come up with bizarre ideas when it comes to what I want to draw (out of the blue, I can easily choose to draw a pigeon with slicked back feathers saying "it ain't easy bein' sleazy"), but for the life of me, I can't sit down and think of a grand idea. I think part of the issue does come from the fact I hadn't chosen musical things as one of my preferences in World Café, which is something I noticed earlier. I have managed to garner an interest, since it mostly stemmed from lack of knowledge in the area, however it has been rather difficult. I know I had trouble in DECO2300 last semester too, and I think knowing I'm not the best at coming up with an excellent idea on the spot puts me down a bit too, along with the other challenges that have recently come into play. Of course, challenges come and go, but I really wish I had a bit more of a creative mind when it came to things that really mattered. I think my focus for now is to not bring myself down, and continue researching my areas of interest, since broadening my knowledge always expands the areas I know I can explore into with concept ideation.

While I did discover there were existing tools for visually impaired people to draw, I still am rather attached to the idea, and wherever I go, I hope I can incorporate a feature such as that into it.

We each went away with the storyboard as reference. I closely followed the storyboard as a reference for my sketches. I took some time deciding what drawing software I wanted to use - depending on how detailed I wanted the sketches to look, I could have used either Adobe Photoshop or Microsoft OneNote, which I use for a lot of my regular doodles. While I was more comfortable using Microsoft OneNote to do quick sketches, it’s not a very complex program, so its interface was a big too simple and restrictive for me to comfortably to draw in. Conversely, Photoshop was too heavy-duty for my liking, especially since it’s more ideal for digital painting and proper illustration rather than sketches. I decided to settle on a safe middleground that I was comfortable with - FireAlpaca. I’ve been using it since I was 12, so I’m incredibly comfortable with it, despite my urge to move to Adobe Photoshop (especially since I am paying for it). Anyhow, it had the right amount of complexity and simplicity that I needed to draw quick sketches with the desired amount of detailing, and had the layering system that OneNote can’t provide.

I maintained a naming system of the layers and the files. The raw illustration files were named accordingly, and when I exported them I labeled them to briefly describe what was being illustrated. This was to ensure Maria knew what each of the images represented. Unfortunately, when it came to being animated by another team member, she had gotten some of the images mixed up. She still represented the actions correctly though, so it was still alright.

When drawing, I usually do a basic sketch then draw over it to achieve clean lineart, but instead I went straight to using an inking brush and created rough-looking illustrations that were still somewhat clean. I used some images and my own hand as a reference for different objects. The following images are what I drew.

Overall, I'm pretty happy with what I drew. Thanks to me notorious perfectionist trait, I usually spend far too long trying to create elaborate illustrations, but I really pushed myself to create decent and quick sketches and I'm satisfied with the outcome. I had enough time left to record my audio in comfort, and provide assistance with other parts of our pitch preparation.

There were some issues with our animation, as the incorrect files had been used for some visual aids, but I think they still successfully represented the intended actions. I was pretty happy with the work we put together, and while there was the tiniest bit of last-minute edits at the end, I think we achieved a lot and managed our time well.

While I had a few concerns about our proposed idea in regards to feasibility, during the production of our pitch, they didn't fully occur to me until after we had presented. Seeing the feedback we got put me in a rather negative mindset of "oh no, we can't do this", but I tried twisting it more into a "we have a lot to do, but we know what areas we need to explore further". We looked at our feedback and broke it down into different categories.

Positive

• Playful
• Interesting focus on those who don’t have music skills
• Using daily objects as interaction tokens is affordable
• If creating new product, could create something that isn’t completely foreign
• Adaptability with different objects
• Recording audio and using it later to make music would be a fun approach

Negative

• Could become bothersome over time especially if you just want to use the item
• Almost too open-ended, have more of a specific purpose (the more open-ended, the more difficult it might be to create)
• Having everything be musical might take away the novelty
• Might be irritating for those around you
• Consider feasibility

Questions

• What are the natural sounds?
• Difference between making music and exploring sounds
• Using sound (from objects or even speaking) to manipulate other objects, or
• Using objects to make new sounds

Explore further into

• Abletons plugins
• Pete and Trevor on Soundscape
• MakeyMakey banana piano
• Scratch mouse
• Break down depending on context (in the classroom)
• Controlling a computer with different sounds
• AudioMostly conference
• Expected sound associations with certain objects
• Collaborative?
• Explore what kinds of sounds people want to hear
• Could be used for educational purposes

There were a lot of elaborate critiques from our peers, and while many shared the same essence, each provided their own unique perception of our concept, and it was overall very valuable. An observation that I had quickly made was that there were no "positive" comments, or at least none that had any value. Many people wrote "This is an interesting idea, but...", which didn't tell us anything about the parts we could actually keep and develop on.

I think the most valuable feedback were the suggestions people provided in researching different concepts. It's easiest to draw inspiration from something, rather than create something completely new. So, we went away with this information, and conducted some individual research and further ideation to see where it would lead us.

This week has started interestingly. Our team went away after class, and we each thought about the concept. Last Friday, we each shared our thoughts on the concept. Most of us certainly liked the idea of creating music with any sort of everyday object - most commonly a pen or an eating utensil. Another team member also mentioned the use of gloves in creating music, however I believe it was something that's not very original (because I know of students who had created the concept in DECO2300 Digital Prototyping), and strayed a bit from our concept of everyday things. With the idea of eating utensils, I brought up the issue of table etiquette, since within my Asian culture my Asian friends and I grew up pretending to play the drums on my rice bowl with chopsticks (how could I not?), but I would get scolded any time I tried.

While our ideas were a bit different to each other, I think the discussion was good with the diverse ideas and perceptions that each of our team members brought to the conversation. We then organised a team meeting for the Monday, and chose to still meet up despite the university shutting down for the week.

Pitch Plan

We planned to meet at uni at 9am. Two of the team members arrived late, which reminded us we needed to clear up and sign off our team agreement.

We started off with discussion about the different ideas we had, and promptly sketched out what our concept would be. Before getting any of the pitch done, we needed clarity on the concept we were going to approach. It was agreed that we would be using some sort of object to be producing music, but we needed clarity on what exactly the “music” would be. In the previous week, we’d outlined it as being some of these categories

• Ambient noises of nature
• The natural and organic sounds of the object
• Tunes and actual music that is emitted according to the movements of the user

This round, we went into more detail about how the audio might be pre-recorded, most likely with tunes rather than the natural sound of an object against a surface (since that could be rather dull, and the feedback time would be rather laggy). I brought up the idea of possibly introducing unexpected sounds, but looking back on it now, it wasn’t a great idea since it would become rather bothersome in everyday use, especially since we couldn’t think of any reason to incorporate that feature. We looked at the duration of sound, and furthermore the variables for which the different kinds of audio are dependent o - we established contact duration, and motion (e.g. strokes) as base examples, but explored them further in our pitch document.

The drawings below it are a visualisation of our ideation. We discussed whether we wanted the concept to be more feasible or interesting, and chose to go with a more feasible approach, while explaining in our presentation that we weren’t sure about which direction to go in, with the hopes that our classmates might provide some extra feedback. The more feasible approach we considered was to use large surface areas as opposed to smaller objects. This would mean placemats, desks, and whiteboards (which I’ve mentioned multiple times - I don’t know why I’m so fond of whiteboards). Users are then able to use /any/ everyday object to interact with this surface, which is much more feasible than trying to create a device that works with an object as small as a pen. Furthermore, it appeals to the initial focus we had to include all everyday objects, which I believe also encourages the use of it, as it is more accessible than something that may be a completely foreign object that isn’t properly integrated into people’s everyday lives.

Moving onto our document, we began outlining the parts we needed to include in our pitch - using the assessment brief and brainstorm sketch to refer to. We broke our pitch down into 4-5 different categories:

1. Introduction and Aim of Concept

• In here, my team mate Maria introduced the team, what theme we were focusing on and what our concept was. She covered the purpose of the concept, and a summary of the interaction and example users.

2. Interaction

• A team mate and I wrote this section separately and combined our paragraphs, describing the basic logistics of the interaction of the proposed concept, and the different ways a user may interact with it.

3. Scenarios/Example Users

• In this section, a team mate covered example users and scenarios in which they might use the product. In my opinion, I feel a more in-depth description of the actual people rather than the scenarios they may use it would have sufficed, but I was concerned with my own tasks since it hadn’t been completed at the same time as the rest of the script.

4. Future Development and End Goal

• We described our end goals as both a team and as individuals, and where we would ideally take this concept.

5. Concerns (Extra)

• We included this small section so we could make brief mention of the extra concerns we had, since I personally think we had/have a lot of uncertainty about the project. Through discussion, we learnt that we shared the same concerns - regarding the feasibility of the project, and how we would manage on our own.

Furthermore, since we had assigned each person to each section of the script, we also roughly assigned the different roles. I was in charge of doing illustrations, two team mates for editing the video or presentation (which was undetermined at the time, due to the uncertainty of the university’s status), each of us doing audio, and some other roles.

I think we'd made a good start on our project, and we set milestones for deadlines for each of our parts. We planned to reconvene online in a few days, to check everyone's progress, then set another date to have a proper online meeting.

From experience, only setting a meeting or deadline, and not having a "check up" date always ended badly, since we'd realise on the day that some team members hadn't done any work - whether that was because they'd forgotten, they weren't sure what to write about, or they didn't have to time. I think it also motivated me to actually get something done, since as much as I was checking my team member's progress, they were checking mine too. It also ensured that each of us knew what we were doing, and if we needed any help, that point before the actual online meeting would ensure we had something sufficient by the date.

Online Meeting

We had an online meeting via zoom, which was certainly interesting. It was a bit slow to begin with, as we originally wanted to clear up the content we had written, but two sections hadn’t been completed, so we worked on them together as a team. Setting our check up deadline and our actual deadline wasn’t as effective as I’d hoped it would be, but it was definitely better than having a single one, so we chose to continue using it. Our planned two hour meeting went on for 4 hours, but we managed to get a lot of work done, which I am happy about. I suggested we made a storyboard, so we broke the script down in smaller paragraphs, and drew up the frames accordingly. I used the whiteboard feature on zoom and my Surface with the stylus to draw up all the frames. We all used our mics and it was a very collaborative effort. Going through each of the frames, I made sure my team members knew what was being illustrated so everyone was on the same page, and I think we had a very good discussion, and honestly a very fun time too. While I was doing most of the drawing, my team members utilised the annotation feature, which is how George came to be. I didn’t draw him, and I won’t say who did - just please believe me when I say those are his hands.

We distributed the work amongst ourselves, with consideration to what we enjoyed and what we believed we were capable of, which naturally came up in our discussion. Joy and Kelsey were to record the video parts, I was to draw the sketches, Elva animate them, and Maria to create a powerpoint presentation and bring everything together. We broke the script down (as shown in the image below), with the paragraphs numbered so that we could name our audio files accordingly to ease the editing process. We again set several milestones, to ensure our individual content was completed, with enough time for Maria to combine all the different elements, and upload it to Youtube well before the submission time. I was very happy with the work we completed in this session.

This week was crucial for team formation. A World Café session was conducted on Tuesday, which consisted of multiple rounds of context, outlining audience and domain, then refining the ideas. This process was rather tedious but it prompted a lot of design thinking, which I enjoyed.

Tables were set up for each of the different themes the tutors had collated:

• Beautify the Self
• Change through discomfort
• Altered Landscapes
• Enhanced mundane spaces
• Everyday sustainability
• Musical things
• Emotional totems
• Musical metrics
• Emotional intelligence
• Sassy tech
• Guided movement
• Bothersome tech
• Change through positive reinforcement
• Digital sensations
• Creative Learning
• Body as controller
• Ability-centric interactions

It was interesting to see the progress from last week's themes - basic and broad to rather specific.

Round 1: Context

This first round required us to look at the theme and ideate what project could be created, and look at what space and problem it might be exploring or solving. Early into World Café I realised I'd be doing a lot of speaking since there wasn't a lot of individual contribution most of the time. Anyhow, those who participated provided diverse insights and there were even some really excellent ideas. World Café wasn't only beneficial for concept development, but for getting to know my classmates, who each bring a different outlook on life and tech into the shared space of our table. Having multiple mini-rounds was a bit hectic, but the entire time I was constantly immersed in the process.

Round 2: Audience/Domain

This round looked at who we were designing for, and where the project could be applied. This part was a bit difficult since as I moved to new tables I realised they hadn't actually developed a new concept and the host would basically explain their critiques of each project, rather than trying to create a new one or clarify a theme. Anyhow, for the ones where I had something to work with, we explored some interesting ideas. One that really stuck with me was one of the music ones (ironic how it stuck with me but I'm not 100% certain what the theme actually was). We looked not only at those who were born without a sense, but also those who lose it either as they age, or due to an incident or developed condition. I enjoyed exploring audiences that don't have products catered towards them.

This also made me interested in the ability-centric interactions but I hadn't considered it when choosing what table to go to, so I never got to see what ideas had been created.

Round 3: Refine

This process was again a bit difficult, since some themes really had nothing to go off. There were a lot of notes but a lot of them seemed to just elaborate on the existing concepts. I think at this point, anyone who was a host struggled to recall everything that had been written down, since I think there was a habit of hosts simply recalling what the group had done during their session, and not really of what past developments had been made.

While I really like the theme of sustainability, and it's something I like applying to my everyday life. The fact that there were pretty much no properly outlined concepts made it difficult to /refine/ as the round requires us to do, so for some of the last groups I ended up coming up with other ideas.

Overall, the experience was rather interesting. It was definitely informative but I only managed to go to about half of the tables. This meant that when it came to picking my favourite themes, I wouldn't actually know about whether the themes were actually good and proposed a great challenge for the course. I blame myself partially for this, as I went to random themes rather than thinking of which ones I was fond of, and wanted to elaborate on. Furthermore, the fact that there was only a little bit of work for some of the themes was incredibly disappointing. There were themes like emotional totems and sustainability that my heart desired, but when it came down to it, there were not many good ideas, and almost no development, so while I did vote for them, I was still unsure.

Teams

So the second class this week, we started off with a call with Bash Isai, a graduate who moved to the UK. He provided some very insightful advice to us about getting out there in the real world, and to be rather honest, I am freaking out constantly about the concept of having to and needing to find a job.

Anyway, he ran an activity where we had to write a 13 word paragraph, extend it to 50 words, then cut it down in half until we were left with 3 words - then 1. I realised I hadn't done the activity correctly and probably had too much fun with it, but I guess that is a good aspect too.

My paragraph ended up being:

Hi, my name is Rika.

I am a digital consultant/UX designer.

I am unique because I like drawing birds. I did synchronised swimming between the ages of 6 and 12 and was thrown 2 metres into the air by my team mates, which I believe was my earliest and most memorable experience of trust and team work. I am also two halves Asian but can only speak English.

Cutting words down, my 3 most defining words were:

1. UX
2. Birds
3. Asian

My single word ended up being birds.

I felt a bit stupid after learning others had words like "outstanding", "victorious", or "challenge". But, when I think about it, I guess it shows my dedication to the things I'm passionate about? I'm mostly bullsh*tting it, but I guess I'm also mostly unashamed about the things I really enjoy as an individual. In the past I've had interviews at rather corporate places like Deloitte and the American Chamber of Commerce, and while I'm dressed in a proper suit with my hair all tidy and presentable, while remaining professional, I think it's important to have a bit of character and stand out. Whether talking about drawing birds is a desirable characteristic is entirely dependent on who I might speak to, but at least my mention of synchronised swimming is something a bit more relevant.

While synchronised swimming occupied most of my childhood and my mindset was always about preparing for the next competition, it was honestly one of the most valuable experiences I've had to date. 6 years of working with various teams - kids my age, women a whole decade older than me, in a team of 4 or 10. Like I mentioned, I was thrown into the air a metre or two by the older swimmers. Not just once. It was part of a routine and we had to get it right so it would happen about 10 times a day for a few weeks. My sister and I were the most experienced in our age group so we took part in our "combination team", and as the youngest I was rightfully the Chosen One. Anyway, enough about that - synchronised swimming has certainly been one of the most intense sports I've played, with the amount of coordination which is reliant on both individual efforts (practising and memorising routines in our own time), and a team effort to move ourselves around a pool to ensure we're all actually synchronised. That's the entire point of the sport.

Anyway, so put simply, while I did come up with some bizarre sentences, I do feel they still have some relevance, and it doesn't hurt to make an impression with potential employers.

Teams (Cont.)

Back to the teams. I was lucky enough to be in a team with two of my friends, and two unfamiliar students. I didn't get any of my theme choices, but I am still happy about the theme we got - musical things. This week, we'd established a team agreement, and got familiar with each other. One of the students were late, so I feel she might've been a bit behind, but I think we all got on pretty well. We established communication as one of our biggest things, since while things may go well or badly, lack of communication really just escalates them, and it is not a good time.

We quickly tried to develop on the concepts that were outlined from World Café.

I'm feeling a bit rushed about assessment, but I am looking forward to what our team comes up with.

This week everyone presented their ideas to the class in a pitch-and-poster format. The audience was required to critique the different works based on the relevance of the concept, and how well it was illustrated and explained. It was an incredibly tedious effort sitting for quite a few hours, but it forced me to pay attention to each and every single concept, and try and understand it to the fullest, rather than if I had sat idly and zoned out. I had written a pitch to aide my poster, however my nerves had gotten to me and I struggled to properly explain my concept. I still believe my poster mostly did it justice.

The feedback received on the concept was mostly positive, with a few suggestions here and there on how to make it slightly more interactive. Some of the key features people took a liking to were its portability, its assistance in describing ones emotions to others (therapist or friend), and how it was not incorporated into a fact.

Anyhow, the second part of the process was to identify common themes among the various concepts. There were many that immediately came to mind, such as "shadows", "music", "emotion", but a friend and I tried exploring into other areas, totalling about 10 different themes between the two of us.

Most people had the same ideas with others, and some ideas weren't so easy to sort into a single category. Themes like health, game, music, emotion, fitness, smart, sustainability, and behavioural change were the most common themes. Themes such as life beautification and memories were more unique categories that only a few individuals had identified.

These themes were explored further and broken down into more specific themes, outlined below.

Some of these included themes like creating music through movement, individual action for sustainability, negative reinforcement for behaviour change, and emotion as input. This is beneficial for next week's World Café activity.

I spent the weekend coming up with some slightly less impossible designs for the pitch and project. I began with a rough ideation of different ideas on paper, then made a more elaborate mind map, and spam of different concepts that popped into my head.

After discarding many ideas, I eventually ended up with something called the Em-Cube.

Em-Cube

The process began with wanting to target the challenge of mental health in physical computing. Mental health may be one of my greatest interests, alongside birds. With the concerns of people misdiagnosing and avoiding help at all for their problems, my mind was brought to mood-tracking apps on phones. I personally haven't used any myself, but

Pitch

Em-Cube is a small portable device that allows you to track and measure different emotions with greater precision. With the rise of products focused on health and well-being, there are still many challenges in the mental health sector. Em-Cube brings a new layer of interactivity to mood-tracking, utilising tap and slide, fitting its functionality into a small cube that can be carried around without the disruption of a phone. The concept comes in three parts. The first is the basic functionality of the cube - Tapping on one of the flat edges turns on the device, and tapping twice turns it off. Tapping on the other flat edge allows the user to swap between negative and positive moods. Each face of the cube represents a specific emotion, associated with a colour, and can be selected by tapping it face-down on a solid surface. Users can slide the cube to record how intense their current emotion is, which is indicated by the strength of vibration.

This concept has been inspired by the psychological evaluation systems in the Blade Runner films. While the Em-Cube can’t identify if someone is a Replicant, it targets the issue of misdiagnosis; this cube allows users to track their moods, with the addition of a reader that can be utilised by a therapist or counsellor. It further allows users to actually remember the different moods they experience over a day or week, and recall either happy moments or things that brought them down. The colours allow users to see an overall summary of their moods on an hourly, daily, or weekly basis, and whether there are improvements or unfortunate declines in their moods, which can further be discussed with a therapist or even shared with friends. Em-Cube enhances everyday life for those who want to keep track of their mental well-being.

Inspiration

As mentioned in the pitch, I was inspired by the psychological evaluation tests in the two Blade Runner films. Referred to as the Voight-Kampff Test, and the Baseline Test, the psych evaluations ask the user questions, and their answers are analysed to detect whether they are a Replicant or not. It sparked some thoughts of how humans are unable to really analyse a person's mind, especially when it comes to themselves. In a therapy session, users don't analyse themselves, they are evaluated by recalling emotions and events, and having the therapist provide a diagnosis or form discussion around it.

The shape was inpired by fidget cubes, which I have bought before for a friend. They are compact, easy to handle, and small enough to put into your pocket. They're easy to access and really you can use it in most situations. My initial idea was to incorporate the fidget sides to the Em-Cube, however that became a bit of a chore to achieve.

Furthermore, the cube was originally called a "Mood Cube", which was so generic I swore there had to be something out there that already existed. And I was correct. Mood cubes didn't really work with the mental health of a person, and didn't hold much in regards to interactivity. The mood cube is simply a soft glowing lamp that calms users down with different colours. I decided to incorporate that into my product, as the use of colour can be effective in representing different emotions.

References

https://bladerunner.fandom.com/wiki/Voight-Kampff_test

https://bladerunner.fandom.com/wiki/Baselinetest?file=BR2049baseline_test.jpg

https://sensorytools.net/products/fidget-cube-hi-quality

https://www.sandhactivities.co.uk/product/sensory-tactile/colour-changing-mood-cube/

This week really went in at full (ludicrous) speed.

The main focuses of this week were an introduction to the course, and a close look at physical computing (as the course name indicates).

Wednesday

The second day looked at ideation in groups, in preparation for our poster and pitch next week. We started off with discussions about the Seven HCI Grand Challenges; I was assigned to Well being, Health, and Eudaimonia. The reading went into detail about the intentions of technology focused on the well being of individuals whether it be their physical or mental health, but raised concerns about a number of matters.

We discussed the positives, negatives, and examples of this challenge, and how we might change it, as pictured above.

In the second session, Awais introduced a technique for ideation that he wanted to test for his studies. It comprised of creating a sentence describing a need and conceptualising ideas (feasible or not) to cater to it.

There were three parts to the activity - the first was as a group (pictured above), the second was done alone, and the third part (also pictured above) was a reiteration of the first one. This really brought about some wild and incredibly interesting ideas, and got me thinking out of the box. I personally believe that I have trouble coming up with new and intriguing ideas, but this activity really helped advance my thought process and get my brain going. I did have some difficulty in coming up with some ideas myself, especially when it came to the second part of the activity.

The domain was rather difficult to work with and honestly alluded to the concept of eternal death, which resting underwater usually leads to. However, in the desperation of wanting to complete the activity, I managed to twist my perception of the target sentence, and come up with a few ideas. The activity was incredibly helpful in understanding different novel interactions, and how literally anything can be created in so many domains, even if they sounded a bit ridiculous.

One idea that was brought to my attention was the fuzzy sand - while it seems a bit dumb (in simple terms), it did make me think a bit about mental health (which happens to be on a mind a lot) and kind of bringing ourselves to be aware of our surroundings. Picking up sand and experiencing something that contradicts your expectations is typically something shunned in design computing, however with the right intention it can be something really interesting. Whether that's simply creating confusion or intrigue in users, or even going as far as being used as a device for those with trauma to bring themselves back to reality. Picking something up, having the sudden feeling of "gosh, this is weird and not what I expected", can really pull you out of a traumatic memory and focus you on the bizarre experience. Furthermore, it would be

As one might have picked up, I've always been very big about mental health, and the different activities and the challenge I read about has felt relevant to me on a personal level, and I believe there's still a need for improvement on the societal level. These different activities allowed me to look at ideas and concepts from different perspectives - whether that's twisting my own, or from my peers. It's helped broaden my own mind about what is possible or can further be explored.

Hi, I'm Rika! I'm in the fourth year of my Bachelor of IT and UX Major and I like drawing birds in my free time. I'm pretty design-based but I am always looking to advance my programming skills since I do enjoy something a bit more technical than drawing nice shapes (not that that's all design is, of course).