post from Dr. Annette Gourgey

Updated Evaluation Plan

Gene and I met this morning to draw up a revised plan for the WiiCane data analysis. We see this taking form in several ways.

First, Gene has identified some case studies from our first exploratory trials with the WiiCane that suggest ways that the device is beneficial for student learning. He has posted some of these findings on this blog. One set of graphs plots the number of corrective feedbacks given over successive trials, showing that the frequency of these feedbacks may decrease with practice. This suggests that students are learning something from repeated use of the device that carries over into later trials.

A second set of graphs Gene has created shows an effect on veering. When the number of nonveering messages (indicating correct positioning) exceeds the number of corrective messages (indicating veering error), improvement has occurred. A plot of the arithmetic difference (nonveering minus veering messages) shows that this balance may improve with practice. Again, this demonstrates the potential for learning with practice using the device.

The cases analyzed so far took place during the exploratory phase of our trials, when we were experimenting with the optimal number of trials and adjustment of parameters such as the tolerance threshold for veering (which varied from 12 to 18 inches). Our next step would be to set up more consistent trials with several participants, in which the number of trials and the veering tolerance are held constant. Thus, we plan to select three representative students from NYISE and to test them under these conditions: three 30-trial sessions over a period of three days, for a total of 90 trials; and a veering tolerance level of 12”. These conditions are consistent with those identified by Guth in his previous research, which demonstrated that improvement in veering could be observed after several successive days of sessions consisting of 30 trials each.

This experiment would provide a more systematic evaluation of the potential of the WiiCane device to reduce the incidence of veering. With more reliable data from such an experiment, we will be better positioned to design future research with larger samples and to advocate for the benefits of using an automated tool to improve mobility training. This knowledge will be beneficial both for instructors who wish to evaluate how the device will help their students and for our ability to market the device.

Another deafblind student using vibratory feedback

This is day 2 and day 3 veering reduction for subject ST. (Day 1 was only exposure and practice on the system.)

There is only a total of 24 trials and the learning curve appears more erratic.

influencing veering behavior with vibratory feedback only

This is exciting to report. We tested the WiiCane for veering reduction only at HK last week. The students were deaf and hard-of-hearing. None used spoken English feedback; all used vibration delivered by remotes attached to the arms.

I reviewed the video data for two students for three days. I made sure that the data are accurate by viewing every trial, not using the computer system-generated data. This means that when the parameters were temporarily changed or the system malfunctioned due to user error, I eliminated the few bad trials. The results are we have some clean data and we can see learning with this subject. I looked at another student and there was learning but less dramatic and the data was not as clean because the parameters for the device were different on two days of trials.

For the one subject I will call SG, the improvement in reduced veering is clear. Here are the mean number of feedback prompts for veering for this subject:

Day 1 cumulative trials 11 6.36
Day 2 cumulative trails: 34 3.30
Day 3 cumulative trials: 49 1.87

This indicates to me that the feedback system can work with vibration only, and can work with deafblind travelers. How consistently and compared to spoken English we do not know for certain.

The graph is above.

design proposal for the tilting belt holder for the center Wii Remote

Here is my idea for a belt holder for the Wii Remote that will go in the center of the user's back.  This one is different from the simpler version for holding the Wii onto the sides of the belt; since this one needs to be able to "see" the light strip, it has to be able to tilt back, so that the camera's view clears the user's body, hair or clothing. The belt has to be pretty tight to keep the belt itself from rotating, which would end up tilting the unit too much, and make it too bouncy. As much as possible, the Wii Remote must be kept stable and as free from bouncing and jiggling as possible, because too much movement will have a negative effect on system accuracy in measuring veering, gait, rotation, and distance. 

belt clip prototype without tilt feature

This picture shows a prototype of the belt holder for the Wii remote units that will be clipped to the user's belt. This is for the upcoming trials at Helen Keller National Center, where we will be testing WiiCane with users who are deaf-blind. Because they can't hear well enough to use audio feedback via wireless headphones, we are experimenting with providing them with vibratory feedback to steer them during veering exercises. This means that the participants will wear one of these belt units on each side, and we will pulse the rumble motors in the Wii remote to signal to them that we want them to correct to the right or the left. This clip grips the belt very snuggly, and so I think that this will lead to good conduction of the vibration to the user's body. While this design is very simple and inexpensive to produce, I don't think that we can also use it for the (third) Wii remote that is being used to "see" the light strip above, because in that case we need the capability to tilt the device backwards for some users so that the camera view is not obscured by their clothing, hair or body bulk (see previous post).  But, for the two side-mounted Wii remotes, this should work better, and will be cheaper to produce.

belt clip design with tilt feature

GUI and adaptive algorithms for feedback

Zach and I met today to discuss our next steps. We came up with an approach to presenting the Graphical User Interface for WiiCane when it becomes a commercial product. Currently, we have a very crowded screen that includes controls and text boxes for setting all of the feedback thresholds and other conditions. That screen is just for our purposes during testing. But, our users will want something much simpler, that is tied to the curriculum that Gene and the reviewers (Bonnie, Donna and Rob) are creating. So, we are proposing to create a different screen for trainees and instructors to use that does away with all of the controls, and instead provide a list of lessons. Once logged on, a user would touch the screen to choose a lesson; choosing a lesson, sets up all of the conditions for that activity. For example, if you choose "Lesson 1: Veering", the system will turn on both corrective and positive feedback for veering, but no other feedback. The system would assume a default value for the veering threshold (the distance away from the center line that you go before you hear the correction "go right" or "go left").  "Lap swimming" would be turned off for Lesson 1, which only calls for a single traverse of the course. When you select Lesson 1, instructions for tapping three times to start and stop the feedback is displayed and is spoken through the computer and the headphones. Later, if you want to switch to Lesson 4, the veering feedback turns off, and the rotational feedback turns on, and the instructions are again displayed and spoken.

With this approach, we control everything with presets or styles. These are factory settings for each feedback condition or operational mode, based on our experience of using the system. The user will not turn things on or off, or set the thresholds (which turns out to be very trick to do).  Instead, the system will configure itself according to the lesson, and the feedback threshold will be controlled dynamically.  So, as the trainee practices one of the lessons, the computer pays attention to his or her performance and spontaneously and continuously adjusts the thresholds.  This way, as a trainee gets better at doing the task, the tolerance for being correct is reduced. This is an adaptive system similar to ones used in standardized assessments, where questions are selected based on the test-taker's success rate in answering earlier ones. I am guessing that a system like this will promote very rapid improvement, even though the user might not realize he was getting better, because it would seem that it was always equally difficult to get the positive feedback!

This approach will also simplify the process of rating student performance so that we can tell whether learning is occurring. If we see a downward trend in the threshold distances for any given exercise, we know that the student is improving, at least in doing that one thing better (such as veering).  That trend, however, tells us nothing about arc width, but that could be measured simultaneously in the same way. So we may end up with a multi-dimensional score, by which a trainee can be given an overall performance rating. Another virtue of the system we have in mind is that it would be easy to add or revise lesson profiles or presets using a single, very tiny text file that could be easily updated automatically if they connect the computer to the internet. This way, we can update the program remotely very easily.

I am interested in knowing what project staff thinks of this proposal.

article from Scientific American Mind that is highly relevant to our project

I just read an interesting article that discusses improved learning outcomes when students are asked to try to do something without any practice or experience first, then undergo training, then try to do it again.  Apparently, the act of trying and failing first primes the brain to acquire the new skill or knowledge more rapidly and with greater retention.  This bears directly on the way we are structuring out tests of WiiCane: we start by asking the student to try to walk straight without any feedback, then we turn on the feedback and have them do repetitive traverses with audio or vibratory coaching, then finally we turn off feedback and see if their performance improves as compared to their initial state.  Given the findings of the research discussed in the article, it might be good for us to rely on this technique in each of the lessons we are constructing for the WiiCane curriculum. 

If you want to read the article, click here.