video showing 9 section (54 feet long) light track at New York Institute for Special Education

New Lightstrip video

In the short video below I demonstrate the newly-redesigned WiiCane lightstrip and working "find-the-user" routines.

New design for the overhead light track

Here's the latest design.  This one uses anodized aluminum channels with the open side facing down.

Discussions with Dr. Rosen

I have spent the past few weeks in discussions with Dr. Rosen regarding the WiiCane, and being educated in the biomechanical and sensory aspects of veering. I will summarize some of our discussions and end with some implications for product design and teaching approaches with the device.

Sandy has reviewed a few hours of video and I have sent her the recordings of two more students. I should note that the data are not ideal for analysis so Sandy’s comments were based on what was possible to glean from the images.

It is highly improbable that training with the WiiCane is influencing students’ gait. Sandy said, “Gait patterns stabilize around age 6-7 according to all medical research.. . . gait is a very hard one to change long term.” As for posture (particularly pelvis-trunk alignment), which is more “fluid,” we probably are not effectively influencing this either, but we could.

So why is it that we see improvement in some students using the device? First, I’ve come to believe that early and dramatic improvement in straight-line travel is probably just becoming adjusted to using the feedback system. But the long-term changes in walking a straight-line is most likely from the training’s (exercise) impact on proprioceptive awareness. The proprioceptive system is a balance system The one that is not vestibular) — that makes normal, slight adjustments to balance. It is (as Gene understands it) neurologically related to muscle systems and tone. Individuals can be trained throughout life (not like gait patterns) to fine tune this system; effective training is active exercise. What might be happening with the Wii is that the repeated feedback influences this system’s patterns.

So, what are the product and training implications? I will simply bullet some of the more salient points that Sandy makes:

- We must eliminate any incorrect feedback that the system issues now – there is plenty of it evident. We need to use the body orientation data and create a better feedback logic. We should not be giving feedback that causes ping-pong patterns of walking.

- We should be including instructions for the teachers and users to make adjustment to the line of travel by using trunk movement, not feet movement. We need to look at the words/text we use for the feedback and the advice in the user guide.

- We should begin at each end of the course with the student aligned from the floor to the pelvis, then ask the student to align their trunk forward. This need serious consideration, and an effective and easy squaring-off platform that reaches from the floor to around the pelvis.

If all this works perfectly there is one part of the assessment of the device that we have not yet looked at. Sandy and I will talk about how I will do this when I return to the Bronx for final testing in June. THE BIG QUESTION: how much will the proprioceptive changes generalize, and will they be effective in the real world when the student does not travel on an ideal surface. Changes in the travel surfaces are inevitable. These will interpret the new patterns achieved with training, and student will revert to their old patterns. How much training and the degree of benefit in the real world need to be looked at. On this topic I have my single experience (n=1) with a student I trained with the equivalent of over 100 trails on a WiiCane at HKNC. He learned to walk in a straight line indoors, and has demonstrated this in the real world consistently since training. We are looking to see if this will be typical.

WiiCane April 2010 with built in ferrules

This is the final version of the WiiCane that we will go into production with.  This is the child size version, which adjusts from 32" to 42".  There's an adult version which adjusts to the maximum length of standard canes.

This version is a big improvement over the last one. Here, we have worked with the cane manufacturer, Ambutech, to develop a better way for inserting the aluminum fixture that holds the wii remote device and allows it to pivot.  In the earlier version aluminum collars were welded onto the ends of the fixture; set screws were used to hold the ends of the graphite shafts in the collars. Now, we have an internal threaded sleeve that is pressure fit into the cut ends of the shaft, and two big machine screws hold it all together. Not having the collars makes it look sleeker, and it weighs less.  But the big benefit is that the cane feels a lot more rigid and stable, and should not get loose after repeated banging.

New design for the overhead light track

Zach and I made progress in our meeting today on the design of the light strip. We figured out how to connect the strips in such a way that makes it easy for one person to hang it up and connect each 6 foot long strip end-to-end.  This is a three part system: the first part is a plastic clip that gets screwed to the ceiling. The first track section slides around this clip, then the next one is added.  As each track section goes up, the installer places a connector that hides the joint, and makes the 8-wire electrical connection. Here's a detail drawing that illustrates the system's components.

post from Dr. Annette Gourgey

Updated Evaluation Plan

Gene and I met this morning to draw up a revised plan for the WiiCane data analysis. We see this taking form in several ways.

First, Gene has identified some case studies from our first exploratory trials with the WiiCane that suggest ways that the device is beneficial for student learning. He has posted some of these findings on this blog. One set of graphs plots the number of corrective feedbacks given over successive trials, showing that the frequency of these feedbacks may decrease with practice. This suggests that students are learning something from repeated use of the device that carries over into later trials.

A second set of graphs Gene has created shows an effect on veering. When the number of nonveering messages (indicating correct positioning) exceeds the number of corrective messages (indicating veering error), improvement has occurred. A plot of the arithmetic difference (nonveering minus veering messages) shows that this balance may improve with practice. Again, this demonstrates the potential for learning with practice using the device.

The cases analyzed so far took place during the exploratory phase of our trials, when we were experimenting with the optimal number of trials and adjustment of parameters such as the tolerance threshold for veering (which varied from 12 to 18 inches). Our next step would be to set up more consistent trials with several participants, in which the number of trials and the veering tolerance are held constant. Thus, we plan to select three representative students from NYISE and to test them under these conditions: three 30-trial sessions over a period of three days, for a total of 90 trials; and a veering tolerance level of 12”. These conditions are consistent with those identified by Guth in his previous research, which demonstrated that improvement in veering could be observed after several successive days of sessions consisting of 30 trials each.

This experiment would provide a more systematic evaluation of the potential of the WiiCane device to reduce the incidence of veering. With more reliable data from such an experiment, we will be better positioned to design future research with larger samples and to advocate for the benefits of using an automated tool to improve mobility training. This knowledge will be beneficial both for instructors who wish to evaluate how the device will help their students and for our ability to market the device.

Another deafblind student using vibratory feedback

This is day 2 and day 3 veering reduction for subject ST. (Day 1 was only exposure and practice on the system.)

There is only a total of 24 trials and the learning curve appears more erratic.

influencing veering behavior with vibratory feedback only

This is exciting to report. We tested the WiiCane for veering reduction only at HK last week. The students were deaf and hard-of-hearing. None used spoken English feedback; all used vibration delivered by remotes attached to the arms.

I reviewed the video data for two students for three days. I made sure that the data are accurate by viewing every trial, not using the computer system-generated data. This means that when the parameters were temporarily changed or the system malfunctioned due to user error, I eliminated the few bad trials. The results are we have some clean data and we can see learning with this subject. I looked at another student and there was learning but less dramatic and the data was not as clean because the parameters for the device were different on two days of trials.

For the one subject I will call SG, the improvement in reduced veering is clear. Here are the mean number of feedback prompts for veering for this subject:

Day 1 cumulative trials 11 6.36
Day 2 cumulative trails: 34 3.30
Day 3 cumulative trials: 49 1.87

This indicates to me that the feedback system can work with vibration only, and can work with deafblind travelers. How consistently and compared to spoken English we do not know for certain.

The graph is above.

design proposal for the tilting belt holder for the center Wii Remote

Here is my idea for a belt holder for the Wii Remote that will go in the center of the user's back.  This one is different from the simpler version for holding the Wii onto the sides of the belt; since this one needs to be able to "see" the light strip, it has to be able to tilt back, so that the camera's view clears the user's body, hair or clothing. The belt has to be pretty tight to keep the belt itself from rotating, which would end up tilting the unit too much, and make it too bouncy. As much as possible, the Wii Remote must be kept stable and as free from bouncing and jiggling as possible, because too much movement will have a negative effect on system accuracy in measuring veering, gait, rotation, and distance. 

belt clip prototype without tilt feature

This picture shows a prototype of the belt holder for the Wii remote units that will be clipped to the user's belt. This is for the upcoming trials at Helen Keller National Center, where we will be testing WiiCane with users who are deaf-blind. Because they can't hear well enough to use audio feedback via wireless headphones, we are experimenting with providing them with vibratory feedback to steer them during veering exercises. This means that the participants will wear one of these belt units on each side, and we will pulse the rumble motors in the Wii remote to signal to them that we want them to correct to the right or the left. This clip grips the belt very snuggly, and so I think that this will lead to good conduction of the vibration to the user's body. While this design is very simple and inexpensive to produce, I don't think that we can also use it for the (third) Wii remote that is being used to "see" the light strip above, because in that case we need the capability to tilt the device backwards for some users so that the camera view is not obscured by their clothing, hair or body bulk (see previous post).  But, for the two side-mounted Wii remotes, this should work better, and will be cheaper to produce.

belt clip design with tilt feature

GUI and adaptive algorithms for feedback

Zach and I met today to discuss our next steps. We came up with an approach to presenting the Graphical User Interface for WiiCane when it becomes a commercial product. Currently, we have a very crowded screen that includes controls and text boxes for setting all of the feedback thresholds and other conditions. That screen is just for our purposes during testing. But, our users will want something much simpler, that is tied to the curriculum that Gene and the reviewers (Bonnie, Donna and Rob) are creating. So, we are proposing to create a different screen for trainees and instructors to use that does away with all of the controls, and instead provide a list of lessons. Once logged on, a user would touch the screen to choose a lesson; choosing a lesson, sets up all of the conditions for that activity. For example, if you choose "Lesson 1: Veering", the system will turn on both corrective and positive feedback for veering, but no other feedback. The system would assume a default value for the veering threshold (the distance away from the center line that you go before you hear the correction "go right" or "go left").  "Lap swimming" would be turned off for Lesson 1, which only calls for a single traverse of the course. When you select Lesson 1, instructions for tapping three times to start and stop the feedback is displayed and is spoken through the computer and the headphones. Later, if you want to switch to Lesson 4, the veering feedback turns off, and the rotational feedback turns on, and the instructions are again displayed and spoken.

With this approach, we control everything with presets or styles. These are factory settings for each feedback condition or operational mode, based on our experience of using the system. The user will not turn things on or off, or set the thresholds (which turns out to be very trick to do).  Instead, the system will configure itself according to the lesson, and the feedback threshold will be controlled dynamically.  So, as the trainee practices one of the lessons, the computer pays attention to his or her performance and spontaneously and continuously adjusts the thresholds.  This way, as a trainee gets better at doing the task, the tolerance for being correct is reduced. This is an adaptive system similar to ones used in standardized assessments, where questions are selected based on the test-taker's success rate in answering earlier ones. I am guessing that a system like this will promote very rapid improvement, even though the user might not realize he was getting better, because it would seem that it was always equally difficult to get the positive feedback!

This approach will also simplify the process of rating student performance so that we can tell whether learning is occurring. If we see a downward trend in the threshold distances for any given exercise, we know that the student is improving, at least in doing that one thing better (such as veering).  That trend, however, tells us nothing about arc width, but that could be measured simultaneously in the same way. So we may end up with a multi-dimensional score, by which a trainee can be given an overall performance rating. Another virtue of the system we have in mind is that it would be easy to add or revise lesson profiles or presets using a single, very tiny text file that could be easily updated automatically if they connect the computer to the internet. This way, we can update the program remotely very easily.

I am interested in knowing what project staff thinks of this proposal.

article from Scientific American Mind that is highly relevant to our project

I just read an interesting article that discusses improved learning outcomes when students are asked to try to do something without any practice or experience first, then undergo training, then try to do it again.  Apparently, the act of trying and failing first primes the brain to acquire the new skill or knowledge more rapidly and with greater retention.  This bears directly on the way we are structuring out tests of WiiCane: we start by asking the student to try to walk straight without any feedback, then we turn on the feedback and have them do repetitive traverses with audio or vibratory coaching, then finally we turn off feedback and see if their performance improves as compared to their initial state.  Given the findings of the research discussed in the article, it might be good for us to rely on this technique in each of the lessons we are constructing for the WiiCane curriculum. 

If you want to read the article, click here. 

taking the blog private

I have begun discussions with Ted Sabety, our patent attorney, about commencing the process of applying for a US patent for some of the ideas in WiiCane. Ted feels that our use of an overhead array of computer-addressable lights for motion tracking may be both novel and non-obvious, requirements for patent protection. Since the blog discloses details of this, it is no longer appropriate to allow for public access, and therefore I have reset the permissions on the blog so that only contributors may read it. needless to say, I wish we didn't have to do this, because I like the idea that other researchers can find out about our work here, but apparently this could interfere with the patent process. Of course, we could also decide not to patent, but Ted feels that this is an important step from a business perspective, and the commercial viability of the product might require that we protect the intellectual property embodied in the WiiCane system. I am certainly open to discussion about this.

Details of the production version of the Wiicane and associated apparatus

Collaboration with Dr. Sandy Rosen

Just met with Dr. Sandra Rosen at SFSU. She was very interested and excited about our WiiCane work. She’s familiar with Guth, and her own work has focused on similar concerns. Her background is O&M and PT.

Sandy's work has looked at gait and posture (both set in place by early teen years) and their influences on veering behavior. Sandy proposed that we could do a joint project. Evaluate gait and posture before treatment, train with the WiiCane system, then re-evaluate gait and posture of individuals whose veering has diminished.. The evals are brief (~30 minutes, I believe she said). We could see if gait is actually altered by the traveler to achieve straight-line travel. Imagine – we could make a significant contribution to understand what cause and how to ameliorate veering. This would be great research!

Thoughts?

LESSON 4 for proprioceptive awareness: body rotation by degrees

I have sent lesson 4 out to the reader/reviewers for feedback. Looking forward to getting the input. This lesson forms the basis for several others in the curriculum/guide.

Designing the overhead light track for WiiCane

Zach and I are beginning to look at the redesign of the overhead light track for a production version. We plan to use a clear 1" or 3/4" square clear acrylic extrusion . This is a highly transparent material that hopefully will not introduce any optical bending or distortion of the IR light emitters. This material costs about 40 per foot when bought in quantities of 1000 feet. For us to manufacture 50 sets we will need  50 times 32 or 1600 feet. At this quantity, the raw material cost is $12.80 for each WiiCane set.  It comes in 8' lengths, so we will cut these in half; when all of these sticks are bundled up for shipping, the entire package will be only 4' x 3" x 3".  With a four-sided extrusion, we will be sliding the circuit boards in through the ends rather than laying them down as with a three-sided shape. we also need to figure out how to fasten the boards to the interior of the tube from the outside. The inside dimension is either 7/8" or 5/8".  It will have to be possible to ensure that the lights lie flat and evenly spaced within each section. Zach is currently deciding whether the material of the boards will be rigid or flexible; this will impinge on how these sections will join up and how they will be mounted to the ceiling. I want to continue with the assumption that the apparatus can be supported either with ceiling clips or on tension poles. I think some people will not be comfortable shooting nails into their ceiling, and they might prefer the pole set up, which will be priced higher ($1,000 vs. $1,250). We will continue to 3D print the suspension clips for now. We will need 50 times 8, or 400 clips for the first batch. 

We should consider the virtue of making a suspension clip that combines the structural and electrical functions. If the board substrate is like a tape, then I do think we need to add some kind of tounge to add rigidity, and this could also slide into the adjacent section acyrlic section and snap into place, thereby making the connections of the three conductors. I am going to start ordering some sample materials and we can run some tests here in the office. It appears that brightness and equivalent light levels in the 64 IR LED's in the array is going to yield the best performance, and the only way we can be sure that it works is by testing. I reviewed a video from the Overbrook testing and noticed that the good chiming feedback stops near the end of the course, which may be caused by some amount of power drop off along the length of the course.

This evening I looked at another subject. It's clear over 43 trials there is noticeable improvement in veering behavior. One of the interesting things is this subject never achieved more "no veering" than "veering" feedback, until after a night's rest when her results went from always negative to general and increasingly positive (after trial 18). Again, this is far less trials than we would use in real world training.

I have some graphic (as in graphs!) information on an Overbrook subject who did almost 60 trials over two days. You can see the changes in veering behavior with these two views. One (bottom) is just looking at the number of veering feedback prompts the student received. The (top graph) is a look at the number or no-veering minus veering prompts (adjusted for the time of the trial) which show another more subtle change in behavior. REMEMBER that according to Guth this student has had less than half the trials he applied before he noticed real change in his subjects.

I won't show it here, but our youngest and more cognitively involved subject showed some improvement in fewer veering prompts but with great variability in behavior throughout his three days of trials.

This looks very promising. We need to remember also that these trials were not tightly controlled, some variables in tolerance were adjusted over the days, and that more rigorous and study with more trials might yield much more significant results. Subjects (the small sample I looked at) do not seem to have consolidated or stabilized the straight-line behavior.


BTW, I looked at these subjects based solely on whether thaey veered at all when they began the trials (a few students did not) and whether we actually were able to conduct good testing with them; I did not select out the best (most iorived) subjects to look at because honestly I couldnt tell you who they would be.

Another look

Here is a chart that takes into effect the time it took to complete each trial (no veer - veer)/time. When we consider the time of the trial, the improvement in veering is even more obvious.

First look at WiiCane feedback on single user. One day only 19 trials

This a subject from Overbrook, on a single day, with 19 trials down the course. You can see the effects. The x-axis is the trial number. The two lines plot the number of "no veering" conditions and "no veering" less the number of veers.

Even in this brief look at a single subject the positive effects can be seen. I'll be doing some more simple presentations soon.

development of the curriculum

I am please to say that Ellen, Dona, Bonnie, and Rob have all agreed to be readers/reviewers of the curriculum materials that Dr. Gourgey and I are developing. With this team we cannot fail!

I have completed a skeleton for the curriculum document, detailing all the major sections and lessons. I will be working on completing some of the core content in coming weeks. Steve has asked that we complete the document by the end of February. I'm not sure we'll make that deadline, but we'll try. Naturally, sections will remain incomplete until the technology is able to monitor the tasks (e.g., cane movement, body movement and placement) and create appropriate feedback for those items not yet incorporated into the device menu.

Thanks. Gene