WiiCane

video showing 9 section (54 feet long) light track at New York Institute for Special Education

2010-07-03T16:37:00.000-04:00

New Lightstrip video

2010-04-27T18:16:00.002-04:00

In the short video below I demonstrate the newly-redesigned WiiCane lightstrip and working "find-the-user" routines.

New design for the overhead light track

2010-04-25T13:31:00.000-04:00

Here's the latest design. This one uses anodized aluminum channels with the open side facing down.

Discussions with Dr. Rosen

2010-04-15T08:30:00.004-04:00

I have spent the past few weeks in discussions with Dr. Rosen regarding the WiiCane, and being educated in the biomechanical and sensory aspects of veering. I will summarize some of our discussions and end with some implications for product design and teaching approaches with the device.

Sandy has reviewed a few hours of video and I have sent her the recordings of two more students. I should note that the data are not ideal for analysis so Sandy’s comments were based on what was possible to glean from the images.

It is highly improbable that training with the WiiCane is influencing students’ gait. Sandy said, “Gait patterns stabilize around age 6-7 according to all medical research.. . . gait is a very hard one to change long term.” As for posture (particularly pelvis-trunk alignment), which is more “fluid,” we probably are not effectively influencing this either, but we could.

So why is it that we see improvement in some students using the device? First, I’ve come to believe that early and dramatic improvement in straight-line travel is probably just becoming adjusted to using the feedback system. But the long-term changes in walking a straight-line is most likely from the training’s (exercise) impact on proprioceptive awareness. The proprioceptive system is a balance system The one that is not vestibular) — that makes normal, slight adjustments to balance. It is (as Gene understands it) neurologically related to muscle systems and tone. Individuals can be trained throughout life (not like gait patterns) to fine tune this system; effective training is active exercise. What might be happening with the Wii is that the repeated feedback influences this system’s patterns.

So, what are the product and training implications? I will simply bullet some of the more salient points that Sandy makes:

- We must eliminate any incorrect feedback that the system issues now – there is plenty of it evident. We need to use the body orientation data and create a better feedback logic. We should not be giving feedback that causes ping-pong patterns of walking.

- We should be including instructions for the teachers and users to make adjustment to the line of travel by using trunk movement, not feet movement. We need to look at the words/text we use for the feedback and the advice in the user guide.

- We should begin at each end of the course with the student aligned from the floor to the pelvis, then ask the student to align their trunk forward. This need serious consideration, and an effective and easy squaring-off platform that reaches from the floor to around the pelvis.

If all this works perfectly there is one part of the assessment of the device that we have not yet looked at. Sandy and I will talk about how I will do this when I return to the Bronx for final testing in June. THE BIG QUESTION: how much will the proprioceptive changes generalize, and will they be effective in the real world when the student does not travel on an ideal surface. Changes in the travel surfaces are inevitable. These will interpret the new patterns achieved with training, and student will revert to their old patterns. How much training and the degree of benefit in the real world need to be looked at. On this topic I have my single experience (n=1) with a student I trained with the equivalent of over 100 trails on a WiiCane at HKNC. He learned to walk in a straight line indoors, and has demonstrated this in the real world consistently since training. We are looking to see if this will be typical.

WiiCane April 2010 with built in ferrules

2010-04-14T23:10:00.001-04:00

This is the final version of the WiiCane that we will go into production with. This is the child size version, which adjusts from 32" to 42". There's an adult version which adjusts to the maximum length of standard canes.

This version is a big improvement over the last one. Here, we have worked with the cane manufacturer, Ambutech, to develop a better way for inserting the aluminum fixture that holds the wii remote device and allows it to pivot. In the earlier version aluminum collars were welded onto the ends of the fixture; set screws were used to hold the ends of the graphite shafts in the collars. Now, we have an internal threaded sleeve that is pressure fit into the cut ends of the shaft, and two big machine screws hold it all together. Not having the collars makes it look sleeker, and it weighs less. But the big benefit is that the cane feels a lot more rigid and stable, and should not get loose after repeated banging.

New design for the overhead light track

2010-04-13T20:14:00.000-04:00

Zach and I made progress in our meeting today on the design of the light strip. We figured out how to connect the strips in such a way that makes it easy for one person to hang it up and connect each 6 foot long strip end-to-end. This is a three part system: the first part is a plastic clip that gets screwed to the ceiling. The first track section slides around this clip, then the next one is added. As each track section goes up, the installer places a connector that hides the joint, and makes the 8-wire electrical connection. Here's a detail drawing that illustrates the system's components.

post from Dr. Annette Gourgey

2010-03-29T17:09:00.000-04:00

Updated Evaluation Plan

Gene and I met this morning to draw up a revised plan for the WiiCane data analysis. We see this taking form in several ways.

First, Gene has identified some case studies from our first exploratory trials with the WiiCane that suggest ways that the device is beneficial for student learning. He has posted some of these findings on this blog. One set of graphs plots the number of corrective feedbacks given over successive trials, showing that the frequency of these feedbacks may decrease with practice. This suggests that students are learning something from repeated use of the device that carries over into later trials.

A second set of graphs Gene has created shows an effect on veering. When the number of nonveering messages (indicating correct positioning) exceeds the number of corrective messages (indicating veering error), improvement has occurred. A plot of the arithmetic difference (nonveering minus veering messages) shows that this balance may improve with practice. Again, this demonstrates the potential for learning with practice using the device.

The cases analyzed so far took place during the exploratory phase of our trials, when we were experimenting with the optimal number of trials and adjustment of parameters such as the tolerance threshold for veering (which varied from 12 to 18 inches). Our next step would be to set up more consistent trials with several participants, in which the number of trials and the veering tolerance are held constant. Thus, we plan to select three representative students from NYISE and to test them under these conditions: three 30-trial sessions over a period of three days, for a total of 90 trials; and a veering tolerance level of 12”. These conditions are consistent with those identified by Guth in his previous research, which demonstrated that improvement in veering could be observed after several successive days of sessions consisting of 30 trials each.

This experiment would provide a more systematic evaluation of the potential of the WiiCane device to reduce the incidence of veering. With more reliable data from such an experiment, we will be better positioned to design future research with larger samples and to advocate for the benefits of using an automated tool to improve mobility training. This knowledge will be beneficial both for instructors who wish to evaluate how the device will help their students and for our ability to market the device.

Another deafblind student using vibratory feedback

2010-03-24T21:20:00.005-04:00

This is day 2 and day 3 veering reduction for subject ST. (Day 1 was only exposure and practice on the system.)

There is only a total of 24 trials and the learning curve appears more erratic.

influencing veering behavior with vibratory feedback only

2010-03-24T19:45:00.008-04:00

This is exciting to report. We tested the WiiCane for veering reduction only at HK last week. The students were deaf and hard-of-hearing. None used spoken English feedback; all used vibration delivered by remotes attached to the arms.

I reviewed the video data for two students for three days. I made sure that the data are accurate by viewing every trial, not using the computer system-generated data. This means that when the parameters were temporarily changed or the system malfunctioned due to user error, I eliminated the few bad trials. The results are we have some clean data and we can see learning with this subject. I looked at another student and there was learning but less dramatic and the data was not as clean because the parameters for the device were different on two days of trials.

For the one subject I will call SG, the improvement in reduced veering is clear. Here are the mean number of feedback prompts for veering for this subject:

Day 1 cumulative trials 11 6.36
Day 2 cumulative trails: 34 3.30
Day 3 cumulative trials: 49 1.87

This indicates to me that the feedback system can work with vibration only, and can work with deafblind travelers. How consistently and compared to spoken English we do not know for certain.

The graph is above.

design proposal for the tilting belt holder for the center Wii Remote

2010-03-06T19:08:00.000-05:00

Here is my idea for a belt holder for the Wii Remote that will go in the center of the user's back. This one is different from the simpler version for holding the Wii onto the sides of the belt; since this one needs to be able to "see" the light strip, it has to be able to tilt back, so that the camera's view clears the user's body, hair or clothing. The belt has to be pretty tight to keep the belt itself from rotating, which would end up tilting the unit too much, and make it too bouncy. As much as possible, the Wii Remote must be kept stable and as free from bouncing and jiggling as possible, because too much movement will have a negative effect on system accuracy in measuring veering, gait, rotation, and distance.

belt clip prototype without tilt feature

2010-03-06T08:31:00.000-05:00

This picture shows a prototype of the belt holder for the Wii remote units that will be clipped to the user's belt. This is for the upcoming trials at Helen Keller National Center, where we will be testing WiiCane with users who are deaf-blind. Because they can't hear well enough to use audio feedback via wireless headphones, we are experimenting with providing them with vibratory feedback to steer them during veering exercises. This means that the participants will wear one of these belt units on each side, and we will pulse the rumble motors in the Wii remote to signal to them that we want them to correct to the right or the left. This clip grips the belt very snuggly, and so I think that this will lead to good conduction of the vibration to the user's body. While this design is very simple and inexpensive to produce, I don't think that we can also use it for the (third) Wii remote that is being used to "see" the light strip above, because in that case we need the capability to tilt the device backwards for some users so that the camera view is not obscured by their clothing, hair or body bulk (see previous post). But, for the two side-mounted Wii remotes, this should work better, and will be cheaper to produce.

belt clip design with tilt feature

2010-03-03T14:13:00.000-05:00

GUI and adaptive algorithms for feedback

2010-03-02T18:47:00.001-05:00

Zach and I met today to discuss our next steps. We came up with an approach to presenting the Graphical User Interface for WiiCane when it becomes a commercial product. Currently, we have a very crowded screen that includes controls and text boxes for setting all of the feedback thresholds and other conditions. That screen is just for our purposes during testing. But, our users will want something much simpler, that is tied to the curriculum that Gene and the reviewers (Bonnie, Donna and Rob) are creating. So, we are proposing to create a different screen for trainees and instructors to use that does away with all of the controls, and instead provide a list of lessons. Once logged on, a user would touch the screen to choose a lesson; choosing a lesson, sets up all of the conditions for that activity. For example, if you choose "Lesson 1: Veering", the system will turn on both corrective and positive feedback for veering, but no other feedback. The system would assume a default value for the veering threshold (the distance away from the center line that you go before you hear the correction "go right" or "go left"). "Lap swimming" would be turned off for Lesson 1, which only calls for a single traverse of the course. When you select Lesson 1, instructions for tapping three times to start and stop the feedback is displayed and is spoken through the computer and the headphones. Later, if you want to switch to Lesson 4, the veering feedback turns off, and the rotational feedback turns on, and the instructions are again displayed and spoken.

With this approach, we control everything with presets or styles. These are factory settings for each feedback condition or operational mode, based on our experience of using the system. The user will not turn things on or off, or set the thresholds (which turns out to be very trick to do). Instead, the system will configure itself according to the lesson, and the feedback threshold will be controlled dynamically. So, as the trainee practices one of the lessons, the computer pays attention to his or her performance and spontaneously and continuously adjusts the thresholds. This way, as a trainee gets better at doing the task, the tolerance for being correct is reduced. This is an adaptive system similar to ones used in standardized assessments, where questions are selected based on the test-taker's success rate in answering earlier ones. I am guessing that a system like this will promote very rapid improvement, even though the user might not realize he was getting better, because it would seem that it was always equally difficult to get the positive feedback!

This approach will also simplify the process of rating student performance so that we can tell whether learning is occurring. If we see a downward trend in the threshold distances for any given exercise, we know that the student is improving, at least in doing that one thing better (such as veering). That trend, however, tells us nothing about arc width, but that could be measured simultaneously in the same way. So we may end up with a multi-dimensional score, by which a trainee can be given an overall performance rating. Another virtue of the system we have in mind is that it would be easy to add or revise lesson profiles or presets using a single, very tiny text file that could be easily updated automatically if they connect the computer to the internet. This way, we can update the program remotely very easily.

I am interested in knowing what project staff thinks of this proposal.

article from Scientific American Mind that is highly relevant to our project

2010-03-02T12:25:00.002-05:00

I just read an interesting article that discusses improved learning outcomes when students are asked to try to do something without any practice or experience first, then undergo training, then try to do it again. Apparently, the act of trying and failing first primes the brain to acquire the new skill or knowledge more rapidly and with greater retention. This bears directly on the way we are structuring out tests of WiiCane: we start by asking the student to try to walk straight without any feedback, then we turn on the feedback and have them do repetitive traverses with audio or vibratory coaching, then finally we turn off feedback and see if their performance improves as compared to their initial state. Given the findings of the research discussed in the article, it might be good for us to rely on this technique in each of the lessons we are constructing for the WiiCane curriculum.

If you want to read the article, click here.

taking the blog private

2010-02-25T16:31:00.000-05:00

I have begun discussions with Ted Sabety, our patent attorney, about commencing the process of applying for a US patent for some of the ideas in WiiCane. Ted feels that our use of an overhead array of computer-addressable lights for motion tracking may be both novel and non-obvious, requirements for patent protection. Since the blog discloses details of this, it is no longer appropriate to allow for public access, and therefore I have reset the permissions on the blog so that only contributors may read it. needless to say, I wish we didn't have to do this, because I like the idea that other researchers can find out about our work here, but apparently this could interfere with the patent process. Of course, we could also decide not to patent, but Ted feels that this is an important step from a business perspective, and the commercial viability of the product might require that we protect the intellectual property embodied in the WiiCane system. I am certainly open to discussion about this.

Details of the production version of the Wiicane and associated apparatus

2010-02-21T16:49:00.003-05:00

Collaboration with Dr. Sandy Rosen

2010-02-16T18:18:00.002-05:00

Just met with Dr. Sandra Rosen at SFSU. She was very interested and excited about our WiiCane work. She’s familiar with Guth, and her own work has focused on similar concerns. Her background is O&M and PT.

Sandy's work has looked at gait and posture (both set in place by early teen years) and their influences on veering behavior. Sandy proposed that we could do a joint project. Evaluate gait and posture before treatment, train with the WiiCane system, then re-evaluate gait and posture of individuals whose veering has diminished.. The evals are brief (~30 minutes, I believe she said). We could see if gait is actually altered by the traveler to achieve straight-line travel. Imagine – we could make a significant contribution to understand what cause and how to ameliorate veering. This would be great research!

Thoughts?

LESSON 4 for proprioceptive awareness: body rotation by degrees

2010-02-16T00:49:00.001-05:00

I have sent lesson 4 out to the reader/reviewers for feedback. Looking forward to getting the input. This lesson forms the basis for several others in the curriculum/guide.

Designing the overhead light track for WiiCane

2010-02-14T18:43:00.002-05:00

Zach and I are beginning to look at the redesign of the overhead light track for a production version. We plan to use a clear 1" or 3/4" square clear acrylic extrusion . This is a highly transparent material that hopefully will not introduce any optical bending or distortion of the IR light emitters. This material costs about 40 per foot when bought in quantities of 1000 feet. For us to manufacture 50 sets we will need 50 times 32 or 1600 feet. At this quantity, the raw material cost is $12.80 for each WiiCane set. It comes in 8' lengths, so we will cut these in half; when all of these sticks are bundled up for shipping, the entire package will be only 4' x 3" x 3". With a four-sided extrusion, we will be sliding the circuit boards in through the ends rather than laying them down as with a three-sided shape. we also need to figure out how to fasten the boards to the interior of the tube from the outside. The inside dimension is either 7/8" or 5/8". It will have to be possible to ensure that the lights lie flat and evenly spaced within each section. Zach is currently deciding whether the material of the boards will be rigid or flexible; this will impinge on how these sections will join up and how they will be mounted to the ceiling. I want to continue with the assumption that the apparatus can be supported either with ceiling clips or on tension poles. I think some people will not be comfortable shooting nails into their ceiling, and they might prefer the pole set up, which will be priced higher ($1,000 vs. $1,250). We will continue to 3D print the suspension clips for now. We will need 50 times 8, or 400 clips for the first batch.

We should consider the virtue of making a suspension clip that combines the structural and electrical functions. If the board substrate is like a tape, then I do think we need to add some kind of tounge to add rigidity, and this could also slide into the adjacent section acyrlic section and snap into place, thereby making the connections of the three conductors. I am going to start ordering some sample materials and we can run some tests here in the office. It appears that brightness and equivalent light levels in the 64 IR LED's in the array is going to yield the best performance, and the only way we can be sure that it works is by testing. I reviewed a video from the Overbrook testing and noticed that the good chiming feedback stops near the end of the course, which may be caused by some amount of power drop off along the length of the course.

2010-02-10T00:25:00.005-05:00

This evening I looked at another subject. It's clear over 43 trials there is noticeable improvement in veering behavior. One of the interesting things is this subject never achieved more "no veering" than "veering" feedback, until after a night's rest when her results went from always negative to general and increasingly positive (after trial 18). Again, this is far less trials than we would use in real world training.

2010-02-08T18:16:00.005-05:00

I have some graphic (as in graphs!) information on an Overbrook subject who did almost 60 trials over two days. You can see the changes in veering behavior with these two views. One (bottom) is just looking at the number of veering feedback prompts the student received. The (top graph) is a look at the number or no-veering minus veering prompts (adjusted for the time of the trial) which show another more subtle change in behavior. REMEMBER that according to Guth this student has had less than half the trials he applied before he noticed real change in his subjects.

I won't show it here, but our youngest and more cognitively involved subject showed some improvement in fewer veering prompts but with great variability in behavior throughout his three days of trials.

This looks very promising. We need to remember also that these trials were not tightly controlled, some variables in tolerance were adjusted over the days, and that more rigorous and study with more trials might yield much more significant results. Subjects (the small sample I looked at) do not seem to have consolidated or stabilized the straight-line behavior.

BTW, I looked at these subjects based solely on whether thaey veered at all when they began the trials (a few students did not) and whether we actually were able to conduct good testing with them; I did not select out the best (most iorived) subjects to look at because honestly I couldnt tell you who they would be.

Another look

2010-02-07T23:34:00.003-05:00

Here is a chart that takes into effect the time it took to complete each trial (no veer - veer)/time. When we consider the time of the trial, the improvement in veering is even more obvious.

First look at WiiCane feedback on single user. One day only 19 trials

2010-02-07T22:36:00.005-05:00

This a subject from Overbrook, on a single day, with 19 trials down the course. You can see the effects. The x-axis is the trial number. The two lines plot the number of "no veering" conditions and "no veering" less the number of veers.

Even in this brief look at a single subject the positive effects can be seen. I'll be doing some more simple presentations soon.

development of the curriculum

2010-01-07T11:26:00.002-05:00

I am please to say that Ellen, Dona, Bonnie, and Rob have all agreed to be readers/reviewers of the curriculum materials that Dr. Gourgey and I are developing. With this team we cannot fail!

I have completed a skeleton for the curriculum document, detailing all the major sections and lessons. I will be working on completing some of the core content in coming weeks. Steve has asked that we complete the document by the end of February. I'm not sure we'll make that deadline, but we'll try. Naturally, sections will remain incomplete until the technology is able to monitor the tasks (e.g., cane movement, body movement and placement) and create appropriate feedback for those items not yet incorporated into the device menu.

Thanks. Gene

thoughts about the upcoming staff meeting and the future

2009-12-27T16:19:00.000-05:00

In preparing for our meeting, which is scheduled for Wednesday morning, December 30 at 10 AM, we have to cover a lot of ground. We are at a critical moment for the project and have to make some rapid decisions that will shape the final product and our activities over the next five months. Here are the general categories of things that I want to talk about.

The results of the testing at overbrook.

Videos
Questionnaires
Data logs

Review of current system design, focusing on:

Feedback behaviors
User interface issues
Belt holder ergonomics and optics
WiiCane weight, length and balance
Physical installation
Data logging

Next tests

Need to be more empirical and hypothesis driven, less usability focused.
Incorporating vibratory feedback for deaf-blind users
Maybe adding a belt unit wtih two additional wii remotes for directional signalling
Scheduling for Guild and HKNC tests

WiiCane Curriculum

Purpose and overview of the wiiCane curriculum
working method
schedule

Commercialization

Publications
Conferences
Marketing
IP

There's a great deal to discuss and plan for, and I am really optimistic about this project right now. While we have yet to experimentally prove that WiiCane has therapeutic benefits, I think that we need to go full speed to bring this to market. the low cost means that people will want it even if it is only fun and not proven (yet) to be beneficial. the main thing that I learned from two rounds of testing we have done thus far is that kids enjoy doing it, if it is presented to them as a game, and it seems like we can make it challenging enough to sustain their interest. So, of course we need to make it truly educational, and not just a toy, especially we are expected to be carrying out evidence-based research, but even this preliminary observation appears positive.

Design for WiiCane pole at the end of the walking course

2009-12-26T14:42:00.001-05:00

Here is a proposal or how to organize the WiiCane system components to make it easier to keep track of everything. In this picture, you can see that I mounted an all-in-one touch screen PC on an adjustable height tension pole, and added a horizontal aluminum rack a few inches above the screen. The rack makes a convenient place to hang the cane, the headphones, the blindfold and the belt unit. The USB controller for the light track is mounted out of sight on the back of the PC, since there is no user access required to that component. Configuring and launching wiiCane will be via a basic graphical user interface with a few big buttons on the PC screen. Instructors and students will operate the system with speech assistance, and we will supply a transparent tactile graphic overlay sheet to be mounted over the screen, so that users can feel the locations of buttons.

In cases where the wiiCane track is being fastented to the ceiling, and there is no pole, the rack for the wiiCane accoutrements can be mounted to the wall adjacent to the walking course.

response to gene's summary of existing literature on cane training

2009-12-26T11:54:00.004-05:00

Gene, this is a great start. It is very helpful to see a complete list of the categories of cane skills that O&M teaches. Now that we know in general the kinds of movements that our apparatus can measure, we need to create a curriculum for the WiiCane so that instructors and students can maximize their results. This curriculum should provide instructions for using WiiCane in a linear, cumulative way, to promote rapid acquisition of skills and habits needed for safe and efficient travel with a long cane. It should present training activities as a series of games, where students are challenged to improve their performance by mastering skills of escalating difficulty.

In looking at the list of skills customarily included in O&M training (according to the text books you have consulted), it appears that some skills will be quite easy to teach using our system, while some others could very hard, with the rest probably impossible or at least impractical to achieve in this 24 month project. In sorting through these, I think we can add program modules to accomplish some things right away. For example, it would be easy to add features that measure pivoting movements, with feedback played in the headphones as the user's orientation changes. In our curriculum, we could add an activity where the user is walking along under the light track, and as he approaches the end of the course, an instruction is played in the headphones to stop and turn 180 degrees. We would then turn off veering feedback and just look at rotation via the belt-mounted camera. We could provide a short pulse (either accoustical or vibratory) when the student has reversed direction. This behavior and feedback scenario could then be built into a "lap swimming" task (for advanced students), where they would have to practice going back and forth on course, turning at each end with the help of this rotational feedback. This is one example of how skills taught in standard cane training could be integrated into exercises carried out with WiiCane. The challenge will be to organize these exercises in some linear fashion to create a curriculum which addresses the needs of each user population.

Developing a WiiCane curriculum

2009-12-24T18:57:00.001-05:00

Developing a WiiCane curriculum; the biomechanics and other aspects of travel that can be detected and that can be influenced by automated feedback.

I reviewed the two classic texts below to compile this list as the beginning of developing an official curriculum for developing and using the WiiCane.

Skills so far tested:
- veering reduction
- cane arc width and coverage, stationary
- cane arc width and coverage, ambulating

Skills that might be developed, orientation:
- proprioception, rotating by degrees: e.g. turning 90 degrees, turning 180 degrees
- using compass directions
- using lateral directs
- interpreting kinesthetic feedback
- consistent forward movement
- adjusting line of travel from a shoreline with two adjustments
- identifying object materials

Skills that might be developed, cane articulation:
- two-point touch technique – arc apex height
- two-point touch technique – in-step
- two-point touch technique – in rhythm
- touch and slide technique
- touch and drag technique
- shortened touch technique
- three-point touch technique
- diagonal cane position
- hand centeredness (mid-sagittal plane)
- wrist role

Hill, E., & Ponder, P. (1976). Orientation and Mobility Techniques: A Guide for the Practitioner. New York, NY: American Foundation for the Blind.

Jacobson, W. H. (1993). The Art and Science of Teaching Orientation and Mobility to Persons With Visual Impairments New York, NY: American Foundation for the Blind.

Overbrook School Testing Report

2009-12-20T21:19:00.004-05:00

Introduction

I just returned from two days of testing WiiCane at Overbrook School for the Blind on Thursday, December 17 and Friday, December 18. I worked with 3 O&M instructors and 7 high school students on wiiCane. The following report documents the physical apparatus and setting, observations made during testing, and implications of the experimental findings on our plans going forward.

Figure 1: Fitness Room with the WiiCane overhead light track installed.

Physical apparatus and setting.

We tested the system in the Fitness Room at the school's Field House (see figure 1, above). The room is approx. 40' x 24', and has an 11' high ceiling. We moved exercise equipment to the perimeter of the room, which opened up a clear floor area of 32' x 14'. The floor surface was hard vinyl tile. The room has windows high up on one of the long sides, and also has high windows on one short end. We set up two telescoping aluminum poles 32' apart; the poles extended from floor to ceiling, and were held firmly in place by pressure. A vinyl-coated suspension cable hung between the two poles, and the WiiCane system's plastic lighting track was hung with clips from the suspension cable. Power and control wires for the 64 IR LED's, spaced out at 6" intervals along the light track, ran down the pole at the goal and of the course, and connected to the computer running a Java-based control and data collection application. The user interface for the program appears in figure 2 below.

Figure 2: WiiCane's Experimental Graphical User Interface

Using this program, a teacher or experimenter can create and manage user accounts; see a real time view of the light strip from cameras mounted on the cane and the user; watch a visualization of the user's and cane's position in real time as the system is used; control which feedback is active, and the intervals for repeating various feedbacks; and start and stop sessions. Observed user actions that the system can respond to with corrective messages are:

veering right
veering left
veering tolerance
not veering or just corrected from veering
hand not centered
arc too wide
arc too narrow
arc correct
arc tolerance

Procedure

Each participant worked with the WiiCane system for 45 minutes on each of two days. Sessions were scheduled by the staff, and participants did not appear at the same time each day. Their instructors stayed with the students as they used the system. Only veering feedback was given on day one, and the student was allowed to practice walking the course until they could walk 32' with no more than two corrections.
On Day Two, feedback was given for both veering and arc width. Participants walked the course an average of 15 times in each session. At the end of both days, participants tried walking the course with all feedback switched off to see if their ability to walk straight and use proper cane technique had improved after extended practice.

Figure 3: A participant walking the course with WiiCane as her instructor looks on.

Data Collection

The tests generated data in the following forms:

A continuous video was made of each session. On the first day of testing, I wore headphones to listen to the feedback as it was simultaneously playing in the wireless headphones worn by the use. This was to eliminate any possibility that the student could get directional information to help him walk straight by listening to external speakers. On day 2, we played the feedback over a speaker on the computer, so it may have been possible to use that sound as a beacon. This means that the feedback is audible on the video, which will make analysis easier.
The Java program produced a log file for each session. These files can be opened directly in Microsoft Excel for statistical operations to be carried out by the Project Evaluator. The log includes all data from a session, including raw accelerometer and IR camera data and user settings and timestamped information on each feedback delivered.
Students filled out a pre-test questionnaires that collected history and status, and a post-test questionnaire that probed them about their general level of comfort using the system. These forms were collected, along with signed informed consent forms, ato be given to the Project Evaluator.

A note about videos: because of an objection from the IRB, we can't post the videos as we did for Overbrook. The person that called me explained that there is no clear rule about posting videos on You Tube, so they have decided to err on the side of caution. I offered to take it down, which she accepted. I thought that, since the person is wearing goggles, it is hard to identify him or her, but I decided not to fight about it. We are looking for a more secure way to make these videos available for streaming. It will be much easier for Annette to do the review on You Tube site, because there are a lot of videos, and the uncompressed size is very large. We should, however, decide on one or two videos that are especially good, then get explicit permission from the student and the school and the IRB, and then make that available for public use.

Observations

The most important aspect of this test was for me to watch a different population work with the system.
As with testing at NYISE, all participants were able to use the system easily after minimal practice. There was a big range of body types this time, so I was able to see the way that the equipment fits each person. In particular, I paid attention to whether the belt-mounted unit consistently pointed up, with a good view of the light strip above. There was a big difference across subjects: some participants walked smoothly, with relatively little bounce in their step or swaying of the hips, which produced much better results and consistency in the system's ability to track position than those with a rolling gait. In one overweight subject, the belt was not long enough to go all the way around the subject's waist, and a belt extension had to be fashioned from cable ties on the spot. This suggests that in future versions, a belt clip should be used, so that the body-mounted Wii device can be attached to the participant's own belt, instead of providing one as part of the product. We are currently working on a total redesign of that part of the system, and I will post about the new design later.

All feedback types were used except for hand-centeredness, which proved to be too inconsistent after the first few trials, so we decided to turn it off. Specifically, we were getting the message to "Center your hand" very often, even with the user's hand appeared to be centered. We did use the arc-too-wide, arc-too-narrow, and correct arc feedbacks. Surprisingly, users were mostly able to understand and follow all feedbacks (this is borne out both in the videos, where it is clear that users could correct their performance as feedbacks were delivered, and in the post-test questionnaire, where they indicated that they had little difficulty in understanding what the system was instructing them to do).

We observed that both the younger students at New York Institute for Special Education, and the older ones at Overbrook thought of WiiCane as a game where they were challenged to recieve as many "ding-dongs" (not veering) and "thump" (cane arc correct) sounds, and as few correction sounds ("go right", "narrower arc") as possible. This finding suggests that we should develop this aspect of the system further, perhaps creating a scoring system that would allow multiple users to compete, or other more complex activities that call for obstacle detection and following a route.

Figure 4: The wiiCane, the suspension pole at the start of the course, and the wood square-off block.

vtesting at Overbrook

2009-12-14T16:17:00.001-05:00

Testing OVERBROOK
Wednesday 12/16/2009 set up
Thursday 12/17/2009 8 AM – 2:30 PM
Friday 12/8/2009 8 AM – 2:00 PM

This procedure is proposed for testing at Overbrook School for the Blind. Steve may need to modify these procedures depending on local conditions.

Overbrook staff are scheduling students for 45 minute periods, based on availability. They will have the consents signed and the pre-questionnaires completed.

Christine will send the student names (by Wednesday) so we can set up the ID numbers.

I met with Jackie and Judy, and three O&M specialist, Christine, John, and Kim.

I met four (ood the 8) students who were available. They were great and I got to explain the entire project to them.

Procedure for trials.
45 minute sessions. Thirty minutes of veering and 15 minutes of cane coverage.

Veering:

1. On Thursday staff will check that the pre-trial survey has been completed.

2. The staff will measure the height of the belt-mounted wii remote and input the values in the program user interface.

3. For the Thursday initial trials for veering, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff. Note: the staff will emphasize that we are testing the cane, not the subject.

4. Subject will be shown the belt-attaching wii remote. The remote band will be put around the subject’s waist and adjusted to fit comfortably. The remote angle will adjusted to point vertical toward the infrared light strip. If subject is wearing loose clothing, the staff will use clips to secure the clothing so it does not interfere with the wii remote. If the subject’s hair is blocking the remote camera, it will be adjusted by the subject and/or the staff.

5. Subject will be shown the wiiCane, the imbedded remote, and the shaft. Subject will demonstrate how they move the cane left and right in constant-contact techniuqe; any wrist rolling behavior will be noted and the subject will be shown how to move the cane without rolling their wrist. (This action is taken now in anticipation of the coverage trials.)

6. Subject (with or without vision) will be shown the MINDFOLD blindfold. The subject will put on the blindfold with a tissue protecting their face and assisted to adjust them to fit comfortably.

7. Subject will put on the wireless headphones and assisted to adjust them to fit comfortably.

8. Subject will walk, guided, the 32 foot-long straight-line course at least once while the procedure is reviewed again.

9. Data collect will be turned on; feedback will be turned off. The subject will walk the course three times to collect pre-feedback data on veering, without the feedback turned on. This should be videotaped and data should be collected.

10. Subject will have the feedback system explained, including the meaning of the feedback terms (terms like “move left”) and feedback chimes.

11. The subjects will be shown the proper amount of adjustment to the left or right when they receive feedback.

12. Subject will be asked if they are ready to try the course.

13. With ascent, the subject will walk the course at least once, with staff behind them, as they listen and respond to the feedback. The staff will advise the subject of errors in responses to feedback. The subject will be encouraged to continue walking forward when he/she hears feedback. The subject will be prompted to move to the left or right in response to feedback. This may be repeated until the subject appears confident and can travel the course alone.

14. Data recording will begin. The subject will walk the course. At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

15. Subject will walk the course repeatedly until the 30 minutes expire.

16. On the second/final day of testing, the subject will walk the course three final times to collect post-feedback data on veering, without the feedback turned on. This should be videotaped and the data collected.

17. Subject will be told the time of his/her next appointment, if any.

Cane coverage:

1. For the initial trials on Thursday for coverage, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff.

2. Subject will have the feedback system explained, including the meaning of the feedback terms and feedback sounds. Terms like “wider on the right” will be explained.

3. The subject will be shown how to produce an appropriate arc width/coverage by moving their cane laterally between the feet of the staff or mobility specialist, who will stand facing the student with feet placed to demarcate the proper distance. If necessary the subject will practice the arc width movements for a short period of time.

4. Data recoding and feedback will begin, and the subject will remain stationary, moving the cane left/right and adjusting the movements according to the feedback. This may continue through several short practice intervals of just moving the cane and adjusting it to follow the feedback while not walking forward.

5. Subject will be asked if they are ready to try the cane on the course.

6. The subject will walk the course. (We anticipate the veering feedback will not typically be turned on for these trials, so staff or specialist may need to manually, from behind the subject, adjust the subject’s line of travel to remain on the course and under the infrared lighting strip.) At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

7. Subject will walk the course repeatedly until the 15 minutes expire.

8. The subject will complete the post-testing questionnaire. (Note, if additional personnel are not available, the above schedule will need to be adjusted to included time – approximately 5 minutes – for the post questionnaire.)

wiicane user interface: tactile overlay for touch screen monitor

2009-12-12T23:33:00.001-05:00

I think that when we should develop a simplified UI to allow audio tactile control of the wiicane controls. There should be a user control screen that is designed to be used in conjunction with a clear tactile overlay. buttons displayed on the monitor would be also outlined in raised lines that delineate controls to be used in conjunction with speech prompts. the monitor would be mounted on the wall or table at the beginning of the course, and users could initiate sessions and change settings in a fully accessible way. I am picturing groups of blind kids playing with this at a training center, like the Colorado Center for the Blind that I just visited in Littleton. They could keep score, restart a game, and adjust settings as they trained themselves while playing various speed and accuracy challenges. the monitor touch screen could also serve as a big slap panel that a person would hit at the end of one lap. Then, to permit people "swimming" back and forth, we would just have to make another button at the other end and mount that to the pole or to the wall, too. Maybe we can extend the wire out of the light strip and down the wall for the second button and use the control wire from the lights strip also for the button? Zach?

Wiicane Test Video

2009-12-12T16:05:00.003-05:00

All Videos are deleted from Youtube.

NYISE Day 3 testing

2009-12-12T15:29:00.002-05:00

NYISE Day 3 testing

The last day of testing was quite eventful!

Steve set up the equipment with relatively little difficulty.

We had the same eight subjects as day two. All the subjects were now familiar with the veering trials and the wii cane (day one they used their own canes), which was introduced day two. Every subject provided us with more veering trials. Most of the trials went really well, and it really seems like there were positive changes with less veering than the first day. Several times we asked subjects to walk the course with the feedback. Watching some of the subjects correct their line of travel WITHOUT THE FEEDBACK is amazing - it seemed magical with a few of them. (The data will tell!) The two subjects that had troubles on day one continued to have some difficulties on this third day. This has to do with ‘overcorrection’ and some other behaviors. When the system asked them to “go left” or “go right,” instead of making the minor adjustment with a small body rotation in the correct direction, these two subjects would make a large body rotation, in effect turning up to 90 degrees and sometimes walking perpendicular to the course. At such a point it is difficult for the subject to re-orient no matter what the feedback says. We attempted to change the feedback to say “go a little left/right” but the software would not accept the new text. Instead, it spoke the standard “go left/right” in the automated voice.

This is something to discuss and solve. The two subjects that became disoriented on the course both had cognitive involvement, and one was our youngest subject. Both were repeatedly coached repeatedly on how to adjust “just a little” before trials. They needed:
- prompts to respond to some of the feedback
- prompts when they did not move forward along the course
- physical manipulation to be reoriented to the course when they overcorrected

Can we design feedback for younger and more cognitively involved students that will address these situations? It might be that more and repeated preparation would be effective, showing the student how to adjust until they “get it” before the actual training. We simply did not have time to do this during the testing.

Right now we cannot tell where on the course the subject is located – we cannot yet track forward movement. When we can, we can certainly easily prompt students to keep moving forward when they stop.

We might need physical barriers to keep younger students on the course, not allowing them to turn 90 degree and walk off. We might try various types of feedback.

We also began to collect data on the cane arc width with most of the subjects (I believe 6 out of eight). I talked to each subject about why they would want to make their cane movements more narrow or more wide. I them let them move their canes left and right between my feet, showing them a correct width. We them had them move the cane back and forth, without walking, with the system feedback. The feedback said either wide or narrow (meaning too wide or narrow) or gave a single chime for a correct arc. Several of the kids were able to achieve repeated chimes after a few trials. For those who did well with stationary trails, we asked them to use the feedback when walking. For the most skilled subject, we gave her both veering feedback (with the positive double chime turned off after she told us it was too much) and cane arc width feedback. For this subject it actually seemed to work well!

The staff at NYISE was fantastic. The children subjects were all well-behaved. I conducted the post-testing surveys, which were also videotaped. Interestingly, not a single subject said the weight of the wiiCane was a problem, and I think 7 or 8 out of eight said that they wanted to spend more time with the wiiCane.

Finally -- dealing with some issues from the previous days. We checked with the kids about their “hearing the wall” – we were concerned that perhaps there was some echolocation happening. It turns out the subject that reported she could hear the wall was actually talking about circumstances when she is not wearing headphone and there is environmental noise to bounce off the walls. The other issue --- detecting the lights on the ceiling through the opaque goggles is still uncertain. We used a MINDFOLD sleepshade for all the subjects on Friday – the kids loved it because it is very padded and comfortable; but, we still may have little kids who are afraid of the dark and the goggles will may still be a better alternative in those situations.

Proposed evaluation plan for WiiCane data

2009-12-12T10:36:00.002-05:00

Below is an outline of a plan for the analysis of our WiiCane data. Our data collection instruments are: a pre-questionnaire asking for participant characteristics (sex, age, grade level, visual impairment, prior mobility training); data collected by the WiiCane equipment on veering and coverage performance in response to feedback; and a post-questionnaire that allows participants to describe in their own words how they liked working with the device.

(1) Descriptive statistics for all of our pre-questionnaire demographic items (sex, age, grade, visual impairment, mobility training experience, etc.)

(2) Descriptive statistics for number of trials for veering and for coverage. Correlations of number of trials with subject characteristics, if they occur (e.g. do participants with prior mobility training, or longer history of visual impairment, or different age levels, do different numbers of trials in the same time period).

(3) The learning trend over successive trials will be plotted to see if it is linear, if it shows a sudden change (indicating that the participant suddenly "gets" what to do), and the rate of learning.

(4) Since number of trials is expected to vary with subjects, we will focus on the change from the first trial to the last trial for each subject as the measure of total amount of progress. For veering, we hypothesize that the average deviation from the center will be smaller in the last trial than in the first and the number of corrective feedbacks triggered will also be smaller in the last trial. For coverage, we hypothesize that the number of corrective feedbacks will be smaller in the last trial than in the first. We can also look at whether there is a bias toward too wide vs. too narrow.

If we look at these alone, essentially we have a series of paired t-tests. We can also build in one or two factors such as age, number of years of prior training, or blind vs. deaf-blind. In that case we will have an analysis of variance.

(5) Qualitative analysis of post-questionnaire comments and recommendations for improvement.

Day 3 at New York Institute of Special Ed

2009-12-11T21:57:00.000-05:00

I am expecting a lot to happen as we go through the next six weeks, and there will be a great deal of interaction between me, Gene, Zach, Ellen and Grace to get it all done. Some other tasks that need to get underway, and that will involve various members of the project staff are:

Design, fabricate and test a new version of the belt holder. We learned a lot about how the user's hair and clothes have to be shifted in order to get good, continuous view of the light strip above. In general, we ran into trouble with hoodies, sweaters, vests, bushy or otherwise bulky hair. Gene and I will work Grace to improve the design and then reprint it in time for testing at Jewish Guild for the Blind and HKNC in January.
make sure that that the lights are being evenly powered along the length of the overhead strip. You could clearly see, in the on-screen display, that the lights were much steadier when the user was standing near the end of the light track where power and control wires connect. Zach has to look into that and make some improvements to the electronics of the controller.
Add visual indicators on screen to show when feedback is playing. It's really hard to know what the student is hearing by reading the messages sequentially, especially since things could occur at the same time.
Solve the problem of not being able to have an instructor listen to the stereo mix that the student is hearing through his own headphones or through a separate speaker. Someone in the world must have confronted this before with Bluetooth, I am hoping.
Begin to develop a strategy for producing a curriculum to go along with the system. Gene and I talked about the three existing texts on cane instruction that are in use, and he will post a discussion comparing these soon, so that as a group we can arrive at a consensus about how to teach using WiiCane. It will also be good to reintroduce Bonnie and Dona to the discussion as we enter into this part.

Wow, that's a lot of things to do. We have barely enough time for everything before the end of the project, but I think that if everyone makes steady progress on their parts, we will do it. I feel like people are going to love this product, because it is oddly fun to do, which is what all of the users agreed on, so we need to do this and make a new product to show how small business in America can work to make niche high tech products like this.

responses from subjects - day 1

2009-12-10T18:58:00.002-05:00

Annette, here are the responses from the subjects on day one. Let me know if you have questions.

8:45 – I001jk
Stereo/binaural feedback/headphones not working
Did not wear goggle
Response to trials – very positive and cooperative

9:25 – I002am
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative

10:05 I003tr
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – cooperated during first few trials, responded to feedback prompts with overcorrecting body rotations. After several trials he was not moving forward or responding to feedback, asked if he wanted to stop he said yes and we terminated. He seems fine but perhaps overwhelmed

10:45 I004gd
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – walked very slow; responded to feedback prompts with overcorrecting body rotations. Given constant prompts from the administrating staff she would respond appropriately, but not to the generated feedback alone. She was shy, very soft spoken . It was difficult to know if she enjoyed the trials.

12:00 I005av
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative

12:40 I006la
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative. She is a dancer, and from the first trial to the last she walked almost a perfect line.

1:20 I007dc
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative

NYISE Testing Day No. 2

2009-12-10T18:07:00.000-05:00

video from yesterday's test

The day got off to a bad start, because I was in charge of setting up the system and running the computer instead of Zach. I spent 90 minutes fighting with the Bluetooth connection, with copious tech support from Zach. Ultimately, it turned out that we had to use the same USB jack that we used yesterday, along with the same Bluetooth "dongle" (the external USB plug that you have to use to add Bluetooth radio service to a computer that doesn't have it built-in). Once I got both wii devices (cane and belt), as well as the bluetooth headphones all connected, everything went very smoothly; but, there is no question that this aspect of the wiiCane system is the most problemmatic right now, and we need to make connecting to all three devices very easy and foolproof (since it is likely that people operating the system will be fools, like me). I was able to catch up with the schedule, and we finished on time. We had one new student today and seven that were there yesterday.

In general, the purpose of today's testing was to see whether user ability to follow the audio prompts during travel showed measurable improvement after a night's rest. I asked them to continue to walk repeatedly along the course in one direction only. Their instructors helped them to return to the starting point each time. The biggest difference was that I asked them to use the wiiCane this time instead of their own canes. Even though no feedback was provided based on their cane movements, this gave them a chance to become familiar with the new cane, which is heavier then a normal cane, due to the additional weight of the wii device. Since the cane had a roller tip, none of them reported that the cane was too heavy. I also asked them to pay attention to maintaining their index finger of their cane hand extended along the flat face of the grip. I explained why this is important, and how the wii device had to be pointing up towards the ceiling to work properly. In most cases, I observed that they did hold the cane correctly, with a minimum of wrist roll. I don't know for sure whether this will work once we turn on the feedback for wrist roll, but to me it appeared that they can learn to hold their wrist straight with minimal training.

In terms of veering, I noticed some improvement, but since many of the kids were quite good at walking straight (with the help of our system's audio prompts ) yesterday, there was not a marked improvement among the more adept travelers. Two students who had significant trouble mastering the system yesterday did much better today, including the youngest subject, who is 7 and very little, so that suggests that the learning curve for using wiiCane is not very significant. In each test today, I asked the student to walk the course at least 10 times (or until they could do it flawlessly), then I asked them to do it a few more times with the feedback turned off. The results of this were highly variable. One girl seemed to do better with the feedback off, but the others veered substantially and never recovered. The implications of this are unclear and further discussion is needed to be sure of what is happening. It may be that when Annette reviews the videos along with the events logs from the tests, it might become clearer, or after we do more testing at the other three sites.

Tomorrow, Gene and I will go back to the Institute for the final day of testing there. We will be having them do cane movement tasks for the first time, so this will be important for us to see. I will post information about that tomorrow.

Comments on today's testing at NYISE

2009-12-09T20:42:00.000-05:00

Testing went very well today, I think. We are learning a lot from watching these kids, who have been incredibly nice and cooperative. It will be easy to get what we need from them, and I believe that we will be able to return later and see how they like the finished product. That's doubly true if we leave the finished system up there for them to use; it will be a great PR thing, maybe we can get news crews up there watching them play with it, if we make it enough like a game. It will be a really cool addition to their rec room. From speaking with the staff, it looks like there are a bunch of additional behaviors they would like us to train students on, including maintaining a low arc, etc. Once we have the horizontal movement of the cane under control, we should move into other kinds of motion capture.

Based on what I saw today, I believe that we are justified in continuing to develop this idea as a commercial product. We need to build a very open-ended framework in this early version of the software so that it will be easy to add subsequent modules. We also have to start to prepare our wiiCane curricula that will go along with the system. Gene and I spoke briefly today about that part of the project. I need to look at the schedule for the final five months of the project. We have to evaluate at the results of the tests and incorporate findings, leading up to a rapid product rollout, hopefully. I seriously think that we can begin selling these system at the end of Phase 2 (June 1, 2010). Grace is starting to do research on a better material and electrical couplings for the light track, and we also need to redesign the belt, but those things can realistically get done quickly. I know it is just the first day of testing, but considering the ease with which most of the students took to it, I am convinced that the basic principles are sound, so we should proceed full blast.

As you can see, I am very upbeat about today, and I think we should be very encouraged about the usefulness and practicality of wiiCane. I also just had an idea for my fencing application: There should be an enormous projection screen at the end of the course where we can display images from a ceiling mounted projector about 15 feet away. There could be huge image on the wall that you would face while sword fighting with a virtual opponent...

I will send a report on events tomorrow. I spoke to Zach about an hour ago, and he said that he has made the changes he needed to, including an on screen display of messages as they are played in the headphones. It looks like we will be ready to do cane movement tasks on Friday. I think that we should maybe have them just do a stationary cane movement task over and over like Gene and I discussed. That will allow us to really study their hand movements to see how serious the weight issue will be. BTW, the roller tips arrived today. They are perfect. I think they will neutralize the weight issue, as long as they are doing it on a nice smooth floor. the roller tip just slides over the pencil tip, so it looks like any slide on tip will work.

steve

REVISE: NYISE procedures and schedule

2009-12-07T18:37:00.002-05:00

Schedule for testing NYISE
December 8 (Tuesday) through December 11 (Friday)
Daily 8:30 AM-2:30PM

Tuesday
Complete setup of all equipment
Testing of equipment
Run through blindfolded TGI staff on course
O&M specialist should prepare kids, perhaps show them the route and explain what will happen

The first trial for each subject, the subject will be have the experiment explained
The testers may use whatever appropriate language is necessary, and take as much time as they feel is reasonable for the subject to understand and be comfortable.
Subjects will be asked if they are comfortable with the headphones and the goggles.
Verbal and non-verbal response will indicate whether the subject should continue. Subjects will then be guided once down the course, shown the end of the course and the safety obstacle.
Subjects will then be walked/guided back to the start of the course.
Subjects will always complete trials for veering before trials for coverage.

I am suggesting we test based on time, not number of trials. There is no way I can think of to stay on a schedule when we might see great variability with each child. Doing as many trials as possible during a prescribed period makes sense. Annette, to you see a study design problem with this?

Pre- and post-testing questionnaires will be done by TG and NYISE personnel outside the subjects’ timeslot for the scheduled trials.

I do not really care the order of subjects. It should not be relevant. Whatever works best for the institute should be fine. If we do have a preference at all, we might want to start with the best cane users.

Day 1: Veering trials only. Each subject should complete as many trials as possible during the 45 minute period.
Wednesday

Time Subject Veering
8:45 I00n
9:25 I00n
10:05 I00n
10:45 I00n
11:20 I00n
12:00 LUNCH
12:30 I00n
1:10 I00n
1:50 I00n

Day 2: Veering only. Each subject should complete as many trials as possible during the 45 minute period.
Thursday

Time Subject Veering
8:45 I00n
9:25 I00n
10:05 I00n
10:45 I00n
11:20 I00n
12:00 LUNCH
12:30 I00n
1:10 I00n
1:50 I00n

Day 3: Cane coverage. Each subject should complete as many trials as possible during the 45 minute period.
Friday

Time Subject Veering
8:45 I00n
9:25 I00n
10:05 I00n
10:45 I00n
11:20 I00n
12:00 LUNCH
12:30 I00n
1:10 I00n
1:50 I00n

Below is the draft of the detailed procedure suggested by Steve to be used at NYISE. I completed the veering procedure first -- and pending feedback, I will create the others. Your comments are necessary.

Procedure for initial trials: veering

1. Staff will check that the pre-trial survey has been completed.

2. The staff will measure the widest area of the body, select the cane length, and input the values in the program interface.

3. For the initial trials for veering, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff. Note: the staff will emphasize that we are testing the cane, not the subject.

4. Subject will be shown the waist-attaching wii remote. The remote band will be put around the subject’s waist and adjusted to fit comfortably. The remote angle will adjusted to point toward the infrared light strip. If subject is wearing loose clothing, the staff will use clips to secure the clothing does not interfere with the wii remote.

5. The staff will check the indicator for the remote vertical alignment, and make necessary adjustments.

6. Subject will be shown the wiiCane, the imbedded remote, and the shaft. Subject will demonstrate how they move the cane left and right; any wrist rolling behavior will be noted and the subject will be shown how to move the cane without rolling their wrist. (This action is taken now in anticipation of the coverage trials.)

7. Subject (with or without vision) will be shown the customized occluding goggles. The subject will put the goggles on and assisted to adjust them to fit comfortably.

8. Subject will put on the wireless headphones and assisted to adjust them to fit comfortably.

9. Subject will walk, guided, the 30 foot-long straight-line course several times while the procedure is reviewed again.

10. Subject will have the feedback system explained, including the meaning of the feedback terms and feedback chimes.

11. Subject will be asked if they are ready to try the course.

12. With ascent, the subject will walk the course several times, with staff behind them, as they listen and respond to the feedback. The staff will advise the subject of errors in responses to feedback. The subject will be encouraged to continue walking forward when he/she hears feedback. The subject will be prompted to move to the left or right in response to feedback. This will be repeated until the subject appears confident and can travel the course alone.

13. Data recording will begin. The subject will walk the course. At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

14. Subject will walk the course repeatedly until the 40 minutes expire.

15. Subject will be told the time of his/her next appointment.

Procedure - draft

2009-12-06T17:24:00.002-05:00

Below is the draft of the detailed procedure suggested by Steve to be used at NYISE. I completed the veering procedure first -- and pending feedback, I will create the others. Your comments are necessary.

Procedure for initial trials: veering

1. Staff will check that the pre-trial survey has been completed

2. For the initial trials for veering, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff.

3. Subject will be shown the waist-attaching wii remote. The remote band will be put around the subject’s waist and adjusted to fit comfortably. The remote angle will adjusted to point toward the infrared light strip.

4. Subject will be shown the wiiCane, the imbedded remote, and the shaft. Subject will demonstrate how they move the cane left and right; any wrist rolling behavior will be noted and the subject will be shown how to move the cane with rolling their wrist. (This action is taken now in anticipation of the coverage trials.)

5. Subject (with or without vision) will be shown the customized occluding goggles. The subject will put the goggles on and assisted to adjust them to fit comfortably.

6. Subject will put on the wireless headphones and assisted to adjust them to fit comfortably.

7. Subject will walk, guided, the 30 foot-long straight-line course several times while the procedure is reviewed again.

8. Subject will have the feedback system explained, and the meaning of the feedback terms and chimes.

9. Subject will be asked if they are ready to try the course.

10. With ascent, the subject will walk the course several times, with staff behind them, as they listen and respond to the feedback. The staff will advise the subject of errors in responses to feedback. The subject will be encouraged to continue walking forward when he/she hears feedback. The subject will be prompted to move to the left or right in response to feedback. This will be repeated until the subject appears confident and can travel the course alone.

11. Data recording will begin. The subject will walk the course. At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

12. Subject will walk the course repeatedly until the 45 minutes expire.

13. Subject will be told the time of his/her next appointment.

behold: wiiCane 2009

2009-12-05T21:57:00.052-05:00

Introduction

Here is the final WiiCane design that will be used in the upcoming user trials. The first trials will occur on Wednesday, December 9 at New York Institute for Special Ed in the Bronx and will continue through the rest of December and into January at a total of four sites. This post is to describe the physical wiiCane apparatus as of December 6, including:

The wiiCane itself
The belt-mount unit
The light track and associated fittings

This post will provide a generic description of each component of the physical system for project documentation. This will also serve as a first public disclosure of the wiiCane system prior to filing for a provisional patent, followed by a full patent filing should test findings indicate that a commercial product based on this technology can achieves the aims set out in the project proposal. To provide a complete picture of our activities and results, other project participants will produce posts over the next couple of months describing our software and hardware approaches and components, as well as discussions about how the apparatus can be used for therapeutic, educational and entertainment purposes. A final project report will provide a public disclosure of raw data and evaluation results from the upcoming user tests.

A note about intellectual property: this project has been funded by a grant from the United States Department of Education's Steppingstones of Technology program. See the sidebar on the right side of the screen for additional information. Touch Graphics' stated goal in pursuing this project is to bring one or more products to market based on technology developed here. Since the company is subject to competitive pressures, we have chosen to obscure some aspects of our design and omit certain details in descriptions of structures and mechanisms, fittings, electronics, and software. But we hope to contribute to the scientific discourse in fields related to orientation and mobility, motion capture and analysis, human computer interaction, and product design and so are committed to publishing our findings in ways that other researchers find helpful and informative. In particular, this work seeks to validate and extend work of David Guth and Robert Wall Emerson at Western Michican University, and Bruce Blasch at the Atlanta VA (see future posts for elaboration). This disclaimer will hopefully clarify our reasoning for leaving some things out of the following discussion.

WiiCane
The central physical component of the system is the modified cane itself. As of now, the cane is identical to an ordinary mobility cane, except that it includes a fixture that permits the wii device to be held and positioned as needed for our purposes. The wiiCane includes a telescoping section so that its length can be adjusted for the individual user. This way, a single wiiCane can be used by many students. Maximum length is 59" and minimum length is 42". The wii device is held in a two-part gimbaled lightweight cradle that allows the device to rotate 360 degrees and then be fixed in position, so the wii device's camera can be pointed straight up or straight down. The handle of the wiiCane is flat on one side so that the the user can easily determine the correct orientation to hold it. When the cane is held in the right hand, the flattened part of the handle points to the right, and the wii device points straight up so that its camera can "see" the light strip above. When switching to a left-handed user, the wii device must be rotated 180 degrees in the cradle. the wiiCane is quite heavy compared to an ordinary mobility cane, but it is well-balanced and easy to hold. The cane can accommodate any of the standard tips, including marshmallow and roller, by unscrewing the standard pencil tip and replacing it with a new one. The wii device can be easily removed from the cradle for replacing the batteries.

Belt unit

Because we need to track both user and cane position during use, we added a second wii device mounted on the user's body. To hold it in place at the right angle, we created a simple belt that goes around the user's waist. The wii device is held in a cradle against the user's back, so that the camera points up at the light strip. Because its important that the camera in the belt-mounted wii device maintain continuous awareness of as many lights on the light strip as possible, we added adjustments that allow us to control the unit's tilt back and away from the user's body and potentially billowing clothing. We also may need to add clips to their hair or other garments that could block the camera's view. The belt is a standard item made of nylon webbing with a quick-release plastic buckle. The cradle can also be attached to the user's own belt.

The light track

The third component of the wiiCane system is the light track. This is a 32' long modular construction that acts as an upside down landing strip: as users walk along under the light track, cameras inside the cane- and belt-mounted wii devices track LED light sources spaced regularly along its length and relay their positions via bluetooth radio to a stationary computer. A Java program evaluates the raw data returned from the cameras and accelerometers on the wii devices, determines the most efficacious feedback and provides (hopefully) helpful corrective messages via wireless headphones and/or vibrating the cane's grip.

The project includes design of a suspension systems for the light track. Depending on existing conditions at the training site, the light track can be mounted in a variety of ways. Using custom made plastic clips, the track sections can be screwed to the ceiling, to an overhead pipe or other linear element, or to a free standing cable structure supported on lightweight aluminum tension poles that we provide. The light track includes IR LED's at 6" intervals along its length, for a total of 64 lights. The light track is made up of 8 4-foot long sections that are held up to the suspension cable, ceiling or exposed pipe using plastic clips.A single CAT 5 cable emerges from the end of the track, and it connects to a serial controller that switches the lights on and off under computer control.

Plastic clips for mounting the lighting track

scheduling for NYISE - draft

2009-12-03T16:45:00.000-05:00

Schedule for testing NYISE
December 8 (Tuesday) through December 11 (Friday)
Daily 8:30 PM

Tuesday
Complete setup of all equipment
Testing of equipment
Run through blindfolded TGI staff on course
O&M specialist should prepare kids, perhaps show them the route and explain what will happen

The first trial for each subject, the subject will be have the experiment explained
The testers may use whatever appropriate language is necessary, and take as much time as they feel is reasonable for the subject to understand and be comfortable.
Subjects will be asked if they are comfortable with the headphones and the goggles.
Verbal and non-verbal response will indicate whether the subject should continue. Subjects will then be guided once down the course, shown the end of the course and the safety obstacle.
Subjects will then be walked/guided back to the start of the course.
Subjects will always complete trials for veering before trials for coverage.

I am suggesting we test based on time, not number of trials. There is no way I can think of to stay on a schedule when we might see great variability with each child. Doing as many trials as possible during a prescribed period makes sense. Annette, to you see a study design problem with this?

The tables do not appear below correctly on the blog.

Day 1: Veering trials only. Each subject should complete as many trials as possible during the 45 minute period.

Wednesday

Time Subject Veering
9:00 I001
9:45 I002
10:30 I003
11:15 I004
12:00 Lunch
12:45 I005
1:00 I006
1:45 I007
2:30 I008

Day 2: Veering trials for 15 minutes, followed by cane coverage trials

Thursday

Time Subject Veering Coverage Total
9:00 I008
9:45 I007
10:30 I006
11:15 I005
12:00 Lunch
12:45 I004
1:00 I003
1:45 I002
2:30 I001

Day 3: Veering trials for 20 minutes, followed by cane coverage trials for 20 minutes

Friday

Time Subject Veering Coverage Total
9:00 I001
9:45 I002
10:30 I003
11:15 I004
12:00 Lunch
12:45 I005
1:00 I006
1:45 I007
2:30 I008

IRB Approval

2009-12-03T13:20:00.003-05:00

As the project evaluator, I'm very happy to announce that the WiiCane project received official IRB (Institutional Review Board) approval on November 24. This means that we are able to move forward to collect data on adult participants and on children age 6 and above. IRB approval is necessary for all research using human participants, to certify that the research meets ethical standards for human subjects testing. Our data collection will begin on December 8.

After we have collected data on children age 6 and above, we will refine our procedure for testing children ages 3-5 and submit our preliminary findings to the IRB for approval to test in that age group.

F u n c t i o n a l i t y

2009-11-10T18:31:00.002-05:00

I had a chance today to meet with Zach at the TG office. I was able to expereince the device provide feedback on veering (only). The system worked, if very imperfectly. The belt mounted Wii could reliably detect when I veered and: make a statement for which way to move, and ring a bell when I returned to the middle of the course.

There was a bias to one side, with the feedback kicking in at a wider point from midline, but Zach said that's an easy fix. The feedback wording is still not coming from the user interface screen (so it's always one word, and sometimes wrong), and we could not narrow the tolerance for veering with an input field on the screen, but I know these features are coming. I seriously suggest that the bell sound (which is just like the classic bell at a hotel desk) be replaced with something friendly and encouraging.

It was remarkable to just have the basic thing, in all of its aspects, work.

Gene's excellent adventure in the Bronx

2009-11-09T13:52:00.000-05:00

I was warmly welcomed at the Institute in the Bronx this morning. First I met with Tina (the school computer teacher and TG liaison) and eventually with 4 of the five O&M specialists. It was nice to see Jim Waters again (he’s been around the profession for decades and has done some work with deafblind travelers), and the other younger COMS.

They were all very interested in the project. I reviewed a bit of the history and theory for the project, reviewed the physical space, and then went out on the campus to see the prospective subjects travel with their canes.

Great news – we have 16 very good potential subjects. Tina will contact Ellen regarding how to proceed: contact all 16 households, or start with eight and see how it goes. They anticipate a very high positive response rate (because of the stipend).

There are some of the issues that you will see followed up upon:

- they need instructions on how the prepare the room, where the lights will be affixed, and what needs to be removed from the room

- many of the COMS want to be present when their students participate.

- I explained that during the project we are not officially asking for the specialists feedback on the device (the IRB would consider them subject which would complicate things) but that we will eventually and outside the experiment protocol love to hear their ideas.

These kids all seems to have the capacity to tap a cane and maintain forward movement, so I think we will be fine. They are all at least 6 and as old as fifth grade takes them.

It was so nice to sit around with these colleagues and talk about the project. It was obvious that they were all excited.

Visit to NYISE

2009-11-04T23:00:00.002-05:00

Today Ellen, Grace and I went to the New York Institute for Special Education to check out sites for the upcoming user trials there. We considered two spaces: along one wall of the big gym, and in the basement rec room of one of the dorms. These are pictured above The basement room is a fairly large rectangular space with small windows high up along one wall. The ceiling height is 9'. The gym is a very large space with 30' ceilings, except under the running track that runs all the way around the gym, where the ceiling is 11'. We chose the rec room, because it is not used much, and there's not much natural light, which we are worried could interfere with the camera's ability to see the IR LEDs on the track clearly.

We also decided that it would be easier in this case to mount the track directly to the ceiling instead of using poles. There is an existing pipe to which we can mount the track, using special clips, but the same clips could also be used to mount a track to a sheetrock or acoustical tile ceiling. I am inserting a sketch of what I have in mind here. Grace will 3D print some so that we can try it. A clip will be glued to the end of each of 8 four-foot long sections of track. That clip will be screwed to the ceiling or tied to a pipe. Then, the next piece will be hung up in the same way, until all 8 sections are hung up. I think that this would work for almost any room or hallway, as long as the space is long and wide enough. For tying the clips to a pipe or other existing ceiling structure, we are putting slots through which you can place plastic wire ties for easy temporary installations. A wire will come out of one end of the track and run to the computer, which could be mounted to the wall or on a table.

Feedback design implementation

2009-10-31T14:57:00.000-04:00

As the date draws closer for our first practical demonstration of the WiiCane system, I am building computer application that processes the raw data and events recognized by Zach's driver. As of now, I have created a single screen that I think includes data entry fields, indicators and graphics we will need for this first iteration. It should be noted that this design is preliminary and will undergo changes as we proceed.

In the image above, I divided the screen into four zones:

Raw Data, where we see numeric values returned from both WiiMote units (cane mounted and body mounted).
Position, where we see a plan view of the WiiCane course. An avatar representing the user moves along the course to indicate progress, and a stick moves back and forth to show the cane's position at each tap event.
Indicators and Settings, which includes large color fields meant to be visible from the far end of the course. The indicators change color in synchronization with tap and veering events as perceived by the WiiCane system. Text boxes in this area allow the administrator to set threshold values for these items.
Feedback. This section includes boxes where the administrator can create or edit text for each utterance that the system will speak into the user's wireless headphones as he walks along the course.

In Gene's post of October 12, he discusses the variety of feedbacks that will be needed. I believe that I have captured his recommendations here, but this may require some discussion from the group, since the way that feedback is delivered is the key factor in affecting student learning and improvement. Keep in mind, we are only dealing with one case here, that is where feedback is being delivered in speech, not sound effects or vibration, and we are only, for now, looking at veering and coverage.

The six feedbacks for now are:

Correct veering right
Correct veering left
Increase coverage right
Increase coverage left
Decrease coverage right
Decrease coverage left

One question that still needs to be addressed is how often feedback should be issued for a particular condition. Gene mentioned three times for a particular behavior before issuing feedback. This needs to be considered carefully. Does the counter reset if a different error is observed? I think that the only way to really know how to do this is by trying it out, so everyone should be prepared for the system to require a lot of revision before it becomes useful.

Testing Protocol as of 25 October 2009

2009-10-25T13:30:00.002-04:00

Version 2, 10/21/09

WiiCane: An Accelerometer-based Tool for Mobility Training

Touch Graphics, Inc.

330 West 38^th Street Suite 900

New York, New York 10018

Phone (212) 375-6341, Fax (646) 452-4211

Testing Protocol

Creation of the Device

The WiiCane project proposes to adapt the Wii gaming technology to develop a feedback device for long-cane mobility training for blind and deaf-blind children and adults. The Nintendo Wii gaming console includes a small, lightweight, inexpensive WiiRemote component with accelerometer-based motion tracking. The current project proposes to use this device attached to the shaft of a cane during mobility training. The device will provide auditory or vibratory feedback to the cane user for immediate correction of movement and will also provide computerized tracking information for instructors and researchers. The project will focus on the device’s ability to monitor two specific cane-use behaviors: maintenance of the user’s direction within a straight line path; and cane arc width adequate to insure footfall preview (for hazard protection). The objective is to create a device that is easy and fun for the user and that provides corrective feedback to improve cane mobility instruction and learning. A diagram of the device is attached to Form 4.13B (Device) showing how the WiiRemote component is mounted below the handle of a cane, enabling it to pivot so that the camera can be directed either in front of the user or under the cane to monitor motion and direction.

Verification Study

The first stage of product testing was a verification experiment. This experiment has taken place at Western Michigan University at Kalamazoo. This location was chosen in order to use the consultation services of Dr. Robert Wall Emerson of the University’s world-renowned Department of Blindness and Low Vision Studies. The verification study has helped the development team confirm the accuracy of their proposed motion capture apparatus by comparing the data it collects with the known accuracy of the optical tools developed by Dr. Emerson. The study also provided recommendations for improving the device’s measurement algorithms. Since Dr. Emerson’s optical tracking equipment is too bulky and delicate to ship, it was preferable to conduct the verification study at his home site. The subjects were ten sighted college students recruited through advertisements and classroom announcements.

Since WMU maintains its own IRB, this experiment was submitted separately for review at their local IRB. Since the subjects were all over 18 years old, and since they wore blindfolds to simulate blindness, they were not considered a protected population, and we expected the IRB submission to qualify for expedited or exempt status. This experiment, and its participants, are completely separate from the experiment in the current application to IRC; it is acknowledged here because of its common funding source and because it will result in information for improvement of the device to be used in the current study.

Recruitment and Consent Process

Following the verification study, the Testing Coordinator will contact staff at each of four test sites to identify potential participants for usability testing of the device, scheduled for November 2009. Test Sites and age ranges for recruitment are as follows; 8 participants per site, for a total of 32 participants, are anticipated.

Jewish Guild for the Blind, participant ages 2-4.
New York Institute for Special Education, participant ages 5-12.
Overbrook School for the Blind, participant ages 13-22.
Helen Keller National Center, participant ages 17-adult.

Testing will start with children aged six and older. We will add younger children to the testing, at the sites indicated, only once success with older ones has been shown.

Participants must be visually impaired, and must use mobility canes for independent travel. Participants at the Helen Keller National Center will be deaf-blind. Participants at all sites should have no additional disabilities, including motor, cognitive or behavioral problems. An appropriate staff person at each institution will be identified as a liaison to participants and will be responsible for collection and transmission of data as needed. Once the liaison at each institution has identified qualified candidates, the liaison will contact adults, or children’s parent or guardian, to discuss the testing protocols, requirements for participating, payment for participation, and procedures put in place for the protection of human subjects in research.

The liaison will contact potential participants or their parents by a letter either given to the adult participant or sent home with the child as is often done with parental permission slips. If this is not feasible in an individual case, the letter script may be communicated by telephone or by e-mail. A copy of the letter/telephone/e-mail script is provided as a separate document. Students asked to participate will be given a consent form to give to their parent to sign and to bring back to the liaison. Consent forms are provided as separate documents. Students and parents will have time to consider the study and to ask questions before they have to make a decision. As noted in the consent form, students or parents with questions about the study may contact the testing coordinator or the project evaluator. Their names, e-mail addresses and home telephone numbers are given, and they will receive their messages every day. The consent form also provides the e-mail address and phone number of IRC in case students or parents have questions about their rights as research subjects. Willing participants or their parent will also receive an electronic version of the Pre-Participation Questionnaire. This document assesses the participant’s degree of visual impairment, educational background and cane experience and is included in the IRC submission. It will be administered and collected by the liaison.

Usability Testing

Usability testing will take place at the named sites over a four-week period, one week per site, beginning with participants aged six or older. Younger children will be tested only after success with older children has been established. The Co-PI for Pedagogy, who is a qualified orientation and mobility specialist on the project staff, and a testing assistant will conduct one to two sessions lasting around one hour with each participant. This time includes equipment setup, interview, preparation, trials with each participant, and a brief post-trial interview. A staff member from the test site host organization will be present during all test sessions but will not administer the testing.

Participants or their parent will be told both in the consent form and at the time of testing that we want them to try out the device to see if it is useful and to solicit their reactions for the purpose of improving the product, not to evaluate them personally. In each session, the participant will first try out the WiiCane device, making sure that he or she understands the meanings of various feedbacks (either audible or vibratory, or, in the case of the deaf-blind participants, only vibratory). Next, the participant will be asked to walk along a 30-foot-long course, using the WiiCane as a mobility aid. The computer will monitor their movements and will generate a complete record of their performance in each session. Benign auditory or vibratory signal feedback will be sent from the computer to the participant through the device or through wireless headphones, and the system will log for future analysis the participant’s ability to use the feedback to correct behaviors that fall outside pre-set allowable ranges. In order to insure uniformity of vision across subjects, partially-sighted participants will be asked to wear a customized occluded goggle so that they cannot use vision to alter their cane use. This will be explained in advance in the consent form. Participants who consent but then express anxiety at wearing the goggle will be allowed to discontinue their participation with full payment. At the end of testing, participants will be asked a series of questions designed to elicit their responses to the system.

Payment to participants ($150 plus travel reimbursement) will be made in cash on completing the session, and they will sign a receipt. It will be given directly to the adult participant or to an accompanying parent. Payment will be made to all participants who attend the session, even if they choose not to complete the study. Payment will not be made to students who do not attend the session.

Data analysis will focus on the usability and effectiveness of the remote feedback to improve participants’ cane use for the two behavioral criteria (maintenance of a straight line path and use of a proper-sized cane arc). All records will be maintained by Dr. Annette Gourgey, project evaluator, for a period of five years. All participants will be identified by an ID number and Dr. Gourgey will be the only person in possession of the master list of names and numbers. Only Mr. Landau and Drs. Bourquin and Gourgey will have access to the records.

The long-term goal is consistent with Touch Graphics, Inc.’s practice of conducting high quality research and converting the findings to practical and needed assistive and accessible educational materials. Thus, the Company will seek to make necessary changes to the WiiCane system as suggested by the research outcomes, to be followed by a product roll-out within one year of the end of the project. We will design and manufacture a custom cane that includes a pivoting fixture for mounting the WiiRemote device. The software will run on any PC that has Bluetooth capability (normally included with most recent laptops, and easy to add to desktop models and older laptops with an inexpensive expansion card). With the addition of Owner’s Manuals and packaging and promotional materials, we hope to start selling up to 100 systems per year, at a total cost to an organizational customer of $1,200, including setup and training. Realistically, students could begin training on the WiiCane as early as late 2010.

IRB language

2009-10-25T11:36:00.004-04:00

I agree with Annette that we should avoid the word "injury" in the informed consent documents. I would have preferred to leave this out altogether, since we have never included a statement like this in the past. But in thinking through potential scenarios, however, I imagine that someone could fall down and get injured while using the system, especially since we are occluding subjects' hearing and vision. Apart from IRB considerations, what are the real risks here? What if someone does crash into a wall or trip over something and fall? Gene, given your experience with this kind of instruction, and based on what you imagine will happen in the testing, is there a real risk of people getting injured, and is this risk greater than what would be encountered in standard O&M training? What special training or precautions should we be taking? While I don't want to raise any red flags that could interfere with our IRB certification, I also don't want to do anything that could be construed later as reflecting a failure to adequately protect our subjects or users of any commercial products that come out of this research.

One precaution that I think we should consider is to allow the test administrator to speak directly to the subject via the wireless headphones. While this will not work for d/b subjects, it could be a good way to intervene in the case that a subject is veering widely and is heading for a crash. The administrator could wear a headset with a microphone, and then we would mix his voice prompts with the system-produced feedback in the computer. Ultimately, I am hoping that the commercial WiiCane project will not include any occlusion of remaining vision or hearing. I understand why it is needed from the experimental design perspective, but I am not sure that this is a requirement from the pedagogical perspective. While we want to add to the literature by developing experiments that are comparable to those carried out by other researchers (e.g., Guth), our ultimate goal is to produce a practical training apparatus for teaching cane travel techniques, so we have to satisfy both of those cases.

I would like to go forward and resubmit the two informed consent documents in their latest form. Let's see what the IRB does; if they are comfortable with this, then we can proceed. But I want to go on record as saying that we will need to be highly vigilant at the test sites. If we are occluding hearing and remaining vision in children and then asking them to walk around, we have to be absolutely sure that we are not placing them in a dangerous situation, regardless of who actually has to pay in the event that one of them gets injured. Our goal is to test a practical teaching system, and we should be less concerned about the absolute purity of our experimental design. I think that this is a very important issue, and while we are addressing it very late in the day, I believe that this is a subject that requires further consideration and discussion before any testing can take place.

I would be grateful for any and all comments on this subject.

optimum criteria for subject selection for testing

2009-10-20T19:36:00.002-04:00

Steve,

I understand you will be visiting some of the sites and meeting with key site personnel.

I suggest you share the criteria below with the administrators and O&M folks so we will have the best chance of success. I hope others respond to this with other ideas or corrections.

Criteria:
- the subject should have the capacity to tap a cane back and forth as they move forward
- the subject should understands the concepts of left and right; or, the subject can move left or right depending on which ear he/she hears a sound.

This is a short list but I think if you present this to the sites they will know which children are appropriate and which are not. We might also ask that teachers work on these skills with the children in anticipation of the testing.I am guessing we can assume we will not get 8 kids at each site that can do these things, but the more the better.

I suspect others might be able to add qualitatively and quantitatively to this post.

posts from recent emails

2009-10-19T05:02:00.002-04:00

Steve requested I place some of our emails here on the blog. Here they are.

- - -

Sorry I've been absent from this discussion all day as I have been out of town. I have thought more about this and I don't want to underestimate the abilities of the young children in this project. I don't know these students so maybe they are not as involved as the students I am accustomed to working with over the years. And are we talking 3, 4, or 5 year olds? There is a BIG difference in even 6 months in regards to cognitive and conceptual development.

I can agree with trying using the feedback in the left ear to see if the child will move more toward this side etc. Young children are typically adept at moving toward sound sources. So this might work for a child who doesn't really have a deep understanding of laterality. It might result in more of a move toward the direction of the sound in my ear type behavior, instead of a conceptual understanding of straight line travel. But if using the Wii feedback system illicits the desired behavior, this is a good thing.

In response to Gene's comment - Here's my big problem with this and maybe Bonnie or you could explain so my simple mind can get it . . . what's the difference between learning the word RIGHT and what it means, from learning a particular sound and what it means. They seem like very similar cognitive tasks. Both use the auditory pathways, both require the child to understand the difference between lef/right, and both require the child to pair a label (sound or word) with the proper concept and react to it. So what's the difference and why would one be better/easier than the other? In my gut I would suspect the word would be easier. But what the heck do I know!?

I understand this and am I don't disagree. What I am questioning is the conceptual understanding of left/right in children 4 and under.

I need to seek out literature on children learning the concepts of laterality. It has been my experience that blind children, even as young as age 4, can respond correctly to "touch your right ear", "touch your left foot", etc but they don't truly understand more advanced concepts such as "move your cane tip to the right", "turn your body left", until months or a year after they understand they have two of some body parts, and a left and a right side. You can see this development when you administer the Cratty & Sams Body Image for Blind Children checklist. Maybe we should ask the O&M specialist to do this with each child before data collection. It only takes about 20 minutes per child. This starts with body parts, then body planes, laterality, manipulating objects in relation to body to demonstrate spatial understanding of laterality, movements to left/rigth and directionality (left/right on others). I'll dig this instrument up and attach a file tomorrow.

Finally, what percentage of the subjects are children age 5 and under? If what we develop works for school age children and adults, then the project has accomplished some of it goals, right?

Bonnie

- - -

Hi everyone! I just saw a video by Dr. Bil Hawkins who used a WHISTLE to give feedback for veering while crossing the street. Now, I would never, ever suggest that we distract students while crossing streets by sending them coded messages about their veering (perfect scenario for the need for isolated veering training OFF THE STREET with the wiicane!), but this auditory feedback might be something that could work with Wiicane. We can ask Bil for details, but from the video it seemed that if the student veered a little to the left, he got a long, slow descending whistle. If he veered sharply to the right, he got a quick rising whistle. Bonnie, do you think kids who can't remember L-R could learn that if the "left veer" (descending) whistle was played in the left ear it meant they had veered to the left, and if the "right veer" (rising) whistle was played in the right ear, they had veeered to the right? P.S. I tried to log into the blog to publish my comments, it said it was emailing me my new password but I've gotten no message. So I'll continue to email comments, hope that's okay. -- Dona

project status and ongoing discussions

2009-10-18T20:09:00.001-04:00

First, I need to apologize for not participating in the ongoing discussions that are occurring on email regarding which feedback types are appropriate for young children and also which style of cane use our system will train students for. As I mentioned, last week for me was consumed with preparing for and then attending a great conference at the Metropolitan Museum. Now that's over and my full attention is shifting onto WiiCane for the next two weeks as we get ready to do our first field tests.

I need to reiterate that our group discussions must take place on this blog, so that we end up with a coherent record of our conversations, and so that all project participants can easily keep abreast of our developments and make contributions as needed. I understand that there were some oversights in distributing invitiations to blog participants, and I also know that it can seem easier or more satisfying to just fire off an email to one or two people, but it is important to the overall project that we use the blog. I expect traffic to pick up significantly now, so it's really a very good idea to subscribe to the email digest that will come to you on every day when there is some blog activity. If you have not signed up for that yet, please do so by filling in your email address in the text box in the upper right corner of the blog and then clicking "subscribe". If you have not received an invitation to participate in the blog, please let me or Gene know, and we will take care of it.

Regarding the question of how best to provide corrective messages to young children undergoing training with WiiCane: I think that this is a question that will be clarified quickly when we start running kids. My guess is that directional audio that plays a tone in the ear on the side toward which the correction should be made will be more effective than saying "move right" or "move left". But, it's easy to create a feedback system that allows for either option to be set, so we should have no difficulty in determining which works best, or, we provide both options and allow the test administrator to decide.

On the question of two-point tap vs. constant contact: my understanding is that most users these days are trained in constant contact, and so we need to ensure that our system reflects this preference. We should not develop a system that is only capable of providing instruction in an obsolete or unpopular method for cane use. But, the proposal was written specifically to address two-point touch, and so we need to do that first. Once we get that working, we can augment the system so that it can also be used for training constant contact also. From a technical perspective, it is much simpler for us to first develop a tap-based system for motion capture. We know that tap events are very easy for the Wii's accelerometers to recognize. So, our current conception calls for taking a "snapshot" of the cane's position at the moment when a single tap occurs. If we define cane arc as the angular or lateral diplacement that occurs between tap events, our work is greatly simplified. Without taps, we will have to develop a much more complicated approach to arc width calculation, in which we would have to recognize the moment where the direction of cane movement reverses. That's not impossible to do, but it is probably a lot harder. My thinking is that we should master 2-point touch first, then introduce constant contact later. The next couple of months are going to include a lot of trial and error as we develop our technology and our pedagogical approach. Nothing is fixed in stone right now, and we need to remain very open to making changes and enhancements suggested by our domain experts and our users. but, we should not abandon plans that were approved by the grant review panel without receiving explicit permission to do so. I also think that its sensible to start with something that we know how to do, get that working, and then move on to them more complex stuff.

On Tuesday, Zach and I will get together to connect his Java WiiCane drivers to the test administration program that I am working on. We should have something nice to show on Wednesday, when I am assuming that we will meet at the office to discuss our plans for responding to the IRB's questions about the test protocols we have requested them to certify.
sl

Progress report on application programming

2009-10-14T11:14:00.001-04:00

Zach and I have made progress on the application that will run the WiiCane system. This application consists of two components:

The device driver, which Zach is programming in Java. This is a program that will run on the PC mounted to the pole at the end of the course. The driver establishes and manages the wireless connection between two Wii Remote units: one mounted on the cane and the other mounted on the user's body. The driver maintains information on:

Linear distance between the starting point and current location.
Body compass bearing, expressed as a positive or negative angle. Pointing straight on the course returns a bearing of zero, and deviation to the left or right returns a negative or positive value, respectively.
Amount of veering, measured linearly from the imaginary line connecting the start and finish points.
Cane taps events. A "tap threshold" is used to distinguish intentional taps from normal accelerometer fluctuations. Three taps in rapid sequence is a special signal that can be used to indicate that the user is about to start running the course.
Wrist roll. As Gene mentioned, the system will not work if the user rotates his or her wrist such that the Wii remote mounted on the cane is not pointing up at the lights. We can assume that this has happened if the body-mounted Wii places the user in proper position on the course, but the cane-mounted unit reports that no lights are seen.
Cane arc, expressed as a positive or negative angle of the cane in respect to the body at the moment when a tap occurs.

The course set up and test administration program, carried out in Adobe Director, which I will be creating. We will use the Moka Xtra to act as a bridge between Zach's Java-based Wii Cane driver and Director. We have tested this, and it works well, and is very fast, which means that we will be able to respond to user actions without perceptible latency, which would result in system sluggishness. The Director application that we will use for testing in November will include the following features:

A user log in screen so that subject parameters (cane height, body width, age) can be set prior to testing, and so that performance statistics can be saved to the computer's hard drive for later analysis.
A course parameter screen that permits an administrator to customize feedback conditions.
An animated plan view of the course with an avatar representing the test subject indicating instantaneous body and cane position and bearing during the test.
A training administration screen that shows elapsed time and a visual display of feedback prompts currently playing in the subject's headphones.

These programs are being developed specifically for the upcoming tests. The actual commercial version of the WiiCane product will include other features and a more polished presentation.

wii cane feedback logic and testing procedures - draft

2009-10-12T21:08:00.003-04:00

WiiCane procedures for initial testing

The following drafts are for everyone’s consideration and critique.

This weekend I visited Touch Graphics and saw the working model of the WiiCane and course. It is quite amazing. On the HD digital display at the end of the 32 foot course, you can see the cane movements precisely as the cane/subject moves. The equipment and remote software are working and it’s impressive. The user interface and the software to create the feedback loops are the next steps.

I will use the term “detectable” to mean that the WiiRemote can detect the infrared lights suspended above the course.

I will use the term “expected cane tip-placement” to mean that the cane tip lands within a tolerance of 2 inches on the left or right on the surface in front of the subject that is equal to the widest part of the subject’s body’s lateral plane.

These comments do not include procedures for deafblind subjects.

General considerations and concerns:
· The subject may not role their wrist as they move the cane; this will cause the cane to be undetectable. Subjects’ rolling behavior will need to be corrected with intervention.
· The subjects will need to stay with the detectable lateral area during testing. If they move out of this area, the tester will intervene to move then back into the detectable area.
· All subject will align for each trial on the course by squaring-off , placing the back of their footwear into cutouts of a wooden board properly located at the start of the course
· All subjects will wear customized goggles to occlude vision to the point of LP.
· All subjects will wear wireless headphones that are expected to deliver the feedback loop and prevent echolocation.

Logic for creating the feedback loops

VEERING
· When the subjects’ midline exceeds 16 inches from the intended straight-line of travel, the subject will hear a message
· The feedback loop will say the following is high quality electronic speech: “move to the left [or right]”
· The feedback message will repeat each time the subject exceeds16 inches from the intended path.

CANE COVERAGE
· The software will check the subject’s cane tip-placements after the first three lateral sweeps. Thereafter, the software will check the most recent three cane sweeps/tip-placement.
· Feedback will, of course, be issued only if the previous feedback statement has completed.
· The software will select a feedback statement based on the following logic:
- if the cane tip-placements for the three most recent sweeps result in the expected cane tip-placement, then the subject will hear: “Good”
- if the cane tip-placements for the three most recent sweeps result in cane tip-placement further from the subject’s midline than the expected cane tip-placement, then the subject will hear: “Too wide”
- if the cane tip-placements for the three most recent sweeps result in cane tip-placement nearer to the subject’s midline than the expected cane tip-placement, then the subject will hear: “Too narrow”
- if the cane tip-placements on the most recent three sweeps are asymmetrical (not too wide or too narrow on both sides), the greater lateral displacement from the expected cane tip-placement will determine the feedback and the subject will hear: “To wide [or narrow] on the right [or left]

Procedures for conducting trials

For the first trial for each subject, the subject will be have the experiment explained; the testers may use whatever appropriate language is necessary, and take as much time as they feel is reasonable for the subject to understand and be comfortable. Subjects will be asked if they are comfortable with the headphones and the goggles. Verbal and non-verbal response will indicate whether the subject should continue. They will then be walked/guided once down the course, shown the end of the course and the safety obstacle. They will then be walked/guided back to the start of the course. Subjects will always complete trials for veering before trials for coverage.

VEERING AND CANE COVERAGE

· The subject will align at the start of the course.
· The subject will be told to swing there cane and walk forward in a straight line
· The subject will walk unimpeded unless they become undetectable. If the subject becomes undetectable, the tester may physically prompt or manipulate him/her back onto the course from behind the subject.
· When the subject ends the course, they will be walked/guided to the beginning of the course.
· Each subject will complete 10 trials on any particular day, on up to 5 separate days. This number will be doubled at one test site. This procedure will be repeated for all trials for each behavior, veering and coverage.

Course construction complete

2009-10-08T09:23:00.002-04:00

This picture shows the WiiCane experimental set up at Touch Graphics' office. The apparatus consists of two poles at either end of a 32' course. A pair of overhead cables supports a plastic strip that houses wiring and electronics for 64 infrared LED's on 6" spacing. We added the second cable after observing 8" of sag at the mid-point of the single cable set up; the second cable takes the shape of a catenary (like in a suspension bridge), and hangers drop from the upper cable to the lower one that carries the light strip (like the roadway in a bridge). This ensure that the light strip is flat, and that should simplify algorithms for determining position and cane movement. The second cable also appears to stabilize the light strip and to reduce sway from air currents. Immobilizing the lights is key to ensuring our system's measurement accuracy.

In the detail picture, we see the two cables at the start of the course, a support pole, and the plastic channel that holds the electronics. The underside of the channel is printed with a metric distance scale that will help us to calibrate the algorithm for calculating linear position on the course. To do this, we will mount a laser pointer on the body-worn Wii device, and (hopefully) we will be able to visually observe the red dot from the pointer when it appears on the measurement scale. It's not clear whether this scale will be of use in an actual Wiicane product.

We have tried out the apparatus in our office with a Wii device mounted to a cane, with very promising results: as we walked along the course, the system was able to easily see the lights. We could watch the lights moving on the video screen we had mounted on the goal-end pole, and it seemed to work for cane swings that were quite wide. The only remaining problem, in my view, is that the user must not roll his or her wrist while running the course, because then the camera is not pointed up at the lights, and we lose position awareness. But, since our second (body-mounted) Wii device will always tell us where the traveler is in relation to the lights, we will know that wrist rotation has occurred if the person is under the lights but the cane-mounted Wii doesn't show any lights visible. Then we can provide spoken feedback to encourage the user to straighten his or her wrist, and we can let them know when this has been done successfully before resuming travel.

discussion about pole placement in the coures

2009-10-05T18:29:00.000-04:00

Bonnie responded to my last post via email, so I am pasting that here, along with responses from Dona, Gene and Zach. I want to make sure that our discussion is captured in the blog so that we end up with a nice record of our process. The question of sagging has been taken care of by adding a second cable above the one holding the lights. think suspension bridge. the upper cable takes the shape of the catenary, and the lower cable, the one holding the lights, is perfectly flat.

(In physics and geometry, the catenary is the theoretical shape a hanging chain or cable will assume when supported at its ends and acted on only by its own weight. Its surface of revolution, the catenoid, is a minimal surface and will be the shape of a soap film bounded by two circles. The curve is the graph of the hyperbolic cosine function, which has a U-like shape, similar in appearance to a parabola.) Click here for more information about the catenary.

Hi Steve,

I am just now reading the blogs about the overhead string of LED lights and the belt mounted Wii remote. Sorry it took me so long. As others have commented, getting rid of the path off the floor seems like a big improvement. I like the idea of the Wii remote being in the center of the user's back as it won't interfere with cane usage. It sounds like you will be able to get more accurate feedback about cane arc with the overhead system.

Have you been able to set up the cable and poles and try it out yet? I'm curious about the length of a path, considering that sagging may lead to inaccurate measurements. If you can't get anywhere near the 30 feet length, would you envision two strings of lights with 4 poles in order to cover the entire 30 foot path?

Gene & Dona, what do you think would be the impact of having a pole in the path of travel and at the end of the path? Do you think any of the subjects would be able to use echolocation to hear the poles? I'm wondering if the poles could be used as cues for alignment in a way that might interfere with the veering data, which was designed to have no audible markers, so to speak. I read that the pole will be padded, which is good because I imagine some of the younger subjects or those who are not covering their body width bumping into a pole. Do you bumping into a pole would result in the person altering their gait or cane width as a response to contacting a pole? This is the real world. When my students bump into something because they were not covering the entire width of their body (or not paying attention, or cane hand not extended), they often will respond by swinging their cane wider. Some students will slow down because they are now aware there is an obstacle and they want to avoid contacting it. A slower speed and having to move around the pole could affect the subject's straight line of travel.

I'm trying to view this pole idea as a positive learning experience from an O&M perspective. If it is the best way to collect the data (cheaper, appartus readily available, easy to ship/assemble) then there just needs to be consideration of how the pole(s) might alter the subject's cane usage. There are obstacles in the environment so maybe the experimental design is now revised to include an obstacle. If it works out that we only need a pole at the beginning and end of the path, they my points may not be relevant. I'm simply thinking aloud as to how a pole IN the path might alter the subject's cane usage and alignment.

Thanks.

Bonnie

-----------------------------------

From Dona:

Wow, Bonnie, interesting points! I hadn't thought about the possibility of the goal pole serving as an aid to aim for, or the effect of having an obstacle in the line of travel. perhaps two things could be done to mitigate that:

1. Surrounding the pole with spongy cushiony material should make the echolocation very difficult (even with clicking );

2. Having a line run perpendicular to line of travel a few feet short of the pole will warn the user that he's reached the end.

-- Dona

--------------------------------

From Gene

Bonnie,

Subjects' vision and hearing are occluded.

A pole in the center of the course would not work. I believe that the 32 foot course will have a single suspended string of lights. The poles are anchored at the top and bottom.

The single forward pole at the end of the course should not interfere at all. It will be padded, of course, and subjects may encounter it when the course is completed.

Does that all make sense.

Gene

---------------------------------------------

From Zach:

I was also skeptical about keeping the line taut but I've seen it at
Steven's office and it looks great.

The starting pole will have a wood block or similar on the pole or on
the floor to help subjects square off. Both poles are wrapped in foam
to prevent injury.

---------------------------------------------

From Bonnie:

Gene - I obviously forgot about the subject's hearing being occluded. That sure eliminates my concern about echolocation. Yes, your comments make sense. I had thought they couldn't string the lights for the entire distance, hence a pole in the middle of the path would be needed. So my concern no longer apply!

Zach - I am glad that the line can be kept taut for the entire distance. This eliminates the pole in the middle. I'm excited that Steve has set it up and it works.

Dona - I like the idea of a textural change in the floor a few steps before the end of the path to let folks know they have reached the end and the pole is near.

Thanks.

Bonnie

belt-mounted Wii remotes

2009-09-19T12:32:00.000-04:00

If it turns out that the overhead string of lights works well for tracking cane arc, I would also like to try a belt mounted unit to watch for veering. It may be that we don't need a camera at the end of the course if we could see at least two lights straight up. Here are some things to consider:

1. if the wii device is mounted on the person's body instead of on the cane, we should be able to determine placement on the course (y-axis), veering (up to the point that lights are no longer visible), rotation in the x-y plane, cane centeredness (by comparing values from both cane and body-mounted sensors.

2. We have to figure out where on the subject's body to attach the wii device. I am thinking that putting it in a belt holster on the midline-rear of the subject's body, because that seems to me to be less likely to be blocked by swinging arms or upper body. But certainly, we will have to test that and it might be necessary to make a special belt that can hold the device at a precise angle.

sl

Overhead course pole design

2009-09-18T23:07:00.002-04:00

Here's an sketch for an idea for making a wiiCane course with an overhead string of LED lights that will allow us to track position along a course and also to observe cane arc and veering. The best part about this is that it is mostly parts from IKEO, so it will be very inexpensive to set up a course. The picture shows a pole with a video screen and a plywood box mounted to it. At a point 8' above the floor, there's a fitting on the pole that is used to hold each of the three cables in tension. A cable comes out of the middle of the pole to supply electric and control to the lights.

new concept: putting the IR lights on a cable overhead

2009-09-18T22:36:00.000-04:00

After thinking through the problems associated with constructing a floor mat system for tracking position of a student during cane travel training with a wii remote, I want to consider a new idea that may resolve many problems. This approach calls for creating an overhead string of LED lights on a three or four wire cable stretched tightly between two poles. The poles would be floor-to-ceiling aluminum rods of the type I have here in the office to hold up the desks. These are telescoping poles from IKEA that extend up to 12'. I believe this could be easily installed in a wide variety of places. Even if they didn't have a full 30', they could use set up and use a shorter course.

Here are the pluses and minuses of the proposed system:

1. We can control the distance between the camera and the lights this way. In the floor mat arrangement, the shortest cane held the Wii device only about 18" or so above the floor, so the portion of the floor that the camera could see at once was very small. If we point the wii device straight up instead of straight down, and if we string the lights on a single cable directly above the course, I believe that we should be able to be far enough from the lights that we would mostly always able to see one light, and usually more than that. In the floor scenario, we were talking about making multiple rows of lights so that the cane would be seen even when the subject was not standing in the middle of the course. If we are far enough from the lights, we may only need one row. This has to be demonstrated experimentally.

2. The poles will be rigidly held in place, so, as long as they are padded, it won't pose too much of a hazard in the case of inevitable collisions. Because the poles are rigidly supported, we can draw the cables very tight, reducing the amount of sag. We will probably need some kind of mechanical tensioner in the cable for setting up the apparatus. While a little sag might not matter, a large amount of sagging could lead to measurement inaccuracies. I think this set up is a fairly low hazard level.

3. This apparatus will be much easier to package and ship, much, much cheaper to produce and less prone to damage and wear, because no one will be walking on it.

4. The system will be very scalable. It will be relatively easy to extend or reduce the length of a course. It would also be possible to include additional legs, as long as the subject is prepared to negotiate a free-standing pole along the route.

5. There may be other applications for the set up that I am envisioning, such as Wii Fencing. I just googled it, and it does not appear that wiiFencing has been done. Here's a link to a thread on that subject. A cheap set up for tracking linear motion along a virtual course may have a range of uses in both therapeutic and gaming domains.

So...I am going to go shopping tomorrow and buy stuff to set this up. I will be ready to start testing next week, and if it looks promising, the next thing we have to do is to go to all of the sites and make sure that they have a location where the poles could be set up.

proposed design for wiiCane modular walking surface

2009-09-13T19:27:00.006-04:00

After this weekend's meetings, in which I met with Zach Eveland, Annette Gourgey, Raphael Baptista and Gene Bourquin to make plans leading up to our test version of the WiiCane apparatus to take place at four sites in November. We made significant progress in several directions, and I am hoping that Gene, Zach, Rafael, and Annette will also produce posts on this blog capturing their view of the work as it is right now.

I am going to discuss the design for a modular walking surface for the WiiCane system. We decided to go this way after considerable back and forth discussion. We decided in the end that the floor mounted lights will be easier to track along the entire distance of the walking course (compared to lights mounted 30' away at the goal end of the course. Also this approach gives us the ability to make any length track, and even to make courses with turns, etc. (although I can't think of a reason to have that yet). This approach will be more expensive, and much more difficult to transport and set up. But, there are also distinct advantages, and it will be much simpler to achieve reliable position awareness doing it this way. And there are additional complexities, for example because the IR LED's have some thickness and they are crushable, we have to cover them with a surface that will protect them and the associated wiring harness. We also have to camoflage them tactilely, so that cane will not discover them, thereby giving away the straight line travel path, which is what we are training our users to do.

I am proposing a series of folding lightweight panels. 5 of these would lie end-to-end, creating a surface that is 8' wide by 30' long. The panels will be extremely light, and when all folded up, they would fit in a box that is 6' by 4' x 6", which is very manageable. It would fit easily in a station wagon. I am thinking that the panels would be shipped in a wooden box that would then become a head unit at the goal end of the course. This would house a camera, flat panel display, speakers, computer and any other devices. Each of the five panels will consist of a 4' x 6' fixed section and two hinged side pieces that are 2' x 6'. The panels will be 3/8" thick masonite board. When the hinged panels are folded in (covering the treading surface), the whole thing is 3/4" thick. The treading surface will be a long permanent self-adhesive floor banner from the printer we use to make tactiles. that printer produces 30" wide prints of any length, so we will burnish the prints to the masonite and trim off the excess. I want to try to use the prints themselves as a continuous hinge for folding up the panels for shipping, but that will take some experimentation.

The printing on the treading surface can include a long ruler in metric and english units, a big wiiCane logo, and different color lines to indicate set ups for various courses (green course, red course, blue course, etc). I just remembered that I once designed a book cover with Elga Joffee that we nicknamed the Corridor of Pain. It was a rendering of a theoretical hallway with one of each of the hazards disallowed in the ADA regulations, things that you can smack your head on, etc. I was thinking that we could base the placement of obstacles in one of the training course on some of those things, as a way of preparing for those kinds of conditions. The obstacles that we actually use would be inflatable or Nerf material. I like the idea for a future line of nerf obstacles! In any case, we have to start thinking about the pedagogical objectives and capabilities of this way of training people. We should end up the project with a producct manual that explains our reasoning and sources. We should also all be starting to think about publishing this work soon, assuming that we get some good outcomes.

The lights will be in three strips that will run the entire length of the course, one right down the middle and one in either of the two wings. These will be marked on the printed floor surface, so we can just use that as a template when drilling holes. We would pop the little LED through a hole so that they are slightly recessed from the surface of the treading surface, then fill the holes with clear epoxy. This will level the surface and protect the LEDs while allowing IR light to pass through. A cable will connect the lights to a control device in the head unit/storage crate. I don't yet know how to join the units to one another mechanically or electrically yet, so any ideas about that would be appreciated. A big neoprene rubber starting block would be at the opposite end of the course. This would have little semicircular cutouts for their heels while squaring off.

So, please give me feedback. I am going to start building a sample on Wedenday, with the hope of bringing it up with me to Boston during the week of September 28.

wiiCane version 3.0

2009-08-25T09:20:00.004-04:00

This new version of the wiiCane is a significant improvement over the previous one. The construction is similar to version 2.0 (machined and welded aluminum), but it is smaller and lighter. the aluminum fixture is very rigid, so you can really feel every nuance of tactile information transmitted up the shaft of the cane. It also seems to be well-balanced, regardless of how you point the wii remote. So, with this design we can experiment with a variety of light placements, including at the goal end of the course, on the floor, on the ceiling, or on the user's body. We now have to decide whether to use the adjustable length cane, or if we should just make four of fived fixed length versions. If everyone agrees that this is a good approach, I will ask Accufab up in Ithaca to make five more, and we can experiment with various finishes and branding strategies (logos, etc.)

WiiCane version 2.0

2009-08-23T12:52:00.003-04:00

This version of the WiiCane is made by chopping standard graphite shaft cane and removing an 8" section near the grip. We then installed a custom aluminum fabrication that returns the cane to its original length. the aluminum part holds a pivoting cradle, which in turn holds the Wii Remote device. The device can be pivoted around 360 degrees without significantly unbalancing the cane so that we can test using IR light sources in various positions. In terms of balance, stiffness and general feel, this version is considerably better than the previous plastic one. The tradeoff here was weight. This version is significantly heavier than version 1.0, and so we have re-engineered the part. It will arrive this week from Accufab in Ithaca. I will put together a new prototype for testing and see if the aluminum version will work. The next cane-based question is whether we should use multiple fixed-length canes or if we should try using an adjustable length cane. Gene is looking into that question now.

Research on veering

2009-08-23T12:26:00.004-04:00

According to an article in the New York Times, our work on WiiCane is highly relevant to ongoing research in the multi-sensory perception. The article discusses work by Dr. Jan L. Souman of Max Planck Institute for Biological Cybernetics. Dr. Souman is seeking scientific explanations for the "lost hiker" phenomenon, where people without access to visual cues always loop back on themselves when they are intending to walk in a straight line. One interesting outcome of this work is that it debunks the theory that the tendency to veer is an outcome of one leg being shorter than the other, since circling can happen in either direction (in the same individual). While WiiCane is not intended as a training system for hikers, I think we should pay attention to this parallel line of inquiry. If we can demonstrate that, using corrective and supportive audio and/or vibratory feedback, it is possible to teach people to walk straight, the research community may be interested to know about this. I sent Dr. Souman an email about WiiCane, I will post his response if he replies.

WiiCane Course Layout-Scheme No. 1

2009-07-03T21:38:00.003-04:00

This image shows one way that a course could be constructed for the WiiCane. Here, a 30' path is marked on the floor. A user is shown walking along the path towards the light bar at the goal end. The light bar is an 8' long horizontal beam that is supported on two adjustable stancions. In this rendering, the light bar is about 40" above the floor. A visual spectrum camera is mounted to the mid point of the light bar. This camera faces the user, and is intended to track a color dot that is pinned on the person's torso. The camera will track the person's absolute x-axis position (left or right of the travel path). Then, lights mounted on the face of the light bar provide information on arc width, arc-centeredness, and arc height at mid-swing. In this version, there are no lights on the floor. In this image, the floor has visual markings to help an observer determine the user's position in terms of distance from the starting point and amount of veering. A 24 inch wide starting block allows the user to square off at the beginning of the route.

wiiCane version 1.0

2009-07-02T22:29:00.004-04:00

This image shows a photograph of our first wiiCane. This is a standard mobility cane that has been modified as follows:

1. An 8" section (just below the grip) of the cane was chopped out.

2. A plastic part was inserted in place of the removed section of the cane. This part looks like two tuning forks connected end-to-end at their tines.

3. A smaller 3d printed part (painted red here) was mounted with screws to the first part so that it can pivot 360 degrees. This smaller part holds the wii device.

The completed wiiCane is slightly heavier than a standard cane, but not so much so that it becomes difficult or awkward to use. It also appears to be well-balanced, regardless of how the wii is rotated in the fixture. Now that we have this assembly, we can begin to do in-depth testing of a variety of different light configurations, so that we can determine which arrangement offers the most options for motion tracking during cane travel exercises.

2009-07-01T17:42:00.003-04:00

Here's a picture of our new 3d printer. It's the UPrint desktop printer from Stratysis. I got it for the purpose of outputting prototype enclosures for the NimblePad that we are working on under contract to Nimble Assessment Systems. But I am hoping to use this for many other things, including making lots of audio-tactile models and other touch-sensitive interactive displays. With the new usb sensor device that Zach is making producing now, and plastic models printed from this machine, we can make detailed very strong exhibits cheaply and rapidly. Right now, I am printing a pivoting fixture for mounting the wii remote to a mobility cane. I am hoping that other interesting new things will emerge from this, and would welcome comments or suggestions.

steve

wiiCane curriculum pedigogical objectives

2009-07-01T16:49:00.003-04:00

Now that we are starting to get more detailed in our understanding of what can be accomplished with the envisioned WiiCane system, we need to start thinking also about how it will be most effectively used in actual teaching. While it is too early to make any decisions about specific lessons or activities, we would be smart to begin developing how instruction with the wiiCane used in a rehab or school setting will probably occur. Gene will be the leader for this part of the work, and I am hoping that he will faciliate a discussion about teaching parallel to the one we are having about the technical aspects. We should start brainstorming about how we image using the system in a teaching environment, because that discussion will shape our decisions about what to emphasize in taking measurements.

As a starting point for this discussion, I suggest that we look at SAL, which is Sally Mangold's curriculum for braille literacy. In that, there are a set of short actitivies that are organized in order of ascending difficulty and skill mastery requirements. I am not sure if the aquisition of travel skills is analogous to learning braille, so it may be the wrong model, but this gives people something to respond to. Gene?

Mobility perspectives for post-Kalamazoo

2009-07-01T13:22:00.001-04:00

For technical reasons please continue by reading the follow up comments to this post.
Thanks Gene

Technology options

2009-07-01T11:22:00.003-04:00

The following is a list of the technology options we are considering for obtaining information about a traveler's movement and their cane handling technique.

Inertial measurement - uses movement information supplied by the Wii Remote's built-in accelerometers (and possibly the Wii MotionPlus gyrometers). This gives relative movement information from inside the Wii Remote's frame of reference.

Good for: detecting taps, determining cane orientation, detecting cane movement in the absence of other sensing methods

OK for: measuring arc width

Not good for: determining cane position, detecting veering

Light strip - an array of computer-controlled infrared lights on the floor which are visible to the Wii Remote's IR sensor. Like the light bar method this gives us an idea of the traveler's absolute position.

Good for: very precise arc width measurement when lights are in view, precisely determining traveler location, detecting veering

OK for: might be able to measure cane-centeredness and coverage

Not good for: easy installation (requires careful positioning of light strips or installation of walking surface)

Light bar - similar to the Wii's Sensor Bar this method uses the same principle as the light strip method - an array of infrared lights which can be seen by the Wii Remote's IR sensor gives us an absolute position reference to work with. In this case, several lights are mounted horizontally at the end of the course and are seen by the Remote's IR sensor.

Good for: arc width measurement, detecting veering, determining traveler location, simple setup

OK for: might be able to measure cane-centeredness and coverage

Not good for: less accurate than light strip

Rear-facing IR sensor - a second Remote or IR sensor mounted on the cane is positioned to view the traveler. Markers or lights affixed to the traveler's clothing are seen by the Remote or sensor.

Good for: very precise cane-centeredness and wrist isolation measurement, precise measurement of coverage if arc width is known

OK for: might be able to measure arc width

Not good for: detecting veering, determining traveler position

External camera - a computer-connected video camera at the end of the course records the traveler's progress. Computer vision software analyzes the video to determine information about traveler and cane movement.

Good for: measuring cane-centeredness and wrist isolation, detecting veering, measuring coverage if arc width is known, easy installation

Not good for: determining traveler location, precise measurement of arc width or coverage, long courses - even a 30-foot course may be too long

As no single method provides all necessary information, it is very likely that the ultimate embodiment of the WiiCane system will use two or more complementary methods. We will continue to refine the inertial measurement methods and investigate the other options.

Feedback design

2009-06-28T23:01:00.015-04:00

One positive aspect of having run the tests in Kalamazoo last week is that it permitted us to begin to see which kinds of feedback are going to be relatively easy to acheive (once we are able to take measurements during travel as expected) and which ones may be very difficult to do. In this posting, I am going to set down some thoughts about feedback, partially based on what we observed last week, and also based on what we have been talking about as a group. I am hoping that this will stimulate a discussion among group members as we enter this next crucial phase in the project.

I will start by listing the kinds of events that will or could occur during a wiiCane session (there may be others that I have not considered):

1. Tap-this is the result of striking the cane's tip on the groud at point of maximum x displacement. this is a necessary component of two-point touch (and our system may not work if the user does not execute a tap at the completion of each swing). Tap events can also occur in mid-swing and are highly recognizeable amid potentially noisy data. In the test in Kalamazoo, we were able to synchronize the Optotrak and WiiCane data streams temporally by having the user produce three rapid taps in succession when he had finished walking. Taps are detected using accelerometer values exclusively, cane orientation is not relevant to tap detection. Due to system latency, I don't think that we can issue any feedback on taps (because the user would probably notice a lag between tapping the cane and perceiving the feedback). Also, I don't think it serves an informational purpose, since the user knows he has tapped the cane (assuming we can filter out non-tap acceleration events). But detecting and recording tap events is a crucial first step in being able to interpret other aspects of the user's technique.

2. Mid-swing. This is probably defined as the point where the cane's azimuth is co-linear with the travel path, although it could also be thought of as the point at which the cane's azimuth is co-linear with the midpoint of a line connecting the right and left tap points of a single swing (when the traveler is swinging the cane but not walking). In some cases, it may be desireable to provide vibration feedback as the cane passes the swing midpoint. This form of feedback would provide an easy-to-follow tactile path; while this kind of guidance is probably not useful in mobility training (because the real world is not equipped with this feature), it may be useful for preliminary exercises (like training wheels on a bike). In practice, creating a vibratory feedback that coincides with the point of mid-swing may be difficult (due to system latency: see discussion of Tap events, above), although there may be ways to predict that the cane is about to pass the arc midpoint, and send the feedback slightly before the event occurs.

3. Arc width. This is defined as the either the maximum angular displacement of the cane as it moves through a single sweep, or as the perpendicular displacement of the cane (from the walking path) at each tap event. Accurate measurement of arc width is essential to the system as envisioned in the proposal. In our discussions so far, there appears to be some concensus that feedback on cane arc should be provided intermittently and not after every tap. In a preliminary version, we provided verbal feedback consisting of the phrases, "too narrow", "too wide" or "good". Probably our administration utility needs to have a means of adjusting the frequency of this kind of advice. My guess is that, for adult users, spoken feedback should be provided remedially, when arc width is observed to deviate from the ideal width by some software-configurable percentage. For kids, it might be better to use a positive-sounding earcon to signal that arc width is correct, and nothing when it is incorrect. I can imagine an exercise where the child is simply asked to walk or stand still and swing the cane to produce the praise sound.

4. Hand at midpoint of body. It will be challenging to detect when the cane hand is not held at midpoint of the body. We are considering a variety of methods for tracking this; it may be, however, that the best we can do is to ask them to start at the beginning of the course with their hand held at midpoint, and we should be able to watch them execute a few stationary (non-walking) swings to make sure that we recognize the gesteral pattern associated with type of movement.

5. Wrist isolation. Similarly, it will be hard to check for wrist isolation during travel. Classic two-point touch technique calls for wrist flexion and forearm immobilization while swinging the cane. We will probably be able to detect good form (and provide confirming or admonitory feedback) during stationary swings, but when motion associated with walking is added, it may not be feasible to track wrist position using a relatively simple set up of lights and cameras like the one we are considering here.

6. Wrist rotation. One big problem we ran into during the verification exercise at Western was that we were not able to correct for wrist rotation. When the user rotated his wrist, all of the lights we were tracking moved significantly, even for a very minor rotation. While this can be compensated for in our measurement algorithms, the problem becomes much more difficult. It is not clear (to me) whether classic two-point touch teaches the traveler to avoid rotating the wrist of the cane hand. However, we need to find a sensing method that is not affected by this kind of movement, or we have to train the user to avoid wrist rotation, something that might be very hard to do with kids. We may find that this problem disappears when we switch to a different arrangement of lights. This issues is an imporant one, and I think that the accuracy of our system might be negatively affected by failure to account for this kind of motion.

8. Cane/step synchronization. While it was not one of our original measurement and feedback objectives, it would be very helpful to know whether the user's cane swings are properly synchronized with their steps. According to Zach, accelerometers mounted to the person could determine gait, but our current plan calls for mounting the wii device on the cane, not on the person, so we cannot assume that periodic motion in the z-axis (as reported by the wii's accelerometers) is related to stepping movements. It is possible to add another wii device and attach that one to the user's belt (for example), but we are trying to avoid adding additional hardware if that's possible.

9. Arc centered-ness. Another feature of classic two point touch technique is that the arc in the x-y plane that is described by the cane as it swings back and forth should be centered. That is, the line that bisects the arc should align with the travel path, and, conversely, the angular displacement of the cane to the right and left sides should be equal. it appears that this will be extremely difficult to determine conclusively, and we need to think carefully about how important this measurement is to the overall usefulness of the eventual wiiCane training system.

6. Veering. The literature indicates that everyone tends to veer in the absence of usable visual, tactile, or auditory information about the environment. Current research is considering ways to train blind pedestrians to overcome or correct for this tendency, because some mobility tasks, such as crossing wide streets, can be dangerous for blind travelers, and experts believe that if individuals could be trained to walk straight, accidents could be reduced and independence improved. One goal of the WiiCane project is to particate in this ongoing inquiry, and to produce a very practical tool for measuring veering.

Veering occurs (in my definition) when the wiiCane user's path deviates from a straight line in respect to a fixed perpendiular surface (the surface against which the user "squared-off" before starting to walk). Veering will be measured by the wii's camera, which can be trained to observe the position of infrared lights that we will array in the environment. Lights can be mounted on a variety of surfaces (including floor, ceiling, walls, the cane and the user's body), and one big job we are facing now is to study a variety of configurations to determine what works best. This is a challenging problem, because we have to be able to detect veering and to correct the user's performance before he or she has walked off to a position where they can no longer be "seen" by the wii's camera. So, our plan for providing feedback in this case is of the utmost importance if we expect to avoid the frustration of having to repeatedly return the user to the starting point because our system has lost track of their location and cannot recover. My guess is that, as with cane width, it will work best to provide spoken corrective messages when veering occurs with adults, and positive earcons when a child is successfully walking straight.

7. Distance. One important measurement that should be relatively easy to take will be distance to the goal. In the event that lights are positioned along a horizontal bar at the goal end of the course, we believe that the wii will be able to use a multi-step process, including a range-finding step (where we determine which are the widest apart lights that can be seen in a typical arc to set the gross distance) and a fine-tuning step where we look at the perceived distance between the lights. This measurement, however, can only take place when the user is more or less pointing the cane in the direction of the goal. Distance measurement will be an important component in constructing a graphical display of cane movement over time. There also may feedback opportunities associated with the student's position along the course, for example, we might want the system to speak "ten feet", "you are halfway there", or "turn left" in the event we construct a game which includes more complex route configurations.

9. Goal attained/Point awarded. Another form of feedback that we need to consider is associated with events that are not, strictly speaking, driven by cane motion. In developing a curriculum for building cane skills using wiiCane, we may want to consider various interactive activities involving game scenarios similar to SAL, which is our braille literacy system. That has a number of simple games that students and teachers find incentivizing, especially when a score for an activity is announced, and the student can practice to get a better result. We need to create a lexicon of feedback options that are stimulating and helpful for all of the populations for whom we are designing.

11. Improper usage. We know that some behaviors with the wiiCane will not be trackable. In the event the user veers significantly, there may be no way to recover location awareness, and the instructor will have to intervene. If this experience is repeated often enough, students may get discouraged and lose interest. In general, we want to promote a feeling of independence and control, and having to go back to the beginning may disincentivize further training. Therefore, we need to consider how we will use feedback to head off extreme veering before it's too late. Other researchers have set up courses that were designed to measure the onset and degree of veering; but in our case, we want to prevent veering to the point that we no longer can sense the user's position. I think feedbacks can be designed that accomplish this, but that are not overly intrusive or interfering with the student's efforts to practice walking straight.

12. Dangerous usage. In extreme cases, it will be necessary to let the user know that their movements with the cane are potentially dangerous. We have to observe a lot of people using the system before we will know all of the potentially dangerous things that could happen, and we have to be careful to carefully supervise subjects while they are using the product at all times. There are also product liablility issues that must be considered in any product where people are mobile, especially if our system calls for occluding hearing or vision. Also, in designing any curriculum for the system, we have to make sure to consider possible injuries, and discuss necessary warnings and precautions with someone knowledgeable about this (like a lawyer).

13. System alerts. Finally, we must consider that there may be a number of system warnings that we may want to include in feedback schedules. For sure, there should be some kind of "low battery" warning.

Data logging and graphical representations

2009-06-28T18:20:00.010-04:00

In our proposal, we talked specifically about the need for maintaining a record of student performance as they carry our various training excercises, games or other activities with the WiiCane. Depending on the applications to which our system is put, we need to capture movement along the course in formats that will be of interest and of use to trainers and teachers. So, we need to begin considering how this will be accomplished in our system, and who will be responsibile for its implementation.

We looked at the output from RoboCane, a project carried out by Bruce Blasch, et. al. as a starting place for thinking about this.

Robocane was a computer simulator for cane travel. Researchers could enter parameters like cane length and traveler dimensions, and

the system produced a graphical representation of theoretical traveler matching those characteristics walking a straight course was produced. The purpose of this software was to allow the developers see the impact of making changes to various paramters on efficiency and safety for the theoretical traveler. It was expected that this information could be interpreted in the context of training actual students, to help the traveler optimize his or her performance by adjusting individual attributes of his or her technique.

Graphical record of cane travel from RoboCane

from: Blasch, Weiner & Welsh, Foundations of Orientation and Mobility, second edition

I am hoping that our system will be capable of generating graphical output like the sketch in this post. This shows a record of cane movement over a course: slanted lines show the top view of the cane at each time increment, and the line that represents the widest sweep (coinciding with the observed cane tap event) is darkened to make it stand out and to emphasize the period of the classic sine wave pattern produced during perfect execution of two-point touch technique. A side view will help illustrate the movement of the cane in the z (vertical) direction; a different sine wave will show the movement of the cane as the tip reaches its apex (height above the floor) at mid-swing.

It should be relatively straightforward to generate a visualization of wiiCane data like I am describing here. I am hoping that we can set up a process to do this as part of the software running on the computer that is being used to communictate with the wii. It would be helpful if a visual display on the computer showed the diagram being produced in real time. In the Optotrak system that we used in the verification experiment at WMU, an image can be produced that shows the movement of the cane in all three axes, however this is shown as three separate lines, and the there is no real-time display of the data. I will ask Rob to post an image of the data that Optotrak produced when we ran some subjects last week.

wiiCane fixture design concept

2009-06-28T15:38:00.004-04:00

Okay, here is my idea for how to fix the wii to the cane so that all of my conditions in the previous post are satisfied. This is a two-part plastic assembly. The wii is slid into the pivoting cradle (labled "pivoting section"). There is a matching slot on the device to accept the little track, so you can slide the device in until a click is heard. This occurs when the part labeled "detente" drops into a matching recess on the device. Once the device is in place, it should not tend to slide back out, regardless of the angle that it is held, although this will have to be tested. It's very important that the device be held motionless in relation to the cane. The pivoting section, then, is placed within the fixed section, and 2 pairs of screws with matching threaded sleeves are used to hold this assembly together. When the screws are loose, it is possible to freely rotate the wii (held by the pivoting section) 360 degrees, with minimal blockage of the wii's camera, even when the angle of rotations is almost 0 (in respect to the cane). The camera will be blocked, and the wiiCane unusable, when the device is exactly aligned with the camera, but it seems to me that this does not limit the kinds of light configurations that we will be able to observe and respond to during travel exercises.

If it turns out that everynoe likes this approach, I will have 3D prints made so that we can begin evaluating the parts quickly.

Designing the WiiCane

2009-06-27T20:59:00.007-04:00

In response to general agreement among team members that I have spoken to, I am going forward with some preliminary sketches for the fitting that we will use to mount the WiiMote device to the cane. My operating assumptions are as follows:

1. The fixture needs to be able to hold the wii so that it can point in any direction along a vertical plane that passes through the idealized travel path. There may be different ways to use the device that call for positioning lights in ways that are hard to predict now. This fixture will let us experiment with various configurations and courses. For example, with this design, we could place lights on the floor, at the goal, on the user's body or shoes, or on the ceiling. This appears to give us enough flexibiliy so that we will be able to determine the best way forward experimentally.

2. The fixture needs to distribute the weight of the device evenly and symetrically regardless of which way the device is pointing. In the current design, as long as the cane is held so that the flat portion of the grip is sideways, the cane will remain balanced. Since the direction that the flat of the grip points is determined by the hand it is held in, it may be necessary to remove the wii from the fixture and reverse it depending on whether the user is a rightly or a lefty.

3. The fixture needs to be lightweight, so that the experience of using WiiCane is not very different from using a regular cane.

4. The fixture has to be strong enough to withstand reasonable abuse. Since we are replacing an 8" section of the cane with the plastic fixture, significant forces may be imposed on this part. While the design will take into account this requirement, we may not be able to acheive the same strength from the 3D print as we would expect from a metal casting or mass produced plastic part.

5. In this approach, where we modify a standard cane with our plastic fixture, it will probably be most practical to use an adjustable length cane like the one we borrowed from Rob in K-z00, rather than multiple fixed-length canes as in our prior approach.

6. there needs to be a very simple way to insert and remove the device in the fixture. I am thinking about a thumb screw or hand-tightened knob on the side that will permit us to loosen the knob or screw, then swivel the unit to the desired angle and then tighten it.

7. There should be some form or pointer or indicator to register the angle of the unit's tilt (in some suitable units), so that teachers can set the angle with precision.

WiiCane Kalamazoo Verification Exercise Report June 23, 2009

2009-06-27T20:50:00.002-04:00

Click here to download the pdf version of this report.

Click here to download the text version of this report.

WiiCane System - Technical Description

2009-06-24T13:53:00.004-04:00

Apologies for the long post, but Steven has asked me to present the tech side of the WiiCane system as it stands now. Lots of new stuff here - I hope some of it is of interest!

The system tested at Western Michigan University was made up of a strip of discretely-controllable infrared emitters (940 nm wavelength) fixed in a wooden strip on 12" centers. This light strip, composed of 32 individual emitters, together with a controller device attached to the host computer, provided reference lights detectable by the Wii Remote infrared sensor.

As the Wii Remote's infrared sensor is sensitive to light in the 940 nm spectrum, we are able to derive positional and orientation information of the Remote relative to the light strip by analyzing the perceived locations of lights. The sensor itself provides Cartesian coordinates and perceived brightness values for up to four lights in the sensor's field of view. In the presence of more than four lights, the sensor reports information on the four it perceives to be brightest. Generally these are the four lights closest to the camera, but the perceived brightness may be affected by other factors such as occlusion of lights or off-axis orientation of the Remote relative to the emitter's emission angle. The top picture in this post shows the host application view of four infrared emitters in a double strip.

The software used for the trial attempts to determine arc width and to measure the amount of veering. It does this by analyzing raw accelerometer and infrared sensor data provided by resources in the Remote. Accelerometer data is normalized and used to detect discrete "tap" events marking the beginning and end of an arc. During an arc, data from the infrared sensor is used to develop a model of the traveler's movements in order to determine arc width and amount of veering.

To analyze movement, the reported infrared sensor data is plotted and examined from frame to frame. In order to calculate arc width, the position of all visible lights is noted at the beginning of an arc (defined as the first "tap") and the changing position of these marker lights is then followed as the traveler completes the arc. The final position of the lights at the end of the arc or second "tap" is noted and the total movement is calculated based on the distance between the initial and final positions. A scaling factor (experimentally determined based on cane length) is then applied to the calculated distance value to produce a reading in mm.

Veering detection follows a similar method. Once an arc is completed, the arc time is first calculated based on the time between the first and second taps. The infrared sensor data collected for the arc is then examined - it is scanned to find the time when the Remote was most closely aligned with the light strip (determined by examining the light locations which outline the strip itself). Since the light strip makes up the desired course of travel and since the arc should cross the light strip approximately halfway through the arc when the traveler is not veering, it is possible, based on the timing figures just calculated, to determine how much veering has occurred and in which direction. The highlighted area in the bottom picture in this post shows typical results for the analysis of three cane arcs.

In addition to the analysis tasks performed by the host software, it also serves to log raw accelerometer and infrared sensor data received from the remote for later analysis. A simple display area is also provided to aid visual analysis and debugging.

2009-06-23T15:44:00.003-04:00

I just got done meeting with Zach, and we came up with an idea that I think will solve several problems at once. This image to the left shows a customized cane that is designed to hold a Wii remote device so that it can be angled such that the camera is pointing either down towards lights in a strip on the floor, or level, towards lights mounted at the goal end of the course. This allows us to continue to experiment with both versions, and also could end up becoming part of the final product. To make this cane, we propose starting with an adjustable length cane, like the one that Rob Wall Emerson showed us when we visited him last week in Kalamazoo. We would cut out a 6" section of the shaft of the cane near the grip, and replace that with a plastic fork that holds the wii device such that it can pivot. A tightening screw would allow us to fix the angle as needed to account for different configurations of sensing lights, and also to compensate for diffences due to the height of the user and cane length. One nice thing about this is that it should not noticably affect the balance of the cane.

latest results on feedback and veering

2009-06-17T15:51:00.004-04:00

After 38 treatments composed of feedback about left and right veering, you can see the trend toward less veeering from an intended path. Guth recommends about 160 treatments to see signifcant and lasting changes to veering behavior. The x axis is the number of treatments; the y axis is the number of paces walked forward before the subject veered into the wall on the left or right.

early tests on feedback and veering - charts

2009-05-26T18:37:00.006-04:00

I have been asking students at my agency who require improvement in the veering to allow me to record the numbers on their normal practice, and the trend in their veering behavior after feedback. One student has had the most (n=34) treatments of feedback (indications to move left or right) and engaged in 34 trials where the number of paces he could walk before contacting a wall (in an 8 foot wide hallway) with his cane. Above is a graph, the x axis being the number of trials and the y axis being the pace walked. There is a trend toward improvement but not dramatic.

The graph above shows the results of trials and the average paces walked (6 trips on the path) without touching a wall. The initial trial is an average about 27 paces, the least number of paces is at about the 24th trials with about 18 paces walked, and the two longest average distances walked without touching a wall are at about the 28th and 34th trials. The trend is for less veering.

I reviewed Guth’s article and his email comments to me. He saw improvement after 6 to 8 trials of 15 treatments each. This means after 120 to 160 individual treatments saw recognizable improvement, so our results seem okay. I infer one major issue from this. We need to figure out how to do many more trials than the originally planned 8 to 20. We must do 60 or more at a minimum with each subject to see improvements.

I’ll continue to record the data on my students, doing about 8 treatments and two trials a week. The preliminary numbers look okay and that is the good news.

Here is a chart on the second student with simlar results - initial paces about 22, and the most paces walked after the 7th and 13th (last) trials:

Arc Width Measurement Demonstration

2009-05-26T12:44:00.002-04:00

As part of the last Friday's project meeting, we demonstrated the latest arc width measurement software application. This application analyzes raw acceleration data from a Wii Remote to compute timing, velocity, and distance for each arc. The resulting distance measurements, together with user set thresholds, are used to generate spoken audio feedback.

On-screen slider controls are used to set thresholds for determining whether an arc is too wide, too narrow, or in the correct range. Other controls are used to set feedback options allowing the user to specify whether feedback should or should not be given when an arc is correct or is too wide or narrow.

Ellen Rubin, Gene Bourquin, and Steven Landau tested the system at the Touch Graphics offices and results were generally positive though the methods used to compute arc width need additional work. Currently, the biggest shortcoming is in the math used to double-integrate instantaneous acceleration to produce distance. Any "roll" in the cane which moves the Wii Remote's Y-Z plane off of alignment with gravity affect the calculated distance traveled along the Remote's X axis.

Future work will concentrate on better using static acceleration readings and an external reference to the light strip to improve arc width distance calculations.

teaching and reinforcing appropriate cane arc width

2009-04-16T23:31:00.002-04:00

teaching appropriate cane arc width
In Rob’s (and Ashmead) 2002 article we find these statements:

“For this study, the widest part of the participants’ bodies were their shoulders. Across groups, conditions, and sessions, arc widths were 38.9 cm (or about 1 ft) wider than the participants’ shoulder widths”

“The participants increased their arc width-to-shoulder width differential when they were first reminded of the proper two-point touch technique, increased it further when they relaxed their technique, and then reduced it when they were reminded of the proper technique the second time. Given the differences in hand position and wrist bending already noted, it appears that the participants were not able to use biomechanical feedback to structure their technique. They erred on the side of caution and used arc widths twice as wide as prescribed so that any other deviation in technique would not shortchange body coverage.

The involvement of wrist movement and hand position is not sufficient to explain the pattern of arc widths across conditions. The movement of horizontal hand position to the right in the Normal and Comfort conditions, with an accompanying lessening of wrist extension, would lead one to expect smaller arc widths in the Normal and Comfort conditions and larger arc widths in the Classic1 and Classic2 conditions. However, arc width-to-shoulder differentials increased from the Normal to the Classic1 to the Comfort conditions and decreased in the Classic2 condition. This pattern of results is different from any of the other measures. It may indicate that arc width is not solely a biomechanical process but is strongly influenced by cognitive demand, more so than other aspects of the long-cane technique."

Does this mean that WiiCane feedback will be an effective tool because it will reinforce via a audio-tatctile-cognitive channel?

We do not have any hard data on teaching the production of an effective cane arc width. Motor-skills learning is well documented, but we have no studies on what constitutes a strategy or approach to reinforcing a target behavior. There is also scant annecdotal data.

Sadowski, J. (2004). Springboard. Re:View, 35(4), 160-164.

She’s Called on the Carpet to Teach Arc Width

I have used several methods to teach arc width. Most give an indication of the arc width by providing a stopping place for the cane on each side of the swing. Two canes extended on the floor, parallel to the line of travel and to each other, or a guide made of PVC pipe, effectively provide information concerning arc width, but they do not teach control of the arc width. I find many students enjoy hitting the boundary objects for the sound feedback and swing harder than is necessary. The result is that when the guide is taken away, the student is not in control of the swing and the arc width is too wide. As a result students may not develop the touch and control needed to demonstrate consistent, correct arc width.

One day while standing in the hall, pondering the problem, I spotted the rugs that were placed outside the classrooms for students to place their winter boots on to dry. The width was about the correct width of a particular student’s arc. We experimented with his standing centered on the carpet and swinging the cane back and forth. The rug edge provided feedback to help him know when to begin the counter arc. It helped him to learn the correct width and to develop the control of the cane that solid obstacle barriersdid not.Longer lengths of carpet can be used to permit the student to walk a distance and practice arcing the cane while moving. This provides an added bonus of feedback concerning straight line of travel.Pieces of rug can be found at carpet store remnant bins. Most dealers are willing to donate a section of carpet when they understand the use. Longer lengths can be used to permit the student to walk a distance and practice arcing the cane while moving. Storage and transport are relatively easy because the carpet pieces can be rolled up and secured with a bungee cord.

Jane Sadowski, COMS
Special Education Districts of McHenry/Lake Counties
1200 Claussen Drive, Woodstock, IL 60098
jafsadowski@ameritech.net

Veering re-visited: Noise and posture cues in walking without sight

2009-04-16T19:59:00.002-04:00

I thought that the information about auditory stim and veering was interesting and something to keep in mind when we do a final design on the spoken/sounds feedback system.

Millar, S. (1999). Veering re-visited: Noise and posture cues in walking without sight. Perception, 28(6), 765-780.

Millar S.
Department of Experimental Psychology, University of Oxford, UK. susanna.millar@psy.ox.ac.uk
Effects of sound and posture cues on veering from the straight-ahead were tested with young blind children in an unfamiliar space that lacked orienting cues. In a pre-test with a previously heard target sound, all subjects walked straight to the target. A recording device, which sampled the locomotor trajectories automatically, showed that, without prior cues from target locations, subjects tended to veer more to the side from which they heard a brief, irrelevant noise. Carrying a load on one side produced more veering to the opposite side. The detailed samples showed that, underlying the main trajectories, were alternating concave and convex (left and right) movements, suggesting stepwise changes in body position. It is argued that the same external and body-centred cues that contribute to reference-frame orientation for locomotion when they converge and concur, influence the direction of veering when the cues occur in isolation in environments that lack converging reference information.

What does not cause veering; what does not help

2009-04-16T19:23:00.002-04:00

Kallie, Schrater, and Legge seem to be good at telling us what does not cause veering: inexperience walking without vision; and they tell us what does not help: an explicit indicator of intended walking direction and a skill for curvature detection. Here are some excerpts:

What causes this veering behavior? Our research suggests that a simple explanation, unperceived motor noise at the level of individual steps, may explain the veering behavior of blind pedestrians and sighted pedestrians who are blindfolded. (p. 183)
A pedestrian’s ability to walk a straight line depends on the availability and quality of sensory information about walking direction (Loomis & Beall, 1998; Philbeck, Loomis, & Beall, 1997; Rieser, Ashmead, Talor, & Youngquist, 1990) and on the capacity to execute movements in an intended direction. (p. 184)
Although blind walkers often use acoustic or tactile cues during walking, those cues are often uninformative and sometimes misleading. For that reason, we asked what factors limit straight-line walking in the absence of visual, acoustic, or tactile cues. (p. 184)
The lack of difference in veering behavior between sighted and blind participants suggests that a history of visual experience is not critical to performance on this task. (p. 187)
The results of the present experiment do not support the hypothesis that people who veer the least by the previously mentioned measures are most sensitive to path curvature. For emphasis of this
point, the participant (P3) who was best at curvature detection (threshold radius = 36.48 m) also was the one who veered the most in the straight-line-walking task (mean unsigned deviation
1.73 m). (p. 188-189)
An explicit indicator of intended walking direction did not reduce veering behavior. In fact, in the case of the static perceptual pointer condition, veering actually increased compared with the physical alignment condition, in which there was no explicit pointer. The increased veer is due mainly to a larger linear component in the participants’ trajectories. (p. 191-192)

Veering? What veering? (and a new article)

2009-04-16T18:21:00.000-04:00

I am posting these excerpts from Guth’s chapter as a reminder of the theoretical and experimental underpinnings of at least one of the variables we are dealing with in the WiiCane project:

Once the veering tendency of each of these individuals had been well documented, he or she participated in a series of about fifteen 20-trial training sessions. The training procedure was modeled after bandwidth feedback approaches used in athletic training (Schmidt, 1991). During each training trial, a participant started with his or her back to a portable wall, walked away from the wall, and attempted to stay within a 2-m wide, 20-m long simulated crosswalk defined by two parallel, ankle-level infrared beams. Whenever either beam was broken, the participant immediately heard—through earphones—
the direction of the error and the distance that had been traveled before veering out of the “crosswalk.” (p. 356)
All participants exhibited marked improvement over the course of this training and, as illustrated in Figure 18–1, the effects were still evident 5 months after the cessation of training. (p. 356).
While one participant reported a cognitive strategy that wasn’t consistently effective (i.e., “step to the left every few steps”), the others reported that they simply “learned what it felt like to walk straight.” (p. 357)
Our feedback system was unlike anything experienced during the everyday travel of blind pedestrians. It required that well-aligned participants attempt to generate a straight line path and stopped them when they had deviated 1 m to either side of the intended path. (p. 358)
Although blind pedestrians have many opportunities to experience straight-line walking, this experience occurs in the presence of continuous guidance. (p 358)
One of the surprising aspects of our training study was that the effects lasted for at least 5 months, the point at which we ceased taking follow-up data. (p. 359)
The training study revealed that practice with feedback is useful, but it remains unclear what elements of locomotion were modified during training. (p. 359)

Also, you may want to check out this other journal article:

Kallie, C. S., Schrater, P. R., & Legge, G. E. (2007). Variability in stepping direction explains the veering behavior of blind walkers. Journal of Experimental Psychology: Human Perception and Performance, 33(1), 183-200.

I cannot attach it here but will email the article to stakeholders.

light strip proof-of-concept

2009-04-08T11:45:00.002-04:00

On April 7th I completed construction and testing of the proof-of-concept WiiCane light strip. The purpose was to test the feasibility of using the WiiCane setup with infrared LEDs embedded in a strip on the floor. The light strip provides waypoint markers which can be detected by the remote and software to provide additional positional and movement data. The active LED strip is an improvement on and supercedes the previous concept of using reflective strips on the floor to provide additional movement data.

This version was built as a six-foot strip, with infrared LEDs spaced 12 inches apart. The LEDs are constantly illuminated, drawing approximately 50 mA each and supplying 7 mW of light with a wavelength of 940 nm. The LED package chosen emits with a 34 degree viewing angle.

Initial results look very good - the remote is able to see the LEDs at a fairly oblique angle, down to about 50 degrees off-axis. The lights show up at a distance of at least two or three feet, even against moderately-bright background lighting. The bottom picture shows the view reported by the remote when two LEDs are in the remote's field of view.

Notes from March 16 meeting

2009-03-18T10:27:00.000-04:00

A short project meeting took place on March 16. In attendance were; Steven, Gene, Ellen, Ting, Dan, and Zach. Discussion centered on methods for providing directional feedback to deaf users of the WiiCane system and integration of the WiiCane and Optotrak systems for testing at WMU. Some brief notes on the discussion:

Gene recapped his headphone tests - two deaf users were unable to feel high- or low-frequency vibrations from the Sennheiser headphones. Gene indicated that he feels the headphone cushions dampen the vibration from the drivers and laid out a few options for making the vibrations felt including; attaching tinfoil to the drivers and removing the headphone cushions.

Dan suggested using two additional Wii Remotes on the user's body to provide additional channels of vibratory feedback. Agreement all around - we'll try this next.

There was further talk of the placement of the Wii remote and infrared illuminator on the cane. Main concerns here seem to be minimizing the effect of the additional weight on the user's cane-handling, and ensuring the placement of the remote doesn't interfere with the user's normal grip. Ellen and Gene gave some direction and Steven will fabricate an adjustable mount to evaluate placement.

We closed with a short discussion about running the WiiCane and Optotrak systems simultaneously. There is some concern that the WiiCane's infrared illuminator will interfere with the Optotrak's infrared-sensitive cameras or that the Optotrak's infrared-emitting markers will interfere with the WiiCane's sensor. Zach will followup with Rob to determine if both systems can be run simultaneously.

headphones testing

2009-03-13T08:12:00.005-04:00

Headphone testing

Sennheiser TR/HDR 120, 22 Hz to 19,500 Hz

This is not a success story. I set up the headphones on a desktop at HKNC. The phones came with batteries and adapters for the plugs. Setup was easy. I could not save the sound file, but I was able to play it from the email. The controls for volume and forward/rewind appeared in the email. One feature of the phones I did not like was the “tune” slide-wheel needed to optimize the signal between the transmitter and the phones. If not set just right the signal was less than strong.

I set the phones and the computer volume on maximum; there was no frequency controls. I tested the phones with on deafblind subject and on deaf subject. Hearing loss was severe-profound to profound. Both subjects did not wear hear aids. Neither subject could feel the vibrations from the phones during several sound playback. I removed the pads from each ear phone, but neither could feel the vibration. I asked the subjects to place there hands on the phones, but neither could feel the vibrations.

Possible problems/solutions:

* It seems I could not get sufficient volume. Is it possible to create a file with significantly higher dB output? I would like to raise the volume and the lower frequencies.
* I plan to try ear buds. I have a pair from my iPod.
* A lighter or different phone material – which might reverberate better – might work.

There was literally no recognition from the subjects of any vibrations, even when focused on the task.

We might need to go to plan B for this. I am open to ideas and suggestions.

Wii Remote and IR Illuminator

2009-03-09T13:27:00.008-04:00

I have finished the first round of tests with the infrared illuminator, Wii Remote, and retro-reflective materials. The results look good all around - using a simple IR flashlight with three LEDs emitting at 940 nm, the Remote's camera is able to track retroreflective samples at a distance of three feet or so.

There are a number of design improvements to make for the next round of testing including; increasing light output, increasing efficiency, and reducing weight of the package. This first unit is headed to Robert now for testing with his Optotrak system.

headphones

2009-03-08T18:52:00.002-04:00

I met with Steve today at TG and we reviewed various parts of the project. We visited B&H and purchased wireless headphones, Sennheiser TR/HDR 120, 22 Hz to 19,500 Hz. I will be bringing them to HKNC and trying them an several deafblind students. If the headphones' output works and the deafblind folks can readily identify left/right, then I'd like to try this with the Wii system.

Steve and I also discussed the vibration-motor output on the Wii, which we want to use as an alternative for veering feedback (spoken feedback via headphones as the other system output).

Wii Remote IR sensitivity

2009-03-06T16:48:00.003-05:00

One of the open questions about using the Wii Remote's infrared-sensitive camera has been which part of the infrared spectrum it was most sensitive to. Our plan to track retroreflective strips on the course will work best if we flood the scene with IR illumination in the right range.

There is no official word on the subject from Nintendo, but there is anecdotal evidence that the optimal wavelength is 940 nm.

I have tested IR illuminators at five different fixed wavelengths: 850, 870, 890, 940, and 950 nm. My results agree with those above - the Remote's camera seems to be most sensitive to illumination at 940 nm.

experimental design for K-zoo verification

2009-03-02T15:06:00.002-05:00

An interesting discussion has been taking place between Zach and Rob regarding the best way to run the comparison between the WiiCane and the Optotrak systems that we have planned for early June. As everyone already knows, our project calls for running test subjects along a 30 foot straight line course using both systems. both apparatus call for mounting IR light sources on the cane, and Zach and Rob think they might interfere with one another, which would significantly complicate matters. They also wonder whether it is necessary or important, from an experimental design perspective, to do them simultaneosly, so that we end up with head-to-head, swing-by-swing comparisons for individual running of the course by study participants. I have always thought that they would be done separately, but I can see very good arguments for doing them together. We will be much more confident in the WiiCane system's accuracy if we can compare results collected simultaneously, because we will see, for each participant, whether the device is accurate or not, and if it is not, we will immediately know which aspects of our system need improvement. But if we do them separately, we will have to aggregate the results and try to draw conclusions, which is certainly possible, but will require more number crunching, and will inevitably be less directly meaningful. I think that Annette, as the Project Evaluator, should weigh in here. Of course, we will not know if this is a problem until we actually try both systems in the presence of IR light sources. I think we need to do this test right away, and certainly not wait until we are in K-zoo to do it. It would be best if we could use a single IR source mounted on the cane that can work for both WiiCane and Optotrak. Having two many devices and cables mounted to the cane and cables will affect weight and balance, and may have other unexpected consequences.

2/23/09 meeting at Touch Graphic

2009-02-24T07:38:00.001-05:00

Steve, Zach, Annette, Ellen, Rob (by phone), and Gene met for a status and update session at the new and improved Touch Graphic’s office space on West 38 Street for approximately 2 hours. Steve announced that we now have a 30 foot area in the office to conduct tests on a WiiCane course. Zach reported that we should have a “working system” with fairly accurate output within a month.

The following details the highlights of the meeting:
* Zach demonstrated the limited audio capacity of the Wii controller. Speech and sounds emitting from the controller are both low in dB and in quality, and speech was unintelligible. Three possible solutions were proposed: wireless headphones that could deliver high-quality speech output, directional sounds, and possibly (low frequency, higher volume) sound that could be received as vibration on the head by deafblind users; a belt-pack-system that could emit speech and sounds and vibratory (rumbler) output when prompted by the controller; a collar-neck pillow that could house an audio-vibratory device(s).and possibly ear buds. Auditory occlusion, positives and negatives for the training device, was discussed. No final selection made. Gene will investigate possible headphone devices.
* Zach demonstrated the limited but detectable vibratory output capacity of the Wii controller. The vibrations were perceptible when the controller was attached to a cane with tape. If this is used, the output would be presented when the cane is not touching a surface.
* Zach demonstrated the infrared sensing capacity of the controller. The controller camera can locate “blobs” of light sources (up to four) and display/plot them on the laptop screen. We discussed the use of a single central strip of light-emitting material on the course, versus a set of two strips defining left-right boundaries. These would be used to detect veering, and as a possible source of data to calculate other variables.
* Testing in Kalamazoo to verify the capabilities of the WiiCane system will probably occur in early June 2009. Arc width detection and veering detection will be the two variables verified. Rob is handling the IRB at WMU; the in-house IRB process is being handled by Annette and this will be delayed until after the WMU tests.

There will be an advisory board meeting soon, probably at the end of March, before an April 2009 report is due to the funder.

If attendees have corrections or additions, they would be appreciated. Likewise, questions and comments are welcomes.

Thanks…..Gene

early intervention O&M article

2009-02-09T12:25:00.001-05:00

A recent article in the International Journal of Orientation & Mobility about “early intervention O&M” was relevant to our project. The author, from western Australia, makes several statements about the approach to children and the use of mobility canes. I do not have an electronic version to share. I’ll summarize what I think are important points from the article.
* cane techniques such as two-point touch are not introduced to children under the age of 6 because of the development of strength and motor skills
* traditional arguments against early cane use have not been found to happen in their programs (e.g., fear of injuries to others)
* keeping the cane “close to the ground” is strongly emphasized with young cane users
* the first formal technique is diagonal cane position with the correct grasp

None of this is striking or new, but I found the age information interesting.

The journal might be found in your academic libraries; I couldn’t find it in mine yet.

Scott, B. (2008). Early intervention orientation and mobility: A Western Australian persepctive. International Journal of Orientation & Mobility, 1(1), 70-72.

Wii MotionPlus coming soon

2009-02-01T10:25:00.002-05:00

A new Nintendo Wii accessory may be worth investigation for the WiiCane project. The MotionPlus controller accessory adds additional sensing capabilities to the controller, complementing the standard 3 degrees-of-freedom accelerometer features with angular rate change sensing.

Nintendo unveiled the MotionPlus at last year's E3 and announced plans to begin shipments in 2009. See Nintendo's press release for more information.

Feedback for young subjects

2009-01-31T11:06:00.003-05:00

This is not a new topic, but one which still seems unresolved. One of the discussions which we had with both the focus group and advisory board is the nature of feedback -- that is the type of reinforcement(s) that will work for system users (i.e., subjects). We are particularly concerned about young visually impaired travelers who may not readily understand that a particular stimulus is meant to be a positive indication of cane behavior. Parents and specialists indicated that some users may respond with behaviors that elicit any type of stim. Bells, chime, and pleasant sounds have been suggested. Spoken messages have been suggested, as well as vibratory output.

We have participants with many years of experience with younger travelers. I am wondering if blog participants can discuss from their experiences what kind of reinforcements makes the most sense. We are still not certain of the exact capacity if the wii output for feedback, but we should consider all options right now.

jan 21 meeting

2009-01-26T15:21:00.001-05:00

Reflecting on our discussion during the meeting, here are my thoughts:

1. Type of feedback (auditory or vibratory):
Since there is no established literature as to what type of feedback is better,
will it be possible to record data in which some kids are given audio feedback
and a few other kids that are given vibratory feedback (in case this feedback
modality is going to be considered). In that way, we can avoid the issue of
learning if the same kids wherein they will end of doing more trails when the
same kids are trained using both types of feedback. By doing few kids with
audio feedback and few other kids with vibratory feedback, it will provide you
with a means of comparing which one is better.

2. Follow-up data recording
It is a must for a project of this nature. But it may be too premature to do it
right now. The first thing (step-1) to do is to establish 'proof-of-concept'
(this was the thought I had when I mentioned about feasibility testing) i.e.
demonstrate that the concept will perform comparable or better than the current
practice - i.e. it measures the same variables or more that give a better
quantification of assessing the training (outcome measures) given with the
WiiCane and that it provides more detailed information pertinent to training
aspects e.g. allows more detailed measurements (what measurements), provides a
training report at the end of each walking trial (### deviations or veering,
initial posture, final posture, time taken to finish, number of times feedback
was given, difference between the initial and final measurements to reflect +ve
ot -ve change, etc)

This step will help to establish what parameters need to be measured and is the
WiiCane a more descriptive and informative training modality as compared to the
normal cane training.

Once the proof-of-concept is established then it will be easier to build a case
to show that it provides better training. Then, step-2 will be to do record
data in follow-up sessions (some kids with WiiCane and some kids with the
normal training) and seeing how long it takes to learn the skill and how well
do they learn the skill etc.

For now, these are my thoughts.

Wii Remote camera frame rate

2009-01-26T15:02:00.002-05:00

During the advisory board call, someone asked what the frame rate is for the Wii Remote. The sensor used isn't particularly well-documented, but there's some agreement that it runs around 100 frames per second, maximum (see Johnny Chung Lee et al.)

radio interview on cbs

2009-01-25T21:53:00.002-05:00

I was interviewed recently by a guy from CBS radio about WiiCane. You can hear the interview here. I'm afraid my language is somewhat imprecise and rambling at places.

steve

children, motor learning, and feedback effect

2009-01-24T20:34:00.002-05:00

I thought participants might want to look at this.

Sullivan, K. J., Kantak, S. S., & Burtner, P. A. (2008). Motor learning in children: Feedback effects on skill acquisition. Physical Therapy, 88(6), 720-732.

Discussion and Conclusions. During motor learning, children use feedback in a manner different from that of adults. To optimize motor learning, children may require longer periods of practice, with feedback reduced more gradually, compared with young adults.

Click here to go to the article: link

Guth veering chapter (warning: copyright)

2009-01-24T14:31:00.002-05:00

Hello participants. If you would like to read the chapter kindly provided by Dave Guth at WMU on veering and instruction/feedback, please click here: link

Please note that we can assume this is copyrighted material, so while I believe it is okay for us to read it, I would ask we not copy or pass the document to anyone else, and quote it in legal ways only.

Thanks,
Gene

Focus group report

2009-01-23T14:56:00.002-05:00

A focus group was held at Touch Graphics office in New York City on January 13, 2009. To read a report that documents this event, please click on either pdf format or txt format.