thoughts about the upcoming staff meeting and the future

In preparing for our meeting, which is scheduled for Wednesday morning, December 30 at 10 AM, we have to cover a lot of ground. We are at a critical moment for the project and have to make some rapid decisions that will shape the final product and our activities over the next five months. Here are the general categories of things that I want to talk about.

• The results of the testing at overbrook. 
• Videos
• Questionnaires
• Data logs 
• Review of current system design, focusing on:
• Feedback behaviors
• User interface issues
• Belt holder ergonomics and optics
• WiiCane weight, length and balance
• Physical installation
• Data logging
• Next tests
• Need to be more empirical and hypothesis driven, less usability focused.
• Incorporating vibratory feedback for deaf-blind users
• Maybe adding a belt unit wtih two additional wii remotes for directional signalling
• Scheduling for Guild and HKNC tests
• WiiCane Curriculum
• Purpose and overview of the wiiCane curriculum
• working method
• schedule
• Commercialization
• Publications
• Conferences
• Marketing
• IP

There's a great deal to discuss and plan for, and I am really optimistic about this project right now. While we have yet to experimentally prove that WiiCane has therapeutic benefits, I think that we need to go full speed to bring this to market. the low cost means that people will want it even if it is only fun and not proven (yet) to be beneficial. the main thing that I learned from two rounds of testing we have done thus far is that kids enjoy doing it, if it is presented to them as a game, and it seems like we can make it challenging enough to sustain their interest. So, of course we need to make it truly educational, and not just a toy, especially we are expected to be carrying out evidence-based research, but even this preliminary observation appears positive.

Design for WiiCane pole at the end of the walking course

Here is a proposal or how to organize the WiiCane system components to make it easier to keep track of everything. In this picture, you can see that I mounted an all-in-one touch screen PC on an adjustable height tension pole, and added a horizontal aluminum rack a few inches above the screen. The rack makes a convenient place to hang the cane, the headphones, the blindfold and the belt unit.  The USB controller for the light track is mounted out of sight on the back of the PC, since there is no user access required to that component. Configuring and launching wiiCane will be via a basic graphical user interface with a few big buttons on the PC screen. Instructors and students will operate the system with speech assistance, and we will supply a transparent tactile graphic overlay sheet to be mounted over the screen, so that users can feel the locations of buttons.

In cases where the wiiCane track is being fastented to the ceiling, and there is no pole, the rack for the wiiCane accoutrements can be mounted to the wall adjacent to the walking course.

response to gene's summary of existing literature on cane training

Gene, this is a great start. It is very helpful to see a complete list of the categories of cane skills that O&M teaches. Now that we know in general the kinds of movements that our apparatus can measure, we need to create a curriculum for the WiiCane so that instructors and students can maximize their results. This curriculum should  provide instructions for using WiiCane in a linear, cumulative way, to promote rapid acquisition of skills and habits needed for safe and efficient travel with a long cane. It should present training activities as a series of games, where students are challenged to improve their performance by mastering skills of escalating difficulty.

In looking at the list of skills customarily included in O&M training (according to the text books you have consulted), it appears that some skills will be quite easy to teach using our system, while some others could very hard, with the rest probably impossible or at least impractical to achieve in this 24 month project. In sorting through these, I think we can add program modules to accomplish some things right away. For example, it would be easy to add features that measure pivoting movements, with feedback played in the headphones as the user's orientation changes. In our curriculum, we could add an activity where the user is walking along under the light track, and as he approaches the end of the course, an instruction is played in the headphones to stop and turn 180 degrees.  We would then turn off veering feedback and just look at rotation via the belt-mounted camera. We could provide a short pulse (either accoustical or vibratory) when the student has reversed direction. This behavior and feedback scenario could then be built into a "lap swimming" task (for advanced students), where they would have to practice going back and forth on course, turning at each end with the help of this rotational feedback. This is one example of how skills taught in standard cane training could be integrated into exercises carried out with WiiCane. The challenge will be to organize these exercises in some linear fashion to create a curriculum which addresses the needs of each user population.

Developing a WiiCane curriculum

Developing a WiiCane curriculum; the biomechanics and other aspects of travel that can be detected and that can be influenced by automated feedback.

I reviewed the two classic texts below to compile this list as the beginning of developing an official curriculum for developing and using the WiiCane.

Skills so far tested:
- veering reduction
- cane arc width and coverage, stationary
- cane arc width and coverage, ambulating


Skills that might be developed, orientation:
- proprioception, rotating by degrees: e.g. turning 90 degrees, turning 180 degrees
- using compass directions
- using lateral directs
- interpreting kinesthetic feedback
- consistent forward movement
- adjusting line of travel from a shoreline with two adjustments
- identifying object materials

Skills that might be developed, cane articulation:
- two-point touch technique – arc apex height
- two-point touch technique – in-step
- two-point touch technique – in rhythm
- touch and slide technique
- touch and drag technique
- shortened touch technique
- three-point touch technique
- diagonal cane position
- hand centeredness (mid-sagittal plane)
- wrist role


Hill, E., & Ponder, P. (1976). Orientation and Mobility Techniques: A Guide for the Practitioner. New York, NY: American Foundation for the Blind.

Jacobson, W. H. (1993). The Art and Science of Teaching Orientation and Mobility to Persons With Visual Impairments New York, NY: American Foundation for the Blind.

Overbrook School Testing Report

Introduction


I just returned from two days of testing WiiCane at Overbrook School for the Blind on Thursday, December 17 and Friday, December 18. I worked with 3 O&M instructors and 7 high school students on wiiCane. The following report documents the physical apparatus and setting, observations made during testing, and  implications of the experimental findings on our plans going forward.

Figure 1: Fitness Room with the WiiCane overhead light track installed.

Physical apparatus and setting.


We tested the system in the Fitness Room at the school's Field House (see figure 1, above). The room is approx. 40' x 24', and has an 11' high ceiling. We moved exercise equipment to the perimeter of the room, which opened up a clear floor area of 32' x 14'. The floor surface was hard vinyl tile.  The room has windows high up on one of the long sides, and also has high windows on one short end. We set up two telescoping aluminum poles 32' apart; the poles extended from floor to ceiling, and were held firmly in place by pressure. A vinyl-coated suspension cable hung between the two poles, and the WiiCane system's plastic lighting track was hung with clips from the suspension cable. Power and control wires for the 64 IR LED's, spaced out at 6" intervals along the light track, ran down the pole at the goal and of the course, and connected to the computer running a Java-based control and data collection application. The user interface for the program appears in figure 2 below.

Figure 2: WiiCane's Experimental Graphical User Interface

Using this program, a teacher or experimenter can create and manage user accounts; see a real time view of the light strip from cameras mounted on the cane and the user; watch a visualization of the user's and cane's position in real time as the system is used; control which feedback is active, and the intervals for repeating various feedbacks; and start and stop sessions.  Observed user actions that the system can respond to with corrective messages are:

• veering right
• veering left
  
• veering tolerance
• not veering or just corrected from veering
• hand not centered
• arc too wide
• arc too narrow
• arc correct
• arc tolerance

Procedure

Each participant worked with the WiiCane system  for 45 minutes on each of two days.  Sessions were scheduled by the staff, and participants did not appear at the same time each day. Their instructors stayed with the students as they used the system. Only veering feedback was given on day one, and the student was allowed to practice walking the course until they could walk 32' with no more than two corrections.
On Day Two, feedback was given for both veering and arc width. Participants walked the course an average of 15 times in each session. At the end of both days, participants tried walking the course with all feedback switched off to see if their ability to walk straight and use proper cane technique had improved after extended practice.

 Figure 3: A participant walking the course with WiiCane as her instructor looks on. 

Data Collection

The tests generated data in the following forms:

1. A continuous video was made of each session. On the first day of testing, I wore headphones to listen to the feedback as it was simultaneously playing in the wireless headphones worn by the use. This was to eliminate any possibility that the student could get directional information to help him walk straight by listening to external speakers. On day 2, we played the feedback over a speaker on the computer, so it may have been possible to use that sound as a beacon. This means that the feedback is audible on the video, which will make analysis easier. 
2. The Java program produced a log file for each session. These files can be opened directly in Microsoft Excel for statistical operations to be carried out by the Project Evaluator. The log includes all data from a session, including raw accelerometer and IR camera data and user settings and timestamped  information on each feedback delivered. 
3. Students filled out a pre-test questionnaires that collected history and status, and a post-test questionnaire that probed them about their general level of comfort using the system. These forms were collected, along with signed informed consent forms, ato be given to the Project Evaluator.

A note about videos: because of an objection from the IRB, we can't post the videos as we did for Overbrook. The person that called me explained that there is no clear rule about posting videos on You Tube, so they have decided to err on the side of caution. I offered to take it down, which she accepted.  I thought that, since the person is wearing goggles, it is hard to identify him or her, but I decided not to fight about it. We are looking for a more secure way to make these videos available for streaming. It will be much easier for Annette to do the review on You Tube site, because there are a lot of videos, and the uncompressed size is very large. We should, however, decide on one or two videos that are especially good, then get explicit permission from the student and the school and the IRB, and then make that available for public use.

Observations


The most important aspect of this test was for me to watch a different population work with the system.
As with testing at NYISE, all participants were able to use the system easily after minimal practice. There was a big range of body types this time, so I was able to see the way that the equipment fits each person. In particular, I paid attention to whether the belt-mounted unit consistently pointed up, with a good view of the light strip above. There was a big difference across subjects: some participants walked smoothly, with relatively little bounce in their step or swaying of the hips, which produced much better results and consistency in the system's ability to track position than those with a rolling gait. In one overweight subject, the belt was not long enough to go all the way around the subject's waist, and a belt extension had to be fashioned from cable ties on the spot.  This suggests that in future versions, a belt clip should be used, so that the body-mounted Wii device can be attached to the participant's own belt, instead of providing one as part of the product. We are currently working on a total redesign of that part of the system, and I will post about the new design later.

All feedback types were used except for hand-centeredness, which proved to be too inconsistent after the first few trials, so we decided to turn it off. Specifically, we were getting the message to "Center your hand" very often, even with the user's hand appeared to be centered. We did use the arc-too-wide, arc-too-narrow, and correct arc feedbacks. Surprisingly, users were mostly able to understand and follow all feedbacks (this is borne out both in the videos, where it is clear that users could correct their performance as feedbacks were delivered, and in the post-test questionnaire, where they indicated that they had little difficulty in understanding what the system was instructing them to do).

We observed that both the younger students at New York Institute for Special Education, and the older ones at Overbrook thought of WiiCane as a game where they were challenged to recieve as many "ding-dongs" (not veering) and "thump" (cane arc correct) sounds, and as few correction sounds ("go right", "narrower arc") as possible. This finding suggests that we should develop this aspect of the system further, perhaps creating a scoring system that would allow multiple users to compete, or other more complex activities that call for obstacle detection and following a route.

Figure 4: The wiiCane, the suspension pole at the start of the course, and the wood square-off block.

vtesting at Overbrook

Testing OVERBROOK
Wednesday 12/16/2009 set up
Thursday 12/17/2009 8 AM – 2:30 PM
Friday 12/8/2009 8 AM – 2:00 PM

This procedure is proposed for testing at Overbrook School for the Blind. Steve may need to modify these procedures depending on local conditions.

Overbrook staff are scheduling students for 45 minute periods, based on availability. They will have the consents signed and the pre-questionnaires completed.

Christine will send the student names (by Wednesday) so we can set up the ID numbers.

I met with Jackie and Judy, and three O&M specialist, Christine, John, and Kim.

I met four (ood the 8) students who were available. They were great and I got to explain the entire project to them.

Procedure for trials.
45 minute sessions. Thirty minutes of veering and 15 minutes of cane coverage.

Veering:

1. On Thursday staff will check that the pre-trial survey has been completed.

2. The staff will measure the height of the belt-mounted wii remote and input the values in the program user interface.

3. For the Thursday initial trials for veering, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff. Note: the staff will emphasize that we are testing the cane, not the subject.

4. Subject will be shown the belt-attaching wii remote. The remote band will be put around the subject’s waist and adjusted to fit comfortably. The remote angle will adjusted to point vertical toward the infrared light strip. If subject is wearing loose clothing, the staff will use clips to secure the clothing so it does not interfere with the wii remote. If the subject’s hair is blocking the remote camera, it will be adjusted by the subject and/or the staff.

5. Subject will be shown the wiiCane, the imbedded remote, and the shaft. Subject will demonstrate how they move the cane left and right in constant-contact techniuqe; any wrist rolling behavior will be noted and the subject will be shown how to move the cane without rolling their wrist. (This action is taken now in anticipation of the coverage trials.)

6. Subject (with or without vision) will be shown the MINDFOLD blindfold. The subject will put on the blindfold with a tissue protecting their face and assisted to adjust them to fit comfortably.

7. Subject will put on the wireless headphones and assisted to adjust them to fit comfortably.

8. Subject will walk, guided, the 32 foot-long straight-line course at least once while the procedure is reviewed again.

9. Data collect will be turned on; feedback will be turned off. The subject will walk the course three times to collect pre-feedback data on veering, without the feedback turned on. This should be videotaped and data should be collected.

10. Subject will have the feedback system explained, including the meaning of the feedback terms (terms like “move left”) and feedback chimes.

11. The subjects will be shown the proper amount of adjustment to the left or right when they receive feedback.

12. Subject will be asked if they are ready to try the course.

13. With ascent, the subject will walk the course at least once, with staff behind them, as they listen and respond to the feedback. The staff will advise the subject of errors in responses to feedback. The subject will be encouraged to continue walking forward when he/she hears feedback. The subject will be prompted to move to the left or right in response to feedback. This may be repeated until the subject appears confident and can travel the course alone.

14. Data recording will begin. The subject will walk the course. At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

15. Subject will walk the course repeatedly until the 30 minutes expire.

16. On the second/final day of testing, the subject will walk the course three final times to collect post-feedback data on veering, without the feedback turned on. This should be videotaped and the data collected.

17. Subject will be told the time of his/her next appointment, if any.

Cane coverage:

1. For the initial trials on Thursday for coverage, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff.

2. Subject will have the feedback system explained, including the meaning of the feedback terms and feedback sounds. Terms like “wider on the right” will be explained.

3. The subject will be shown how to produce an appropriate arc width/coverage by moving their cane laterally between the feet of the staff or mobility specialist, who will stand facing the student with feet placed to demarcate the proper distance. If necessary the subject will practice the arc width movements for a short period of time.

4. Data recoding and feedback will begin, and the subject will remain stationary, moving the cane left/right and adjusting the movements according to the feedback. This may continue through several short practice intervals of just moving the cane and adjusting it to follow the feedback while not walking forward.

5. Subject will be asked if they are ready to try the cane on the course.

6. The subject will walk the course. (We anticipate the veering feedback will not typically be turned on for these trials, so staff or specialist may need to manually, from behind the subject, adjust the subject’s line of travel to remain on the course and under the infrared lighting strip.) At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

7. Subject will walk the course repeatedly until the 15 minutes expire.

8. The subject will complete the post-testing questionnaire. (Note, if additional personnel are not available, the above schedule will need to be adjusted to included time – approximately 5 minutes – for the post questionnaire.)

wiicane user interface: tactile overlay for touch screen monitor

I think that when we should develop a simplified UI to allow audio tactile control of the wiicane controls. There should be a user control screen that is designed to be used in conjunction with a clear tactile overlay.  buttons displayed on the monitor would be also outlined in raised lines that delineate controls to be used in conjunction with speech prompts. the monitor would be mounted on the wall or table at the beginning of the course, and users could initiate sessions and change settings in a fully accessible way. I am picturing groups of blind kids playing with this at a training center, like the Colorado Center for the Blind that I just visited in Littleton. They could keep score, restart a game, and adjust settings as they trained themselves while playing various speed and accuracy challenges. the monitor touch screen could also serve as a big slap panel that a person would hit at the end of one lap. Then, to permit people "swimming" back and forth, we would just have to make another button at the other end and mount that to the pole or to the wall, too. Maybe we can extend the wire out of the light strip and down the wall for the second button and use the control wire from the lights strip also for the button? Zach?

Wiicane Test Video

All Videos are deleted from Youtube.

NYISE Day 3 testing

NYISE Day 3 testing

The last day of testing was quite eventful!

Steve set up the equipment with relatively little difficulty.

We had the same eight subjects as day two. All the subjects were now familiar with the veering trials and the wii cane (day one they used their own canes), which was introduced day two. Every subject provided us with more veering trials. Most of the trials went really well, and it really seems like there were positive changes with less veering than the first day. Several times we asked subjects to walk the course with the feedback. Watching some of the subjects correct their line of travel WITHOUT THE FEEDBACK is amazing - it seemed magical with a few of them. (The data will tell!) The two subjects that had troubles on day one continued to have some difficulties on this third day. This has to do with ‘overcorrection’ and some other behaviors. When the system asked them to “go left” or “go right,” instead of making the minor adjustment with a small body rotation in the correct direction, these two subjects would make a large body rotation, in effect turning up to 90 degrees and sometimes walking perpendicular to the course. At such a point it is difficult for the subject to re-orient no matter what the feedback says. We attempted to change the feedback to say “go a little left/right” but the software would not accept the new text. Instead, it spoke the standard “go left/right” in the automated voice.

This is something to discuss and solve. The two subjects that became disoriented on the course both had cognitive involvement, and one was our youngest subject. Both were repeatedly coached repeatedly on how to adjust “just a little” before trials. They needed:
- prompts to respond to some of the feedback
- prompts when they did not move forward along the course
- physical manipulation to be reoriented to the course when they overcorrected

Can we design feedback for younger and more cognitively involved students that will address these situations? It might be that more and repeated preparation would be effective, showing the student how to adjust until they “get it” before the actual training. We simply did not have time to do this during the testing.

Right now we cannot tell where on the course the subject is located – we cannot yet track forward movement. When we can, we can certainly easily prompt students to keep moving forward when they stop.

We might need physical barriers to keep younger students on the course, not allowing them to turn 90 degree and walk off. We might try various types of feedback.

We also began to collect data on the cane arc width with most of the subjects (I believe 6 out of eight). I talked to each subject about why they would want to make their cane movements more narrow or more wide. I them let them move their canes left and right between my feet, showing them a correct width. We them had them move the cane back and forth, without walking, with the system feedback. The feedback said either wide or narrow (meaning too wide or narrow) or gave a single chime for a correct arc. Several of the kids were able to achieve repeated chimes after a few trials. For those who did well with stationary trails, we asked them to use the feedback when walking. For the most skilled subject, we gave her both veering feedback (with the positive double chime turned off after she told us it was too much) and cane arc width feedback. For this subject it actually seemed to work well!

The staff at NYISE was fantastic. The children subjects were all well-behaved. I conducted the post-testing surveys, which were also videotaped. Interestingly, not a single subject said the weight of the wiiCane was a problem, and I think 7 or 8 out of eight said that they wanted to spend more time with the wiiCane.

Finally -- dealing with some issues from the previous days. We checked with the kids about their “hearing the wall” – we were concerned that perhaps there was some echolocation happening. It turns out the subject that reported she could hear the wall was actually talking about circumstances when she is not wearing headphone and there is environmental noise to bounce off the walls. The other issue --- detecting the lights on the ceiling through the opaque goggles is still uncertain. We used a MINDFOLD sleepshade for all the subjects on Friday – the kids loved it because it is very padded and comfortable; but, we still may have little kids who are afraid of the dark and the goggles will may still be a better alternative in those situations.

Proposed evaluation plan for WiiCane data

Below is an outline of a plan for the analysis of our WiiCane data. Our data collection instruments are: a pre-questionnaire asking for participant characteristics (sex, age, grade level, visual impairment, prior mobility training); data collected by the WiiCane equipment on veering and coverage performance in response to feedback; and a post-questionnaire that allows participants to describe in their own words how they liked working with the device.

(1) Descriptive statistics for all of our pre-questionnaire demographic items (sex, age, grade, visual impairment, mobility training experience, etc.)

(2) Descriptive statistics for number of trials for veering and for coverage. Correlations of number of trials with subject characteristics, if they occur (e.g. do participants with prior mobility training, or longer history of visual impairment, or different age levels, do different numbers of trials in the same time period).

(3) The learning trend over successive trials will be plotted to see if it is linear, if it shows a sudden change (indicating that the participant suddenly "gets" what to do), and the rate of learning.

(4) Since number of trials is expected to vary with subjects, we will focus on the change from the first trial to the last trial for each subject as the measure of total amount of progress. For veering, we hypothesize that the average deviation from the center will be smaller in the last trial than in the first and the number of corrective feedbacks triggered will also be smaller in the last trial. For coverage, we hypothesize that the number of corrective feedbacks will be smaller in the last trial than in the first. We can also look at whether there is a bias toward too wide vs. too narrow.

If we look at these alone, essentially we have a series of paired t-tests. We can also build in one or two factors such as age, number of years of prior training, or blind vs. deaf-blind. In that case we will have an analysis of variance.

(5) Qualitative analysis of post-questionnaire comments and recommendations for improvement.

Day 3 at New York Institute of Special Ed

 I am expecting a lot to happen as we go through the next six weeks, and there will be a great deal of interaction between me, Gene, Zach, Ellen and Grace to get it all done. Some other tasks that need to get underway, and that will involve various members of the project staff are:

• Design, fabricate and test a new version of the belt holder. We learned a lot about how the user's hair and clothes have to be shifted in order to get good, continuous view of the light strip above. In general, we ran into trouble with hoodies, sweaters, vests, bushy or otherwise bulky hair. Gene and I will work Grace to improve the design and then reprint it in time for testing at Jewish Guild for the Blind and HKNC in January.
• make sure that that the lights are being evenly powered along the length of the overhead strip. You could clearly see, in the on-screen display, that the lights were much steadier when the user was standing near the end of the light track where power and control wires connect.  Zach has to look into that and make some improvements to the electronics of the controller.
• Add visual indicators on screen to show when feedback is playing.  It's really hard to know what the student is hearing by reading the messages sequentially, especially since things could occur at the same time.
• Solve the problem of not being able to have an instructor listen to the stereo mix  that the student is hearing through his own headphones or through a separate speaker.   Someone in the world must have confronted this before with Bluetooth, I am hoping. 
• Begin to develop a strategy for producing a curriculum to go along with the system.  Gene and I talked about the three existing texts on cane instruction that are in use, and he will post a discussion comparing these soon, so that as a group we can arrive at a consensus about how to teach using WiiCane. It will also be good to reintroduce Bonnie and Dona to the discussion as we enter into this part. 

Wow, that's a lot of things to do.  We have barely enough time for everything before the end of the project, but I think that if everyone makes steady progress on their parts, we will do it. I feel like people are going to love this product, because it is oddly fun to do, which is what all of the users agreed on, so we need to do this and make a new product to show how small business in America can work to make niche high tech products like this. 

responses from subjects - day 1

Annette, here are the responses from the subjects on day one. Let me know if you have questions.

8:45 – I001jk
Stereo/binaural feedback/headphones not working
Did not wear goggle
Response to trials – very positive and cooperative

9:25 – I002am
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative

10:05 I003tr
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – cooperated during first few trials, responded to feedback prompts with overcorrecting body rotations. After several trials he was not moving forward or responding to feedback, asked if he wanted to stop he said yes and we terminated. He seems fine but perhaps overwhelmed

10:45 I004gd
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – walked very slow; responded to feedback prompts with overcorrecting body rotations. Given constant prompts from the administrating staff she would respond appropriately, but not to the generated feedback alone. She was shy, very soft spoken . It was difficult to know if she enjoyed the trials.


12:00 I005av
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative

12:40 I006la
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative. She is a dancer, and from the first trial to the last she walked almost a perfect line.


1:20 I007dc
Used stereo feedback system with left/right differentiate feedback
Used small goggles
Response to trials – very positive and cooperative

NYISE Testing Day No. 2

video from yesterday's test

The day got off to a bad start, because I was in charge of setting up the system and running the computer instead of Zach.  I spent 90 minutes fighting with the Bluetooth connection, with copious tech support from Zach. Ultimately, it turned out that we had to use the same USB jack that we used yesterday, along with the same Bluetooth "dongle" (the external USB plug that you have to use to add Bluetooth radio service to a computer that doesn't have it built-in).  Once I got both wii devices (cane and belt), as well as the bluetooth headphones all connected, everything went very smoothly; but, there is no question that this aspect of the wiiCane system is the most problemmatic right now, and we need to make connecting to all three devices very easy and foolproof (since it is likely that people operating the system will be fools, like me).  I was able to catch up with the schedule, and we finished on time. We had one new student today and seven that were there yesterday.

In general, the purpose of today's testing was to see whether user ability to follow the audio prompts during travel showed measurable improvement after a night's rest. I asked them to continue to walk repeatedly along the course in one direction only. Their instructors helped them to return to the starting point each time. The biggest difference was that I asked them to use the wiiCane this time instead of their own canes. Even though no feedback was provided based on their cane movements, this gave them a chance to become familiar with the new cane, which is heavier then a normal cane, due to the additional weight of the wii device. Since the cane had a roller tip, none of them reported that the cane was too heavy. I also asked them to pay attention to   maintaining their index finger of their cane hand extended along the flat face of the grip.  I explained why this is important, and how the wii device had to be pointing up towards the ceiling to work properly.  In most cases, I observed that they did hold the cane correctly, with a minimum of wrist roll. I don't know for sure whether this will work once we turn on the feedback for wrist roll, but to me it appeared that they can learn to hold their wrist straight with minimal training.

In terms of veering, I noticed some improvement, but since many of the kids were quite good at walking straight (with the help of our system's audio prompts ) yesterday, there was not a marked improvement among the more adept travelers.  Two students who had significant trouble mastering the system yesterday did much better today, including the youngest subject, who is 7 and very little, so that suggests that the learning curve for using wiiCane is not very significant.  In each test today, I asked the student to walk the course at least 10 times (or until they could do it flawlessly), then I asked them to do it a few more times with the feedback turned off. The results of this were highly variable. One girl seemed to do better with the feedback off, but the others veered substantially and never recovered. The implications of this are unclear and further discussion is needed to be sure of what is happening. It may be that when Annette reviews the videos along with the events logs from the tests, it might become clearer, or after we do more testing at the other three sites.

Tomorrow, Gene and I will go back to the Institute for the final day of testing there. We will be having them do cane movement tasks for the first time, so this will be important for us to see.  I will post information about that tomorrow.

Comments on today's testing at NYISE

Testing went very well today, I think. We are learning a lot from watching these kids, who have been incredibly nice and cooperative.  It will be easy to get what we need from them, and I believe that we will be able to return later and see how they like the finished product. That's doubly true if we leave the finished system up there for them to use; it will be a great PR thing, maybe we can get news crews up there watching them play with it, if we make it enough like a game. It will be a really cool addition to their rec room. From speaking with the staff, it looks like there are a bunch of additional behaviors they would like us to train students on, including maintaining a low arc, etc. Once we have the horizontal movement of the cane under control, we should move into other kinds of motion capture. 

Based on what I saw today, I believe that we are justified in continuing to develop this idea as a commercial product. We need to build a very open-ended framework in this early version of the software so that it will be easy to add subsequent modules. We also have to start to prepare our wiiCane curricula that will go along with the system. Gene and I spoke briefly today about that part of the project. I need to look at the schedule for the final five months of the project.  We have to evaluate at the results of the tests and incorporate findings, leading up to a rapid product rollout, hopefully. I seriously think that we can begin selling these system at the end of Phase 2 (June 1, 2010).  Grace is starting to do research on a better material and electrical couplings for the light track, and we also need to redesign the belt, but those things can realistically get done quickly. I know it is just the first day of testing, but considering the ease with which most of the students took to it, I am convinced that the basic principles are sound, so we should proceed full blast.

As you can see, I am very upbeat about today, and I think we should be very encouraged about the usefulness and practicality of wiiCane. I also just had an idea for my fencing application: There should be an enormous projection screen at the end of the course where we can display images from a ceiling mounted projector about  15 feet away.  There could be huge image on the wall that you would face while sword fighting with a virtual opponent...

I will send a report on events tomorrow. I spoke to Zach about an hour ago, and he said that he has made the changes he needed to, including an on screen display of messages as they are played in the headphones. It looks like we will be ready to do cane movement tasks on Friday. I think that we should maybe have them just do a stationary cane movement task over and over like Gene and I discussed. That will allow us to really study their hand movements to see how serious the weight issue will be.  BTW, the roller tips arrived today. They are perfect. I think they will neutralize the weight issue, as long as they are doing it on a nice smooth floor. the roller tip just slides over the pencil tip, so it looks like any slide on tip will work. 

steve

REVISE: NYISE procedures and schedule

Schedule for testing NYISE
December 8 (Tuesday) through December 11 (Friday)
Daily 8:30 AM-2:30PM

Tuesday
Complete setup of all equipment
Testing of equipment
Run through blindfolded TGI staff on course
O&M specialist should prepare kids, perhaps show them the route and explain what will happen


The first trial for each subject, the subject will be have the experiment explained
The testers may use whatever appropriate language is necessary, and take as much time as they feel is reasonable for the subject to understand and be comfortable.
Subjects will be asked if they are comfortable with the headphones and the goggles.
Verbal and non-verbal response will indicate whether the subject should continue. Subjects will then be guided once down the course, shown the end of the course and the safety obstacle.
Subjects will then be walked/guided back to the start of the course.
Subjects will always complete trials for veering before trials for coverage.

I am suggesting we test based on time, not number of trials. There is no way I can think of to stay on a schedule when we might see great variability with each child. Doing as many trials as possible during a prescribed period makes sense. Annette, to you see a study design problem with this?

Pre- and post-testing questionnaires will be done by TG and NYISE personnel outside the subjects’ timeslot for the scheduled trials.

I do not really care the order of subjects. It should not be relevant. Whatever works best for the institute should be fine. If we do have a preference at all, we might want to start with the best cane users.


Day 1: Veering trials only. Each subject should complete as many trials as possible during the 45 minute period.
Wednesday

Time Subject Veering
8:45 I00n
9:25 I00n
10:05 I00n
10:45 I00n
11:20 I00n
12:00 LUNCH
12:30 I00n
1:10 I00n
1:50 I00n


Day 2: Veering only. Each subject should complete as many trials as possible during the 45 minute period.
Thursday

Time Subject Veering
8:45 I00n
9:25 I00n
10:05 I00n
10:45 I00n
11:20 I00n
12:00 LUNCH
12:30 I00n
1:10 I00n
1:50 I00n



Day 3: Cane coverage. Each subject should complete as many trials as possible during the 45 minute period.
Friday

Time Subject Veering
8:45 I00n
9:25 I00n
10:05 I00n
10:45 I00n
11:20 I00n
12:00 LUNCH
12:30 I00n
1:10 I00n
1:50 I00n


Below is the draft of the detailed procedure suggested by Steve to be used at NYISE. I completed the veering procedure first -- and pending feedback, I will create the others. Your comments are necessary.


Procedure for initial trials: veering

1. Staff will check that the pre-trial survey has been completed.

2. The staff will measure the widest area of the body, select the cane length, and input the values in the program interface.

3. For the initial trials for veering, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff. Note: the staff will emphasize that we are testing the cane, not the subject.

4. Subject will be shown the waist-attaching wii remote. The remote band will be put around the subject’s waist and adjusted to fit comfortably. The remote angle will adjusted to point toward the infrared light strip. If subject is wearing loose clothing, the staff will use clips to secure the clothing does not interfere with the wii remote.

5. The staff will check the indicator for the remote vertical alignment, and make necessary adjustments.

6. Subject will be shown the wiiCane, the imbedded remote, and the shaft. Subject will demonstrate how they move the cane left and right; any wrist rolling behavior will be noted and the subject will be shown how to move the cane without rolling their wrist. (This action is taken now in anticipation of the coverage trials.)

7. Subject (with or without vision) will be shown the customized occluding goggles. The subject will put the goggles on and assisted to adjust them to fit comfortably.

8. Subject will put on the wireless headphones and assisted to adjust them to fit comfortably.

9. Subject will walk, guided, the 30 foot-long straight-line course several times while the procedure is reviewed again.

10. Subject will have the feedback system explained, including the meaning of the feedback terms and feedback chimes.

11. Subject will be asked if they are ready to try the course.

12. With ascent, the subject will walk the course several times, with staff behind them, as they listen and respond to the feedback. The staff will advise the subject of errors in responses to feedback. The subject will be encouraged to continue walking forward when he/she hears feedback. The subject will be prompted to move to the left or right in response to feedback. This will be repeated until the subject appears confident and can travel the course alone.

13. Data recording will begin. The subject will walk the course. At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

14. Subject will walk the course repeatedly until the 40 minutes expire.

15. Subject will be told the time of his/her next appointment.

Procedure - draft

Below is the draft of the detailed procedure suggested by Steve to be used at NYISE. I completed the veering procedure first -- and pending feedback, I will create the others. Your comments are necessary.


Procedure for initial trials: veering

1. Staff will check that the pre-trial survey has been completed

2. For the initial trials for veering, the subject will have the goals and procedures of the exercise explained in appropriate language by the staff.

3. Subject will be shown the waist-attaching wii remote. The remote band will be put around the subject’s waist and adjusted to fit comfortably. The remote angle will adjusted to point toward the infrared light strip.

4. Subject will be shown the wiiCane, the imbedded remote, and the shaft. Subject will demonstrate how they move the cane left and right; any wrist rolling behavior will be noted and the subject will be shown how to move the cane with rolling their wrist. (This action is taken now in anticipation of the coverage trials.)

5. Subject (with or without vision) will be shown the customized occluding goggles. The subject will put the goggles on and assisted to adjust them to fit comfortably.

6. Subject will put on the wireless headphones and assisted to adjust them to fit comfortably.

7. Subject will walk, guided, the 30 foot-long straight-line course several times while the procedure is reviewed again.

8. Subject will have the feedback system explained, and the meaning of the feedback terms and chimes.

9. Subject will be asked if they are ready to try the course.

10. With ascent, the subject will walk the course several times, with staff behind them, as they listen and respond to the feedback. The staff will advise the subject of errors in responses to feedback. The subject will be encouraged to continue walking forward when he/she hears feedback. The subject will be prompted to move to the left or right in response to feedback. This will be repeated until the subject appears confident and can travel the course alone.

11. Data recording will begin. The subject will walk the course. At the end of the course, the subject will be guided back to the beginning of the course. Staff will ensure that the student is comfortable and will provide encouragement.

12. Subject will walk the course repeatedly until the 45 minutes expire.

13. Subject will be told the time of his/her next appointment.

behold: wiiCane 2009

Introduction

Here is the final WiiCane design that will be used in the upcoming user trials. The first trials will occur on Wednesday, December 9 at New York Institute for Special Ed in the Bronx and will continue through the rest of December and into January at a total of four sites.  This post is to describe the physical wiiCane apparatus as of December 6, including:

• The wiiCane itself
• The belt-mount unit
• The light track and associated fittings

This post will provide a generic description of  each component of the physical system for project documentation. This will also serve as a first public disclosure of the wiiCane system prior to filing for a provisional patent, followed by a full patent filing should test findings indicate that a commercial product based on this technology can achieves the aims set out in the project proposal.  To provide a complete picture of our activities and results, other project participants will produce posts over the next couple of months describing our software and hardware approaches and components, as well as discussions about how the apparatus can be used for therapeutic, educational and entertainment purposes.  A final project report will provide a public disclosure of raw data and evaluation results from the upcoming user tests.

A note about intellectual property: this project has been funded by a grant from the United States Department of Education's Steppingstones of Technology program. See the sidebar on the right side of the screen for additional information. Touch Graphics' stated goal in pursuing this project is to bring one or more products to market based on technology developed here. Since the company is subject to competitive pressures, we have chosen to obscure some aspects of our design and omit certain details in descriptions of structures and mechanisms, fittings, electronics, and software.  But we hope to contribute to the scientific discourse in fields related to orientation and mobility, motion capture and analysis, human computer interaction, and product design and so are committed to publishing our findings in ways that other researchers find helpful and informative. In particular, this work seeks to validate and extend work of David Guth and Robert Wall Emerson at Western Michican University, and Bruce Blasch at the Atlanta VA (see future posts for elaboration). This disclaimer will hopefully clarify our reasoning for leaving some things out of the following discussion.

WiiCane
The central physical component of the system is the modified cane itself. As of now, the cane is identical to an ordinary mobility cane, except that it includes a fixture that permits the wii device to be held and positioned as needed for our purposes.  The wiiCane includes a telescoping section so that its length can be adjusted for the individual user. This way, a single wiiCane can be used by many students. Maximum length is 59" and minimum length is 42".  The wii device is held in a two-part gimbaled lightweight cradle that allows the device to rotate 360 degrees and then be fixed in position, so the wii device's camera can be pointed straight up or straight down.  The handle of the wiiCane is flat on one side so that the the user can easily determine the correct orientation to hold it. When the cane is held in the right hand, the flattened part of the handle points to the right, and the wii device points straight up so that its camera can "see" the light strip above. When switching to a left-handed user, the wii device must be rotated 180 degrees in the cradle.  the wiiCane is quite heavy compared to an ordinary mobility cane, but it is well-balanced and easy to hold.  The cane can accommodate any of the standard tips, including marshmallow and roller, by unscrewing the standard pencil tip and replacing it with a new one. The wii device can be easily removed from the cradle for replacing the batteries.

Belt unit

Because we need to track both user and cane position during use, we added a second wii device mounted on the user's body. To hold it in place at the right angle, we created a simple belt that goes around the user's waist. The wii device is held in a cradle against the user's back, so that the camera points up at the light strip.  Because its important that the camera in the belt-mounted wii device maintain continuous awareness of as many lights on the light strip as possible, we added adjustments that allow us to control the unit's tilt back and away from the user's body and potentially billowing clothing. We also may need to add clips to their hair or other garments that could block the camera's view. The belt is a standard item made of nylon webbing with a quick-release plastic buckle. The cradle can also be attached to the user's own belt.

The light track

The third component of the wiiCane system is the light track. This is a 32' long modular construction that acts as an upside down landing strip: as users walk along under the light track, cameras inside the cane- and belt-mounted wii devices track LED light sources spaced regularly along its length and relay their positions via bluetooth radio to a stationary computer. A Java program evaluates the raw data returned from the cameras and accelerometers on the wii devices, determines the most efficacious feedback and provides (hopefully) helpful corrective messages via wireless headphones and/or vibrating the cane's grip.

The project includes design of a suspension systems for the light track.  Depending on existing conditions at the training site, the light track can be mounted in a variety of ways. Using custom made plastic clips, the track sections can be screwed to the ceiling, to an overhead pipe or other linear element, or to a free standing cable structure supported on lightweight aluminum tension poles that we provide. The light track includes IR LED's at 6" intervals along its length, for a total of 64 lights.   The light track is made up of 8  4-foot long sections that are held up to the suspension cable, ceiling or exposed pipe using plastic clips.A single CAT 5 cable emerges from the end of the track, and it connects to a serial controller that switches the lights on and off under computer control.

Plastic clips for mounting the lighting track

scheduling for NYISE - draft

Schedule for testing NYISE
December 8 (Tuesday) through December 11 (Friday)
Daily 8:30 PM

Tuesday
Complete setup of all equipment
Testing of equipment
Run through blindfolded TGI staff on course
O&M specialist should prepare kids, perhaps show them the route and explain what will happen


The first trial for each subject, the subject will be have the experiment explained
The testers may use whatever appropriate language is necessary, and take as much time as they feel is reasonable for the subject to understand and be comfortable.
Subjects will be asked if they are comfortable with the headphones and the goggles.
Verbal and non-verbal response will indicate whether the subject should continue. Subjects will then be guided once down the course, shown the end of the course and the safety obstacle.
Subjects will then be walked/guided back to the start of the course.
Subjects will always complete trials for veering before trials for coverage.

I am suggesting we test based on time, not number of trials. There is no way I can think of to stay on a schedule when we might see great variability with each child. Doing as many trials as possible during a prescribed period makes sense. Annette, to you see a study design problem with this?

The tables do not appear below correctly on the blog.

Day 1: Veering trials only. Each subject should complete as many trials as possible during the 45 minute period.

Wednesday

Time Subject Veering
9:00 I001
9:45 I002
10:30 I003
11:15 I004
12:00 Lunch
12:45 I005
1:00 I006
1:45 I007
2:30 I008


Day 2: Veering trials for 15 minutes, followed by cane coverage trials

Thursday

Time Subject Veering Coverage Total
9:00 I008
9:45 I007
10:30 I006
11:15 I005
12:00 Lunch
12:45 I004
1:00 I003
1:45 I002
2:30 I001


Day 3: Veering trials for 20 minutes, followed by cane coverage trials for 20 minutes

Friday

Time Subject Veering Coverage Total
9:00 I001
9:45 I002
10:30 I003
11:15 I004
12:00 Lunch
12:45 I005
1:00 I006
1:45 I007
2:30 I008

IRB Approval

As the project evaluator, I'm very happy to announce that the WiiCane project received official IRB (Institutional Review Board) approval on November 24. This means that we are able to move forward to collect data on adult participants and on children age 6 and above. IRB approval is necessary for all research using human participants, to certify that the research meets ethical standards for human subjects testing. Our data collection will begin on December 8.

After we have collected data on children age 6 and above, we will refine our procedure for testing children ages 3-5 and submit our preliminary findings to the IRB for approval to test in that age group.