Overbrook School Testing Report

Introduction


I just returned from two days of testing WiiCane at Overbrook School for the Blind on Thursday, December 17 and Friday, December 18. I worked with 3 O&M instructors and 7 high school students on wiiCane. The following report documents the physical apparatus and setting, observations made during testing, and  implications of the experimental findings on our plans going forward.

Figure 1: Fitness Room with the WiiCane overhead light track installed.

Physical apparatus and setting.


We tested the system in the Fitness Room at the school's Field House (see figure 1, above). The room is approx. 40' x 24', and has an 11' high ceiling. We moved exercise equipment to the perimeter of the room, which opened up a clear floor area of 32' x 14'. The floor surface was hard vinyl tile.  The room has windows high up on one of the long sides, and also has high windows on one short end. We set up two telescoping aluminum poles 32' apart; the poles extended from floor to ceiling, and were held firmly in place by pressure. A vinyl-coated suspension cable hung between the two poles, and the WiiCane system's plastic lighting track was hung with clips from the suspension cable. Power and control wires for the 64 IR LED's, spaced out at 6" intervals along the light track, ran down the pole at the goal and of the course, and connected to the computer running a Java-based control and data collection application. The user interface for the program appears in figure 2 below.

Figure 2: WiiCane's Experimental Graphical User Interface

Using this program, a teacher or experimenter can create and manage user accounts; see a real time view of the light strip from cameras mounted on the cane and the user; watch a visualization of the user's and cane's position in real time as the system is used; control which feedback is active, and the intervals for repeating various feedbacks; and start and stop sessions.  Observed user actions that the system can respond to with corrective messages are:

• veering right
• veering left
  
• veering tolerance
• not veering or just corrected from veering
• hand not centered
• arc too wide
• arc too narrow
• arc correct
• arc tolerance

Procedure

Each participant worked with the WiiCane system  for 45 minutes on each of two days.  Sessions were scheduled by the staff, and participants did not appear at the same time each day. Their instructors stayed with the students as they used the system. Only veering feedback was given on day one, and the student was allowed to practice walking the course until they could walk 32' with no more than two corrections.
On Day Two, feedback was given for both veering and arc width. Participants walked the course an average of 15 times in each session. At the end of both days, participants tried walking the course with all feedback switched off to see if their ability to walk straight and use proper cane technique had improved after extended practice.

 Figure 3: A participant walking the course with WiiCane as her instructor looks on. 

Data Collection

The tests generated data in the following forms:

1. A continuous video was made of each session. On the first day of testing, I wore headphones to listen to the feedback as it was simultaneously playing in the wireless headphones worn by the use. This was to eliminate any possibility that the student could get directional information to help him walk straight by listening to external speakers. On day 2, we played the feedback over a speaker on the computer, so it may have been possible to use that sound as a beacon. This means that the feedback is audible on the video, which will make analysis easier. 
2. The Java program produced a log file for each session. These files can be opened directly in Microsoft Excel for statistical operations to be carried out by the Project Evaluator. The log includes all data from a session, including raw accelerometer and IR camera data and user settings and timestamped  information on each feedback delivered. 
3. Students filled out a pre-test questionnaires that collected history and status, and a post-test questionnaire that probed them about their general level of comfort using the system. These forms were collected, along with signed informed consent forms, ato be given to the Project Evaluator.

A note about videos: because of an objection from the IRB, we can't post the videos as we did for Overbrook. The person that called me explained that there is no clear rule about posting videos on You Tube, so they have decided to err on the side of caution. I offered to take it down, which she accepted.  I thought that, since the person is wearing goggles, it is hard to identify him or her, but I decided not to fight about it. We are looking for a more secure way to make these videos available for streaming. It will be much easier for Annette to do the review on You Tube site, because there are a lot of videos, and the uncompressed size is very large. We should, however, decide on one or two videos that are especially good, then get explicit permission from the student and the school and the IRB, and then make that available for public use.

Observations


The most important aspect of this test was for me to watch a different population work with the system.
As with testing at NYISE, all participants were able to use the system easily after minimal practice. There was a big range of body types this time, so I was able to see the way that the equipment fits each person. In particular, I paid attention to whether the belt-mounted unit consistently pointed up, with a good view of the light strip above. There was a big difference across subjects: some participants walked smoothly, with relatively little bounce in their step or swaying of the hips, which produced much better results and consistency in the system's ability to track position than those with a rolling gait. In one overweight subject, the belt was not long enough to go all the way around the subject's waist, and a belt extension had to be fashioned from cable ties on the spot.  This suggests that in future versions, a belt clip should be used, so that the body-mounted Wii device can be attached to the participant's own belt, instead of providing one as part of the product. We are currently working on a total redesign of that part of the system, and I will post about the new design later.

All feedback types were used except for hand-centeredness, which proved to be too inconsistent after the first few trials, so we decided to turn it off. Specifically, we were getting the message to "Center your hand" very often, even with the user's hand appeared to be centered. We did use the arc-too-wide, arc-too-narrow, and correct arc feedbacks. Surprisingly, users were mostly able to understand and follow all feedbacks (this is borne out both in the videos, where it is clear that users could correct their performance as feedbacks were delivered, and in the post-test questionnaire, where they indicated that they had little difficulty in understanding what the system was instructing them to do).

We observed that both the younger students at New York Institute for Special Education, and the older ones at Overbrook thought of WiiCane as a game where they were challenged to recieve as many "ding-dongs" (not veering) and "thump" (cane arc correct) sounds, and as few correction sounds ("go right", "narrower arc") as possible. This finding suggests that we should develop this aspect of the system further, perhaps creating a scoring system that would allow multiple users to compete, or other more complex activities that call for obstacle detection and following a route.

Figure 4: The wiiCane, the suspension pole at the start of the course, and the wood square-off block.