I will start by listing the kinds of events that will or could occur during a wiiCane session (there may be others that I have not considered):
1. Tap-this is the result of striking the cane's tip on the groud at point of maximum x displacement. this is a necessary component of two-point touch (and our system may not work if the user does not execute a tap at the completion of each swing). Tap events can also occur in mid-swing and are highly recognizeable amid potentially noisy data. In the test in Kalamazoo, we were able to synchronize the Optotrak and WiiCane data streams temporally by having the user produce three rapid taps in succession when he had finished walking. Taps are detected using accelerometer values exclusively, cane orientation is not relevant to tap detection. Due to system latency, I don't think that we can issue any feedback on taps (because the user would probably notice a lag between tapping the cane and perceiving the feedback). Also, I don't think it serves an informational purpose, since the user knows he has tapped the cane (assuming we can filter out non-tap acceleration events). But detecting and recording tap events is a crucial first step in being able to interpret other aspects of the user's technique.
2. Mid-swing. This is probably defined as the point where the cane's azimuth is co-linear with the travel path, although it could also be thought of as the point at which the cane's azimuth is co-linear with the midpoint of a line connecting the right and left tap points of a single swing (when the traveler is swinging the cane but not walking). In some cases, it may be desireable to provide vibration feedback as the cane passes the swing midpoint. This form of feedback would provide an easy-to-follow tactile path; while this kind of guidance is probably not useful in mobility training (because the real world is not equipped with this feature), it may be useful for preliminary exercises (like training wheels on a bike). In practice, creating a vibratory feedback that coincides with the point of mid-swing may be difficult (due to system latency: see discussion of Tap events, above), although there may be ways to predict that the cane is about to pass the arc midpoint, and send the feedback slightly before the event occurs.
3. Arc width. This is defined as the either the maximum angular displacement of the cane as it moves through a single sweep, or as the perpendicular displacement of the cane (from the walking path) at each tap event. Accurate measurement of arc width is essential to the system as envisioned in the proposal. In our discussions so far, there appears to be some concensus that feedback on cane arc should be provided intermittently and not after every tap. In a preliminary version, we provided verbal feedback consisting of the phrases, "too narrow", "too wide" or "good". Probably our administration utility needs to have a means of adjusting the frequency of this kind of advice. My guess is that, for adult users, spoken feedback should be provided remedially, when arc width is observed to deviate from the ideal width by some software-configurable percentage. For kids, it might be better to use a positive-sounding earcon to signal that arc width is correct, and nothing when it is incorrect. I can imagine an exercise where the child is simply asked to walk or stand still and swing the cane to produce the praise sound.
4. Hand at midpoint of body. It will be challenging to detect when the cane hand is not held at midpoint of the body. We are considering a variety of methods for tracking this; it may be, however, that the best we can do is to ask them to start at the beginning of the course with their hand held at midpoint, and we should be able to watch them execute a few stationary (non-walking) swings to make sure that we recognize the gesteral pattern associated with type of movement.
5. Wrist isolation. Similarly, it will be hard to check for wrist isolation during travel. Classic two-point touch technique calls for wrist flexion and forearm immobilization while swinging the cane. We will probably be able to detect good form (and provide confirming or admonitory feedback) during stationary swings, but when motion associated with walking is added, it may not be feasible to track wrist position using a relatively simple set up of lights and cameras like the one we are considering here.
6. Wrist rotation. One big problem we ran into during the verification exercise at Western was that we were not able to correct for wrist rotation. When the user rotated his wrist, all of the lights we were tracking moved significantly, even for a very minor rotation. While this can be compensated for in our measurement algorithms, the problem becomes much more difficult. It is not clear (to me) whether classic two-point touch teaches the traveler to avoid rotating the wrist of the cane hand. However, we need to find a sensing method that is not affected by this kind of movement, or we have to train the user to avoid wrist rotation, something that might be very hard to do with kids. We may find that this problem disappears when we switch to a different arrangement of lights. This issues is an imporant one, and I think that the accuracy of our system might be negatively affected by failure to account for this kind of motion.
8. Cane/step synchronization. While it was not one of our original measurement and feedback objectives, it would be very helpful to know whether the user's cane swings are properly synchronized with their steps. According to Zach, accelerometers mounted to the person could determine gait, but our current plan calls for mounting the wii device on the cane, not on the person, so we cannot assume that periodic motion in the z-axis (as reported by the wii's accelerometers) is related to stepping movements. It is possible to add another wii device and attach that one to the user's belt (for example), but we are trying to avoid adding additional hardware if that's possible.
9. Arc centered-ness. Another feature of classic two point touch technique is that the arc in the x-y plane that is described by the cane as it swings back and forth should be centered. That is, the line that bisects the arc should align with the travel path, and, conversely, the angular displacement of the cane to the right and left sides should be equal. it appears that this will be extremely difficult to determine conclusively, and we need to think carefully about how important this measurement is to the overall usefulness of the eventual wiiCane training system.
6. Veering. The literature indicates that everyone tends to veer in the absence of usable visual, tactile, or auditory information about the environment. Current research is considering ways to train blind pedestrians to overcome or correct for this tendency, because some mobility tasks, such as crossing wide streets, can be dangerous for blind travelers, and experts believe that if individuals could be trained to walk straight, accidents could be reduced and independence improved. One goal of the WiiCane project is to particate in this ongoing inquiry, and to produce a very practical tool for measuring veering.
Veering occurs (in my definition) when the wiiCane user's path deviates from a straight line in respect to a fixed perpendiular surface (the surface against which the user "squared-off" before starting to walk). Veering will be measured by the wii's camera, which can be trained to observe the position of infrared lights that we will array in the environment. Lights can be mounted on a variety of surfaces (including floor, ceiling, walls, the cane and the user's body), and one big job we are facing now is to study a variety of configurations to determine what works best. This is a challenging problem, because we have to be able to detect veering and to correct the user's performance before he or she has walked off to a position where they can no longer be "seen" by the wii's camera. So, our plan for providing feedback in this case is of the utmost importance if we expect to avoid the frustration of having to repeatedly return the user to the starting point because our system has lost track of their location and cannot recover. My guess is that, as with cane width, it will work best to provide spoken corrective messages when veering occurs with adults, and positive earcons when a child is successfully walking straight.
7. Distance. One important measurement that should be relatively easy to take will be distance to the goal. In the event that lights are positioned along a horizontal bar at the goal end of the course, we believe that the wii will be able to use a multi-step process, including a range-finding step (where we determine which are the widest apart lights that can be seen in a typical arc to set the gross distance) and a fine-tuning step where we look at the perceived distance between the lights. This measurement, however, can only take place when the user is more or less pointing the cane in the direction of the goal. Distance measurement will be an important component in constructing a graphical display of cane movement over time. There also may feedback opportunities associated with the student's position along the course, for example, we might want the system to speak "ten feet", "you are halfway there", or "turn left" in the event we construct a game which includes more complex route configurations.
9. Goal attained/Point awarded. Another form of feedback that we need to consider is associated with events that are not, strictly speaking, driven by cane motion. In developing a curriculum for building cane skills using wiiCane, we may want to consider various interactive activities involving game scenarios similar to SAL, which is our braille literacy system. That has a number of simple games that students and teachers find incentivizing, especially when a score for an activity is announced, and the student can practice to get a better result. We need to create a lexicon of feedback options that are stimulating and helpful for all of the populations for whom we are designing.
11. Improper usage. We know that some behaviors with the wiiCane will not be trackable. In the event the user veers significantly, there may be no way to recover location awareness, and the instructor will have to intervene. If this experience is repeated often enough, students may get discouraged and lose interest. In general, we want to promote a feeling of independence and control, and having to go back to the beginning may disincentivize further training. Therefore, we need to consider how we will use feedback to head off extreme veering before it's too late. Other researchers have set up courses that were designed to measure the onset and degree of veering; but in our case, we want to prevent veering to the point that we no longer can sense the user's position. I think feedbacks can be designed that accomplish this, but that are not overly intrusive or interfering with the student's efforts to practice walking straight.
12. Dangerous usage. In extreme cases, it will be necessary to let the user know that their movements with the cane are potentially dangerous. We have to observe a lot of people using the system before we will know all of the potentially dangerous things that could happen, and we have to be careful to carefully supervise subjects while they are using the product at all times. There are also product liablility issues that must be considered in any product where people are mobile, especially if our system calls for occluding hearing or vision. Also, in designing any curriculum for the system, we have to make sure to consider possible injuries, and discuss necessary warnings and precautions with someone knowledgeable about this (like a lawyer).
13. System alerts. Finally, we must consider that there may be a number of system warnings that we may want to include in feedback schedules. For sure, there should be some kind of "low battery" warning.
No comments:
Post a Comment