The film Million Dollar Baby, by Clint Eastwood, was striking for Luís Paulo Reis. On the night he saw the film in 2006, the professor of the Faculty of Engineering of the University of Porto (FEUP) dreamt that, similarly to the protagonist of the story played by Hillary Swank, he also became paralysed. In the following day, he had an idea: how could we apply our vast robotics experience to a project that allows us to increase the autonomy of quadriplegics and people suffering from cerebral palsy or other incapacitating problems?

“I thought that it would be interesting to create an intelligent wheelchair that could not only be driven safely (avoiding obstacles or falling down the stairs, for instance), but also simple to control, even for users with reduced capacities,” explains the researcher who is the current president of the Portuguese Robotics Society.

With this goal in mind, he joined a group of professors from the Artificial Intelligence and Computer Science Laboratory at FEUP and from the Institute for Systems and Computer Engineering of Porto (INESC TEC). These were later joined by four new partners: the University of Aveiro, the University of Minho, the Escola Superior de Tecnologia de Saúde do Porto (health college in Porto, Portugal) and the Porto Association of Cerebral Palsy. The team have developed the IntellWheels, a platform that makes it possible to turn a conventional electrical wheelchair into intelligent equipment capable of avoiding obstacles and of being driven using a joystick, voice controls or facial expressions.

During the first stage, and without any funding, the researchers used a wheelchair that was offered to them – “it was practically junk,” according to Luís Paulo Reis – to do the first tests on the concept. After that, the project grew as it received support from various entities.

Although there are now dozens of projects for intelligent wheelchairs being developed all over the world, the IntellWheels has some distinguishing features. The most important is the multimodal control interface, which adapts to the abilities of each user. When the user sits on the chair for the first time, a series of tests (user profiling) are made where the goal is to verify the capacities of each user and define the command that best adapts to their features. The chair can be controlled using a common USB joystick or voice controls defined during the first use, captured by a microphone and interpreted by a processor capable of recognising and ignoring surrounding noise.

DRIVING WITH THE BLINK OF AN EYE

Using a camera and a WiiMote motion sensor, developed for Wii, the IntellWheels can also be moved by simple head movements, with the blink of an eye, facial expressions or simple thoughts. Furthermore, the different types of controls can either be used separately or combined. “For instance, for a quadriplegic user we can define that blinking once means stopping, blinking twice is moving forwards and blinking twice means turning left, and so on. However, if the users are capable, they can also use more efficient and precise controls, such as the joystick,” Luís Paulo Reis explains.

The team has already tested a brain-computer interface using existing technology, which consists of a helmet that makes it possible to ‘read’ cerebral waves: “The user can train thoughts which are linked to the chair controls. For instance, if I think of water the chair will go; if I think of fire the chair will stop.”

However, this technology was put aside for now because the device’s precision in detecting thoughts was too low, which makes it impossible to move the chair efficiently and safely. During the initial experiments, none of the volunteers managed, after two hours of practice, to form a well-defined cognitive thought that was useful to guide the IntellWheels, and for that reason the interface was not used in the final experiments.

Another advantage of the IntellWheels is the fact that the system can be installed in practically every common wheelchair. “Unlike many intelligent wheelchairs, which are designed from scratch and look kind of robotic and scary, the IntellWheels maintains the look of a conventional wheelchair,” Luís Paulo Reis stresses. “For that reason, the IntellWheels is a low-cost system because it uses non-intrusive devices that are affordable.”

Besides the different controls, the chair can automatically avoid obstacles. Using a sensor bar that is placed around the chair and a Kinect device, the equipment can capture images and calculate distances from objects, at a 360-degree angle.

The same technology can be applied in domotics

Task planning is another asset of the IntellWeels, especially in environments which are familiar to the users of the chair, such as their homes or institutions where they spend most of their time. At the Porto Association of Cerebral Palsy, where most of the tests were conducted, the research team did a map of the facilities and from there they defined automatic actions for the chair, from basic orders like “follow the wall,” to more complex actions such as “go to the cafeteria.” “The chair knows its location, as well as the location of the cafeteria, and so it can plan the path from the starting point until the desired destination,” Luís Paulo Reis explains.

CHAIRS THAT COMMUNICATE WITH EACH OTHER

The IntellWheels can also communicate with other chairs. “When the system (in this case the chair) shares the environment with other systems capable of communicating, it can interact with them and define a cooperative plan,” the researcher explains. In other words, it is possible to program the chairs so that they don’t run into each other (avoiding collisions, for example) or, on the other hand, so that they meet at a certain location. Similarly, the chair can communicate with other devices, such as automatic doors, allowing them to open as the chair passes.

Another important feature is the 3D simulator that the Portuguese team created, which is very realistic and reproduces the real environment of the Porto Association of Cerebral Palsy. Other than the features of the association’s headquarters, the simulator includes a faithful copy of the chair itself, of its sensors, engines and dynamic behaviour, and it uses precisely the same software as the real IntellWheels. It was thanks to this virtual reproduction that various children and adults suffering from cerebral palsy were able to try a wheelchair for the first time. “It was very emotional for some people who have never had this opportunity because there are no chairs adapted to patients with cerebral palsy. In order to train them and assess their performance controlling the chair, it is necessary to know exactly where the chair is at all times, and that would be very complicated in a real environment,” Luís Paulo Reis explains.

After building the prototype, which has received five national and international awards, there is still much work ahead until IntellWheels can be commercialised. For now, the team are looking for funding to start designing a product that is more economically feasible to put on the market. According to Luís Paulo Reis, the project may go ahead with Portuguese and Brazilian funding in order to reach a vaster market.

In the meantime, some of the principles that have led to the multimodal interface of the IntellWheels may be applied in other projects focusing on people with physical limitations. Using a multimodal interface, boccio players, for instance, will be able to control the ramp used in this Paralympic sport, allowing them to play autonomously. Similar interfaces can also be applied in areas such as domotics, making it possible to control the environment in the house with different electronic devices and intelligent furniture.

Super Interessante, April 2014