A project that developed a system based on mobile devices to assist persons with visual impairment or who need more information about their surroundings has won the Internet Foundation prize for 2011.

The device enables the user to take a picture with the mobile’s camera, send it over the internet and get information about the surroundings- Martin Debattista

Judie Attard, a BSc ICT graduate at the University of Malta who is currently studying at the Digital Enterprise Research Institute in Galway, Ireland, presented the project entitled “White Cane Device” as part of her undergraduate studies.

Ms Attard was awarded for her innovative work describing or demonstrating a mobile and location-based application that profits from the use of the internet and addresses a need that is particularly relevant to the Maltese context.

Her project exploits the portability and availability of mobile technology to provide a means of guidance to visually impaired people while they are on the move or persons who need more information while on the road, such as travellers and tourists.

“I wanted to research an area which, apart from being relevant to the degree I was pursuing, would also be an area to which many people could relate to,” Ms Attard told i-Tech. “While brainstorming with my tutor Matthew Montebello, the discussion turned to problems that visually challenged people face every day. The development of an application which would help the latter people to locate themselves when in difficulty seemed a feasible research niche that satisfied both these aims.”

This system assists a visually impaired person while travelling through the city by implementing image recognition through a mobile phone and offering information of the immediate vicinity.

The “White Cane Device” enables the user to take a picture with the mobile’s in-built camera, send it over internet, and then receive back information about the surroundings. The device was implemented in a client-server architecture, where the client (mobile device) acts as a peripheral device, and the server performs the image recognition through software. The GPS coordinates, or geo-tagging, of the images are used to optimise the latter process.

“The real effort would actually lie in the building of a good database. The database should contain geo-tagged images of the various locations of interest. The larger the database, the more accurate the results would be. Such an application would not only be ideal to be used by visually challenged persons, but it could be also useful to mentally challenged persons or persons suffering from dementia. It could potentially also be tweaked for the use in the tourism industry,” added Ms Attard.

Though the project was successfully carried out and won the prize, Ms Attard admitted it presented its own challenges.

“First and foremost, the research part was very time consuming. It required finding existing related work and current state of the art for the image matching section of the system. From the latter I had to research on the techniques most ideal for my system. Another challenge was the merging of the various sections of the system, as I wanted the data to travel quickly from one process to another in such a way that the user of the system is given the results in the smallest amount of time possible. The evaluation was quite challenging as well as various aspects of the system needed to be evaluated, such as usability, accuracy and the effect of occlusion.”

A thorough evaluation of the system produced exciting results and showed that the “White Cane Device” performs reliable matching even when images contain occlusion or are taken from various perspectives. The tested system returned results within more than acceptable time spans and received positive feedback from the usability evaluations held.

Ms Attard has presented a scientific paper on her project at the IADIS Mobile Learning Conference which took place in Berlin last March.