The AI4SD’s Sign Talk project is an initiative aimed at improving communication for speech and hard-of-hearing individuals in Ghana.
As part of the broader Artificial Intelligence For Sustainable Development (AI4SD) initiative, the project began with a thorough review of existing literature and community-based feasibility studies to better understand the needs of the speech and hard of hearing population in the country.
Dr. Emmanuel Ahene, the lead of the project, shared key details about the initiative’s objectives and progress.
“We spent most of last year conducting a review of literature and community-based feasibility studies to understand the scope of the problem and what interventions would best serve the speech and hard of hearing community in Ghana,” Dr. Ahene explained.
Dr. Emmanuel Ahene, AI4SD Sign Talk project lead
The AI4SD project is funded by the French Embassy in Ghana and the Agence Française de Développement (AFD) and supported by the Foreign Commonwealth and Development Office (FCDO), International Development Research Center (IDRC) and the Artificial Intelligence for Development (AI4D).
A key issue identified was the difficulty speech and hard of hearing individuals face when communicating with healthcare professionals. “We realized that many speech and hard of hearing individuals face significant challenges in communicating their health concerns to physicians.
While some are able to hire translators, it’s often very expensive, and this becomes a barrier to effective communication,” Dr. Ahene said.
AI4SD’s Sign Talk project aims to bridge this communication gap.
“Our goal is to develop an intervention that helps speech and hard of hearing individuals effectively communicate their health concerns with healthcare providers,” Dr. Ahene stated.
Photograph of the recording process
As part of the project, the team has generated 4,000 sentences focusing on interactions between healthcare workers and speech and hard of hearing individuals.
The project also incorporated existing datasets of Ghanaian Sign Language to support its work.
“We found that there are existing datasets of Ghanaian Sign Language, and we are building upon them to create a more comprehensive set of data,” Dr. Ahene noted.
To expand these datasets, five qualified sign language experts from Kwame Nkrumah University of Science and Technology (KNUST) were recruited to sign the generated sentences.
The team recording videos for Ghanaian sign language
This effort resulted in the creation of 10,000 video datasets, which have been accurately labeled and validated. “With the help of these experts, we’ve been able to generate 10,000 video datasets that are labeled and validated, ensuring their quality for future use,” Dr. Ahene explained.
Looking ahead, AI4SD’s Sign Talk project team plans to make these datasets available to the public, particularly to Ghanaian AI enthusiasts and experts, with the goal of developing AI models that can translate Ghanaian Sign Language into text or audio.
“We hope to make these datasets available for AI developers to work with, enabling the creation of systems that can translate Ghanaian Sign Language into text or audio,” Dr. Ahene said.
The project envisions an AI system that can interpret sign language from video or camera feeds, improving communication in healthcare settings.
“Our vision is to have a system where individuals can upload a video or use a camera feed to translate Ghanaian Sign Language into text or audio, making communication in healthcare more efficient,” he added.
The team is working on building an enhanced Ghanaian Sign Language translation model.
Dr. Ahene emphasized the potential impact of the project, stating, “If successful, this project will not only improve healthcare access for speech and hard of hearing individuals, but it will also contribute to greater inclusivity and help reduce mortality rates within this community in Ghana.”