The AI-powered device will help speech therapists determine which preschoolers need speech therapy and why. The device connected to the app will facilitate diagnosis, provide guidance to parents and enable remote consultations between specialists: speech therapists, orthodontists or physiotherapists.
Equipped with the device, children will pronounce words and sentences, and the app will indicate if any irregularities are seen or heard.
The reason for creating a team working on a device and software for recording and analyzing sound and images for speech therapy was the personal motivation of a young scientist, currently the head of the NCBR LIDER program. The mother of Michał Kręcichwost, then a student of biomedical engineering at the Silesian University of Technology in Gliwice, suffered a stroke. The result was aphasia, or the loss of the ability to speak. “For our family, it was a completely new and difficult experience. We did not know what aphasia was or how to start rehabilitation,” recalls the researcher.
See alsoHow artificial intelligence will affect the work of accountants
Dr. Eng. Michał Kręcichwost decided to help – first adult patients with aphasia, and then children who need support in speech development. Currently, his team combines several technologies and creates the SpeechCAD system, which is to support speech therapists throughout Poland.
Earlier, together with engineers Zuzanna Miodońska and Krzysztof Mrozowski, he created the web application “Afast! Mówi to”, supporting the rehabilitation of people with aphasia. As the scientist explains, it works like a virtual rehabilitator, filling the time between visits to a specialist. The platform is available for free and offers speech therapy exercises. According to the authors, the project carried out as part of the Social Innovations program was a success, and the application still helps people with speech disorders.
“Studies involving older people with aphasia, however, proved to be a demanding task – each of them struggles with slightly different problems related not only to articulation, but also to comprehension. Collecting the appropriate database necessary to develop algorithms was quite a challenge. Therefore, in further studies we focused on preschool children, who were easier to access,” explains Dr. Kręcichwost.
As part of his doctorate, the scientist built a device for recording a spatial acoustic signal (speech sound from multiple microphones) and a deep model for detecting incorrect pronunciation of the sz sound. At the same time, Dr. Eng. Miodońska, based on the data recorded by this device, developed methods for classifying different realizations of the sz sounds. Their methods indicate which sounds the child pronounces correctly and which require correction.
Dr. Eng. Michał Kręcichwost (Silesian University of Technology)
The team then created a device equipped with two cameras that record facial images, which allows for the generation of a 3D image. Work began on the objective assessment of speech movements. The device created as part of the LIDER program, called by scientists a “4D tool,” records not only sound, but also the image of the face and changes over time, creating a 3D model of the speaker.
“The data will be processed by an advanced form of artificial intelligence. The system will analyze the movements of the articulators and pronunciation, and the results will be presented in the application in the form of a 3D model of the speaker over time. Thanks to this, the speech therapist will be able to show the parents the progress of the therapy – how the child's pronunciation is changing. Such a model will also be used to share with various specialists in order to deepen the diagnosis. Then the speech therapist will be able to simply send the data record in the form of a 4D model to other specialists, e.g. an orthodontist,” emphasizes Dr. Kręcichwost.
The LIDER project involves the cooperation of an interdisciplinary team. It consists of three biomedical engineers specializing in image and sound processing, four speech therapists from different regions of Poland (this is related to different articulation and different approaches to diagnosis in individual speech therapy centers). The team also includes a programmer who will supervise the programming work.
“Artificial intelligence must be fed with data to know what to indicate to speech therapists. Algorithms must be trained on large data sets to recognize subtle differences in pronunciation or facial movements during articulation. It is speech therapists who provide the knowledge that allows teaching the system how to support therapy,” the scientist explains.
After three years, the LIDER project is to be completed with the production of a prototype ready for implementation of the SpeechCAD system. “This tool will not replace a speech therapist, but it will help to objectify the diagnostic process and speed it up. The system will, among other things, indicate which children require a detailed diagnosis, which is extremely important in kindergartens, where speech therapists often have limited time for screening tests. In addition, it will allow for the recording of the test in a repeatable manner,” indicated Dr. Kręcichwost.
The main recipients of the device will be kindergartens, psychological and pedagogical clinics and speech therapy offices. The researcher adds that the device does not require complicated certification, like medical products, which will make it easier to introduce it to the market. The system will be secured in accordance with medical data protection standards, so that the recordings and images of children are safe.
Although the project is based on advanced technology, Dr. Kręcichwost emphasizes that the most important thing is to help people in an accessible way. “In the beginning there was Afast, which still supports people with aphasia. Now we want to help the youngest in kindergartens, to take proper care of them, saving time for speech therapists and making their work easier,” the researcher sums up.
Karolina Duszczyk (PAP)
col/ bar/ ktl/
