Artificial Intelligence powered mobile sign language translator that has easily developable vocabulary.
Sesgoritma is a real time sign language vocalization application that has developable vocabulary. Sesgoritma aimed to minimize the problems of speaking and hearing impaired people, we put forward on an alternative and useful study to make them adapt to social life easier.
Sesgoritma, which perceives the words that people with speech impairments want to express and turns them into sounds, gives the feeling of speaking to the individual by working synchronously with the hand movements of the speech impaired person.
Guidebook of developable translator and user interface.
Sign to speech and text section.
|Dark Mode||Light Mode|
|Button: Switch to speech to text section|
|Button: Switch between front and back camera|
|Button: View vocabulary|
|Button: Add sign to vocabulary|
|Button: Turn on-off flashlight|
|Button: Get information|
|Warning: Hand guide|
|Warning: Low speaker level|
If translator can’t find your sing meaning, you can quickly add new image to your translator’s vocabulary with using Add Sign button.
You can use Add Sign button to add new word, letter and number to your translator’s vocabulary or you can add samples to solve problems about existing signs.
Show sign that you want to add to translator’s vocabulary to camera and tap to Add Sign button. Add Sign Sectin will appear to you and you can write meaning of taked image and add to yor sign library.
You can reach and edit your saved signs into Sign Library.
Leave blank the text filed in Add Sign Sectin if you want to add images that don’t have sign language gesture. These will appear as “No Sign” in the sign library section. “No Sign” labeled images won’t transcribed or voiced in home page.
Important Note: If you leave a space or write “No Sign” to text field for images that don’t have sign language gesture, this will prevent your translator from working efficiently. Beacuse they will be saved as a new vocabulary and processed by the translator.
You can edit your sign library here.
|Button: Sync translator with library|
|Button: Reset your translator (on one press) or library (on long press)|
|Button: Go back|
You can view samples or delete all samples of signs by a tap or long press to image. One tap to view samples, long press to delete.
Important note: If you try to delete a sign, you will see a warning alert first and if you accept the deleting there is not an undo button for this operating.
If made changes on your library and want to sync your translator with your library you must press to Sync button before continue to use Sesgoritma. When you press the Sync button, the app prevents you from returning to the home page about a few seconds. After this realy little time, your translator will have been synchronized and you will be able to continue using Sesgoritma.
Note: You don’t need to sync your translator after adding sample in Home (Sign Language Translator) section.
The reset button has two kinds of use:
1) On one press: You can reset your translator and continue to use the app with first time installation translator.
With this section, it won’t make any change on your sign library and you can sync your translator with your library and get your translator again.
2) On long press: You can hard reset your app. This will delete your entire sign library and reset your translator. So you can continue to use Sesgoritma as first installation time.
Important note: In the case of both options, you will see a warning screen first and if you accept the reset there is no going back.
You can edit your samples of selected sign library.
You can delete one sign sample with long press to image.
You can listen to environment by the reading in this section.
|Speech to Text|
|Button: Return to home page|
If you want to add new sign to your translator, try to add as many and various examples as possible for each sign you add, which will greatly affect your translator’s success.
In this case various means; light level in the environment, background, distance of the hand from the camera and its position in the frame.
If you want to get the best result while using sign language translator, you can keep your hand in center of the frame. The hand warning light on the screen will help you with this.
Your sign library data stored on your local device and is not shared with anyone, includes Sesgoritma developers.
Zeynep Dikle: Sesgoritma V1 - Project Partner - Dataset Collection Assistant
Sinan Dağ: Sesgoritma V1 - National TUBITAK Competition Supervisor
Kadir Efe Dilek: Suggestion and comments on project.
Thank to all of volunteer students and teachers from Ayrancı Anadolu High School for their help in collecting data.