Sesgoritma

Artificial Intelligence powered mobile sign language translator that has easily developable vocabulary.


Contents:


What is Sesgoritma?

Sesgoritma is a real time sign language vocalization application that has developable vocabulary. Sesgoritma aimed to minimize the problems of speaking and hearing impaired people, we put forward on an alternative and useful study to make them adapt to social life easier.
Sesgoritma, which perceives the words that people with speech impairments want to express and turns them into sounds, gives the feeling of speaking to the individual by working synchronously with the hand movements of the speech impaired person.


How to use?

Guidebook of developable translator and user interface.

Home - Sign Language Translator

Sign to speech and text section.
Dark Mode Light Mode


Symbols Explanations
Button: Switch to speech to text section
Button: Switch between front and back camera
Button: View vocabulary
Button: Add sign to vocabulary
Button: Turn on-off flashlight
Button: Get information
Warning: Hand guide
Warning: Low speaker level


If translator can’t find your sing meaning, you can quickly add new image to your translator’s vocabulary with using Add Sign button.
You can use Add Sign button to add new word, letter and number to your translator’s vocabulary or you can add samples to solve problems about existing signs.
Show sign that you want to add to translator’s vocabulary to camera and tap to Add Sign button. Add Sign Sectin will appear to you and you can write meaning of taked image and add to yor sign library.
You can reach and edit your saved signs into Sign Library.
Leave blank the text filed in Add Sign Sectin if you want to add images that don’t have sign language gesture. These will appear as “No Sign” in the sign library section. “No Sign” labeled images won’t transcribed or voiced in home page.
Important Note: If you leave a space or write “No Sign” to text field for images that don’t have sign language gesture, this will prevent your translator from working efficiently. Beacuse they will be saved as a new vocabulary and processed by the translator.

Library Section

You can edit your sign library here.
Sign Library


Symbols Explanations
Button: Sync translator with library
Button: Reset your translator (on one press) or library (on long press)
Button: Go back


View or edit sign:

You can view samples or delete all samples of signs by a tap or long press to image. One tap to view samples, long press to delete.
Important note: If you try to delete a sign, you will see a warning alert first and if you accept the deleting there is not an undo button for this operating.

Sync Button:

If made changes on your library and want to sync your translator with your library you must press to Sync button before continue to use Sesgoritma. When you press the Sync button, the app prevents you from returning to the home page about a few seconds. After this realy little time, your translator will have been synchronized and you will be able to continue using Sesgoritma.
Note: You don’t need to sync your translator after adding sample in Home (Sign Language Translator) section.

Reset Button:

The reset button has two kinds of use: 1) On one press: You can reset your translator and continue to use the app with first time installation translator.
With this section, it won’t make any change on your sign library and you can sync your translator with your library and get your translator again.
2) On long press: You can hard reset your app. This will delete your entire sign library and reset your translator. So you can continue to use Sesgoritma as first installation time.

Important note: In the case of both options, you will see a warning screen first and if you accept the reset there is no going back.


Sign Samples Section

You can edit your samples of selected sign library.
Sign Samples


You can delete one sign sample with long press to image.


Speech to Text Section

You can listen to environment by the reading in this section.
Speech to Text


Symbols Explanations
Button: Return to home page


Tricks for using Sesgoritma efficiently

1) Sample size and variation:

If you want to add new sign to your translator, try to add as many and various examples as possible for each sign you add, which will greatly affect your translator’s success.
In this case various means; light level in the environment, background, distance of the hand from the camera and its position in the frame.

2) Hand position:

If you want to get the best result while using sign language translator, you can keep your hand in center of the frame. The hand warning light on the screen will help you with this.


Open Source and Publishes

Publications:

Published Dataset:

Artificial Intelligence:

Mobile Application:


Awards

About Developer

All rights of Sesgoritma belong to Arda Mavi.
Arda Mavi’s
LinkedIn
GitHub
Twitter
Blog

Contact Arda Mavi by sending an e-mail.

Privacy Policy

Your sign library data stored on your local device and is not shared with anyone, includes Sesgoritma developers.
View Privacy Policy


Special Thanks

Zeynep Dikle: Sesgoritma V1 - Project Partner - Dataset Collection Assistant,
Zümra Uğur : Contributions to the research on speech-impaired people,
Sinan Dağ: Sesgoritma V1 - National TUBITAK Competition Supervisor,
Kadir Efe Dilek: Suggestion and comments on project.
Thank to all of the volunteer students and teachers from Ayrancı Anadolu High School, Ankara, Turkey for their help in collecting data.

App Store

The Sesgoritma app remained publicly available on the App Store between 2018 and 2022.
Demo video of the app can be watched here: Sign language to voice translation demo.
App Store Previews:



Copyright © 2017-2022 Arda Mavi All rights reserved.