A few weeks ago I received an email from Harley Hamilton, a researcher at Georgia Tech's Center for Accessible Technology in Sign (CATS) asking if I'd like to collaborate on a project. Harley's team has spent years compiling the largest ASL dictionary on YouTube, SmartSign ASL Dictionary, and asked if I'd be interested in including it in Sign Search, my ASL dictionary Google Chrome extension. I actually wrote about working with CATS in my college application essay to Georgia Tech, so I was super excited about this collaboration!
When a user looks up a word in Sign Search, it sends requests to several ASL dictionary websites, parses the data it receives in response, and pulls out relevant bits of media content (gifs, videos, etc.) to show to the user. For the sites I currently work with, such as Lifeprint and Spread the Sign, that means crawling through a web page's HTML to find media. I had to take a different approach for SmartSign, because their website uses an interface that I couldn't programmatically crawl through. I asked Harley for his source code and took a look at how things worked. His code was an impressivley long series of if-statements that matched user queries to YouTube video IDs. Since a human went through and wrote all of these pairings, the search results were as accurate as possible. All I needed was a way for my extension to have access to those YouTube IDs, and we'd be good to go.
I cleaned up Harley's dictionary code so that it could stand alone as a Javascript library rather than a webpage and built a simple API with Express and Node.js to query the library. I'm hosting this API on Heroku at https://smart-sign-dictionary.herokuapp.com. It's pretty easy to use, and hopefully someone else out there will find it useful.
(Side note: the time is almost nigh for me to start paying for Heroku. I've just about reached the limits of the hobbyist tier with 5 free projects. The good news is that apps on paid tiers have no sleep time, meaning way less initial load time on my blog and Sign Search!)
Hooking Sign Search up to my API was pretty straightforward, and I was able to fetch a list of YouTube IDs that related to a search term. I could've just taken those IDs and embedded the corresponding YouTube videos, but I went a bit further and used YouTube's API to get more information before deciding to surface a video to the user. I want Sign Search's results to be narrow, not overwhelming the user with tangentially related signs. Some of SmartSign's results were synonyms or related concepts to a sign. For example: searching for 'red' gives signs for the color red, as well as communist and republican. By looking at each video's metadata I added a bit of logic to filter out these extra signs and move them to the 'related signs' section of my search results.
The updated app should hit the Google Chrome store soon. I'm close to 200 users in 20 different countries, which is pretty crazy considering I built Sign Search out of boredom in lockdown after Cornell sent students home in Spring 2020. An assistive technology specialist from a school district in Connecticuit also reached out to me about using Sign Search with their students, so I'm excited to release this new and improved version of the extension with the help of CATS.