AWS Rekognition, Comprehend, and Transcribe

For the Metal Toad 2020 Hackathon, our teams were tasked with creating a fully functional video platform using AWS Media Services. Team Babu Frik (Tony Rost, Julie Scott, Brent Logan, and Ron Shafii) walk us through their project in the interview below.

What was the challenge/problem your team wanted to solve?

Babu Frik, an Anzellan droidsmith with a talent for bypassing droid security systems, erased C3PO's memory.  He felt very sad about it.  He decided to use AWS Media Services and Rekognition to reprogram C3PO with knowledge about his friends.  

What was your initial idea that you wanted to build?

We built a secured React portal with the following features:

• A C3PO Reprogramming Station to upload and analyze video of major Star Wars characters (Rekognition/Transcribe/Comprehend)
• A Babu Frik Reminder Station to teach Babu about major characters.
• A library of memories and results that is searchable with ElasticSearch
• Authentication with AWS Cognito

What were the top 1-2 obstacles you encountered?

• Chewbacca was identified a ‘beard’ for several rounds of training.  We eventually got him properly recognized.
• We had a couple issues with MediaConvert via the CLI and eventually used the AWS Console instead.

What were the top 1-2 wins/successes you had while working on the project?

• We learned into AWS CloudFormation to help configure AWS Steps — there are a lot of processes to orchestrate and this step was pivotal.
• The React portal, with the results from Rekognition/Transcribe/Comprehend, was visually fun and thematic.

What end product did you ultimately build?

Exactly what we set out to build!

What were the top 3 things you learned?

• AWS Step Functions are powerful — and can get overwhelming.  The temptation to add ‘wait’ calls is strong.
• AWS has a strong open source library of snippets to speed up development of APIs and JS code.
• Chewbacca is more than just a beard — he has a rich inner world fueled by powerful feelings.