Frequently Asked Questions about AircraftML

If you have been following the CloudGeeks blog, you have probably heard me talk a lot about AircraftML, whether I’m using it to compare managed computer vision services or show you how to build an image classification system from the ground up. So I wanted to answer a few questions about this bot.

What is AircraftML?

@AircraftML is a Twitter bot designed to classify aircraft found in images using a machine learning backend. For example, AircraftML can successfully classify many modern commercial aircraft like the Boeing 747 and the Airbus A380.

How do I use AircraftML?

You can interact with AircraftML on Twitter by mentioning AircraftML and attaching an image of an aircraft to your tweet, like this:

The actual text in the request doesn’t matter, so long as you ‘mention’ @AircraftML.

AircraftML will respond to your tweet and tell you what type(s) of aircraft are found in the image (if any):

How does AircraftML work?

The AircraftML system is built on Amazon Web Services and includes the use of services like API Gateway, AWS Lambda, and Amazon SQS. The AircraftML Twitter bot is considered a serverless application.

Here’s an overview of the architecture of AircraftML and how it currently works (subject to change, of course):

1. Twitter

AircraftML is a Twitter bot with an AWS machine learning backend. Users send requests and receive responses using tweets. The Twitter Account Activity API is used to deliver events to AircraftML.

2. Request Webhook - API Gateway

API Gateway provides a webhook URL for Twitter. Two methods are defined: GET requests are used by Twitter to do periodic checks; POST requests are used by Twitter to deliver tweets. API Gateway uses a Lambda Proxy integration to forward requests to the Request Dispatcher.

3. Request Dispatcher - AWS Lambda

The Request Dispatcher Lambda function processes events delivered by Twitter via the webhook. This function parses events delivered by Twitter, filters out non-tweet events, and then queues up individual tweets to the SQS Request Queue.

4. Request Queue - Amazon SQS

An SQS queue is used to store Twitter events and to decouple delivery and processing of events. A dead letter queue is used for debugging events that fail to be processed. The Request Handler function is triggered when events are placed in this queue by the Request Dispatcher.

5. Request Handler - AWS Lambda

The Request Processor Lambda function coordinates aircraft classification and is triggered when a tweet event is placed in the Request Queue. This function validates incoming tweets, coordinates the processing of incoming images, and assembles and delivers responses via Twitter.

6. Aircraft Detector - AWS Lambda

The Aircraft Detector Lambda function examines images to determine if there are any aircraft in the image and, if so, returns a bounding box for each aircraft found. Aircraft Detector is implemented as a Single Shot Detector built with the MxNet framework.

7. Aircraft Classifier - AWS Lambda

The Aircraft Classifier function classifies each aircraft detected and returns a list of possible aircraft and confidence levels. The model was built using the MxNet machine learning framework and has been trained to recognize many common commercial aircraft.

8. Tail Number Detector - AWS Lambda

The Tail Number Detector Lambda function looks for aircraft tail numbers in each of the detected aircraft. It uses Amazon Rekognition to find text and then determines if the text is a tail number. It currently recognizes tail numbers from the US, UK, Canada, Mexico, and France.

9. Model Trainer - Amazon EC2

The Model Trainer is one or more EC2 instances used to build and train the deep-learning models used by both the Aircraft Detector and Classifier. These instances are used only when training a new model (at most once per month) and usually less than 3 hours. Spot instances are purchased to minimize cost.

10. Model Store - Amazon S3

Amazon S3 is used to store all models, training material, and temporary files used during processing. Lifecycle policies are used to ensure temporary files are deleted in a timely fashion.

11. Text Detector - Amazon Rekognition

Amazon Rekognition is used by the Tail Number Detector Function to identify text in images.

12. Secret Store - AWS Secrets Manager

The Secret Store is used to store Twitter API keys and secrets. Both the Request Dispatcher and the Request Handler functions retrieve Twitter secrets from Secrets Manager before interacting with the Twitter APIs.

13. Permissions - AWS IAM

All AWS services, applications, and functions use AWS IAM roles to ensure they have the appropriate (and minimum necessary) permissions to accomplish their assigned tasks. No more, no less.

Why doesn’t AircraftML use service X?

Wondering why AircraftML doesn’t run on a different service? It probably did, or maybe it will. AircraftML isn’t intended to be a reference architecture—but more a platform to experiment with many services.

I originally built AircraftML using Amazon SageMaker, which worked extremely well. But now the app is based in Lambda, not because Lambda is better suited for this project, but simply because I wanted to find out how much AI/ML I could fit into a Lambda framework. (Turns out—all of it.)

AircraftML will probably all be container-based in a future version. ;)