How I built Defects AI a machine learning service to predict GitHub labels.

Defects AI is a fully automated service to predict labels for GitHub issues, using machine learning models that are trained in particular to that use case. Don’t waste your time on labeling issues, use it to solve them instead.

Update: I took the service down, it will now rest in peace as my first unsuccessfull side project. I decided to spend my time on my main project

Automatically labeled issue

Defects AI began as a side project for a statistics based tool that provides insights about product quality. While digging deep into different data-sets which were based on varied GitHub repositories, I notice that many of the issues were not labeled.

After a couple of days and talks I realized why: For many projects, it is difficult to analyze issues in an acceptable amount of time.

That was the turning point, I paused working on the statistics tool and started working on the automated issue labeling for GitHub. Starting with the core feature, the machine learning model to predict the GitHub issue type.

A few first successful tests and further improvements of the machine learning model gave me the confidence to continue with that idea.

The setup is simple, register at Defects AI, install the GitHub Defects AI App and select the repositories which you want to enable for the automatic issue labeling.

Each time when a new issue is created, GitHub sends a request to our server and as a response, we send the predicted issue back to your repository. The predicted label is then set automatically on the issue.

Automatic Issue Labeling in action
  • Reduced the features to the bare minimum
  • Kept my weekends for my family and relaxing
  • I usually worked 5 days a week on Defects AI, each day at least 1 hour before my daily job starts.
    (You are still responsible to deliver great results to your employer and it is up to you to manage both. If you aren’t capable of combining both, please don’t risk your job)

I spent a bit more than 3 months (95 Days) to build the initial release of Defects AI. To be exact I started on the 15–06–2018. Here is the first commit in the repository:

While collecting feedback from my user I continue working on Insights and improving the models further. Feel free to follow me on Twitter.

Supported currently are the labels: Bug, Feature, Documentation, and Question.

Like everything else also a SaaS costs money, the biggest expenses are the training costs for the models, hosting and collaboration tools. I spend around 12$ for each training, as you can imagine in the beginning I trained a lot.

I plan to publish my financial reports each month.

Defects AI provides also a simple and easy way to experiment with the prediction. Just follow the link and start predicting the labels of issues:

❤ Thank you for having you here. Please feel free to leave any comments if you have questions about building a SaaS.
I appreciate your feedback which helps me to get inspired for my next posts.


Hi, I am Sascha, AI/ML Expert at @doitint and Founder of ioannotator an AI Platform.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store