
Getting started with computer vision using CoreML and CreateML
iOS MacOver the past several years the barrier to entry for adopting machine learning and computer vision-related features has dropped substantially. Still, many developers are...
So, you’ve heard about machine learning and Apple’s framework CoreML and want to give it a whirl. If your initial thought is that it’s too complicated and that didn’t know where to begin, don’t worry—it’s not, and I’m here to walk you through it.
But wait, why do we even need machine learning and how can it help us? Machine learning allows you to take large data sets and apply complex mathematical calculations over and over, faster and faster.
Apple has made the entry-level for machine learning quite accessible and wrapped it up in an all-in-one package. Swift, Xcode, and Playgrounds are all the tools you’re going to need to train a model, then implement it into a project. If you haven’t already, you’ll want to download Xcode. Now, let’s jump right in.
A note before we begin. This tutorial will be done in Playgrounds
 to help understand the code behind training models, however, CreateML is a great alternative to training models with no machine learning experience. It allows you to view model creation workflows in real-time.
One of the first things you’re going to need before opening Xcode is a dataset. What’s a dataset you say? It’s simple, a dataset is a collection of data, such as movie reviews, locations of dog parks, or images of flowers. For your purposes, we’re going to be using a file named appStore_description.csv
 to train your model. There are a handful of resources to find datasets, but we’re using kaggle.com, which is a list of AppStore app descriptions and the app names. We’ll use this text to help predict an app for the user based on their text input. Our model will be a TextClassifier
, which learns to associate labels with features of our input text. This could come in the form of a sentence, paragraph, or a whole document.
track_name
, you can open the csv
 file and rename that column to app_name
 so it’s consistent with this example.Now that you have your dataset you can open Xcode 🙌.
Playground
 using the macOS
 template and choose Blank
. We use macOS
 because the CreateML
 framework is not available on iOS
.Foundation
 and CreateML
.Resources
 folder.Here is what your Playground will look like and what is going on inside it:
import Foundation import CreateML //: Create a URL path to your dataset and load it into a new MLDataTable let filePath = Bundle.main.url(forResource: "appStore_description", withExtension: "csv") let data = try MLDataTable(contentsOf: filePath) //: Create two mutually exclusive, randomly divided subsets of the data table //: The trainingData will hold the larger portion of rows let (trainingData, testData) = data.randomSplit(by: 0.8) //: Create your TextClassifier model using the trainingData //: This is where the `training` happens and will take a few minutes let model = try MLTextClassifier(trainingData: trainingData, textColumn: "app_desc", labelColumn: "app_name") //: Test the performance of the model before saving it. See an example of the error report below let metrics = model.evaluation(on: testData, textColumn: "app_desc", labelColumn: "app_name") print(metrics.classificationError) let modelPath = URL(fileURLWithPath: "/Users/joshuawalsh/Desktop/AppReviewClassifier.mlmodel") try model.write(to: modelPath)
Once you have this in your Playground, manually run it to execute training and saving the model. This may take a few minutes. Something to note is that our original csv
 dataset file is 11.5 MB
 and our training and test models are both 1.3 MB
. While these are relatively small datasets, you can see that training our model drastically reduces the file size 👍.
Printing the metrics is optional when creating your model, but it’s good practice to do this before saving. You’ll get an output something like this:
Columns: actual_count integer class string missed_predicting_this integer precision float predicted_correctly integer predicted_this_incorrectly integer recall float Rows: 1569 Data: +----------------+----------------+------------------------+----------------+---------------------+ | actual_count | class | missed_predicting_this | precision | predicted_correctly | +----------------+----------------+------------------------+----------------+---------------------+ | 1 | "HOOK" | 1 | nan | 0 | | 1 | ( OFFTIME ) ...| 0 | nan | 1 | | 1 | *Solitaire* | 0 | nan | 1 | | 1 | 1+2=3 | 0 | nan | 1 | | 1 | 10 Pin Shuff...| 0 | nan | 1 | | 1 | 10 – �.. | 0 | nan | 1 | | 1 | 100 Balls | 0 | nan | 1 | | 1 | 1010! | 0 | nan | 1 | | 1 | 12 Minute At...| 0 | nan | 1 | | 1 | 20 Minutes.f...| 0 | nan | 1 | +----------------+----------------+------------------------+----------------+---------------------+ +----------------------------+----------------+ | predicted_this_incorrectly | recall | +----------------------------+----------------+ | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | | 0 | 0 | +----------------------------+----------------+ [1569 rows x 7 columns]
Now that you have your model trained and saved somewhere on your computer, you can create a new iOS
 project.
AppPredictor
, and use Storyboards.ViewController.swift
 import UIKit
, NaturalLanguage
 and CoreML
.import UIKit import NaturalLanguage import CoreML
For simplicity’s sake, your UI will have 3 elements. A text field, a label, and a button. We’re going for functionality here, but feel free to update your designs however you see fit. Next, add the text field, label, and button to your view controller in the storyboard. The text field and label will be IBOutlets
, and your button will be an IBAction
.
@IBOutlet weak var textField: UITextField! @IBOutlet weak var appNameLabel: UILabel! @IBAction func predictApp(_ sender: Any) { }
Now add a reference to your classifier like so:
private lazy var reviewClassifier: NLModel? = { // Create a custom model trained to classify or tag natural language text. // NL stands for Natual Language let model = try? NLModel(mlModel: AppReviewClassifier().model) return model }()
Let’s then create a function that takes in a string, and returns a string based on the user’s input.
private func predict(_ text: String) -> String? { reviewClassifier?.predictedLabel(for: text) }
Back in predictApp
, add the predict
function and pass the text fields text in the argument.
appNameLabel.text = predict(textField.text ?? "")
Build and run your app and let’s see what you get. Describe an app that you can’t quite remember the name of, but you know what it does. I described two similar types of apps but got different results 👍.
![]() ![]() |
![]() ![]() |
Machine learning isn’t all that scary or complicated once you break it down into digestible chunks. Finding the right dataset can often be the biggest hurdle. Now that you have the basics down you should explore some of the other classifier types, and train different models.
Over the past several years the barrier to entry for adopting machine learning and computer vision-related features has dropped substantially. Still, many developers are...
This year Apple teased a packed WWDC with over 200 sessions. Monday’s (2-hour long!) Keynote moved at breakneck speed, and the Platforms State of...
Navigation in SwiftUI is very different than it is in UIKit. This is particularly troublesome for complex applications or those that require programmatic navigation...