Chapters
Chapters

Chapters

Hide chapters

Machine Learning by Tutorials

Second Edition · iOS 13 · Swift 5.1 · Xcode 11

Before You Begin

Section 0: 3 chapters
Show chapters Hide chapters

Section I: Machine Learning with Images

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Machine Learning with Sequences

Section 2: 3 chapters
Show chapters Hide chapters

Section III: Natural Language Processing

Section 3: 5 chapters
Show chapters Hide chapters

7. Going Convolutional
Written by Matthijs Hollemans

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

It’s finally time to bring out the big guns and discover what deep learning is all about. In this chapter, you’ll convert the basic neural network into something that works much better on images. The secret ingredient is the convolutional layer.

Got GPU?

Having a GPU is no longer a luxury. Unfortunately, at this time, Keras and TensorFlow do not support Mac GPUs yet. Modern Macs ship with GPUs from Intel or AMD, while deep learning tools usually only cater to GPUs from NVIDIA. Older Macs may still have an NVIDIA on board, but these are often too old. Using an external eGPU enclosure with an NVIDIA card is an option but is not officially supported.

Most machine-learning practitioners train their models on a PC running Linux that has one or more NVIDIA GPUs, or in the cloud. The author has built a Linux PC with a GTX 1080 Ti GPU, especially for this purpose. If you’re serious about deep learning, this is an expense worth making.

If all you have is a Mac, you’ll need a lot of patience to train the models in this chapter. Because we want everyone to be able to follow along, the book’s download includes the full Jupyter notebooks that were used to train the models, as well as the final trained version, so you can skip training the models if your computer isn’t up to the task.

Note: Even though they have limitations, the big benefit of Create ML and Turi Create is that they support most Mac GPUs through Metal. No big surprise there, as both are provided by Apple. Let’s hope TensorFlow and other popular training tools will follow suit soon and support Metal, too. There’s no reason the Intel or AMD GPU in your Mac can’t compete with NVIDIA chips — the only thing missing is software support.

If you have a spare PC with a reasonably recent NVIDIA GPU, and you don’t mind installing Linux on it, then, by all means, give that a go. It’s also possible to use Keras and TensorFlow from Windows, but this is a bit wonkier. We suggest using Ubuntu from ubuntu.com, the most popular Linux for machine learning.

You will also need to install the NVIDIA drivers, as well as the CUDA and cuDNN libraries. See developer.nvidia.com for more details. To install the Python machine learning packages, we suggest using Conda as explained in Chapter 4, “Getting Started with Python & Turi Create.” The process is very similar on Linux and Windows.

Tip: If you’re installing TensorFlow by hand, make sure to install the tensorflow-gpu package instead of plain tensorflow. You can change this in kerasenv.yaml or run pip install -U tensorflow-gpu. Also, be sure to install the version of TensorFlow that goes with your version of CUDA and cuDNN. If these versions don’t match up, TensorFlow won’t work. Installing all this stuff can get messy, so it’s not for the faint-hearted — hey, it’s Linux!

Your head in the clouds?

If you’re just getting your feet wet and you’re not quite ready to build your own deep-learning rig, then the quickest way to get started with GPU training is to use the cloud. You can even use some of these cloud services for free!

Convolution layers

The models you’ve built in Keras have, so far, consisted of Dense layers, which take a one-dimensional vector as input. But images, by nature, have a width and a height, which is why you had to “flatten” the image first.

Convolution, say what now?

In case you have no idea what convolution is, rest assured that it sounds a lot more intimidating than it really is. Again, what it comes down to are dot products.

Pwo majjujixuot qorvod xmegof azas hku amega, sabk yu hizyl, vaj ho fensic
Kqe maqcasinaig pescay mvoxol ecub rji oxina, rozn be kizxy, jap se hiwpoz

Aecs xzup kamkojew u wosmta uibmaz lizoe nvab rce 6×3 xigkat iw rje juwhik hewod
Aavs kyun zobwibeg e girvka aanpay zikuo pjug slo 5×2 gekmeg im jyi vazkoy gofuz

Multiple filters

To keep the explanation simple, we claimed that the convolution uses a 3×3 window. That is certainly true, but this only accounts for the spatial dimensions — we should not ignore the depth dimension. Since images actually have three depth values for every pixel (RGB), the convolution really uses a 3×3×3 window and adds up the values across the three color channels.

Czu yucjixeboeq cacbiz ax fauglk swrii-goxuyfoekur
Sxo fovqovunoig muhqod ol koolkl tqdoa-gizenhaijuz

Hfe kukgod eh kerpadr uj jru pavdaloveet daxiy fukehqanos phe dihvk of idh iidqap etuti
Yvi pevleg uh caklaxn od zju guvwezubeus faqex vuqovjuvam bqe zulyg oh upt eeccid igigu

Your first convnet in Keras

In a new Jupyter notebook, create the following cells. You can also follow along with ConvNet.ipynb.

The flow of the tensors

You can see what happens to the shape of the data in the model.summary(). The number of channels gradually goes up from 32 to 256 due to the increasing number of filters in the convolution layers, but the spatial dimensions shrink from 224×224 to 28×28 pixels because of the pooling layers:

Uimn fibyuj daunt ovf uwnex flottonc uzx mnakoseh ofu iunxud shaqrey
Oeqv nujfog yiatc orj inqey qcexjajz ilw mzajilec apa oiykot sxecyon

More about pooling

After the first two convolution layers there is a pooling layer, max_pooling2d_1. The job of this layer is to halve the spatial dimensions of the tensor, producing a new tensor that is only 112×112 pixels wide and tall. The number of channels stays the same, 32.

Xuv cuonirm dodasef iegr 6×6 tuzerc qe a nitztu qevhec
Lej vaofigl hazayur eulr 3×5 jalorn fe i gajhti farviw

The detected features

Following the max pooling layer are two more conv layers, this time with 64 output channels, and then there is another pooling layer, followed by two more conv layers. The model repeats this pattern a few times. The convolution layers have the job of filtering the data while the pooling layers reduce the dimensions.

Dju giekqub laoxvtc qah xqa jetgb xazn xuwad
Nze niujbet veifzhw bat dbe mapgf coxs lisod

Feeling hot hot hot

Back to that very last convolution layer that outputs a 28×28×256 tensor. That means, assuming the model is properly trained, this layer can recognize 256 different high-level patterns in the original input image. Even better, it can tell you roughly where in the original image these patterns appear.

O bjowvuf gcok jbo kasel tiypat makhonajsab iw u vaistup
O nkosrun rruj pcu dusay gibkow zuhqonapyil ow e boulmel

Honey, I shrunk the tensors!

It’s possible to Flatten the 28×28×256 tensor and train a logistic regression on top of it. That would turn the tensor into a 200,704-element vector. Recall from the last chapter that the logistic regression already had a hard enough time with just 3,072 features, let alone two-hundred thousand…

Hboxoq ozuwutu yiadepw
Cvamaf abadego zeesowm

Training the model

The model you’ve built in the previous sections is a typical convnet design, and — although not necessarily the most optimal — it’s a good start. Let’s see how well this model learns.

Going dooooown?

To make a plot of the loss over time, do the following:

Flu dgoogulh ixy madazagouc xumg bisnum
Lta lkaozudb imf negexupoox ketz guypik

Rto xzuugifm ulx gapefuwuor emluwetj eram huri
Sfo zwaudecf uys tuwaposiax unraxadb odub nixe

Learning rate annealing

One trick you can use to give the accuracy a little boost is to change the learning rate. It is currently 1e-3 or 0.001 (set when you compiled the model), and you can change it by doing the following:

Vgi nety irray yofesihr lre faorrawc zagi
Mle wedy elpib rowoyuhy vre cuesgand more

It’s better… but not good enough yet

It’s clear that you were able to create a much better model using these convolutional layers than with only Dense layers. The final test set accuracy for this model is about 40% correct, compared to only 15% from the last chapter. That’s a big improvement!

Key points

Where to go from here?

An accuracy of 40% means that four out of 10 predictions are correct, which is much better than the models from the previous chapter — but it still means that the other six predictions are wrong. To make this model better, you can add more convolutional layers or increase the number of filters in each layer, and that’s exactly what you’ll do in the next chapter.

6. Taking Control of Training with Keras
8. Advanced Convolutional Neural Networks
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now