Chapters
Chapters

Chapters

Hide chapters

Apple Augmented Reality by Tutorials

Second Edition · iOS 15 · Swift 5.4 · Xcode 13

Before You Begin

Section 0: 5 chapters
Show chapters Hide chapters

Section I: Reality Composer

Section 1: 5 chapters
Show chapters Hide chapters

Section II: Reality Composer & Image Tracking

Section 2: 2 chapters
Show chapters Hide chapters

Section III: Reality Composer & Object Tracking

Section 3: 1 chapter
Show chapters Hide chapters

Section IV: RealityKit & Face Tracking

Section 4: 3 chapters
Show chapters Hide chapters

Section V: ARKit & SpriteKit

Section 5: 3 chapters
Show chapters Hide chapters

Section VI: ARKit & SceneKit

Section 6: 2 chapters
Show chapters Hide chapters

Section VII: ECS & Collaborative Experiences (Bonus Section)

Section 7: 1 chapter
Show chapters Hide chapters

Section VIII: Creating 3D Models With Object Capture (Bonus Section))

Section 8: 2 chapters
Show chapters Hide chapters

17. ECS & Collaborative Experiences
Written by Chris Language

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

In the early days of ARKit, it quickly became apparent that something important was missing: the ability to share augmented reality experiences among multiple users.

Later versions of ARKit addressed the issue by introducing ARWorldMap. The map contains a space-mapping state along with a set of anchors from a world-tracking AR session. This map can be shared, allowing multiple users to experience persistent AR anchors within the same space.

With the assistance of a peer-to-peer network, multiple users can share an ARWorldMap in real time, creating a collaborative experience. Using ARKit, the process is somewhat painful, requiring vast amounts of manual labor from a coding perspective.

Apple created a fantastic ARKit example project that you can explore. Find the project here: Creating a Multiuser AR Experience

However, since iOS 13, you’ve been able to pair RealityKit with ARKit to automate most of the manual effort that ARKit-based apps typically require.

In this chapter, you’ll create a modern take on the classic Tic-Tac-Toe game and deliver a RealityKit-based collaborative experience. The new project will borrow from Apple’s ARKit example project, but will mainly focus on the RealityKit side of things.

It’s time to get going!

Exploring the Project

There’s a starter project waiting for you in the starter/XOXO folder. The project is a basic Swift-based app that uses a classic style storyboard UI.

Load the project in Xcode so you can take a quick tour of the important components within.

ViewController.swift

Open ViewController.swift. By now, you’re very familiar with the inner workings of the ViewController.

MultipeerController.swift

Open MultipeerController.swift.

Main.storyboard

Open Main.storyboard and flip the orientation to landscape.

Info.plist

Open Info.plist.

Creating the AR View

Now that you’ve gotten the basics out of the way, you’ll start filling in the missing pieces, beginning with the AR View.

Setting the Player’s Color

Add the following variable to the Properties section:

Sending Messages

Add the following helper function to Helper Functions:

Creating the AR Configuration

You start all AR experiences by creating an ARConfiguration and running the AR session. Well, this AR experience is no different.

What is ECS?

When using the RealityKit framework to create content for your AR experiences, it’s important to note that the framework runs a CPU-based entity-component system (ECS) to manage physics, animations, audio processing and network synchronization. The framework then relies on Metal for GPU-based multithreaded rendering.

Predefined Entities

With RealityKit, you can easily create your own custom entities with custom behaviors based on the various components you add to them.

Creating the Game Board

Now that you have some background, it’s time to create some of your very own entities, starting with the game board.

Creating the Model Entities

Look carefully at the game board and you can see that the entire board is constructed out of just three distinct shapes: the two grid bars and the square tiles. You’ll create those shapes next.

Cloning Model Entities

Now that you’ve created the three main shapes, you’ll use them to construct the game board. Instead of re-creating each element from scratch, you’ll clone the original entities.

Adding the Grid

Now, you’re going to use the helper function above to create the grid. Add the following function to Model Entity Functions:

Adding the Tiles

With the grid out of the way, you need to add the tiles to the game board. There are nine slots to fill.

Adding the Anchor

Now that you’ve now completed the grid and all the tiles, your next step is to add the game board to the AR scene.

Placing Content

Now that your game board is ready to place in the scene, you need some user input to know where to place it. All the user needs to do is tap the horizontal surface and the game board should appear in that position. Your next step is to ensure the app recognizes the user’s tap.

Creating a Tap Gesture

You’ll start by creating a basic tap gesture to handle user touch input.

Handling Tap Gestures

But hang, on there’s an error. You still need to define handleTap().

Getting the Touch Location

After the user taps the screen, you’ll cast a ray into the scene to see where on the surface the tap actually occurred. This lets you position the game board just where they want it.

Tapping a Surface

Now, to perform the actual ray-cast into the AR scene.

Tapping a Tile

OK, now that the game board is visible in the scene, what’s next? Well, when the user touches a tile, that tile should change to the player’s color.

Collaborative Experiences

When multiple people share an augmented reality experience from their own personal viewpoints on separate devices, it’s known as a collaborative experience. To achieve such an experience, all the devices should be connected to one another via a local network or Bluetooth. The devices share an AR world map, which localizes each device within the same space. During an active collaborative session, entities in the augmented space synchronize across all the other devices.

Creating a Multi-Peer Network with MCSession

Thankfully, all the hard work is already done, thanks to MultipeerSession, which is part of your project. It acts as a basic wrapper class for MCSession, which is the network session class that connects multiple peers.

Adding Multi-Peer Connectivity

Now that the network is ready, you’ll create the multi-peer session.

Handling Session ID Changes

When a peer connects or when your session ID changes, you need to inform the connected peers of your current peer ID.

Handling the “Peer” Discovered Event

When the network session discovers a new peer, it triggers peerDiscovered(_:), asking it for permission to allow the new peer to connect.

Handling the “Peer Joined” Event

When the peer is allowed to connect, the network session will trigger peerJoined(_:).

Handling the “Peer Left” Event

When a peer leaves, you need to update peerSessionIDs. To do this, add the following to peerLeft(_:):

Configuring RealityKit for Collaboration

Well, that’s all you need to do to create a multi-peer network, but you’re not quite done yet. You still need to configure RealityKit for collaboration.

Enabling Collaboration

To use collaboration, you need to enable it when you create the AR configuration. Do this by adding the following line of code to initARView(), just before running the AR session:

Setting the Synchronization Service

When you use RealityKit, you have to synchronize all of its entities and their components with all the connected peers.

Handling a Successful Connection

Now that everything’s in place, once a new peer successfully joins, RealityKit will create an ARParticipationAnchor for that peer.

Requesting Network Permissions

Oh, you’re not quite done yet. There’s one last thing that you have to do and that’s to request network permissions.

Managing Ownership

During a collaborative experience, when you create an entity, you become the owner of that entity. Should another peer attempt to modify an entity that belongs to you, they’ll be blocked.

Enabling Automatic Ownership

To keep things simple, when another peer requests ownership of an entity that belongs to you, you’ll simply transfer ownership to that peer automatically.

Requesting Ownership

Now, you need to make sure you request ownership when you tap on a tile.

Removing Anchors

As a final touch, when you’ve played a few games with a friend, it would be nice to clear out the playing field so that you can play some more.

Key Points

Congratulations, you’ve reached the end of this chapter and section. You can find a copy of the project in its final state under final/XOXO.

16. Focus Nodes & Billboards
18. Creating 3D Models With Object Capture
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now