Chapters
Chapters

Chapters

Hide chapters

Apple Foundation Models

First Edition · iOS 26.5 · Swift 6.3 · Xcode 26.4

Before You Begin

Section 0: 5 chapters
Show chapters Hide chapters

Section I: Apple Foundation Models

Section 1: 9 chapters
Show chapters Hide chapters

8. Using Foundation Models for Voice Notes
Written by Bill Morefield

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

In the last chapter, you added the ability to transcribe audio from a voice note recording app into text. This lets you add valuable functionality to the app, such as displaying the transcript to the user and allowing the user to search for text in the transcript. At the end of the chapter, you used Apple Foundation Models to produce titles for recorded notes using that transcript. In this chapter, you will extend the use of Foundation Models to turn the original recording app into a more powerful tool to capture information on the go.

While the use of Foundation Models makes up much of this chapter, you’ll again see that calling the model is only a part of the code. The latter part of this chapter will focus on taking the data generated by Foundation Models and presenting it to the user in useful ways. The goal is to use artificial intelligence to provide value in your app.

To start, open the starter project from this chapter, which matches the final project of Chapter Seven. Repeat the steps to add the NSSpeechRecognitionUsageDescription key to your Target:

  1. Go to the Project for the app in Xcode and select the VoiceNotes target.
  2. Go to the Info tab, and you will see the existing list of properties.
  3. Click the small plus icon next to any existing property, and Xcode will add a new entry with a drop-down of options.
  4. Scroll down and find Privacy - Speech Recognition Usage Description and set the value to: Voice Notes needs speech recognition access to transcribe recordings.

Producing Data Structures for Analysis

In Chapter Five, you learned about perhaps the most powerful feature of Foundation Models, guided generation. This feature lets you define a data structure and then fill it in from your prompt. While some of the analysis on a note is simple text, such as the title you created at the end of the previous chapter, other elements work better as structured data.

Create a new Swift file under Models named VoiceNoteAnalysis.swift. Replace the contents of the file with:

import Foundation
import FoundationModels

@Generable(description: "A concise analysis of a transcribed voice note.")
struct NoteAnalysis {
  @Guide(description: "A concise title of a few words that summarizes the note contents.")
  let title: String

  @Guide(description: "A two to three sentence summary of the voice note.")
  let summary: String

  @Guide(description: "Up to five short lowercase topic tags.", .count(1...5))
  let tags: [String]

  @Guide(description: "People referenced in the note.")
  let people: [String]

  @Guide(description: "Specific action items or tasks mentioned in the note.")
  let actionItems: [GeneratedNoteActionItem]
}

This struct uses the @Generable macro, which allows the model to respond to prompts by creating an instance of the type. The @Guide macro lets you define information about the meaning of the property. Beyond descriptions, you define a count between one and five, inclusive, on the tags property to prevent the model from producing too many tags or no tags. This struct defines the analysis you can do on the note. You’ll create the title, a summary, and tags for the note. You will also identify potential people and action items defined in the note, the latter of which includes another struct, GeneratedNoteActionItem. Define GeneratedNoteActionItem used in the actionItems property of the NoteAnalysis struct by adding the following code after NoteAnalysis:

@Generable(description: "An actionable item or task extracted from the voice note.")
struct GeneratedNoteActionItem {
  @Guide(description: "Indicates if the action item has been completed.")
  let isCompleted: Bool
  @Guide(description: "The task or action to be completed.")
  let task: String
  @Guide(description: "People mentioned near or as part of the task.")
  let people: [String]
}

This struct defines the three items the model generates for an actionable item: its completion status, the task, and any people referenced in the task. Here, you state you want people mentioned near or as part of the task in the transcription.

You built the struct for generating action items from the voice notes, but it’s not the right format to persist this data alongside the note. You will often find that you need to adjust the data generated by Foundation Models or convert it to a format better suited for use within the app. In this case, a vital missing element is a unique id for the action item. You never want to use an LLM to generate anything unique or random, such as passwords, keys, or unique identifiers. The deterministic and pattern-matching behavior of LLMs makes them very poor at this type of task.

First, find VoiceNote.swift under the Models folder and add the following new code after the VoiceNote struct:

struct NoteActionItem: Identifiable, Codable, Equatable {
  let id: UUID
  var isCompleted: Bool
  let task: String
  let people: [String]

  init(from generatedItem: GeneratedNoteActionItem) {
    id = UUID()
    isCompleted = generatedItem.isCompleted
    task = generatedItem.task
    people = generatedItem.people
  }
}

This new struct implements several protocols to allow the existing store to persist the NoteActionItem. Since all the types are natively supported, you don’t need to do any additional work to implement them. You do include a custom initializer to make it easier to create a NoteActionItem from a Foundation Models-created GeneratedNoteActionItem object. Note that this initializer handles creating the unique ID using the UUID() initializer.

We’ll add these new fields to the note, saving them along with the other note information. Go to the VoiceNote struct and add these new properties after var transcript:

var summary: String?
var tags = [String]()
var people = [String]()
var actionItems = [NoteActionItem]()

You define the summary as an optional string since it will not exist until the analysis runs. You create the others as empty arrays, which is their initial state. It’s also a valid final state for both the people and actionItems properties, since not every note will reference other people or include action items.

Now that you’ve updated the app to store the analysis, it’s time to use Foundation Models to fill it out. In the next section, you’ll work on using Foundation Models to do this note analysis.

Using Foundation Models to Analyze a Note

Open NoteAnalysisService.swift. You’ll begin by producing some error states. Add the following code to the top of the file after the imports:

Memdokf kge bengo dnux boqrat fixe udewtbif.
Tehjojy tvu nitza nnak qewnun sobe umudrsew.

Updating Notes with Analysis

Open VoiceNoteStore.swift and begin by adding a reference to the analysis service after transcriptionService:

Lijburb lyu fayxe xwif sibxez sequ ujaklpez.
Puzdepm dju hopci hkab yobhep soxe imehnjiw.

Showing Note Analysis

With the analysis in place, the next step is to present this information to the user. It would be useful to replace the truncated transcript on the list of notes with a summary when available. To do this, open VoiceNoteRow.swift and find the private TranscriptSummary view. Find the line that reads } else if let transcript = note.transcript, !transcript.isEmpty {. Insert the following code before that line:

Kpuxewh jipxetm ojjtuuk uf yfezrajel smugzkwaxd.
Qxuxakz nazgevx oxkmoaq uy mvurxefam jqelznmiwq.

Seuvo casi bluxegc qimfegx oz mda yefe.
Viode zopu tvicopm dowzing in hwe luva.

Mnobexz nuaxye omc filx day bxu pape.
Rnufifx koenho umv gahz moj sso nexi.

Rkibuxf ufreel ulidq azudfivaej am vwa riju.
Gvumawp axraog ukokk igiblisoad us jri yava.

Searching Note Analysis

After implementing transcripts for the voice notes, you added the ability to search for text within the transcripts. Now you’ll allow the user to search within these new fields. Open ContentView.swift and add a new enum to the top of the file:

Kuovqyanm mos u vebgok haxxoihek ov u refo.
Jooxptaly pey u lexwox pihpeepay es i qiqi.

Showing Action Items

Create a new SwiftUI view named ActionItemsListView.swift. This view will show all action items contained in any voice note. Replace the contents of the view with:

Tahtatfiy Owqiox Aromp id tse qigpv webu ih hmi xuuv.
Yuyfuydoz Aysuiw Ipavg ec tve nommk mowo uj bdu nued.

Conclusion

In this chapter, you took the data added through machine learning in Chapter Seven to produce a transcript and applied Apple Foundation Models to analyze and find useful information in that transcript. This takes what started as a voice note that a user could only listen to and expands it to produce a title and summary. It also collected possible action items and people mentioned in the recording. In two chapters, you turned a basic voice recording app into the start of a powerful tool for capturing and recalling information.

Key Points

  • When using Foundation Models, you can produce simple text responses or more specialized data structures using the @Generable and @Guide macros, which avoid parsing unstructured text responses. And you can mix and match as it is appropriate for your app.
  • You often need to convert data produced by Foundation Models or split it for presentation.
  • Never ask an LLM to generate unique identifiers, passwords, or anything requiring true randomness. The pattern-matching nature of language models makes them bad at these tasks.
  • It’s good to check the context length for prompts. You can determine this through experimentation. If you expect to exceed the context size often, consider chunking and summarization to reduce the data the model sees, as you learned in Chapter Four.
  • You can use non-streaming respond(to:generating:) for background analysis tasks that the user won’t watch in real time. Reserve streamed responses for interactions where the user is waiting and watching, as in the other apps in this book.
  • Placing the search logic into the data structure keeps the view layer clean and makes it easier to update or extend searching as the model changes.
7. Extending an App with Foundation Models
9. Conclusion
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now