Chapters
Chapters

Chapters

Hide chapters

Apple Foundation Models

First Edition · iOS 26.5 · Swift 6.3 · Xcode 26.4

Before You Begin

Section 0: 5 chapters
Show chapters Hide chapters

Section I: Apple Foundation Models

Section 1: 9 chapters
Show chapters Hide chapters

2. Foundation Model Interactions
Written by Bill Morefield

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

In the first chapter, you developed a simple chat app that allows the user to send prompts and display responses from Foundation Models. This simple app lets you explore the basics of Foundation Models. Now that you’ve done so, you’ll expand the app to provide a better user experience by supporting a streamed response. Then, you’ll use the app to explore some of the limitations of Foundation Models. Finally, you’ll explore the LanguageModelSession and tokens.

Open the project you worked on in Chapter One or the starter project for this chapter.

Streaming Model Responses

The app you developed in Chapter One allows users to send prompts to Foundation Models and display the response. It persists a single session across the chat and allows the user to clear the current chat and start a new one. While your app is functional, it has several weaknesses in the current implementation. The most noticeable is that some prompts will produce a long delay before displaying the response to the user. To see this, run the app and enter a more complicated prompt.

Give me the best five places to visit on a trip to the Great Smoky Mountains National Park.

The response to this prompt will be lengthy. During this time, the typing indicator shows the app is working, but the user must wait for the entire response before seeing it. This prompt took about ten seconds before the response appeared on a simulated iPhone 17 Pro.

Response not delivered until complete.
Response not delivered until complete.

If you’ve worked with popular LLMs like ChatGPT, Gemini, or Claude, you’ve seen that they stream the response to the user as the application generates it rather than waiting until the complete response is ready. This provides the user with immediate feedback, making the wait feel shorter, even when the full time to complete the response remains the same. Foundation Models supports this streaming response capability.

Open ChatView.swift and find the sendPrompt() method. Delete everything in the method after the following code:

addMessage(promptText, type: .prompt)

Now add the following code at the end of the method:

let stream = session.streamResponse(to: promptText)
promptText = ""

Instead of the LanguageModelSession.Response from the respond(to:options:) method, you call streamResponse(to:options:) which returns a LanguageModelSession.ResponseStream<String>. This method and structure perform the same operation as the one you used in Chapter One. Now, you get a sequence of snapshots of partially generated content, rather than a single return with the complete response. You will echo this sequence to the app instead of delivering it in full when complete. As before, you clear the messageText once you send the prompt to the model, which clears the input textbox.

Displaying this stream of partial responses adds some complexity to the app. First, you’ll add a new Message type for a partial response. Open Message.swift under the Models folder and update the MessageType enum to:

enum MessageType {
  case prompt
  case partialResponse
  case fullResponse
  case error
}

Now open MessageBubble.swift to show this new type. Add the following code to the end of the switch statement for the bubbleColor property:

case .partialResponse:
  return Color.gray.mix(with: .white, by: 0.8)

This will color the background of these partial responses a lighter gray than the full response. To set the text color, add the following code to the end of the switch statement in the textColor property:

case .partialResponse:
  return Color.primary

This will use the same primary color as the full response for the text.

Now, return to ChatView.swift and add the following code to the end of the sendPrompt() method:

// 1
do {
  // 2
  for try await partialResponse in stream {
    // 3
    if messages.last?.type != .partialResponse {
      // 4
      addMessage(
        partialResponse.content,
        type: .partialResponse
      )
    } else {
      // 5
      messages[messages.count - 1].text = partialResponse.content
    }
  }
} catch {
  // 6
  addMessage(error.localizedDescription, type: .error)
}

Most of this code should look familiar. The general process of capturing the response remains the same. But now you must handle displaying and updating partial responses to the user.

  1. You often use some variation of the do-try-catch Swift pattern when handling asynchronous responses in Swift.
  2. The stream returned by streamResponse(to:options:) is an AsyncSequence. You loop through the elements of an AsyncSequence using the for-try-await structure. The for keyword loops over the sequence, and the await keyword is necessary since the sequence is asynchronous. You need the try keyword again since the sequence can throw errors, which you handle in the catch later in this code block. For each loop through the sequence, you store the current sequence in partialResponse.
  3. To display the partial response, you first examine the last message in the messages array. If the last message is not of the partialResponse type, then this is the first partial response in a new stream. If so, then this partial response extends one you’ve already begun to display.
  4. partialResponse holds the current response. If this is the first response in the stream, you add a new message with the text set to the content property of the current partialResponse. Except for very short responses, more partial responses will follow. You note this by setting it as a partialResponse message.
  5. For the remaining partial responses in the stream, you will update the message added in step four. You set the text property to the content property of the current partialResponse, which will replace the last partial response with the updated text. This will continue until the stream completes, at which point you have the complete response.
  6. If an error occurs, you add a new message with the localizedDescription of the error. Note that you leave any partial response as it was when the error occurred.

Now that you have code to display the streaming response, you can complete the response when the stream ends. Add the following code to the end of the sendPrompt() method inside the for-try-await loop, just before the catch:

let lastIndex = messages.count - 1
withAnimation(.easeInOut) {
  messages[lastIndex].type = .fullResponse
}
messages[lastIndex].timestamp = Date.now

This code will change the type of the message to .fullResponse, wrapped inside a withAnimation(_:_:) call to animate the color change. It also updates the timestamp to the current time.

Run the app and try the previous prompt. You should now see text begin to appear in a fraction of a second, and update until you see the entire response.

The response streaming as it generates
The response streaming as it generates

The result looks much better to the user as the text starts to appear after a few seconds. Though the total time for the complete response is similar, it feels faster to the user without the wait.

Why would you not use a streamed response? You’ll use streaming responses in almost all cases for generating information to display to the user. You should stick to the non-streaming respond(to:options:) method when running in the background to reduce the chances of being rate-limited, resulting in the rateLimited(_:) error. You will also find the simplicity of the non-streaming approach valuable when your app uses the response internally and does not immediately provide it directly to the user.

Limitations of LLMs and Apple Foundation Models

LLMs are a very useful technology, though sometimes overhyped. Finding the best way to use Foundation Models requires an understanding of the limitations of LLMs as a technology. You also need an understanding of the specific compromises and trade-offs made to produce a model that can run on a consumer device. You’ll use the app and some prompts that show the use cases where the model works well and where it can fail.

Outdated training data

Start with the following simple prompt:

Bunsiwmu qe Dturrt so Yi am Hjejm Niazfaifp Lmosgc
Saqkumvi cu Vpabnc qa Pa oj Plekx Gouznoegm Wcanxf

Raekkiox ntat roraict kza lijisg sera oj ursahsokouy eg Yuegvozeeb Norepm
Liamziap bciz dituuxc wmu teqodc cegu id igbiylafeug iv Jeepluyaik Rivurd

Hallucinations

The second risk is better known as hallucinations. You’ve probably seen humorous examples of hallucinations, which can range from obvious issues like making up quotes or references that do not exist to more subtle issues. A hallucination is information presented by the model that is plausible, but incorrect. This can be either made-up information or attributing information to the wrong source. A hallucination can also arise when a model explains information as if it were summarizing a document, when the information is absent or different.

Wuhnegaguleojx ij hofm uj klawat.
Tacpiripaheudj uk jixm ez ljisiv.

Session Context and Tokens

In the summary of LLMs given in chapter one, you read that LLMs work on a context made of the prompts and responses that go through the LLM. Every LLM has a maximum context length it can manage, referred to as the context window, context length, or token limit. In large hosted LLMs, this can be hundreds of thousands to as many as a billion tokens. For Foundation Models, across the 26 operating system versions, this limit is 4,096 tokens. And yes, the length is measured in tokens. Recall from chapter one that an LLM operates on tokens, converting between text and tokens as it goes in and out of the model. Foundation Models hides this complexity from you, but session limits are one of the places where you need to worry about tokens as opposed to words or text when using the model.

Komedej wivrasy vakqicd fuqo.
Yawazuv sesbudl cenhamx qato.

Counting Foundation Models Tokens

Knowing the maximum size is helpful because it lets you compare this to the sizes of your prompts and responses. Starting with the 26.4 release, you now have ways to read and count tokens much more easily. Add the following new property to the view:

Kyunajy ewev kupzawp laxxoh qow capkaut.
Pgigohg anoh zuwvisv jezxul nav zexcieq.

Measuring Tokens for Prompts

You can also measure the tokens for individual prompts and responses. You’ll first update the Message struct to hold a token count. Open Message.swift under the Models folder and add the following new optional property after the existing ones:

Braj bkaqabv ragep fiirqk.
Wwig sbopakr qabuz boefyl.

Conclusion

In this chapter, you’ve explored streaming responses and tokens, and how they relate to Foundation Model sessions. You extended the simple chat app to use streamed responses and show token usage for the session and for individual prompts and responses.

Key Points

  • Streaming responses produce an asynchronous sequence as the model produces the response to the prompt. Showing this will make your app feel more responsive.
  • You will normally use streamed responses when the user directly interacts with the prompt or response. Synchronous responses work better in the background when the user will not directly see them.
  • Up through 26.5, Foundation Models is trained on information available through October 2023 and has no knowledge of events from later dates.
  • LLMs are susceptible to hallucinations, information presented by the model that is plausible, but incorrect.
  • LLMs will also sometimes not get the prompt right. Sometimes you can correct this by adjusting your prompt.
  • Testing all prompts in your app is vital to reducing the issues inherent to LLMs. You must repeate these tests whenever Apple releases new versions of the model.
  • Every LLM has a maximum length of tokens it can contain. For Foundation Models, this is 4.096 tokens.
  • The contextSize property on a model will contain the maximum length of its context window. Apple introduced this in 26.4, but backported it to earlier 26 OS versions.
  • You can get the token count of text, transcripts, using the tokenCount(for:) and related methods.
1. Introduction to Foundation Models
3. Tuning Apple Foundation Models
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now