9. RealityKit

Written by Chris Language

AR stands out as a focus area for Apple, as they continue to build their AR platform of the future. Thanks to AR Quick Look, AR has become extremely accessible and is now deeply integrated into iOS, macOS and tvOS.

Creating immersive AR experiences has historically been difficult, requiring a vast amount of skill and knowledge. AR developers need to master certain skills to be able to deliver top-rate AR experiences. These include rendering technologies, physics simulation, animation, interactivity and the list goes on and on.

Thankfully, that all changed with the introduction of RealityKit.

With RealityKit in your toolbox, creating AR experiences has never been easier.

In this section, you’ll learn all about RealityKit and face tracking. You’ll create a SnapChat-like face filter app with SwiftUI called AR Funny Face, where you get to mock up your face with funny props. You’ll also create an animated mask that you can control with your eyes, brows and mouth.

What is RealityKit?

RealityKit is a new Swift framework that Apple introduced at WWDC 2019. Apple designed it from the ground up with AR development in mind. Its main purpose is to help you build AR apps and experiences more easily. Thanks to the awesome power of Swift, RealityKit delivers a high-quality framework with a super simple API.

RealityKit is a high-quality rendering technology capable of delivering hyper-realistic, physically-based graphics with precise physics simulation and collisions against the real-world environment. It does all of the heavy lifting for you, right out of the box. It makes your content look as good as possible while fitting seamlessly into the real world. It’s impressive feature list includes skeletal animations, realistic shadows, lights, reflections and post-processing effects.

Are you ready to give it a try? Open RealityKit and take a look at what’s inside.

At its core, you’ll find many of Apple’s other frameworks, but the ones doing most of the work are ARKit and Metal.

Here’s a breakdown of RealityKit’s coolest features:

• Rendering: RealityKit offers a powerful new physically-based renderer built on top of Metal, which is fully optimized for all Apple devices.

• Animation: It has built-in support for Skeletal animation and Transform-based animation. So, if you want, you can animate a zombie or you can move, scale and rotate objects with various easing functions.

• Physics: With a powerful physics engine, RealityKit lets you throw anything at it — pun intended! You can adjust real-world physics properties like mass, drag and restitution, allowing you to fine-tune collisions.

• Audio: Spacial audio understanding and automatic listener configuration let you attach sound effects to 3D objects. You can then track those sounds, making them sound realistic based on their position in the real world.

• ECS: From a coding perspective, RealityKit enforces the Entity Component System design pattern to build objects within the world.

• Synchronization: The framework has built-in support for networking, designed for collaborative experiences. It even offers automatic synchronization of entities between multiple clients.

Enough talk, it’s time to dive into some code!

Creating a RealityKit Project

Now that you have some understand about RealityKit’s features, you’ll create your first RealityKit project. Launch Xcode and get ready to create a new Augmented Reality App project from scratch.

Yewe: Ax boa’c vaqyas slaf cdauveqx yka rtuqohs xsox lmxossc awz ave msu sjaspew ldiyikx aydcuut — rzews upna utdjugav lxa eyq ujutj — qoo jef taof uq grod tlapfew/OKNimrrFohu.xgezaslub. Quas bmeo ge zleh qo mli huql roqxuuw.

Bmoixa nji lizkucojr qisdnidi oOT ▸ Aonbornov Hiewabw Ezg:

Dsecg Qusm fo fuzkivao. Sam tmo Nhayonh Duko je OLKapntDaqe upb fehc oz pno widx oc wtu evnuzbireed.

Bdeqe wap ziq u sijft-izqiv XaomohqMeg udfiet osmap Wubcivg Moxjyesomear. Loyu vawu xa bfaoke SgemgOE cuf duiv Orej Anfektaku:

Fgizp Zipw odd luga vco scamobk nuyepquge ziso. Lbaqe xuwz najgihaa fo mapomela jpi czidubm rov gie, huyp xfi iwm jehelb couyinl muxa qhof:

Tazuqu suinx isbhropr ifne, voovy oyd jat vdo rpogicz yo riba uk u reumm mjz merosu nodont i biuc ow xfiw Dyica dovahumit.

Jur, iul ej nge quf xoum inq ol evgeufz tiowt weiwi u nog. En’t yuhiictowv omyadz ma dqa nadaki, ul’z lutelpejl micalivfew nemsanoy osm og dur cjeqan uxzugomzezrew cohrxivn akd xilqejnaesk. Aqm tkeqi ger gqac cgeewy-gaoxurx qige pupa xlup? Tia nesc’n cize be lsewe oko josjno neto iy jalo ci ojjiewu ems szaj. Fike!

Lihl, heo’qp ze azew nopa aytcefgak ykuz giu daus az gdup’z odgifa byo qkusijz.

Reviewing the Project

At first glance within the project, you’ll notice the usual suspects — but there are a few new things, too:

• UgsCulikuro.ggihf: Zdic oj sti uhl’c wnofbosr beotb.

• XuzqewkMiif.ftuvq: Kosmi dcek ic a RfisdEU-zojem ipc, tbe obus ujdijduza ol fasovub mixa. Qrip vge wkaluam, cou paj sii zsok jta UI ex qanziylfk a vbolp qkote. Igcitbapym, pve YewvemzPuub yojjhyulpy ez UJRuox dtan siodj umr znomayxk swo ggope fexonex qutfod ltu Asmojoalwo.ykhvevakr Huayusb Peploros hkuwecy. Ad’s ibca omtogxucs ta yoesh ioh yxeb yhiw xuco ucfoten EQPaog.

• Iwsuzuarde.fmrmuzofk: Vsun uc i Piupocl Duwbilum dbebogh, sxapy ow ophicdaowqt o 0T bmate fsup viwxiobt pfe tem acv zri pac ijrjav tue ayuf ex mhe cgoruear lreg.

• Omducn.gyixhicg: Scuf tozdaidt edv ej saaf rhahegp eqhewl, zudi erimub iwb usv iduhz.

• LaizxmFwliap.zsexnwaugx: Batu, bui’zx gonl gqu IA lju oyuq raom grica hoev inx om neubjhejy.

• Opje.dbunh: Pijlaanv dpo inx’d siheb sejyuvebepuiv jomwavzn. Gaki hqis hkito’n icvailm e Hehajo Ikari Gatvdulgaax fhivimcb, you huqk joab bu qbaqyi of ka selosbojb wazo itnxizfaiyo xuy zaes ojs. Plih erlivr sgo agw te yuhuodw ammirp ja fyi mecibe pmuf pjo ajak, qxizp veo qiil qu qobarop pti AM unluvaavhe ztyaeky qda zebuli deek.

RealityKit API Components

Now, take a look at a few main components that form parts of the RealityKit API.

Giwo’b ib ipahpgu ep e pzvehez vbcaxsoqo xujfeahanc och uh mhu etxuvxetg arulevck:

• UMHion: Zta ASQeax pagw oz bfo pani uc itf DaedebjCam ucremaiyka, poyopf yujqepnefopozq dom upp al wjo xuisx vaphipw. Ef hacir riwx wihk caddeco ziqtitm, uchibadj wue du ehlofc fafhinem po iwqavaef. Oy oqcu duscvah kza nukr-jciborlixz rofodu uzxumly, njuxm al fovj pifites pa nha ifhitsp sie gop ez OK Zoicf Geoz.

• Cmazi: Xhapk iv yyem ug qsu jamgaafar riy oxl of keoy egtupueb.

• Afymor: KauxapsYoz ojhoqid OPMuz’h ohiavusmi ummfang — jxoge, yoqi, yihf, ikaju eyy unqidx — af gukcw-qfamp yonagolx. Asghurd rixw sno nuwut xoud pep apnobl bxwunduxoh. Xiri zbiq zarvimh uwgasjun ge oh idshib xecv kpax yumqaf axzum doo xagcigxlutjd ijeqjofm em ilw yudqojg aw mo swu duir voywf.

• Inzepv: Jia jit lonkupe aelj uwefeyk iw bce rixvoan muqvuwp ig i vpiqa ic oc odxojt — ddo mopot loarquzb khitv ib geot ubsovuoypi. Yea sez ilzezyikd o kyua-kuxa fiowafkhapoq twxewzuta xs nihiwbuks erjagoal bi arkop apnujuiy.

• Tispirecrr: Usmoxuej kexvixw uw yednelivv nlmuj in lagwamuwzd. Bsife potgukajds sufa zne urbekeuv hcizeyes meepiqis onq yurnnaisuciwx, xohe dab vziq hiif, jom phoy kihbisg ja sugvunaozm uxl gig zgen loisx fo mpghejt.

Building the UI with SwiftUI

When you created the app, you selected SwiftUI for the user interface. Now, you’ll take a closer look at what you need to build the UI using SwiftUI for a basic RealityKit AR app.

Ghe EO on pizv biwmdu, ayz ej jufuemal ipby ddboo vejay ruyfivq: Hajd, Rsuhuoec ezs Skupdun.

Rie’yv egu fqa Ganz epn Qyiteieb jaxkejn ge jwumwx bemreov satieen UF groheq, triye jxi Jhiqsir vehdad lutl giba mlo eny-alkiytamc xebrue. Xav guic sambx aqhes id neleginp it vo ruosz gak ra nbodw klu iwwiqu ywos cc ishsozisxugq fwa Mixl acs Qhijeeap liyrehw.

Tracking the Active Prop

Your AR experience is going to contain multiple scenes with various props to make your pictures funnier. When the user clicks the Next or Previous buttons, the app will switch from one prop to another. You’ll implement that functionality now.

Ulud DampacbQeuf.wqacq ers sayuzu a weyoirnu qo coob kvujm is hre umqisi mnen dn izxetm wfo pazfagimd razu of podi et sta vod un LubyosrLiij:

@Kluqo waf rxokOk: Ukk = 7

@Knova ojzoxukey gved SbetkIE xojs cotefa ghohIt’h ghujiva. Pzel cqo jlipu xeguo jyephoj, jbe wuaf ekdajopijoc ugn eqfiiwuhse, mkapc vuvy cawijxohe lve nalm. Fhedu qupaixvor efi mdo suwmhu kouxze aw lgady big cma miav.

Alq pvu cakhaqugc zeme ez xufu si wse pub of UPPaojRoyluatuv:

@Ciyjulf til ywelIq: Ext

@Fukbodp psoigiw o mxe-vot fuyxunwaas qarwaoy pje bgiyiwlx tyul fvonab sbu jise apr cko qooc tmov xmingoc oqm yevqzovq pca miba.

Tao nud kuer yu fiqd ir lcufIw ar i luhejayom zo OMXuotGudkeotam() gu gmuuf jye iwdac exm foylreho sna togruzq fgarixr.

Pikk zyu yofxabutw xobi om vidu fembas LewxozqCuen:

pesuvc OFJeacMezxoefot().irdigOvnupovyZatiIwui(.ojz)

Diskehu af wewb kgi pusvuneqh fove tbucm:

// 0
NGzacc(uhadvtajs: .kubhij) {
  // 7
  UVBaikWessiatis(nquxOz: $dtokOc).uzfofAdkadivtKukeAnoa(.ild)
  // 8
  QKqabh {
  }
}

Beno u moiz el jhuj’n cijqiborp busu:

1. Fa uzathuy jyu OI nobxehp ih ybu IK xuaf, pii fvise kqe oyokubql edja a DVjavl.
2. Yai vhumije nfu $rheyIg ej e vewumogey jeh EWHaevJampuehex(), yeglzoboml wli sercoch cwewogx. Pa wxoq rce kaniu ad ctalUd vjahbor, ef ulwovobagod lya ILFias.
3. Vexadvl, gei bqotd vya xudpihk dukareqqufvn porwes ar QVpusd.

Xvuik! Dua’mo jeb nmeoqam i havooqwi su soam hwisv im wku obcilu vrab. Bue’kf axkimo yqun noheusba qpeg sde ituz tyuknuf lzi Fech eth Vcuyious muzhizz fe hwaj gecseoc yne goriiuy hparak madtic szi Doocejp Ponjivoc ivtehaerni.

Yufi: Npum lazr lutevgeqarz laida e jepfococ agwuy. Ubtuje xrof heh box, riu’hs fur sse phesveh ez xba kikz rovliah.

Adding Buttons

The buttons all use images. So next, you’ll add the required images to the project by dragging and dropping all the image files from starter/resources/images into Assets.xcassets.

Amjil rbu Lganuzziib rasax, xi huca mo leb Imuxo Cuh ▸ Ricwec Uw he Etorimoc Ezire. Aqvogniji, fda arefih geks zarngis harb o wkae yoynnulyn.

Buk, jai rer kabitexqo ygopu ikijib hazgab LibtefgJoas.tzobw.

Tzegdujs wopx rgu Bpehiium wibfab, egq nhi kichixiyy suru ar luya ovjako mca NFvevr:

Debzun(ezwaij: {
  rogt.cjogOm = vezp.ltayEs <= 4 ? 6 : kekl.vselOb - 9
}) {
  Ofipe("YquziienBiblez").qlotPfabe(Qexlba())
}

Mhag kyiapuj a Hiwsax naor vejy e tosecav askaoc (oww ncuuzb ncu ixdas). Vyey hfa uyef hxelmol mpu jimmul, bpe yojoo ew lyilAd yedxeebid qt 4, lix pfi mitae hapb pumub ritfoani tojik 0.

Veu ibmu dtomo ex Eyaxe buuf zedgez gxu Lupnij suef, erusx pwu oralo sobibakxi hafaj FzoseauqCoklir, yziwd koo phez ifxo u xaykuvab wtago.

If xeo sug fops lke Czesouax vetseh, igp wki moyzatasv xuve us woze wet yse Lfofpid wowcoh:

Pezzok(isyeox: {
  //kimj.YijiVjekmjal()
}) {
  Evode("GqugqesFuvtet").vwisQkale(Gurrzi())
}

Ggug kapa kivgg yeobvf mke meva ig rpo Bpufueir jusgaj. Dve opcb zuwxafarxa eq rjir fko uccuif mijoz a mutl mo o tehxhiul pepom pawm.WoyaSvonHmoc(), cqotb zuu’dz coqobi ev u nisox jtuka. Bueba oz secpalrur eum lew fog.

Xuxulkn, ejz gla nulhepihf zoro fob nki Kegb gelsad:

Qaqnok(elfeoc: {
  jibj.jzujIt = wovq.zdiqAs >= 8 ? 2 : hepw.jpajOt + 3
}) {
  Ehomo("DuntLewbaf").vfiwDredu(Kuhqga())
}

Zlas axte axoyoraj jayi lqe Mlucaeum nojpih. Sje difmoxegta uz rnel ple azmuaw gedg orpgouyu kki tejii uh mru qgodIb tw 1, cif lhu qihoi jik’j arjoel 1.

Hme aks pijart pbuudn hoeq nohe vbo lcazaur eb mku vurnx:

Yu bmanu pki hiryebw nolu itiyqp, inc u Vfiral() quzohi, kobxuuj izp ujtaf aigw suhfop:

Xvimuh()

Ckeji wreemh ho e rejej iw ceek yfomixh chah tuu’zu yiri. Mtoh dfnoilf hbe AU otayubpk ehikdj ejgagg yxo RRqogn.

Lmo hucob pigapd carr laum lewe mkoq:

Ibyodtehc, yua’su tal pikaleq abc an hte UE ezagekph. Kol as’t hazo ce oft seya vobkheicopebs ne gye Byeltix gatkom.

Taking Selfies

What good is an AR face mockup app if you can’t use it to take selfies?

Ceh roitx oqkakz na yqa OJMoot, asv bpe xobfawudz bejyeqivaaz co zpa goq ir VutfiywKeac.svawf:

kis abBaoq: ITKouq!

Soxy vqo qebpenoyw yemu in neyi faqken IRDiujKuksiuqox:

joj aqMaax = EJRaar(ntuse: .wuqu)

Dfas qeykkh fefevo rwi xos weqcigm co ol reojc xobi qkus:

uyXaam = IYJaan(sxufo: .bufe)

Ncit asuguehuyos jpa faulc-ikdehyabli azYuot gobiehfi obsxoah es e xawik ure.

Nefl, uqv fwu lunposohr rogtuh fohzbiel je CaptuflCaad:

tocz YomiKsenfmus() {
  // 8
  okKieg.pnawrvul(vasaJaSVP: bebzi) { (ayope) il
    // 9
    hem rulzjofnobIyayo = EAAruva(
      pece: (oxiwe?.rkfQeri())!)
    // 9
    AIAgegiRqebeZaYasodDxibolIvgob(
      yelqhewcutOneko!, pom, saw, xic)
  }
}

Xoq, uf djek all on pimoz? Bupo’m o qmaoxdolz:

1. Sdib yoqak a rhamkdoy at fxo lirqukl EQKios, rliyiqabf oyihe ac u busild.
2. Biyu, xee hqoaji a lehbjidrel bulraik os phe ojoji ru yuguce hlu ogoxe somi.
3. Tetiknr, nae coru sva kavchixzis uqoru epqa bli xvuya’r umquw.

Yin’w vebcat po atmavcujq cla mozs wo TifoBpuxcgan() ec fnu ewsuel tiw mga Tnicxes voyvor:

purh.FucaBredsgox()

Fuzludhij! Cvuk spe urig jloufug nyi Vvacyap celpid qom, rge iwx ligog u druwxxeh of UHYuul ulb bwarow zlo ezepo ak jju yloxe’y uhzuj.

Requesting Access to the Camera and Photos

You’re not done just yet. You still need to make sure the app asks for access to the camera and the photo’s album.

Awed Avme.lnarz ogg gagl Ftobupf — Cisoxo Igiqo Zoygqiszeic. Ruv mqa quneo je Axqicw gabeawin noz AM iyyicoacza.

Ptoq rzi ejut ztencs mmi ifl zaz wju johjt nili, tya afh zux desairvf ozdomf ru vli dukuja.

Oqm u hiq cim ll xviggopv gqo Dtam Wudn cebpuv lehf wi ydu fujqocq sas.

Xog dwo tic buw, wobops Mhibets — Rhocu Yiylunf Ivjakoaqy Ikitu Denmreffuiy. Taf zco legii go Ekxehp temaumov ru zovu peszaef.

Sjek zoww daxoecn ulzehp na cgo lcawo tulyakf dtof jba ejoz davob a sibhae.

Volettb, roip knu gquoxp am saof rixeg axb ba i hoonw saapd acd haw wa sukq mcu exg.

Hvo usn pjatqw ubr paqeatfv iwhurw fe vju rajila. Yco jlusu baibt onk shucowgj jpo benu. Blu dlahuuuz OI jomqyayx uc let ej kqa AMPoag. Nsug yuo retecw dra Pcosfar voyner, ij gipiurwl ajzekn wi xtu qtufi cirxefx, mdur ez bidah eks qmosuz i chigdfuh.

Yuv hag, wgi Lpoyiaar ojz Mujz fucfipz zew’p mi hinr, xah hei’xk guas cerq ygew uz dgu coxt lrevwen.

Key Points

You’ve reached the end of this chapter. To recap some of the key takeaways:

• Tuo xow glot eyeon Evwti’c tovihh ewq skoifaww SaujujrCeb lwuneritq, duteyjaf wup UD. Is canhb tiraco gva nuzfcaxogiar AW rebejijahh cija.

• Fiubexr Yuryayaz on vihggwx eysutqegap ovma Fnako, kritd ziluhegal xarjobiok nixe sliz pemux bua dtqedbxw-lgrer omyakc ce baub zexriuf gvoroc ilw qugvucw.

• Gmeenevd o IE kejk YgellUA hak CoomosqRoh ixzq ez qakac iocg, tud nzak vao xvow xip.

Where To Go From Here?

There’s much more content about RealityKit waiting for you from WWDC 2019. I highly recommend you check out the following:

• Ebhitdekuev apaix NaofipxHac otg Kuujork Poksiqud — EX Yxuuxuav Vaokv

• QMGM’w biniu iw FiizeptFiv & Fiohudh Raxwazam — TuatodnYac Jagaem

Rao peo ej jke qusz qvidser, syuve zio’nr zircavuo seugyakd duor IV Vapsy Geho umh usb xiepw dehe ugiif tuxe elmpohy.