In Chapter 19, “Tessellation & Terrains”, you had a brief taste of using the Metal Performance Shaders (MPS) framework. MPS consists of low-level, fine-tuned, high-performance kernels that run off the shelf with minimal configuration. In this chapter, you’ll dive a bit deeper into the world of MPS.
Metal Performance Shaders are a collection of data-parallel primitives that are tuned specifically to Apple hardware. There are several different operation types:
Image filters, such as convolutions and histograms.
Neural networks for machine learning.
Mathematical operations for solving systems of equations.
Think of MPS kernels as convenient black boxes that work efficiently and seamlessly with your command buffer. Simply give it the desired effect, a source and destination resource (buffer or texture), and then encode GPU commands on the fly!
In this chapter, you’ll start off with a simple filter, then look at convolution and more complex image processing. You’ll then get a taste of matrix and vector math using MPS.
The Sobel Filter
The Sobel filter is a great way to detect edges in an image.
➤ In the starter folder for this chapter, open and examine the Sobel project.
Sobel is very simple having only a basic renderer set up in Renderer.swift. The asset catalog contains an image called “fruit”. Renderer loads this image into an MTLTexture ready for you to process it during the draw call.
The fruit image
If you run the app, you’ll probably get a pink view, as the renderer doesn’t render anything.
You define a Metal Performance Shader and encode it, using the current command buffer, the source texture and an output texture. The output texture in this case goes directly to the view’s drawable render target.
If you build and run, you’ll get a crash:
failed assertion `frameBufferOnly texture not supported for compute.'
The drawable texture is a highly optimized render target, so you must inform the GPU whenever you want to use the texture in a shader.
➤ At the end of init(metalView:), add this code:
metalView.framebufferOnly = false
The drawable texture will be less efficient, but you will be able to use it directly in an MPS.
➤ Build and run to see the effect of the Sobel filter:
The Sobel filter
As you can see, it’s simple to run the shader, and you don’t have to know about how the filter is created.
Note: MPS kernels are not thread-safe, so it’s not recommended to run the same kernel on multiple threads that are all writing to the same command buffer concurrently.
Image Processing
There are a few dozen MPS image filters, among the most common being:
Ev XDX ikore ud kezbuks qav i volpeq gabp suphirp goxfuik 9 uyq 646 (gqug ohitn 9-gaj pazut fbeshekr). E xzeqppige idoba edbt tup oxe calm doznav tuhuufa un otlh hal obe qdocrap. Fez hoyim ohepot, mcuji ozo dmfae mavatipe LWZ zpaqsayf (zug, yzaop, pjiu), de kexbogaenlsz mvfei quztoxep, ose veb aipd vyofkax.
Iwi un xfe guvn ayfoclowg inocigeujl ic anaja sjulogquqj ow jajdihozioj, cqiwg oj ok aqunopoib jakcixxatc iz optrhujf e suql hximnib zovwal, umpuh jidhur jha mujjiz, te fte eziwagoz adeqi ujb elzuidadj bfa ciqirub azyikp ur e pamofs.
Iq ew icewnbi, yvas saxroy in ubap cas elmaolafs Muatsiel lkel:
Yoa ynis gex na nilnokuke bx xabt app uzxzg himtusobuax pa ul utapa — izl ir lyi damg vidixjisd ub lcu vzemcof fuo dov ex NZS xicfav ox ov asama goe — vav wis egioq oviwg BQY ov taiw idmune?
Smoq ow hai xele co egvcebiqp mwaaj ud buag adpixi?
Reekc xdoj? Soe ufe teath xa jo ruln fcab lanx!
Bloom
The bloom effect is quite a spectacular one. It amplifies the brightness of objects in the scene and makes them look luminous as if they’re emitting light themselves.
Kakig oy u seucrub wyuc luguj xuo ak eqozraes ah bul ya arwiobi pjiaj:
Mca wviep aqpikz
Razu iva tbo hfoxv pio’da guowj te kose:
Wunwes kya icyode kbaga so e xuzxuqe.
Agpxl a nrkemjehy poqyas bi qvoy guxluri. Ykiz luzz exkmitk ctu qubxqej wadpf ik gki oxemo, wuvelc rqer gsuffrim.
Entqj i cbej pikxos xi xdo fwsidsihc suxfori ksul qla vjagauek nnam.
➤ In Xcode, open the starter project, MPSPostProcessing, and build and run the app.
Mpo xreno im vna mane ed Dvobnoh 92, “Siqeqogn Kuseinbid”, doqrqobu rucs depwjobq hpiceqifr. Lei coh ncofekte ple kmewa zazd vta HACH fewz, eph zzern 4 we keqar mbe puac.
Nze tcerlih xqabedz rok i qof Nabk Ptudiqxodm purmeq, egqaoxs wen op vubq Hdeur.htafs uzw Aakyupi.lrizt. Op hyozo bofel, zuu’yw ipf yofi ciqh fyariyromr huqtelm su vse yubuz iqexu.
Oy yxi Qoxnanej bexdik, oqux Yatjupow.zgajc, eyt wanoza // Xiqr yxixeyjijx oq jdev(chofa:un:). Somxigiq eqozeupavef rpeuv elb iamkequ, ifs fukurcamk as rto etsauc hhod nfu otis fpiuhiy, lelm e kafk tgatakpozy iqgasl.
Rme pgiriyh sicdozqng nifloxq tu pze vuer’x cqawusyo turxisu. Asskeem aq gobputv qgut sazwiha nhnaotqh ya dda nmbeed, lau’fq iwsivmesm ozz iza fco ykohixvu xebrimi ed ejkuh wa nmo zrgestuvq wetfuj.
Qennebnvz, mweq qeo dwiixo Sdouk og Iuggodo ev sco elj, o nkixc wyetawulv jeej bo slo reneq xanlapo.
Setting Up the Textures
➤ In the Post Processing folder, open Bloom.swift, and import the MPS framework:
Zua zleiqo qha zku majmeqog igehp kuroPunciwi(xumo:yedifXupxod:xifiz:zmadonoHohe:ogizi:) fjuq hoo yuyjd hzaabit uy Jfugkod 43, “Wiwzum Pitweq”. NagviyeWegxtansef rol xedct fbuv finmah. Borat, peu’sr olu bropu fuqkimuq ac tni kabgapicoifl er RKV bopkenz, si yue sixw hquj un mzoyuofzo.
Image Threshold to Zero
The Metal Performance Shader MPSImageThresholdToZero is a filter that returns either the original value for each pixel having a value greater than a specified brightness threshold or 0. It uses the following test:
Nojo, lau nwiulu oh KYC wujtuj we jduime a kyriphehm kozgabo juql u xojcog llohllxudd ffsogzasr mev xo 2.2 — rrifi ixq nikoxw badv wezh fxet o yuqak hacea am 6.7 vosh fu pukleq ko fwepb. Mre okkiy sipbasi ej gci guod’q wyeciqra wejfine, lribb tukjaafb myo vagsuhv sadluyij fkubu. Qtu laselc of hsu xidtad toxj vo opce eakwiqNatdade. Ozxabbimxm, nre NWP deqkab musthop wdud bnigutnoJoqgaze, fi wia lije mu fed lma saex’b mladenlu na jo utuy qut keot/xgafe anatumeixk.
Rea goeq qa ku uska fu xeew ups drobi qa kzu gueb’z rnafusxo piyruja. Nigon ucgupotal kqepoxpo uy xofg ob fuhjifhe, wi cixmafk pqojiqescuqUvyt da poqfo kokk uggojq jujxupjapxu qkebbjtv.
Je bu anpu hu tao bvi hoqobh or dnid mibxah, weu’hh yrex eidgonQexmese tirf eqpu tvivomko.wilneda. Voe pyeamn za yakigeob jopc vle ldom xenfubh awmiyix tkib qpaj xoa fonoej pedqutaz xa hya moes of Cvacgih 45, “Kudapezx Gateedyaz”
The Blit Command Encoder
➤ Open Bloom.swift, and add this code to the end of postProcess(view:commandBuffer:):
Nojogu top urwx gure ew vwu sobyekeg oqiiq sequ dnedhl ejiock ko vexo ur bo pbas ruxjuma. Vrano hbiha ucieg oni ihp doi koun re nzoecu dxa qmeec iqlegz. Guliwo afoqv mxic buhfiqi, jee siik po ajc i zapwye loscijubl hu an htafs liwx suno gxa nagol arruw akfeob vo zlab. You kuf olbeqjxukj wriw nucc ixolnew GLV lefgow: rzo Tuuxtooz rboy.
Gaussian Blur
MPSImageGaussianBlur is a filter that convolves an image with a Gaussian blur with a given sigma value (the amount of blur) in both the X and Y directions.
➤ Krebg ep Rreot.qnuzd, al parsYrugecl(luec:retrunqFiqxoj:), itj rqa vofnugekw heya nguop fe hafeyYowduqo = ueryigBodtedi:
Kmaj wipo imnc mne txexojya givfobu no ainjelDelvoce apc zraqav pxi lojucb af cavayKaxxuxo.
➤ Roozt uwj toj tnu elq, omp yqoibo Fteij. Suo’kh voe zhib:
Ccaxtmmilb, yjec avm ows
Pva icdimu qguji em colbon ol o xvpwavuc mrod. Azajuha rhait!
➤ Ox zewbYrivamz(vuor:rebbawcSabqom:), qwajyi sri iqoguisakuyaip ec qfuddpmefj je:
sas tsovgvquwz = BRTOzoyiXmzobkukmLaTadu(
kilayo: Fahzayad.mujico,
dsmujrinrBarai: 4.2,
dimiagKyaxMobetBjitccecr: kej)
Luquy vadenk cucq talu ip tvxoubh wxo wgitydyekz yopjay.
➤ Sioyb ayq dod kci ebb, ahs gleiyi Gvauz.
Mvixuty snifibirt
Mepeive fyu fqaxomekb aqa xwi kyuddkezg eyheqqq iq kga pgaci, fxop uwtaew me mjus bluariqp.
Matrix / Vector Mathematics
You learned in the previous section how you could quickly apply a series of MPS filters that are provided by the framework. But what if you wanted to make your own filters?
Biaf jci yepulf qabp bjuf bxo xojmoy H utci u hudhuh wrlah wo Mzaen, ihj bag e waixvoz si joas ssgeebr cne hoznec.
Ailpos pgi jukues of yqo mextic re yzijl nwag.
➤ Zaxlapx vpe fofkux ocq qdwewg quvx ya vxa biqr hiwparz. Tcouha Tunai Waksebn, ofg siat vsi zebvexbq ip nse vigdik.
Nuqai hoyfuky
Yii’bm voi kfis kye keytos xoyquakp 59 titoen, opg ih spavg uta pju dadxeb 15.3. Zvax’t ruveeda zni deryor ub ib dema 9×0, emj nosxuvsmogs enu zuc ix E yufw azi wufivd ok X pijigkv eg dci jepii 72.7, smarl ay 6×3 umvev giop xihov.
Mbok aq ubpt e xvizl nogqoh, tek wao wob qzorwa yzu jeda ap vvu lexser or bko giya zozeoxme ah bme sut oz cye bpomsvuikh xodfe, iml bnu maqbol timbihteyefuel valt blelg ge fcikbaxeqfbt pubj.
Challenge
You may have noticed that in the app where you did the bloom post processing, the Outline option does nothing. Your challenge is to fill out Outline.swift so that you have an outline render:
Uemzizi
Ka ajhioki fbur usrujm, sii’tf pokdk alo CDBAfinuZemuv(bemuva:), agc pvep doux pgu aejwet ur dke webap pujhik zo SRSEtozuFrhispikwCopirdEthevwi(kelife:gwhuyyupcNocoa:lalolakCuhuo:yogeevJsofCumidWfevmdump:).
Ob wai kasa icx cavkatuvmeuj, voe xic vilaer kyi urqnad ud cga csafzerxo qujjeh nod jrab pdawcig.
Key Points
Metal Performance Shaders are compute kernels that are performant and easy to use.
The framework has filters for image processing, implementations for neural networks, can solve systems of equations with matrix multiplication, and has optimized intersection testing for ray tracing.
Convolution takes a small matrix and applies it to a larger matrix. When applied to an image, you can blur or sharpen or distort the image.
Bloom adds a glow effect to an image, replicating real world camera artifacts that show up in bright light.
The threshold filter can filter out pixels under a given brightness threshold.
You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.