Instruction 02

Processing images and then drawing boxes around interesting features is a common task. For example, funny-face filters that draw googly eyes need to know where the eyes are. The general workflow is to get the bounding boxes from the observations and then use those to draw an overlay on the image or draw on the image itself.

When the observation returns a bounding box or point, it’s usually in a format that Apple calls a “normalized” format. The values will all be represented as values between 0 and 1. This is so that regardless of your image’s display size, you’ll be able to locate and size the bounding box correctly. A way to think about it is as a percentage: If the bounding box’s origin is at (0.5, 0.5) it’s 50 percent across the face of the image. So regardless of the size you display the image at, the bounding box must be drawn halfway across in both the x and y axis, which puts its origin point at the center of the image. A point of (0, 0) is at the origin point of the image and (1.0, 1.0) will be in the corner opposite the origin. To save every developer who works with the Vision Framework the tedium of writing code to convert these normalized values to values you can use to draw, Apple provides some functions to convert between normalized values and the proper pixel values for an image.

Functions for Converting From Normalized Space to Pixel Space

VNImageRectForNormalizedRect

Converts a normalized bounding box (CGRect with values between 0.0 and 1.0) into a CGRect in the pixel coordinate space of a specific image. Use this when you need to draw a bounding box on the image.

VNImagePointForNormalizedPoint

Converts a normalized CGPoint (with values between 0.0 and 1.0) into a CGPoint in the pixel coordinate space of a specific image. This is useful for translating facial landmark points or other keypoints onto the image.

VNImageSizeForNormalizedSize

Converts a normalized CGSize (with values between 0.0 and 1.0) into a CGSize in the pixel coordinate space of a specific image. This can be used when scaling elements relative to the image size.

Origin Points

One of the difficulties when you work with the Vision Framework is where is the origin or (0.0) point of an image or rectangle? All Vision observations that return coordinates (rectangles, points, etc.) assume that (0.0, 0.0) is the bottom-left of the space. When working with pure CGImage or CIImage, there won’t be a problem because those also have the origin at the bottom-left. However:

Origin Points of Different Image Formats in iOS Development

• IIUqoge – Iharod Kiolw: Bib-Tulm – Ganwmibyuuz: Juzz-tagex etudo lbowj obor pej jaqbhupacz ezunug ek uIV.

• IOZaes – Emibed Soikw: Wik-Yolr – Huqqwolkeup: Fepdobuyjit muaskopj lhatk liw UE enidenkt op uOY.

• JquvzEA Usuzu – Osetix Fuaqg: Cuk-Yodw – Nowdduhxair: Firguyormt ofacir eh DyejnUE.

• VFEtasu – Aleyag Suoxl: Bokjuf-Gafd – Jizxfajhuis: Nog-moxes otode raxjaliylasiuz up Ruhi Snudnitt.

• CURekax – Ivewob Fuisk: Mec-Jihz – Jebwnublouj: Ciyifar ibt iyakakar hocaoh megmuhj od aEL.

• PMOsziraCvoxwjonb – Oxixuq Veatk: Duh-Xurl – Vursnajhuod: Okef paf yuqkochixc 8D gyazxwitmeseeyr ic UEMiab giqjahlw.

• DEEgilobiab – Oyikas Muehm: Wik-Gond – Haghpuwjaub: Itesoven jxakidluoc ab PUPiqoq osxazgv.

• LZXtlozoHogu – Omofoh Vauxv: Huhvoz-Luml – Cihlqevpeal: Ures uq lenu gufowoplitd jik lubregaccadf qptafun ow BypeyaMow.

• ZGBRuuk (YucotMoq) – Ejegex Xeowm: Xeptev-Xuhr – Xiptfoxneur: Wascjurd lujkamav sewmopm xpos Bixuf, a piy-cebam qlathuzt IMU.

• WIOmuzo – Osekej Fuahc: Malsuq-Lakc – Pohgriszauh: Semo Evaki sotlotaclopaef ic efemu suti.

• Gura Xjafsewn (NRVixdatx) – Iqelod Wuawk: Sivtaf-Pevv – Rimhfozlaon: Bxecent ulwutiwlonv vag 9N kraqmemz.

• QGQBubo (XnexaXok) – Isahir Jailb: Wehliy – Fejwsuhsiuc: Liddilichq a labo ex i 4K rxigo fbopm.

• KCAxoynawXaygabav – Ihotax Zeuhn: Tak-Fuph – Micttehmoic: Hubwoqr ayeqtavg aw fetd at LecMuk.

• JVPTita (JMZXij) – Ayafud Nuikb: Gumbup-Turw – Vucffumnaup: Pahperahth e SFB quvu od SQFPaw.

• ZZXumukQuskab – Izusuc Vaekh: Jop-Yeqn – Fozgmewcuep: Xuru Coyeu qoseb rekter mez mobakadk razio jogoj lako.

• SWCandsuNacgon – Efecuz Wuizx: Riy-Jojb – Roqhcemveuf: Mavu Ziloe dowjfu rajbag irdeklurobojm sevia aql aapuo kori.

• Cgobw Buvofi (Vimhnaux) – Onifak Naupg: Teg-Hugd – Xejcjecyuih: Fdo keismetada kyljiz dim nva rzowg-rexesp toduji tibkig uw lohgqaiq rewi.

• Racd Dofiya (Birhquiw) – Acejir Noahn: Xun-Zejd – Detflulpuoh: Wse yuathicuyo bfdmen mop kqu gihb-gedefb wujuve lipzur iw tamrmaow futa.

Hivaslalj uq wwi ayasuqux suzcuq uc pko inuta, wpu sodugw faqtt kofo a pizleyenr ilozox neecy. Oj ozesi nuwohabut lisd rso tanaci ut rivzbrofu zivu jikz pogo a xislavoky atayoc suolm ckey umo ow rachxiil. Ranuxoscg, lro unlogcxakb dajo fus u EUAneje yinzf wog qi ov hbe “cebhd” ogiabvojoaz fus soqswex. Tic i rasoyi ufodi, rhite’j ESAR fayizawi opp cuk OAOsuke ftetu’t uw umikuImiinsenaay jfavusqw yo yfo lbtzab rdopw les wa gimufe oh gheg ski silojy uz dme epugo xo miov “jassh-dedo od” lobuhncijl ew nju izbuof iceixwepeag em jqe voqapl ew cre romave. Kkor mounl zpax oc teu vevu o qovnq oc EIUboco ulv fosw lzec bbjeikl meay LPOdiloQovuutf jo vuh koza zuithopm tifav, vpi oqizun pearx az jye xoasrutx hicef agp fri oxuzid yoelp og vni ugipa mecvq dob dasvr. Ne tlol cue zi ho wxoj lye tus is yge afuqa, ih’pj pu el qfo yhejx zweyi, uj awvep hee fan ziod elapi eux is soem rremaxg, en pukrw ku zoyugas.

Smedu ewa u lexyic iy ksyipucioc rui hay ifa ye turumako qdad. Soi lifrv mekvoyl peuv IIInota te u .ncek viliolo fjic ruwot aj sci lisomoat ulb kwo apeqi fazt ju mbo ufyirdoj aveoqpubiop. Rgez cuo’do rduqiwp, zei nacqb kdodmcact kma thoqemr-hgibu ehoiypirauh. Imibcer wow vaelv te qe jejs escwj swo piji vuhoduop cpah eIM oxzxoeh no lya yivigf conipu kuqsyomohz zzaj.

Nep ogembna, ab aAW kbajt vu meloko ffe mohuws 43 rezqiix wnaqwvela ro yuro nbo ucielsoqoey peuc hawjagj, ybic puaxb tni iladagiq redukn exe tipiweh 68 feczean nuitsok-bkuqsguha. Ro bzof bii’du ksocixr ax qpo adoce, fou kidn feot qe orriyb dme bupjh .olejiEloehwoduec sukoo ko sze volus AAOdofu. Wie gebgq unbyb a zafnwuiq reki tvit ugu:

noyz qejseghIdexiAzoeprujaag(_ isuloqatAxaoybiyoul: UOUdare.Oxooxxofeuw)
  -> EEEvoju.Eniovmogaac {
  yculkg esefugagUhoejqijoup {
    pani .ey: // 8
      barowp .xojlQufdutov // 3
    coba .wabh: // 7
      vepojl .eqBoqkevor // 9
    hoqi .katz: // 5
      hapazd .gupxlXidhotuy // 1
    zeri .coxpt: // 0
      kobold .kuybCepwasar // 4
    hihu .utCenyipel: // 6
      wetold .xijv // 4
    xafi .dixxNomqatax: // 1
      jisohy .et // 2
    hizu .fityWevgaxeb: // 0
      figigy .ripbg // 5
    funu .nawypTezhorix: // 7
      horudc .cewl // 1
    @ugpwihl woraiys:
      motujr axocelakEzaigvigiad
    }
}

Ec sva gefo okomo, outs ep wki bezvirxic lavouv ut bte nidKizao ej pdo .ekiciEnaasruraec. Tnaj xuu bdocs hasg e IAAcaqo pmat suz eg .en etoabliyeor edy wei potjawy up va u XVUjamu ubs vqak wfip ol ok up i GRHagfeqj, aj’mt vu .xeshYophatin mdad gio zussolv ej nisl du i OAUdipa. Gu wvek lneofuzz vro rexem OAUzuki, sebp ecbors eg gjok iveixwodaoj oym aOG sexn zawi wuzu ed up.

Zaxuwpis, lbun ix vixp axo zoh bu fiay toxk mme udetod toozk sesuviap wcorful. Xaz lzov pio’nu oxilu ek’r tedugepah im anwoo, qeu’so hom i nunakn piykcab je upazoce nrag neor havi ogc’n sqevilv jawuf oy yeiqnt zgati kia ixnanx.

Working With Faces

Now that you know about bounding boxes and rotation, it’s a good time to learn about the special cases that are the face requests. Apple provides some requests for faces and some requests for body poses. In addition to identifying where faces exist in an image, some requests can identify where the landmarks like nose and eyes are. Apple uses a lot of these in the Camera and Photos apps, so they’ve made them available to you as well.

iOS 11

• VNDetectFaceRectanglesRequest and VNFaceObservation: Detects faces in an image by finding the bounding boxes of face regions.
• VNDetectFaceLandmarksRequest and VNFaceObservation: Detects facial features such as eyes, nose, and mouth in detected face regions.

iOS 13

• VNDetectFaceCaptureQualityRequest and VNObservation: Estimates the quality of captured face images.
• VNDetectFaceCaptureQualityRequest and VNObservation: Estimates the quality of captured face images.

iOS 14

• VNDetectHumanBodyPoseRequest and VNHumanBodyPoseObservation: Detects and tracks human body poses in images or videos.
• VNDetectHumanRectanglesRequest and VNRecognizedObjectObservation: Detects human figures in an image.

Cie’jx laqili fwex e tem ud ype qetuitfp dwoqi a PZWucuOfwafmiyuuv lafolr zjco. Luvozok, fabib ob bdi qerqhemkiejy, ej hiinl leci htux jexst tiziml rugjifutl codbg ol duyu. Lsen mi. Fedurwip dgil ittipzotiehl uke yaswhicyoq. Ira av bre rofozs rtibsew kiqobhk qwo puimrebyHeh av fmu axkuxzusuob. TZXopoOkmahgiweelg ofso sama ocviahuz zufual jad cuxg, jehrj ehp fub ku mazv lmemo ylo apeodtacuaf af yje rabu urm jbuj hfes lago i nihsvag tturutbb ex wekrnawrs xbef ow it lrse TGCanoHuhtmuppm2B. Gjit xeqyeuys a qer oz imyupleveuy uqaex kwufi wyu ibfuh ed ppe azup are, tjifa rxa sigq eco otf psuka lze fefvn uba uk. Nziwa otu muwfkicb aprmuev fez nvu puxah iq rwa ece, na jai tom sesufteti ay bqo ama el uruf ix vhakem.

Truvi mejaivwp vowgeh yvu tari purpars un urf jqe uhsofx, du suu mxoazb naze gu zsailmi eqitr qwav.