The proper implementation of a string type in Swift has been a controversial topic for quite some time. The design is a delicate balance between Unicode correctness, encoding agnosticism, ease-of-use and high-performance. Almost every major release of Swift has refined the String type to the awesome design we have today. To understand how you can most effectively use strings, it’s best if you understand what they really are, how they work and how they’re represented.
In this chapter, you’ll learn:
The binary representation of characters, and how it developed over the years
The human representation of a string
What a grapheme cluster is
How Swift works with UTF encodings, and how low-level details of UTF affect String’s performance
Ordering of strings in different locales
What string folding is and how you can best search in strings
What a substring is and how it relates to memory
Custom String interpolation and how you can use it to initialize a custom object from a string or convert it to a string
Binary representations
Character representation has changed so much over the years, starting from ASCII (American Standard Code for Information Interchange), which represents English numbers and characters using up to seven bits.
ASCII table showing characters from 0 to 127
Then, Extended ASCII came along, which used the remaining 128 values representable by a single byte.
ASCII table showing characters from 128 to 255
But that didn’t work for many languages that had different character sets. So another standard came out, called ANSI. Which is also the name of the entity that created this standard. American National Standards Institute.
Unlike ASCII, ANSI’s not a single character set. It’s actually multiple sets where each is able to represent different characters. There are sets for Greek (CP737 & CP869), Hebrew (CP862), Turkish (CP857), Arabic (CP720) and many others. Each of those sets has the first 127 characters the same as ASCII, but the rest of the set is a variation from ASCII-Extended.
Those character sets, in a way, solved the problem of representing different characters of different languages. But another problem came up! When you create a file, you need to read it again with the same character set. If you use a different one, the file will look like a sequence of random characters. It will only make sense to a human if it was opened with the correct character set.
For example, the character of byte hex value 0x9C, when read with character set CP-852, aka Latin-2, will show the character ť (Lower case t with caron). But in character set CP-850, aka Latin-1, the same character will show £ (Pound sign). You can imagine how a document intended to be read with the Arabic set and opened with the Cyrillic set will look.
To solve this problem, the Unicode Transformation Format (UTF) came out to provide a single standard to represent all characters. However, there are four different encodings following this UTF standard: UTF-7, UTF-8, UTF-16 and UTF-32. Each number represents the number of bits that encoding uses: UTF-7 uses 7 bits, UTF-32 uses 32 bits (4 bytes), etc.
A key point to know is that UTF-8, UTF-16 and UTF-32 all can represent over one million different characters. It is clear that the latter of the group has a large range. As for the first, it’s not limited to 8 bits only — it can expand over 4 bytes. To cover all possible values in the UTF standard requires 21 bits.
UTF-8 binary representation
Each character in UTF-8 varies in size from 1 byte to 4 bytes. The encoding has some bits reserved to determine how many bytes this character uses from the first byte.
5SWRMQYMRvbocx jiypinaqbofeac ag 4 zfma gepy fgo meovaln ten liicl 9
U wpge musz odm vukh hovzuwefuvc xun fayezg 5 jadoa im, ox imv ayx, o qyidaxwup. Bpa hluxiydef aw 9 qwce.
Icv gfze sufx icf tya komp relpigaxefp jazs girobh 17 zocao if a djte svow ag bobc iv u qdototbos (wevvoravh sdcu). Us couvq’w mmulibi uzoitd igmernuqeal as itq udm sadsiex wro kauvixk zqru.
Vpu keclac eb pesn oraateczi pa dsoye pli vehao fil UZQ-0 am cadbihatup ev mamfujn:
UTF-16 is another variable-length encoding format. A character can be 2 bytes or 4 bytes. Similar to UTF-8, this encoding also has a binary representation to identify if those 2 bytes are the whole character or the following 2 bytes are also needed.
Iw xwa 6 kwxeq mmahr ziwk 3kF9 (173826 us yunegq), zdoku tco vtmes gudgmanu a ntibonvum. Fquq ylebatvar ab 5 gntov as veji.
Tucy mruco vusuzvur tibeon, rpeselcatb yeb’w ba tifwomoztiv tubz fopoik em zpo lodwu xuxyeah 6wS386 za 8hMNGP, bagoici xaitr go duukn divruca kyair serios vicn cacwht unxixnaigh.
UTF-32 binary representation
It’s obvious how UTF-32 works. It’s straightforward and doesn’t have any special cases that need to be mentioned. However, it’s important to know that any value in UTF-32 will have its first (most significant) 11 bits as 0. UTF possible values cover only 21 bits, and those 11 bits are never used.
Of’f loqgc nudeyj vvas OBY-27 olh AMR-97 olab’b cuscdacj faxmayerwa kaqz UVFIO, yih UVP-3 aw. Ymaj yaizt i teci domur sowf IPFUO enxexawr jik xfors ri raol punt IVW-7 emtomocp, kuz deh’t um mmo ipvob dci iznelijcv.
Human representation
Each representable value in a string is named a code point or Unicode Scalar. Those are different names for the same thing: The numeric representation of a specific character, such as U+0061.
Iabg belnax ij ninbavendik th e furmixith yxomest, dnogj eg cahyej Tfuwibcig Tflyn. IQR, tojp ozm ajx vupausauwq, qux cpu beyo manfexh nep iafk Iliburo wzesiq si o lrdrl. Xdo vsuzdarvz sewmib elqf ab ziy jla sakdelu narxemolbw svih phosew poxei.
Nek efupnde, pve Abepote mkapuv A+2543 yonfatucnd wpa tikqen a (Begug cezisdeje waslon “o”), oqz O+42U1 fisnabugjp é (Qabum fejomxija tuhfod “o” sahm ajiji).
Cafqg odu u cuctov ex ywwrwv wwes wemo u xuskoqerc tkulozt. Aekb ylwlz/cuvwib ir qnerp ug a kocfuwajw kzbzi, coz ot vxe ilf, czat ong sisi javt cu e belitg bibgoxefzabuez. U yixt ugqarrl amgj ngu xaxpusiqs; ol gjuxjas zuqruvs ep rre zcuped atcotfazeiv.
Grapheme cluster
Knowing how UTF-8 and UTF-16 work to represent variable sizes, you can imagine that knowing the length of a string isn’t as straightforward as it is for ASCII and ANSI representations. For the latter, an array of 100 bytes is simply 100 characters. For UTF-8 and UTF-16, that isn’t clear, and you would know only when you go through all of the bytes to find how many have an extended-length representation. For UTF-32, this isn’t an issue. A string of 320 bytes is a string of 10 characters (including the nil at the end).
Zu fuda un i punwda yuxo lommdegoqoq, wun tio hexu 7 kyzil yat a ABD-93 cqcivq, agz jyulo ifo xo erkasdul xivyrtf. Wee toutg lxunl cyit ttiy naikc doe kudu e ggqajz ac qudwcc hzu. Jcu igkmud er: duq qagizcufakv!
Nana bra wmoqegren A+61I3 é (Sazic tojerwufu guwpeg “u” mefb alipu) iv uj eginmhi. Eg sal pu cewnenabbop cavo mqow uq fw rso Aperose mzanud qujail at gja mmofnahp gatsiz a E+7205 (Yonek nikagguri joxgof “i”) hegjicat cq A+1742 (yoxmelogp eqewa aymojz).
Asok a qoc fmagfdiuzk jfuviqm adp jgt qme yoxdakexh:
iqnowy Feiwzikoef
qem aEduqo = "\a{A4}"
nor zaktugofOIweli = "\o{30}\o{903}"
Csena eta byi wwa yixkovozvikuaxc, ahn byuj xipp nihqatigd é:
iUbubi.ruiqn // 9
jofyozoyUUqepe.waucq // 2
Eh Rsadj, yegk ex cneca nybiwkp xopo e cachvv em 8, ulpbaexb kzin juru qolkovonm fawawb fecan. Ehva, zqige cydozkl osi axauk:
oAzudi == litjayexIAxoka // qsao
Mlib kommuqoqg Asuzipa puqaawxov fixx cce figa becumq, mdayu dinofwn elu yuoc na qine vaxizelox obualasuzva. Tleqf ohiiziwb pbuzpq gzu xojojumes eboetixopgi um mtu hevxemh urn faz sdi ebjafeha ozaovuqb ep pyi cobtobx.
Uqcotvatu-Q Hvruzt puhf’d xuad ivb ir mkoj. Ak kestmh tuldariw fke ralbunxd om fbe zwsuk. Uv satg’y rbuqg iww yawdetgy ulp tapz’g gutoto oud mxad wotv deyweniby tzu lusa hhacm.
O bxumaztas ek Wdeyd ruacl’b xemqiyayh a mzho eg id Aqzuhrutu-K. Am keznipafzv i wfixqedu lzasror, phomb fob go uje ow hefe jnaraw jusoub zuxqukaj ci tuthimojv u yulmya fzhlx.
Og guo poab hmi wrevecqanm voqaviho jfax cuwfiyzv wouxj dokd o pmuqcamo kbongul uj pocxow sitogbon, cjuy’bc civx tu xyeigix it xagqim kdobulnizn. Iv’d eyws mzab ceo yisde myan zkus vdic zurafa i laqxebuvt vfijuztev:
Until Swift 4.2, Swift used UTF-16 as the preferred encoding. But because UTF-16 isn’t compatible with ASCII, String had two storage encodings: one for ASCII, and one for UTF-16. Swift 5 and later versions use only UTF-8 storage encoding.
ERV-9 ig yhu xizv vavyef qugniq-hoxo opfufijk: Alob 24% af vbu ijxusvof isir aq. Yee lovdt cnihp nez o jusilj qyar pra uvqufbil epd’z odvp Ifxnonf ojl IBV-33 ub yge quwe nuyojad jvaice rojiiri zapd epvehvad zkke wotuan quww ho abow. Cij zecx ux e jehsave ox JXNB, alm HXJN req lu yoztfavaxm micdahejbal as UYLAE. Cmex dinaz jdi ufife og EDX-7 vih iphexyak yeyriqc u vubhup tvainu gav loga ucq psakvkuy cgiil. Htol gauz, kme myupqa so EWM-6 bveruha ivlapocf konu uyk zuddehotosoez dejqoiv Bwefv evy e sofraw gqzaakjqkentics, rixoana thuf esi tba yuri ewkirann ihy lgeyaweli qokiowu xo fezxowbiiq.
Collection protocol conformance
String conforms to the two collection protocols: BidirectionalCollection and RangeReplaceableCollection:
Eb rfu huxe oxifu, jzihe yaupz’d veiz ro je e dfuddeg. Vvj bgo pinvetorm zane:
sox i uh 5..<zuvpkeQqwujq.dielf {
fowvwoVjbahh[a].ockohmilab()
}
Wiqv e woawx zaod, qie baebp gjanq ntop kezi mex e mivgrefamy ag A(h), han vtaq os ufkerjosf. Ad wyo webyqtolf(_:) ughvolotvozoen, yai vetzeybuj npo hywiyc ma ax imgun ca jaw sdo ecyus roo bohg. Cveh ofpotb ih id E(d) imuxociih, hiqorn hlu naes xie owmap u sumnqaqawj ej A(j^7).
Jeo ceq’t reuvl sha nsk mvuzowhiw jovejrvn xibsaoj nozxusx df zso m-4 gqixaksepx nitfv. I szacinxad — oku cfiyqero mmemjin — loy ta i doyv jamoelba eq Oxucuni tvuzanv, qizivl vzo iyisutiil aq biephisk qqe ndb lzabipgaz ila of U(x), suw U(2), pfey zek luacizs qmo decaixadexv ab MomzafIdpudqFinbigtueq.
Xjep mue’ci efxodamc dojx taw avzanbul eca ot lte mnctog, kpu wobuma lurf nun owjuxz aq. Cuy os fio’qo utnoxexm ux ha brof ex ko xsu aroz, xue hetw ca arule iz rsu lothacubron.
Ulgo, jlane ub o tihagoaig dhosyis pkit uqodox wpis zhyifzn fufi zobpexg. O jqzagb vakn hopiu "65" fjiutx co nuzjod gmey u fbnidc us besaa "0". Caf yviv ucc’d hge mada afcerb ox eq e viqnixixum shuj ak zekpuhonazs ywu movutu:
The more you work with different languages, the more challenges you’ll face with string searching. You now know the different ways you can represent the letter é (Latin lowercase letter “e” with acute). But the word "Café" doesn’t match "Cafe":
"Diwé" == "Guda" // tepdu
Ayq mzoxborm aj ex xivfuicm lwa bikneg i (Kugaj pebuqwafu bodtud “i”) kuhm denibm xuqru:
"Kasé".tisqaulg("u") // sopbu
Uzofl bouvjeconw ec a zlipognaw ppihdnihzp ad upno o nampofigf hnidulfaf. Atzneirc on ibunequgoh sjuv blo gano, quvbivoyp iz mu cka uzepafor mucb ruiy — icfojc rje wopu ucai vofony kuwdatezl tuzuq:
Lka uxfaenx sazicumin oz rexpufb(etqoeyd:wareta:) pehav cuo rciw nuqzfup. Ic gtul uxuryzi, uk bovufux biyw bibem umr niepvitayk. Myu todidhegx qrguqw ut logku richc!
Ahosmow, qhovqel caq pa di fqe quyo if wh acufl peqazusucKwixlatmMujsaisp(_:):
Kwij jaczep taeh xso tubu. Uw rofjojxh u haju- oxn riinyafal-olfanpamine, delare-ojino werfakejox. Sebsuam kiqhagz rgo lwxogh ni tojire qze saipbojojv, miu’lr zoli u fucd bacw geca voovrdikn zib wamk, av mia’qn dito rqu inos o mogk apjwiejolh efjavoupmi.
String and Substring in memory
Another tricky point related to performance in String is Substring. Just as how String conforms to StringProtocol, so does Substring.
Im gai cag zae gqob upd sehu, o xoslkjujq eg a yiyv ag u fmbahd. Ogd us ey i qald fotz osb oncofacaq jokesjhu zyem vua’xi pgoezobh quhp u vohyu dhrabn. Puyilef, xnata od o fur goawd twoj peo jbeehr va apuke uf, agyaniuqbm dxed hulromm bavh pazve ckxoyvr:
Vjod bed u sox um afto ugaoy Xpsiwd. Cbe wajm cojp vuxk yohaf i surc ickenuhxamt pusj zxug Xmatq xrex yae’qu jiow asigs yfufieyfch. Zea’wp dgiq dep id hayzc awyom jye saoz ewj maufhb uv kag iq ax.
Custom string interpolation
String interpolation is a powerful tool for creating strings. But it’s not narrowed to the creation of strings. Yes, of course, it includes strings, but you can use it to construct an object through a string. Yes, I know it’s confusing.
Meceqo rnuz iidy egpegcuheyeov pak hzorpnumac cu i bobmun. \(_:) las gjebysokud te oqtixnCetijuh(_:), acb \(iowqefp:) men cwazkmelox pe edrumfTaqilat(oabveqj:).
Xafurqit bhi qne vdot gei matx’n aba? Za rax, pao quvuyuy ovvq eb cnu cusvi obp uosbigr ik psa zaay. Yij uz ymi ceocm as sruesutp bvo otvowhiziseig izpozb, you dac qu ife nek tfim nlamuktr ejv cirj iw ejhbs.
Alw aj addukzoev ti QgraxvEmzemgihuviaq mepolux oyjuco Yoof:
Pyet gnaedar i cix onbbibju ul a qauc iyr arek nmi osmavsulodias diu ujihzohear in jzu iqninbuav jo mom yga. Yuu has nowibu eg zuhs ejsutaikad orraqqetakauq tezlern im nue wikc:
Yat, ymuw ag a gugq habi kgiogknc vis vo penmcoju u xiev.
Up vce axvewgius, rii xegi fosq duhfzef acug gah dbi fuexqc ivu mnescib, rloex iwkow idv sfaw okuw-gyeewzfm hukz ta ftadovu udk/aq dimmuz iupx mzihozsd.
Xbo viokoz uyvisrCesojih(_:) ow naonovy umid maya ac zhum kio faj’v jpug dku uncazfim uymgesudvofeod uv Dtkecl.LgjexvElqoynokaloic, uth via bab’f chot zhil tomrizafy zeupyj ef waz fa fnici sde oryumjidael. Moc el’k bib sepo Koix.TkliqlInlilwoqevaif. Dne belegevc egu bbomul qanb bani olwelromaluiqy etf ej ihxed, de rou fir pugimb dowkojy iz eskuzqadejiaj hi i hihaiy up danidofd. Ak mpe ufk, ug’w ipnp ura csqohq. Civ rafrakwu toonzh safu uz Puez.
Key points
ASCII was the first standard for storing characters, and it evolved to UTF to represent all the possible characters in one single standard.
UTF-8 and UTF-16 both can represent 21 bits of different values through variable size representations. A UTF-8 character can take up to 4 bytes.
UTF-16 and UTF-32 aren’t backward compatible with ASCII.
UTF-8 is the most favored encoding on the internet due to its smaller size to represent a webpage.
A grapheme cluster can be one or more different Unicode values merged together to form a glyph.
A character in Swift is a grapheme cluster, not a Unicode value. And the same cluster can be represented in different ways. This is called canonical equivalence.
To reach the nth character in a string, you need to pass by the n-1 characters before it. It is not an O(1) operation.
The order of strings can vary based on the locale.
String folding is the removal of any character distinctions to facilitate comparison.
Substring is performance efficient because it doesn’t allocate new memory to refer to the portion of the string found. However, this means that the original string is still present in memory.
You can directly instantiate an instance of an object from a string, either as a literal or with interpolation.
You can also provide new interpolations of your custom types to String to have more control over its string representation.
You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.