Umberto Raimondi's Blog2021-05-28T08:39:51+02:00https://www.uraimo.com/Umberto Raimondime@uraimo.comA Long Overdue Update on Swift 5.0 For Raspberry Pi Zero/1/2/32019-04-11T00:00:00+02:00https://www.uraimo.com/2019/04/11/A-long-overdue-update-on-Swift-5-0-for-raspberry-pi-zero-1-2-3The current status of Swift 5.0 on ARM-based boards like the Raspberry Pis.<p><strong>Update 3/2020:</strong> <em>5.1.5 binaries <a href="https://github.com/uraimo/buildSwiftOnARM/releases/tag/5.1.5">are now available</a>.</em></p>
<p>It has been nearly a year since the last update on the state of Swift on the Raspberry Pi and other ARM boards so let’s start with a short recap of what happened since the release of Swift 4.1.2.</p>
<p>TLDR: After a bloodbath with 4.2, for the first time we got Swift 5.0 running with relatively minimal effort and you can already download it now, <a href="#prebuilt-binaries">see below</a>.</p>
<p>If you have kept an eye on updates to <a href="https://github.com/uraimo/buildSwiftOnARM">buildSwiftOnARM</a> you already know that by the end of October we had a working, albeit with a lingering small bug, Swift 4.2.</p>
<p>This was the result of the work (a huge amount of it, really) done by <a href="https://github.com/kaiede">Kaiede</a> (you can buy him a cup of coffee <a href="https://ko-fi.com/kaiede">here</a> to show your appreciation), that lead to multiple PRs opened for Swift and related libraries and took a few months to get something that could be considered stable.</p>
<p>And it goes without saying that the community built around the <a href="https://slackpass.io/swift-arm">swift-arm</a> Slack channel was responsible for ironing out a plethora of other additional issues and testing on various SBCs.</p>
<p>A few additional small issues were found and fixed in the <a href="https://forums.swift.org/t/announcing-swift-4-2-2-and-monthly-swift-4-2-x-dot-releases-for-linux/20148">monthly 4.2.x Linux-only releases</a> and 4.2.4 now compiles without patches even on 32-bit ARM.</p>
<p>Considering what we are used to when a new major release comes out, getting 5.0 to compile was a breeze (in part thanks to some preemptive work done by Kaiede before the final release).</p>
<h2 id="current-status">Current Status</h2>
<p>Right now a few patches are required to compile Swift 5.0 but all the issues identified have a fix or a workaround.</p>
<p>A new temporary patch that limits the number of parallel jobs launched by SPM has been added by <a href="https://twitter.com/hpux735">hpux735</a> since from this release SPM spawns a number of compiler threads equal to the number of the cpu’s cores every time you build something. This behaviour, not really optimal for platforms with a small amount of RAM, cannot be changed yet, and so for now we are limiting the number of jobs used by SPM to 1. A proper fix should come in one of the future monthly limux releases.</p>
<p>The Swift binaries provided below contain this hotfix if needed.</p>
<p>Feel free to open new issues on <a href="https://github.com/uraimo/buildSwiftOnARM/issues">buildSwiftOnARM</a> if you spot additional problems and crashes.</p>
<h2 id="prebuilt-binaries">Prebuilt binaries</h2>
<table>
<thead>
<tr>
<th>OS</th>
<th>Architecture</th>
<th>Boards</th>
<th>Download</th>
</tr>
</thead>
<tbody>
<tr>
<td>Raspbian Stretch</td>
<td>ARMv6</td>
<td>RaspberryPi Classic, All versions of Pi Zero</td>
<td><a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/4.2.3/swift-4.2.3-RPi01-RaspbianStretch.tgz">4.2.3</a> - <a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/5.0/swift-5.0-RPi01-RaspbianStretch.tgz">5.0</a></td>
</tr>
<tr>
<td>Raspbian Stretch</td>
<td>ARMv7</td>
<td>All versions of RaspberryPi 2/3</td>
<td><a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/4.2.3/swift-4.2.3-RPi23-Ubuntu1604.tgz">4.2.3</a> - <a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/5.0/swift-5.0-threads_hotfix-RPi23-RaspbianStretch.tgz">5.0-hotfix1</a></td>
</tr>
<tr>
<td>Ubuntu 16.04</td>
<td>ARMv7</td>
<td>All versions of RaspberryPi 2/3</td>
<td><a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/4.2.3/swift-4.2.3-RPi23-RaspbianStretch.tgz">4.2.3</a> - <a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/5.0/swift-5.0-threads_hotfix-RPi23-Ubuntu1604.tgz">5.0-hotfix1</a></td>
</tr>
<tr>
<td>Ubuntu 18.04</td>
<td>ARMv7</td>
<td>All versions of RaspberryPi 2/3</td>
<td><a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/4.2.3/swift-4.2.3-RPi23-Ubuntu1804.tgz">4.2.3</a> - <a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/5.0/swift-5.0-threads_hotfix-RPi23-Ubuntu1804.tgz">5.0-hotfix1</a></td>
</tr>
<tr>
<td>Ubuntu 18.10</td>
<td>ARMv7</td>
<td>All versions of RaspberryPi 2/3</td>
<td><a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/4.2.3/swift-4.2.3-RPi23-Ubuntu1810.tgz">4.2.3</a> - <a href="https://github.com/uraimo/buildSwiftOnARM/releases/download/5.0/swift-5.0-threads_hotfix-RPi23-Ubuntu1810.tgz">5.0-hotfix1</a></td>
</tr>
</tbody>
</table>
<p>Just decompress the tgz archive and both <code class="inlinecode">swiftc</code> and <code class="inlinecode">swift</code> will be available under <code class="inlinecode">./usr/bin</code>. Use the former to compile Swift files directly or the swift binary to access additional tools like SPM (as usual the REPL will not be available).</p>
<p>If you have patience and a few hours to spare, check out the updated <a href="https://github.com/uraimo/buildSwiftOnARM">buildSwiftOnARM</a> scripts to build your own 5.0 binaries.</p>
<p>And for those interested in IoT projects, <a href="https://github.com/uraimo/SwiftyGPIO">SwiftyGPIO</a> has been verified with Swift 5.0, that is now the recommended Swift release.</p>
<h2 id="swift-and-aarch64">Swift and AArch64</h2>
<p>AArch64 should not require additional patches now (other than small patches to select the proper linker on some releases of Ubuntu), <a href="https://twitter.com/futurejonesapps">Neil Jones</a> provides downloadable prebuilt binaries <a href="https://github.com/futurejones/swift-arm64/releases">here</a> or the same binaries via more practical <a href="https://github.com/futurejones/swift-arm64">apt packages</a>.</p>
All about Concurrency in Swift - Part 2: The Future2017-07-22T00:00:00+02:00https://www.uraimo.com/2017/07/22/all-about-concurrency-in-swift-2-the-futureDiscussions on how concurrency should be handled natively in Swift will soon start, new paradigms will be introduced and a swifty approach to concurrency will be defined. This article is an introduction to these topics, it could be useful if you plan to contribute to swift-evolution or even if you just want to experiment with something new using the recommended opensource libraries.<p>In the <a href="/2017/05/07/all-about-concurrency-in-swift-1-the-present/">first part</a> of this series we have seen what you’ll have to deal with when working with concurrency, how to reason about some typical multithreading problems, what you’ll need to pay special attention to, and we’ve also looked into some of the traditional instruments we have at our disposal when writing Swift applications.</p>
<p>This time we’ll look into what the future could have in store for us in one of the next major releases of Swift.</p>
<p>We’ll analyze some popular paradigms and abstractions used in other programming languages highlighting their strengths and weaknesses, trying to envision how and if they could fit in the overall design of the Swift language.</p>
<p><img src="/imgs/concurr2.jpg" srcset="/imgs/concurr2@2x.jpg 2x" alt="A future" /></p>
<p>This article should give you the introduction you’ll need to take part in the discussions on <em>swift-evolution</em> that will revolve around introducing native concurrency functionalities in a future release of Swift and give you a few pointers to start experimenting with new paradigms using the open source libraries that are already available.</p>
<p>Some of these paradigms will still tackle directly the problems related to sharing resources in a multithreaded environment, while others will address the same underlying issues proposing a better way to handle asynchronous tasks guaranteeing that each one will operate without external side effects.</p>
<p>Most languages provide a basic set of common concurrency primitives you’ll find everywhere (locks, semaphores, conditions, etc…) but then select a few specific concurrency paradigms to define their own idiomatic approach.</p>
<p>Some of the paradigms that will be described here could be suitable to be included in the Swift’s standard library sooner or later, while others will probably never be.</p>
<p>Each concurrency model has its supporters and detractors and the community is usually very polarized. This article will not discuss my personal preferences, I’ll try to give you a good overview of the most popular models and I recommend you to invest some time in testing them to see how they fare in the real world, maybe building some throwaway application.</p>
<p>I’ll make frequent references to other languages throughout the article and will include some code samples in languages you may not be familiar with, but don’t worry, I’ll explain syntax quirks when necessary.</p>
<h3 id="contents">Contents</h3>
<ul>
<li><a href="#memory-model-and-standard-library">Memory Model and Standard Library</a>
<ul>
<li><a href="#concurrent-data-structures">Concurrent Data Structures</a></li>
<li><a href="#stream-processing-and-parallelism">Stream Processing and Parallelism</a></li>
<li><a href="#coroutines-and-green-threads">Coroutines and Green Threads</a></li>
</ul>
</li>
<li><a href="#futures-and-promises">Futures and Promises</a></li>
<li><a href="#asyncawait">Async/Await</a></li>
<li><a href="#stm-software-transactional-memory">STM: Software Transactional Memory</a></li>
<li><a href="#the-actor-model">The Actor Model</a></li>
<li><a href="#communicating-sequential-processes-channels">Communicating Sequential Processes: Channels</a></li>
<li><a href="#swift-where-are-we-now">Swift: Where are we now</a></li>
<li><a href="#a-few-words-on-kotlin">A few words on Kotlin</a></li>
<li><a href="#closing-thoughts">Closing Thoughts</a></li>
</ul>
<h2 id="memory-model-and-standard-library">Memory Model and Standard Library</h2>
<p>In this section we’ll take a look at some areas of the Swift language and its runtime that could be improved from the point of view of concurrency without introducing new paradigms.</p>
<p>As described <a href="/2017/05/07/all-about-concurrency-in-swift-1-the-present/#language-guarantees">in the first part</a>, Swift does not provide substantial concurrency guarantees and it hasn’t really defined a <a href="https://en.wikipedia.org/wiki/Memory_model_(programming)">memory model</a> similar to what you could find elsewhere.</p>
<p>We know that global variables are initialized atomically for example, but concurrent read/write or multiple writes to the same variable are still classified under undefined behaviour, the developer will have to guard those critical sections to avoid concurrent variable modifications.</p>
<p>The proposal <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md">SE-0176</a> addresses this issue proposing that modifications to a variable must be <em>excluvive</em> and should not happen at the same time of other accesses to that variable. Enforcing exclusivity would allow only reads to happen at the same time while forcing writes to be performed in isolation without other threads trying to read or modify the same variable half way through the original write.</p>
<p>Exclusivity could be guaranteed statically during compilation, checking your code for possible invalid accesses to shared variables, or could be enforced dynamically while your program is running keeping track of the access state of your variables.</p>
<p>The proposal describe this problem and discusses what exclusivity entails for different types, but it’s also part of a bigger effort to make Swift’s handling of memory more safe and predictable described in the <a href="https://github.com/apple/swift/blob/master/docs/OwnershipManifesto.md">Ownership Manifesto</a>.</p>
<p>I will not try to dissect this complex document here, if you are interested in knowing more, I recommend to put aside some time (and maybe grab a cup of coffee or tea) and slowly delve into it.</p>
<p>The functionalities described in the manifesto will likely lead to the definition of a memory model, that as described in part one of the series is essential, among other things, to show how the compiler will be allowed to evolve, defining which new optimization strategies will be practicable and which will not.</p>
<p>Now, let’s consider how the data structures of the runtime could be improved and then we’ll talk about managing the data flow with streams and about some lightweight threading functionalities that could help to improve scalability.</p>
<h3 id="concurrent-data-structures">Concurrent Data Structures</h3>
<p>Some languages have a rich set of data structures included in the default runtime that provide both thread-safe implementations and faster unsafe variants.</p>
<p>The developer will have then to choose each time the right implementation depending on how the object will be used used.</p>
<p>Let’s take a quick look, just for this section, to Java, that more or less ten years ago extended its basic data structures to provide thread-safe variants.</p>
<p>This was part of a larger effort to give Java something more than the extremely basic concurrency functionalities it had at the time, an effort that resulted in the definition of the <a href="https://en.wikipedia.org/wiki/Java_concurrency">JSR-166</a> specification and its implementation in Java 5.</p>
<p>This extension to the language contained things like locks, semaphores and thread pools that you’ll find in libdispatch or Foundation but also concurrent collections, atomic objects and thread-safe queues that are not among the default data structures available from Swift. The majority of those new features were implemented in a lock-less fashion.</p>
<p>While you could implement on your own a thread safe queue or extend arrays or dictionaries to support concurrency, it could still make sense to include efficient implementations of some of these data structures in the standard library.</p>
<p>In addition to the expected lists, maps and sets that perform modifications atomically guaranteeing that all read operations will always complete correctly (using fine-grained locking) and that we’ll always be able to iterate on them even in presence of concurrent changes, Java also provides <em>Blocking Queues and Double Ended Queues</em>.</p>
<p>Blocking queues are fixed-size FIFO queues that have additional methods that allow to wait for the queue to become non-empty when we want to remove a value or wait for the queue to have some free space when we are adding an element on a full queue. An these queue also have add and remove methods that fail if their preconditions are not met right away or after a specific amount of time.</p>
<p>This data structure is well suited to be employed in producers/consumers scenario, where you have multiple producer threads that generate data that other consumers threads will elaborate.</p>
<p>A simple scenario can be implemented in fifty or so lines of Java:</p>
<pre><code class="swift">
import java.util.concurrent.*;
class Producer implements Runnable {
private final BlockingQueue<String> queue;
Producer(BlockingQueue<String> q) { queue = q; }
public void run() {
try {
while (true) {
//Create a new string
queue.put(produce());
}
} catch (InterruptedException ex) {}
}
String produce() {
//Do something and return a string
return "";
}
}
class Consumer implements Runnable {
private final BlockingQueue<String> queue;
Consumer(BlockingQueue<String> q) { queue = q; }
public void run() {
try {
while (true) {
//Wait on take() or consume
consume(queue.take());
}
} catch (InterruptedException ex) {}
}
void consume(String data) {
//Do something
}
}
public class Main {
public static void main(String... args) {
BlockingQueue<String> q = new ArrayBlockingQueue<String>(10);
Producer p1 = new Producer(q);
Producer p2 = new Producer(q);
Consumer c1 = new Consumer(q);
Consumer c2 = new Consumer(q);
new Thread(p1).start();
new Thread(p2).start();
new Thread(c1).start();
new Thread(c2).start();
}
}
</code></pre>
<p>If you get past the verbosity of the language and those semicolons, this implementation is quite an improvement on the classic locks implementation. This data structure hides away all the gritty details and provides an extremely simple interface.</p>
<p>As we’ll see in a few sections, the basic idea of the blocking queue with finite size will come back again, in another, better, form.</p>
<h3 id="stream-processing-and-parallelism">Stream Processing and Parallelism</h3>
<p>Now, let me introduce briefly an approach that you could already be sort of familiar with if you have ever used the <a href="/2015/10/08/Swift2-map-flatmap-demystified/">map and flatMap</a> methods that Swift’s sequences provide.</p>
<p>Swift adds some <a href="/2015/11/12/experimenting-with-swift-2-sequencetype-generatortype/">useful methods to sequences</a> that allow to process their content in a more functional way, turning series of iterations typical of procedural code to a sequence of transformations on some input data:</p>
<pre><code class="swift">
func getData() -> [String] {
var ret = [String]()
for i in (0..<10000).reversed() {
ret.append("i"+String(i))
}
return ret
}
let res = getData()
.lazy
.map{$0[$0.index(after: $0.startIndex)..<$0.endIndex]}
.flatMap{Int($0)}
.filter{$0 < 100}
for n in res.prefix(5) {
print(n)
}
</code></pre>
<p>Each element contained in data sequence will be lazily processed, first removing the initial character from each string and then converting the remaining substring to an integer. Values lower than 100 will be filtered out.</p>
<p>Since we have enclosed the initial sequence in a lazy collection with <code class="inlinecode">lazy</code>, and no operations in the sequence require the complete output from the previous step to produce a result, only five elements will be actually processed to produce the 5 integers we’ll print.</p>
<p>Normally, on a non-lazy sequence, these operation would have worked through the whole sequence.</p>
<p><strong>Stream processing</strong> is a way to perform multiple sequential computations on a series of values, either consuming a set of data (of any kind), available since the beginning of the processing, or a flow of data that will be progressively available over time (with new values coming in every now and then) processed <em>reactively</em>.</p>
<p>Some languages, like Java 8, Scala and others provide those streaming functionality available with Swift’s sequences and much more through more practical <code class="inlinecode">Stream</code> types, that can be obtained wrapping common sequence or collection types but that can be also used to implement streams of data with elements that can become available <em>asynchronously</em>, and that will be processed in a non-blocking way.</p>
<p>Streams have all the familiar functional operators you are familiar with, like map, filter and reduce, and as we did with sequences, multiple steps, each one producing a Stream with intermediate results, are pipelined to obtain the final output.</p>
<p><strong>Parallelism</strong> can easily be introduced during stream processing for those operations like <em>map</em> that do not require the whole content of the stream to perform their duty but just execute the same task on each element, without side-effects.</p>
<p>And this is maybe where stream processing shine, you will not need to break your computation in blocks to perform separately those phases that can take advantage of parallel execution on systems with multiple CPUs.</p>
<p>Some stream operations will behave differently when parallel execution will be requested and all of this will just require from the developer some planning about how the data will be used between these multiple steps.</p>
<p>I will not include examples of parallel execution (since I would have to do it in Java, that has the most simple API for streams) but you just need to know that most libraries will wrap streams in a specific <em>parallel stream</em> object that will alter the behavior of some operators <em>transparently</em>.</p>
<p>While there not seem to be traditional stream-oriented Swift libraries (likely for the overlap with what the language already provide), the most pupular libraries that use reactive stream processing are probably the principal Swift reactive frameworks, <a href="https://github.com/ReactiveX/RxSwift">RxSwift</a> and <a href="https://github.com/ReactiveCocoa/ReactiveSwift">ReactiveSwift</a>, that under the hood use stream of data or events to handle the asynchronous flow of data typical in mobile/desktop applications.</p>
<h3 id="coroutines-and-green-threads">Coroutines and Green Threads</h3>
<p>From the first part of this series, we already know that the concept of executing multiple tasks concurrently predates the general availability of platforms with multiple physical execution units.</p>
<p>But how can we handle the execution of multiple tasks on a single CPU, giving the illusion of simultaneous execution, in a multitasking system?</p>
<p>The software layer responsible for the execution of multiple tasks needs to implement the machinery necessary to share the available CPUs between the currently running tasks to increase utilization, for example to temporarily pause tasks waiting for I/O that leave the CPU idle and execute some other task.</p>
<p>There are two main flavor of multitasking systems, that adopt different approaches for CPU allocation.</p>
<p><strong>Cooperative</strong> multitasking allows each running task to decide when to relinquish the CPU, giving complete freedom to the developer on how the resources should be used but increasing the chance that the whole system hangs because one or more tasks are monopolizing the available execution units.</p>
<p><strong>Preemptive</strong> multitasking instead, takes this responsibility away from the developer and delegates the job of deciding which task should run to a specific <em>Scheduler</em> component, that schedules tasks depending on their priority, but normally allocates the same slice of CPU time to each one, and perform the necessary <em>context switch</em> when a task has used up all its time slice or is waiting for I/O or is otherwise idle.</p>
<p>The preemptive approach is nowadays used in every modern OS kernel (with wildly different scheduling strategies) to allow the execution of multiple threads and processes, making ample use of interrupts to guarantee good performance when handling I/O.</p>
<p>As we already know, the kernel can manage multiple threads and we can use user-space APIs to make use of these threads in our applications.</p>
<p>Multitasking doesn’t have to be limited to the OS layer, no one stop us from implementing user-space threads if we are so inclined.</p>
<p>But does it make sense to do so? It turns out that, <em>yes, it does</em>.</p>
<p>Kernel-level threads need to guarantee data integrity when performing the context switch, making this operation way more complex than one could initially imagine (multiple steps are involved, from saving a well defined set of registers to adjust the processor privilege level). Furthermore, the initial setup required when creating a new thread and the need for a per-thread stack make the creation of a new thread a relatively costly operation.</p>
<p>While with a small number of threads the costs of creation and context switch are negligible, this does not hold true when we start to have hundred or thousands of active threads simultaneously.</p>
<p>The requirement for a stack local to each thread alone would reduce the number of threads you could create in a real world scenario pretty fast.</p>
<p>For these reasons, we can’t use classic OS threads when we plan to have hundreds of thousands (or way more) of concurrent threads in our application (we’ll see in the next sections why we could need these huge amount of threads).</p>
<p>The common solution to this problem, that you’ll see implemented in many languages, consist in creating <strong>lightweight user-space threads</strong>.</p>
<p>These threads don’t need the full context switch required by kernel threads to provide a similar functionality and can operate in the context of a single kernel thread without having their own local stacks.</p>
<p>User-space threads can be implemented following both multitasking approaches described above, but most of the times you’ll see an implementation ascribable to the <em>cooperative</em> category, where data integrity is less of an issue since there is no scheduler and the developer decides where to yield control of the CPU and guarantees integrity across context switches.</p>
<p>The system that manages this lightweight threads can be backed by one or more classic kernel threads (organized in a pool) but most implementations tend to employ by default a number of kernel threads equal to the number of hardware CPUs.</p>
<p><em>Coroutines</em> and <em>Green Threads</em> are two examples of user-space lightweight threads with different characteristics.</p>
<p>Green threads recreate what is available at kernel level, with usual system threads, completely at user level, with a lightweight implementation of threads performing <strong>preemptive scheduling</strong> with a user level scheduler built from scratch that don’t need to invoke threading system calls to function.</p>
<p>Even if the first implementation of green threads was used in the first releases of Java but was later on abandoned in favor of system threads, green threads are now in vogue again for their scalability properties, after improvements in memory performance in the last decade or so have made their advantages quite evident.</p>
<p>At the moment there are no Swift implementations of green threads, but the important thing to know is that green threads have the same API of normal threads, the developer does not need to do anything to make them work the same way system threads would.</p>
<p>Coroutines implement <strong>cooperative multitasking</strong> instead, leaving the responsibility to guarantee that a routine will play well with other concurrently running routines to the developer, that will have to include in the code special instructions that will specify when the closure will be ready to be paused to run other routines, giving the illusion of closure running concurrently.</p>
<p>While control of the CPU is usually yielded to the next routine waiting to be run with a specific <code class="inlinecode">yield</code> other operations that are normally blocking have the same effect. Examples of this are sleep instructions, blocking operations on files and network and many more. Implementations of coroutines usually provide coroutine-aware versions of those basic operations.</p>
<p>The most serious attempt at implementing coroutines in Swift (even if this is not a pure-Swift implementation since it’s built on top of libdill) is represented by the <a href="https://github.com/Zewo/Venice">Venice</a> library that offers them among other features.</p>
<p>Let’s see some basic examples using Venice</p>
<pre><code class="swift">
let coroutine = try Coroutine {
while true {
performExpensiveComputation()
try Coroutine.yield()
}
}
let printer = try Coroutine {
while true {
try Coroutine.wakeUp(100.milliseconds.fromNow())
print(".")
}
}
try Coroutine.wakeUp(5.second.fromNow())
coroutine.cancel()
printer.cancel()
</code></pre>
<p>The first coroutine perform a series of expensive operations in a loop, <em>yielding</em> the control of the CPU to allow other coroutines to execute cooperatively. Other coroutines will do the same, for example breaking up what they need to do in a series of steps and yielding back control periodically if needed.</p>
<p>The second coroutine waits for a non-blocking timer to elapse, yielding back control every time, before printing a dot.</p>
<p>The same <code class="inlinecode">wakeUp</code> is used in the main thread to wait for 5 seconds before cancelling both coroutines. Cancelling will result in the coroutine throwing a <em>VeniceError</em> and exiting.</p>
<p>While Venice coroutines don’t return values, other implementations of coroutines could look a bit more like Swift’s generators and returning intermediate values like <a href="https://github.com/JadenGeller/Yield">Yield</a> does. But note that the library has not been updated to Swift 3 and uses system threads, it’s presented here just to give you an idea of how the API would look like.</p>
<p>Here is an example from the documentation that <em>generates</em> the Fibonacci sequence:</p>
<pre><code class="swift">
let maxFibbValue = 100
let fibbGenerator = Coroutine<Int> { yield in
var (a, b) = (1, 1)
while b < maxValue {
(a, b) = (b, a + b)
yield(a) // Returns `a`, and then continues from here when `next` is called again
}
}
let fibb = AnySequence { fibbGenerator }
for x in fibb {
print(x) // -> 1, 2, 3, 5, 8, 13, 21, 55, 89
}
</code></pre>
<p>Could some form of lightweight thread be a useful addition to the Swift language?</p>
<p>Definitely, and these new mechanisms would make the language way more scalable, something essential when developing server applications that deal with the network, an area where languages like Go and Erlang are at the moment a better choice.</p>
<p>And lightweight threads, as we’ll see, could be the scalable base mechanism needed to implement effectively the concurrency paradigms that will be described in the next sections.</p>
<h2 id="futures-and-promises">Futures and Promises</h2>
<p>Let’s now introduce another concept you could already be familiar with from other languages: tracking the execution of asynchronous closures through objects that contain a reference to their future return value (we’ve already seen something similar when discussing <em>OperationQueues</em>).</p>
<p>This pattern is especially useful on platforms that don’t have the luxury of being able to distribute asynchronous calls between multiple threads and, to avoid callback hell, need to provide some practical high level API built on top of some form of cooperative multitasking runtime.</p>
<p>Both <em>Futures</em> and <em>Promises</em> are placeholder for the real return value of a function, their completion status can be checked and handlers can be registered to do something when the return value of the function they refer to will be available or to handle errors.</p>
<p>Our programs can also wait for the value of the future to be available or combine multiple futures in a chain when they are dependent on each other.</p>
<p>But what’s the difference between futures and promises?</p>
<p>If we follow the original nomenclature, <em>futures</em> are a simple container that store a value and can only be read once ready, while <em>promises</em> are objects that manage the life-cycle of a future, performing the transition needed to complete it (normally only once) successfully or not (in presence of errors).</p>
<p>Libraries tend to use one or the other interchangeably.</p>
<p>There are quite a few open source Swift libraries that implement Futures and Promises (and as we’ll see in the next section async/await): <a href="https://github.com/mxcl/PromiseKit">PromiseKit</a>, <a href="https://github.com/malcommac/Hydra">Hydra</a>, <a href="https://github.com/freshOS/then">then</a>, <a href="https://github.com/Thomvis/BrightFutures">BrightFutures</a>, <a href="https://github.com/arikis/Overdrive">Overdrive</a> and many others.</p>
<p>While all these implementations provide the basic features needed to use Futures and Promises (sometime also called <em>Tasks</em>) with more or less the same API, some add extended functionalities that could make you code more concise.</p>
<p>Let’s see an example to understand how Futures and Promises work in practice using <a href="https://github.com/malcommac/Hydra">Hydra</a>.</p>
<p>Suppose we have a <code class="inlinecode">retrieveUserData</code> that retrieves the profile data for a user with a specific numeric id that once done calls a completion handler. Let’s improve the API using Promises and Futures.</p>
<pre><code class="swift">
func getUserData(for id: Int) -> Promise<UserStruct> {
return Promise<UserStruct>(in: .background, { resolve, reject in
retrieveUserData(id: id, completion: { data, error in
if let error = error {
reject(error)
} else {
resolve(data)
}
})
})
}
getUserData(42).then(.main, {user in
// On successful completion
print("Loaded data for user: \(user.username)")
}).catch(.main, {error in
// On error
print("Error: \(error)")
})
</code></pre>
<p>In the example above, the <code class="inlinecode">getUserData</code> returns a <code class="inlinecode">Promise<UserStruct></code> object, that is set with a value or an error in the completion handler using either the <code class="inlinecode">resolve(value:UserStruct)</code> or <code class="inlinecode">reject(error:Error)</code> methods.</p>
<p>We then register a function to be called when the promise will be fulfilled using <code class="inlinecode">.then</code>, specifying that the closure should be executed on the main queue. The <code class="inlinecode">.catch</code> method handles errors for all promises <em>in a chain</em>.</p>
<p>Yes, promises can be chained and have also a few other useful operators, they are definitely not just a cleaner alternative to callbacks.</p>
<p>We will now define two new functions that returns promises and we’ll chain them together to perform three operations in sequence:</p>
<pre><code class="swift">
func getPhotos(user: UserData) -> Promise<[Photos]>
func bulkDeletePhotos(photos: [Photos]) -> Promise<Int>
getUserData(42).then(getPhotos)
.then(.bulkDeletePhotos)
.catch{ err in
print("Error while deleting photos: \(err)")
}
</code></pre>
<p>After the information about the user have been retrieved, we try to get a list of all the photos of this account and then perform a mass deletion. If any of those operations fails, the final catch block will handle the error.</p>
<p>Promises libraries also provide some operators like: <code class="inlinecode">all</code>, <code class="inlinecode">any</code>, <code class="inlinecode">retry</code>, <code class="inlinecode">recover</code>, <code class="inlinecode">validate</code> and many others.</p>
<p>Combining these operators, complex logic with multiple nested statements can be turned into something more concise but still keeping a good degree of clarity.</p>
<pre><code class="swift">
getUserData(42).retry(3)
.validate($0.age > 18)
.then{ user in
print("Username: \(user.username)")
}.always{
print("Done.")
}
</code></pre>
<p>This time we’ll try to retrieve information on the user with id 42, retrying on failure at most three times, and if successful we’ll print the username after we have verified that the user is over 18 years old.</p>
<p>Using the <code class="inlinecode">always</code> method, we can also add a finalizing closure that will always be executed no matter what happens to the closures that precede it in the chain.</p>
<p>The <code class="inlinecode">all</code> and <code class="inlinecode">any</code> methods create a composite future that will be fulfilled only when either all or at least one of the underlining futures will complete successfully.</p>
<pre><code class="swift">
let promises = [40,41,42,43,44,45].map {return getUserData($0)}
all(promises).then {users in
print("All \(users.count) users retrieved!")
}.catch {err in
print("Error: \(err)")
}
any(promises).then {users in
print("At least some users retrieved!")
}
</code></pre>
<p>While these basic functionalities are common among different implementations, other methods, that for example allow to introduce delays along a chain or that perform additional transformations, could be available.</p>
<h2 id="asyncawait">Async/Await</h2>
<p>Now let’s talk about the Async/Await functionality, that is quite popular lately since it has been finally added to Javascript in the ES2017 version of the standard, but that was already available in other languages (C# introduced this API first, in release 5.0 a few years ago).</p>
<p>Async/Await can be considered an extension over the basic futures API.</p>
<p>In the mono-thread world of Javascript, futures provided a better alternative to callback nesting, allowing to enclose asynchronous closures in convenient objects that could be used to chain sequential operations, with the side-effect of simplifying stacktrace browsing during debugging.</p>
<p>These new keywords allow to do something that at a first look appears to be only a small improvement over using promises directly: <code class="inlinecode">await</code> allows to block execution until a function returning a promise completes and <code class="inlinecode">async</code> allows to annotate functions that during their execution wait for completion of one or more futures.</p>
<p>But let’s see a simple example in Javascript that makes use of both keywords:</p>
<pre><code class="swift">
async function getUserProfileImageUrl(id) {
let user = await getUserData(id);
let img = await getImageData(user.imageId)
return img.url;
}
</code></pre>
<p>With futures we could have obtained a reference to the <code class="inlinecode">UserData</code> structure and to <code class="inlinecode">ImageData</code> in a series of <code class="inlinecode">then</code> closures extracting the URL as last step.</p>
<p>Using Async/Await this asynchronous code looks more like classic synchronous code, an improvement from the point of view of aesthetic but something that could have been replicated waiting in a blocking way for the fulfillment of those two promises.</p>
<p>But look at this from another point of view.</p>
<p>With Async you are also delimiting a context that contains asynchronous calls and with Await you are clearly marking where the execution will stop waiting for a result.</p>
<p>These two information would be extremely useful to implement some form of <em>cooperative multitasking</em> under the hood, and this is what is likely to happen in the context of Javascript in most implementations.</p>
<p>That wait will thus be a non-blocking wait in most of the implementations, with every await function being still executed in sequence, as we saw in the first part of the series, waiting in a non-blocking way is useful when you need to perform I/O or just to increase CPU/resources utilization.</p>
<p>Let’s go back to Swift with some examples with <a href="https://github.com/malcommac/Hydra#awaitasync">Hydra</a>, but int this case too, other libraries provide a mostly identical API.</p>
<pre><code class="swift">
let asyncFunc = async({
let user = try await(getUserData(42))
let photos = try await(getPhotos(user))
let success = try await(bulkDeletePhotos(photos))
})
</code></pre>
<p>The async block will be executed in the <code class="inlinecode">.background</code> thread as the await functions, that run in the context in which they are called.</p>
<p>Additionally, Hydra and other libraries allows to return a promise from within the async block so that you’ll be able to use all the operators seen in the previous section or just simply chain the block with other promises.</p>
<p>How is the failure of a promise handled with async/await?</p>
<p>To handle errors, await calls are usually surrounded by a catch statement, in Javascript but also in the Swift libraries listed above, since as you could have guessed , the <code class="inlinecode">try</code> in the previous sample is a sign that an Error will be thrown by await when a promise completes with a failure.</p>
<p>So, for most libraries you’ll be able to catch errors adding a <code class="inlinecode">do/catch</code> inside the async block.</p>
<p>If you are interested in learning more about async/await I recommend you to check out the opensource libraries listed above and look under the hood at how the various functions have been implemented.</p>
<h2 id="stm-software-transactional-memory">STM: Software Transactional Memory</h2>
<p>In this section we’ll take a look at Clojure, a LISP language running on the JVM that is usually a good source of interesting ideas.</p>
<p>If you want to know more about languages of the LISP family, check out the article I wrote a while ago about <a href="/2017/02/05/building-a-lisp-from-scratch-with-swift/">implementing a LISP from scratch in Swift</a>, that will guide you through the implementation of a minimal LISP while describing the fundamental components of an interpreter.</p>
<p><em>Pure</em> functional languages do not allow mutable data or the use of mutable state and as such are inherently thread-safe. If there is no mutable memory that needs to be protected from concurrent access, the problems of concurrency are already solved.</p>
<p>Clojure is an <em>impure</em> functional languages, and manages mutable data through <em>immutable</em> and <em><a href="https://en.wikipedia.org/wiki/Persistent_data_structure">persistent</a></em> data structures that are concurrency-aware.</p>
<p>Managing mutability with immutable structures may seem a contradiction until you grok the meaning of the term <em>persistent</em> in this context.</p>
<p>Quoting <a href="https://www.cs.cmu.edu/~sleator/papers/making-data-structures-persistent.pdf">the article</a> that coined the term:</p>
<blockquote>
<p>Ordinary data structures are ephemeral in the sense that making a change to the structure destroys the old version, leaving only the new one.
We call a data structure persistent if it supports access to multiple versions.
The structure is partially persistent if all versions can be accessed but only the newest version can be modified, and fully persistent if every version can be both accessed and modified.</p>
</blockquote>
<p>You could be already familiar with the mechanism of <a href="https://en.wikipedia.org/wiki/Copy-on-write">Copy-On-Write</a>, used profusely in Swift, that speeds up the copy of a value when it’s reassigned performing a real copy only when one of the clones of the original value is modified, saving memory and avoiding pointless copying.</p>
<p>In other words, when a data structure that supports <em>COW</em> get copied to a new variable, the new variable will still point to the original area of memory, until a real copy of the original content is triggered by an attempt to change it.</p>
<p><em>Persistent</em> data structures take this idea a step further and try to share with the original value the part of its structure that didn’t change after the update to perform something akin to a selective copy. All of this with the guarantee that the original memory content will be untouched and still accessible through the other references.</p>
<p>The image below shows a simple example of persistent list, that highlights how the actual copy is performed only when the original data structure will become unusable because the last element down the chain will not be needed anymore.</p>
<p><img src="/imgs/persistent.png" srcset="/imgs/persistent@2x.png 2x" alt="Persistent list" /></p>
<p>Similarly to COW variables, using persistent data structure has the effect of isolating changes to the reference on which they were applied.</p>
<p>This property is available with all the basic collection types in Clojure and in addition to this, <em>transient</em>, read-only views of persistent data structures can be created in those scenarios where mutability is not needed.</p>
<p>Clojure guarantees thread-safety with different mechanisms building upon persistence through <a href="http://www.thiagotnunes.com/blog/2013/06/10/clojure-concurrency">4 container primitives</a>: <em>Vars</em>, <em>Refs</em>, <em>Atoms</em> and <em>Agents</em>.</p>
<p>The most interesting and most sophisticated are <em>Refs</em> that are an integral component of a system based on <em>Software Transactional Memory</em>.</p>
<p>STM is based on the idea of grouping a series of operations on mutable variables, that need to happen atomically, in a single transaction that is guaranteed to be executed completely or not at all. You could already be familiar with this concept from the world of databases.</p>
<p>Multiple transactions can be executed concurrently but they will never be allowed to interfere with each other, transactions that try to perform conflicting modifications on the same set of data will simply fail and retried by the STM runtime.</p>
<p>Refs contains the objects that can be modified during a transaction and they can be either <em>dereferenced</em> to obtain the value they contain or modified setting a new value or executing a function that alters the current value.</p>
<pre><code class="swift">
(def counter (ref 0))
(def counter2 (ref 0))
(deref counter) // Dereferenced -> 0
(dosync
(alter counter inc)
(alter counter2 inc)
)
</code></pre>
<p>Let me just do a quick and dirty recap of the LISP notation, each couple of parenthesis contain a series of terms that define a symbolic expression. These expression are written in prefix notation, where the first element of this list of terms is an operator and the rest are operands. A sum of two integers would be expressed as <code class="inlinecode">(+ 1 2)</code>. As said above, check out <a href="/2017/02/05/building-a-lisp-from-scratch-with-swift/">this article</a> if you want to know more.</p>
<p>In that example we defined two Refs to an integer variable named <code class="inlinecode">counter</code> and <code class="inlinecode">counter2</code> and then <em>alter</em> their value in a transaction enclosed in a <code class="inlinecode">dosync</code> block, incrementing them.</p>
<p>Both values will always contain the same number regardless of how much contention there will be on those two Refs but when run concurrently, for example with a few threads that just try to execute the transaction, a good amount of those transaction will fail (silently) and will be retried automatically and transparently.</p>
<p>STM implementations are usually lock-free and as such scale better that alternative solutions based on the classic locks we saw in part one, and as you can see there is not much else to do other than defining the content of the transaction.</p>
<p>There are just a few implementation of STM in Swift with varying degree of sophistication, like <a href="https://github.com/typelift/Concurrent">Typelift’s Concurrent</a> and <a href="https://github.com/lbrndnr/swift-stm">Swift STM</a>.</p>
<p>Let’s see how STM looks with Concurrent using the bank account example, straight from their documentation:</p>
<pre><code class="swift">
typealias Account = TVar<UInt>
/// Some atomic operations
func withdraw(from account : Account, amount : UInt) -> STM<()> {
return account.read().flatMap { balance in
if balance > amount {
return account.write(balance - amount)
}
throw TransactionError.insufficientFunds
}
}
func deposit(into account : Account, amount : UInt) -> STM<()> {
return account.read().flatMap { balance in
return account.write(balance + amount)
}
}
func transfer(from : Account, to : Account, amount : UInt) -> STM<()> {
return from.read().flatMap { fromBalance in
if fromBalance > amount {
return withdraw(from: from, amount: amount)
.then(deposit(into: to, amount: amount))
}
throw TransactionError.insufficientFunds
}
}
</code></pre>
<p>The <code class="inlinecode">TVar</code> type represent a container that supports the transactional memory interface and in this case it contains an integer with the account balance and the <code class="inlinecode">withdraw</code>, <code class="inlinecode">deposit</code> and <code class="inlinecode">transfer</code> functions change its value or perform a transfer between different accounts.</p>
<p>As you can see in the transfer function, a transaction is built concatenating multiple operations:</p>
<pre><code class="swift">
let alice = Account(200)
let bob = Account(100)
let finalStatement =
transfer(from: alice, to: bob, amount: 100)
.then(transfer(from: bob, to: alice, amount: 20))
.then(deposit(into: bob, amount: 1000))
.then(transfer(from: bob, to: alice, amount: 500))
.atomically()
</code></pre>
<p>The transaction will be executed only when <code class="inlinecode">atomically()</code> is called.</p>
<p>To learn more about STM and how it can be implemented I recommend the article <a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/beautiful.pdf">Beautiful Concurrency</a> from Simon Peyton Jones.</p>
<h2 id="the-actor-model">The Actor Model</h2>
<p>In the first part of this series, we dealt with concurrency using locking primitives and more complex APIs based on the use of pool of threads to control multiple accesses to shared memory and to perform concurrent operations.</p>
<p>Locks and derivatives are the most basic primitives we have at how disposal, through them we can limit the access to code that modifies a shared area of memory imposing entry checks and regulating the execution flow of multiple threads.</p>
<p>Locks does not scale well, with increasing number of threads or with the same memory (or most of the times, data structure) shared in too many places reasoning about the state of your programs can become quite challenging.</p>
<p>High level APIs like DispatchQueue allows to model concurrent operations in a way that is easy to reason about but are based on threads, a low level primitive that as we saw in the <a href="#coroutines-and-green-threads">green threads section</a> is quite resource hungry when you consider the cost of the context switch and the memory usage.</p>
<p>And everything we have described until now does not really take into consideration that our programs could need to synchronize their state <em>across multiple machines</em> and <em>between multiple instances</em> of our application.</p>
<p>Wouldn’t be nice to be able to write concurrent code that could nearly transparently be able to run in a distributed environment without any modification?</p>
<p>To support this additional use case we need to introduce new paradigms based on an idea that you are already familiar with from the days of Objective-C: <em>Message Passing</em>.</p>
<p>This approach allows to handle concurrency without sharing memory (implicitly solving a lot of the concurrency issues we’ve seen until now) through higher level constructs that handle the exchange of messages between concurrent entities operating in isolation from each other.</p>
<p>We’ll take a look at two paradigms based on message passing: <em>The Actor Model</em> and <em>Communicating Sequential Processes</em>.</p>
<p>Implementations of the actor model can be found in various languages, from the most notable, Erlang, to languages like Java that support the paradigm through external libraries like Akka.</p>
<p>An actor is an object that provides various services through an interface that processes asynchronous messages and that maintains a mutable internal state (if the actor is <em>stateful</em> instead of <em>stateless</em>) that is <strong>never shared</strong> with the actor’s clients to guarantee thread-safety.</p>
<p>In most implementations, direct messages are received in the same order they were sent and are always processed sequentially since, internally, actors are single-threaded (but not usually backed by a real system thread) and as such, message processing can be considered as an atomic operation.</p>
<p>But messages are not actually sent <em>directly</em> to an actor object.</p>
<p>Each actor has its own <em>Mailbox</em>, a queue that stores the asynchronous messages directed to a specific actor, working as a buffer for incoming messages. A mailbox can have additional characteristics that change the way the actor receives messages, it could for example, limit the number of messages it holds or introduce the concept of message priority.</p>
<p>An actor receives messages but can also send messages to other actors if it knows <em>their address</em> or if when it needs it, it can create a new actor instance for the destination actor. This property is known as <em>locality</em>.</p>
<p>Every message has a reference to its sender, so that is always possible to reply back maintaining locality.</p>
<p>A system that uses the actor model is made up by a network on multiple actors with various functionalities, that communicate with each other constantly. Since each actor can create new actors, the topology of the network is dynamic. Smaller network of actors can be interconnected to build bigger systems.</p>
<p>Since there are no mature Swift libraries implementing the actor model, let’s see some concrete examples using Scala, a language running on the JVM with lot’s of interesting features, and the Akka library.</p>
<p>We’ll create a counter actor, that will maintain an integer value as its state and that will expose to other actors methods to increment this counter or get its value.</p>
<p>No concurrent access to this integer value will be possible, the actor will implicitly take care of that. This is functionally equivalent to having an integer counter protected by multiple accesses through a lock or some other equivalent basic concurrency primitive.</p>
<pre><code class="swift">
import akka.actor.Actor
import akka.actor.Props
import akka.event.Logging
class Counter extends Actor {
var count = 0
def receive = {
case "incr" => count += 1
case "get" => sender() ! count
case _ => println("Unknown message received")
}
}
class Main extends Actor {
val counter = context.actorOf(Props[Counter], “counter”)
counter ! “incr”
counter ! “incr”
counter ! “incr”
counter ! “get”
def receive = {
case count: Int =>
println(s“count was $count”)
context.stop(self)
case _ => println("Unknown message received")
}
}
</code></pre>
<p>Above we are defining two actors, the <em>Counter</em> actor that will manage the mutable <code class="inlinecode">count</code> and the <em>Main</em> actor that we’ll use to send command to the counter.</p>
<p>The <code class="inlinecode">receive</code> method contains a message loop that defines the <em>behavior</em> of the actor, associating at each message string a closure that will be executed every time that specific message is executed.</p>
<p>Akka requires that all the loops must be exhaustive, so, in this case we’ll have to add a final catch all case to our receive loops. Using something akin to enums with associated values (i.e. Scala’s case classes) and specifying all the cases would remove this requirement. Other implementations (e.g. Erlang actors) do not require exhaustiveness.</p>
<p>An interesting feature of actors is the ability to <em>alter their behavior</em> while they are running. Multiple message loops can be defined and the actor will be able to switch between them selecting one of them as active loop.</p>
<p>In the sample above, the Counter replies to <code class="inlinecode">incr</code> and <code class="inlinecode">get</code>, respectively incrementing the local counter or sending the value of the counter to the actor that sent the message.</p>
<p>In this example all the messages are sent in “fire and forget” mode using the <code class="inlinecode">!</code> operator, that sends the message asynchronously and returns right away. Alternatively, messages can be sent encapsulating the status of the call into a future.</p>
<p>The Main actor creates a Counter actor and then proceeds to incrementing the counter three times before retrieving its value. The value is then obtained asynchronously in its receive loop, where the value is printed before the actor stops itself.</p>
<p>Now that we’ve got this basic example down, let’s talk again about behaviors.</p>
<p>The fact that multiple receive loops can be defined before-hand and then swapped at runtime can be useful to implement a <a href="https://en.wikipedia.org/wiki/Finite-state_machine">Finite State Machine</a> to manage the various states the actor will find itself during its lifetime.</p>
<p>An actor could for example be started in a uninitialized state and require a series of messages to be configured and perform a transition to its normal running state. Or it could have a more complex logic, easy to represent with a FSM, where every state could be implemented with a different receive loop and where transitions would be performed when specific messages are received. Furthermore, the ability to change the behavior could be used to encapsulate the internal state of stateful actors.</p>
<p>Let’s see a quick example going back to the counter actor, we’ll remove the need for an external integer counter embedding the counter value in the receive loop.</p>
<pre><code class="swift">
class Counter extends Actor {
def counter(n: Int) = {
case "incr" => context.become(counter(n + 1))
case "get" => sender() ! n
case _ => println("Unknown message received")
}
def receive = counter(0)
}
</code></pre>
<p>A simple actor with two states returning two alternating values can be easily defined as follow:</p>
<pre><code class="swift">
class PingPong extends Actor {
def ping = {
case "get" =>
sender() ! "Ping!"
context.become(pong)
case _ => println("Unknown message received")
}
def pong = {
case "get" =>
sender() ! "Pong!"
context.become(ping)
case _ => println("Unknown message received")
}
def receive = ping
}
</code></pre>
<p>Now, we have our system based on a huge number of actors all operating concurrently and processing asynchronous message, how do we handle abnormal errors and fault conditions?</p>
<p>While normal error condition related to the logic of an actor can still be handled through error messages, unexpected exceptions (remember that Scala and Akka run on the JVM) that may crash an actor can’t.</p>
<p>Systems based on actors follow a <strong>“let it crash”</strong> approach, and focus on bringing back the system to a normal functioning state deciding what to do with components marked as failed, instead of trying to keep components alive stabilizing their state perturbed by an unexpected error.</p>
<p>Actor system are fault tolerant and allow to choose a recovery strategy to make an actor (or a series of actors in a network) fully functional again after a fault.</p>
<p>A failing actor could be for example simply restarted with a clean slate, discarding its internal state, it could be stopped completely or we could just ignore a class of errors and go on as if nothing happened. The strategy you choose depends on the error and what you need to do to bring back the actor to its running state.</p>
<p>What you have read in this section barely scratches the surface of what modern actor systems have to offer, to learn more check out this <a href="http://letitcrash.com/post/20964174345/carl-hewitt-explains-the-essence-of-the-actor">video with Carl Hewitt</a>, the <a href="http://doc.akka.io/docs/akka/current/scala/actors.html">Akka documentation</a> or read about <a href="http://learnyousomeerlang.com/the-hitchhikers-guide-to-concurrency">Erlang’s implementation</a>.</p>
<p>To conclude, we’ve seen that the actor model has a lot of interesting characteristics until now, but it also has some weaknesses.</p>
<p>In some situations, it could be quite hard to model your problem following the rigid actor model and other approaches could be better suited for what you need to do.</p>
<p>When using actors you’ll also need to consider how messages flow in your system and configure accordingly the mailbox of each actor. Failing to do this could lead to mailbox’s overflows.</p>
<p>Actors are also the wrong tool when you want to parallelize sections of your code, that’s not the problem the actor model is trying to solve.</p>
<h2 id="communicating-sequential-processes-channels">Communicating Sequential Processes: Channels</h2>
<p>The Communicating Sequential Process model is another concurrency model based on message passing that, instead of focusing on the objects receiving or sending messages and what they do, revolves around the idea of <em>channels</em>, that compose the infrastructure needed to exchange messages between different entities running concurrently.</p>
<p>A channel can be used to send and receive messages between entities running in different tasks that will use it as a <em>communication channel</em>. You could have for example a task that takes care of centralized logging that exposes its services through a channel that every other task in the system would use.</p>
<p>From the point of view of the implementation, a channel is nothing more than a thread-safe FIFO queue, that can hold multiple messages awaiting to be received if needed(<em>buffered</em> channels).</p>
<p>The sender usually blocks when the channel is <em>full</em> and waits until a receiver removes a message from the list. Conversely, receivers blocks when a queue is empty and wait for new messages. Channels can be <em>closed</em> when they are not needed anymore, unblocking all the senders and receivers that were waiting on that queue.</p>
<p>Implementations of channels have usually a rich API that allows to handle messages from different channels in a single reception block and are usually coupled with some lightweight thread implementation to make the system more scalable.</p>
<p>And channels are usually paired with some form of lightweight threads, either coroutines or green threads, since these two functionality pair quite well together.</p>
<p>A few Swift implementations for CSP channels, each one with a different API and a different set of functionalities, are already available: <a href="https://github.com/Zewo/Venice">Venice</a>, <a href="https://github.com/typelift/Concurrent">Concurrent</a> and <a href="https://github.com/tidwall/Safe">Safe</a>.</p>
<p>The only library that also provides coroutines is <a href="https://github.com/Zewo/Venice">Venice</a>, but on the other hand has a channel API a bit different from what you’d normally expect and is not a pure-Swift framework. As far as I know there are no Swift implementation of green threads.</p>
<p>The Go programming language is probably the reason behind the recent surge in popularity of CSP channels since channels and goroutines (go’s implementation of coroutines) are the corner stone of its approach to concurrency.</p>
<p>Considering that every Swift implementation available at the moment lacks something (Safe is Swift 2 only, Concurrent and Venice do not support the <em>select</em> statement), I’ll present a few examples in Go, that has a clean and minimalistic API for both channels and goroutines. But I still recommend to check out those libraries since what they have could be enough for your use case.</p>
<p>We’ll create a simple program that defines a channel and uses it to send a string between a goroutine and the main thread. The main thread will wait for the string using the channel and it will just print it once it’s available.</p>
<pre><code class="swift">
package main
import "fmt"
import "time"
func main() {
messages := make(chan string)
go func(){
time.Sleep(time.Second)
messages <- "Hello!"
}()
msg := <- messages
fmt.Println(msg)
}
</code></pre>
<p>In the <code class="inlinecode">main</code> function, the entry point of the program, we define a channel that will contain a single string with the <code class="inlinecode">make</code> function.</p>
<p>What follows is the declaration of a goroutine that will execute an anonymous closure responsible for sending a string through the channel.</p>
<p>As we said in the <a href="#coroutines-and-green-threads">Coroutines and Green Threads</a> section, goroutines are just lightweight threads, we could happily create hundred of thousands of them.</p>
<p>After sleeping for a second, the goroutine will send a string message through the channel using the <code class="inlinecode"><-</code> operator. Depending on what is on the right side of the operator, it send or receive a message through a channel.</p>
<p>The main thread will block trying to extract a string from the messages channel until the goroutine will send one. Once it gets a hold of one, it will print it.</p>
<p>Optionally, channels can be buffered and be able to store more than one object and they can also be restricted to a specific direction (send-only or receive-only).</p>
<p>They can be used in different ways to coordinate work between a chain of goroutines or to send back results from a background worker goroutine that had some task to execute.</p>
<p>Since most implementations will be based on multiple channels to coordinate multiple goroutines, each one with a different job to perform, a construct able to handle messages from more than one channel in a single place exists, the <code class="inlinecode">select</code> block.</p>
<pre><code class="swift">
package main
import(
"fmt"
"time"
)
func main() {
ch1 := make(chan string, 3)
ch2 := make(chan string, 3)
done := make(chan bool)
go func() {
for i := 0; i < 10; i++ {
ch1 <- "Message from 1"
time.Sleep(time.Second * 1)
}
}()
go func() {
for i := 0; i < 10; i++ {
ch2 <- "Message from 2"
time.Sleep(time.Second * 2)
}
done <- true
}()
go func() {
for {
select {
case msg1 := <- ch1:
fmt.Println(msg1)
case msg2 := <- ch2:
fmt.Println(msg2)
}
}
}()
<- done
}
</code></pre>
<p>This time we have three anonymous closures executed in as many goroutines.</p>
<p>The first two will send a string and wait, with the longer second goroutines sending a boolean through a boolean channel when is done with its strings. This channel is used as completion signal of the main thread (not a perfect implementation).</p>
<p>The third goroutines receives the messages checking continuously the two string channels and prints everything it receives. The select block allows to wait on more than one channel and be able to receive all the messages from the channels regardless of the order in which they are sent.</p>
<p>Considering that each wait on a channel would have been otherwise blocking, this new construct <em>is essential</em>, without it channels are instantly a lot less useful.</p>
<p>Proponents of actors and CSP are member of two opposing school of thoughts, so, you’ll rarely found someone advocating the use of both approaches in a language.</p>
<p>Channels have great performance characteristics are way more simple than actors and that can lead to simple and clean architectures sometime.</p>
<p>But in the end they are just blocking queues, they don’t solve any of the concurrency problems described in the first part of the series, and you’ll have to use them as building blocks of more complex mechanisms on your own.</p>
<p>While CSP channels introduce a lot of flexibility compared to the <em>one message queue per task</em> approach of the Actor model and allow to build a network of lightweight processes connected by multiple communication channels, they do not have much to say in regards to fault tolerance.</p>
<p>You will not have something analogous to the independent software components that can be restarted at will available with the actor model.</p>
<p>But probably the greatest weakness of CSP channels is that you’ll be rarely able to structure your program around the use of channels alone without creating convoluted hierarchies of channels.</p>
<p>In most cases, you’ll still need to resort to the basic locks seen in part one.</p>
<h2 id="swift-where-are-we-now">Swift: Where are we now</h2>
<p>Discussions on the future of concurrency on swift-evolution have yet to start, and the community will start introducing the first proposal likely a few months after the release of Swift 4 (assuming that improving concurrency will be one of the goals of Swift 5).</p>
<p>So, until the end of 2017 it’s unlikely that discussions around concurrency will start.</p>
<p>But when it will happen, I expect heated discussions with lot’s of input from the community, on a scale similar to what happened when discussing the access keywords or SE-110.</p>
<p>For now, in addition to the already cited <a href="https://github.com/apple/swift/blob/master/docs/OwnershipManifesto.md">Ownership Manifesto</a>, the only document that touches the subject is this <a href="https://github.com/apple/swift/blob/master/docs/proposals/Concurrency.rst">unmaintained draft</a> that contains quite a few interesting ideas.</p>
<p>The document outlines some of the challenges involved in making concurrency more safe and discusses possible approaches to implement Actors, CSP channels and Async/await.</p>
<p><strong>Update 8/17</strong></p>
<p>Chris Lattner wrote his toughts on concurrency in a <a href="https://gist.github.com/lattner/31ed37682ef1576b16bca1432ea9f782">manifesto</a> and discusses all the topics of this post, check out his take on it. He and Joe Groff also published a <a href="https://gist.github.com/lattner/429b9070918248274f25b714dcfc7619">proposal for async/await</a>.</p>
<h2 id="a-few-words-on-kotlin">A few words on Kotlin</h2>
<p>Now let’s talk about concurrency in Kotlin, the de-facto counterpart of Swift on Android.</p>
<p>When using Java interoperability, Kotlin inherits all the concurrency functionalities available in Java, from threads and locks to thread pools and concurrent data structures but since one day Kotlin could not have an underlying JVM to depend on, a new alternative approach to concurrency is in development.</p>
<p>I’m referring to an experimental implementation of some of the paradigms described in the previous sections based on <a href="https://kotlinlang.org/docs/reference/coroutines.html#experimental-status-of-coroutines">coroutines</a>, that at the moment are distributed as an external <a href="https://github.com/Kotlin/kotlinx.coroutines">kotlinx.coroutines</a> library.</p>
<p>The coroutines library <a href="https://github.com/Kotlin/kotlinx.coroutines/blob/master/coroutines-guide.md#composing-suspending-functions">contains implementations</a> of async/await, generators(that I didn’t describe in this article, but that have a structural similarity with Swift’s generators) and channels with support for select.</p>
<p>Swift could follow the same route, instead of choosing one paradigm in particular, multiple options could be offered like in the proposal linked in the previous section, and they could be built on top of some form of lightweight threads.</p>
<h2 id="closing-thoughts">Closing Thoughts</h2>
<p>This article should contain a good overview of the most popular paradigms and should give you enough pointers to experiment and learn further.</p>
<p>We’ve discussed the basic principles behind Promises, Await/Async, STM, Actors and CSP, paradigms that could have a place in the future of Swift, maybe (and I realize I’ve said this more than one time) built leveraging some lightweight mechanism like coroutines or green threads.</p>
<p>And while it’s too soon to speculate which new features will be added to Swift and when, we can all be sure of one thing: concurrency is coming.</p>
All about Concurrency in Swift - Part 1: The Present2017-05-07T00:00:00+02:00https://www.uraimo.com/2017/05/07/all-about-concurrency-in-swift-1-the-presentThere are a few alternatives regarding how to handle concurrency from Swift, this article will discuss everything you have at your disposal right now, to prepare the ground to the next part of this series that will discuss what is likely to come next.<p><strong>Update 10/17:</strong> <em>Updated for Swift 4</em></p>
<p><strong>Update:</strong> <em>The second part of this series is now available: <a href="/2017/07/22/all-about-concurrency-in-swift-2-the-future/">All About concurrency in Swift - Part 2: The Future</a></em>.</p>
<p>The current release of the Swift language doesn’t include yet any native concurrency functionality like other modern languages such as Go or Rust do.</p>
<p>If you plan to perform tasks concurrently and when you’ll need to deal with the resulting race conditions, your only option is to use external libraries like libDispatch or the synchronization primitives offered by Foundation or the OS.</p>
<p>In the first part of this series, we’ll take a look at what we have at our disposal with Swift 3, covering everything from Foundation locks, threads and timers to the language guarantees and the recently improved Grand Central Dispatch and Operation Queues.</p>
<p>Some basic concurrency theory and a few common concurrency pattern will also be described.</p>
<p><img src="/imgs/concurr.png" srcset="/imgs/concurr@2x.png 2x" alt="Example of klingon code with critical section" /></p>
<p>Even if they are available on every platform where Swift is available, the functions and primitives from the pthread library will not be discussed here, since for all of them, higher level alternatives exist.
The NSTimer class will also not be described here, take a look <a href="/swiftbites/nstimer-in-swift-3/">here</a> for info on how to use it with Swift 3.</p>
<p>As has already been announced multiple times, one of the major releases after Swift 4.0 (not necessarily Swift 5) will expand the language to better define the memory model and include new native concurrency features that will allow to handle concurrency, and likely parallelism, without external libraries, defining a swifty idiomatic approach to concurrency.</p>
<p>This will be the topic of the next article in this series, where we’ll discuss a few alternative approaches and paradigms implemented by other languages, how they could be implemented in Swift and we’ll analyze a few open-source implementations that are already available today and that allow to use the Actors paradigm, Go’s CSP channels, Software Transactional Memory and more with the current release of Swift.</p>
<p>This second article will be completely speculative and its main goal will be to give you an introduction to these subjects so that you’ll be able to participate to the, likely heated, discussions that will define how concurrency will be handled in the future releases of Swift.</p>
<h4 id="the-playgrounds-for-this-and-other-articles-are-available-from-github-or-zipped"><em>The playgrounds for this and other articles are available from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="/archives/2017-05-07-ConcurrencyInSwift.playground.zip">Zipped</a>.</em></h4>
<h3 id="contents">Contents</h3>
<ul>
<li><a href="#multithreading-and-concurrency-primer">Multithreading and Concurrency Primer</a></li>
<li><a href="#language-guarantees">Language Guarantees</a></li>
<li><a href="#threads">Threads</a></li>
<li><a href="#synchronization-primitives">Synchronization Primitives</a>
<ul>
<li><a href="#nslock">NSLock</a></li>
<li><a href="#nsrecursivelock">NSRecursiveLock</a></li>
<li><a href="#nsconditionlock">NSConditionLock</a></li>
<li><a href="#nscondition">NSCondition</a></li>
<li><a href="#nsdistributedlock">NSDistributedLock</a></li>
<li><a href="#osatomic-where-art-thou">OSAtomic Where Art Thou?</a></li>
<li><a href="#on-synchronized-blocks">On Synchronized Blocks</a></li>
</ul>
</li>
<li><a href="#gcd-grand-central-dispatch">GCD: Grand Central Dispatch</a>
<ul>
<li><a href="#dispatch-queues">Dispatch Queues</a></li>
<li><a href="#using-queues">Using Queues</a></li>
<li><a href="#barriers">Barriers</a></li>
<li><a href="#singletons-and-dispatch_once">Singletons and Dispatch_once</a></li>
<li><a href="#dispatch-groups">Dispatch Groups</a></li>
<li><a href="#dispatch-work-items">Dispatch Work Items</a></li>
<li><a href="#dispatch-semaphores">Dispatch Semaphores</a></li>
<li><a href="#dispatch-assertions">Dispatch Assertions</a></li>
<li><a href="#dispatch-sources">Dispatch Sources</a></li>
</ul>
</li>
<li><a href="#operations-and-operationqueues">Operations and OperationQueues</a></li>
<li><a href="#closing-thoughts">Closing Thoughts</a></li>
</ul>
<h2 id="multithreading-and-concurrency-primer">Multithreading and Concurrency Primer</h2>
<p>Nowadays, regardless of what kind of application you are building, sooner or later you’ll have to consider the fact that your application will be running in an environment with multiple threads of execution.</p>
<p>Computing platforms with more than one processors or processors with more than one hardware execution core have been around for a few decades and concept like <em>thread</em> and <em>process</em> are even older than that.</p>
<p>Operating systems have been exposing these capabilities to user programs in various ways and every modern framework or application will likely implement a few well known design pattern involving multiple threads to improve flexibility and performance.</p>
<p>Before we start to delve into the specifics of how to deal with concurrency with Swift, let me explain briefly a few basic concepts that you need to know before starting to consider if you should use <em>Dispatch Queues</em> or <em>Operation Queues</em>.</p>
<p>First of all, you could ask, even if Apple’s platform and frameworks use threads, why should you introduce them in your applications?</p>
<p>There are a few common circumstances that make the use of multiple threads a no-brainer:</p>
<ul>
<li>
<p><strong>Task groups separation</strong>: Threads can be used to modularize your application from the point of view of execution flow, different threads can be used to execute in a predictable manner a group of task of the same type, isolating them from the execution flow of other parts of your program, making it easier to reason about the current state of your application.</p>
</li>
<li>
<p><strong>Parallelize data-independent computations</strong>: Multiple software threads, backed by hardware threads or not(see next point), can be used to parallelize multiple copies of the same task operating on a subset of an original input data structure.</p>
</li>
<li>
<p><strong>Clean way to wait for conditions or I/O</strong>: When using blocking I/O or when performing other kinds of blocking operations, background threads can be used to cleanly wait for the completion of these operations. The use of threads can improve the overall design of your application and make handling blocking calls trivial.</p>
</li>
</ul>
<p>But when multiple threads are executing your application code, a few assumptions that made sense when looking at your code from the point of view of a single thread cease to be valid.</p>
<p>In an ideal world where each thread of execution behave independently and there is no sharing of data, concurrent programming is actually not much more complex that writing code that will be executed by a single thread. But if, as often happens, you plan to have multiple threads operating on the same data, you’ll need a way to regulate access to those data structures and to guarantee that every operation on that data completes as expected without unwanted interaction with operations from other threads.</p>
<p>Concurrent programming requires additional guarantees from the language and the operating system, that need to explicitly state how variables (or more generically “resources”) will behave when multiple threads try to alter their value accessing them at the same time.</p>
<p>The language needs to define a <em>Memory Model</em>, a set of rules that explicitly states how some basic statements will behave in presence of concurrent threads, defining how memory can be shared and which kind of memory accesses are valid.</p>
<p>Thanks to this, the user will have a language that behave predictably in the presence of threads and we’ll know that the compiler will only perform optimizations that respect what has been defined in the memory model.</p>
<p>Defining a memory model is a delicate step in the evolution of a language, since a model too strict could limit how the compiler will be allowed to evolve. New clever optimizations could be invalid as a consequence of past decisions on the memory model.</p>
<p>The memory model defines for example:</p>
<ul>
<li>
<p>Which language statements can be considered <em>atomic</em> and which are not, operations that can be executed only as a whole where no thread can see partial results. It’s for example essential to know if variables are initialized atomically or not.</p>
</li>
<li>
<p>How shared variables are handled by threads, if they are cached by default or not and if it would be possible to influence the caching behaviour with specific language modifiers.</p>
</li>
<li>
<p>Concurrency operators that are used to mark and to regulate access to <em>critical sections</em>, sections of code that operate on shared resources, allowing for example only one thread to follow a specific code path at a time.</p>
</li>
</ul>
<p>Now let’s go back to discussing the use of concurrency in your programs.</p>
<p>To handle concurrency correctly, you’ll have to identify the <em>critical sections</em> in your program and use concurrency primitives or concurrency-aware data structure to regulate access to data shared among different threads.</p>
<p>Imposing access rules to these sections of code or data structures open the way to another set of problems, that derive from the fact that while the desired outcome is that every thread gets to be executed and has a chance to modify the shared data, under some circumstances some of them could not execute at all or the data could be altered in unexpected and unpredictable ways.</p>
<p>You’ll face an additional set of challenges and you’ll have to work around some common problems:</p>
<ul>
<li>
<p><strong>Race conditions</strong>: With multiple threads operating on the same data, for example reading and writing it concurrently, the outcome of the execution of a series of operations could become unpredictable and dependent on the order of execution of the threads.</p>
</li>
<li>
<p><strong>Resources contention</strong>: Multiple threads, that could be performing different tasks, trying to access the same resources will increase the amount of time needed to obtain the required resources safely. These delays needed to acquire the resources you need could lead to unexpected behaviour or could require that you structure your program to regulate access to these resources.</p>
</li>
<li>
<p><strong>Deadlocks</strong>: Multiple threads waiting for each other to release the resources/locks they need, forever, blocking the execution of that group of threads.</p>
</li>
<li>
<p><strong>Starvation</strong>: A thread could never be able to acquire the resource, or a set of resources in a specific order, it needs for various reasons and keep trying forever unsuccessfully to acquire them.</p>
</li>
<li>
<p><strong>Priority Inversion</strong>: A thread with lower priority could keep acquiring resources needed for a thread with higher priority effectively inverting the priority assigned by the system.</p>
</li>
<li>
<p><strong>Non-determinism and Fairness</strong>: We can’t make assumptions on when and in what order a thread will be able to acquire a shared resource, this delay <a href="https://en.wikipedia.org/wiki/Unbounded_nondeterminism">cannot be determined a priori</a> and is heavily influenced by the amount of contention. A thread could even never be able to acquire a resource. But concurrency primitives used to guard a critical section can also be built to be <em>fair</em> or to support <em>fairness</em>, guaranteeing access to the critical section to all the threads that are waiting, also respecting the request order.</p>
</li>
</ul>
<h2 id="language-guarantees">Language Guarantees</h2>
<p>Even if at the moment the Swift language itself doesn’t have features related to concurrency, it still offers some guarantees related to how properties are accessed.</p>
<p>Global variables for example are initialized atomically, we will never need to handle manually the case in which multiple threads try to initialize the same global variable concurrently or worry that someone could see a partially initialized variable while initialization is still ongoing.</p>
<p>We’ll discuss again this behaviour when talking about implementing singletons below.</p>
<p>But it’s important to remember that lazy properties initialization is instead not performed atomically, and the language for now does not provide annotations or modifiers to change this behaviour.</p>
<p>Access to class properties is again not atomic, and if you need to make it so, you’ll have to implement exclusive access manually using locks or similar mechanisms.</p>
<h2 id="threads">Threads</h2>
<p>Foundation offers a Thread class, internally based on pthread, that can be used to create new threads and execute closures.</p>
<p>Threads can be created using the method <code class="inlinecode">detachNewThreadSelector:toTarget:withObject:</code> of the Thread class or we can create a new thread declaring a custom Thread class and then overriding the <code class="inlinecode">main()</code> method:</p>
<pre><code class="Swift">
class MyThread : Thread {
override func main(){
print("Thread started, sleep for 2 seconds...")
Thread.sleep(forTimeInterval:2)
print("Done sleeping, exiting thread")
}
}
</code></pre>
<p>But since iOS 10 and macOS Sierra, it’s finally possible on all platforms to just create a new thread using the initializer that allows to specify the closure that the thread will execute. All the example in this article will still extend the base Thread class though, so that you don’t have to worry about having the right OS to try them out.</p>
<pre><code class="Swift">
var t = Thread {
print("Started!")
}
t.stackSize = 1024 * 16
t.start() //Time needed to spawn a thread around 100us
</code></pre>
<p>Once we have a thread instance we need to manually start it. As an optional step, we can also choose a custom stack size for this new thread.</p>
<p>Threads can be stopped abruptly calling <code class="inlinecode">exit()</code> but that’s never recommended since it doesn’t give you the opportunity to cleanly end the current task, most of the times you’ll implement the stopping logic yourself if you need it or just use the <code class="inlinecode">cancel()</code> method and check the <code class="inlinecode">isCancelled</code> property inside your main closure to know if the thread is required to stop the current job before its natural end.</p>
<h2 id="synchronization-primitives">Synchronization Primitives</h2>
<p>When we have different threads that want to mutate shared data, is essential to handle synchronization of those threads in some way to prevent data corruption and non-deterministic behavior.</p>
<p>The basic facilities usually used to synchronize threads are locks, semaphores and monitors.</p>
<p>Foundation provides all of them.</p>
<p>As you’ll see momentarily, the classes (yes, all of them are reference types) implementing these constructs <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0086-drop-foundation-ns.md#proposed-solution">have not lost the NS prefix</a> in Swift 3, but could in one of the next releases of Swift.</p>
<h3 id="nslock">NSLock</h3>
<p>NSLock is the basic type of lock that Foundation offers.</p>
<p>When a thread tries to lock this object two things can happen, the thread will acquire the lock and proceed if it hasn’t already been acquired by a previous thread or alternatively the thread will wait, blocking its execution, until the owner of the lock unlocks it. In other words, locks are object that can be acquired (or locked) only by one thread at a time and this make them perfect to monitor access to critical sections.</p>
<p>NSLock and the other Foundation’s locks are <em>unfair</em>, meaning that when a series of threads is waiting to acquire a lock, they will <strong>not</strong> acquire it in the same order in which they originally tried to lock it.</p>
<p>You can’t make assumption on the execution order, and in cases of high thread contention, when a lot of threads are trying to acquire the resource, some of your threads may be subject to <em>starvation</em> and never be able to acquire the lock they are waiting for (or not able to acquire it in a timely fashion).</p>
<p>The time needed to acquire a lock, without contention, is measurable in 100s of nanoseconds, but that time grows rapidly when more than one thread tries to acquire the locked resource. So, from a performance point of view, locks are usually not the best solution to handle resource allocation.</p>
<p>Let’s see an example with two threads and remember that since the order in which the lock will be acquired is not deterministic, it could happen that T1 acquires the Lock two times in a row (but that wouldn’t be the norm).</p>
<pre><code class="Swift">
let lock = NSLock()
class LThread : Thread {
var id:Int = 0
convenience init(id:Int){
self.init()
self.id = id
}
override func main(){
lock.lock()
print(String(id)+" acquired lock.")
lock.unlock()
if lock.try() {
print(String(id)+" acquired lock again.")
lock.unlock()
}else{ // If already locked move along.
print(String(id)+" couldn't acquire lock.")
}
print(String(id)+" exiting.")
}
}
var t1 = LThread(id:1)
var t2 = LThread(id:2)
t1.start()
t2.start()
</code></pre>
<p>Let me just add a word of warning for when you’ll decide to use locks. Since it’s likely that sooner or later you’ll have to debug concurrency issues, always try to circumscribe your use of locks inside the bounds of some sort of data structure and try not to refer directly to a single lock object in multiple places in your code base.</p>
<p>Checking the status of a synchronized data structure with few entry points while debugging a concurrency problem is way more pleasant than having to keep track of which part of your code is holding a lock and having to remember the local status of multiple functions. Go the extra mile and structure well your concurrent code.</p>
<h3 id="nsrecursivelock">NSRecursiveLock</h3>
<p>Recursive locks can be acquired multiple times from the thread that already holds that lock, useful in recursive function or when calling multiple functions that check the same lock in sequence. This <strong>would not work</strong> with basic NSLocks.</p>
<pre><code class="Swift">
let rlock = NSRecursiveLock()
class RThread : Thread {
override func main(){
rlock.lock()
print("Thread acquired lock")
callMe()
rlock.unlock()
print("Exiting main")
}
func callMe(){
rlock.lock()
print("Thread acquired lock")
rlock.unlock()
print("Exiting callMe")
}
}
var tr = RThread()
tr.start()
</code></pre>
<h3 id="nsconditionlock">NSConditionLock</h3>
<p>Condition locks provides additional sub-locks that can be locked/unlocked independently from each other to support more complex locking setups (e.g. consumer-producer scenarios).</p>
<p>A global lock (that locks regardless of a specific condition) is also available and behaves like a classic NSLock.</p>
<p>Let’s see a simple example with a lock that guards a shared integer, that a consumer prints and a producer updates every time it has been shown on screen.</p>
<pre><code class="Swift">
let NO_DATA = 1
let GOT_DATA = 2
let clock = NSConditionLock(condition: NO_DATA)
var SharedInt = 0
class ProducerThread : Thread {
override func main(){
for i in 0..<5 {
clock.lock(whenCondition: NO_DATA) //Acquire the lock when NO_DATA
//If we don't have to wait for consumers we could have just done clock.lock()
SharedInt = i
clock.unlock(withCondition: GOT_DATA) //Unlock and set as GOT_DATA
}
}
}
class ConsumerThread : Thread {
override func main(){
for i in 0..<5 {
clock.lock(whenCondition: GOT_DATA) //Acquire the lock when GOT_DATA
print(i)
clock.unlock(withCondition: NO_DATA) //Unlock and set as NO_DATA
}
}
}
let pt = ProducerThread()
let ct = ConsumerThread()
ct.start()
pt.start()
</code></pre>
<p>When creating the lock, we need to specify the starting condition, represented by an integer.</p>
<p>The <code class="inlinecode">lock(whenCondition:)</code> method will acquire the lock when the condition is met or will wait until another thread sets that value when releasing the lock using <code class="inlinecode">unlock(withCondition:)</code>.</p>
<p>A small improvement over basic locks that allows us to model slightly more complex scenarios.</p>
<h3 id="nscondition">NSCondition</h3>
<p>Not to be confused with Condition Locks, a condition provide a clean way to wait for a <em>condition</em> to occur.</p>
<p>When a thread that has acquired a lock verifies that an additional condition it needs (some resource it needs, another object being in a particular state, etc…) to perform its work is not met, it needs a way to be put on hold and continue its work once that condition is met.</p>
<p>This could be implemented by continuously or periodically checking for that condition (busy waiting) but doing so, what would happen to the locks the thread holds? Should we keep them while we wait or release them hoping that we’ll be able to acquire them again when the condition is met?</p>
<p>Conditions provide a clean solution to this problem, once acquired a thread can be put on a <em>waiting</em> list for that condition and is woken up once another thread <em>signals</em> that the condition has been met.</p>
<p>Let’s see an example:</p>
<pre><code class="Swift">
let cond = NSCondition()
var available = false
var SharedString = ""
class WriterThread : Thread {
override func main(){
for _ in 0..<5 {
cond.lock()
SharedString = "😅"
available = true
cond.signal() // Notify and wake up the waiting thread/s
cond.unlock()
}
}
}
class PrinterThread : Thread {
override func main(){
for _ in 0..<5 { //Just do it 5 times
cond.lock()
while(!available){ //Protect from spurious signals
cond.wait()
}
print(SharedString)
SharedString = ""
available = false
cond.unlock()
}
}
}
let writet = WriterThread()
let printt = PrinterThread()
printt.start()
writet.start()
</code></pre>
<h3 id="nsdistributedlock">NSDistributedLock</h3>
<p>Distributed locks are quite different from what we’ve seen until now and I don’t expect that you’ll need them frequently.</p>
<p>They are made to be shared between multiple applications and are backed by an entry on the file system (for example a simple file). The file system will obviously need to be accessible by all the applications that need to acquire it.</p>
<p>This kind of lock is acquired using the <code class="inlinecode">try()</code> method, a non blocking method that returns right away with a boolean indicating if the lock was acquired or not. Acquiring a lock will usually require more than one try, to be performed manually and with a proper delay between successive attempts.</p>
<p>Distributed locks are released as usual using the <code class="inlinecode">unlock()</code> method.</p>
<p>Let’s see a basic example:</p>
<pre><code class="Swift">
var dlock = NSDistributedLock(path: "/tmp/MYAPP.lock")
if let dlock = dlock {
var acquired = false
while(!acquired){
print("Trying to acquire the lock...")
usleep(1000)
acquired = dlock.try()
}
// Do something...
dlock.unlock()
}
</code></pre>
<h3 id="osatomic-where-art-thou">OSAtomic Where Art Thou?</h3>
<p>Atomic operations like those that were provided by <a href="https://www.mikeash.com/pyblog/friday-qa-2011-03-04-a-tour-of-osatomic.html">OSAtomic</a> are simple operations that allow to set, get or compare-and-set variables without using the classic locking logic because they leverage specific CPU functionalities (sometimes native atomic instructions) and that provide way better performance than the locks described previously.</p>
<p>It goes without saying that they are extremely useful to build concurrent data structures, since the overhead needed to handle concurrency is reduced to a minimum.</p>
<p>OSAtomic has been deprecated since macOS 10.12 and was never available on Linux, but a few open source project like <a href="https://github.com/glessard/swift-atomics">this</a> with its useful Swift extensions or <a href="https://github.com/bignerdranch/AtomicSwift">this</a> provide similar functionalities. Also, check out the recently released <a href="https://github.com/macmade/AtomicKit">AtomicKit</a>.</p>
<h3 id="on-synchronized-blocks">On Synchronized Blocks</h3>
<p>In Swift you can’t create a @synchronized block out of the box as you would do in Objective-C, since there is no equivalent keyword available.</p>
<p>On Darwin, with a bit of code you could roll out something similar to the original implementation of @synchronized using <code class="inlinecode">objc_sync_enter(OBJ)</code> and <code class="inlinecode">objc_sync_exit(OBJ)</code> to enter and exist an @objc object monitor like @synchronized does under the hood, but it’s not recommended, it’s better to simply use a lock if you need something like that, more versatile.</p>
<p>And as we’ll see when describing Dispatch Queues, we can use queues to replicate this functionality with even less code performing a synchronous call on a serial queue:</p>
<pre><code class="Swift">
serialQueue.sync {
// Only a thread at a time!
v += 1
print("Current value \(v)")
}
</code></pre>
<h4 id="the-playgrounds-for-this-and-other-articles-are-available-from-github-or-zipped-1"><em>The playgrounds for this and other articles are available from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="/archives/2017-05-07-ConcurrencyInSwift.playground.zip">Zipped</a>.</em></h4>
<h2 id="gcd-grand-central-dispatch">GCD: Grand Central Dispatch</h2>
<p>For those that are not already familiar with this API, the Grand Central Dispatch (GCD) is a queue based API that allows to execute closures on workers pools.</p>
<p>In other words, closures containing a job that need to be executed can be added to a queue that will execute them using a series of threads either sequentially or in parallel depending on the queue’s configuration options. But regardless of the type of queue, jobs will always be started following the <em>First-in First-out</em> order, meaning that the jobs will always be started respecting the insertion order. The completion order will depend on the duration of each job.</p>
<p>This is a common pattern that can be found in nearly every relatively modern language runtime that handles concurrency. A thread pool is way more easy to manage, inspect and control than a series of free and unconnected threads.</p>
<p>The GCD API had a few changes in Swift 3, <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0088-libdispatch-for-swift3.md">SE-0088</a> modernized its design and made it more object oriented.</p>
<h3 id="dispatch-queues">Dispatch Queues</h3>
<p>The GCD allows the creation of custom queues but also provide access to some predefined system queues.</p>
<p>To create a basic serial queue, a queue that will execute your closures sequentially, you just need to provide a string label that will identify it and it’s usually recommended to use a reverse order domain name prefix to simplify tracking back the owner of the queue in stack traces.</p>
<pre><code class="Swift">
let serialQueue = DispatchQueue(label: "com.uraimo.Serial1") //attributes: .serial
let concurrentQueue = DispatchQueue(label: "com.uraimo.Concurrent1", attributes: .concurrent)
</code></pre>
<p>The second queue we created is concurrent, meaning that the queue will use all the available threads in its underlying thread pool when executing the jobs it contains. Order of execution is unpredictable in this case, don’t assume that the completion order of your closures will be in any way related to the insertion order.</p>
<p>The default queues can be retrieved from the <code class="inlinecode">DispatchQueue</code> object:</p>
<pre><code class="Swift">
let mainQueue = DispatchQueue.main
let globalDefault = DispatchQueue.global()
</code></pre>
<p>The <em>main</em> queue is the sequential main queue that handles the <em>main event loop</em> for graphical applications on either iOS or macOS, responding to events and updating the user interface. As we know, every alteration to the user interface should be performed on this queue and every long operation performed on this thread will render the user interface less responsive.</p>
<p>The runtime also provides access to other global queues with different priorities that can be identified by their <em>Quality of Service (Qos)</em> parameter.</p>
<p>The different levels of priority are declared in the <code class="inlinecode">DispatchQoS</code> class, from higher to lower:</p>
<ul>
<li>.userInteractive</li>
<li>.userInitiated</li>
<li>.default</li>
<li>.utility</li>
<li>.background</li>
<li>.unspecified</li>
</ul>
<p>It’s important to note that on mobile devices that provide a low power mode, <a href="https://mjtsai.com/blog/2017/04/03/beware-default-qos/">background queues will be suspended</a> when the battery is running low.</p>
<p>To retrieve a specific default global queue, use the <code class="inlinecode">global(qos:)</code> getter specifying the desired priority:</p>
<pre><code class="Swift">
let backgroundQueue = DispatchQueue.global(qos: .background)
</code></pre>
<p>The same priority specifier can be used with or without other attributes also when creating custom queue:</p>
<pre><code class="Swift">
let serialQueueHighPriority = DispatchQueue(label: "com.uraimo.SerialH", qos: .userInteractive)
</code></pre>
<h3 id="using-queues">Using Queues</h3>
<p>Jobs, in the form of closures, can be submitted to a queue in two ways: <em>synchronously</em> using the <code class="inlinecode">sync</code> method or <em>asynchronously</em> with the <code class="inlinecode">async</code> method.</p>
<p>When using the former, the <code class="inlinecode">sync</code> call will be blocking, in other words, the call to the <code class="inlinecode">sync</code> method will complete when its closure will complete (useful when you need to wait for the closure to end, but there are better approaches), whereas the former will add the closure to the queue and complete, scheduling the closure for deferred execution and allowing the current function to continue.</p>
<p>Let’s see a quick example:</p>
<pre><code class="Swift">
globalDefault.async {
print("Async on MainQ, first?")
}
globalDefault.sync {
print("Sync in MainQ, second?")
}
</code></pre>
<p>Multiple dispatch calls can be nested, for example when after some background, low priority, operation executed on a queue of our choosing, we need to update the user interface form the main queue.</p>
<pre><code class="Swift">
DispatchQueue.global(qos: .background).async {
// Some background work here
DispatchQueue.main.async {
// It's time to update the UI
print("UI updated on main queue")
}
}
</code></pre>
<p>Closures can also be executed after a specific delay, Swift 3 finally allows to specify in a more comfortable way the desired time interval with the utility enum <code class="inlinecode">DispatchTimeInterval</code> that allows to compose intervals using these four time units: <code class="inlinecode">.seconds(Int)</code>, <code class="inlinecode">.milliseconds(Int)</code>, <code class="inlinecode">.microseconds(Int)</code> and <code class="inlinecode">.nanoseconds(Int)</code>.</p>
<p>To schedule a closure for future execution use the <code class="inlinecode">asyncAfter(deadline:execute:)</code> method with a time interval:</p>
<pre><code class="Swift">
globalDefault.asyncAfter(deadline: .now() + .seconds(5)) {
print("After 5 seconds")
}
</code></pre>
<p>If you need to execute multiple iteration of the same closure concurrently (like you were used to to with the <em>dispatch_apply</em>) you can use the <code class="inlinecode">concurrentPerform(iterations:execute:)</code> method, but beware, these closure will be executed concurrently <em>if possible in the context of the current queue</em>, so remember to always enclose a call to this method in a sync or async call running on a queue that support concurrency.</p>
<pre><code class="Swift">
globalDefault.sync {
DispatchQueue.concurrentPerform(iterations: 5) {
print("\($0) times")
}
}
</code></pre>
<p>While normally a queue is ready to process its closures upon creation, it can be configured to start in an idle state and to start processing jobs only when manually enabled.</p>
<pre><code class="Swift">
let inactiveQueue = DispatchQueue(label: "com.uraimo.inactiveQueue", attributes: [.concurrent, .initiallyInactive])
inactiveQueue.async {
print("Done!")
}
print("Not yet...")
inactiveQueue.activate()
print("Gone!")
</code></pre>
<p>This is the first time we need to specify more than one attribute, but as you can see, you can just add multiple attributes with an array if needed.</p>
<p>Execution of jobs can also be suspended or resumed temporarily with methods inherited from <code class="inlinecode">DispatchObject</code>:</p>
<pre><code class="Swift">
inactiveQueue.suspend()
inactiveQueue.resume()
</code></pre>
<p>A <code class="inlinecode">setTarget(queue:)</code> method that is to be used only to configure the priority of inactive queues (using it on active queues will result in a crash) is also available. The result of calling this method is that the priority of the queue is set to the same priority of the queue given as parameter.</p>
<h3 id="barriers">Barriers</h3>
<p>Let’s say you added a series of closures to a specific queue (with different durations) but you now want to execute a job <em>only after</em> all the previous asynchronous task are completed. You can use barriers to do it.</p>
<p>Let’s add 20 tasks (that will sleep for a timeout of 1 second) to the concurrent queue we created previously and use a barrier to print something once the other jobs complete, we’ll do this specifying a flag <code class="inlinecode">DispatchWorkItemFlags.barrier</code> in our final <em>async</em> call:</p>
<pre><code class="Swift">
let concurrentQueue = DispatchQueue(label: "com.uraimo.Concurrent", attributes: .concurrent)
concurrentQueue.async {
DispatchQueue.concurrentPerform(iterations: 5) { (id:Int) in
sleep(1)
print("Async on concurrentQueue, 5 times: "+String(id))
}
}
concurrentQueue.async (flags: .barrier) {
print("All 5 concurrent tasks completed")
}
</code></pre>
<p>The 20 tasks will be executed in parallel without a specific order by the concurrent queue and you’ll see those messages appearing in groups of a size equal to the number of execution cores of your Mac, but the final call will always be executed last.</p>
<p>Barriers are a way to impose ordering on concurrent queues that normally don’t execute the registered tasks in a repeatable order.</p>
<p>As Arthur Hammer notes, it’s important to remember that dispatch barriers have no effect on serial queues <em>and on <a href="https://developer.apple.com/documentation/dispatch/1452797-dispatch_barrier_async?language=objc">any of the global concurrent queues</a></em>. You’ll need to define a new custom concurrent queue if you plan to use them.</p>
<h3 id="singletons-and-dispatch_once">Singletons and Dispatch_once</h3>
<p>As you could have already noticed, in Swift 3 there is no equivalent of <code class="inlinecode">dispatch_once</code>, a function used most of the times to build thread-safe singletons.</p>
<p>Luckily, Swift guarantees that global variables are initialized atomically and if you consider that constants can’t change their value after initialization, these two properties make global constants a great candidate to easily implement singletons:</p>
<pre><code class="Swift">
final class Singleton {
public static let sharedInstance: Singleton = Singleton()
private init() { }
...
}
</code></pre>
<p>We’ll declare the class as <code class="inlinecode">final</code> to deny the ability to subclass and we’ll make the designated initializer private, so that it will not be possible to manually create additional instances of this object. A public static constant will be the only entry point of the singleton and will be used to retrieve the single, shared, instance.</p>
<p>The same behaviour can be used to define blocks of code that will be executed only once:</p>
<pre><code class="Swift">
func runMe() {
struct Inner {
static let i: () = {
print("Once!")
}()
}
Inner.i
}
runMe()
runMe() // Constant already initialized
runMe() // Constant already initialized
</code></pre>
<p>It’s not really pretty to look at but it works, and it could be an acceptable implementation if it’s just a <em>one time thing™</em>.</p>
<p>But if we need to replicate exactly the functionality and API of <code class="language-plaintext highlighter-rouge">dispatch_once</code> we need to implement it from scratch, as described in the <a href="#on-synchronized-blocks">synchronized blocks section</a> with an extension:</p>
<pre><code class="Swift">
import Foundation
public extension DispatchQueue {
private static var onceTokens = [Int]()
private static var internalQueue = DispatchQueue(label: "dispatchqueue.once")
public class func once(token: Int, closure: ()->Void) {
internalQueue.sync {
if onceTokens.contains(token) {
return
}else{
onceTokens.append(token)
}
closure()
}
}
}
let t = 1
DispatchQueue.once(token: t) {
print("only once!")
}
DispatchQueue.once(token: t) {
print("Two times!?")
}
DispatchQueue.once(token: t) {
print("Three times!!?")
}
</code></pre>
<p>As expected, only the first of the three closures will be actually executed.</p>
<h3 id="dispatch-groups">Dispatch Groups</h3>
<p>If you have multiple tasks, even if added to different queues, and want to wait for their completion, you can group them in a dispatch group.</p>
<p>Let’s see an example, a task can be added to a specific group directly with the <em>sync</em> or <em>async</em> call:</p>
<pre><code class="Swift">
let mygroup = DispatchGroup()
for i in 0..<5 {
globalDefault.async(group: mygroup){
sleep(UInt32(i))
print("Group async on globalDefault:"+String(i))
}
}
</code></pre>
<p>The tasks are executed on <code class="inlinecode">globalDefault</code>, but we can register an handler for <code class="inlinecode">mygroup</code> completion that will execute a closure on the queue we prefer once all of them will be completed. The <code class="inlinecode">wait()</code> method can be used to perform a blocking wait.</p>
<pre><code class="Swift">
print("Waiting for completion...")
mygroup.notify(queue: globalDefault) {
print("Notify received, done waiting.")
}
mygroup.wait()
print("Done waiting.")
</code></pre>
<p>Another way to do track a task with groups, consists in manually entering and leaving a group instead of specifying it when performing the call on the queue:</p>
<pre><code class="Swift">
for i in 0..<5 {
mygroup.enter()
sleep(UInt32(i))
print("Group sync on MAINQ:"+String(i))
mygroup.leave()
}
</code></pre>
<h3 id="dispatch-work-items">Dispatch Work Items</h3>
<p>Closures are not the only way to specify a job that needs to be executed by a queue, sometimes you might need a container type able to keep track of its execution status and for that we have <code class="inlinecode">DispatchWorkItem</code>. Every method that accepts a closure has a variant for work items.</p>
<p>Work Items encapsulate a closure that is executed by the thread pool of the queue invoking the <code class="inlinecode">perform()</code> method:</p>
<pre><code class="Swift">
let workItem = DispatchWorkItem {
print("Done!")
}
workItem.perform()
</code></pre>
<p>And WorkItems also provide other useful methods, like <code class="inlinecode">notify</code> that as it did with groups allows to perform a closure on a specific queue on completion:</p>
<pre><code class="Swift">
workItem.notify(queue: DispatchQueue.main) {
print("Notify on Main Queue!")
}
defaultQueue.async(execute: workItem)
</code></pre>
<p>We can also wait until the closure has been executed or flag it for removal before the queue tries to execute it with the <code class="inlinecode">cancel()</code> method (that <em>does not</em> cancel closures during execution).</p>
<pre><code class="Swift">
print("Waiting for work item...")
workItem.wait()
print("Done waiting.")
workItem.cancel()
</code></pre>
<p>But it’s important to know that <code class="inlinecode">wait()</code> doesn’t just block the current thread waiting for completion but also <em>elevates</em> the priority of all the preceding work items in its queue, to try to complete this specific item as soon as possible.</p>
<h3 id="dispatch-semaphores">Dispatch Semaphores</h3>
<p>Dispatch Semaphores are locks that can be acquired by more than one thread depending on the current value of a counter.</p>
<p>Threads <code class="inlinecode">wait</code> on a semaphore when the counter, decremented every time the semaphore is acquired, reaches 0.</p>
<p>A slot to access the semaphore is released for the waiting threads calling <code class="inlinecode">signal</code> that has the effect of incrementing the counter.</p>
<p>Let’s see a simple example:</p>
<pre><code class="Swift">
let sem = DispatchSemaphore(value: 2)
// The semaphore will be held by groups of two pool threads
globalDefault.sync {
DispatchQueue.concurrentPerform(iterations: 10) { (id:Int) in
sem.wait(timeout: DispatchTime.distantFuture)
sleep(1)
print(String(id)+" acquired semaphore.")
sem.signal()
}
}
</code></pre>
<h3 id="dispatch-assertions">Dispatch Assertions</h3>
<p>Swift 3 introduces a new function to perform assertions on the current execution context, that allows to verify if a closure is being executed on the expected queue. We can build predicates using the three enum cases of <code class="inlinecode">DispatchPredicate</code>: <code class="inlinecode">.onQueue</code>, to verify that we are on a specific queue, <code class="inlinecode">.notOnQueue</code>, to verify the opposite and <code class="inlinecode">.onQueueAsBarrier</code> to check if the current closure or work item are acting as a barrier on a queue.</p>
<pre><code class="Swift">
dispatchPrecondition(condition: .notOnQueue(mainQueue))
dispatchPrecondition(condition: .onQueue(queue))
</code></pre>
<h4 id="the-playgrounds-for-this-and-other-articles-are-available-from-github-or-zipped-2"><em>The playgrounds for this and other articles are available from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="/archives/2017-05-07-ConcurrencyInSwift.playground.zip">Zipped</a>.</em></h4>
<h3 id="dispatch-sources">Dispatch Sources</h3>
<p>Dispatch Sources are a convenient way to handle system-level asynchronous events like kernel signals or system, file and socket related events using event handlers.</p>
<p>There are a few kind of Dispatch Sources available, that can be grouped as follow:</p>
<ul>
<li><strong>Timer Dispatch Sources:</strong> <em>Used to generate events at a specific point in time or periodic events (DispatchSourceTimer).</em></li>
<li><strong>Signal Dispatch Sources:</strong> <em>Used to handle UNIX signals (DispatchSourceSignal).</em></li>
<li><strong>Memory Dispatch Sources:</strong> <em>Used to register for notifications related to the memory usage status (DispatchSourceMemoryPressure).</em></li>
<li><strong>Descriptor Dispatch Sources:</strong> <em>Used to register for different events related to files and sockets (DispatchSourceFileSystemObject, DispatchSourceRead, DispatchSourceWrite).</em></li>
<li><strong>Process dispatch sources:</strong> <em>Used to monitor external process for some events related to their execution state (DispatchSourceProcess).</em></li>
<li><strong>Mach related dispatch sources:</strong> <em>Used to handle events related to the <a href="http://fdiv.net/2011/01/14/machportt-inter-process-communication">IPC facilities</a> of the Mach kernel (DispatchSourceMachReceive, DispatchSourceMachSend).</em></li>
</ul>
<p>And you can also build your own dispatch sources if needed. All dispatch sources conform to the <code class="inlinecode">DispatchSourceProtocol</code> protocol that defines the basic operations required to register handlers and modify the activation state of the Dispatch Source.</p>
<p>Let’s see an example with <code class="inlinecode">DispatchSourceTimer</code> to understand how to use these objects.</p>
<p>Sources are created with the utility methods provided by <code class="inlinecode">DispatchSource</code>, in this snippet we’ll use <code class="inlinecode">makeTimerSource</code>, specifying the dispatch queue that we want to use to execute the handler.</p>
<p>Timer Sources don’t have other parameters, so we’ll just need to specify the queue to create a source, as we’ll see, dispatch source able to handle multiple events will usually require that you specify the identifier of the event you want to handle.</p>
<pre><code class="Swift">
let t = DispatchSource.makeTimerSource(queue: DispatchQueue.global())
t.setEventHandler{ print("!") }
t.scheduleOneshot(deadline: .now() + .seconds(5), leeway: .nanoseconds(0))
t.activate()
</code></pre>
<p>Once the Source is created, we register an event handler with <code class="inlinecode">setEventHandler(closure:)</code> and if no other configurations are required enable the dispatch source with <code class="inlinecode">activate()</code> (previous releases of libDispatch used the <em>resume()</em> method for this purpose).</p>
<p>Dispatch Sources are initially inactive, meaning that they will not start delivering events right away allowing further configuration. Once we are ready, the source can be activated with <code class="inlinecode">activate()</code> and if needed the event delivery can be temporarily suspended with <code class="inlinecode">suspend()</code> and resumed with <code class="inlinecode">resume()</code>.</p>
<p>Timer Sources require an additional step to configure which kind of timed events the object will deliver. In the example above we are defining a single event that will be delivered 5 seconds after the registration with a strict deadline.</p>
<p>We could have also configured the object to deliver periodic events, like we could have done with the <a href="/swiftbites/nstimer-in-swift-3/">Timer</a> object:</p>
<pre><code class="Swift">
t.scheduleRepeating(deadline: .now(), interval: .seconds(5), leeway: .seconds(1))
</code></pre>
<p>When we are done with a dispatch source and we want to just stop completely the delivery of events, we’ll call <code class="inlinecode">cancel()</code>, that will stop the event source, call the cancellation handler if we did set one and perform some final cleanup operations like unregistering the handlers.</p>
<pre><code class="Swift">
t.cancel()
</code></pre>
<p>The API is still the same for the other dispatch source types, let’s see for example how <a href="https://github.com/IBM-Swift/Kitura-net/blob/master/Sources/KituraNet/IncomingSocketHandler.swift#L96">Kitura</a> initializes the read source it uses to handle asynchronous reads on an established socket:</p>
<pre><code class="Swift">
readerSource = DispatchSource.makeReadSource(fileDescriptor: socket.socketfd,
queue: socketReaderQueue(fd: socket.socketfd))
readerSource.setEventHandler() {
_ = self.handleRead()
}
readerSource.setCancelHandler(handler: self.handleCancel)
readerSource.resume()
</code></pre>
<p>The function <code class="inlinecode">handleRead()</code> will be called on a dedicated queue when new bytes will be available in the socket’s incoming data buffer. Kitura also uses a <em>WriteSource</em> to perform buffered writes, using the dispatch source events <a href="https://github.com/IBM-Swift/Kitura-net/blob/master/Sources/KituraNet/IncomingSocketHandler.swift#L328">to efficiently pace the writes</a>, writing new bytes as soon as the socket channel is ready to send them. When doing I/O, read/write dispatch sources can be a good high level alternative to other lower level APIs you’ll normally use on *nix platforms.</p>
<p>And on the topic of dispatch sources related to files, another one that could be useful in some use cases is <code class="inlinecode">DispatchSourceFileSystemObject</code>, that allows to listen to changes to a specific file, from its name down to changes to its attributes. With this dispatch source you’ll be also able to receive notifications if a file has been modified or deleted, essentially a subset of the events that on Linux are managed by the <em>inotify</em> kernel subsystem.</p>
<p>The remaining source types operate similarly, you can check out the complete list of what’s available in <a href="https://developer.apple.com/reference/dispatch/dispatchsource">libDispatch’s documentation</a> but remember that some of them like the Mach sources and the memory pressure source will work only on Darwin platforms.</p>
<h2 id="operations-and-operationqueues">Operations and OperationQueues</h2>
<p>Let’s talk briefly of Operation Queues, and additional API built on top of GCD, that uses concurrent queues and models tasks as Operations, that are easy to cancel and that can have their execution depend on other operations completion.</p>
<p>Operations can have a priority, which defines the order of execution, and are added to <code class="inlinecode">OperationQueues</code> that be executed asynchronously.</p>
<p>Let’s see a basic example:</p>
<pre><code class="Swift">
var queue = OperationQueue()
queue.name = "My Custom Queue"
queue.maxConcurrentOperationCount = 2
var mainqueue = OperationQueue.main //Refers to the queue of the main thread
queue.addOperation{
print("Op1")
}
queue.addOperation{
print("Op2")
}
</code></pre>
<p>We can also create a <em>Block Operation</em> object and configure it before adding it to the queue and if needed we can also add more than one closure to this kind of operations.</p>
<p>Note that <code class="inlinecode">NSInvocationOperation</code>, that creates an operation with target+selector, is not available in Swift.</p>
<pre><code class="Swift">
var op3 = BlockOperation(block: {
print("Op3")
})
op3.queuePriority = .veryHigh
op3.completionBlock = {
if op3.isCancelled {
print("Someone cancelled me.")
}
print("Completed Op3")
}
var op4 = BlockOperation {
print("Op4 always after Op3")
OperationQueue.main.addOperation{
print("I'm on main queue!")
}
}
</code></pre>
<p>Operations can have a priority and a secondary completion closure that will be run once the main closure completes.</p>
<p>We can add a dependency from <code class="inlinecode">op4</code> to <code class="inlinecode">op3</code>, so that <code class="inlinecode">op4</code> will wait for the completion of <code class="inlinecode">op3</code> to execute.</p>
<pre><code class="Swift">
op4.addDependency(op3)
queue.addOperation(op4) // op3 will complete before op4, always
queue.addOperation(op3)
</code></pre>
<p>Dependencies can also be removed with <code class="inlinecode">removeDependency(operation:)</code> and are stored in a publicly accessible <code class="inlinecode">dependencies</code> array.</p>
<p>The current state of an operation can be examined using specific properties:</p>
<pre><code class="Swift">
op3.isReady //Ready for execution?
op3.isExecuting //Executing now?
op3.isFinished //Finished naturally or cancelled?
op3.isCancelled //Manually cancelled?
</code></pre>
<p>You can cancel all the operations present in a queue calling the <code class="inlinecode">cancelAllOperations</code> method, that sets the <code class="inlinecode">isCancelled</code> flag on the operations remaining in the queue. A single operation can be canceled invoking its <code class="inlinecode">cancel</code> method:</p>
<pre><code class="Swift">
queue.cancelAllOperations()
op3.cancel()
</code></pre>
<p>It’s recommended to check the <code class="inlinecode">isCancelled</code> property inside your operation to skip execution if the operation was cancelled after it was scheduled to run by the queue.</p>
<p>And finally, you can also stop the execution of new operations on an operation queue (the currently running operation will not be affected):</p>
<pre><code class="Swift">
queue.isSuspended = true
</code></pre>
<h4 id="the-playgrounds-for-this-and-other-articles-are-available-from-github-or-zipped-3"><em>The playgrounds for this and other articles are available from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="/archives/2017-05-07-ConcurrencyInSwift.playground.zip">Zipped</a>.</em></h4>
<h2 id="closing-thoughts">Closing Thoughts</h2>
<p>This article should have given you a good summary of what is possible today from the point of view of concurrency using the external frameworks that are available from Swift.</p>
<p><a href="/2017/07/22/all-about-concurrency-in-swift-2-the-future/">Part 2 will focus on what could come next</a>, in term of language facilities that could handle concurrency “natively”, without resorting to external libraries. A few interesting paradigms will be described with the help of a few open source implementations already available today.</p>
<p>I hope that these two articles will be a good introduction to the world of concurrency and that they will help you understand and participate to the discussions that will take place on the swift-evolution mailing list when the community will start considering what to introduce in, let’s hope, Swift 5.</p>
<p>For more interesting content on concurrency and Swift, check out the <a href="https://www.cocoawithlove.com/tags/asynchrony.html">Cocoa With Love</a> blog.</p>
Building a LISP from scratch with Swift2017-02-05T00:00:00+01:00https://www.uraimo.com/2017/02/05/building-a-lisp-from-scratch-with-swiftThis article describes how you can build a simple LISP, based on the 1978 article <i>'A Micro Manual For LISP - Not The Whole Truth'</i> with Swift, taking advantage where possible of the features the language offers.<p>Some say that building a small language interpreter, especially if a LISP, is one of those things you have to do at least one time in your life as a programmer, an eye opening experience that will give you new insights into how the tools you use everyday work and demystify a few concepts that seem daunting when seen from afar.</p>
<p>In this article, we’ll implement a minimal LISP based on the 1978 paper by John McCarthy titled <em><a href="https://github.com/jaseemabid/micromanual">A Micro-Manual For Lisp - Not The Whole Thruth</a></em>, that defines a small and self-contained LISP, as a Swift framework that will be able to evaluate strings containing LISP symbolic expressions.</p>
<p>We’ll eventually use this compact interpreter to build a simple REPL (Read-Eval-Print Loop) that will interactively execute statements and print out the result of the evaluation. A playground to play around with the interpreter is also available.</p>
<p>This article will explain everything you need to known to roll out you own LISP interpreter, something that could be a great weekend project. Feel free to follow along or just read the introduction, write your own interpreter using this post as a starting point for your alternative implementation.</p>
<p>The diagram below shows the overall design of what we are going to build:</p>
<p><img src="/imgs/lisp.png" srcset="/imgs/lisp@2x.png 2x" alt="REPL Diagram" /></p>
<p>The first functional block, the <em>Read</em> phase, reads some text containing code and with a two-phase process produces an syntax tree with an internal representation of the input program.</p>
<p>The first phase, represented by the <em>Lexer</em> separates the input text in tokens (the building blocks, from a textual point of view, of the program) and then the <em>Parser</em> takes this series of tokens and produces an <em><a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Abstract Syntax Tree</a> (AST)</em>, a hierarchical representation of the source code.</p>
<p>Once we have an <em>AST</em> we’ll be able to evaluate the expression to produce a result that we’ll then print on screen for the user.</p>
<h4 id="the-library-with-the-interpreter-described-in-this-article-and-a-playground-to-test-it-are-avaliable-on-github"><em>The library with the interpreter described in this article and a playground to test it are avaliable on <a href="https://github.com/uraimo/SwiftyLISP">GitHub</a>.</em></h4>
<p><strong>Contents:</strong></p>
<ul>
<li><a href="#lisp-basics">LISP Basics</a></li>
<li><a href="#building-the-interpreter">Building the interpreter</a>
<ul>
<li><a href="#lexer-and-parser">Lexer and Parser</a></li>
<li><a href="#evaluation-and-default-global-environment">Evaluation and Default Global Environment</a></li>
</ul>
</li>
<li><a href="#swiftylisp-repl">SwiftyLisp REPL</a></li>
<li><a href="#conclusion">Conclusion</a></li>
</ul>
<h2 id="lisp-basics">LISP Basics</h2>
<p>Let’s start with a brief recap of what we are going to implement, looking into the <a href="/files/MicroManual-LISP.pdf">McCarthy’s article</a> that essentially contains the definition of the language.</p>
<p>First of all, if you are not familiar with LISP, the acronym derives from <em>LISt Processor</em> and that’s a good way do describe languages of the LISP family. Their essential data structure is the list and your programs will perform operations on those lists.</p>
<p>As you have guessed since I used the term <em>family</em> a lot of variants or dialects of the original LISP defined by John McCarthy exist nowadays, from traditional languages like Racket to languages like Clojure that are built on top of different technologies (the Java virtual machine and the Java runtime in this case) and are able to extend the underlying platform with the functionalities that a LISP can provide through its different paradigm.</p>
<p>What we are going to implement here is a minimal LISP that contains the bare essential elements to do something useful.</p>
<p>A LISP interpreter can be described as an evaluator of programs expressed using a peculiar recursive data structure called <em><a href="https://en.wikipedia.org/wiki/S-expression">symbolic expression</a></em> or <em>form</em>, something that can assume the appearance of either an <em>Atom</em> or a <em>List</em>. Atoms are simple series of alphanumeric characters that can assume different meaning whereas Lists (also called compound forms) are sequences of other symbolic expressions, represented as a sequence of values enclosed inside parenthesis.</p>
<p>An additional kind of form exists in this LISP, the <em>special form</em> that differs from other kind of symbolic expressions because has different evaluation rules for its sub-expressions.</p>
<p>To represent the data that your program will manipulate, we will again use the same symbolic expression data type, ending up using the same data structure to represent both you source code and the data it uses.</p>
<p>But what about the <em>AST</em>? A syntactically valid program is structured as a series of symbolic expression, in other words a series of nested lists, so when converting the source code to an <em>AST</em> we will again use a data structure able to store lists to model our program.</p>
<p>Languages like LISP, where programs and their internal representation can be expressed with the language fundamental data type are called <em>homoiconic</em> and this property makes meta-programming, the ability that a program has to modify itself or other programs in the same language, easier than what it is in classic non-homoiconic languages(most of those you know, Swift included). You will be able to leverage the fact that code and data share the same representation to modify your code at runtime without using complex mechanisms.</p>
<p>If you look at the end of the McCarthy’s paper you’ll notice that also building a LISP interpreter in LISP, called a <a href="http://wiki.c2.com/?MetaCircularEvaluator">Meta Circular Evaluator</a>, will just be a matter of a few lines of code.
The Swift interpreter we are about to build will do the same thing, it will evaluate these symbolic expressions recursively and will produce another symbolic expression as result.</p>
<p>But let’s see an example of a LISP program expressed using symbolic expressions:</p>
<pre><code class="lisp">
(COUNT (QUOTE (A B C) ) 42)
</code></pre>
<p>In the example above, <code class="inlinecode">COUNT</code>, <code class="inlinecode">QUOTE</code>, <code class="inlinecode">A</code>, <code class="inlinecode">B</code>, <code class="inlinecode">C</code>, <code class="inlinecode">42</code> are all atoms (let’s ignore their meaning for now), and each sequence between parenthesis is a list. Note how a list can contain any kind of symbolic expression, even sub-lists.</p>
<p>How will our interpreter evaluate this expression?</p>
<p>This expression will be evaluated as an expression that uses <a href="https://en.wikipedia.org/wiki/Polish_notation">polish notation</a>, where each list will be considered as an operator followed by the operands it needs to be applied on, e.g. a sum between two integers would be represented as <code class="inlinecode">(+ 1 2)</code>.</p>
<p>In the example above, the operator/function <code class="inlinecode">COUNT</code> will be applied on the operands/parameters <code class="inlinecode">(QUOTE (A B C))</code> and <code class="inlinecode">42</code>.</p>
<p>And you have certainly noticed that in this definition of our language, atoms are not typed, we have a single type of atom and common types like integers, booleans and strings are not available. This LISP does not have the complex type system we find in languages like Swift.</p>
<p>The micro-manual defines a series of atoms that perform basic operations and describes the value they produce once a list that contains them is evaluated. In the table below, <em>e</em> will denote generic symbolic expressions whereas <em>l</em> will be used for lists.</p>
<table>
<thead>
<tr>
<th>Atom</th>
<th>Structure</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>Quote</td>
<td>(quote e1)</td>
<td>This atom once evaluated returns its sub expression <strong>as is</strong>, <em>e.g. (quote A) = A</em></td>
</tr>
<tr>
<td>Car</td>
<td>(car l)</td>
<td>Returns the first element of a non-empty sub-list, <em>e.g. (car (quote (A B C))) = A</em></td>
</tr>
<tr>
<td>Cdr</td>
<td>(cdr l)</td>
<td>Returns all the elements of the sub-list after the first in a new list, <em>e.g. (cdr (quote (A B C))) = (B C)</em></td>
</tr>
<tr>
<td>Cons</td>
<td>(cons e l)</td>
<td>Returns a new list with e as first element and then the content of the sublist <em>e.g. (cons (quote A) (quote (B C))) = (A B C)</em></td>
</tr>
<tr>
<td>Equal</td>
<td>(equal e1 e2)</td>
<td>Returns an atom aptly named <strong>true</strong> if the two symbolic expressions are recursively equal and the empty list <strong>()</strong> (that serves as both nil and false value) if they are not, <em>e.g. (equal (car (quote (A B))) = (quote A))</em></td>
</tr>
<tr>
<td>Atom</td>
<td>(atom e)</td>
<td>Returns true if the symbolic expression is an atom or an empty list if it is a lis, <em>e.g. (atom A) = true</em></td>
</tr>
<tr>
<td>Cond</td>
<td>(cond (p1 e1) (p2 e2) … (pn en))</td>
<td>Returns the first <strong>e</strong> expression whose <strong>p</strong> predicate expression is not equal to the empty list. This is basically a conditional atom with a slightly more convoluted syntax than a common if construct. <em>e.g. (cond ((atom (quote A)) (quote B)) ((quote true) (quote C) = B</em></td>
</tr>
<tr>
<td>List</td>
<td>(list e1 e2 … en)</td>
<td>Returns a list of all the given expressions, identical to applying cons recursively to a sequence of expressions.</td>
</tr>
</tbody>
</table>
<p>The description contains the set of rules that will be used to evaluate these expressions.</p>
<p>If you look closely you’ll notice that <code class="inlinecode">cond</code> is slightly different from the others since it conditionally evaluates its body depending on the sublists it contains. This is our first example of <em>special form</em>, we’ll pay special attention to this detail when implementing the evaluator.</p>
<p>Now let’s see another category of these operators, the one that is able to define functions:</p>
<table>
<thead>
<tr>
<th>Atom</th>
<th>Structure</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>Lambda</td>
<td>( (lambda (v1 … vn) e) p1 … pn)</td>
<td>Defines a lambda expression with body <strong>e</strong> that describes an anonymous function that uses a series of environment variables <strong>v</strong>. This function will be evaluated using the provided parameters as value for the variables. <em>e.g. ((lambda (X Y) (cons (car x) y) (quote (A B)) (cdr (quote (C D)))) = (A D)</em></td>
</tr>
<tr>
<td>Defun</td>
<td>(defun <name> (v1 ... vn) e)</name></td>
<td>Define a lambda expression and registers it in the current context to be used when we need it. We’ll be able to define a function like <em>(defun cadr (X) (car (cdr x)))</em> and use it in another expression like <em>(cadr (quote (A B C D)))</em>.</td>
</tr>
</tbody>
</table>
<p>McCarthy’s paper describe an additional operator that can be used to define local labeled lambda expressions but we are not going to implement it, when we’ll need something similar we’ll use <em>defun</em> instead.</p>
<h2 id="building-the-interpreter">Building The Interpreter</h2>
<p>Now that we are done describing the content of the paper it’s time to discuss the implementation of the interpreter.</p>
<p>In this section, each functional module that composes the interpreter will be analyzed in detail, for the full code of the interpreter check out this repository on <a href="https://github.com/uraimo/SwiftyLISP">Github</a>.</p>
<p>The first aspect that need to be addressed is how symbolic expressions will be represented inside the interpreter defining how the <em>AST</em> will be structured. This is an important aspect since a good structure simplifies the evaluation.</p>
<h3 id="modeling-symbolic-expressions">Modeling Symbolic Expressions</h3>
<p>The most obvious way to model the symbolic expressions is to use a recursive <em>enum</em>:</p>
<pre><code class="Swift">
public enum SExpr{
case Atom(String)
case List([SExpr])
}
</code></pre>
<p>Usually you need <code class="inlinecode">indirect</code> when declaring a recursive enum, but in this case the array is acting as a container, so we can do without it. Other than this, there is not much to see here, this enum simply mimics the definition of symbolic expression.</p>
<p>Now let’s add a few other things to this enum, we’ll need a way to understand if two expressions are equal and a way to print them. For that we are going to implement <code class="inlinecode">Equatable</code> and <code class="inlinecode">CustomStringConvertible</code> declaring two extensions.</p>
<pre><code class="Swift">
extension SExpr : Equatable {
public static func ==(lhs: SExpr, rhs: SExpr) -> Bool{
switch(lhs,rhs){
case let (.Atom(l),.Atom(r)):
return l==r
case let (.List(l),.List(r)):
guard l.count == r.count else {return false}
for (idx,el) in l.enumerated() {
if el != r[idx] {
return false
}
}
return true
default:
return false
}
}
}
extension SExpr : CustomStringConvertible{
public var description: String {
switch self{
case let .Atom(value):
return "\(value) "
case let .List(subxexprs):
var res = "("
for expr in subxexprs{
res += "\(expr) "
}
res += ")"
return res
}
}
}
</code></pre>
<p>Both functions recursively traverse the symbolic expression structure, triggering a call to themselves (using the equality operator or converting a <code class="inlinecode">SExpr</code> to String) to perform their duty.</p>
<p>Now that the data structure has been defined let’s look into how each component of the REPL diagram can be implemented.</p>
<p><img src="/imgs/lisp.png" srcset="/imgs/lisp@2x.png 2x" alt="REPL Diagram" /></p>
<h3 id="lexer-and-parser">Lexer and Parser</h3>
<p>The <em>Read</em> phase that translates the source code to an <em>AST</em> to be evaluated can be divided in two stages, each one performed by a dedicated component: the <em>Lexer</em> and the <em>Parser</em>.</p>
<p>The main job of the <em>Lexer</em> or <em>tokenizer</em> is to perform <a href="https://en.wikipedia.org/wiki/Lexical_analysis">lexical analysis</a> on an input block of text containing the source code.</p>
<p>The lexer is able to break down a series of characters to a series of <em>lexemes</em> or <em>tokens</em> that represent strings that have meaning when considered in the context of a language. Tokens can be language keywords like <code class="inlinecode">if</code>, operators like <code class="inlinecode">=</code> or various identifiers (e.g. variable names) and literals.</p>
<p>Since the <a href="https://en.wikipedia.org/wiki/Lexical_grammar">lexical grammar</a> of our language, the definition of what a valid token is, is extremely simple, the lexer/tokenizer will also be simple. The Lexer will just identify string tokens separated by spaces or parenthesis.</p>
<p>Let’s add a <code class="inlinecode">read()</code> method to <code class="inlinecode">SExpr</code> to convert Strings to our enum representation and let’s start discussing the <em>tokenize</em> stage of the process.</p>
<pre><code class="Swift">
extension SExpr {
/**
Read a LISP string and convert it to a hierarchical S-Expression
*/
public static func read(_ sexpr:String) -> SExpr{
enum Token{
case pOpen,pClose,textBlock(String)
}
/**
Break down a string to a series of tokens
- Parameter sexpr: Stringified S-Expression
- Returns: Series of tokens
*/
func tokenize(_ sexpr:String) -> [Token] {
var res = [Token]()
var tmpText = ""
for c in sexpr.characters {
switch c {
case "(":
if tmpText != "" {
res.append(.textBlock(tmpText))
tmpText = ""
}
res.append(.pOpen)
case ")":
if tmpText != "" {
res.append(.textBlock(tmpText))
tmpText = ""
}
res.append(.pClose)
case " ":
if tmpText != "" {
res.append(.textBlock(tmpText))
tmpText = ""
}
default:
tmpText.append(c)
}
}
return res
}
// Parser
// ...
// Read: tokenize -> parse -> result
let tokens = tokenize(sexpr)
let res = parse(tokens)
return res.subexpr ?? .List([])
}
}
</code></pre>
<p>The <code class="inlinecode">tokenize</code> method will go through all the characters of the input string turning an opaque (from the point of view of syntax) string to a series of values defined in the <code class="inlinecode">Token</code> enum. The possible values will be: <em>pOpen</em> (for open parenthesis), <em>pClose</em> (for close parenthesis) and <em>textBlock</em> (for every other string, representing an atom). Everything quite straightforward, since there are no special rules that could make the content read invalid.</p>
<p>The next phase is performed by the <em><a href="https://en.wikipedia.org/wiki/Parsing#Computer_languages">Parser</a></em>.</p>
<p>The purpose of a parser is to convert a series of tokens into an <em>AST</em> that represent our code in a form easy to check for syntax errors and easy to evaluate (and optimize and compile if we are building a compiler instead of an interpreter).</p>
<p>We are going to implement a very simple <em><a href="https://en.wikipedia.org/wiki/Top-down_parsing">Top-down parser</a></em> that will consume the token array in its natural order and build the <em>AST</em>. If you plan to build a parser for a language with a more complex grammar you’d likely need something slightly more sophisticate like a <em><a href="https://en.wikipedia.org/wiki/Recursive_descent_parser">Recursive Descent Parser</a></em> (easy to hand code) or a <em><a href="https://en.wikipedia.org/wiki/LL_parser">LL Parser</a></em>.</p>
<p>But for languages with complex grammar the parser is usually generated with a parser generator (e.g. <a href="http://www.antlr.org/">ANTLR</a>, that recently introduced support for Swift), so you’ll have to describe your grammar in a DSL instead of coding manually the parser.</p>
<p>The parser will be definitely more convoluted than the lexer, but again, thanks to how simple this language is, it will be a really small and simple parser.</p>
<pre><code class="Swift">
extension SExpr {
/**
Read a LISP string and convert it to a hierarchical S-Expression
*/
public static func read(_ sexpr:String) -> SExpr{
// Tokenizer
// ...
func appendTo(list: SExpr?, node:SExpr) -> SExpr {
var list = list
if list != nil, case var .List(elements) = list! {
elements.append(node)
list = .List(elements)
}else{
list = node
}
return list!
}
/**
Parses a series of tokens to obtain a hierachical S-Expression
- Parameter tokens: Tokens to parse
- Parameter node: Parent S-Expression if available
- Returns: Tuple with remaning tokens and resulting S-Expression
*/
func parse(_ tokens: [Token], node: SExpr? = nil) -> (remaining:[Token], subexpr:SExpr?) {
var tokens = tokens
var node = node
var i = 0
repeat {
let t = tokens[i]
switch t {
case .pOpen:
//new sexpr
let (tr,n) = parse( Array(tokens[(i+1)..<tokens.count]), node: .List([]))
assert(n != nil) //Cannot be nil
(tokens, i) = (tr, 0)
node = appendTo(list: node, node: n!)
if tokens.count != 0 {
continue
}else{
break
}
case .pClose:
//close sexpr
return (Array(tokens[(i+1)..<tokens.count]), node)
case let .textBlock(value):
node = appendTo(list: node, node: .Atom(value))
}
i += 1
}while(tokens.count > 0)
return ([],node)
}
let tokens = tokenize(sexpr)
let res = parse(tokens)
return res.subexpr ?? .List([])
}
}
</code></pre>
<p>The <code class="inlinecode">parse(tokens:node:)</code> method goes through every token produced by the lexer using <code class="inlinecode">.pOpen</code> and <code class="inlinecode">.pClose</code> to delimit lists and converts every other token to atoms.</p>
<p>Notice that the parsing is performed recursively, with every nested call receiving the array of tokens left to parse and the parent list that will contain the values parsed during the next recursion step (starting with <em>nil</em> for the root expression). When a close parenthesis is found, the list is considered complete and is returned to the caller along with the remaining tokens that still have to be parsed.</p>
<p>After these functions you can see the actual body of the <code class="inlinecode">read()</code> method, that executes every step in sequence returning the top-level form or an empty list (that doubles as false as we saw in the previous section) on error.</p>
<pre><code class="Swift">
let tokens = tokenize(sexpr)
let res = parse(tokens)
return res.subexpr ?? .List([])
}
}
</code></pre>
<p>Now that we have a working read module, let’s add something to the <code class="inlinecode">SExpr</code> enum that will allow us to obtain an expression directly from a string literal without invoking manually the <code class="inlinecode">read()</code> method by implementing the <code class="inlinecode">ExpressibleByStringLiteral</code> protocol:</p>
<pre><code class="Swift">
extension SExpr : ExpressibleByStringLiteral,
ExpressibleByUnicodeScalarLiteral,
ExpressibleByExtendedGraphemeClusterLiteral {
public init(stringLiteral value: String){
self = SExpr.read(value)
}
public init(extendedGraphemeClusterLiteral value: String){
self.init(stringLiteral: value)
}
public init(unicodeScalarLiteral value: String){
self.init(stringLiteral: value)
}
}
</code></pre>
<p>With this we’ll be able to read programs directly from a string:</p>
<pre><code class="Swift">
let expr: SExpr = "(cond ((atom (quote A)) (quote B)) ((quote true) (quote C)))"
print(expr)
print(expr.eval()!) //B
</code></pre>
<h3 id="evaluation-and-default-global-environment">Evaluation and Default Global Environment</h3>
<p>The evaluation phase will be more complex than what we’ve seen until now, the <code class="inlinecode">eval()</code> function will recursively evaluate the <em>AST</em> and return the resulting evaluated symbolic expression.</p>
<p>First of all let’s collect all the basic operators defined by our language in a private dictionary called <code class="inlinecode">defaultEnvironment</code> that will associate to every operator atom name a function of type <code class="inlinecode">(SExpr, [SExpr]?, [SExpr]?)->SExpr</code> that implements it.</p>
<p>These function will take a <code class="inlinecode">SExpr</code> parameter containing the original list (function name and parameters), evaluate it and return a <code class="inlinecode">SExpr</code> as result. Those two optional arrays as second and third parameter will contain a list of variables with their values, and will be used for user-defined functions defined via <code class="inlinecode">defun</code> and <code class="inlinecode">lamdba</code>, in all the other cases they will just be nil. But we’ll come back to this when we’ll take a look at those operators.</p>
<p>To keep track of the basic builtin operators the <code class="inlinecode">Builtin</code> enum has been declared with a function that identifies which operators don’t need sub-expression evaluation. Those are operators like <code class="inlinecode">quote</code> (that exists with the sole purpose to disable sub-expressions evaluation), special forms like <code class="inlinecode">cond</code> or the lambda defining operators that will handle internally the evaluation of the sub-expressions.</p>
<pre><code class="Swift">
/// Basic builtins
fileprivate enum Builtins:String{
case quote,car,cdr,cons,equal,atom,cond,lambda,defun,list,
println,eval
/**
True if the given parameter stop evaluation of sub-expressions.
Sub expressions will be evaluated lazily by the operator.
- Parameter atom: Stringified atom
- Returns: True if the atom is the quote operator
*/
public static func mustSkip(_ atom: String) -> Bool {
return (atom == Builtins.quote.rawValue) ||
(atom == Builtins.cond.rawValue) ||
(atom == Builtins.defun.rawValue) ||
(atom == Builtins.lambda.rawValue)
}
}
</code></pre>
<p>All the <code class="inlinecode">defaultEnvironment</code> functions start with a simple check to verify that the minimum number of parameters has been provided and then proceed in building up the result to return.</p>
<p>Let’s take a look to a few of those, check the <a href="https://github.com/uraimo/SwiftyLISP">full project</a> for the complete list.</p>
<pre><code class="Swift">
/// Global default builtin functions environment
///
/// Contains definitions for: quote,car,cdr,cons,equal,atom,cond,lambda,label,defun.
private var defaultEnvironment: [String: (SExpr, [SExpr]?, [SExpr]?)->SExpr] = {
var env = [String: (SExpr, [SExpr]?, [SExpr]?)->SExpr]()
env[Builtins.quote.rawValue] = { params,locals,values in
guard case let .List(parameters) = params, parameters.count == 2 else {return .List([])}
return parameters[1]
}
env[Builtins.cdr.rawValue] = { params,locals,values in
guard case let .List(parameters) = params, parameters.count == 2 else {return .List([])}
guard case let .List(elements) = parameters[1], elements.count > 1 else {return .List([])}
return .List(Array(elements.dropFirst(1)))
}
env[Builtins.equal.rawValue] = {params,locals,values in
guard case let .List(elements) = params, elements.count == 3 else {return .List([])}
var me = env[Builtins.equal.rawValue]!
switch (elements[1].eval(with: locals,for: values)!,elements[2].eval(with: locals,for: values)!) {
case (.Atom(let elLeft),.Atom(let elRight)):
return elLeft == elRight ? .Atom("true") : .List([])
case (.List(let elLeft),.List(let elRight)):
guard elLeft.count == elRight.count else {return .List([])}
for (idx,el) in elLeft.enumerated() {
let testeq:[SExpr] = [.Atom("Equal"),el,elRight[idx]]
if me(.List(testeq),locals,values) != SExpr.Atom("true") {
return .List([])
}
}
return .Atom("true")
default:
return .List([])
}
}
env[Builtins.atom.rawValue] = { params,locals,values in
guard case let .List(parameters) = params, parameters.count == 2 else {return .List([])}
switch parameters[1].eval(with: locals,for: values)! {
case .Atom:
return .Atom("true")
default:
return .List([])
}
}
// ...
return env
}()
</code></pre>
<p>While functions like <code class="inlinecode">quote</code> or <code class="inlinecode">cdr</code> just manipulate the parameter list to build an output list, other functions like <code class="inlinecode">equal</code> implement a more complex logic (in this case it performs a recursive equality check). To keep the source readable for didactic purposes, error checks have been kept to a minimum, additional parameters are ignored and when something goes wrong the empty list is returned.</p>
<p>For special forms like the conditional <code class="inlinecode">cond</code> a different handling of the evaluation is required.</p>
<p>Conditional operators are essential to implement recursion, because only with this kind of statements we are able to decide if we have to stop the recursion or proceed with another iteration.</p>
<pre><code class="Swift">
env[Builtins.cond.rawValue] = { params,locals,values in
guard case let .List(parameters) = params, parameters.count > 1 else {return .List([])}
for el in parameters.dropFirst(1) {
guard case let .List(c) = el, c.count == 2 else {return .List([])}
if c[0].eval(with: locals,for: values) != .List([]) {
let res = c[1].eval(with: locals,for: values)
return res!
}
}
return .List([])
}
</code></pre>
<p>The implementation of <code class="inlinecode">cond</code>, once it has dropped the first element of the list containing the <em>cond</em> atom, iterates through the list until it finds a sublist where the first member is a form with a value different than the empty list (that means <em>false</em> as we already saw), and when it finds it, evaluates the second member of the sublist and returns it. With this kind of evaluation we evaluate only what we actually need and when evaluating a recursive function we don’t follow the infinite series of nested recursive calls that the body of those functions contains.</p>
<p>Among those default functions, the <code class="inlinecode">defun</code> and <code class="inlinecode">lambda</code> operators allow the creation of user-defined functions that are then registered in a globally accessible dictionary called <code class="inlinecode">localContext</code>:</p>
<pre><code class="Swift">
/// Local environment for locally defined functions
public var localContext = [String: (SExpr, [SExpr]?, [SExpr]?)->SExpr]()
</code></pre>
<p>Let’s see how <code class="inlinecode">defun</code> (<code class="inlinecode">lambda</code> is mostly identical) can be implemented.</p>
<pre><code class="Swift">
env[Builtins.defun.rawValue] = { params,locals,values in
guard case let .List(parameters) = params, parameters.count == 4 else {return .List([])}
guard case let .Atom(lname) = parameters[1] else {return .List([])}
guard case let .List(vars) = parameters[2] else {return .List([])}
let lambda = parameters[3]
let f: (SExpr, [SExpr]?, [SExpr]?)->SExpr = { params,locals,values in
guard case var .List(p) = params else {return .List([])}
p = Array(p.dropFirst(1))
// Replace parameters in the lambda with values
if let result = lambda.eval(with:vars, for:p){
return result
}else{
return .List([])
}
}
localContext[lname] = f
return .List([])
}
</code></pre>
<p>This function requires a list with four symbolic expressions, one for the operator name, one for the name (this as expected will be a simple atom) and the last two for the variables list and the lambda body respectively. Therefore, once we store each component in a constant (note that again, the empty list is used as error value), we define and register in <code class="inlinecode">localContext</code> a function with type <code class="inlinecode">(SExpr, [SExpr]?, [SExpr]?)->SExpr</code> that as we’ll see momentarily will be invoked by <code class="inlinecode">eval()</code> when the evaluator will find it in an expression.</p>
<p>During invocation, this anonymous function will evaluate the body of the lambda replacing the variables contained in the original variables list with the current parameters and return the result.</p>
<p>To better understand what is happening there, let’s finally take a look at the <code class="inlinecode">eval()</code> function:</p>
<pre><code class="Swift">
public enum SExpr{
case Atom(String)
case List([SExpr])
/**
Evaluates this SExpression with the given functions environment
- Parameter environment: A set of named functions or the default environment
- Returns: the resulting SExpression after evaluation
*/
public func eval(with locals: [SExpr]? = nil, for values: [SExpr]? = nil) -> SExpr?{
var node = self
switch node {
case .Atom:
return evaluateVariable(node, with:locals, for:values)
case var .List(elements):
var skip = false
if elements.count > 1, case let .Atom(value) = elements[0] {
skip = Builtins.mustSkip(value)
}
// Evaluate all subexpressions
if !skip {
elements = elements.map{
return $0.eval(with:locals, for:values)!
}
}
node = .List(elements)
// Obtain a a reference to the function represented by the first atom and apply it, local definitions shadow global ones
if elements.count > 0, case let .Atom(value) = elements[0], let f = localContext[value] ?? defaultEnvironment[value] {
let r = f(node,locals,values)
return r
}
return node
}
}
private func evaluateVariable(_ v: SExpr, with locals: [SExpr]?, for values: [SExpr]?) -> SExpr {
guard let locals = locals, let values = values else {return v}
if locals.contains(v) {
// The current atom is a variable, replace it with its value
return values[locals.index(of: v)!]
}else{
// Not a variable, just return it
return v
}
}
}
</code></pre>
<p>The evaluator traverses the <em>AST</em> performing different operations depending on the type of the form under evaluation.</p>
<p>When an atom is encountered, it tries to resolve it as a variable with the current context of local variables (set initially by <code class="inlinecode">defun</code> or <code class="inlinecode">lambda</code> and propagated between calls) but most of the times it will just return the atom as it is.</p>
<p>This is where the variables substitution for user-defined lambda is performed, we simply verify for each atom with <code class="inlinecode">evaluateVariable</code> if its name is present in the array of the variables and if it is, we replace the atom with the one with the same index from the values array.</p>
<p>We have more to consider when evaluating a list or compound form.</p>
<p>We’ll first try to evaluate recursively all the sub-expressions in the current list, but only if the current operator does not need to handle this evaluation itself. As said above in this simple LISP only <code class="inlinecode">quote</code>, special forms and lambda definition operator fall in this category.</p>
<p>Once the sub-expressions have been evaluated, it’s time to apply the operator to its operands performing a lookup for a lambda with the same name of the operator atom in <code class="inlinecode">localContext</code> and then in <code class="inlinecode">defaultEnvironment</code>. The order is important, since we want to be able to <em>shadow</em> the default definitions with new functions we would want to define manually.</p>
<p>If a lambda with that name exists, the function is invoked and the result returned to the previous step of the recursive evaluation.</p>
<p>This concludes the description of the basic interpreter, the whole thing needs more or less 400 lines of code.</p>
<h2 id="swiftylisp-repl">SwiftyLisp REPL</h2>
<p>It’s time to implement the REPL, but it won’t take long, the interpreter has all the basic functionalities we need.</p>
<p>We’ll read a line from the terminal, convert it to a <code class="inlinecode">SExpr</code>, evaluate it and print the result, that will be well formatted thanks to the <code class="inlinecode">CustomStringConvertible</code> protocol.</p>
<pre><code class="Swift">
import SwiftyLisp
var exit = false
while(!exit){
print(">>>", terminator:" ")
let input = readLine(strippingNewline: true)
exit = (input=="exit") ? true : false
if !exit {
let e = SExpr.read(input!)
print(e.eval()!)
}
}
</code></pre>
<p>The REPL is also available on Github in a <a href="https://github.com/uraimo/SwiftyLISP-REPL">separate repository</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>This article described a minimal LISP interpreter to show you the basic building blocks of interpreters in general regardless of the language.</p>
<p>If you never built something like this before it could seem daunting at first but I hope to have shown that it is definitely something that with a bit of work everyone can do.</p>
<p>Check out the complete project on <a href="https://github.com/uraimo/SwiftyLISP">Github</a> and let me know in the comments if you’d like to read more about interpreters and compilers!</p>
<p>For more interesting articles on building interpreters and compilers check out the <a href="https://github.com/aalhour/awesome-compilers">awesome-compilers</a> list.</p>
Unowned or Weak? Lifetime and Performance2016-10-27T00:00:00+02:00https://www.uraimo.com/2016/10/27/unowned-or-weak-lifetime-and-performanceWhile the usual explanation that when dealing with retain cycles you should choose between unowned or weak considering references lifetime is by now well known, sometimes you are still in doubt about which one you should actually use between the two and if defensively using only weak references is a good idea. This article helps you choose which one to use discussion about object lifetimes and weak references performance.<p><strong>Update 10/17:</strong> <em>Mike Ash wrote about some recent improvements to weak references handling <a href="https://mikeash.com/pyblog/friday-qa-2017-09-22-swift-4-weak-references.html">here</a>.</em></p>
<p>The usual explanation that when dealing with retain cycles you should choose between unowned or weak references considering the object lifetimes is well known, but sometimes you could be still in doubt about which one you should actually use and wondering if defensively using only weak references is a good idea.</p>
<p>In this article, after a brief introduction, I’ll analyze the differences between the two in term of lifetime and performance with excerpts from the Swift sources to, I hope, help you choose which flavor of weak reference you should use in different circumstances.</p>
<p><strong>Contents:</strong></p>
<ul>
<li><a href="#the-basics">The Basics</a></li>
<li><a href="#the-question-unowned-or-weak">The Question: Unowned or Weak?</a></li>
<li><a href="#performance-a-look-under-the-hood">Performance: A Look Under The Hood</a>
<ul>
<li><a href="#deconstructing-capture-lists-handling">Deconstructing capture lists handling</a></li>
</ul>
</li>
<li><a href="#conclusion">Conclusion</a></li>
<li><a href="#footnotes">Footnotes</a></li>
</ul>
<h4 id="get-the-playground-for-this-and-other-articles-from-github-or-zipped-click-here-instead-for-the-closure-sample-and-the-silsilgen-and-llvm-ir-output"><em>Get the playground for this and other articles from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-10-27-UnownedWeakPlayground.playground.zip">zipped</a>. Click <a href="https://www.uraimo.com/archives/closure.zip">here</a> instead, for the closure sample and the SIL,SILGen and LLVM IR output.</em></h4>
<h2 id="the-basics">The Basics</h2>
<p>As we all know, Swift leverages the good old ARC (<em>Automatic Reference Counting</em>) to manage memory and as a consequence, as we were used to with Objective-C, we’ll have to deal manually with retain cycles through a wise use of weak references.</p>
<p>If you are not familiar with ARC, you just need to know that every <em>reference type</em> instance will have a reference count (a simple integer value) associated with it, that will be used to keep count of the number of times an object instance is currently being referred to by a variable or a constant. Once that counter reaches zero, instances are deallocated and the memory and resources they held are made available again.</p>
<p>You have a retain cycle every time two instances refer in some way to each other (e.g. two classes instances that have a property that refers to the other class instance as will happen with two adjacent node instances in a doubly linked list) preventing those instances from being deallocated because the retain count has always a value greater than zero.</p>
<p>To solve this, in Swift but also in many other languages, the concept of <em>weak references</em> has been introduced, references that are not considered by ARC and that as such will not increment the retain count of your objects.</p>
<p>Considering that weak references do not prevent instances from being deallocated, it’s essential to always remember that at any point a weak reference could not point anymore to a valid object. Not a problem impossible to overcome, but something that we need to consider when we deal with this kind of references.</p>
<p>Swift has two kinds of <em>weak</em> references: <code class="inlinecode">unowned</code> and <code class="inlinecode">weak</code>.</p>
<p>While they serve the same purpose, they are slightly different in regard to the assumptions they do related to your instance lifetime and have different performance characteristics.</p>
<p>Instead of looking at this from the perspective of retain cycles between classes, we’ll discuss this in the context of closures, that from the days of Objective-C is maybe the most common situation in which you’ll have to deal with retain cycles. As happens with classes, using an external instance inside the closure creates a strong reference to that instance, or <em>captures</em> it, blocking its deallocation.</p>
<p>In Objective-C, following the standard pattern you would have declared a weak reference to that instance outside the block and then declared a strong reference to that instance inside the block to get a hold of it during the execution of the block. And obviously, checking that the reference was still valid was necessary.</p>
<p>To help deal with retain cycles, Swift introduces a new construct to simplify and make more explicit the capturing of external variables inside closures, the <em>capture list</em>. With capture lists, you can declare on top of your function the external variables that will be used specifying which kind of reference should be created internally.</p>
<p>Let’s see a few examples of what is the result of capturing variables in different ways.</p>
<p>When you don’t use capture lists, the closure will create a strong reference to the value from the outer scope:</p>
<pre><code class="Swift">
var i1 = 1, i2 = 1
var fStrong = {
i1 += 1
i2 += 2
}
fStrong()
print(i1,i2) //Prints 2 and 3
</code></pre>
<p>Modification happening inside the closure will alter the value of the original variable as you would expect.</p>
<p>Using a capture list, a new constant valid inside the closure’s scope is created instead. If you don’t specify a capture modifier the constant will simply be a copy of the original value and this will work with both value types and reference types.</p>
<pre><code class="Swift">
var fCopy = { [i1] in
print(i1,i2)
}
fStrong()
print(i1,i2) //Prints 2 and 3
fCopy() //Prints 1 and 3
</code></pre>
<p>In the example above we are declaring the <code class="inlinecode">fCopy</code> function before the call to <code class="inlinecode">fStrong</code>, and it’s when the function is declared that the private constant is initialized. As you can see, when we call the second function we still print the original value for <code class="inlinecode">i1</code>.</p>
<p>Specifying either <code class="inlinecode">weak</code> or <code class="inlinecode">unowned</code> before the name of an external variable with a reference type, this constant will be instead initialized as a weak reference to the original value, and this specific form of capturing is the one we use to break retain cycles.</p>
<pre><code class="Swift">
class aClass{
var value = 1
}
var c1 = aClass()
var c2 = aClass()
var fSpec = { [unowned c1, weak c2] in
c1.value += 1
if let c2 = c2 {
c2.value += 1
}
}
fSpec()
print(c1.value,c2.value) //Prints 2 and 2
</code></pre>
<p>The difference in how the two <code class="inlinecode">aClass</code> captured instances are handled inside the closure is a consequence of their different characteristics.</p>
<p>Unowned references are used when the original instance will never be <em>nil</em> while the closure is reachable and are declared as implicitly unwrapped optionals. Trying to use its captured value when the original instance has been deallocated will result in a crash.</p>
<p>If instead, the original instance we want to capture <em>could</em> be <em>nil</em> at some point during execution, we must declare the reference as <code class="inlinecode">weak</code> and verify that the reference is still valid before using it.</p>
<h2 id="the-question-unowned-or-weak">The Question: Unowned or Weak?</h2>
<p>Which one of the two weak reference types should you use?</p>
<p>This question can be answered simply <strong>reasoning about the lifetime</strong> of the original object and the closure that references it.</p>
<p><img src="/imgs/unownedbig.png" srcset="/imgs/unownedbig@2x.png 2x" alt="unowned vs weak" /></p>
<p>There are two possible scenarios:</p>
<ul>
<li>
<p>The closure has the same lifetime of the captured object, so the closure will be reachable only until the object will be reachable. The external object and the closure have the same lifetime (e.g. simple back-references between and object and its parent). In this case, you should declare the reference as <strong>unowned</strong>.</p>
<p>A common example is the <code class="inlinecode">[unowned self]</code> used in many examples of small closures that do something in the context of their parent and that not being referenced <em>or passed</em> anywhere else do not outlive their parents.</p>
</li>
<li>
<p>The closure lifetime is independent from the one of the object being captured, the closure could still be referenced when the object is not reachable anymore. In this case you should declare the reference as <strong>weak</strong> and verify it’s not nil before using it (don’t force unwrap).</p>
<p>A common example of this is the <code class="inlinecode">[weak delegate = self.delegate!]</code> you can see in some examples of closure referencing a completely unrelated (lifetime-wise) delegate object.</p>
</li>
</ul>
<p>What if you are unsure about the lifetime relationship between two objects and you don’t want to risk having an invalid unowned reference? Always capturing defensively as <em>weak</em> could be a good approach?</p>
<p>No, and not only because having a clear idea of your objects lifetime is a good thing, the two attributes have also wildly different performance characteristics.</p>
<p>The most common implementation for weak references requires that each time a new reference is created it must be registered in a side-table where every weak reference is associated with the object it refers to.</p>
<p>When an object does not have any strong references pointing to it, the runtime will start the deallocation process but before this happens, it will set to <em>nil</em> all the weak references that were pointing to the object. Because of this behavior, weak references implemented this way are called <em>zeroing weak references</em>.</p>
<p>This implementation has a tangible overhead if you consider that an additional data structure needs to be maintained and that we need to guarantee the correctness of all operations on these global reference holding structures even in presence of concurrent access. It should not be possible under any circumstance to access the value pointed by a weak reference once the deallocation process has started.</p>
<p>Weak references (<em>unowned</em> and with some variations <em>weak</em> too) in Swift employ a less convoluted and faster mechanism instead.</p>
<p>Every object in Swift keeps two reference counters, the usual <em>strong reference counter</em>, used to decide when ARC will be able to safely <em>deinitialize</em> an object and an additional <em>weak reference counter</em> that counts how many <em>unowned</em> or <em>weak</em> references have been created toward this object, when this counter reaches zero, the object is <em>deallocated</em>.</p>
<p>It is important to understand that an object is not really deallocated until all its <em>unowned</em> references have been released, it will be kept reachable but it will be in an uninitialized state, with its content being just garbage after the deinitialization occurs.</p>
<p>Every time and <em>unowned</em> reference is declared its <em>unowned reference counter</em> is incremented atomically (using <a href="http://llvm.org/docs/Atomics.html#libcalls-atomic">atomic gcc/llvm operations</a>, that allow to perform basic operations like increment, decrement, compare and compare&swap in a fast and thread-safe way) to guarantee thread-safety and each time it’s used, the strong reference count will be checked to understand if the object it’s still valid before safely retaining it.</p>
<p>Trying to access an invalid object will result in a failed assertion and your application will fail gracefully with an error at runtime (that’s why this <em>unowned</em> implementation is defined <em>unowned(safe)</em> implementation).</p>
<p>As further optimization, if your application is compiled with <code class="inlinecode">-OFast</code> <em>unowned</em> references will not be checked for object validity anymore and the references will behave like the <em>__unsafe_unretained</em> you have in Objective-C. If the object is invalid, your reference will point to deinitialized garbage memory (implementation known as <em>unowned(unsafe)</em>).</p>
<p>When an <em>unowned</em> reference is released, if there are no more <em>strong</em> or <em>unowned</em> references the object will finally be deallocated and this is why the object cannot just be deallocated completely when the strong counter reaches zero, all the reference counters must still be accessible to verify the <em>unowned</em> and <em>strong</em> count.</p>
<p>Swift’s <em>weak</em> references add an additional layer of indirection wrapping <em>unowned</em> references in an optional container, something that is useful to handle cleanly all those cases when the pointed object could become null after a deallocation. But this does not come for free, additional machinery is required to manage this optional correctly.</p>
<p>Considering all this, the use of <em>unowned</em> <strong>should be preferred</strong> over <em>weak</em> every time it’s possible and lifetime relationships allow it. But this is not the end of our story, let’s talk about performance<sup><a href="#myfootnote1">1</a></sup> now.</p>
<h2 id="performance-a-look-under-the-hood">Performance: A Look Under The Hood</h2>
<p>Before we look into the source of the Swift project to verify what we said in the previous section, we need to understand how each kind of reference is managed by ARC and to do that, I need to explain a few things about swiftc, LLVM and SIL.</p>
<p>I’ll try to give you a short overview and explain only what is strictly necessary, if you want to learn more you’ll find some useful links in the footnotes.</p>
<p>Let’s start with a diagram that contains the basic functional blocks of <em>swiftc</em>, the Swift compiler, to give you an idea of what the whole compilation process entails:</p>
<p><img src="/imgs/swiftc.png" srcset="/imgs/swiftc@2x.png 2x" alt="swiftc block diagram" /></p>
<p>Swiftc follows an approach for the most part similar to other compilers built on top of LLVM like <em>clang</em>.</p>
<p>In the first part of the compilation process, managed by a specific language frontend, the source code is parsed to produce an AST representation<sup><a href="#myfootnote2">2</a></sup> of your source code and the resulting AST is then analyzed from a semantic point of view to identify semantic errors.</p>
<p>At this point, in other LLVM-based compilers, after an additional step that performs a static analysis of your code (and if necessary displays errors and warnings) through a dedicated component, the content of the AST is converted to a light-weight and low-level machine independent representation called <a href="http://llvm.org/docs/LangRef.html">LLVM IR</a> (LLVM Intermediate Representation) by the <em>IRGen</em> component.</p>
<p>These two components, static analyzer and IRGen, are separated even if some checks needs to be performed in both of them, there is usually a lot of code duplication between these two modules.</p>
<p>The IR is a <a href="https://en.wikipedia.org/wiki/Static_single_assignment_form">Static Single Assignment form</a> (SSA-form) compliant language and can be considered the RISC-style <a href="https://idea.popcount.org/2013-07-24-ir-is-better-than-assembly/">assembly language</a> of the LLVM register-based virtual machine. Being SSA based simplifies greatly the next step of the compilation process, where multiple passes of optimization are applied to the IR obtained from the internal representation provided by the language frontend.</p>
<p>It’s important to know, that one of the characteristics of IR is that it can be represented in three different forms: an in-memory representation (used internally), a serialized bitcode representation (the same bitcode <a href="https://developer.apple.com/library/tvos/documentation/IDEs/Conceptual/AppDistributionGuide/AppThinning/AppThinning.html">you already know</a>) and a human-readable form.</p>
<p>This last form is quite useful to verify the final structure of the IR code that will be passed to the last step of the process, that will convert our machine-independent IR to a platform-specific representation (e.g. x86, ARM, etc…). This last step will be performed by dedicated LLVM platform backends.</p>
<p>But what makes swiftc different from other compilers based on LLVM?</p>
<p>The fundamental structural difference between swiftc and other compilers is the presence of an additional component, <em>SILGen</em>, right before <em>IRGen</em> that performs diagnostic and optimization passes on your source producing an intermediate high-level representation called <em>SIL</em>(Swift Intermediate Language), that will be converted to LLVM IR. This allows to consolidate in a single software component all the language-specific checks and simplifies <em>IRGen</em>.</p>
<p>The conversion from AST to IR is a two step process. <em>SILGen</em> converts the source represented as an AST into <em>raw SIL</em> and then the compiler performs Swift diagnostic checks (printing errors or warnings if necessary) and optimizes the validated raw SIL through multiple passes producing <em>canonical SIL</em>. As the diagram above shows, the canonical SIL is then converted into LLVM IR.</p>
<p>SIL<sup><a href="#myfootnote3">3</a></sup> is, again, a SSA-form language and extends the Swift syntax with additional constructs. It relies on Swift’s type system and is able to understand Swift declaration but it’s important to remember that top level Swift code or function content will be ignored when compiling hand-coded (yes, we can write SIL and compile it) SIL sources.</p>
<p>In the second half of this section, we’ll analyze an example of canonical SIL to understand how unowned and weak references are handled by the compiler. Looking at the SIL generated from our code, a basic closure with a capture list, we’ll be able to see all the ARC-related function calls added by the compiler.</p>
<h4 id="get-the-playground-for-this-and-other-articles-from-github-or-zipped-click-here-instead-for-the-closure-sample-and-the-silsilgen-and-llvm-ir-output-1"><em>Get the playground for this and other articles from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-10-27-UnownedWeakPlayground.playground.zip">zipped</a>. Click <a href="https://www.uraimo.com/archives/closure.zip">here</a> instead for the closure sample and the SIL,SILGen and LLVM IR output.</em></h4>
<h3 id="deconstructing-capture-lists-handling">Deconstructing capture lists handling</h3>
<p>Let’s start with this simple Swift example that declares two variables and captures them weakly in a closure:</p>
<pre><code class="Swift">
class aClass{
var value = 1
}
var c1 = aClass()
var c2 = aClass()
var fSpec = {
[unowned c1, weak c2] in
c1.value = 42
if let c2o = c2 {
c2o.value = 42
}
}
fSpec()
</code></pre>
<p>To generate canonical SIL for this sample, just compile the swift source file with <code class="inlinecode">xcrun swiftc -emit-sil sample.swift</code>. Raw SIL can be generated using the <code class="inlinecode">-emit-silgen</code> option.</p>
<p>If you run the command above you’ll notice that swiftc produces quite a lot of code, let’s take a look at an excerpt of the swiftc output to learn what some basic SIL directives do and to understand the overall structure.</p>
<p>I’ve added a few multiline comments with explanations (convenient single line comments are generated by the compiler) where needed that should be enough to clarify what’s happening:</p>
<pre><code class="Swift">
/*
This file contains canonical SIL
*/
sil_stage canonical
/*
Some special import available only internally that can be used in SIL
*/
import Builtin
import Swift
import SwiftShims
/*
Definitions for three global variables for c1,c2 and the fSpec closure
@_Tv4clos2c1CS_6aClass is the symbol name for this variable and $aClass
its type (types start with $). Variable names are mangled here but can
be transformed in something more readable as we'll see below.
*/
// c1
sil_global hidden @_Tv4sample2c1CS_6aClass : $aClass
// c2
sil_global hidden @_Tv4sample2c2CS_6aClass : $aClass
// fSpec
sil_global hidden @_Tv4sample5fSpecFT_T_ : $@callee_owned () -> ()
...
/*
A hierarchical scope definition that refers to positions in the original source.
Each SIL instruction will point to the sil_scope it was generated from.
*/
sil_scope 1 { parent @main : $@convention(c) (Int32, UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>) -> Int32 }
sil_scope 2 { loc "sample.swift":14:1 parent 1 }
/*
An autogenerated @main function that contains the code of our original global
scope.
It follows the familiar c main() structure accepting the number of
arguments and an arguments array. The function conforms to the c calling convention.
This function contains the instructions needed to invoke the closure above.
*/
// main
sil @main : $@convention(c) (Int32, UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>) -> Int32 {
/*
Registers starts with a % followed by a numeric id.
Every time a new register is defined (or at the beginning of a function for function
parameters) the compiler adds a trailing comment with the list of registers or instructions
that depend on its value (called users).
For other instructions, the id of the current instruction is provided.
In this case, register 0 will be used to calculate the content of register 4 and register 1
will be used to create the value of register 10.
*/
// %0 // user: %4
// %1 // user: %10
/*
Every function is decomposed in a series of basic blocks of instructions and each block ends
with a terminating instruction (a branch or a return).
This graph of blocks represents all the possible execution paths of the function.
*/
bb0(%0 : $Int32, %1 : $UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>):
...
/*
Each SIL instruction has a reference to the source location that contains the Swift
instruction from which it originated and a reference to the scope it's part of.
We'll look to some of this below when analyzing this method.
*/
unowned_retain %27 : $@sil_unowned aClass, loc "sample.swift":9:14, scope 2 // id: %28
store %27 to %2 : $*@sil_unowned aClass, loc "sample.swift":9:14, scope 2 // id: %29
%30 = alloc_box $@sil_weak Optional<aClass>, var, name "c2", loc "sample.swift":9:23, scope 2 // users: %46, %44, %43, %31
%31 = project_box %30 : $@box @sil_weak Optional<aClass>, loc "sample.swift":9:23, scope 2 // user: %35
%32 = load %19 : $*aClass, loc "sample.swift":9:23, scope 2 // users: %34, %33
...
}
...
/*
A series of autogenerated methods for aClass, init/deinit,
setter/getter and other utility methods.
The comments added by the compiler clarify what they do.
*/
/*
Hidden function are visible only inside their module.
@convention(method) is the default Swift method calling convention, an additional
parameter is added at the end to contain a reference to self.
*/
// aClass.__deallocating_deinit
sil hidden @_TFC4clos6aClassD : $@convention(method) (@owned aClass) -> () {
...
}
/*
@guaranteed parameters are guaranteed to be valid for all the duration of the call.
*/
// aClass.deinit
sil hidden @_TFC4clos6aClassd : $@convention(method) (@guaranteed aClass) -> @owned Builtin.NativeObject {
...
}
/*
Functions annotated with [transparent] are small function that can be inlined.
*/
// aClass.value.getter
sil hidden [transparent] @_TFC4clos6aClassg5valueSi : $@convention(method) (@guaranteed aClass) -> Int {
...
}
// aClass.value.setter
sil hidden [transparent] @_TFC4clos6aClasss5valueSi : $@convention(method) (Int, @guaranteed aClass) -> () {
...
}
// aClass.value.materializeForSet
sil hidden [transparent] @_TFC4clos6aClassm5valueSi : $@convention(method) (Builtin.RawPointer, @inout Builtin.UnsafeValueBuffer, @guaranteed aClass) -> (Builtin.RawPointer, Optional<Builtin.RawPointer>) {
...
}
/*
@owned specifies that the object is owned by the caller.
*/
// aClass.init() -> aClass
sil hidden @_TFC4clos6aClasscfT_S0_ : $@convention(method) (@owned aClass) -> @owned aClass {
...
}
// aClass.__allocating_init() -> aClass
sil hidden @_TFC4clos6aClassCfT_S0_ : $@convention(method) (@thick aClass.Type) -> @owned aClass {
...
}
/*
The closure.
*/
// (closure #1)
sil shared @_TF4closU_FT_T_ : $@convention(thin) (@owned @sil_unowned aClass, @owned @box @sil_weak Optional<aClass>) -> () {
...
/* SIL for the closure, see below */
...
}
...
/*
sil_vtable defines the virtual function table for the aClass class.
It contains as expected all the autogenerated methods.
*/
sil_vtable aClass {
#aClass.deinit!deallocator: _TFC4clos6aClassD // aClass.__deallocating_deinit
#aClass.value!getter.1: _TFC4clos6aClassg5valueSi // aClass.value.getter
#aClass.value!setter.1: _TFC4clos6aClasss5valueSi // aClass.value.setter
#aClass.value!materializeForSet.1: _TFC4clos6aClassm5valueSi // aClass.value.materializeForSet
#aClass.init!initializer.1: _TFC4clos6aClasscfT_S0_ // aClass.init() -> aClass
}
</code></pre>
<p>Now let’s go back to the main function, to see how the two class instances are retrieved and passed to the closure when it’s invoked.</p>
<p>This time all the symbols are demangled<sup><a href="#myfootnote4">4</a></sup> to make the snippet slightly more readable:</p>
<pre><code class="Swift">
// main
sil @main : $@convention(c) (Int32, UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>) -> Int32 {
// %0 // user: %4
// %1 // user: %10
bb0(%0 : $Int32, %1 : $UnsafeMutablePointer<Optional<UnsafeMutablePointer<Int8>>>):
...
/*
References to the global variables are placed in three registers.
*/
%13 = global_addr @clos.c1 : $*aClass, loc "sample.swift":5:5, scope 1 // users: %26, %17
...
%19 = global_addr @clos.c2 : $*aClass, loc "sample.swift":6:5, scope 1 // users: %32, %23
...
%25 = global_addr @clos.fSpec : $*@callee_owned () -> (), loc "sample.swift":8:5, scope 1 // users: %48, %45
/*
c1 is unowned_retained.
This instruction increments the unowned reference count of the variable.
*/
%26 = load %13 : $*aClass, loc "sample.swift":9:14, scope 2 // user: %27
%27 = ref_to_unowned %26 : $aClass to $@sil_unowned aClass, loc "sample.swift":9:14, scope 2 // users: %47, %38, %39, %29, %28
unowned_retain %27 : $@sil_unowned aClass, loc "sample.swift":9:14, scope 2 // id: %28
store %27 to %2 : $*@sil_unowned aClass, loc "sample.swift":9:14, scope 2 // id: %29
/*
For c2 the process is more complex.
alloc_box creates a reference-counted container for this variable that will be stored
on the heap.
After the box has been created, an optional variable is initialized to point to c2 and stored
in the box. The box retains the value it contains, so, as you see below, once the box is
populated, the optional can be released.
At one point, while the value of c2 is being stored in the optional, the object is
temporarily strong_retained and then released.
*/
%30 = alloc_box $@sil_weak Optional<aClass>, var, name "c2", loc "sample.swift":9:23, scope 2 // users: %46, %44, %43, %31
%31 = project_box %30 : $@box @sil_weak Optional<aClass>, loc "sample.swift":9:23, scope 2 // user: %35
%32 = load %19 : $*aClass, loc "sample.swift":9:23, scope 2 // users: %34, %33
strong_retain %32 : $aClass, loc "sample.swift":9:23, scope 2 // id: %33
%34 = enum $Optional<aClass>, #Optional.some!enumelt.1, %32 : $aClass, loc "sample.swift":9:23, scope 2 // users: %36, %35
store_weak %34 to [initialization] %31 : $*@sil_weak Optional<aClass>, loc "sample.swift":9:23, scope 2 // id: %35
release_value %34 : $Optional<aClass>, loc "sample.swift":9:23, scope 2 // id: %36
/*
A reference to the closure is retrieved.
*/
// function_ref (closure #1)
%37 = function_ref @sample.(closure #1) : $@convention(thin) (@owned @sil_unowned aClass, @owned @box @sil_weak Optional<aClass>) -> (), loc "sample.swift":8:13, scope 2 // user: %44
/*
c1 is tagged with @unowned and the variable is then unowned_retained.
*/
strong_retain_unowned %27 : $@sil_unowned aClass, loc "sample.swift":8:13, scope 2 // id: %38
%39 = unowned_to_ref %27 : $@sil_unowned aClass to $aClass, loc "sample.swift":8:13, scope 2 // users: %42, %40
%40 = ref_to_unowned %39 : $aClass to $@sil_unowned aClass, loc "sample.swift":8:13, scope 2 // users: %44, %41
unowned_retain %40 : $@sil_unowned aClass, loc "sample.swift":8:13, scope 2 // id: %41
strong_release %39 : $aClass, loc "sample.swift":8:13, scope 2 // id: %42
/*
The box containing an optional with the value of c2 is strong_retained.
*/
strong_retain %30 : $@box @sil_weak Optional<aClass>, loc "sample.swift":8:13, scope 2 // id: %43
/*
Creates a closure object binding the function to its parameters.
*/
%44 = partial_apply %37(%40, %30) : $@convention(thin) (@owned @sil_unowned aClass, @owned @box @sil_weak Optional<aClass>) -> (), loc "sample.swift":8:13, scope 2 // user: %45
store %44 to %25 : $*@callee_owned () -> (), loc "sample.swift":8:13, scope 2 // id: %45
/*
Performs release on the c1 and c2's box variables (using the matching *_release functions).
*/
strong_release %30 : $@box @sil_weak Optional<aClass>, loc "sample.swift":14:1, scope 2 // id: %46
unowned_release %27 : $@sil_unowned aClass, loc "sample.swift":9:14, scope 2 // id: %47
/*
Loads the previously stored closure object, retains it strongly and invoke the function.
*/
%48 = load %25 : $*@callee_owned () -> (), loc "sample.swift":17:1, scope 2 // users: %50, %49
strong_retain %48 : $@callee_owned () -> (), loc "sample.swift":17:1, scope 2 // id: %49
%50 = apply %48() : $@callee_owned () -> (), loc "sample.swift":17:7, scope 2
...
}
</code></pre>
<p>The closure has a more complex structure:</p>
<pre><code class="Swift">
/*
The closure parameters are annotated with @sil annotations that specify how they will be
retained, we have an unowned aClass, c1, and a weak box with and optional containing c2.
*/
// (closure #1)
sil shared @clos.fSpec: $@convention(thin) (@owned @sil_unowned aClass, @owned @box @sil_weak Optional<aClass>) -> () {
// %0 // users: %24, %6, %5, %2
// %1 // users: %23, %3
/*
This function has three blocks, with the last two being executed conditionally depending
on the value of the c2 optional.
*/
bb0(%0 : $@sil_unowned aClass, %1 : $@box @sil_weak Optional<aClass>):
...
/*
c1 is strongly retained.
*/
strong_retain_unowned %0 : $@sil_unowned aClass, loc "sample.swift":10:5, scope 17 // id: %5
%6 = unowned_to_ref %0 : $@sil_unowned aClass to $aClass, loc "sample.swift":10:5, scope 17 // users: %11, %10, %9
/*
Using the internal Builtin package, an Int with value 42 is initialized using an integer
literal as parameter for an Int struct.
This value is then set as new value of c1 and once done the variable is released.
The class_method instruction that we see here for the first time, retrieves a reference
to a function from the vtable of an object.
*/
%7 = integer_literal $Builtin.Int64, 42, loc "sample.swift":10:16, scope 17 // user: %8
%8 = struct $Int (%7 : $Builtin.Int64), loc "sample.swift":10:16, scope 17 // user: %10
%9 = class_method %6 : $aClass, #aClass.value!setter.1 : (aClass) -> (Int) -> () , $@convention(method) (Int, @guaranteed aClass) -> (), loc "sample.swift":10:14, scope 17 // user: %10
%10 = apply %9(%8, %6) : $@convention(method) (Int, @guaranteed aClass) -> (), loc "sample.swift":10:14, scope 17
strong_release %6 : $aClass, loc "sample.swift":10:16, scope 17 // id: %11
/*
And now it's the turn of c2.
The optional is retrieved and branch to one the the last to blocks is performed depending
on its content.
If the optional has a value the bb2 block will be executed before jumping
to bb3, if it doesn't after a brief jump to bb1, the function will proceed to bb3 releasing
the retained parameters.
*/
%12 = load_weak %3 : $*@sil_weak Optional<aClass>, loc "sample.swift":11:18, scope 18 // user: %13
switch_enum %12 : $Optional<aClass>, case #Optional.some!enumelt.1: bb2, default bb1, loc "sample.swift":11:18, scope 18 // id: %13
bb1: // Preds: bb0
/*
Jumps to the end of the closure.
*/
br bb3, loc "sample.swift":11:18, scope 16 // id: %14
// %15 // users: %21, %20, %19, %16
bb2(%15 : $aClass): // Preds: bb0
/*
Invokes the setter for aClass setting a value of 42 and procedes.
*/
...
%17 = integer_literal $Builtin.Int64, 42, loc "sample.swift":12:21, scope 19 // user: %18
%18 = struct $Int (%17 : $Builtin.Int64), loc "sample.swift":12:21, scope 19 // user: %20
%19 = class_method %15 : $aClass, #aClass.value!setter.1 : (aClass) -> (Int) -> () , $@convention(method) (Int, @guaranteed aClass) -> (), loc "sample.swift":12:19, scope 19 // user: %20
%20 = apply %19(%18, %15) : $@convention(method) (Int, @guaranteed aClass) -> (), loc "sample.swift":12:19, scope 19
strong_release %15 : $aClass, loc "sample.swift":13:5, scope 18 // id: %21
br bb3, loc "sample.swift":13:5, scope 18 // id: %22
bb3: // Preds: bb1 bb2
/*
Releases both captured parameters and returns.
*/
strong_release %1 : $@box @sil_weak Optional<aClass>, loc "sample.swift":14:1, scope 17 // id: %23
unowned_release %0 : $@sil_unowned aClass, loc "sample.swift":14:1, scope 17 // id: %24
%25 = tuple (), loc "sample.swift":14:1, scope 17 // user: %26
return %25 : $(), loc "sample.swift":14:1, scope 17 // id: %26
}
</code></pre>
<p>At this point, ignoring for a moment the performance characteristics of the various ARC instructions we can do a quick recap of what needs to be done for each kind of captured variable at different stages:</p>
<table>
<thead>
<tr>
<th>Action</th>
<th>Unowned</th>
<th>Weak</th>
</tr>
</thead>
<tbody>
<tr>
<td>Pre-call #1</td>
<td>unowned_retain the object</td>
<td>Create a @box, strong_retain the object, create an optional and store it in the @box,release the optional</td>
</tr>
<tr>
<td>Pre-call #2</td>
<td>strong_retain_unowned, unowned_retain and strong_release</td>
<td>strong_retain</td>
</tr>
<tr>
<td>Closure execution</td>
<td>strong_retain_unowned, unowned_release</td>
<td>load_weak, switch on Optional, strong_release</td>
</tr>
<tr>
<td>Post-call</td>
<td>unowned_release</td>
<td>strong_release</td>
</tr>
</tbody>
</table>
<p>As we saw in the SIL excerpts above, handling weak references involves more work because they make use of an optional that needs to be handled.</p>
<p>Here is a brief explanation of what each one of the ARC instructions listed above do as described in the <a href="https://github.com/apple/swift/blob/master/docs/SIL.rst"">documentation</a>:</p>
<ul>
<li><strong>unowned_retain:</strong> <em>Increments the unowned reference count of the heap object.</em></li>
<li><strong>strong_retain_unowned:</strong> <em>Asserts that the strong reference count of the object is still positive, then increases it by one.</em></li>
<li><strong>strong_retain:</strong> <em>Increases the strong retain count of the object.</em></li>
<li><strong>load_weak:</strong> <em>Not really an ARC call but it increments the strong reference count of the object referenced by the optional.</em></li>
<li><strong>strong_release:</strong> <em>Decrements the strong reference count of the object. If the release operation brings the strong reference count of the object to zero, the object is destroyed and the weak references are cleared. When both its strong and unowned reference counts reach zero, the object’s memory is deallocated.</em></li>
<li><strong>unowned_release:</strong> <em>Decrements the unowned reference count of the object. When both its strong and unowned reference counts reach zero, the object’s memory is deallocated.</em></li>
</ul>
<p>Now let’s dig deeper in the Swift runtime to see how these instructions are implemented, the files that contain what we need are <a href="https://github.com/apple/swift/blob/master/stdlib/public/runtime/HeapObject.cpp">HeapObject.cpp</a>, <a href="https://github.com/apple/swift/blob/master/include/swift/Runtime/HeapObject.h">HeapObject.h</a>, <a href="https://github.com/apple/swift/blob/master/stdlib/public/SwiftShims/RefCount.h">RefCount.h</a> and for a few minor definitions <a href="https://github.com/apple/swift/blob/master/stdlib/public/runtime/Heap.cpp">Heap.cpp</a> and <a href="https://github.com/apple/swift/blob/master/stdlib/public/runtime/SwiftObject.mm">SwiftObject.mm</a>. Boxes implementation can be found in <a href="https://github.com/apple/swift/blob/master/stdlib/public/runtime/MetadataImpl.h">MetadataImpl.h</a>, but I will not talk about them in this post.</p>
<p>Many of the ARC functions declared in these file come in three variants, a basic implementation for Swift objects and two additional implementations for non native Swift objects: Bridge objects and Unknown objects. The last two variants will not be discussed here.</p>
<p>The first set of instructions we’ll discuss is the one related to unowned references.</p>
<p>The functions that implement <code class="inlinecode">unowned_retain</code> and <code class="inlinecode">unowned_release</code> can be found halfway through <a href="https://github.com/apple/swift/blob/master/stdlib/public/runtime/HeapObject.cpp">HeapObject.cpp</a>:</p>
<pre><code class="cpp">
SWIFT_RT_ENTRY_VISIBILITY
void swift::swift_unownedRetain(HeapObject *object)
SWIFT_CC(RegisterPreservingCC_IMPL) {
if (!object)
return;
object->weakRefCount.increment();
}
SWIFT_RT_ENTRY_VISIBILITY
void swift::swift_unownedRelease(HeapObject *object)
SWIFT_CC(RegisterPreservingCC_IMPL) {
if (!object)
return;
if (object->weakRefCount.decrementShouldDeallocate()) {
// Only class objects can be weak-retained and weak-released.
auto metadata = object->metadata;
assert(metadata->isClassObject());
auto classMetadata = static_cast<const ClassMetadata*>(metadata);
assert(classMetadata->isTypeMetadata());
SWIFT_RT_ENTRY_CALL(swift_slowDealloc)
(object, classMetadata->getInstanceSize(),
classMetadata->getInstanceAlignMask());
}
}
</code></pre>
<p>While <code class="inlinecode">swift_unownedRetain</code>, the implementation of <code class="inlinecode">unowned_retain</code>, simply increments atomically the unowned reference count (here called <code class="inlinecode">weakRefCount</code>), <code class="inlinecode">swift_unownedRelease</code> is more complex because as described above it needs to handle object deallocation, performing it only when there are no other unowned references left.</p>
<p>But nothing particularly complex here, as you can see <a href="https://github.com/apple/swift/blob/master/stdlib/public/SwiftShims/RefCount.h#L242">here</a> the <code class="inlinecode">doDecrementShouldDeallocate</code> function, called by a similarly named function in the snippet above, doesn’t do much and <code class="inlinecode">swift_slowDealloc</code> just frees the given pointer.</p>
<p>And once we have an unowned reference to an object, another instruction, <code class="inlinecode">strong_retain_unowned</code>, is used to create a strong reference:</p>
<pre><code class="cpp">
SWIFT_RT_ENTRY_VISIBILITY
void swift::swift_unownedRetainStrong(HeapObject *object)
SWIFT_CC(RegisterPreservingCC_IMPL) {
if (!object)
return;
assert(object->weakRefCount.getCount() &&
"object is not currently weakly retained");
if (! object->refCount.tryIncrement())
_swift_abortRetainUnowned(object);
}
</code></pre>
<p>Since this object should already be weakly referenced, an assert is performed to verify that the object is indeed weakly retained and once done, an attempt to increment its strong retain count is performed. The attempt will fail if the object is in the process of being deallocated.</p>
<p>All the functions like <code class="inlinecode">tryIncrement</code> that modify in some way the retain counters are located in <a href="https://github.com/apple/swift/blob/master/stdlib/public/SwiftShims/RefCount.h">RefCount.h</a> and require just a few atomic operations to perform their task.</p>
<p>Let’s talk about weak references now, as we saw before, <code class="inlinecode">swift_weakLoadStrong</code> is used to obtain a strong reference to the object contained in the optional:</p>
<pre><code class="cpp">
HeapObject *swift::swift_weakLoadStrong(WeakReference *ref) {
if (ref->Value == (uintptr_t)nullptr) {
return nullptr;
}
// ref might be visible to other threads
auto ptr = __atomic_fetch_or(&ref->Value, WR_READING, __ATOMIC_RELAXED);
while (ptr & WR_READING) {
short c = 0;
while (__atomic_load_n(&ref->Value, __ATOMIC_RELAXED) & WR_READING) {
if (++c == WR_SPINLIMIT) {
std::this_thread::yield();
c -= 1;
}
}
ptr = __atomic_fetch_or(&ref->Value, WR_READING, __ATOMIC_RELAXED);
}
auto object = (HeapObject*)(ptr & ~WR_NATIVE);
if (object == nullptr) {
__atomic_store_n(&ref->Value, (uintptr_t)nullptr, __ATOMIC_RELAXED);
return nullptr;
}
if (object->refCount.isDeallocating()) {
__atomic_store_n(&ref->Value, (uintptr_t)nullptr, __ATOMIC_RELAXED);
SWIFT_RT_ENTRY_CALL(swift_unownedRelease)(object);
return nullptr;
}
auto result = swift_tryRetain(object);
__atomic_store_n(&ref->Value, ptr, __ATOMIC_RELAXED);
return result;
}
</code></pre>
<p>Obtaining a strong reference in this case requires <a href="https://github.com/apple/swift/pull/1454">more complex synchronization</a> that will reduce performance under heavy thread contention.</p>
<p>The <code class="inlinecode">WeakReference</code> object we see here for the first time is a simple struct that contains an integer <code class="inlinecode">Value</code> field pointing to the target object that as every Swift object is represented in the runtime with the <code class="inlinecode">HeapObject</code> class.</p>
<p>Right after the weak reference in acquired for the current thread setting the <code class="inlinecode">WR_READING</code> flag, the Swift object is retrieved from the <code class="inlinecode">WeakReference</code> container and if it’s not valid anymore or if it has become eligible for deallocation while we were waiting to acquire the resource, the current reference is set to <em>null</em>.</p>
<p>If the object is still valid, an attempt to retain is performed as expected.</p>
<p>Therefore, even from this point of view we can expect the performance of weak reference during common operations to be lower than what we would expect from simpler unowned references (but from what I’ve seen the major overhead seems to be optional handling).</p>
<h2 id="conclusion">Conclusion</h2>
<p>Does using defensively only weak references make sense? No, from both the point of view of performance and code clarity.</p>
<p>Using the right type of capture modifier makes explicit some lifetime characteristics of our code and makes it harder to reach wrong conclusions about how the code behaves when someone else, or future you, will read what you wrote.</p>
<h3 id="footnotes">Footnotes</h3>
<div style="font-size: 0.9em">
<br /><a name="myfootnote1" style="font-weight:bold">1</a>: <i> The first discussion on the weak/unowned dilemma with input from Apple can be found <a href="https://devforums.apple.com/message/987086#987086">here</a>, and a later discussion on twitter with Joe Groff has been <a href="http://mjtsai.com/blog/2015/11/24/how-swift-implements-unowned-and-weak-references/">summarized here by Michael Tsai</a>. This article starts from there with the intention of providing a throughout and approachable explanation.</i>
<br /><a name="myfootnote2" style="font-weight:bold">2</a>: <i>A good description of ASTs can be found on <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Wikipedia</a> while <a href="https://medium.com/@slavapestov/the-secret-life-of-types-in-swift-ff83c3c000a5#.jyxx86n2x">this article from Slava Pestov</a> has more details on how this is implemented for the Swift compiler.</i>
<br /><a name="myfootnote3" style="font-weight:bold">3</a>: <i>For more information about SIL, check out the detailed <a href="https://github.com/apple/swift/blob/master/docs/SIL.rst">official SIL guide</a> and <a href="https://www.youtube.com/watch?v=Ntj8ab-5cvE">this video</a> from the 2015 LLVM Developers's Meeting. A quick reference for SIL instructions written by Lex Chou is available <a href="https://github.com/lexchou/swallow/tree/master/docs/en/sil-3-instruction-references">here</a>.</i>
<br /><a name="myfootnote4" style="font-weight:bold">4</a>: <i>To learn more about how name mangling is performed in Swift, read <a href="https://github.com/lexchou/swallow/tree/master/docs/en/sil-1-mangling">this reference</a> from Lex Chou.</i>
<br /><a name="myfootnote5" style="font-weight:bold">5</a>: <i>Mike Ash talked about weak references with an experimental approach in one of his <a href="https://www.mikeash.com/pyblog/friday-qa-2015-12-11-swift-weak-references.html">Friday Q&A</a>, it's not completely up-to-date with the current way things are named and implemented in Swift but the explanations are still valid.</i>
</div>
Recursive Tail Calls and Trampolines in Swift2016-05-05T00:00:00+02:00https://www.uraimo.com/2016/05/05/recursive-tail-calls-and-trampolines-in-swiftThe use of recursion can improve the overall design of your algorithms but it leaves you susceptible to stack overflows if the compiler is unable to perform specific optimizations. In this post we'll see what tail recursion and TCO are and how the use of trampolines can overcome the fact that we cannot rely on the Swift compiler performing tail call elimination.<p><strong>Update 10/17:</strong><em>This post has been verified with Swift 4, no changes required.</em></p>
<p><strong>Update 10/16:</strong><em>This post has been updated to Swift 3.</em></p>
<p><img src="/imgs/trampoline.png" srcset="/imgs/trampoline@2x.png 2x" alt="Recursion vs Trampoline" /></p>
<p>The use of recursion can often lead to a cleaner implementation of your algorithms but when compared with implementations <a href="http://c2.com/cgi/wiki?RecursionVsLoop">based on loops</a>, recursive ones incur in the additional cost of allocating and managing a <a href="http://www.cs.uwm.edu/classes/cs315/Bacon/Lecture/HTML/ch10s07.html">new stack frame</a> for every method call performed, something that makes the recursive implementation slower and that can also quickly lead to stack exhaustion (aka stack overflow).</p>
<p>To avoid the risk of overflowing the stack, the recommended approach suggests to rewrite your algorithm employing <em>tail recursion</em> to leverage the <em>tail call optimization</em> that some compilers provide.</p>
<p>But what’s the difference between recursion and tail recursion and what are those compilers actually doing to solve the issues above?</p>
<p>Tail recursion differs from generic recursion for the fact that the result of the recursive call is always returned without any additional manipulation to the caller and calculation are performed through an accumulator variable that passes partial results to the successive recursive calls along the chain, until the end of the recursion is reached.</p>
<p>If this definition seems confusing, an example in the next section will make it more clear, for now you just need to know that this specific kind of recursion can be optimized and these recursive algorithms can be easily identified and converted by a compiler to a more efficient loop-based implementation, not affected by stack size limitations.</p>
<p>But in Swift, <em>we can’t rely on</em> the fact that the compiler will always <a href="https://twitter.com/jl_hfl/status/551168151497748480">perform the tail call elimination</a> described above in every circumstance.</p>
<p>This limitation has already been discussed on <a href="https://www.natashatherobot.com/functional-swift-tail-recursion/">Natasha’s blog</a> before and some work has been done on the draft of a <a href="https://github.com/apple/swift-evolution/pull/103/files">proposal</a> that aimed at adding a few new attributes that would have made the behavior of the optimizer more verifiable, allowing to explicitly state which recursive method call we expected to be optimized (and if that didn’t happen an error would have been thrown).</p>
<p>In this post we’ll see how the lack of predictable tail call elimination in Swift could be overcome using trampolines and a few alternatives to recursion will be described.</p>
<h4 id="get-the-playground-for-this-post-from-github-or-zipped"><em>Get the playground for this post from <a href="https://github.com/uraimo/Swift-Playgrounds/">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-05-05-recursive-trampoline.zip">zipped</a>.</em></h4>
<h2 id="triangular-numbers-with-recursion">Triangular numbers with recursion</h2>
<p>Let’s see an example of an algorithm that calculates the n-th <a href="https://en.wikipedia.org/wiki/Triangular_number">triangular number</a> recursively:</p>
<pre><code class="Swift">
func tri(n:Int)->Int{
if n <= 0 {
return 0
}
return n+tri(n: n-1)
}
tri(n: 300) //45150
</code></pre>
<p>In this example of simple recursion, the result of the recursive call is added to the value passed as parameter and the result of our initial method call <code class="inlinecode">tri(300)</code> will be the sum of all those integer values, chained together with recursion.</p>
<p>To improve this algorithm adding tail recursion, we’ll add an accumulator that will be passed along to the next call:</p>
<pre><code class="Swift">
func ttri(n:Int, acc:Int=0)->Int {
if n<1 {
return acc
}
return ttri(n: n-1,acc:acc+n)
}
ttri(n: 300) //45150
</code></pre>
<p>Notice how the result of this algorithm is now built with the accumulator and the last step of the recursion just returns the accumulator value to complete the calculation.</p>
<p>But both functions will crash your application or playground when invoked with a big enough integer as parameter. Let’s see how trampolines can solve this issue.</p>
<h2 id="trampolines">Trampolines</h2>
<p>The idea behind trampolines is actually quite simple.</p>
<p>A trampoline is not much more than a loop that executes iteratively functions that can either return the next function in the sequence (in form of a <a href="https://en.wikipedia.org/wiki/Thunk">thunk</a> or continuation, a data structure that contains the information needed to perform a specific function call) or another kind of value (in this case the content of the accumulator) to signal the end of the iteration.</p>
<p>The tail recursive function we wrote will have to be slightly modified if we want to execute it sequentially through a trampoline, we’ll need to rewrite it in <a href="https://en.wikipedia.org/wiki/Continuation-passing_style">continuation-passing style</a>.</p>
<blockquote>
<p><strong>Update</strong></p>
<p><span style="font-style: normal;">As <a href="https://twitter.com/oisdk">@oisdk</a> pointed out, the modified function we’ll see below will only slightly resemble actual CPS:</span></p>
<p>Here, closures are letting you simulate lazy evaluation to do pseudo tail-call optimisation. In Continuation-Passing style, you pass in a continuation as an extra parameter to your recursive function. The continuation represents what happens after the function it’s being passed into is called. Then, to evaluate the continuation, you (usually) pass in the identity function. This lets you transform non tail-recursive functions into tail-recursive ones. Obviously, since TCO isn’t guaranteed in Swift, this isn’t very useful.</p>
<p>Regardless, here’s what the triangular numbers example would look like in CPS:</p>
<pre><code class="Swift">
func triCont(n: Int, cont:@escaping Int -> Int) -> Int {
return n <= 1 ? cont(1) : triCont(n: n-1) { r in cont(r+n) }
}
func id<A>(x: A) -> A { return x }
triCont(n: 10, cont: id) // 55
</code></pre>
<p><span style="font-style: normal;">Thanks for the great explanation.</span></p>
</blockquote>
<p>Instead of performing directly a recursive call, our <code class="inlinecode">ttri</code> function will now return an object <em>that will wrap the actual call we performed previously</em> and once the point where the execution should complete will be reached, we’ll return a sentinel value with the content on the accumulator.</p>
<p>We start defining a <code class="inlinecode">Result</code> enum that represents the range of values that the modified recursive function will be able to return: a <code class="inlinecode">.Done</code> value that will signal the end of the recursion and that will contain the accumulator, and a <code class="inlinecode">.Call</code> value that will contain a closure with the next function call to perform.</p>
<pre><code class="Swift">
enum Result<A>{
case Done(A)
case Call(()->Result<A>)
}
</code></pre>
<p>And after that we define a new function, containing the modified <code class="inlinecode">ttri</code> tail recursive function and a section of code implementing the trampoline. This last part is usually contained in a separate function, but in this example everything will be kept together to make the code more readable.</p>
<pre><code class="Swift">
func tritr(n:Int)->Int {
func ttri(n:Int, acc:Int=0)->Result<Int> {
if n<1 {
return .Done(acc)
}
return .Call({
()->Result<Int> in
return ttri(n: n-1,acc: acc+n)
})
}
// Trampoline section
let acc = 0
var res = ttri(n: n,acc:acc)
while true {
switch res {
case let .Done(accu):
return accu
case let .Call(f):
res = f()
}
}
}
tritr(300)
</code></pre>
<p>Once you wrap your head around it, understanding what is happening in the trampoline section it’s not too hard.</p>
<p>After an initial call to the <code class="inlinecode">ttri</code> method to bootstrap the trampoline, the functions contained in the <code class="inlinecode">.Call</code> enums are executed in sequence while the value of the accumulator is being updated at each step:</p>
<pre><code class="Swift">
return .Call({
()->Result<Int> in
return ttri(n: n-1,acc: acc+n)
})
</code></pre>
<p>Even if the code is different, the behavior it’s still the same of our original recursive call.</p>
<p>Once we are done, the <code class="inlinecode">ttri</code> function returns a <code class="inlinecode">.Done</code> enum with the final result to be returned.</p>
<p>And even if this implementation is slower than the original one for all the code needed to operate the trampoline that was added, this version solves our biggest issue related to stack exhaustion, we’ll now be able to calculate every triangular number our heart desires… until we hit the size limit for integers.</p>
<p><strong>Update:</strong> Following a suggestion from <a href="https://twitter.com/oisdk">@oisdk</a>, the design of the <code class="inlinecode">ttri</code> function could be improved using the often forgotten <code class="inlinecode">@autoclosure</code> attribute:</p>
<pre><code class="Swift">
func call<A>(c:@escaping () -> Result<A>) -> Result<A> {
return .Call(c)
}
func ttri(n: Int, acc:Int=1) -> Result<Int> {
return n <= 1 ? .Done(acc) : call(c: tri(n: n-1, acc: acc+n))
}
</code></pre>
<p>Before we move on, let me add one more thing about that example, enclosing a block of code within a <code class="inlinecode">while true</code> it’s usually a <em>code smell</em> but this time I did it to make the example more compact, a more proper loop check would have looked something like this:</p>
<pre><code class="Swift">
while case .Call(_) = res {
switch res {
case let .Done(accu):
return accu
case let .Call(f):
res = f()
}
}
if case let .Done(ac) = res {
return ac
}
return -1
</code></pre>
<p>And for an even more proper check, since we are using enums with associated values, we should have implemented the comparison operator for this specific enum and check for <em>not done</em> on top of the loop.</p>
<p>Now that the basics of how trampolines work are explained, we can build a generic function that given a function converted to use the <code class="inlinecode">Result</code> enum, returns a closure that will execute the original function with a trampoline, hiding what’s happening under the hood:</p>
<pre><code class="Swift">
func withTrampoline<V,A>(f:@escaping (V,A)->Result<A>) -> ((V,A)->A){
return { (value:V,accumulator:A)->A in
var res = f(value,accumulator)
while true {
switch res {
case let .Done(accu):
return accu
case let .Call(f):
res = f()
}
}
}
}
</code></pre>
<p>The body of the closure we return is essentially what we had in the trampoline section in the previous example and <code class="inlinecode">withTrampoline</code> expects as parameter a function with the form <code class="inlinecode">(V,A)->Result<A></code>, that is what we had before.
The more obvious difference with the previous version is that since we can’t initialize the generic accumulator <code class="inlinecode">A</code> because we don’t know yet its concrete type, we’ll need to expose it as a parameter in the returned function, a minor annoyance.</p>
<p>Let’s see how to use our new utility function:</p>
<pre><code class="Swift">
var fin: (_ n:Int, _ a:Int) -> Result<Int> = {_,_ in .Done(0)}
fin = { (n:Int, a:Int) -> Result<Int> in
if n<1 {
return .Done(a)
}
return .Call({
()->Result<Int> in
return fin(n-1,a+n)
})
}
let f = withTrampoline(fin)
f(30,0)
</code></pre>
<p>This is probably a bit more verbose than what you expected.</p>
<p>Since we need a reference to the current function inside the closure to use it in the thunk, we must declare a dummy reference before declaring the actual closure and use that valid reference in our function.</p>
<p>Declaring the <code class="inlinecode">fin</code> closure directly without the dummy and attempting to use it would get us a <em>Variable used within its own initial value</em> error. If you feel adventurous, an alternative to this ugly workaround consists in using a <a href="https://stackoverflow.com/questions/24717460/cant-make-weak-reference-to-closure-in-swift">Z Combinator</a>.</p>
<p>But if moving away from the traditional trampoline design is not an issue, we can improve what we have above simplifying the <code class="inlinecode">Result</code> enum and keeping track of the function to call inside the trampoline instead of saving it as an associated value of the enum:</p>
<pre><code class="Swift">
enum Result2<V,A>{
case Done(A)
case Call(V, A)
}
func withTrampoline2<V,A>(f:@escaping (V,A)->Result2<V,A>) -> ((V,A)->A){
return { (value:V,accumulator:A)->A in
var res = f(value,accumulator)
while true {
switch res {
case let .Done(accu):
return accu
case let .Call(num, accu):
res = f(num,accu)
}
}
}
}
let f2 = withTrampoline2 { (n:Int, a:Int) -> Result2<Int, Int> in
if n<1 {
return .Done(a)
}
return .Call(n-1,a+n)
}
f2(30,0)
</code></pre>
<p>Way cleaner and compact.</p>
<h4 id="get-the-playground-for-this-post-from-github-or-zipped-1"><em>Get the playground for this post from <a href="https://github.com/uraimo/Swift-Playgrounds/">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-05-05-recursive-trampoline.zip">zipped</a>.</em></h4>
<h2 id="swifty-alternatives-to-recursion">Swifty Alternatives to recursion</h2>
<p>As you already know if you’ve read some of the posts in the <a href="/category/functional">Swift and the functional approach series</a>, Swift provides a few features that can be helpful to build alternative implementations for algorithms that are usually implemented recursively.</p>
<p>For example, triangular numbers could have been calculated with just a <a href="/2016/01/06/10-Swift-One-Liners-To-Impress-Your-Friends/">simple functional one liner</a> using reduce:</p>
<pre><code class="Swift">
(1...30).reduce(0,+) //465
</code></pre>
<p>Or we could have created a <a href="/2015/11/12/experimenting-with-swift-2-sequencetype-generatortype/">Sequence or an Iterator</a> to generate a sequence with all the possible triangular numbers:</p>
<pre><code class="Swift">
class TriangularSequence :Sequence {
func makeIterator() -> AnyIterator<Int> {
var i = 0
var acc = 0
return AnyIterator(body:{
print("# Returning "+String(i))
i=i+1
acc = acc + i
return acc
})
}
}
var fs = TriangularSequence().makeIterator()
for i in 1...30 {
print(fs.next())
}
</code></pre>
<p>And these are only two alternative implementations we could have built using what Swift provides.</p>
<h2 id="closing-thoughts">Closing Thoughts</h2>
<p>This post describes the limitations that Swift has in regard to recursion and shows how trampolines, the by-the-book workaround for languages lacking TCO, could be implemented in Swift. But, am i advocating the use of trampolines in your code?</p>
<p>Definitely not.</p>
<p>In Swift, considering that the language is not purely functional, what could be solved with a complex construct like trampolines can always be solved in a better way (producing code that is more readable and with a behavior easier to grasp) using one of the features the language provide, <em>don’t over-engineer your code</em>, your future self will be glad you didn’t.</p>
<p><em>Thanks to <a href="https://twitter.com/oisdk">@oisdk</a> for his insightful comments.</em></p>
Swift And C: Everything You Need to Know on Types, Pointers and more2016-04-07T00:00:00+02:00https://www.uraimo.com/2016/04/07/swift-and-c-everything-you-need-to-knowNow that Swift is available on Linux and other OSes, there are even more situations where you could need to interact with C code from your Swift 3 application. This article will hopefully shed some light on the most non-obvious details and give you some practical examples and tips of how to interact with C APIs like the C standard library.<p><strong>Update 5/17:</strong><em>A few minor improvements here and there related to other additions in Swift 3</em></p>
<p><strong>Update 3/17:</strong><em>This post has been updated to Swift 3 and extended with way more details and examples on UnsafePointers.</em></p>
<p><br /></p>
<p>Only a few months have passed since the open-source release of Swift but the language has already <a href="http://uraimo.com/category/swiftporting/">been ported</a> to numerous new platforms and new projects to port Swift somewhere else pop up every month.</p>
<p>The availability on different platforms turned mixing Swift and C from something that appeared to be an esoteric practice with a very limited practical utility other than wrapping native libraries to something you could have to deal with daily depending on where your code is running.</p>
<p>While some of the basics of C interoperability are well explained in the official guide <a href="https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/BuildingCocoaApps/InteractingWithCAPIs.html#//apple_ref/doc/uid/TP40014216-CH8-ID17">Using Swift with Cocoa and Objective-C</a>, more than a few things, especially related to the actual usage of bridged functions in real world scenarios, still remain mysterious and documented and explained properly only in an handful of blog posts.</p>
<p>This article will hopefully shed some light on the most non-obvious details and give you some practical examples of how to interact with C APIs, and while this post have been written mainly for people that plan to start developing in Swift on Linux, everything explained here also applies to Darwin-based OSes.</p>
<p>After a brief description of how C types are imported in Swift, we’ll delve into the specifics of pointers, strings and functions, and conclude with a short tutorial about creating mixed Swift/C projects using LLVM modules.</p>
<h4 id="get-the-mixed-swiftc-playground-for-this-post-from-github-or-zipped"><em>Get the mixed Swift/C playground for this post from <a href="https://github.com/uraimo/Swift-Playgrounds/tree/swift2">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-04-07-Swift-And-C.zip">zipped</a>.</em></h4>
<h3 id="contents">Contents</h3>
<ul>
<li><a href="#c-types">C Types</a>
<ul>
<li><a href="#arrays-and-structs">Arrays and Structs</a></li>
<li><a href="#enums">Enums</a></li>
<li><a href="#unions">Unions</a></li>
<li><a href="#the-size-of-things">The size of things</a></li>
<li><a href="#null-nil-0">Null, nil, 0</a></li>
</ul>
</li>
<li><a href="#macros">Macros</a></li>
<li><a href="#working-with-pointers">Working with Pointers</a>
<ul>
<li><a href="#allocating-memory">Allocating memory</a></li>
<li><a href="#pointer-conversion">Pointer Conversion</a></li>
<li><a href="#pointer-arithmetic">Pointer arithmetic</a></li>
<li><a href="#working-with-addresses">Working with addresses</a></li>
</ul>
</li>
<li><a href="#working-with-strings">Working with Strings</a></li>
<li><a href="#working-with-functions">Working with Functions</a>
<ul>
<li><a href="#unmanaged">Unmanaged</a></li>
</ul>
</li>
<li><a href="#working-with-files">Working with Files</a></li>
<li><a href="#bitwise-operations">Bitwise operations</a></li>
<li><a href="#swift-and-c-mixed-projects">Swift and C: Mixed Projects</a>
<ul>
<li><a href="#swift-package-manager">Swift Package Manager</a></li>
</ul>
</li>
<li><a href="#swift-3-changes">Swift 3 Changes</a></li>
<li><a href="#closing-thoughts">Closing Thoughts</a></li>
</ul>
<h2 id="c-types">C Types</h2>
<p>For each one of the basic C types, Swift provides a corresponding equivalent type that can be used when interoperating with C functions from Swift:</p>
<table>
<thead>
<tr>
<th>C Type</th>
<th>Swift C Type</th>
<th>Typealias of</th>
</tr>
</thead>
<tbody>
<tr>
<td>bool</td>
<td>CBool</td>
<td>Bool</td>
</tr>
<tr>
<td>char,unsigned char</td>
<td>CChar, CUnsignedChar</td>
<td>Int8, UInt8</td>
</tr>
<tr>
<td>short, unsigned short</td>
<td>CShort, CUnsignedShort</td>
<td>Int16, UInt16</td>
</tr>
<tr>
<td>int, unsigned int</td>
<td>CInt, CUnsignedInt</td>
<td>Int32, UInt32</td>
</tr>
<tr>
<td>long, unsigned long</td>
<td>CLong, CUnsignedLong</td>
<td>Int, UInt</td>
</tr>
<tr>
<td>long long, unsigned long long</td>
<td>CLongLong, CUnsignedLongLong</td>
<td>Int64, UInt64</td>
</tr>
<tr>
<td>wchar_t, char16_t, char32_t</td>
<td>CWideChar, CChar16, CChar32</td>
<td>UnicodeScalar, UInt16, UnicodeScalar</td>
</tr>
<tr>
<td>float, double</td>
<td>CFloat, CDouble</td>
<td>Float, Double</td>
</tr>
</tbody>
</table>
<p>The table above, in addition to what is already described in the official documentation, shows the actual Swift type the C typealias refers to.</p>
<p>Even if when writing code that interacts with C APIs you should use the Swift C types when possible, you’ll notice that the result of the import from C performed by Swift will most of the times simply use the usual Swift fixed-size types you are already familiar with.</p>
<h3 id="arrays-and-structs">Arrays and Structs</h3>
<p>Let’s now talk about compound data structures like arrays and structs.</p>
<p>In an ideal world, you would expect that a global array like this one:</p>
<pre><code class="c">
//header.h
char name[] = "IAmAString";
</code></pre>
<p>would be translated as a Swift String or at least as an array of some kind of character type. Well… this is what happens instead, once we try to use that imported <em>name</em> array in Swift:</p>
<pre><code class="swift">
print(name) // (97, 115, 100, 100, 97, 115, 100, 0)
</code></pre>
<p>This alone is more than enough to recommend using pointer to sequences of objects instead of vanilla arrays every time you can on the C layer of mixed Swift/C applications, to avoid painful translations once you reach the Swift layer.</p>
<p>But wait, can that global string declared with an array be recovered only with a convoluted piece of code converting the tuple to something more useful? Actually no, we’ll see how to fix that tuple with a few lines of code when discussing pointers.</p>
<p>Luckily, the situation it’s not so dire when dealing with structs, that are converted as Swift structs as expected, and its members are treated in the same predictable way, each one is converted recursively to the related Swift type.</p>
<p>For example, this struct:</p>
<pre><code class="c">
typedef struct {
char name[5];
int value;
int anotherValue;
} MyStruct;
</code></pre>
<p>is converted to a <code class="inlinecode">MyStruct</code> Swift struct. This clean conversion simplifies struct initialization too that is also performed as usual:</p>
<pre><code class="swift">
let ms = MyStruct(name: (0, 0, 0, 0, 0), value: 1, anotherValue:2)
print(ms)
</code></pre>
<p>In one of the next sections we’ll see that this is not the only way to allocate and initialize an instance of a struct and especially if we just need a pointer to an empty object it could be easier to just allocate a new empty struct pointer instance manually.</p>
<h3 id="enums">Enums</h3>
<p>If you need to access a C enum from Swift, declaring it as you usually do in C:</p>
<pre><code class="c">
typedef enum ConnectionError{
ConnectionErrorCouldNotConnect = 0,
ConnectionErrorDisconnected = 1,
ConnectionErrorResetByPeer = 2
}
</code></pre>
<p>Will get you something completely different from what you expected, once imported in Swift that enum will be represented by a structure and some global variables:</p>
<pre><code class="Swift">
struct ConnectionError : RawRapresentable, Equatable{ }
var ConnectionErrorCouldNotConnect: ConnectionError {get}
var ConnectionErrorDisconnected: ConnectionError {get}
var ConnectionErrorResetByPeer: ConnectionError {get}
</code></pre>
<p>And it’s quite obvious that this way we’ll lose all the capabilities that native Swift enums provide. But getting what we want is just a matter of using a specific macro in C:</p>
<pre><code class="c">
typedef NS_ENUM(NSInteger,ConnectionError) {
ConnectionErrorCouldNotConnect,
ConnectionErrorDisconnected,
ConnectionErrorResetByPeer
}
</code></pre>
<p>Using the <code class="inlinecode">NS_ENUM</code> macro (more details on this macro that is equivalent to declaring a classic C enum, <a href="http://nshipster.com/ns_enum-ns_options/">here</a>), this is how Swift will import the enum:</p>
<pre><code class="Swift">
enum ConnectionError: Int {
case CouldNotConnect
case Disconnected
case ResetByPeer
}
</code></pre>
<p>Notice that the conversion also stripped away the prefix that the enum had, this is one of the conversion rules of Swift that you can also see in action when you use the standard iOS/OSX frameworks in Swift.</p>
<p>Additionally, Swift also provides a <code class="inlinecode">NS_OPTIONS</code> macro that can be used to declare option sets conforming to <code class="inlinecode">OptionSetType</code>. To learn more about this macro, check out the official <a href="https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/BuildingCocoaApps/InteractingWithCAPIs.html#//apple_ref/doc/uid/TP40014216-CH8-ID17">documentation</a>.</p>
<h3 id="unions">Unions</h3>
<p>Let’s talk about unions, an interesting C type that has no Swift counterpart.</p>
<p>Swift supports unions only partially, meaning that while unions will be imported, not every kind of field is supported and as a consequence, some of the fields that you have declared in C could not be available (at the moment there is no documentation about what is not supported).</p>
<p>Let’s see an example of actual usage of this scarcely documented C type:</p>
<pre><code class="c">
//header.h
union TestUnion {
int i;
float f;
unsigned char asChar[4];
} testUnion;
</code></pre>
<p>Here we’ve declared a <code class="inlinecode">TestUnion</code> type with a related <code class="inlinecode">testUnion</code> union variable, each field represents a different view on the same 4 byte chunk of memory, in C we would be able to access <code class="inlinecode">testUnion</code> either as an integer, a float or as a set of bytes.</p>
<p>Since there is nothing similar to unions in Swift, this type will be imported as a <em>struct</em>:</p>
<pre><code class="Swift">
strideof(TestUnion) // 4 bytes
testUnion.i = 33
testUnion.f // 4.624285e-44
testUnion.i // 33
testUnion.asChar // (33, 0, 0, 0)
testUnion.f = 1234567
testUnion.f // 1234567
testUnion.i // 1234613304
testUnion.asChar // (56, 180, 150, 73)
</code></pre>
<p>The first line verifies that this type, like we would expect from an union, is indeed only 4 bytes long and the next lines modify one of the fields to verify that the values contained in the others are updated too. But why when we set <code class="inlinecode">testUnion</code> with <code class="language-plaintext highlighter-rouge">33</code> using the integer field we get <code class="language-plaintext highlighter-rouge">4.624285e-44</code> when we read the field as a float?</p>
<p>This is related to how unions work. You can think of an union as a bag of bytes that can be set or read using the formatting rules of each one of the fields it’s composed of, what we did above was setting that 4 bytes memory area with the same bit content an <code class="inlinecode">Int32(33)</code> would have had, and then we read the 4 bytes memory area interpreting its bit pattern like a IEEE float.</p>
<p>But let’s verify this using the useful (but dangerous) <code class="inlinecode">unsafeBitCast</code> function:</p>
<pre><code class="Swift">
var fv:Float32 = unsafeBitCast(Int32(33), to: Float.self) // 4.624285e-44
</code></pre>
<p>Here we are doing exactly the kind of conversion that happens when we access the union as a float, we are taking the bits that compose an <code class="inlinecode">Int32</code> with value <code class="language-plaintext highlighter-rouge">33</code> and assigning to them a variable with <code class="inlinecode">Float</code> type without any conversion or safety checks.</p>
<p>Now that we’ve see how this behave, could we implement a similar struct manually in Swift?</p>
<p>Even without checking the source, we can guess that <code class="inlinecode">TestUnion</code> it’s just a simple struct backed by a memory block of 4 bytes (it’s not important in which form) and the properties we access are just computed properties that hide all the conversion operations with their set/get.</p>
<h3 id="the-size-of-things">The size of things</h3>
<p>In Swift you can obtain the data-only or memory size of a specific type (primitive or compound) using the <code class="inlinecode">MemoryLayout<T></code> generic struct and the properties and functions it provides.</p>
<p>If you are interested in the size of the data contained in a variable, ignoring any additional space added to guarantee memory alignment, you can use the <code class="inlinecode">size</code> property of the type or its static method <code class="inlinecode">size(ofValue:)</code>.</p>
<p>As you could have guessed, <code class="inlinecode">MemoryLayout</code> also provides an additional property and a function to correctly retrieve the size of variables or types that take into account the additional space needed for alignment and that you should favor instead of the previous ones most of the times: <code class="inlinecode">stride</code> and its static method <code class="inlinecode">stride(ofValue:)</code>.</p>
<p>Let’s see an example where you’ll notice the difference between the values returned by size and stride:</p>
<pre><code class="Swift">
print(MemoryLayout<CChar>.stride) // 1 byte
struct Struct1{
let anInt8:Int64
let anInt:Int16
let b:Bool
}
print(MemoryLayout<CStruct1>.size)) // 11 (8+2+1) byte
print(MemoryLayout<CStruct1>.stride) // 16 (8+4+4) byte
</code></pre>
<p>And while the amount of additional space added to abide to the alignment rules of the processor architecture can be obtained doing a difference between the value returned by <code class="inlinecode">stride</code> and <code class="inlinecode">size</code>, an additional property <code class="inlinecode">alignment</code> is also available.</p>
<h3 id="null-nil-0">Null, nil, 0</h3>
<p>Luckily, Swift does not have an additional constant to represent null values, you can just use the Swift <code class="inlinecode">nil</code>, regardless of the type of the specific variable or parameter.</p>
<p>As we’ll see when talking about pointers, <code class="inlinecode">nil</code> gets also automatically translated to a null <code class="inlinecode">Unsafe[Mutable]RawPointer?</code> when passed as a parameter.</p>
<p>If you need a typed null pointer instead, you can create one assigning the <code class="inlinecode">nil</code> value to an optional pointer:</p>
<pre><code class="Swift">
let p: UnsafeMutablePointer<UInt8>? = nil
</code></pre>
<p>Just don’t try to unwrap this optional since it contains a nil value, assigning nil to an implicitly unwrapped optional is also ok.</p>
<h2 id="macros">Macros</h2>
<p>Simple C defines are translated in Swift as global constants, something like this in C:</p>
<pre><code class="Swift">
#define MY_CONSTANT 42
</code></pre>
<p>Will be translated as:</p>
<pre><code class="Swift">
let MY_CONSTANT = 42
</code></pre>
<p>More complex macros and preprocessor directives will be <em>completely ignored by Swift</em> and will not be available.</p>
<p>Swift also provides a simple conditional compilation statement and some support functions that can be used to include certain sections of code only for specific OSes, architectures or versions of Swift.</p>
<pre><code class="Swift">
#if arch(arm) && os(Linux) && swift(>=2.2)
import Glibc
#elseif !arch(i386)
import Darwin
#else
import Darwin
#endif
puts("Hello!")
</code></pre>
<p>In this example, we’ll import the standard c library from a different source depending on the fact that we are compiling this application on an ARM Linux host or not.</p>
<p>The functions available to customize the compilation behavior are: <code class="inlinecode">os()</code> (with valid values: <em>OSX, iOS, watchOS, tvOS, Linux</em>), <code class="inlinecode">arch()</code> (with valid values: <em>x86_64, arm, arm64, i386</em>) and <code class="inlinecode">swift()</code> (that requires its parameter to be specified as <em>>=version number</em>). The results of these functions can be combined with some basic logical operators to build complex rules: <strong>&&</strong>, <strong>||</strong>, <strong>!</strong>.</p>
<p>And since this is the first time we see it, remember that as on OSX you would import <em>Darwin</em> (or one of the frameworks that depends on it) in your projects to gain access to the libc functions and on a platform like Linux you’ll need to import <em>Glibc</em>.</p>
<h2 id="working-with-pointers">Working with Pointers</h2>
<p>Pointers are automatically translated to different kinds of <code class="inlinecode">UnsafePointer<Pointee></code> objects depending on the characteristics of the value they point to:</p>
<table>
<thead>
<tr>
<th>C Pointer</th>
<th>Swift Type</th>
</tr>
</thead>
<tbody>
<tr>
<td>int *</td>
<td>UnsafeMutablePointer<Int32></td>
</tr>
<tr>
<td>const int *</td>
<td>UnsafePointer<Int32></td>
</tr>
<tr>
<td>NSDate**</td>
<td>AutoreleasingUnsafeMutablePointer<NSDate></td>
</tr>
<tr>
<td>struct UnknownType *</td>
<td>OpaquePointer</td>
</tr>
<tr>
<td>void *</td>
<td>UnsafeMutableRawPointer</td>
</tr>
<tr>
<td>const void *</td>
<td>UnsafeRawPointer</td>
</tr>
</tbody>
</table>
<p>While the general rule is that mutable Pointer instances point to mutable variables, for class objects as in the third example <code class="inlinecode">NSDate**</code>, pointers to object passed as pointers are translated as <code class="inlinecode">AutoreleasingUnsafeMutablePointer</code>.</p>
<p>Moreover, if the type we are pointing to is not completely defined or cannot be represented in Swift (structs or unions that can only partially be translated from C), the pointer will be translated as an <code class="inlinecode">OpaquePointer</code>, an untyped pointer, essentially just a struct that contains some bits. Values pointed to by an <code class="inlinecode">OpaquePointer</code> cannot be accessed directly, the Pointer variable will need to be converted first.</p>
<p>Conversions from the three mutable Unsafe[Mutable]Pointer types to <code class="inlinecode">UnsafePointer<Type></code> are performed automatically (for example when you are passing a mutable pointer to a function requiring an immutable pointer) while a compiler error is raised the other way around.</p>
<p>A pointer to an immutable value cannot be converted to a pointer to a mutable value <em>implicitly</em>, Swift tries to guarantee a minimum of safety even in this circumstance, and this is an example of one of the operations on pointers that as we’ll see need to be performed manually. To obtain a mutable unsafe pointer you must now use the initialize <code class="inlinecode">UnsafeMutablePointer(mutating:)</code></p>
<p>It’s important to note that since Swift 3.0 conversion between pointers with different element types using the init method is not possible anymore, casting will need to be done explicitly with specific methods as we’ll see in <a href="#pointers-conversion">one of the following sections</a>.</p>
<p>The release 3.0 of Swift most notably added the <code class="inlinecode">UnsafeRawPointer</code> type to deal with untyped pointers, pointers that will normally be represented in C as void pointers and that were represented in Swift 2.x as <code class="inlinecode">Unsafe[Mutable]Pointer<Void></code></p>
<p>UnsafeRawPointers have a few methods that as we’ll see in the following sections will make our life a lot more easier simplifying all operations related to pointer conversions.</p>
<p>All the pointer types, with the exclusion of raw pointers and opaque pointers, are type safe (the compiler will perform type checks on the pointer and its content) and guarantee address alignement with the <em>Pointee</em> type they point to (the pointer, even after operations that alter its address moving int forwards or backwards, will always point at the start of a Pointee value).</p>
<p>The “unsafe” prefix in all those type names instead, refers to how we access the content, because interacting with pointers directly allows to circumvent the safety measures that the language has put in place when we access objects and is therefore considered inherently unsafe.</p>
<p>But what about the lifetime of the pointed object? How are they handled, if they are, through ARC?</p>
<p>As we already know, Swift uses ARC to manage the lifetime of reference types (and some struct and enums are <em>tracked</em> too when they contain reference types) and to track ownership, does unsafe pointers behave in some peculiar way?</p>
<p>No, <code class="inlinecode">Unsafe[Mutable]Pointer<Type></code> structs will be tracked if they are pointing to a reference type (a class object) or if they contain some reference to be tracked, that’s something you should know and that will help us explain something apparently weird when we’ll talk about memory allocation.</p>
<p>But again, this is not true for raw pointers and opaque pointers, that are not managed by ARC. And because of this, in some circumstances you’ll need to use a specific utility class called <code class="inlinecode">Unmanaged</code> to control the lifetime of the pointed objects, we’ll discuss this later.</p>
<p>Now that we know how pointers are represented, there are two things left to describe: how pointers can be dereferenced to obtain or modify the value they point to and how we can obtain a pointer to a new or preexisting Swift variable.</p>
<p>Once you get a hold of a non-Void <code class="inlinecode">Unsafe[Mutable]Pointer<T></code> retrieving or modifying the pointed value is straightforward using the <code class="inlinecode">pointee</code> property:</p>
<pre><code class="Swift">
var anInt:Int = myIntPointer.pointee //UnsafeMutablePointer<Int> --> Int
myIntPointer.pointee = 42
myIntPointer[0] = 43
</code></pre>
<p>And you can also access a specific element in a sequence of pointers of the same type as you would do in C, using a convenient array subscript, where each increment of the index will move you to the next element of size <code class="inlinecode">MemoryLayout<T>.stride</code> of the sequence.</p>
<p>On the other hand, if you need to obtain an <code class="inlinecode">Unsafe[Mutable]Pointer</code> to a Swift variable to use it as parameter of a function call and <em>only in that situation</em>, this can be done easily using the same operator we use to pass <strong>inout</strong> parameters by address to functions:</p>
<pre><code class="Swift">
let i = 42
functionThatNeedsAPointer(&i)
</code></pre>
<p>Considering that the operator cannot be used to perform that conversion outside of the described function invocation context, if you need to get a hold of the pointer variable to perform further computation (e.g. a pointer type conversion), Swift provides the <code class="inlinecode">withUnsafePointer</code> and <code class="inlinecode">withUnsafeMutablePointer</code> utility functions:</p>
<pre><code class="Swift">
withUnsafePointer(&i, { (ptr: UnsafePointer<Int>) -> Void in
var vptr= UnsafeRawPointer(ptr)
functionThatNeedsAVoidPointer(vptr)
})
let r = withUnsafePointer(&i, { (ptr: UnsafePointer<Int>) -> Int in
var vptr= UnsafeRawPointer(ptr)
return functionThatNeedsAVoidPointerAndReturnsInt(vptr)
})
</code></pre>
<p>This function creates a pointer object for the given variable and passes it to a closure that can then use it and optionally return a value. The pointer is guaranteed to be valid for the duration of the closure and considering it’s meant to be used only in that context <em>it can’t be returned</em> to the outside scope.</p>
<p>This way, the ability to access the variable unsafely is limited to the well-defined scope of the closure. In the sample above we are converting the int pointer to a void pointer before passing it to a function, in the <em><a href="#pointers-conversion">“Pointers Conversion”</a></em> section we’ll see how to convert one pointer type to another.</p>
<p>And let’s talk again briefly about <code class="inlinecode">OpaquePointer</code>, there is nothing special about it and it can be easily converted to a specific typed pointer thanks to the initializers of the <code class="inlinecode">Unsafe[Mutable]Pointer</code> and then accessed using the <code class="inlinecode">pointee</code> property like any other Unsafe[Mutable]Pointer:</p>
<pre><code class="Swift">
// ptr is an untyped OpaquePointer
var iptr = UnsafePointer<Int>(ptr)
print(iptr.pointee)
</code></pre>
<p>Swift also provides unsafe buffers, that are used to manipulate areas of memory that contain a sequence of values of the same type.</p>
<p>We can obtain a pointer to a classic Swift array with a slightly different syntax that converts it to an <code class="inlinecode">Unsafe[Mutable]BufferPointer</code> using one of its methods <code class="inlinecode">withUnsafe[Mutable]BufferPointer</code>:</p>
<pre><code class="Swift">
let array: [Int8] = [ 65, 66, 67, 0 ]
puts(array) // ABC
array.withUnsafeBufferPointer { (ptr: UnsafeBufferPointer<Int8>) in
puts(ptr.baseAddress! + 1) //BC
}
</code></pre>
<p>Note that <code class="inlinecode">Unsafe[Mutable]BufferPointer</code> also exposes a <code class="inlinecode">baseAddress</code> property that contains the base address of the buffer.</p>
<p>Following the choice made with the <code class="inlinecode">withUnsafePointer(to:body:)</code> method, a global static method with the same name is also provided for unsafe buffer.</p>
<p>The proposal <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md">SE-0138</a> implemented in Swift 3.0.1, adds a new global and a new array type method, both called <code class="inlinecode">withUnsafeBytes</code> that allows to manipulate the area of memory pointed by a typed pointer via a <em>raw buffer pointer</em> to a series of bytes (UInt8).</p>
<p>Something that can come in handy when you want inspect or alter the underlying raw representation of a high level data structure.</p>
<p>Let’s try this with Swift’s strings, that are built using a <code class="inlinecode"> _StringCore </code> struct internally, a struct contains three components: a pointer to an area of memory with the actual UTF-8 characters, a bit mask with the length and other flags, and an owner reference.</p>
<pre><code class="Swift">
public struct _StringCore {
// Internals
public var _baseAddress: UnsafeMutableRawPointer?
var _countAndFlags: UInt
public var _owner: AnyObject?
...
</code></pre>
<p>We’ll use <code class="inlinecode">withUnsafeBytes</code> to print each byte that makes up the structure and then we’ll build a pointer with the first 8 bytes of the struct (obtaining the <code class="inlinecode">_baseAddress</code> pointer, stored as little endian, with the Least Significant Byte placed in the first position of the byte sequence) and finally we’ll create new string using the character buffer pointed by <code class="inlinecode">_baseAddress</code> as input.</p>
<pre><code class="Swift">
import Foundation
var str = "iAmAStringHello"
withUnsafeBytes(of:&str){ ptr in
ptr.forEach{
//Print the content of the underlying _StringCore structure
print(String(format:"0x%x",$0))
}
let strptr = UnsafePointer<UInt8>(bitPattern:
UInt(ptr[0]) | // e.g. 0x26
(UInt(ptr[1]) << 8) | // 0x58
(UInt(ptr[2]) << 16) | // 0xef
(UInt(ptr[3]) << 24) | // 0x0e
(UInt(ptr[4]) << 32) | // 0x01
(UInt(ptr[5]) << 40) | // 0x00
(UInt(ptr[6]) << 48) | // 0x00
(UInt(ptr[7]) << 56) ) // 0x00
print(String(format:"0x%x",UInt(bitPattern: strptr)))
//Print the c string stored at address 0x000000010eef5826
print(String(cString:strptr!)) // Prints: iAmAStringHello
}
</code></pre>
<p>As we’ll see in a while, we’ll not need to reconstruct the addresses we stored in memory every time, that piece of code can be greatly simplified as follow, changing the way we access that block of memory and with a few pointer castings:</p>
<pre><code class="Swift">
let tmpp = UnsafeMutableRawPointer(ptr.baseAddress!).assumingMemoryBound(to: Int.self)
let strptr = UnsafeMutablePointer<UInt8>(bitPattern: tmpp.pointee)
</code></pre>
<p>I’m not going to explain this yet, after reading the next sections you’ll understand what I did here.</p>
<p>There is another kind of pointer we haven’t discussed yet: function pointers.</p>
<p>Since Swift 2.0, C function pointers are imported in Swift as closures with a special attribute <code class="inlinecode">@convention(c)</code> that specifies that the closure conforms to C calling conventions, we’ll see what this means in one of the <a href="#working-with-functions">following sections</a>.</p>
<p>Ignoring these details for a moment, the essential thing to know about function pointers is that every time an imported C function expects a function pointer parameter we will be able to use a closure declared in-place or a Swift function reference as parameter.</p>
<h3 id="allocating-memory">Allocating memory</h3>
<p>Until now we have only obtained pointers to existing Swift objects but we have never allocated memory manually, in this section we’ll see how this can be done in Swift using the recommended approach or with the functions of the <em>malloc</em> family like we would have done in C (that could make sense in a few very specific circumstances).</p>
<p>But before we start, we should be aware that unsafe pointers like good old C pointers can have three possible states during their lifetime:</p>
<ul>
<li><em>Unallocated</em>: No memory has been reserved for the pointer.</li>
<li><em>Allocated</em>: The pointer points to a valid allocated memory location but its value is uninitialized.</li>
<li><em>Initialized</em>: The pointer points to an allocated and initialized memory location.</li>
</ul>
<p>Pointers will move between these states in response to the operations we’ll perform on them.</p>
<p>The recommended approach for dealing with pointers, the one you should choose most of the times, consists in using the methods that the UnsafeMutablePointer class provides, to allocate a new object and then retrieve a pointer to the instance, initialize it and once we’re done with it, clean up its content and deallocate the memory it refers to.</p>
<p>Let’s see a basic example:</p>
<pre><code class="Swift">
var ptr = UnsafeMutablePointer<CChar>.allocate(capacity: 10)
ptr.initialize(from: [CChar](repeating: 0, count: 10))
//Do something with the object
ptr[3] = 42
ptr.deinitialize() //Clean up
ptr.deallocate(capacity: 10) //Let's free the memory we allocated
</code></pre>
<p>Here we have allocated a block of memory of 10 CChars (UInt8) using <code class="inlinecode">allocate(capacity:)</code> (that for raw pointers has an additional parameter to specify the alignment), that’s basically equivalent to calling <code class="inlinecode">malloc</code> specifying the total size of the memory chunk and then casting the result to the specific pointer type we need, but way less error prone since we don’t have to specify manually the size.
c
Once the space for the <code class="inlinecode">UnsafeMutablePointer</code> has been allocated, we must initialize the mutable object with one of the <code class="inlinecode">initialize</code> methods, for example <code class="inlinecode">initialize(to:Pointee, count:Int)</code> to initialize with <em>count</em> number of <em>Pointee</em> values or the <code class="inlinecode">initialize(from: Collection)</code> method specifying the initial content with another sequence. When we are done with the object and we want to free the resources allocated, we first clean up its content with <code class="inlinecode">deinitialize</code> and then proceed to deallocate the pointer with <code class="inlinecode">deallocate(capacity:)</code>.</p>
<p>It’s important to note that the Swift runtime will not do this last two things for you, as was your responsibility to allocate the memory needed by the variable it’s also your responsibility to deallocate the object once you are done working on that memory area.</p>
<p>Let’s see another example with a pointer to a more complex Swift value type this time:</p>
<pre><code class="Swift">
var ptr = UnsafeMutablePointer<String>.allocate(capacity: 1)
sptr.initialize(to:"Test String")
print(sptr[0])
print(sptr.pointee)
ptr.deinitialize()
ptr.deallocate(capacity: 1)
</code></pre>
<p>The sequence of operations comprising the allocation&initialization and cleanup&deallocation phase are the same for value types and reference types, but if you play around with it you’ll notice that for some value types (like integers,floats or some simple structs) the initialization it’s not actually necessary and you can just initialize the content with the <code class="inlinecode">pointee</code> property or via array subscript.</p>
<p>But this will definitely not work when your pointer points to a class or to some specific structs or enums. Sometimes initialization is required, but why?</p>
<p>The reason behind this behavior is related to what happens, from the memory management point of view, when you modify the content in one of the way described above, let’s see a snippet that doesn’t require manual initialization and then one that crashes miserably if we don’t initialize the UnsafeMutablePointer before altering its content.</p>
<pre><code class="Swift">
struct MyStruct1{
var int1:Int
var int2:Int
}
var s1ptr = UnsafeMutablePointer<MyStruct1>.allocate(capacity: 5)
s1ptr[0] = MyStruct1(int1: 1, int2: 2)
s1ptr[1] = MyStruct1(int1: 1, int2: 2) //This always works!
s1ptr.destroy()
s1ptr.dealloc(5)
</code></pre>
<p>No problem here, this just works, but let’s see this other example:</p>
<pre><code class="Swift">
class TestClass{
var aField:Int = 0
}
struct MyStruct2{
var int1:Int
var int2:Int
var tc:TestClass // we have introduced a field with a reference type
}
var s2ptr = UnsafeMutablePointer<MyStruct2>.alloc(5)
s2ptr.initialize(from: [MyStruct2(int1: 1, int2: 2, tc: TestClass()), // Remove the initialization
MyStruct2(int1: 1, int2: 2, tc: TestClass())]) // and you'll have a crash below
s2ptr[0] = MyStruct2(int1: 1, int2: 2, tc: TestClass())
s2ptr[1] = MyStruct2(int1: 1, int2: 2, tc: TestClass())
s2ptr.deinitialize()
s2ptr.deallocate(capacity: 5)
</code></pre>
<p>What happens here, is related to what was described at the beginning of the <a href="#working-with-pointers">Working with Pointers</a> section, <code class="inlinecode">MyStruct2</code> contains a reference and it’s lifetime is managed with ARC. When we modify one of the values in the memory block it points to, the Swift runtime will try to release the previous object contained in that slot, and if the memory contains garbage because it was never initialized, your application will crash.</p>
<p>Be aware of this, and to be on the safe side, favor initializing your UnsafeMutablePointers using <code class="inlinecode">initialize</code> once they have been allocated over just setting the memory directly.</p>
<p>The alternative approach we hinted at at the beginning of this section, consists in importing the standard C library (<em>Darwin</em> or <em>Glibc</em> on Linux) and using the familiar functions of the <em>malloc</em> family:</p>
<pre><code class="Swift">
var ptr = malloc(10*MemoryLayout<CChar>.stride).bindMemory(to: CChar.self, capacity: 10*MemoryLayout<CChar>.stride)
ptr[0] = 11
ptr[1] = 12
free(ptr)
</code></pre>
<p>As you can see we are not initializing the instance like we did when we followed the recommended approach, because as noted in the last paragraph with a type like CChar or some basic structs this will work.</p>
<p>Let’s now see two additional examples of usage of two common functions: <code class="inlinecode">memcpy</code> and <code class="inlinecode">mmap</code>:</p>
<pre><code class="Swift">
var val = [CChar](repeating: 0, count: 10)
var buf = [CChar](repeating: 0, count: val.count)
memcpy(&buf, &val, buf.count*MemoryLayout<CChar>.stride)
buf // [1,1,1,1,1,1,1,1,1,1]
let mptr = mmap(nil, Int(getpagesize()), PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0)!
if (Int(bitPattern: mptr) == -1) { //MAP_FAILED not available, but its value is (void*)-1
perror("mmap error")
abort()
}
// Bind the *uninitialized* memory to the Int type, for initialized memory we should have used .assumingMemoryBound(to:)
let iptr = mptr.bindMemory(to: Int.self, capacity: Int(getpagesize())/MemoryLayout<Int>.stride)
iptr[0] = 3
munmap(ptr, Int(getpagesize()))
</code></pre>
<p>This code is similar to what you would have done in C, note that you can easily retrieve the size of a memory page with <code class="inlinecode">getpagesize()</code>.</p>
<p>But while the first example just shows that we can use <code class="inlinecode">memcpy</code>, the second shows a real use case for the alternative approach to memory allocation, here we are mapping a new memory page but we could have mapped a specific memory area or a specific file pointer, and in that case we would have needed only direct access to the preexisting content, without any initialization.</p>
<p>Let’s see a real world example taken from <a href="https://github.com/uraimo/SwiftyGPIO">SwiftyGPIO</a>, here I’m <a href="https://github.com/uraimo/SwiftyGPIO/blob/master/Sources/SwiftyGPIO.swift#L294">mapping the area of memory</a> that contains the registers for the digital GPIO pins of the Raspberry Pi that will be used throughout the library to read or write values:</p>
<pre><code class="Swift">
// BCM2708_PERI_BASE = 0x20000000
// GPIO_BASE = BCM2708_PERI_BASE + 0x200000 /* GPIO controller */
// BLOCK_SIZE = 4*1024
private func initIO(id: Int){
let mem_fd = open("/dev/mem", O_RDWR | O_SYNC)
guard (mem_fd > 0) else {
print("Can't open /dev/mem")
abort()
}
let gpio_map = mmap(
nil, //Any adddress in our space will do
BLOCK_SIZE, //Map length
PROT_READ|PROT_WRITE,// Enable reading & writting to mapped memory
MAP_SHARED, //Shared with other processes
mem_fd, //File to map
off_t(GPIO_BASE) //Offset to GPIO peripheral
)!
close(mem_fd)
let gpioBasePointer = gpio_map.assumingMemoryBound(to: Int.self)
if (Int(bitPattern:gpioBasePointer) == -1) { //MAP_FAILED not available, but its value is (void*)-1
perror("GPIO mmap error")
abort()
}
gpioGetPointer = gpioBasePointer.advanced(by: 13)
gpioSetPointer = gpioBasePointer.advanced(by: 7)
gpioClearPointer = gpioBasePointer.advanced(by: 10)
inited = true
}
</code></pre>
<p>Once we’ve mapped the 4KB area starting at <code class="language-plaintext highlighter-rouge">0x20200000</code>, we retrieve the address of the 3 registers we are interested in and from there we just read or write their values through the <code class="inlinecode">memory</code> property.</p>
<h3 id="pointer-conversion">Pointer Conversion</h3>
<p>Conversions between Unsafe[Mutable]Pointers with a different type can be performed in a few ways.</p>
<p>If you need to bind the content pointed by a pointer to a different type <em>temporarily</em>, obtaining what we could consider a different <em>view</em> of the pointed data, you can use <code class="inlinecode">withMemoryRebound(to:capacity:body:)</code> that provides an <code class="inlinecode">UnsafaMutablePointer</code> in the context of a closure where you can modify the data, essentially circumventing momentarily the type safety rules of the original pointer.</p>
<pre><code class="Swift">
// With ptr being an UnsafeMutablePointer<UInt8>
let charPtr = ptr.withMemoryRebound(to: CChar.self, capacity: 11, {
(cptr) -> String in
return String(validatingUTF8: cptr)!
})
</code></pre>
<p>To perform a permanent type conversion, you’ll need raw pointers.</p>
<p>A raw pointer pointing to the same address can simply be created initializing a raw pointer with the typed pointer you want to convert, using either the method <code class="inlinecode">assumingMemoryBound(to:)</code> or <code class="inlinecode">bindMemory(to:capacity)</code> depending on the state of the memory you are binding to the new type. If the block of memory the pointer points to is uninitialized use <code class="inlinecode">bindMemory(to:capacity)</code>, using this method on already bound memory rebinds it to the new type. On already initialized memory that was already bound to the new type and is correctly aligned, use <code class="inlinecode">assumingMemoryBound(to:)</code> instead.</p>
<p>We have already seen a few examples with these methods in the previous section but now let’s isolate only what we really need to perform a simple pointer conversion:</p>
<pre><code class="Swift">
UnsafeRawPointer(typedPointer).bindMemory(to: UInt8.self capacity:1024)
</code></pre>
<p>In some very specific circumstances, and only after serious consideration, you could reach the conclusion that, for example for performance reasons, it could make sense to do without the safety features that Swift provides, and just work with raw pointers and maybe perform some <a href="https://en.wikipedia.org/wiki/Type_punning">type punning</a>.</p>
<p>As we have already seen, most of the functionalities you’ll need in this cases are offered by the <code class="inlinecode">Unsafe[Mutable]RawPointer</code> object. Let’s see a few other methods that are available to operate on chunks of memory without having to worry about type safety checks.</p>
<pre><code class="Swift">
// Get a value starting at a specific offset and cast it to a type
let value = ptr.load(fromByteOffset: 8, as: UInt8.self)
// Set a value at a specific offset and with the specific type
ptr.storeBytes(of: 0xFA, toByteOffset: 8, as: UInt8.self)
// Copy bytes from another pointer
ptr.copyBytes(from: anotherPtr, count: 10 * MemoryLayout<UInt8>.stride)
</code></pre>
<p>Now we are ready to go back to the char array we had at the beginning of this article, with all the information we have now and knowing that a <code class="inlinecode">CChar</code> tuple is automatically converted to a pointer of a sequence of <code class="inlinecode">CChar</code> we can convert the tuple to a String easily:</p>
<pre><code class="Swift">
let namestr = withUnsafePointer(to: &name, { (ptr) -> String? in
let charPtr = ptr.withMemoryRebound(to: CChar.self, capacity: 11, {
(cptr) -> String in
return String(validatingUTF8: cptr)!
})
return charPtr
})
print(namestr!) //IA#AString
</code></pre>
<h3 id="pointer-arithmetic">Pointer arithmetic</h3>
<p>In C is quite common to use pointers arithmetic to move through sequences or to get a reference to a specific member of compound variables, can this be done in Swift too?</p>
<p>Sure, <code class="inlinecode">UnsafePointer</code> and its mutable variants provide a few convenient methods that allow to perform the same operations that in C we’ll perform leveraging pointer arithmetic incrementing or decrementing pointers: <code class="inlinecode">successor()</code>, <code class="inlinecode">predecessor()</code>, <code class="inlinecode">advanced(by:)</code> and <code class="inlinecode">distance(to:UnsafePointer<T>)</code>.</p>
<pre><code class="Swift">
var aptr = UnsafeMutablePointer<CChar>.allocate(capacity: 5)
aptr.initialize(from: [33,34,35,36,37])
print(aptr.successor().pointee) // 34
print(aptr.advanced(by: 3).pointee) // 36
print(aptr.advanced(by: 3).predecessor().pointee) // 35
print(aptr.distance(to: aptr.advanced(by: 3))) // 3
aptr.deinitialize()
aptr.deallocate(capacity: 5)
</code></pre>
<p>Even if I’ve presented these methods first and even if those are the one I recommend you to use, it’s still possible to increment and decrement an unsafe pointer adding an integer to obtain a pointer to one of the other elements in a sequence:</p>
<pre><code class="Swift">
print((aptr+1).pointee) // 34
print((aptr+3).pointee) // 36
print(((aptr+3)-1).pointee) // 35
</code></pre>
<p>When you increment or decrement an Unsafe[Mutable]Pointer, the actual pointer is moved by multiples of <code class="inlinecode">MamoryLayout<Pointee>.alignement</code>. Raw pointers or opaque pointers are instead just incremented or decremented by the given number of positions.</p>
<p>The classic use case for pointer arithmetic is the traversal of components of structures or of structures that contain nested sub-structures.</p>
<p>Let’s see an example that uses some of the techniques explained until now with a data structure composed by an header, defined as a <code class="inlinecode">Header</code> struct with a few fields, and a body with a series of float values. We will first allocate an untyped pointer to the whole block of memory and then obtain two typed pointers to the two distinct logical segments:</p>
<pre><code class="Swift">
struct Header{
let field1: Int64 // 8 bytes
let field2: Int32 // 4 bytes
let field3: Int64 // 8 bytes
}
let numValues = 100 // 100 float values in the body
let ptr = UnsafeMutableRawPointer.allocate(
bytes: MemoryLayout<Header>.stride + numValues * MemoryLayout<Float>.stride,
alignedTo: MemoryLayout<Header>.alignment)
let header = ptr.bindMemory(to: Header.self, capacity: 1)
let data = (ptr + MemoryLayout<Header>.stride).bindMemory(to: Float.self, capacity: numValues)
</code></pre>
<p>With the code above we’ll end up with two <code class="inlinecode">UnsafeMutablePointer</code> instances, one pointing at the beginning of the memory area and one shifted of 20 bytes containing a series of 100 floats.</p>
<h3 id="working-with-addresses">Working with addresses</h3>
<p>Sometimes you could need to retrieve the actual pointer value to perform some calculation with it or to store it like an integer. This could be done before Swift 3.0 with an unsafe bit cast to an integer of compatible size:</p>
<pre><code class="Swift">
import Foundation
func address<T>(of: UnsafePointer<T>) -> UInt {
return unsafeBitCast(of, to:UInt.self)
}
var a = 1
print( String(format:"0x%016x",address(of:&a)) )
</code></pre>
<p>But since Swift 3.0 you can accomplish the same thing way more safely using an <code class="inlinecode">Int</code> or <code class="inlinecode">UInt</code> initializer with <em>every</em> pointer type:</p>
<pre><code class="Swift">
import Foundation
func address<T>(of: UnsafePointer<T>) -> UInt {
return UInt(bitPattern: of)
}
var a = 1
print( String(format:"0x%016x", address(of: &a)) )
</code></pre>
<p>What if you already have and integer representation of an address pointer and need to build an unsafe pointer from that?</p>
<p>Swift has a specific unsafe pointer initializer to do it without resorting again to the unsafe bit cast. You can either create a typed pointer or a raw pointer using a specific integer bit pattern as address.The address will need to be correctly aligned to the Pointee type, meaning that <code class="inlinecode">pattern % MemoryLayout<Pointee>.alignment</code> should be 0.</p>
<pre><code class="Swift">
let ptr = UnsafeMutableRawPointer(bitPattern: 0x12345678)!
</code></pre>
<p>But please not that trying to unwrap a pointer with bit pattern equal to zero (a null pointer) will result in a crash, like it would for any other optional Swift type with a nil value.</p>
<h4 id="get-the-mixed-swiftc-playground-for-this-post-from-github-or-zipped-1"><em>Get the mixed Swift/C playground for this post from <a href="https://github.com/uraimo/Swift-Playgrounds/">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-04-07-Swift-And-C.zip">zipped</a>.</em></h4>
<h2 id="working-with-strings">Working with Strings</h2>
<p>When a C functions has a char pointer parameter, as we now know, this parameter is imported in Swift as an <code class="inlinecode">Unsafe[Mutable]Pointer<Int8></code>, but since Swift automatically converts strings into pointer to UTF8 buffers, you’ll be able to invoke those functions simply using a string as parameter, without manually converting it first.</p>
<p>Alternatively, if you need to perform additional operations with that pointer before invoking the function expecting a char pointer, Swift strings also provide the <code class="inlinecode">withCString</code> method that passes the UTF8 char buffer to a closure that can optionally return a value.</p>
<pre><code class="Swift">
puts("Hey! I was a Swift string!") //Passing a swift string to a libc method
var testString = "AAAAA"
testString.withCString { (ptr: UnsafePointer<Int8>) -> Void in
// Do something with ptr
functionThatExpectsAConstCharPointer(ptr)
}
</code></pre>
<p>Turning a C string into a full fledged Swift string is straightforward, just use the <code class="inlinecode">init(cString:)</code> or <code class="inlinecode">init?(validatingUTF8:)</code> String initializers, but remember that the C string must be <strong>null terminated</strong>.</p>
<pre><code class="Swift">
let swiftString = String(validatingUTF8: aCString)!
</code></pre>
<p>If you are porting some C code that deals with strings to Swift, for example something handling with user input, you could have the need to compare the value of each character in a string against a single ASCII character value or an ASCII range, can this be done in Swift considering how strings are structures?</p>
<p>Yes, but i will not delve too much into the specifics of Swift strings, if you want to learn more about how strings in Swift are structured read <a href="http://oleb.net/blog/2014/07/swift-strings/">this article from Ole Begemann</a> and <a href="http://andybargh.com/unicode/">this article by Andy Bargh</a> to learn more about Unicode.</p>
<p>Let’s see an example of a function that verifies if a string is composed only by basic ASCII printable characters that looks like something that could have been ported from C:</p>
<pre><code class="Swift">
func isPrintable(text:String)->Bool{
for scalar in text.unicodeScalars {
let charCode = scalar.value
guard (charCode>31)&&(charCode<127) else {
return false // Unprintable character
}
}
return true
}
</code></pre>
<p>What in C was likely a comparison between a char integer value and an ASCII range for each character composing the string, in Swift doesn’t change much and just uses the value of each one of the unicode scalars for the comparison. Clearly, this approach makes sense only if those strings have graphemes composed by a single scalar, not much in the general case.</p>
<p>And what about simple conversions between characters and their numerical ASCII value?</p>
<p>To convert a numerical value to the corresponding <code class="inlinecode">Character</code> or <code class="inlinecode">String</code> we must first convert it to an <code class="inlinecode">UnicodeScalar</code>, while the most compact way to do the opposite uses the specific constructor that only the <code class="inlinecode">UInt8</code> type provide:</p>
<pre><code class="Swift">
let c = Character(UnicodeScalar(70)) // "F"
let s = String(UnicodeScalar(70)) // "F"
let asciiForF = UInt8(ascii:"F") // 70
</code></pre>
<p>The guard statement in the previous example could have been improved with <code class="inlinecode">UInt8(ascii:)</code> to increase readability.</p>
<h2 id="working-with-functions">Working with Functions</h2>
<p>As happened with strings, Swift is able to automatically convert closure into C function pointers when used as parameters, but with a major twist, closures that will be used as a C function pointer parameter <em>cannot capture any value outside of their context</em>.</p>
<p>To enforce this, this kind of closures (and closures that are the result of a conversion from a C function pointer) are annotated automatically with a specific type attribute <code class="inlinecode">@convention(c)</code> that, as described in the chapter on type attributes of <a href="https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/Attributes.html#//apple_ref/doc/uid/TP40014097-CH35-ID347">Swift Language Reference</a>, indicate the calling convention the closure will conform to. The possible value are : <em>c</em>, <em>objc</em> and <em>swift</em>.</p>
<p>A few alternatives to work around this limitation are described in <a href="http://chris.eidhof.nl/posts/swift-c-interop.html">this article by Chris Eidhof</a> and consist in using a block-based function, if you are on a Darwin-derived OS and are calling a function that also have a block variant, or passing a retained environment object to the function following a common C pattern.</p>
<p>Now let’s talk briefly about variadic functions.</p>
<p>Swift does not support traditional variadic C functions and this will be clear the first time you try to call a function like <code class="inlinecode">printf</code> form Swift, compile time error. If you really need to use one of those, the only viable alternative is building a wrapper function in C that limits the number of parameters or a wrapper that accepts multiple parameters indirectly using a <em>va_list</em> (that is supported by Swift).</p>
<p>So, even if <code class="inlinecode">printf</code> does not work, <code class="inlinecode">vprintf</code> or similar functions will.</p>
<p>To turn an array of parameters or a variadic Swift parameter list into a va_list pointer, each parameter must implement <code class="inlinecode">CVarArgType</code> and then you just need to call <code class="inlinecode">withVaList</code> to obtain a <code class="inlinecode">CVaListPointer</code> that points to your list of parameters (<code class="inlinecode">getVaList</code> is also available but the documentation recommends to avoid it). Let’s see a short example with <em>vprintf</em>:</p>
<pre><code class="Swift">
withVaList(["a", "b", "c"]) { ptr -> Void in
vprintf("Three strings: %s, %s, %s\n", ptr)
}
</code></pre>
<h3 id="unmanaged">Unmanaged</h3>
<p>We’ve learn more or less everything we have to know about pointers, but there is still something that we’ll not be able to handle with what we know now.</p>
<p>What if we pass to a C function that returns its result in a callback a Swift reference object as parameter? Can we be sure that during this switch of context the Swift object will still be there and that ARC will not have released it? No, we can’t make the assumption that the object will still be there.</p>
<p>Meet <code class="inlinecode">Unmanaged</code>, a class with some interesting utility methods that we’ll use to manage situations like the one described above. With <em>Unmanaged</em> you will be able alter the retain count of an object and convert it to a <code class="inlinecode">COpaquePointer</code> if you need to pass it around.</p>
<p>Let’s get right into it and solve the issue we described, here is an example of a C function similar to what was described before:</p>
<pre><code class="Swift">
// cstuff.c
void aCFunctionWithContext(void* ctx, void (*function)(void* ctx)){
sleep(3);
function(ctx);
}
</code></pre>
<p>And some Swift code that calls it:</p>
<pre><code class="Swift">
class AClass : CustomStringConvertible {
var aProperty:Int=0
var description: String {
return "A \(type(of: self)) with property \(self.aProperty)"
}
}
var value = AClass()
let unmanaged = Unmanaged.passRetained(value)
let uptr = unmanaged.toOpaque()
let vptr = UnsafeMutableRawPointer(uptr)
aCFunctionWithContext(vptr){ (p:UnsafeMutableRawPointer?) -> Void in
var c = Unmanaged<AClass>.fromOpaque(p!).takeUnretainedValue()
c.aProperty = 2
print(c) //A AClass with property 2
}
</code></pre>
<p>With the <code class="inlinecode">passRetained</code> and <code class="inlinecode">passUnretained</code> methods, <em>Unmanaged</em> retains a given object for us, respectively incrementing or not its reference count.</p>
<p>Since the callback needs a void pointer, we first obtain a <code class="inlinecode">COpaquePointer</code> with <code class="inlinecode">toOpaque()</code> and then convert it to <code class="inlinecode">UnsafeMutablePointer<Void></code>.</p>
<p>In the callback, we perform the same operations in reverse to obtain a reference to the original class and modify its value.</p>
<p>When extracting the class from the unmanaged object, we could have called either <code class="inlinecode">takeRetainedValue</code> or <code class="inlinecode">takeUnretainedValue</code>, that following the same pattern described before, respectively decrement or leave unmodified the reference count of the value.</p>
<p>In this case, we are not decrementing the reference count so that the class will not become releasable once we are out of the closure scope, this class will have to be released manually somewhere else through the <code class="inlinecode">unmanaged</code> instance we declared initially.</p>
<p>And this is just one simple, and maybe not the best, example of a category of problems that <code class="inlinecode">Unmanaged</code> could solve, to learn more about Unmanaged check out this <a href="http://nshipster.com/unmanaged/">NSHipster’s article</a>.</p>
<h2 id="working-with-files">Working with Files</h2>
<p>Since on some platform we could have to deal directly with files frequently using the standard C library functions, let’s see some examples of how to read and write from files:</p>
<pre><code class="Swift">
let fd = fopen("aFile.txt", "w")
fwrite("Hello Swift!", 12, 1, fd)
let res = fclose(file)
if res != 0 {
print(strerror(errno))
}
let fd = fopen("aFile.txt", "r")
var array = [Int8](repeating:0, count: 13)
fread(&array, 12, 1, fd)
fclose(fd)
let str = String(validatingUTF8: array)!
print(str) // Hello Swift!
</code></pre>
<p>As you can see there is nothing weird or convoluted about file access, this is essentially the same code we would have written in C. Notice that we have full access to <code class="inlinecode">errno</code> and all the related functions.</p>
<h2 id="bitwise-operations">Bitwise Operations</h2>
<p>Since it’s highly likely that you’ll need to perform bit mask operations when interoperating with C, <a href="/2016/02/05/Dealing-With-Bit-Sets-In-Swift/">i recommend a post I wrote a while ago</a> on the subject that should cover all you need to know.</p>
<h4 id="get-the-mixed-swiftc-playground-for-this-post-from-github-or-zipped-2"><em>Get the mixed Swift/C playground for this post from <a href="https://github.com/uraimo/Swift-Playgrounds/">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-04-07-Swift-And-C.zip">zipped</a>.</em></h4>
<h2 id="swift-and-c-mixed-projects">Swift and C: Mixed Projects</h2>
<p>Swift projects can access libraries written in C using a bridging header following the same procedure used for Objective-C libraries.</p>
<p>But since this does not work for framework projects, let’s see a more general alternative approach that has only slightly more of typing involved. We’ll create a <a href="http://clang.llvm.org/docs/Modules.html">LLVM module</a> that will contain the C code we want to export to Swift.</p>
<p>Let’s suppose we’ve added to our Swift project a C source file:</p>
<pre><code class="c">
// CExample.c
#include "CExample.h"
#include <stdio.h>
void printStuff(){
printf("Printing something!\n");
}
void giveMeUnsafeMutablePointer(int* param){ }
void giveMeUnsafePointer(const int * param){ }
</code></pre>
<p>And its header:</p>
<pre><code class="c">
// CExample.h
#ifndef CExample_h
#define CExample_h
#include <stdio.h>
#define IAMADEFINE 42
void printStuff();
void giveMeUnsafeMutablePointer(int* param);
void giveMeUnsafePointer(const int * param);
typedef struct {
char name[5];
int value;
} MyStruct;
char name[] = "IAmAString";
char* anotherName = "IAmAStringToo";
#endif /* CExample_h */
</code></pre>
<p>To keep the C sources separated from the rest we’ve put these files in the <em>CExample</em> sub-directory in the root of the project.</p>
<p>We must now create a <em>module.map</em> file in that same directory and this file will declare what our C module exports and from which header file.</p>
<pre><code class="c">
module CExample [system] {
header "CExample.h"
export *
}
</code></pre>
<p>As you can guess, we are exporting all the content declared in the header but modules can also selectively export only part of what’s declared.</p>
<p>Furthermore, in this example the actual source of the library is contained in the project, but if you need to expose to Swift one of the libraries you have installed on your system you can just create a <em>module.map</em> (not necessarily in its own directory) and specify as <code class="inlinecode">header</code> one or more of your system’s headers. In this case you’ll likely need to specify the name of the library your headers refer to too using the <code class="inlinecode">link libname</code> directive in your modulemap (that will link that library as you would manually do with <em>-llibname</em>). And you can also declare more than one module in a single <em>module.map</em>.</p>
<p>To learn more about LLVM modules and all the options available check out the <a href="http://clang.llvm.org/docs/Modules.html">official documentation</a>.</p>
<p>The last step consist in adding the module directory to the search path of the compiler. To do this, open the project properties and add the module path (<em>${SRCROOT}/CExample</em>) to <strong>Import Paths</strong> under <strong>Swift Compiler - Search Paths</strong>:</p>
<p><img src="/imgs/swiftandc_1.png" alt="Importing c module" /></p>
<p>And that’s it, we can now import from Swift the new module and use what it contains:</p>
<pre><code class="Swift">
import CExample
printStuff()
print(IAMADEFINE) //42
// Pass a bogus pointer at address 0x1
giveMeUnsafePointer(UnsafePointer<Int32>(bitPattern: 1))
giveMeUnsafeMutablePointer(UnsafeMutablePointer<Int32>(bitPattern: 1))
let ms = MyStruct(name: (0, 0, 0, 0, 0), value: 1)
print(ms)
print(name) // (97, 115, 100, 100, 97, 115, 100, 0)
//print(String(validatingUTF8:name)!) // Cannot convert it
print(anotherName) //0xXXXXXX pointer address
print(String(validatingUTF8:anotherName)!) //IAmAStringToo
</code></pre>
<h3 id="swift-package-manager">Swift Package Manager</h3>
<p>The Swift Package Manager supports the creation of C modules via module.map as <a href="https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#require-system-libraries">described in the documentation</a>. You can either create mixed projects or create a main project importing various sub-projects, each one providing a wrapper module (with likely some Swift to decorate the original C API) for each different C library you need to use.</p>
<h2 id="swift-3-changes">Swift 3 Changes</h2>
<p>The third major release of Swift introduces many changes related to pointers and other functionalities shown in this article, some of the pertaining Swift Evolution proposals are: <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0016-initializers-for-converting-unsafe-pointers-to-ints.md">SE-0016</a>, <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0055-optional-unsafe-pointers.md">SE-0055</a>, <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0076-copying-to-unsafe-mutable-pointer-with-unsafe-pointer-source.md">SE-0076</a>, <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0101-standardizing-sizeof-naming.md">SE-0101</a>, <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md">SE-0107</a>, <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0136-memory-layout-of-values.md">SE-0136</a> and finally <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md">SE-0138</a> that extends SE-0107 and has been implemented in Swift 3.0.1.</p>
<p>With <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md">SE-0107</a>, a new <code class="inlinecode">UnsafeRawPointer</code> type has been introduced to replace the <code class="inlinecode">UnsafePointer<Void></code> that was used in Swift 2.2 code for pointer without type information and strict aliasing for the pointers element type is now enforced. A comprehensive <a href="https://swift.org/migration-guide/se-0107-migrate.html">migration guide</a> is available to help the transition.</p>
<h2 id="closing-thoughts">Closing Thoughts</h2>
<p>I hope this article will at least shed some light on the mysterious and scantily documented world of Swift and C interoperability but I don’t really expect to have covered everything you could encounter in your projects.</p>
<p>You’ll find yourself in situations where a bit of experimenting will be needed to make things work as you want and it’s likely that C interoperability will be improved over the next releases of Swift (UnsafePointer and all the related functions were introduced just in Swift 2.0, before that, interoperability with C was a bit more convoluted) with new constructs.</p>
A Short Swift GYB Tutorial2016-02-09T00:00:00+01:00https://www.uraimo.com/2016/02/09/a-short-swift-gyb-tutorialThe GYB tool is used internally in Swift to simplify source files with many snippets of code that follow a common pattern. GYB provides some additional directives that are parsed by the tool to generate the final source files. This short tutorial describes how to use GYB in your own projects.<p><em>Update 9/2019:</em> Updated the post and added a detailed description of the line-directive tool.</p>
<p>The GYB (Generate Your Boilerplate) tool <a href="https://github.com/apple/swift/blob/master/utils/gyb.py">is a Swift tool</a> intended for internal use that generates source files starting from a template.</p>
<p>It’s extremely useful when you have more than one struct/class/enum that share a common structure, and you are not eager to maintain multiple versions of what is, actually, the same code. Every time you are writing the same set of methods or properties for slightly different objects, your maintenance effort (and bugs resulting from careless copy/paste) could be reduced using GYB. The tool is used extensively throughout the <a href="https://github.com/apple/swift/search?utf8=%E2%9C%93&q=filename%3A*.gyb&type=Code">Swift codebase</a> and there are just a few things you need to know to use it in your projects.</p>
<p>As a diligent guinea-pig (AFAIK, the only other project that uses GYB at the moment is <a href="https://github.com/SwiftAndroid/swift-jni">Swift-JNI</a>, part of the Android Swift porting project), I’ve used GYB extensively in the first release of <a href="https://github.com/uraimo/Bitter">Bitter</a>, a Swift library that simplifies working with bits and bitwise operations, where I had a lot of very similar code inside extensions for each one of the fixed size Swift Ints.</p>
<p>With this tool, I was able to define a single template that the tool was able to expand to the 10 separate extensions I initially coded by hand.</p>
<p>Let’s see what you need to know to start using GYB.</p>
<h2 id="gyb-engine-elements">GYB Engine Elements</h2>
<p>The GYB templating engine is quite simple but requires a minimal knowledge of Python. A template is composed by these elements:</p>
<ul>
<li>
<p>Lines starting with <strong>%</strong> (leading whitespace is ignored) are followed by Python code and are used for control flow statements, as you would expect those statement are closed with an %end. Statements can be nested.</p>
</li>
<li>
<p>Lines that do not start with an <strong>%</strong> are treated as text and simply inserted into the output.</p>
</li>
<li>
<p>Elements with the form <strong>${VARIABLE_NAME}</strong> are replaced with the value of <strong>VARIABLE_NAME</strong></p>
</li>
<li>
<p>The <strong>%</strong> and <strong>$</strong> characters are escaped respectively as <strong>%%</strong> and <strong>$$</strong>.</p>
</li>
<li>
<p>Blocks of Python code can be added to the template and are delimited by <strong>%{</strong> and <strong>}%</strong> , the indentation outside the block is stripped, so, it’s irrelevant for your Python code.</p>
</li>
</ul>
<p>Let’s see what we can do with these few simple rules with an example, taken from the <a href="https://github.com/uraimo/Bitter/blob/9e9d048c7ee4e503ae7e386380f32b0a587f0126/Templates/Bitter.swift.gyb">Bitter template</a>, that adds to all fixed size integers a computed property <code class="inlinecode">allOnes</code>, that returns an Int/UInt initialized with a bit pattern with all ones:</p>
<pre><code class="Swift">
%{
intTypes = [8,16,32,64]
}%
% for intType in intTypes:
% for sign in ['','U']:
/// Extension that adds a few additional functionalities to ${sign}Int${intType}
extension ${sign}Int${intType} {
/// Returns a ${sign}Int${intType} with all ones
%if sign == '':
public static var allOnes:Int${intType}{return Int${intType}(bitPattern: UInt${intType}.max)}
%else:
public static var allOnes:UInt${intType}{return UInt${intType}.max}
%end
}
%end
%end
</code></pre>
<p>With a Python block we declare an array with all the fixed sizes of the Ints available in Swift and then iterate over it, using an internal loop to consider signed and unsigned integers too. We than output two different snippets depending on the value of the <code class="inlinecode">sign</code> variable, we’ll output the first one if the sign variable is empty (signed integers) or the second one if it’s not (unsigned integers).</p>
<p>In this example we are using a simple if/else and foreach statements, but we could have used everything that is valid in Python like an <code class="inlinecode">elif</code> or a variation of that for.</p>
<p>Running this through GYB we’ll get 8 extensions, one for each fixed size integer, from Int8/UInt8 to Int64/UInt64.</p>
<h2 id="generating-the-source">Generating the source</h2>
<p>You can download GYB from the Swift repository:</p>
<pre><code class="Bash">
wget https://github.com/apple/swift/raw/master/utils/gyb
wget https://github.com/apple/swift/raw/master/utils/gyb.py
chmod +x gyb
</code></pre>
<p>And parse your template this way:</p>
<pre><code class="Bash">
./gyb --line-directive '' -o ../Sources/Bitter/Bitter.swift Bitter.swift.gyb
</code></pre>
<p>The <code class="inlinecode">-o</code> option specifies the output file, while the last filename specifies the name of the file containing the template.</p>
<p>Without the <code class="inlinecode">--line-directive ''</code> parameter, GYB outputs additional debugging comments, that for each section of the output describe which element in the original template was executed to generate it.</p>
<p>Useful while you are still in the process of writing your template but once you are done the debug comments can be disabled to obtain a clean output.</p>
<h2 id="better-debugging-of-generated-sources-with-line-directive">Better debugging of generated sources with line-directive</h2>
<p>But what about debugging code that has been generated with GYB?</p>
<p>That’s where the comments that we purged from the output in the previous step come in handy, comments that will be parsed by an additional tool, <em>line-directive</em>, that will simplify what would otherwise be an excruciating process.</p>
<p>Download it from the main repository on GitHub:</p>
<pre><code class="Bash">
wget https://github.com/apple/swift/raw/master/utils/line-directive
chmod +x line-directive
</code></pre>
<p>To explain how this works, here is a detailed explanation by GYB’s author, Dave Abrahams, directly from the line-directive usage documentation.</p>
<p><code class="inlinecode">#sourceLocation</code> is a directive used by tools like the Swift compiler and debugger to adjust the lines reported in diagnostics and to determine what source you see when you’re stepping.</p>
<p><code class="inlinecode">#sourceLocation</code> corresponds to <code class="inlinecode">#line</code> in C/C++ which is inserted by code generators like Lex/Flex/Yacc/Bison so that you deal with the actual code you wrote and not the generated result. For dealing with errors in the Swift generated by your gyb source it’s important that your tools can take you to the right line in your gyb file rather than in generated .swift file. If you don’t have such a tool, manually indirecting through the generated code is tedious, but at least it’s possible since gyb leaves <code class="inlinecode">#sourceLocation</code> information behind.</p>
<p>But Swift’s <code class="inlinecode">#sourceLocation</code> directive is suboptimal for the purposes of the freeform code generation done with gyb because it can only appear between grammatically-complete declarations and statements. So instead of inserting <code class="inlinecode">#sourceLocation</code> directives, gyb inserts <code class="inlinecode">//###sourceLocation</code> comments (by default, it’s tunable).</p>
<p>This line-directive tool remaps file and line information in the output of your swift compiler (or whatever tool you are using to process generated source, gyb is not swift-specific) so that the error points to the right place in the gyb source.</p>
<p>You invoke it as follows:</p>
<pre><code class="Bash">
line-directive <generated-sources> -- <compilation command>
</code></pre>
<p>For example, if you have foo.swift.gyb, bar.swift.gyb, and baz.swift, instead of:</p>
<pre><code class="Bash">
gyb foo.swift.gyb -o foo.swift
gyb bar.swift.gyb -o bar.swift
swiftc foo.swift bar.swift baz.swift
</code></pre>
<p>You do this:</p>
<pre><code class="Bash">
gyb foo.swift.gyb -o foo.swift
gyb bar.swift.gyb -o bar.swift
line-directive foo.swift bar.swift -- swiftc foo.swift bar.swift baz.swift
</code></pre>
Dealing with Bit Sets in Swift2016-02-05T00:00:00+01:00https://www.uraimo.com/2016/02/05/Dealing-With-Bit-Sets-In-SwiftSwift provides a convenient set of functionalities for fixed size integers and binary operations but you'll soon discover that in some cases the language is a bit opinionated in regard to how those operations should be performed. This post explains some of the gotchas and describe Bitter, a Swift bit manipulation library.<p><strong>Update 10/17:</strong><em>This post has been updated to Swift 4.</em></p>
<p><strong>Update 10/16:</strong><em>This post has been updated to Swift 3.</em></p>
<p>Swift provides convenient fixed size integers and the usual set of bitwise operators you already know, so, dealing with bit sets would seem pretty straightforward.</p>
<p>But you’ll soon discover that the language and the standard library put always safety first and dealing with bits and different integer types requires a few more type conversion than what you could be used to. This post describes the a gotcha you should be aware of.</p>
<p>Before explaining what i mean, let’s get up to speed with the basics of integer types and bitwise operations.</p>
<h4 id="get-the-playground-for-this-article-from-github-or-zipped"><em>Get the playground for this article from <a href="https://github.com/uraimo/Swift-Playgrounds/tree/swift2">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-02-5-Bitwise-Playground.playground.zip">zipped</a>.</em></h4>
<h2 id="integer-types-and-bitwise-operators">Integer Types and bitwise operators</h2>
<p>Swift offers a set of integer types with different fixed length and signedness: <strong>Int/UInt</strong>, <strong>Int8/UInt8</strong>(8 bits), <strong>Int16/UInt16</strong>(16 bits), <strong>Int32/UInt32</strong>(32 bits) and <strong>Int64/UInt64</strong>(64 bits).</p>
<p>The types Int and UInt are platform dependent and are equivalent to <strong>Int32/UInt32</strong> on 32 bit platforms and to <strong>Int64/UInt64</strong> on 64 bit platforms. The others have the specified length regardless of the target platform you compiled for.</p>
<p>Fixed length type are extremely useful when used in combination with bitwise operators because they make explicit the size of the data you are working on, and when performing operations on single bit you’ll rarely use the platform dependent <code class="inlinecode">Int</code> or <code class="inlinecode">UInt</code>.</p>
<p>Variables with a fixed size integer type can be initialized with a binary, octal or hexadecimal value this way:</p>
<pre><code class="Swift">
var int1:UInt8 = 0b10101010
var int2:UInt8 = 0o55
var int3:UInt8 = 0xA7
</code></pre>
<p>In regards to bitwise operations, Swift supports what you’d expect: <strong>unary NOT</strong>(~ operator), <strong>AND</strong>(& operator), <strong>OR</strong>(| operator), <strong>XOR</strong>(^ operator) and the <strong>left and right shift</strong>(« and » operators).</p>
<p>It’s important to remember that for unsigned integers, shifting left or right by the given number of positions inserts a 0 in the opposite direction. For signed integers instead, the sign is preserved inserting the sign bit instead of a zero when shifting to right.</p>
<p>For integers more than one byte long, Swift also provides a few useful computed properties to deal with endianness conversion: <code class="inlinecode">littleEndian</code>, <code class="inlinecode">bigEndian</code>, <code class="inlinecode">byteSwapped</code>, that respectively convert from the current integer representation to big or little endian or convert to the opposite endianness.
One last thing, is there a way to understand if we are on a 32 or 64 bits platform?</p>
<p>Sure, but considering that the <a href="http://ankit.im/swift/2016/01/12/swift-mysterious-builtin-module/">Builtin module</a> is not accessible, we can only compare the size of <em>Int</em> with one of the two fixed size integers that correspond to the two supported platform widths:</p>
<pre><code class="Swift">
MemoryLayout<Int>.stride==MemoryLayout<Int32>.stride //Are we on a 32bits platform? Nope.
</code></pre>
<p>I’m using the <code class="inlinecode">stride</code> property here, but in this case <code class="inlinecode">size</code> would have worked as well.</p>
<h2 id="integer-type-conversions">Integer type conversions</h2>
<p>Swift does not perform implicit type conversions, and as you may have already noticed when doing mixed types arithmetic operations, you need to explicitly convert the variables of your expression to a type big enough to hold your results.</p>
<p>In case of multiple integers in the same expression, Swift can only infer the type of free integer literals when they are used with some typed variable of one single type, as before, no implicit conversion toward the bigger integer type is performed.</p>
<p>Let’s see an example of what is allowed and what isn’t:</p>
<pre><code class="Swift">
var u8:UInt8 = 1
u8 << 2 //4: The number 2 is considered an UInt8 and u8 is shifted
// to the left by 2 positions
var by2:Int16 = 1
u8 << by2 //Error: Operands of different types, doesn't compile
u8 << UInt8(by2) //4: It works, we manually converted the Int type,
// but this is NOT SAFE!
</code></pre>
<p>Why it’s not safe you could ask?</p>
<p>Because when converting a bigger integer type to a smaller one or an unsigned integer to an signed one, Swift does not perform any truncation of the contained value, and <em>fails at runtime</em> when the value being converted overflows the receiving type.</p>
<p>This must be kept in mind and it’s extremely important when you perform conversions on integer that are the result of something entered by the user or coming from some external component.</p>
<p>But luckily, Swift provides a way to perform <em>bit</em> truncating conversions using the <code class="inlinecode">init(truncatingIfNeeded:)</code> constructor, quite useful when you are performing operations on bits and are <strong>not interested in the actual decimal value of the integer</strong>.</p>
<pre><code class="Swift">
var u8:UInt8=UInt8(truncatingIfNeeded:1000)
u8 // 232
</code></pre>
<p>In this sample we converted the <em>Int</em> 1000 that has a binary representation of <code class="inlinecode">0b1111101000</code> to an <em>UInt8</em> just keeping the 8 least significant bits and discarding everything else. That way we obtained 232, that has a binary representation of <code class="inlinecode">0b11101000</code>.</p>
<p>And this works the same way for every combination of Int<em>n</em> or UInt<em>n</em> types, the sign of the signed <em>Int</em> is ignored and the bit sequence is simply used to initialize a new integer variable. Between signed and unsigned integers of the same size, <code class="inlinecode">init(bitPattern:)</code> is also available but the result is the same of the usual truncating conversion.</p>
<p>The only drawback of this safety-first/no-assumptions approach, is that when you need to perform a lot of type conversions, you code starts to become bloated with all those truncating conversions.</p>
<p>But luckily, in Swift we can extend base types with new methods and we can use this to add some utility methods that truncate to a specific size to all the integer types, for example:</p>
<pre><code class="Swift">
extension Int {
public var toU8: UInt8{ get{return UInt8(truncatingIfNeeded:self)} }
public var to8: Int8{ get{return Int8(truncatingIfNeeded:self)} }
public var toU16: UInt16{get{return UInt16(truncatingIfNeeded:self)}}
public var to16: Int16{get{return Int16(truncatingIfNeeded:self)}}
public var toU32: UInt32{get{return UInt32(truncatingIfNeeded:self)}}
public var to32: Int32{get{return Int32(truncatingIfNeeded:self)}}
public var toU64: UInt64{get{
return UInt64(self) //No difference if the platform is 32 or 64
}}
public var to64: Int64{get{
return Int64(self) //No difference if the platform is 32 or 64
}}
}
extension Int32 {
public var toU8: UInt8{ get{return UInt8(truncatingIfNeeded:self)} }
public var to8: Int8{ get{return Int8(truncatingIfNeeded:self)} }
public var toU16: UInt16{get{return UInt16(truncatingIfNeeded:self)}}
public var to16: Int16{get{return Int16(truncatingIfNeeded:self)}}
public var toU32: UInt32{get{return UInt32(self)}}
public var to32: Int32{get{return self}}
public var toU64: UInt64{get{
return UInt64(self) //No difference if the platform is 32 or 64
}}
public var to64: Int64{get{
return Int64(self) //No difference if the platform is 32 or 64
}}
}
var h1 = 0xFFFF04
h1
h1.toU8 // Instead of UInt8(truncatingIfNeeded:h1)
var h2:Int32 = 0x6F00FF05
h2.toU16 // Instead of UInt16(truncatingIfNeeded:h2)
</code></pre>
<h2 id="common-bitwise-patterns">Common Bitwise Patterns</h2>
<p>Now, let’s see some of this in action with some common bitwise operation patterns, just as an excuse to talk about something really useful that is not available in Swift.</p>
<h3 id="multi-byte-component-extraction">Multi-byte Component Extraction</h3>
<p>A combination of AND and right shift is commonly used to extract single bits or bytes from longer sequences, let’s see an example where we want to extract the single color components from an RGB representation of the color:</p>
<pre><code class="Swift">
let swiftOrange = 0xED903B
let red = (swiftOrange & 0xFF0000) >> 16 //0xED
let green = (swiftOrange & 0x00FF00) >> 8 //0x90
let blue = swiftOrange & 0x0000FF //0x3B
</code></pre>
<p>Here we are isolating the bits we are interested in performing an <em>AND</em> with a bitmask that has only the bits we want to get in the result at <code class="inlinecode">1</code>, the rest is zeroed out. To obtain and 8 bits representation of the component we need we shift the result of the <em>AND</em> to the right, moving of 16 positions for the red component(2 bytes to the right) and 8 for the green component (1 byte to the right).
And that’s it, this masking+shifting pattern has a wide range of applications, but when used in sub-expression it can make your expression unreadable really fast, but why not implement it as a subscript of all the integer types? In other words, why not add the ability to access with an index the single byte components of an Int like we do with arrays?</p>
<p>For example, let’s add the subscript to <code class="inlinecode">Int32</code>:</p>
<pre><code class="Swift">
extension UInt32 {
public subscript(index: Int) -> UInt32 {
get {
precondition(index<4,"Byte set index out of range")
return (self & (0xFF << (index.toU32*8))) >> (index.toU32*8)
}
set(newValue) {
precondition(index<4,"Byte set index out of range")
self = (self & ~(0xFF << (index.toU32*8))) | (newValue << (index.toU32*8))
}
}
}
var i32:UInt32=982245678 //HEX: 3A8BE12E
print(String(i32,radix:16,uppercase:true)) // Printing the hex value
i32[3] = i32[0]
i32[1] = 0xFF
i32[0] = i32[2]
print(String(i32,radix:16,uppercase:true)) //HEX: 2E8BFF8B
</code></pre>
<h3 id="the-magical-xor">The magical XOR</h3>
<p>Some of you may know XOR from the simple and quite useless <em>XOR</em> cipher, that allows to encrypt a bit stream with a key and retrieve the original value performing again an <em>XOR</em> with the same key.
For the sake of brevity, let’s use a message and a key with the same size:</p>
<pre><code class="Swift">
let secretMessage = 0b10101000111110010010101100001111000 // 0x547C95878
let secretKey = 0b10101010101010000000001111111111010 // 0x555401FFA
let result = secretMessage ^ secretKey // 0x12894782
let original = result ^ secretKey // 0x547C95878
print(String(original,radix:16,uppercase:true)) // Printing the hex value
</code></pre>
<p>The same property of the <em>XOR</em> can be used for other things, the most simple one is the <a href="https://en.wikipedia.org/wiki/XOR_swap_algorithm">XOR swap</a> that swaps two integer variables without an additional temporary variable:</p>
<pre><code class="Swift">
var x=1
var y=2
x = x ^ y
y = y ^ x // y is now 1
x = x ^ y // x is now 2
</code></pre>
<p>Not really useful in Swift since you can do the same trick using tuples (see item #11 <a href="https://www.uraimo.com/2016/01/06/10-Swift-One-Liners-To-Impress-Your-Friends/">here</a>).</p>
<p>Another thing you can do with the <em>XOR</em> but that i will not describe here, is building a variation of the classic doubly linked lists: the XOR linked list. A way more interesting use of the <em>XOR</em>, learn more about this on <a href="https://en.wikipedia.org/wiki/XOR_linked_list">wikipedia</a>.</p>
<h3 id="double-negation-is-that-bit-set">Double Negation: Is That Bit Set?</h3>
<p>Another common pattern, similar to the first presented, is using a bitmask in conjunction with a suspicious looking double negation to understand if a specific bit or a specific bit pattern is set in the input bit sequence:</p>
<pre><code class="Swift">
let input:UInt8 = 0b10101101
let mask:UInt8 = 0b00001000
let isSet = !!(input & mask) // If the 4th bit is set this is equal to 1.
// But this code is not valid in Swift...
</code></pre>
<p>The double negation is based on the specific behavior of the logical negation in C/C++ (and a few other languages) and the fact that in C/C++ booleans are implemented with integers (0 as false, 1 as true), quoting from the C99 standard:</p>
<blockquote>
<p>The result of the logical negation operator ! is 0 if the value of its operand compares unequal to 0, 1 if the value of its operand compares equal to 0. The result has type int. The expression !E is equivalent to (0==E).</p>
</blockquote>
<p>Considering this, the behavior of the double negation becomes more clear. The first logical <em>NOT</em> turns our masked input into either a 0 or 1 if the operand was respectively a number greater than 0 or 0 (effectively inverting its boolean value as we’d expect), while the second logical <em>NOT</em> turns the input back to the original boolean value, but now, the only possible value will be 0 or 1. Maybe the explanation is a bit messy, but you should get the idea of how this works.</p>
<p>But Swift has a proper boolean type and the logical <em>NOT</em> works only on those booleans. So, what can we do?</p>
<p>Let’s define a custom operator(usually i don’t really like them, but let’s make an exception) that implements the double negation for the <code class="inlinecode">UInt8</code> type!</p>
<pre><code class="Swift">
prefix operator ~~
prefix func ~~(value: UInt8)->UInt8{
return (value>0) ? 1 : 0
}
~~7 // 1
~~0 // 0
let isSet = ~~(input & mask) // 1 as expected
</code></pre>
<p>As an improvement we could return a <code class="inlinecode">Bool</code> instead of an <code class="inlinecode">UInt8</code>, to use it directly in conditional statements, but we’ll loose the ability to embed it in other integer expressions.</p>
<h2 id="bitter-a-library-for-bits-manipulation">Bitter: A library for bits manipulation</h2>
<p><img src="https://raw.githubusercontent.com/uraimo/Bitter/master/logo.png" srcset="https://raw.githubusercontent.com/uraimo/Bitter/master/logo.svg?sanitize=true" alt="Bitter" style="width:500px" /></p>
<p>All the alternative approaches to manipulate bit sets presented in this post are part of <a href="https://github.com/uraimo/Bitter">Bitter</a>, a library that try to offer a more “swifty” interface for bit sets manipulation.</p>
<p>To recap what you’ll find in Bitter (available via CocoaPods,Carthage, SwiftPM):</p>
<ul>
<li>Convenient computed properties for bit pattern truncating conversions</li>
<li>Integer byte indexed subscripting for every integer type</li>
<li>The double negation operator</li>
<li>and more…</li>
</ul>
<p>The library is still young and feedback is highly appreciated, feel free <a href="https://github.com/uraimo/Bitter">to try it out</a> and open issues if something doesn’t work or you have ideas for additional features!</p>
10 Swift One Liners To Impress Your Friends2016-01-06T00:00:00+01:00https://www.uraimo.com/2016/01/06/10-Swift-One-Liners-To-Impress-Your-FriendsFollowing a programming meme of a few years ago, this post presents a few Swift one liners that solve common problems following a functional approach. Will some of your friends be impressed?<p><strong>Update 10/17:</strong><em>This post has been updated for Swift 4</em></p>
<p><strong>Update 09/16:</strong><em>This post has been updated for Swift 3</em></p>
<p>A few years ago, at the peak of the Functional Renaissance™, a blog post that presented <a href="https://mkaz.github.io/2011/05/31/10-scala-one-liners-to-impress-your-friends/">10 Scala functional one liners</a> became quite popular and was rapidly followed by a series of articles that implemented the same one liners in other languages like <a href="http://blog.fogus.me/2011/06/03/10-haskell-one-liners-to-impress-your-friends/">Haskell</a>, <a href="http://programmingzen.com/2011/06/02/10-ruby-one-liners-to-impress-your-friends/">Ruby</a>, <a href="http://arturoherrero.com/10-groovy-one-liners-to-impress-your-friends/">Groovy</a>, <a href="http://freegeek.in/blog/2011/06/10-clojure-one-liners/">Clojure</a>, <a href="http://codeblog.dhananjaynene.com/2011/06/10-python-one-liners-to-impress-your-friends/">Python</a>, <a href="https://gist.github.com/1004837">C#</a>, <a href="http://willwhim.wordpress.com/2011/06/02/fsharp-one-liners-to-impress-your-friends/">F#</a>, <a href="http://ricardo.cc/2011/06/02/10-CoffeeScript-One-Liners-to-Impress-Your-Friends.html">CoffeeScript</a>.</p>
<p>We’ll never know how many people were actually impressed by those one liners during social gatherings, but my guess is that at least for the uninitiated the more complex examples were a good incentive to learn more about functional programming.</p>
<p>Let’s see how Swift fares against the other languages, trying to solve the same 10 exercises using one liners, maybe obtaining something interesting in the process (see #6 and #10).</p>
<h4 id="get-the-playground-for-this-article-from-github-or-zipped"><em>Get the playground for this article from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="https://www.uraimo.com/archives/2016-01-6-Swift-One-Liners-Playground.playground.zip">zipped</a>.</em></h4>
<h3 id="1-multiply-each-element-of-an-array-by-2">#1 Multiply each element of an array by 2</h3>
<p>Not much to see in this first example, easily solvable using map as <a href="https://www.uraimo.com/2015/10/08/Swift2-map-flatmap-demystified/">we all know</a>.</p>
<pre><code class="Swift">
(1...1024).map{$0 * 2}
</code></pre>
<h3 id="2-sum-a-list-of-numbers">#2 Sum a list of numbers</h3>
<p>This exercise is solved using reduce and the plus operator, leveraging the fact that the plus operator is a function, but the solution is obvious, we’ll see in a moment a few more creative uses of <code class="inlinecode">reduce</code>.</p>
<pre><code class="Swift">
(1...1024).reduce(0,+)
</code></pre>
<h3 id="3-verify-if-exists-in-a-string">#3 Verify if Exists in a String</h3>
<p>Let’s verify if a tweet contains at least one of a few selected keywords using <code class="inlinecode">filter</code>:</p>
<pre><code class="Swift">
let words = ["Swift","iOS","cocoa","OSX","tvOS"]
let tweet = "This is an example tweet larking about Swift"
let valid = !words.filter({tweet.contains($0)}).isEmpty
valid //true
</code></pre>
<p><em>Update:</em> <a href="https://twitter.com/oisdk">@oisdk</a> suggests a few better alternatives:</p>
<pre><code class="Swift">
words.contains(where:tweet.contains)
</code></pre>
<p>Way more concise, and this one:</p>
<pre><code class="Swift">
tweet.characters.split(separator:" ")
.lazy
.map(String.init)
.contains(where:Set(words).contains)
</code></pre>
<h3 id="4-read-in-a-file">#4 Read in a File</h3>
<p>Reading a file into an array of lines is not possible through an easy built-in like in other languages but we can create something short that doesn’t need a <em>for</em> using a combination of <code class="inlinecode">split</code> and <code class="inlinecode">map</code>:</p>
<pre><code class="Swift">
let path = Bundle.main.path(forResource:"test", ofType: "txt")
let lines = try? String(contentsOfFile: path!).characters
.split{$0 == "\n"}
.map(String.init)
if let lines=lines {
lines[0] // O! for a Muse of fire, that would ascend
lines[1] // The brightest heaven of invention!
lines[2] // A kingdom for a stage, princes to act
lines[3] // And monarchs to behold the swelling scene.
}
</code></pre>
<p>That last step with map and the string constructor turns our arrays of characters into strings.</p>
<h3 id="5-happy-birthday-to-you">#5 Happy Birthday to You!</h3>
<p>This will display the Happy Birthday song to console, a simple use of map with a range and the ternary operator.</p>
<pre><code class="Swift">
let name = "uraimo"
(1...4).forEach{print("Happy Birthday " + (($0 == 3) ? "dear \(name)":"to You"))}
</code></pre>
<h3 id="6-filter-list-of-numbers">#6 Filter list of numbers</h3>
<p>In this case we are asked to partition a sequence using a provided filtering function. Many languages have in addition to the usual map, flatMap, reduce, filter, etc… also a <code class="inlinecode">partitionBy</code> function that does exactly that, Swift as you know doesn’t have something similar (the <em>NSArray</em> function that filters by <em>NSPredicate</em> is not what we want here).</p>
<p>Therefore, we could solve this extending <code class="inlinecode">Sequence</code> with a <code class="inlinecode">partitionBy</code> function that we’ll use to partition an integer array:</p>
<pre><code class="Swift">
extension Sequence{
typealias Element = Self.Iterator.Element
func partitionBy(fu: (Element)->Bool)->([Element],[Element]){
var first: [Element] = []
var second: [Element] = []
for el in self {
if fu(el) {
first.append(el)
}else{
second.append(el)
}
}
return (first,second)
}
}
let part = [82, 58, 76, 49, 88, 90].partitionBy{$0 < 60}
part // ([58, 49], [82, 76, 88, 90])
</code></pre>
<p>It’s not really a one liner and the approach is imperative. But could we use filter to improve it a little?</p>
<pre><code class="Swift">
extension Sequence{
func anotherPartitionBy(fu: (Self.Iterator.Element)->Bool)->
([Self.Iterator.Element],[Self.Iterator.Element]){
return (self.filter(fu),self.filter({!fu($0)}))
}
}
let part2 = [82, 58, 76, 49, 88, 90].anotherPartitionBy{$0 < 60}
part2 // ([58, 49], [82, 76, 88, 90])
</code></pre>
<p>This is slightly better, but it traverses the sequence two times and trying to turn this into a one liner removing the enclosing function will get us something with too much duplicated stuff (the filtering function and the array that will be used in two places).</p>
<p>Can we build something that will transform the original sequence into a partition tuple using a single stream of data? Yes we can, using <code class="inlinecode">reduce</code>.</p>
<pre><code class="Swift">
var part3 = [82, 58, 76, 49, 88, 90].reduce( ([],[]), {
(a:([Int],[Int]),n:Int) -> ([Int],[Int]) in
(n<60) ? (a.0+[n],a.1) : (a.0,a.1+[n])
})
part3 // ([58, 49], [82, 76, 88, 90])
</code></pre>
<p>What we are doing here is building the result tuple that contains the two partitions, an element at a time, testing each element of the original sequence using the filtering function and appending this element to the first or second partition array depending on the filtering result.</p>
<p>Finally a true one liner but notice that the fact that the partition arrays are being built via append will actually make it way slower than the two previous implementations.</p>
<h3 id="7-fetch-and-parse-an-xml-web-service">#7 Fetch and Parse an XML web service</h3>
<p>Some of languages above don’t rely on external libraries and have more than one option available by default to deal with XML (e.g. Scala that “natively” albeit awkwardly supports XML parsing into objects), but Foundation provides only the SAX parser NSXMLParser, and as you may have already guessed we are not going to use it.</p>
<p>There are a few alternative open source libraries we could use in this case, some of them written in C or Objective-C and others in pure Swift.</p>
<p>This time we are going to use the pure-Swift <a href="https://github.com/tadija/AEXML">AEXML</a>:</p>
<pre><code class="Swift">
let xmlDoc = try? AEXMLDocument(xmlData: NSData(contentsOf: URL(string:"https://www.ibiblio.org/xml/examples/shakespeare/hen_v.xml")!)!)
if let xmlDoc=xmlDoc {
var prologue = xmlDoc.root.children[6]["PROLOGUE"]["SPEECH"]
prologue.children[1].stringValue // Now all the youth of England are on fire,
prologue.children[2].stringValue // And silken dalliance in the wardrobe lies:
prologue.children[3].stringValue // Now thrive the armourers, and honour's thought
prologue.children[4].stringValue // Reigns solely in the breast of every man:
prologue.children[5].stringValue // They sell the pasture now to buy the horse,
}
</code></pre>
<h3 id="8-find-minimum-or-maximum-in-a-list">#8 Find minimum (or maximum) in a List</h3>
<p>We have various ways to find the minimum and maximum of a sequence, among those the <code class="inlinecode">min</code> and <code class="inlinecode">max</code> functions:</p>
<pre><code class="Swift">
//Find the minimum of an array of Ints
[10,-22,753,55,137,-1,-279,1034,77].sorted().first
[10,-22,753,55,137,-1,-279,1034,77].reduce(Int.max, min)
[10,-22,753,55,137,-1,-279,1034,77].min()
//Find the maximum of an array of Ints
[10,-22,753,55,137,-1,-279,1034,77].sorted().last
[10,-22,753,55,137,-1,-279,1034,77].reduce(Int.min, max)
[10,-22,753,55,137,-1,-279,1034,77].max()
</code></pre>
<h3 id="9-parallel-processing">#9 Parallel Processing</h3>
<p>Some languages allows to enable in a simple and transparent way parallel processing of sequences for functionalities like <em>map</em> and <em>flatMap</em>, to speed up the execution of sequential and independent operations using thread pools under the hood.</p>
<p>This feature is not yet available in Swift but can be built using GCD: <a href="http://moreindirection.blogspot.it/2015/07/gcd-and-parallel-collections-in-swift.html">http://moreindirection.blogspot.it/2015/07/gcd-and-parallel-collections-in-swift.html</a></p>
<h3 id="10-sieve-of-erathostenes">#10 Sieve of Erathostenes</h3>
<p>The good old sieve of Erathostenes is used to find all the prime numbers until a given upper limit <em>n</em>.</p>
<p>Starting with a sequence of all the integers smaller than <em>n</em>, the algorithm removes all the multiples of each integer, until we are left with just prime numbers. And to speed up the execution, we don’t actually need to check each integer for its multiple, we can just stop at the square root of <em>n</em>.</p>
<p>Based on that definition the first implementation could look like this:</p>
<pre><code class="Swift">
var n = 50
var primes = Set(2...n)
(2...Int(sqrt(Double(n)))).forEach{
primes.subtract(Set(stride(from:2*$0, to:n+1, by:$0)))
}
primes.sorted()
</code></pre>
<p>We use the outer range to iterate over the integers we want to check and for each one we calculate a sequence of multiples of those numbers using <code class="inlinecode">stride(from:Int, to:Int, by:Int)</code>. Those sequences are then substracted from a <em>Set</em> initialized with all the integers from 2 to <em>n</em>.</p>
<p>But as you can see, to actually remove the multiples we use an external mutable <em>Set</em>, introducing a side-effect.</p>
<p>To eliminate side-effects, as we should always try to do, we will first calculate all the subsequences, flatMap them in a single array of multiples and remove these integers from the original Set.</p>
<pre><code class="Swift">
var sameprimes = Set(2...n)
sameprimes.subtract(Set(2...Int(sqrt(Double(n))))
.flatMap{stride(from:2*$0, to:n+1, by:$0)})
sameprimes.sorted()
</code></pre>
<p>Way cleaner and a nice example of <a href="https://www.uraimo.com/2015/10/08/Swift2-map-flatmap-demystified/">usage of <em>flatMap</em> to flatten nested arrays</a>.</p>
<h3 id="11-bonus-tuple-swap-via-destructuring">#11 Bonus: Tuple swap via destructuring</h3>
<p>As a bonus, not everyone knows that like in other languages that have a tuple type, you can leverage tuple destructuring to perform a compact variable swap:</p>
<pre><code class="Swift">
var a=1,b=2
(a,b) = (b,a)
a //2
b //1
</code></pre>
<p>And that’s all, as expected Swift is as expressive as many other languages.</p>
<p><em>Thanks to <a href="https://twitter.com/oisdk">@oisdk</a> for reviewing the post</em></p>
Xcode 7.2 and Swift 2.1.1 Released [Updated, XCode 7.2.1]2015-12-08T00:00:00+01:00https://www.uraimo.com/2015/12/08/XCode-7.2-and-Swift-2.1.1-releasedA new XCode version and a maintenance release of the Swift compiler are now available<p><strong>Update 01/03/16:</strong></p>
<p>XCode 7.2.1 has been released, this release it’s a maintenance release with minor fixes and stability improvements, no updated to Swift are included.
Those minor fixes include:</p>
<ul>
<li><code class="Swift">xcodebuild test</code> will no longer timeout.</li>
<li>Resolved a debugger crash that could occur when your code was depending on a binary Swift library or framework.</li>
<li>The certificates used for Apple Wallet passes, Safari Push notifications and Safari extensions have been updated.</li>
</ul>
<p><strong>Original XCode 7.2 Release Post:</strong></p>
<p>With the new version of XCode released today a maintenance release of the Swift compiler will be also available, as expected there is nothing new regarding the language itself, only a few fixes as described in the <a href="https://developer.apple.com/library/ios/releasenotes/DeveloperTools/RN-Xcode/Chapters/xc7_release_notes.html">release notes</a>.</p>
<p>For more informations about what changed from the release 2.0 to 2.1.x check out this <a href="http://www.uraimo.com/2015/09/20/Swift2.1-released/">specific post</a> and this post on <a href="http://www.uraimo.com/2015/09/29/Swift2.1-Function-Types-Conversion-Covariance-Contravariance/">Swift 2.1 Function Types Conversion: Covariance and Contravariance</a>.</p>
<p>Here is a summary of the changes:</p>
<ul>
<li>
<p>In previous releases of Swift, if a type had a mutable property of protocol type, chained accesses to properties of that property were always treated as mutations of the property, even if the second property was only read, not written. For example:</p>
<pre><code class="Swift">
protocol Countable {
var count: Int { get }
}
class MyObject {
var widgets : Countable {
didSet { print("in didSet") }
}
}
var obj : MyObject = ...
let count = obj.widgets.count
</code></pre>
<p>The example above would perform a spurious write back to the property widgets, causing didSet to unexpectedly fire. The workaround was to split the access into separate expressions. For example:</p>
<pre><code class="Swift">
let widgets = obj.widgets
let count = widgets.count
</code></pre>
<p>This bug has now been fixed(radar 22953072).</p>
</li>
<li>
<p>Swift data types that are imported into Swift from C struct types (such as CGRect and CGPoint) can now be inspected in the debugger (radar 23088739).</p>
</li>
<li>
<p>A bug that caused some Swift projects to crash in whole module optimization with a “program too clever” LLVM error has been fixed (radar 23200656).</p>
</li>
<li>
<p>A bug has been fixed where Swift code calling an Objective-C method that took a block returning a nonnull NSString * with a Swift closure would be miscompiled, causing the compiled program to crash (radar 23285766).</p>
</li>
</ul>
Swift 3.0 Upcoming Changes2015-12-03T00:00:00+01:00https://www.uraimo.com/2015/12/03/Swift-3-upcoming-changesSwift is now finally opensource, this post introduces the features and modifications that will be available with release 3.0 of the language.<p>It finally happened, today <a href="http://swift.org/">Swift has been opensourced</a> and it’s <a href="https://github.com/apple/swift">available on GitHub</a>.</p>
<p>And with its public release of the source, the first informations on what will be included in the next major release, Swift 3.0, are finally available.</p>
<p>From now on, everything related to the evolution of the language will be tracked through the aptly named <a href="https://github.com/apple/swift-evolution">Swift-Evolution</a> repository on GitHub.</p>
<p>Proposals for updates to the language (in the form of new features or alterations to the current behaviour, that will come from Apple or the community) will be discussed following the <a href="https://github.com/apple/swift-evolution/blob/master/process.md">Swift Evolution Process</a> on the dedicated <a href="https://swift.org/community/#mailing-lists">mailing lists</a>, and Apple will review each proposal and ultimately decide what will be accepted or rejected.</p>
<p>At the moment, Swift 3.0 is planned to be released in the fall of 2016.</p>
<p>The update will focus on a few key areas:</p>
<ul>
<li>
<p><strong>ABI Stabilization:</strong> Stabilize the application binary interface (ABI) to guarantee binary backward compatibility. This will involve the stabilization of core details of the language implementation like function calling conventions and internal language objects name mangling.</p>
</li>
<li>
<p><strong>Language refinement:</strong> Even if the language is still in its infancy, the next releases will start to make the language more consistent outlining a set of core principles that define the language and eliminating features that deviate from those principles.</p>
</li>
<li>
<p><strong>Improvements to generics:</strong> The work on generics will reach completion and some known gaps will be filled.</p>
</li>
<li>
<p><strong>Type system cleanup and documentation of behaviour</strong></p>
</li>
<li>
<p><strong>Complete the alignement of core libraries to the API guidelines</strong></p>
</li>
</ul>
<p>No updates related to new concurrency primitives or C++ interoperability will be included in the next major release, but these are among the new features that will be available in future releases.</p>
<p>Regarding the actual changes coming with Swift 3.0, for now we know that a few proposals have been accepted and will be included:</p>
<h3 id="better-translation-of-objective-c-apis-into-swift"><a href="https://github.com/apple/swift-evolution/blob/master/proposals/0005-objective-c-name-translation.md">Better Translation of Objective-C APIs Into Swift</a></h3>
<p>The automatic translation of Objective-C APIs will be improved to adhere more strictly to what is described in the <a href="https://swift.org/documentation/api-design-guidelines.html">Swift API Design Guidelines</a>, so, as you imagine, more than a few method names will change.</p>
<p>To give you and idea of some of the changes, the translator will drop the NS prefix from all Foundation classes, boolean properties will gain an <em>is</em> prepended to their name (e.g. a boolean <code class="inlinecode">empty</code> property will become <code class="inlinecode">isEmpty</code>) methods will lose redundant portions of their name (e.g. <code class="inlinecode">UIBezierPath.moveToPoint(p : CGPoint)</code> will become <code class="inlinecode">UIBezierPath.moveTo(p : CGPoint)</code> and methods like <code class="inlinecode">UIBezierPath.bezierPathByReversingPath()</code> will simply become <code class="inlinecode">UIBezierPath.reversing()</code>).</p>
<h3 id="removal-of-currying-func-declaration-syntax"><a href="https://github.com/apple/swift-evolution/blob/master/proposals/0002-remove-currying.md">Removal of currying func declaration syntax</a></h3>
<p>To simplify the language, removing some rarely used syntactic sugar, Swift 3.0 will lose the curried function declaration syntax (e.g. func foo(a:Int)(b:Int)). Not a big loss in my opinion.</p>
<h3 id="removal-of--var-from-function-parameters-and-pattern-matching"><a href="https://github.com/apple/swift-evolution/blob/master/proposals/0003-remove-var-parameters-patterns.md">Removal of var from Function Parameters and Pattern Matching</a></h3>
<p>It will not be possible anymore to create local mutating copies of function parameters or binding variables inside if/switch/guard/for/etc… using the <em>var</em> keyword. Thus, no more implicit variable shadowing will be performed, and that’s a good thing.</p>
<p>To give you a practical example, this will not be valid Swift 3.0 code:</p>
<pre><code class="Swift">
if var x = getOptionalInt() {
x += 1
return x
}
</code></pre>
<p>But something like that could be easily updated this way, introducing a new temporary variable:</p>
<pre><code class="Swift">
if let x = getOptionalInt() {
var x = x
x += 1
return x
}
</code></pre>
<h3 id="removal-of-the--and--operators"><a href="https://github.com/apple/swift-evolution/blob/master/proposals/0004-remove-pre-post-inc-decrement.md">Removal of the ++ and – operators</a></h3>
<p>The unary increment and decrement operators will be completely removed. Implemented only by a few types, prone to be implemented with some non-obvious implicit behaviour, and prone to encourage the implementation of some overly “clever” code.</p>
<p>And that’s all for now.</p>
<p>The list of features is obviously not complete yet, more proposals will be discussed and accepted in the following months. I’ll keep and eye on <a href="https://github.com/apple/swift-evolution">Swift-Evolution</a> and the mailing-list to follow the progress of the release.</p>
<h3 id="updates">Updates</h3>
<h3 id="101215"><em>10/12/15</em></h3>
<p>The first proposal from the community <a href="https://github.com/apple/swift-evolution/blob/master/proposals/0007-remove-c-style-for-loops.md">has been accepted</a>, there will be no more c-style For (the good old indexed for) in Swift 3.0 (a warning will be displayed in Swift 2.2). A lot of additional modifications are being proposed on the mailing list, I recommend to check it out, there are many discussions that detail the rationale behind current language choices, explanations you will not find anywhere else.</p>
<h3 id="81215"><em>8/12/15</em></h3>
<p>Read an <a href="http://arstechnica.com/apple/2015/12/craig-federighi-talks-open-source-swift-and-whats-coming-in-version-3-0/">interesting interview</a> to Craig Federighi about the future of open source Swift.</p>
<p>After just one day, more than a few additional proposal for modifications are already being discussed on the specific mailing list (<a href="https://lists.swift.org/pipermail/swift-evolution/2015-December/date.html">archive here</a>) and proposal documents <a href="https://github.com/apple/swift-evolution/tree/master/proposals">are starting to appear</a> on github.</p>
Experimenting with Swift 3 Sequences and Iterators2015-11-12T00:00:00+01:00https://www.uraimo.com/2015/11/12/experimenting-with-swift-2-sequencetype-generatortypeIn this article, part of a series on Swift and the functional approach, we'll explore what we need to do to build our own sequences in Swift 3, discuss the differences between finite and infinite sequences and examine what we can do with them in a few example scenarios.<p><strong>Update 02/17:</strong> <em>Improved the code snippets, with less cryptic variable names.</em></p>
<p><strong>Update 10/16:</strong> <em>This post has been updated to Swift 3.</em></p>
<p>In this article, part of a series on <a href="http://uraimo.com/category/functional/">Swift and the functional approach</a>, we’ll explore what we need to do to build our own sequences in Swift 2, discuss the differences between finite and infinite sequences and examine what we can do with them in a few example scenarios.</p>
<p><img src="/imgs/sequences.png" srcset="/imgs/sequences@2x.png 2x" alt="A Sequence" /></p>
<h4 id="get-the-playground-for-this-post-from-github-or-zipped"><em>Get the playground for this post from <a href="https://github.com/uraimo/Swift-Playgrounds/">GitHub</a> or <a href="https://github.com/uraimo/Swift-Playgrounds/raw/master/archives/2015-11-12-SequenceTypeGeneratorTypePlayground.playground.zip">zipped</a>.</em></h4>
<p>The <code class="inlinecode">Sequence</code> standard protocol is defined in the documentation simply as <em>a type that can be iterated with a for…in loop</em>. The section of the protocol definition relevant to our custom implementation is in the top half:</p>
<pre><code class="Swift">
public protocol Sequence {
/// A type that provides the sequence's iteration interface and
/// encapsulates its iteration state.
associatedtype Iterator : IteratorProtocol
/// Returns an iterator over the elements of this sequence.
func makeIterator() -> Iterator
...
...
}
</code></pre>
<p>The protocol contains an <em>associated type</em> (Swift weird way of making protocols generic) that refers to the <code class="inlinecode">IteratorProtocol</code> protocol, we’ll need to implement this too in some way when creating a sequence. Our custom <code class="inlinecode">Sequence</code> will return a custom iterator with a specific element type when <code class="inlinecode">makeIterator()</code> is called.</p>
<p>Sequences also provide many other interesting methods, already implemented via protocol extensions, like <strong>map</strong>, <strong>flatmap</strong> (check out my in-depth article on <a href="http://www.uraimo.com/2015/10/08/Swift2-map-flatmap-demystified/">map and flatMap</a>), <strong>filter</strong>, <strong>reduce</strong>, <strong>subsequence functions</strong> and others.</p>
<p>Having these for free makes <code class="inlinecode">Sequence</code> a bit more useful that just a type that can be used with a for each.</p>
<p>Let’s take a look at the <code class="inlinecode">IteratorProtocol</code> definition:</p>
<pre><code class="Swift">
public protocol IteratorProtocol {
/// The type of element traversed by the iterator.
associatedtype Element
...
...
/// - Returns: The next element in the underlying sequence if a next element
/// exists; otherwise, `nil`.
mutating func next() -> Element?
}
</code></pre>
<p>This simple protocol contains just a <code class="inlinecode">next()</code> method, responsible for returning the next element in the sequence managed by this iterator. It’s <strong>very</strong> important that the iterator returns <em>nil</em> when the sequence ends, we’ll see why below, when we’ll build an infinite sequence.</p>
<p>Let’s build a simple iterator that produces numbers from the well known Fibonacci sequence:</p>
<pre><code class="Swift">
class FibonacciIterator : IteratorProtocol {
var nextValues = (0,1)
var stopsAt:Int
var iterationsCount = 0
init(iteratorLength:Int){
stopsAt = iteratorLength
}
func next() -> Int?{
guard iterationsCount<stopsAt else {
return nil
}
iterationsCount += 1
let next = nextValues.0
nextValues = (nextValues.1,nextValues.0+nextValues.1)
return next
}
}
</code></pre>
<p>To return a finite sequence we need an additional constructor that we’ll use to specify the sequence length and return <em>nil</em> instead of a new element when we reach it. There is not much else to see here other than the tuple swap trick that save us a few lines, but let’s see how to use this iterator:</p>
<pre><code class="Swift">
var fg = FibonacciIterator(iteratorLength:10)
while let fib = fg.next() {
print(fib)
}
</code></pre>
<p>This way we’ll iterate on the elements until <em>nil</em> is returned.</p>
<p>Implementing a <em>Sequence</em> for this iterator is straightforward:</p>
<pre><code class="Swift">
class FibonacciSequence : Sequence {
var stopsAt:Int
init(sequenceLength:Int){
stopsAt = sequenceLength
}
func makeIterator() -> FibonacciIterator{
return FibonacciIterator(iteratorLength: stopsAt)
}
}
let arr = Array(FibonacciSequence(sequenceLength:10))
for f in FibonacciSequence(sequenceLength: 10) {
print(f)
}
</code></pre>
<p>The sequence can be used in a foreach as expected but can also be used to create other sequences like an array as seen above.</p>
<p>But there is no need to declare the iterator as a separated entity, we can use the <code class="inlinecode">AnyIterator<T></code> class to make this example more compact:</p>
<pre><code class="Swift">
class CompactFibonacciSequence : Sequence {
var stopsAt:Int
init(sequenceLength:Int){
stopsAt = sequenceLength
}
func makeIterator() -> AnyIterator<Int> {
var nextValues = (0,1)
var iterationsCount = 0
return AnyIterator{
guard iterationsCount<self.stopsAt else {
return nil
}
iterationsCount += 1
let next = nextValues.0
nextValues = (nextValues.1,nextValues.0+nextValues.1)
return next
}
}
}
</code></pre>
<p>This will work exactly like the previous sequence, the only substantial difference is that the <code class="inlinecode">AnyIterator<Int></code> returned by <code class="inlinecode">makeIterator()</code> conforms to <em>Sequence</em> too now, it’s not anymore just a simple object implementing <em>IteratorProtocol</em> like the one we started with.</p>
<p>Not really that useful here, considering that the iterator is already embedded in a sequence, but in some circumstances, a simple sequence generated with <code class="inlinecode">AnyIterator(body:)</code> could be more than enough for what we want to do.</p>
<p>For instance, we could create a sequence with the first 10 numbers of the <a href="https://en.wikipedia.org/wiki/Lucas_number">Lucas sequence</a>, a numeric series similar to Fibonacci that starts with <em>2,1</em> instead of <em>0,1</em> generating a quite different sequence (i.e. 2, 1, 3, 4, 7, 11, 18, 29, etc…) , using just an iterator and initializing an array with it:</p>
<pre><code class="Swift">
var nextValues = (2,1)
var iterationsCount = 0
let lucas = AnyIterator{
()->Int? in
guard iterationsCount<10 else {
return nil
}
iterationsCount += 1
let next = nextValues.0
last = (nextValues.1,nextValues.0+nextValues.1)
return next
}
let a = Array(lucas) //[2, 1, 3, 4, 7, 11, 18, 29, 47, 76]
</code></pre>
<p>Definitely not bad, we removed a lot of boilerplate, but since we can improve our algorithm further with a formula involving the <a href="https://en.wikipedia.org/wiki/Golden_ratio">golden ratio</a>, let’s do it:</p>
<pre><code class="Swift">
import Darwin
let Phi = (sqrt(5)+1.0)/2
let phi = 1/Phi
func luc(n:Int)->Int {
return Int(pow(Phi, Double(n))+pow(-phi,Double(n)))
}
c = 0
var compactLucas = AnyIterator{ c<10 ? luc(n: c+1): nil }
let a2 = Array(compactLucas) //[2, 1, 3, 4, 7, 11, 18, 29, 47, 76]
</code></pre>
<p>Does it really work? Yes, feel free to play around with it using the <a href="https://github.com/uraimo/Swift-Playgrounds/">playground</a> (<a href="https://github.com/uraimo/Swift-Playgrounds/raw/master/archives/2015-11-12-SequenceTypeGeneratorTypePlayground.playground.zip">zip</a>).</p>
<p>To try out some of the functional(ish) facilities that <code class="inlinecode">Sequence</code> provide, we’ll now build a derived sequence that will only return <em>even</em> numbers from the Lucas sequence:</p>
<pre><code class="Swift">
c = 0
var evenCompactLucas = AnyIterator{ c<10 ? luc(n: c+1): nil }.filter({$0 % 2 == 0})
let a3 = Array(evenCompactLucas) //[2, 4, 18, 76]
</code></pre>
<p>Notice that we are redeclaring our <code class="inlinecode">AnyIterator</code> because the previous one has already been used up and it will return no more elements, it has reached the end of the sequence it was able to generate and from now on it will return only <em>nil</em>. But this aside, you can also notice how easily we modified the original sequence to return a modified set of objects. We could perform even bolder transformations using map methods.</p>
<h3 id="infinite-sequences">Infinite Sequences</h3>
<p>But now, what it we remove the nil termination requirement described above to build an infinite sequence of all the possible Lucas numbers?</p>
<pre><code class="Swift">
c = 0
var infiniteLucas = AnyIterator{luc(n: c+1)}
</code></pre>
<p>Converting the original finite sequence we had was easy, and now we have a new sequence that does not have any limit to the number of results it can generate, but it’s also easy to understand that we’ll now need a way to limit the number of elements it generates to be able to traverse the sequence using the usual control flow constructs.</p>
<p>And luckily the <code class="inlinecode">Sequence</code> protocol comes to the rescue with one of its methods:</p>
<pre><code class="Swift">
let a4 = Array(infiniteLucas.prefix(10)) //[2, 1, 3, 4, 7, 11, 18, 29, 47, 76]
for var f in infiniteLucas.prefix(10){
print(f)
}
</code></pre>
<p>This way we’ll extract 10 elements from the sequence into a newly created sequence and use it like we were used to from the previous examples.</p>
<p>But let’s go a step further and again apply a <em>filter</em> to our sequence, to obtain a sequence of even Lucas numbers:</p>
<pre><code class="Swift">
var onlyEvenLucas = infiniteLucas.filter({$0 % 2 == 0})
for var f in onlyEvenLucas.prefix(10){
print(f)
}
</code></pre>
<p><em>Well… this will not work as expected.</em></p>
<p>Assuming you are using a playground, you’ll see an error where we declared <code class="inlinecode">onlyEvenLucas</code> that will highlight that a stack overflow happened. If you wrote that in an application, you’ll see your application likely crashing instead.</p>
<p>The reason why this happens is related to how <em>filter</em> works on normal sequences, as you could already know. When we apply a filter to the original sequence the filtering is carried out instantly and all the elements of the sequence consumed eagerly, but without the terminating <em>nil</em> we have no way to specify when this operation should complete.</p>
<p>Let’s see visually what’s happening using a more verbose infinite sequence of integers that will print some text every time a value is requested from the iterator:</p>
<pre><code class="Swift">
class InfiniteSequence :Sequence {
func makeIterator() -> AnyIterator<Int> {
var i = 0
return AnyIterator{
print("# Returning "+String(i))
i += 1
return i
}
}
}
var fs = InfiniteSequence().filter({$0 % 2 == 0}).makeIterator()
for i in 1...5 {
print(fs.next())
}
</code></pre>
<p>If you run this you’ll verify the behavior described above, the filtering on <code class="inlinecode">InfiniteSequence</code> will start consuming the sequence… <em>until it will not be able to proceed anymore a few minutes/hours later.</em></p>
<p>Luckily, obtaining the behavior we expected is again quite easy, we just need to <strong>lazily evaluate</strong> the infinite Lucas sequence:</p>
<pre><code class="Swift">
var onlyEvenLucas = infiniteLucas.lazy.filter({$0 % 2 == 0})
for var f in onlyEvenLucas.prefix(10){
print(f)
}
</code></pre>
<p>Retrieving <code class="inlinecode">.lazy</code> from the original sequence, we’ll get a new <code class="inlinecode">LazySequenceType</code> on which operations such as <strong>map</strong>, <strong>flatMap</strong>, <strong>reduce</strong> or <strong>filter</strong> will be executed lazily and the real evaluation will be performed on demand only when a terminal operation (other languages call them this way) down the chain such as <code class="inlinecode">next</code> or something that needs the whole sequence content will be performed.</p>
<p>Making your infinite sequences lazy is a required step, since Swift sequences are not lazy by default (they were in the first few releases of Swift 1.0). Detailed information about how to implement directly <em>LazySequenceProtocol</em> (that most of the times could be the right approach) are available in the <a href="http://swiftdoc.org/v3.0/type/LazySequence/">documentation</a>, and it’s likely I’ll do a future post on it.</p>
<p><em>Note: Thanks to Rennie for his suggestion of making the samples a bit less concise :)</em></p>
Error Handling: From Objective-C to Swift and Back2015-11-03T00:00:00+01:00https://www.uraimo.com/2015/11/03/error-handling-from-objective-c-to-swift-2-and-backSwift introduces error handling constructs like do,try,catch and try?, in this, I hope, comprehensive article we'll discuss this new feature in detail, how it affects the base frameworks and how Swift modules with the new error handling can be integrated in legacy Objective-C applications.<p><strong>Update 10/16:</strong><em>This post has been updated to Swift 3</em></p>
<h4 id="the-code-for-this-article-is-available-as-a-playground-on-github-or-zipped"><em>The code for this article is available as a playground on <a href="https://github.com/uraimo/Swift-Playgrounds">Github</a> or <a href="https://www.uraimo.com/archives/2015-11-Swift2ErrorHandling.zip">zipped</a>.</em></h4>
<p>Swift introduces error handling constructs like <strong>do/catch</strong> and <strong>try</strong> and its variants. In this article we’ll discuss this new feature, how it affects the base frameworks and how Swift modules that manage errors that way can be integrated in legacy Objective-C applications.</p>
<p><img src="/imgs/error_header.png" alt="trycatch" /></p>
<p>In the first releases of Swift, error handling was performed in a way that essentially mimicked how it has always been performed in Objective-C (the approach recommended by <a href="https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/ErrorHandling/ErrorHandling.html">Apple</a>).</p>
<p>In the good old days of Objective-C, when a method could fail with a recoverable error a <code class="inlinecode">NSError</code> pointer was added as the last parameter of the function, and in case of errors it was used to return a description of what happened.</p>
<p>Unrecoverable errors, that could prevent the application from being able to continue its execution normally, were <em>sometimes</em> handled with exceptions, that Objective-C also supported. Using <code class="inlinecode">NSError</code> for error handling was definitely the favorite approach.</p>
<p>Back to Swift, something like this in Objective-C:</p>
<pre><code class="Objective-C">
NSError *error = nil;
NSArray *imageFiles = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:@"./" error: &error];
</code></pre>
<p>was translated in Swift 1.2 as:</p>
<pre><code class="Swift">
var error: NSError? = nil
let manager = NSFileManager.defaultManager()
var array = manager.contentsOfDirectoryAtPath(path:"./", error: error)
</code></pre>
<p>The <code class="inlinecode">contentsOfDirectoryAtPath</code> function was defined this way:</p>
<pre><code class="Swift">
contentsOfDirectoryAtPath(path: String,error error: NSErrorPointer) -> [AnyObject]?
</code></pre>
<p>And NSErrorPointer was defined as:</p>
<pre><code class="Swift">
typealias NSErrorPointer = AutoreleasingUnsafePointer<NSError?>
</code></pre>
<p>Even without going into the details of the types involved, it’s easy to see that in Swift 1.x there was nothing new about how errors were handled.</p>
<p>Everything changed with Swift 2.0.</p>
<p>Let’s see how error handling works (<a href="https://github.com/uraimo/Swift-Playgrounds">download the playground</a> if you want to play with these examples) before discussing how a Swift component using the new constructs can be integrated in a legacy Objective-C application (hint: it’s not that hard).</p>
<h2 id="error-handling-in-swift">Error Handling in Swift</h2>
<p>The following simple example shows the basic syntax:</p>
<pre><code class="Swift">
enum MyError : Error{
case AnError
case AnotherError
case JustAnotherError
}
func throwsError()throws ->Int {
throw MyError.AnotherError
}
do{
try throwsError()
}catch MyError.AnError {
print("AnError")
}catch MyError.AnotherError {
print("AnotherError") //AnotherError will be catched and printed
}catch{
print("Something else happened")
}
do{
do{
try throwsError()
}catch MyError.AnError {
print("AnError")
}
}catch MyError.AnotherError {
print("AnotherError") //AnotherError will be catched and printed
}catch{
print("Something else happened")
}
</code></pre>
<p>This example creates a custom error defining a new enum that conforms to the <code class="inlinecode">Error</code> protocol, each value will refer to a different condition for this error.</p>
<p>When an error needs to be returned from the function, we simply <em>throw</em> one of the values available in the <code class="inlinecode">MyError</code> enum and the function will complete its execution and return control to the caller, that will handle the error. Notice that the throwing function has to explicitly state that it could, under some circumstances, throw an error specifying the <code class="inlinecode">throws</code> keyword in its declaration.</p>
<p>To perform the actual error handling, the functions that could throw need to be preceded by a <code class="inlinecode">try</code>(or one of its alternatives as we’ll see) and need to be enclosed in a <code class="inlinecode">do/catch</code> block, that defines the context in which the errors will be managed.</p>
<p>Each catch will handle a specific error in its body. In the example above, our function throws always the same error that’s handled by the second catch, resulting in the name of the exception printed to console.</p>
<p>The <em>do/catch</em> does not have to cover all possible error conditions, if no catch block is able to handle an error, the error is simply propagated to the outer scope an so on, until a catch able to manage the error is found.</p>
<p>The nested <em>do/catch</em> of the next snippet shows that in action, the first <em>do/catch</em> handles only <code class="inlinecode">.AnError</code> errors, while the surrounding <em>do/catch</em> is able to handle the remaining alternatives.</p>
<p>But Swift error handling has a lot more to offer, as shown in this more complex example:</p>
<pre><code class="Swift">
enum MyError2 : Error{
case GenericError
case DetailedError(String)
case NumericError(Int)
}
func throwsDetailedError()throws ->Int {
throw MyError2.DetailedError("Some details here")
}
func shouldNeverThrow()throws ->Int {
return 0
}
do{
defer{
//Clean up
}
try throwsDetailedError()
var value = try! shouldNeverThrow()
var imNil = try? throwsDetailedError()
}catch MyError2.GenericError {
print("GenericError")
}catch MyError2.DetailedError(let message) {
print("Error: \(message)") //Will print Error: Some details here
}catch MyError2.NumericError(let number) where number>0{
print("Error with id: "+String(number))
}catch{
print("Something else happened: "+String(describing:error))
}
</code></pre>
<p>There is much more going on in this example, this time our custom error can also have parameters for specific conditions and those parameters can then be bound to a variable in the catch clause. If no error or no binding variable is specified, like in the last catch, the error is automatically bound to an <code class="inlinecode">error</code> variable.</p>
<p>Another interesting thing is the use of <code class="inlinecode">defer</code> (I have placed it inside the <em>do/catch</em> but it doesn’t necessarily need to be there, it could be at the beginning of the current function), that provides the same functionality that is usually provided by a <code class="inlinecode">finally</code> block in Objective-C and other languages. The code contained in defer is guaranteed to be executed, no matter what, and because of this is usually used to perform mandatory clean-up operations.</p>
<p>Maybe more interesting it’s the ability to perform pattern matching as you are used to with switches, the code above contains a very simple example, just to show that it can be done, but everything available for switches is available here.</p>
<p>And then there is the first variation on <code class="inlinecode">try</code>.</p>
<p>Using <code class="inlinecode">try!</code> you are disabling error propagation for the <code class="inlinecode">shouldNeverThrow</code> function and wrapping it in a runtime assertion that will generate a runtime error (and crash your application) if the function throws an error. This allows you to ignore errors and their handling in those situations when you can be completely sure that even if a function is declared to throw, no error will actually ever be thrown.</p>
<p>The second and last variation of try is <code class="inlinecode">try?</code>, that handle errors producing an optional value that will contain the returned value if available or that will be nil in case of an error. You lose information about what kind of error was thrown (it could not be important) but you gain the ability to use the resulting optional in combination with all the statements that support them, from <code class="inlinecode">if let</code> to <code class="inlinecode">[map & flatMap](http://www.uraimo.com/2015/10/08/Swift2-map-flatmap-demystified/)</code>.</p>
<p>This is just an example of what you could do using <a href="http://www.uraimo.com/2015/10/08/Swift2-map-flatmap-demystified/">map/flatMap</a>:</p>
<pre><code class="Swift">
var convertedInt = (try? shouldNeverThrow()).map{String($0)}
convertedInt
</code></pre>
<p>Not really that useful if taken alone, but with a lot of function possibly throwing errors, something like this or simply a wise use of optionals could help, in some cases, turn your code littered by do/catches into something more readable.</p>
<h2 id="base-frameworks-and-error-handling">Base Frameworks and Error Handling</h2>
<p>With the introduction of the new error handling in Swift 2.0, the <code class="inlinecode">contentsOfDirectoryAtPath</code> function and all the functions of the base frameworks that return NSErrors now have a slightly different prototype:</p>
<pre><code class="Swift">
func contentsOfDirectoryAtPath(path: String) throws -> [String]
</code></pre>
<p>Every function that returned errors using an <code class="inlinecode">NSError</code> is now a function that throws but with one less parameter, and what the function throws is no other than the original <code class="inlinecode">NSError</code>. And if you check the documentation on <code class="inlinecode">NSError</code>, you’ll notice that among other protocols it now implements the <code class="inlinecode">Error</code> protocol.</p>
<p>And this alteration of the function prototype is the result of automatic bridging, as described in Apple’s “<a href="https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/BuildingCocoaApps/WorkingWithCocoaDataTypes.html#//apple_ref/doc/uid/TP40014216-CH6-ID61">Using Swift with Cocoa and Objective-C</a>”:</p>
<blockquote>
<p>Swift automatically bridges between the Error type and the NSError class.</p>
</blockquote>
<blockquote>
<p>Objective-C methods that produce errors are imported as Swift methods that throw, and Swift methods that throw are imported as Objective-C methods that produce errors, according to Objective-C error conventions.</p>
</blockquote>
<p>Let’s see what this means for Objective-C projects that need to integrate Swift code.</p>
<h2 id="swift-error-handling-and-legacy-objective-c-applications">Swift Error Handling and Legacy Objective-C Applications</h2>
<p>Suppose that you have an existing Objective-C project and that you plan to migrate gradually to Swift or that you simply want to extend an existing Objective-C application with a new Swift component, essentially turning your project in a mixed language application.</p>
<p>This section will use a simple OSX console application to show how this can be done, to follow along create a new console application or get the <a href="https://github.com/uraimo/Swift-Playgrounds">full sample on Github</a>.</p>
<p>Once the project has been created, to start writing some Swift, simply add a new Swift file to your project, when asked if a bridging header needs to be created click <em>No</em>. With a bridging header you would be able to use Objective-C classes in Swift, in this section we’ll do the opposite.</p>
<p>Paste this code in your newly created Swift file:</p>
<pre><code class="Swift">
import Foundation
@objc enum MyError:Int, Error{
case AnError
case AnotherError
}
public class MyClass:NSObject{
public func throwAnError() throws {
throw MyError.AnotherError
}
public func callMe(){
print("Someone called!")
}
}
</code></pre>
<p>We are going to invoke these two methods from Objective-C, but before that it’s important to know the module name of our project as shown under <em>Product Module Name</em> in <em>Build Settings</em>:</p>
<p><img src="/imgs/modulename.png" alt="product module name" /></p>
<p>To access the Swift code we just wrote from Objective-C we just need to import an auto-generated header with a name that follows the format <em>ModuleProductName-Swift.h</em> (the actual file will be placed in the temporary build directory of your project and regenerated when needed).</p>
<p>Considering that our product name was <em>ErrorHandling</em>, the generated Swift header file will be named <em>ErrorHandling-Swift.h</em>.</p>
<p>The <code class="inlinecode">main.m</code> of your project should look like this:</p>
<pre><code class="Objective-C">
#import <Foundation/Foundation.h>
#import "ErrorHandling-Swift.h"
int main(int argc, const char * argv[]) {
@autoreleasepool {
MyClass* c = [MyClass new];
NSError* err=nil;
[c throwAnErrorAndReturnError:&err];
NSLog(@"Domain:%@ Code:%d Message:%@",err.domain,err.code,err.localizedDescription);
[c callMe];
}
return 0;
}
</code></pre>
<p>In this example, we are simply creating a new instance of <code class="inlinecode">MyClass</code> and then invoking its two methods in sequence. There is not much to see other than the fact that the name of our <code class="inlinecode">throwAnError</code> function is now <code class="inlinecode">throwAnErrorAndReturnError</code>.
This is the result of the automatic bridging process described in the previous section.</p>
<p>The header that has been generated for us has this content:</p>
<pre><code class="Objective-C">
SWIFT_CLASS("_TtC13ErrorHandling7MyClass")
@interface MyClass : NSObject
- (BOOL)throwAnErrorAndReturnError:(NSError * __nullable * __null_unspecified)error;
- (void)callMe;
- (nonnull instancetype)init OBJC_DESIGNATED_INITIALIZER;
@end
typedef SWIFT_ENUM(NSInteger, MyError) {
MyErrorAnError = 0,
MyErrorAnotherError = 1,
};
static NSString * const MyErrorDomain = @"ErrorHandling.MyError";
</code></pre>
<p>As expected, the bridging process has added an NSError parameter to our error throwing function but also appended <em>AndReturnError</em> to its name following a common naming convention.</p>
<p>Also, since we added an <code class="inlinecode">@objc</code> modifier to our enum, but this was an optional step (not required to perform the bridging), we now have a convenient Objective-C enum that can be used in conjunction with the error code contained in the NSError.</p>
<p>Running out program produces the following output:</p>
<pre><code class="Objective-C">
ErrorHandling[8104:413890] Domain:MyError Code:1 Message:The operation couldn’t be completed. (MyError error 1.)
Someone called!
Program ended with exit code: 0
</code></pre>
<p>Even if calling Swift code from Objective-C is not as straightforward as doing the opposite, the process is still painless, as promised at the beginning of the article error handling in a mixed language project it’s not that hard.</p>
Effective Method Swizzling in Swift2015-10-23T00:00:00+02:00https://www.uraimo.com/2015/10/23/effective-method-swizzling-with-swiftDespite the static nature of the language, it's still possible to perform method swizzling (runtime method implementation substitution) in Swift following a few simple rules.<p><strong>Update 11/16:</strong><em>This post and the example project have been updated to Swift 3 with the new dispatch_once syntax.</em></p>
<h4 id="get-the-sample-project-for-this-article-from-github-or-zipped"><em>Get the sample project for this article from <a href="https://github.com/uraimo/SwizzlingInSwift/">GitHub</a> or <a href="https://github.com/uraimo/SwizzlingInSwift/archive/master.zip">zipped</a>.</em></h4>
<p>Method Swizzling is a well known practice in Objective-C and in other languages that support dynamic method dispatching.</p>
<p>Through swizzling, the implementation of a method can be replaced with a different one <em>at runtime</em>, by changing the mapping between a specific #selector(method) and the function that contains its implementation.</p>
<p><img src="/imgs/swizzling.png" srcset="/imgs/swizzling@2x.png 2x" alt="Swizzling diagram" /></p>
<p>While this seems extremely convenient, this functionality does not come without its drawbacks. Performing this sort of alterations at runtime, you can’t take advantage of all the safety checks that are usually available at compile time. Swizzling is something that should be used with care.</p>
<p>The definitive article on how to swizzle in Objective-C is available on <a href="http://nshipster.com/method-swizzling/">NSHipster</a> (and some additional details are <a href="https://www.mikeash.com/pyblog/friday-qa-2010-01-29-method-replacement-for-fun-and-profit.html">here</a>) and a comprehensive discussion on the perils of using method swizzling can be found on <a href="http://stackoverflow.com/q/5339276">Stackoverflow</a>.</p>
<p>Swift takes a static approach regarding method dispatching, but it’s still possible to perform method swizzling if some conditions are met.</p>
<p>Before giving you some pointers on how to use swizzling with Swift, let me reiterate that this technique should be used sparingly, only when a more “swifty” alternative to solve your problem does not exist and not considered as a real alternative to subclassing or to the use of protocols and extensions.</p>
<p>As described in another article on <a href="http://nshipster.com/swift-objc-runtime/">NSHipster</a>, performing swizzling in Swift with a class from one of the base frameworks (Foundation, UIKit, etc…), except for a few gotchas, is not that different from what you were used to in Objective-C:</p>
<pre><code class="Swift">
extension UIViewController {
public override static func initialize() {
// make sure this isn't a subclass
if self !== UIViewController.self {
return
}
struct Inner {
static let i: () = {
let originalSelector = #selector(UIViewController.viewWillAppear(_:))
let swizzledSelector = #selector(UIViewController.newViewWillAppear(_:))
let originalMethod = class_getInstanceMethod(UIViewController.self, originalSelector)
let swizzledMethod = class_getInstanceMethod(UIViewController.self, swizzledSelector)
let didAddMethod = class_addMethod(UIViewController.self, originalSelector, method_getImplementation(swizzledMethod), method_getTypeEncoding(swizzledMethod))
if didAddMethod {
class_replaceMethod(UIViewController.self, swizzledSelector, method_getImplementation(originalMethod), method_getTypeEncoding(originalMethod))
} else {
method_exchangeImplementations(originalMethod, swizzledMethod);
}
}()
}
let _ = Inner.i
}
// MARK: - Method Swizzling
func newViewWillAppear(animated: Bool) {
self.newViewWillAppear(animated)
if let name = self.descriptiveName {
print("viewWillAppear: \(name)")
} else {
print("viewWillAppear: \(self)")
}
}
}
</code></pre>
<p>In this example, additional operations needs to be performed for every UIViewController in the application but the original behavior of the <code class="inlinecode">viewWillAppear</code> method needs to be preserved, this can be done only through swizzling.</p>
<p>The <code class="inlinecode">viewWillAppear</code> method implementation will be replaced with the implementation of a new method named <code class="inlinecode">newViewWillAppear</code> in <code class="inlinecode">initialize</code>. <em>Note that after the swizzling, what in the code seems to be a recursive call to <code class="inlinecode">newViewWillAppear</code> will become a call to the original <code class="inlinecode">viewWillAppear</code> method.</em></p>
<p>The first difference from the recommended Objective-C approach is that the swizzling is not performed in <code class="inlinecode">load</code>.</p>
<p>The <code class="inlinecode">load</code> method is guaranteed to be called when the definition of a class is loaded and this makes it the right place to perform method swizzling.</p>
<p>But <code class="inlinecode">load</code> is a Objective-C only method and cannot be overridden in Swift, trying to do it anyway will result in a compile time error. The next best place to perform the swizzling is in <code class="inlinecode">initialize</code>, a method called right before the first method of your class is invoked.</p>
<p>Enclosing all the operations that modify the methods in the lazy initialization block of a computed global constant ensures that the procedure will be performed only once (since initialization of these variables or constants uses dispatch_once behind the scenes).</p>
<p>And that’s what you need to know for classes from base frameworks or for bridged Objective-C classes. When instead, you plan to use pure Swift classes there are a few additional things you should keep in mind to be able to perform method swizzling correctly.</p>
<h2 id="method-swizzling-with-swift-classes">Method Swizzling with Swift classes</h2>
<p>To use method swizzling with your Swift classes there are two requirements that you must comply with:</p>
<ul>
<li>The class containing the methods to be swizzled must extend NSObject</li>
<li>The methods you want to swizzle must have the dynamic attribute</li>
</ul>
<p>More information about why this is necessary can be found in Apple’s “<a href="https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/BuildingCocoaApps/InteractingWithObjective-CAPIs.html#//apple_ref/doc/uid/TP40014216-CH4-XID_38">Using Swift with Cocoa and Objective-C</a>”:</p>
<blockquote>
<p><strong>Requiring Dynamic Dispatch</strong></p>
<p>While the @objc attribute exposes your Swift API to the Objective-C runtime, it does not guarantee dynamic dispatch of a property, method, subscript, or initializer. <strong>The Swift compiler may still devirtualize or inline member access to optimize the performance of your code, bypassing the Objective-C runtime</strong>. When you mark a member declaration with the dynamic modifier, access to that member is always dynamically dispatched. Because declarations marked with the dynamic modifier are dispatched using the Objective-C runtime, they’re implicitly marked with the @objc attribute.</p>
<p>Requiring dynamic dispatch is rarely necessary. <strong>However, you must use the dynamic modifier when you know that the implementation of an API is replaced at runtime</strong>. For example, you can use the <code class="inlinecode">method_exchangeImplementations</code> function in the Objective-C runtime to swap out the implementation of a method while an app is running. If the Swift compiler inlined the implementation of the method or devirtualized access to it, the new implementation would not be used.</p>
</blockquote>
<p>This also means that you can’t perform swizzling if the method that you want to replace has not been declared as dynamic.</p>
<p>Let’s see how this translates to code:</p>
<pre><code class="Swift">
class TestSwizzling : NSObject {
dynamic func methodOne()->Int{
return 1
}
}
extension TestSwizzling {
//In Objective-C you'd perform the swizzling in load() , but this method is not permitted in Swift
override class func initialize()
{
// Perform this one time only
struct Inner {
static let i: () = {
let originalSelector = #selector(TestSwizzling.methodOne)
let swizzledSelector = #selector(TestSwizzling.methodTwo)
let originalMethod = class_getInstanceMethod(TestSwizzling.self, originalSelector)
let swizzledMethod = class_getInstanceMethod(RestSwizzling.self, swizzledSelector)
method_exchangeImplementations(originalMethod, swizzledMethod)
}()
}
let _ = Inner.i
}
func methodTwo()->Int{
// It will not be a recursive call anymore after the swizzling
return methodTwo()+1
}
}
var c = TestSwizzling()
print(c.methodOne()) //2
print(c.methodTwo()) //1
</code></pre>
<p>In this simplified example the implementations of <code class="inlinecode">methodOne</code> and <code class="inlinecode">methodTwo</code> will be replaced with one another, just before the first method of the <code class="inlinecode">TestSwizzling</code> object is called.</p>
<h3 id="closing-remarks">Closing remarks</h3>
<p>As you have seen, it’s still possible to perform method swizzling in Swift, but in my opinion, most of the times it should never end up in actual production code. What a quick fix using swizzling can solve, can be better solved (for various definition of better) refactoring your code and thinking a better architecture.</p>
Swift 3: Map and FlatMap Demystified2015-10-08T00:00:00+02:00https://www.uraimo.com/2015/10/08/Swift2-map-flatmap-demystifiedSwift is a language still slightly in flux, with new functionalities and alterations of behavior being introduced in every release. Much has already been written about the functional aspects of Swift and how to approach problems following a more functional approach.<br/>This short article will try to give a clear and complete explanation of how <i>map</i> and especially <i>flatMap</i> work for different types in Swift 2.0 and 3.0, with references to the current library headers.<p><strong>Update 12/16:</strong><em>This post has been verified with Swift 3, minimal changes were required.</em></p>
<h4 id="get-this-and-other-playgrounds-from-github-or-zipped"><em>Get this and other playgrounds from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="https://www.uraimo.com/archives/2015-10-MapFlatMapPlayground.playground.zip">zipped</a>.</em></h4>
<p>Swift is a language still slightly in flux, with new functionalities and alterations of behavior being introduced in every release. Much has already been written about the functional aspects of Swift and how to approach problems following a more “pure” functional approach.</p>
<p><img src="/imgs/bind.png" srcset="/imgs/bind@2x.png 2x" alt="Bind" />
<span class="imgcaption">Misterious but correct depiction of the monadic bind</span></p>
<p>Considering that the language is still in its infancy, often, trying to understand some specific topics you’ll end up reading a lot of articles referring to old releases of the language, or worst, descriptions that mix up different releases. Sometimes, searching for articles on <code class="inlinecode">flatMap</code>, you could even fortuitously find more than one <a href="http://khanlou.com/2015/09/what-the-heck-is-a-monad/">really good articles</a> explaining <a href="http://codeplease.io/2015/08/05/monads/">Monads</a> in the context of Swift.</p>
<p>Add to the lack of comprehensive and recent material the fact that many of these concepts, even with examples or daring metaphors, are not obvious, especially for someone used to the imperative way of thinking.</p>
<p>With this short article (part of a series on <a href="http://uraimo.com/category/functional/">Swift and the functional approach</a>) I’ll try to give a clear and throughout explanation of how <code class="inlinecode">map</code> and especially <code class="inlinecode">flatMap</code> work for different types, with references to the current library headers.</p>
<h3 id="contents">Contents</h3>
<ul>
<li><a href="#map">Map</a>
<ul>
<li><a href="#map-on-optionals">Map on Optionals</a></li>
<li><a href="#map-on-sequences">Map on Sequences</a></li>
</ul>
</li>
<li><a href="#flatmap">FlatMap</a>
<ul>
<li><a href="#flatmap-on-optionals">FlatMap on Optionals</a></li>
<li><a href="#flatmap-on-sequences">FlatMap on Sequences</a></li>
</ul>
</li>
</ul>
<h2 id="map">Map</h2>
<p>Map has the more obvious behavior of the two *map functions, it simply performs a closure on the input and, like <code class="inlinecode">flatMap</code>, it can be applied to <a href="http://swiftdoc.org/v3.0/type/Optional/">Optionals</a> and <a href="http://swiftdoc.org/v3.0/protocol/Sequence/">Sequences</a> (i.e. arrays, dictionaries, etc..).</p>
<h3 id="map-on-optionals">Map on Optionals</h3>
<p>For Optionals, the map function has the following prototype:</p>
<pre><code class="Swift">
public enum Optional<Wrapped> : ... {
...
/*
- Parameter transform: A closure that takes the unwrapped value
of the instance.
- Returns: The result of the given closure. If this instance is `nil`,
returns `nil`.
*/
public func map<U>(_ transform: (Wrapped) throws -> U) rethrows -> U?
...
}
</code></pre>
<p>The map function expects a closure with signature <code class="inlinecode">(Wrapped) -> U</code>, if the optional has a value applies the function to the unwrapped optional and then wraps the result in an optional to return it (an additional declaration is present for implicitly unwrapped optionals, but this does not introduce any difference in behavior, just be aware of it when map doesn’t actually return an optional).</p>
<p>Note that the output type can be different from the type of the input, that is likely the most useful feature.</p>
<p>Straightforward, this does not need additional explanations, let’s see some real code from the <a href="https://github.com/uraimo/Swift-Playgrounds">playground</a> for this post:</p>
<pre><code class="Swift">
var o1:Int? = nil
var o1m = o1.map({$0 * 2})
o1m /* Int? with content nil */
o1 = 1
o1m = o1.map({$0 * 2})
o1m /* Int? with content 2 */
var os1m = o1.map({ (value) -> String in
String(value * 2)
})
os1m /* String? with content 2 */
os1m = o1.map({ (value) -> String in
String(value * 2)
}).map({"number "+$0})
os1m /* String? with content "number 2" */
</code></pre>
<p>Using map on optionals could save us an if each time we need to modify the original optional (map applies the closure to the content of the optional only if the optional has a value, otherwise it just returns nil), but the most interesting feature we get for free is the ability to concatenate multiple map operations that will be executed sequentially, thanks to the fact that a call to <code class="inlinecode">map</code> always return an optional. Interesting, but quite similar and more verbose than what we could get with optional chaining.</p>
<h3 id="map-on-sequences">Map on Sequences</h3>
<p>But it’s with <code class="inlinecode">Sequences</code> like arrays and dictionaries that the convenience of using map-like functions is hard to miss:</p>
<pre><code class="Swift">
var a1 = [1,2,3,4,5,6]
var a1m = a1.map({$0 * 2})
a1m /* [Int] with content [2, 4, 6, 8, 10, 12] */
let ao1:[Int?] = [1,2,3,4,5,6]
var ao1m = ao1.map({$0! * 2})
ao1m /* [Int] with content [2, 4, 6, 8, 10, 12] */
var a1ms = a1.map({ (value) -> String in
String(value * 2)
}).map { (stringValue) -> Int? in
Int(stringValue)
}
a1ms /* [Int?] with content [.Some(2),.Some(4),.Some(6),.Some(8),.Some(10),.Some(12)] */
</code></pre>
<p>This time we are calling the .map function defined on <code class="inlinecode">Sequence</code> as follow:</p>
<pre><code class="Swift">
/*
- Parameter transform: A mapping closure. `transform` accepts an
element of this sequence as its parameter and returns a transformed
value of the same or of a different type.
- Returns: An array containing the transformed elements of this
sequence.
*/
func map<T>(_ transform: (Element) throws -> T) rethrows -> [T]
</code></pre>
<p>The transform closure of type <code class="inlinecode">(Element) -> T</code> is applied to every member of the collection and all the results are then packed in an array with the same type used as output in the closure and returned. As we did in the optionals example, sequential operation can be pipelined invoking <code class="inlinecode">map</code> on the result of a previous <code class="inlinecode">map</code> operation.</p>
<p>This basically sums up what you can do with <code class="inlinecode">map</code>, but before moving to <code class="inlinecode">flatMap</code>, let’s see three additional examples:</p>
<pre><code class="Swift">
var s1:String? = "1"
var i1 = s1.map {
Int($0)
}
i1 /* Int?? with content 1 */
var ar1 = ["1","2","3","a"]
var ar1m = ar1.map {
Int($0)
}
ar1m /* [Int?] with content [.Some(1),.Some(2),.Some(3),nil] */
ar1m = ar1.map {
Int($0)
}
.filter({$0 != nil})
.map {$0! * 2}
ar1m /* [Int?] with content [.Some(2),.Some(4),.Some(6)] */
</code></pre>
<p>Not every String can be converted to an Int, so our integer conversion closure will always return an Int?.
What happens in the first example with that Int??, is that we end up with an optional of an optional, for the additional wrapping performed by map. To actually get the contained value will need to unwrap the optional two times, not a big problem, but this starts to get a little inconvenient if we need to chain an additional operation to that map. As we’ll see, <code class="inlinecode">flatMap</code> will help with this.</p>
<p>In the example with the array, if a String cannot be converted as it happens for the 4th element of <code class="inlinecode">ar1</code> the that element in the resulting array will be nil. But again, what if we want to concatenate an additional map operation after this first map and apply the transformation just to the valid (not nil) elements of our array to obtain a shorter array with only numbers?</p>
<p>Well, we’ll just need intermediate filtering to sort out the valid elements and prepare the stream of data to the successive map operations. Wouldn’t it be more convenient if this behavior was embedded in <code class="inlinecode">map</code>?
We’ll see that this another use case for <code class="inlinecode">flatMap</code>.</p>
<h2 id="flatmap">FlatMap</h2>
<p>The differences between <code class="inlinecode">map</code> and <code class="inlinecode">flatMap</code> could appear to be minor but they are definitely not.</p>
<p>While <code class="inlinecode">flatMap</code> is still a map-like operation, it applies an additional step called <code class="inlinecode">flatten</code> right after the mapping phase.
Let’s analyze <code class="inlinecode">flatMap</code>’s behavior with some code like we did in the previous section.</p>
<h3 id="flatmap-on-optionals">FlatMap on Optionals</h3>
<p>The definition of the function is a bit different, but the functionality is similar, as the reworded comment implies:</p>
<pre><code class="Swift">
public enum Optional<Wrapped> : ... {
...
/*
- Parameter transform: A closure that takes the unwrapped value
of the instance.
- Returns: The result of the given closure. If this instance is `nil`,
returns `nil`.
*/
public func flatMap<U>(_ transform: (Wrapped) throws -> U?) rethrows -> U?
...
}
</code></pre>
<p>There is a substantial difference regarding the closure, <code class="inlinecode">flatMap</code> expects a <code class="inlinecode">(Wrapped) -> U?)</code> this time.</p>
<p>With optionals, flatMap applies the closure returning an optional to the content of the input optional and after the result has been “flattened” it’s wrapped in another optional.</p>
<p>Essentially, compared to what <code class="inlinecode">map</code> did, <code class="inlinecode">flatMap</code> also unwraps one layer of optionals.</p>
<pre><code class="Swift">
var fo1:Int? = nil
var fo1m = fo1.flatMap({$0 * 2})
fo1m /* Int? with content nil */
fo1 = 1
fo1m = fo1.flatMap({$0 * 2})
fo1m /* Int? with content 2 */
var fos1m = fo1.flatMap({ (value) -> String? in
String(value * 2)
})
fos1m /* String? with content "2" */
var fs1:String? = "1"
var fi1 = fs1.flatMap {
Int($0)
}
fi1 /* Int? with content "1" */
var fi2 = fs1.flatMap {
Int($0)
}.map {$0*2}
fi2 /* Int? with content "2" */
</code></pre>
<p>The last snippet contains and example of chaining, no additional unwrapping is needed using <code class="inlinecode">flatMap</code>.</p>
<p>As we’ll see again when we describe the behavior with Sequences, this is the result of applying the flattening step.</p>
<p>The <code class="inlinecode">flatten</code> operation has the sole function of “unboxing” nested containers. A container can be an array, an optional or any other type capable of containing a value with a container type. Think of an optional containing another optional as we’ve just seen or array containing other array as we’ll see in the next section.</p>
<p>This behavior adheres to what happens with the <code class="inlinecode">bind</code> operation on Monads, to learn more about them, read <a href="http://khanlou.com/2015/09/what-the-heck-is-a-monad/">here</a> and <a href="http://robnapier.net/flatmap">here</a>.</p>
<h3 id="flatmap-on-sequences">FlatMap on Sequences</h3>
<p>Sequence provides the following implementations of <code class="inlinecode">flatMap</code>:</p>
<pre><code class="Swift">
/// - Parameter transform: A closure that accepts an element of this
/// sequence as its argument and returns a sequence or collection.
/// - Returns: The resulting flattened array.
///
public func flatMap<SegmentOfResult : Sequence>(_ transform: (Element) throws -&gt: SegmentOfResult) rethrows -> [SegmentOfResult.Iterator.Element]
/// - Parameter transform: A closure that accepts an element of this
/// sequence as its argument and returns an optional value.
/// - Returns: An array of the non-`nil` results of calling `transform`
/// with each element of the sequence.
///
public func flatMap<ElementOfResult>(_ transform: (Element) throws -> ElementOfResult?) rethrows -> [ElementOfResult]
</code></pre>
<p><code class="inlinecode">flatMap</code> applies those transform closures to each element of the sequence and then pack them in a new array with the same type of the input value.</p>
<p>These two comments blocks describe two functionalities of <code class="inlinecode">flatMap</code>: sequence flattening and nil optionals filtering.</p>
<p>Let’s see what this means:</p>
<pre><code class="Swift">
var fa1 = [1,2,3,4,5,6]
var fa1m = fa1.flatMap({$0 * 2})
fa1m /*[Int] with content [2, 4, 6, 8, 10, 12] */
var fao1:[Int?] = [1,2,3,4,nil,6]
var fao1m = fao1.flatMap({$0})
fao1m /*[Int] with content [1, 2, 3, 4, 6] */
var fa2 = [[1,2],[3],[4,5,6]]
var fa2m = fa2.flatMap({$0})
fa2m /*[Int] with content [1, 2, 3, 4, 6] */
</code></pre>
<p>While the result of the first example doesn’t differ from what we obtained using <code class="inlinecode">map</code>, it’s clear that the next two snippets show something that could have useful practical uses, saving us the need for convoluted manual flattening or filtering.</p>
<p>In the real world, there will be many instances where using <code class="inlinecode">flatMap</code> will make your code way more readable and less error-prone.</p>
<p>And an example of all this is the last snippet from the previous section, that we can now improve with the use of <code class="inlinecode">flatMap</code>:</p>
<pre><code class="Swift">
var far1 = ["1","2","3","a"]
var far1m = far1.flatMap {
Int($0)
}
far1m /* [Int] with content [1, 2, 3] */
far1m = far1.flatMap {
Int($0)
}
.map {$0 * 2}
far1m /* [Int] with content [2, 4, 6] */
</code></pre>
<p>I may look just a minimal improvement in this context, but with longer chain it would become something that greatly improves readability.</p>
<p>And let me reiterate this again, in this context too, the behavior of swift flatMap is aligned to the <code class="inlinecode">bind</code> operation on Monads (and “flatMap” is usually used as a synonym of “bind”), you can learn more about this reading <a href="http://khanlou.com/2015/09/what-the-heck-is-a-monad/">here</a> and <a href="http://robnapier.net/flatmap">here</a>.</p>
<p>Learn more about Sequence and IteratorProtocol protocols in the <a href="http://www.uraimo.com/2015/11/12/experimenting-with-swift-2-sequencetype-generatortype/">next article</a> in the series.</p>
<h6>Drawing inspired by <a href="https://github.com/VincentToups/emacs-utils/blob/master/monads.md">emacs-utils</a> documentation.</h6>
Swift 2.1 Function Types Conversion: Covariance and Contravariance2015-09-29T00:00:00+02:00https://www.uraimo.com/2015/09/29/Swift2.1-Function-Types-Conversion-Covariance-ContravarianceWith Swift 2.1 function types will support type conversion, discover covariance and contravariance and what this means for you<p><strong>Update 10/16:</strong><em>This post has been updated for Swift 3, click <a href="https://github.com/uraimo/Swift-Playgrounds/commit/c7436ecb90c50eaa6685be50d953e8c6a84b2097#diff-893181bb364e4fc6f00f690be4bbac97">here</a> to see what changed in the code samples.</em></p>
<h4 id="this-swift-21-post-requires-xcode71-beta-or-later-get-this-and-other-playgrounds-from-github-or-zipped"><em>This Swift 2.1 post requires Xcode7.1 beta or later, get this and other playgrounds from <a href="https://github.com/uraimo/Swift-Playgrounds">GitHub</a> or <a href="https://www.uraimo.com/archives/2015-09-VariancePlayground.playground.zip">zipped</a>.</em></h4>
<p>In Swift 2.1, coming with XCode 7.1 (see the <a href="http://adcdownload.apple.com/Developer_Tools/Xcode_7.1_beta_2/Xcode_7.1_beta_2_Release_Notes.pdf">change log</a>), function types will support covariance and contravariance, let’s see why this will matter.</p>
<p>In the context of computer science and types, the term variance refers to how the relationship between two types influences the relationship between the complex types that have been derived from them. This relationship between complex types is the result of either invariance, covariance and contravariance applied to the original types. Understanding how this derived relationship is defined is essential to effectively use complex types.</p>
<p>To clarify this, using pseudocode, let’s consider a complex parametric type <code class="inlinecode">List<T></code> and two other simple types: <code class="inlinecode">Car</code> and <code class="inlinecode">Maserati</code>, a subtype of <code class="inlinecode">Car</code>.</p>
<p>Invariance, Covariance and contravariance can be explained as follow considering the relationship that could bind the two types obtained choosing a specific <code class="inlinecode">T</code> for <code class="inlinecode">List</code>:</p>
<ul>
<li>
<p><strong>Covariance</strong>: If <code class="inlinecode">List<Maserati></code> is also a subtype of <code class="inlinecode">List<Car></code>, then the type relationship between the original types is preserved on <code class="inlinecode">List</code> because <code class="inlinecode">List</code> is <em>covariant</em> on his original type.</p>
</li>
<li>
<p><strong>Contravariance</strong>: If instead, <code class="inlinecode">List<Car></code> is a subtype of <code class="inlinecode">List<Maserati></code>, then the type relationship between the original types is reversed on <code class="inlinecode">List</code> because <code class="inlinecode">List</code> is <em>contravariant</em> on his original type.</p>
</li>
<li>
<p><strong>Invariance</strong>: <code class="inlinecode">List<Car></code> is not subtype of <code class="inlinecode">List<Maserati></code> and neither the opposite, the two complex types have no derived relationship.</p>
</li>
</ul>
<p>Each language employs a specific set of variance approaches, knowing how complex types are related to each other helps understanding if two complex types are compatible and could be use interchangeably in some situations, in the same way a type and its subtypes do.</p>
<p>In the context of function types, and this is what changes with Swift 2.1, complex type compatibility boils down to one simple question, when it’s safe to use an alternative function of a type <em>B</em> where a function of type <em>A</em> was expected?</p>
<p>The general rule is that a compatible function type can have more generic parent types for parameters (the client of <em>A</em> function will be able to manage a more specialized parameter than what <em>A</em> declared) and can return as result a more specific subtype (the clients of the <em>A</em> function will treat the result as the simpler parent type declared in <em>A</em>). Contravariance applied to parameters and Covariance to what is returned.</p>
<p>Before Swift 2.1, function types behaved in an invariant way, if you try to run the code below in a playground, you will get a few variations of this error: <code class="inlinecode">Cannot convert value of type '(Int)->Int' to expected argument type '(Int) -> Any'</code>.</p>
<pre><code class="Swift">
func testVariance(foo:(Int)->Any){foo(1)}
func innerAnyInt(p1:Any) -> Int{ return 1 }
func innerAnyAny(p1:Any) -> Any{ return 1 }
func innerIntInt(p1:Int) -> Int{ return 1 }
func innerIntAny(p1:Int) -> Any{ return 1 }
testVariance(p1:innerIntAny)
testVariance(p1:innerAnyInt)
testVariance(p1:innerAnyAny)
testVariance(p1:innerIntInt)
</code></pre>
<p>With Swift 2.1, this changes, function type conversion is supported and function types are now <strong>contravariant regarding parameter types</strong> and <strong>covariant regarding the result type</strong>.</p>
<p>Back to the code sample, is now legal to pass all the three <code class="inlinecode">Any->Any</code>, <code class="inlinecode">Any->Int</code>, <code class="inlinecode">Int->Int</code> functions to the <code class="inlinecode">testVariance</code> function with a <code class="inlinecode">Int->Any</code> parameter of the example above.</p>
Swift 2.1 Released2015-09-20T00:00:00+02:00https://www.uraimo.com/2015/09/20/Swift-2.1-releasedThe release 2.1 of Swift is now available<p>With the release of XCode 7.1, Swift 2.1 is now available.</p>
<p>It’s not a major release so don’t expect major changes to the language, here is a list of what changed:</p>
<ul>
<li>
<p>Expressions interpolated in strings may now contain string literals. For example, <code class="inlinecode">"My name is \(attributes["name"]!)"</code> is now a valid expression. <em>Highly appreciated</em>.</p>
</li>
<li>
<p>Conversions between function types are supported, exhibiting covariance in function result types and contravariance in function parameter types (See the specific post on this: <a href="http://www.uraimo.com/2015/09/29/Swift2.1-Function-Types-Conversion-Covariance-Contravariance/">Swift 2.1 Function Types Conversion</a>).</p>
</li>
<li>
<p>Enums imported from C now automatically conform to the <code class="inlinecode">Equatable</code> protocol, including a default implementation of the == operator. This conformance allows you to use C enum pattern matching in switch statements with no additional code.</p>
</li>
</ul>
<p>And some less evident changes:</p>
<ul>
<li>
<p>The <code class="inlinecode">NSNumberunsignedIntegerValue</code> property now has the type <code class="inlinecode">UInt</code> instead of <code class="inlinecode">Int</code>, as do other methods and properties that use the <code class="inlinecode">NSUInteger</code> type in Objective-C and whose names contain “unsigned..”. Most other uses of <code class="inlinecode">NSUInteger</code> in system frameworks are imported as <code class="inlinecode">Int</code> as they were in Xcode 7.</p>
</li>
<li>
<p>Field getters and setters are now created for named unions imported from C. In addition, an initializer with a named parameter for the field is provided.</p>
<p>For example, given the following Objective-C typdef:</p>
<pre><code class="Swift">
typedef union IntOrFloat {
int intField;
float floatField;
} IntOrFloat;
</code></pre>
<p>Importing this typedef into Swift generates the following interface:</p>
<pre><code class="Swift">
struct IntOrFloat {
var intField: Int { get set }
init(intField: Int)
var floatField: Float { get set }
init(floatField: Float)
}
</code></pre>
</li>
<li>
<p>Bitfield members of C structs are now imported into Swift.</p>
</li>
<li>
<p>The type <code class="inlinecode">dispatch_block_t</code> now refers to the type <code class="inlinecode">@convention(block) () -> Void</code>, as it did in Swift 1.2.</p>
<p>This change allows programs using <code class="inlinecode">dispatch_block_create</code> to work as expected, solving an issue that surfaced in Xcode 7.0 with Swift 2.0.</p>
</li>
<li>
<p>Editing a file does not trigger a recompile of files that depend upon it if the edits only modify declarations marked private.</p>
</li>
<li>
<p>Error messages produced when the type checker cannot solve its constraint system continue to improve in many cases.</p>
<p>For example, errors in the body of generic closures (for instance, the argument closure to map) are much more usefully diagnosed.</p>
</li>
</ul>
A Brief iOS9 UIStackView Guide2015-09-08T00:00:00+02:00https://www.uraimo.com/2015/09/08/ios9-uistackview-guide-swiftHow to use the new iOS9 UIStackViews, all you need to know with a practical example project<h4 id="this-swift-20-guide-requires-xcode7-beta-or-later-get-the-complete-project-from-github-or-zipped"><em>This Swift 2.0 Guide requires Xcode7 beta or later, get the complete project from <a href="https://github.com/uraimo/uistackview-sample">GitHub</a> or <a href="https://github.com/uraimo/uistackview-sample/archive/master.zip">zipped</a>.</em></h4>
<p>iOS9 introduces UIStackViews, a new component that greatly simplifies building layouts that can be broken down to vertical or horizontal sequences of UIViews, providing an alternative to manually positioning views using auto-layout.</p>
<p>Acting as an invisible container, each UIStackView is able to display a single sequence of subviews (<em>arranged views</em>) aligned either vertically or horizontally, automatically resizing its content according to the current screen size and adapting to changes in orientation. How these subviews are actually positioned depends on a few properties that define how the subviews should be aligned, spaced and, if needed, resized.</p>
<p>What happens under the hood is that the UIStackView class manages auto-layout constraints for you.
Think of UIStackView as an abstraction layer above auto-layout that simplifies the creation of a well defined subset of layouts. You can start building your layout from a main UIStackView and add nested UIStackView until all your UIViews are positioned correctly.</p>
<p>If you have done any Android development, you’ll notice that the UIStackView concept is quite similar to LinearLayouts, likely the most used android layout scheme, that in turn borrowed some ideas and improved upon the multitude of layouts that were already available in Java Swing.</p>
<h2 id="the-basics">The Basics</h2>
<p>As usual, UIStackViews can be created both programmatically and in Interface Builder.</p>
<p>In Interface Builder you can add a new vertically or horizontally aligned UIStackView choosing the right control from the Object Library and once the view is in place, new views can be added dragging controls inside the UIStackView.</p>
<p>A new UIStackView can also be wrapped around one of more existing views, just select them and click the new <em>Stack icon</em> <img src="/imgs/uistackview00.png" style="display:inline;margin:0;" /> you’ll find in the bottom bar of Interface Builder.</p>
<p>Quite simple, but in this guide we’ll create a basic nested layout programmatically.</p>
<p><img src="/imgs/uistackview02.gif" alt="UIStackView example" /></p>
<p>In this brief example, a vertical UIStackView, placed right below the status bar, will contain four controls: two UILabels, one horizontal UIStackView and one UIButton. Three buttons with default icons will be placed inside an inner horizontal UIStackView.</p>
<p>Let’s start, create a new <em>Single View Application</em>, verifying that the selected <em>Deployment Target</em> is 9.0+.</p>
<p>Open your only <code class="inlinecode">ViewController</code> and replace the <code class="inlinecode">viewDidLoad</code> method with this one:</p>
<pre><code class="Swift">
var stackView:UIStackView!
var nestedStackView=UIStackView()
override func viewDidLoad() {
super.viewDidLoad()
stackView.translatesAutoresizingMaskIntoConstraints=false
self.view.addSubview(stackView)
// Main UIStackView contraints, nearly fills its parent view
self.view.addConstraints(NSLayoutConstraint.constraintsWithVisualFormat("V:|-30-[stackView]-30-|",
options: NSLayoutFormatOptions.AlignAllLeading,metrics: nil, views: ["stackView":stackView]))
self.view.addConstraints(NSLayoutConstraint.constraintsWithVisualFormat("H:|-10-[stackView]-10-|",
options: NSLayoutFormatOptions.AlignAllLeading,metrics: nil, views: ["stackView":stackView]))
stackView.axis = .Vertical
stackView.alignment = .Fill
stackView.spacing = 25
stackView.distribution = .FillEqually
var lbl = UILabel()
lbl.text="Label 1"
lbl.backgroundColor = UIColor.redColor()
stackView.addArrangedSubview(lbl)
lbl = UILabel()
lbl.text="Label 2"
lbl.backgroundColor = UIColor.greenColor()
stackView.addArrangedSubview(lbl)
nestedStackView.axis = .Horizontal
nestedStackView.alignment = .Fill
nestedStackView.spacing = 25
nestedStackView.distribution = .FillEqually
nestedStackView.addArrangedSubview(UIButton(type: .InfoDark))
nestedStackView.addArrangedSubview(UIButton(type: .InfoLight))
nestedStackView.addArrangedSubview(UIButton(type: .ContactAdd))
stackView.addArrangedSubview(nestedStackView)
let btn=UIButton(type: .System)
btn.setTitle("Press Me", forState: .Normal)
stackView.addArrangedSubview(btn)
}
</code></pre>
<p>To specify a vertical orientation for the main UISTackView we are setting the <code class="inlinecode">axis</code> property to <code class="inlinecode">.Vertical</code>, the first three controls will be equally spaced and the UIButton will fill the rest of the available space. The three buttons with default styles contained in the inner <code class="inlinecode">nestedStackView</code> will be arranged in a similar fashion. The alignment, distribution and spacing properties will be explained in the next section, ignore them for now.</p>
<p>Sooner or later you will need to hide and show some of the arranged views and this is quite straightforward for UIStackViews, just set the <code class="inlinecode">hidden</code> property of one of your views.</p>
<p>To test this, let’s add an action to the UIButton and a new <code class="inlinecode">pressedMe</code> method as follow:</p>
<pre><code class="Swift">
...
btn.setTitle("Press Me", forState: .Normal)
btn.addTarget(self, action: "pressedMe:", forControlEvents: UIControlEvents.TouchUpInside)
stackView.addArrangedSubview(btn)
}
func pressedMe(sender: UIButton!){
UIView.animateWithDuration(0.5) {
self.nestedStackView.hidden = !self.nestedStackView.hidden
}
}
</code></pre>
<p>Clicking the button will now hide or show with a short animation the inner UIStackView and the main UIStackView will reposition the remaining views according to the properties specified in <code class="inlinecode">viewDidLoad</code>.</p>
<p>If needed, subviews can also be completely removed from the UIStackView and all the contained arranged views will be, again, repositioned according to the current properties.</p>
<pre><code class="Swift">
func pressedMe(sender: UIButton!){
stackView.removeArrangedSubview(nestedStackView)
nestedStackView.removeFromSuperview()
}
</code></pre>
<p>Removal is a two step process, calling the <code class="inlinecode">removeArrangedSubview</code> method will remove the view from the UIStackView and reposition the remaining subviews but <strong>will not remove</strong> the view from its superview.
The removed views need to be completely removed from their super views too, or they will still be shown outside the UIStackView. To do this, simply invoke <code class="inlinecode">removeFromSuperview</code> on the removed view.</p>
<h2 id="uistackview-alignment-distribution-and-spacing">UIStackView: Alignment, Distribution And Spacing</h2>
<p>Let’s take a look at the positioning properties UIStackView exposes:</p>
<p><img src="/imgs/uistackview01.png" alt="UIStackView properties" /></p>
<h3 id="axis">Axis</h3>
<p>Defines along which axis your views will be positioned, has two possible values: <em>Vertical, Horizontal</em>.</p>
<h3 id="alignment">Alignment</h3>
<p>The alignment property specifies the perpendicular (to the selected axis) alignment for your views, the value <em>Fill</em> will also resize all your views to fill the available space, the other values will not modify your views size. Available values are: <em>Fill, Leading, Top, FirstBaseline, Center, Trailing, Bottom, LastBaseline</em>.</p>
<h3 id="distribution">Distribution</h3>
<p>Distribution specifies how the subviews should be resized or distributed to fill <em>all</em> the available space along the axis, the possible values can be divided in two groups: fill and spacing values.</p>
<p><em>Fill</em> values modify the size of the subviews if they don’t fill (or fit in) all the available space. The spacing between the subviews will be the one specified with the <em>spacing</em> property.</p>
<ul>
<li><em>Fill</em>: The subviews will be shrinked or stretched according to their content content resistance or hugging priority. If you didn’t set any, one of the subviews will be modified to fill the space available.</li>
<li><em>FillEqually</em>: Disregarding any constraint, the subviews will be resized to the same size along the axis.</li>
<li><em>FillProportionally</em>: The subviews will be proportionally resized according to the original size of each subview.</li>
</ul>
<p><em>Spacing</em> values fill the space along the axis altering the spacing between the subviews, the size of the subviews will be modified, according to the compression resistance, only if the subviews still don’t fit or if any auto-layout ambiguity arises.</p>
<ul>
<li><em>EqualSpacing</em>: The subviews will be equally spaced</li>
<li><em>EqualCentering</em>: The subviews center axis will be equally spaced</li>
</ul>
<h3 id="spacing">Spacing</h3>
<p>The spacing property is expressed in points and its meaning depends on the current <em>distribution</em> value.</p>
<p>If the UIStackView <em>distribution</em> property is either EqualSpacing or EqualCentering, the spacing property will represent the minimum spacing among subviews. Alternatively, if a FillProportionally <em>distribution</em> was selected, the requested spacing value will be exactly the chosen value.</p>
<h2 id="uistackviews-ios7-backports">UIStackViews: iOS7+ Backports</h2>
<p>UIStackViews are supported only from iOS9 onward but a few backports, that partially implements this feature in iOS7 or later, have already been built:</p>
<ul>
<li><a href="https://github.com/oarrabi/OAStackView">OAStackView</a></li>
<li><a href="https://github.com/tomvanzummeren/TZStackView">TZStackView</a></li>
</ul>