How async/await works internally in Swift

Introducing this pattern in Swift specifically would not only improve the experience of working with completion handlers but also technically allow the compiler to detect and prevent common concurrency mistakes. It’s easy to see why Chris Lattner made the pattern the central piece of his manifesto, in which he declared, in his own words: “I suggest that we do the obvious thing and support this in Swift.”

Over the years these features were gradually integrated into Swift, culminating in the Swift 5.5 release and the “official” release of async/await in Swift.

Async/await under the hood

Now that we understand why async/await became a part of Swift, we’re ready to take a look at how it works under the hood!

But first, I have to set some expectations with you. Because what we refer to as “async/await” is actually multiple different compiler features working in unison, and because each of these features is complicated enough to warrant their own separate article(s), there’s no way I can possibly cover every single detail of how it works in just one article. I would lose my sanity within the first section if I did that.

So instead of doing that, I’ve decided that a good plan would be to cover only what I believe to be async/await’s “core” functionality and to leave the remaining bits for future articles. But although we don’t go into details about those other bits here, I still made sure to mention some of them in the appropriate sections for you to know where they come into play. For simplicity, we’re also not going to cover compatibility modes and Obj-C bridging here.

With that said, let’s get started! When I reverse-engineer something to learn more about it, I always start from the bottom and move my way up. So when I decided that I wanted to understand how async/await works, my first question was: “Who is managing the program’s background threads?”

The Cooperative Thread Pool

One of the most important aspects of how async/await works in Swift, and one that we must cover before anything else, is that while async/await technically uses GCD under the hood, it does not use the DispatchQueues that we are familiar with. Instead, what powers async/await is a completely new feature of libdispatch called The Cooperative Thread Pool.

Unlike traditional DispatchQueues, which creates and terminates threads dynamically as it deems necessary, the Cooperative Thread Pool manages a fixed number of threads that are constantly helping each other with their tasks.

The “fixed number”, which in this case is equal to the system’s number of CPU cores, is an intentional move that aims to prevent thread explosion and improve system performance in general, something which DispatchQueues were notoriously not very good at.

In other words, the cooperative thread pool is similar to traditional GCD from an interface perspective (it’s a service that receives a job and arranges some thread to run it), but is more efficient and designed to better suit the Swift runtime’s special needs.

We can see exactly how the Cooperative Thread Pool works by exploring the open-source libdispatch repo, but I would like to reserve that for a future article. In general, I find that this WWDC session from 2021 provides great information about how the pool works internally.

Gotcha: Starving threads in the pool

It must be noted that the fact that the pool holds a fixed number of threads has a significant impact on how asynchronous code is supposed to be written in the language. There is now an expectation that threads should always make forward progress, which means you need to be really careful with when and how you do expensive operations in async/await in order to avoid starving the system’s threads. Swift’s async/await has many gotchas like this, and we’ll uncover some of them as we proceed.

Let’s move up the abstraction layers. How does the compiler “speak” to the pool?

Keep the jobs flowing: The Executors

In Swift, you don’t interact with the Cooperative Thread Pool directly. This is hidden by several layers of abstractions, and at the lowest of these layers, we can find the executors.

Executors, just like the pool itself, are services that accept jobs and arrange for some thread to run them. The core difference between them is that while the pool is just, well, the pool, executors come in many shapes and forms. They all end up forwarding jobs to the pool, but the way they do so may change depending on which type of executor you’re using. As of writing, executors can be either concurrent (jobs can run in parallel) or serial (one at a time), and the compiler provides built-in implementations for both of them.

The Built-in Concurrent Executor

The built-in concurrent executor is referred to internally as the Global Concurrent Executor.

The implementation of this executor in the Swift compiler is for the most part nothing more than an abstraction on top of the cooperative thread pool that we mentioned in the previous section. It starts by creating an instance of the new thread pool, which we can see is done by calling the good ol’ GCD API with a special new flag:

constexpr size_t dispatchQueueCooperativeFlag = 4;
queue = dispatch_get_global_queue((dispatch_qos_class_t)priority,
                                  dispatchQueueCooperativeFlag);

Then, when the executor is asked to run a job, it forwards it to the pool via a special dispatch_async_swift_job API:

JobPriority priority = job->getPriority();
auto queue = getGlobalQueue(priority);
  dispatch_async_swift_job(queue, job, (dispatch_qos_class_t)priority,
                  DISPATCH_QUEUE_GLOBAL_EXECUTOR);

I would like to leave the details of libdispatch and dispatch_async_swift_job for another time, but as mentioned in the previous section, this is supposed to be a special/more efficient variant of the regular dispatch_async API that iOS developers are familiar with that better suits the Swift runtime’s special needs.

Another aspect of this executor worth mentioning is that it’s “global”, meaning there is only one instance of it for the entire program. The reasoning for this is similar to why a serial DispatchQueue would deep-down forward its jobs to the global ones: while from a systems perspective, it makes sense for the responsibilities to appear to be divided, from a performance perspective it would be a nightmare for each component to have their own dedicated threads. It’s sensible then to have a single, global executor that will ultimately schedule most of the work in the system, and have everyone else forward their jobs to it.

The Global Concurrent Executor is Swift’s default executor in general. If your async code is not explicitly requesting that it should go through a specific executor (we will see some examples of that as we continue to explore the abstractions), this is the executor that will handle it.

(Swift uses a different global executor in platforms that don’t support libdispatch, but I will not go into details of that as the primary focus of this article is iOS development.)

The Built-in Serial Executor

Unlike the concurrent executor, the purpose of the serial executor is to make sure jobs are executed one by one and in the order in which they were submitted.

The built-in serial executor is referred internally to as the “Default Actor” (spoiler alert!), and it is in its essence an abstraction of the concurrent executor that keeps track of a linked list of jobs:

class DefaultActorImpl : public HeapObject {
public:
  void initialize();
  void destroy();
  void enqueue(Job *job);
  bool tryAssumeThread(RunningJobInfo runner);
  void giveUpThread(RunningJobInfo runner);
}

struct alignas(2 * sizeof(void*)) State {
  JobRef FirstJob;
  struct Flags Flags;
};

enum class Status {
  Idle,
  Scheduled,
  Running,
};

swift::atomic<State> CurrentState;

static void setNextJobInQueue(Job *job, JobRef next) {
  *reinterpret_cast<JobRef*>(job->SchedulerPrivate) = next;
}

if (priority > oldState.getMaxPriority()) {
  newState = newState.withEscalatedPriority(priority);
}

public protocol Executor: AnyObject, Sendable {
    func enqueue(_ job: consuming Job)
}

public protocol SerialExecutor: Executor { ... }

func example() async {
    let fooResult = await foo()
    let barResult = await bar()
    doSomething(fooResult, barResult)
}

func foo() async -> FooResult {
    // Some async code
}

func bar() async -> BarResult {
    // Some async code
}

func foo() async throws {
    // …
    throw MyError.failed // Can’t do this without async/await!
}

// example()
sil hidden @$s4test7exampleyyYaF : $@convention(thin) @async () -> () {
bb0:
  hop_to_executor foo
  foo()
  hop_to_executor example
  hop_to_executor bar
  bar()
  hop_to_executor example
  return void
} // end sil function '$s4test7exampleyyYaF'

func example() async {
    (original executor)
    let fooResult = await foo() // POTENTIAL job 1 (go to foo’s executor)
    // POTENTIAL job 2 (back to original context)
    let barResult = await bar()  // POTENTIAL job 3 (go to bar’s executor)
    // POTENTIAL job 4 (back to original context)
    doSomething(fooResult, barResult)
}

protocol MyProto {
    func asyncFunction() async
}

struct MyType: MyProto {
    func asyncFunction() {
        // This is not an async function, but the Swift is fine with it
        // because `async` and `await` doesn’t mean that the
        // function is _actually_ async, only that it _may_ be.
    }
}

func example() async {
    doSomething() // Running in thread A
    await somethingElse()
    doSomethingAgain() // This COULD also be running in thread A, but it’s probably not!
    // Also, thread A has likely moved on to do other things while we were waiting for somethingElse()!
}

Task {
    await foo()
}

extension Task where Failure == Never {
  public init(
    priority: TaskPriority? = nil,
    @_inheritActorContext @_implicitSelfCapture operation: __owned @Sendable @escaping () async -> Success
  ) {
    let flags = taskCreateFlags(
      priority: priority, isChildTask: false, copyTaskLocals: true,
      inheritContext: true, enqueueJob: true,
      addPendingGroupTaskUnconditionally: false,
      isDiscardingTask: false)

    let (task, _) = Builtin.createAsyncTask(flags, operation)
    self._task = task
  }
}

static AsyncTaskAndContext swift_task_create_commonImpl(…) {
  // The actual function is a lot more complicated than this.
  // This is just a pseudo-coded simplification for learning purposes.

  task.executor = task.parent.executor ?? globalConcurrentExecutor;

  task.checkIfItsChildTask(flags);
  task.checkIfItsTaskGroup(flags);
  task.inheritPriorityFromParentIfNeeded(flags);

  task.asJob.submitToExecutor();
}

func synchronous() {
  withUnsafeCurrentTask { maybeUnsafeCurrentTask in 
    if let unsafeCurrentTask = maybeUnsafeCurrentTask {
      print("Seems I was invoked as part of a Task!")
    } else {
      print("Not part of a task.")
    }
  }
}

func example() async {
  Task {
    // This is NOT a parentless task, as much as it looks like one!
  }
}

actor MyExample {
    var fooInt = 0
}

func foo() {
    let example = MyExample()
    example.fooInt = 1 // Error: Actor-isolated `fooInt`
    // cannot be mutated from a non-isolated context
}

actor MyExample {
    var fooInt = 0
    func mutateFooInt() {
        fooInt = 1
    }
}

func foo() {
    let example = MyExample()
    Task {
        await example.mutateFooInt()
        // The actor itself is allowed to call mutateFooInt() synchronously,
        // but the example() function is not.
    }
}

public protocol Actor: AnyObject, Sendable {
    nonisolated var unownedExecutor: UnownedSerialExecutor { get }
}

public struct UnownedSerialExecutor: Sendable {
    internal var executor: Builtin.Executor
    public init<E: SerialExecutor>(ordinary executor: __shared E) {
      self.executor = Builtin.buildOrdinarySerialExecutorRef(executor)
    }
}

// What you write:
actor MyActor {}

// What is compiled:
final class MyActor: Actor {

    nonisolated var unownedExecutor: UnownedSerialExecutor {
        return Builtin.buildDefaultActorExecutorRef(self)
    }

    init() {
        _defaultActorInitialize(self)
    }

    deinit {
        _defaultActorDestroy(self)
    }

}

public protocol Sendable { }

actor BankAccount {
    nonisolated let accountNumber: Int
}

@globalActor public final actor MainActor: GlobalActor {
  public static let shared = MainActor()

  public nonisolated var unownedExecutor: UnownedSerialExecutor {
    return UnownedSerialExecutor(Builtin.buildMainActorExecutorRef())
  }

  public nonisolated func enqueue(_ job: UnownedJob) {
    _enqueueOnMain(job)
  }
}

// The function where “regular” async/await jobs ends up in
static void swift_task_enqueueGlobalImpl(Job *job) {
  auto queue = getCooperativeThreadPool();
  dispatchEnqueue(queue, job);
}

// The function where MainActor jobs ends up in
static void swift_task_enqueueMainExecutorImpl(Job *job) {
  auto mainQueue = dispatch_get_main_queue();
  dispatchEnqueue(mainQueue, job);
}

// What you write:
Task {
    await myMainActorMethod()
}

// What (sort of) actually happens:
// (Actual behavior explained above)
Task {
    DispatchQueue.main.async {
        myMainActorMethod()
    }
}

extension MainActor {
    func myMainActorMethod() {}
}

func example() {
    Task {
        await MainActor.shared.myMainActorMethod()
    }
}

///////////// or:

func example() {
    Task {
        await MainActor.run {
            myMainActorMethod()
        }
    }
}

func myMainActorMethod() {}

@MainActor
func myMainActorMethod() {}

func example() {
    await myMainActorMethod() // This method is annotated as @MainActor,
    // so it will run in the MainActor’s context.
}

public protocol GlobalActor {
  associatedtype ActorType: Actor
  static var shared: ActorType { get }
}

func example() {
    // SIL: hop_to_executor(MainActor.shared)
    await myMainActorMethod()
    // SIL: hop_to_executor(DefaultExecutor)
}