Custom Operators in Swift Combine

The Combine framework in Swift is a powerful declarative API for the asynchronous processing of values over time. It takes full advantage of Swift features such as Generics to provide type-safe algorithms that can be composed into processing pipelines. These pipelines can be manipulated and transformed by components called operators. Combine ships with a large assortment of built-in operators that can be chained together to form impressive conduits through which values can be transformed, filtered, buffered, scheduled, and more.

Despite the usefulness of Combine’s built-in operators, there are times when they fall short. This is when constructing your own custom operators adds needed flexibility to perform often complex tasks in a concise and performant manner of your choosing.

Combine Lifecycle

In order to create our own operators, it is necessary to understand the basic lifecycle and structure of a Combine pipeline. In Combine, there are three main abstractions: Publishers, Subscribers, and Operators.

Publishers are value types, or Structs, that describe how values and errors are produced. They allow the registration of subscribers who will receive values over time. In addition to receiving values, a Subscriber can potentially receive a completion, as a success or error, from a Publisher. Subscribers can mutate state, and as such, they are typically implemented as a reference type or Class.

Subscribers are created and then attached to a Publisher by subscribing to it. The Publisher will then send a subscription back to the Subscriber. This subscription is used by the Subscriber to request values from the Publisher. Finally, the Publisher can start sending the requested values back to the Subscriber as requested. Depending on the Publisher type, it can send values that it has indefinitely, or it can complete with a success or failure. This is the basic structure and lifecycle used in Combine.

Operators sit in between Publishers and Subscribers where they transform values received from a Publisher, called the upstream, and send them on to Subscribers, the downstream. In fact, operators act as both a Publisher and as a Subscriber.

Creating a Custom Operator

Let’s cover two different strategies for creating a custom Combine operator. In the first approach, we’ll use the composition of an existing chain of operators to create a reusable component. The second strategy is more involved but provides the ultimate in flexibility.

Composing a Combine Operator

In our first example, we’ll be creating a histogram from a random array of integer values. A histogram tells us the frequency at which each value in the sample data set appears. For example, if our sample data set has two occurrences of the number one, then our histogram will show a count of two as the number of occurrences of the number one.

// random sample of Int
let sample = [1, 3, 2, 1, 4, 2, 3, 2]    

// Histogram
//     key: a unique Int from the sample
//     value: the count of this unique Int in the sample
let histogram = [1: 2, 2: 3, 3: 2, 4: 1]

We can use Combine to calculate the histogram from a sample of random Int.

// random sample of Int

// 1
let sample = [1, 3, 2, 1, 4, 2, 3, 2]

// 2
sample.publisher
      // 3
      .reduce([Int:Int](), { accum, value in
        var next = accum

        if let current = next[value] {
            next[value] = current + 1
        } else {
            next[value] = 1
        }

        return next
    })
    // 4
    .map({ dictionary in
        dictionary.map { $0 }
    })
    // 5
    .map({ item in
        item.sorted { element1, element2 in
            element1.key < element2.key
        }
    })
    .sink { printHistogram(histogram: $0) }
    .store(in: &cancellables)

Which gives us the following output.

histogram standard operators:
1: 2
2: 3
3: 2
4: 1

Here is a breakdown of what is happening with the code:

1. Define our sample data set
2. Get a Publisher of our sample data
3. Bin each unique value in the data set and increase a counter for each occurrence.
4. Convert our Dictionary of binned values into an Array of key/value tuples. eg [(key: Int, value: Int)]
5. Sort the array in ascending order by key

As you can see, we have created a series of chained Combine operators that calculates a histogram for a published data set of Int. But what if we use this sequence of code in more than one location? It would be really nice if we could use a single operator to perform this entire operator chain. This reuse not only makes our code more concise and easier to understand but easier to debug and maintain as well. So let’s do just that by composing a new operator based on what we’ve already done.

// 1
extension Publisher where Output == Int, Failure == Never {
    // 2
    func histogramComposed() -> AnyPublisher<[(key:Int, value:Int)], Never>{
        // 3
        self.reduce([Int:Int](), { accum, value in
            var next = accum

            if let current = next[value] {
                next[value] = current + 1
            } else {
                next[value] = 1
            }

            return next
        })
        .map({ dictionary in
            dictionary.map { $0 }
        })
        .map({ item in
            item.sorted { element1, element2 in
                element1.key < element2.key
            }
        })
        // 4
        .eraseToAnyPublisher()
    }
}

What is this code doing:

1. Create an extension on Publisher and constrain its output to type Int
2. Define a new function on Publisher that returns an AnyPublisher of our histogram output
3. Perform the histogram chain of operators as in the previous example but this time on self. We use self here since we are executing on the current Publisher instance
4. Type erase our publisher to be an AnyPublisher

Now let’s use our new Combine operator.

// 1
let sample = [1, 3, 2, 1, 4, 2, 3, 2]

// 2
sample.publisher
    .histogramComposed()
    .sink { printHistogram(histogram: $0) }
    .store(in: &cancellables)

Which gives us the following output.

histogram composed: 
1: 2
2: 3
3: 2
4: 1

Using the new composed histogram operator:

1. Define our sample data set
2. Directly use our new composed Combine histogram operator

From the example usage of our new histogram operator, you can see that the code at the point of usage is quite simple and reusable. This is a fantastic technique for creating a toolbox of reusable Combine operators.

The Complete Combine Operator

Creating a Combine operator through composition, as we have seen, is a great way to refactor existing code for reuse. However, composition does have its limitations, and that is where creating a native Combine operator becomes important.

A natively implemented Combine operator utilizes the Combine Publisher, Subscriber, and Subscription interfaces and relationships in order to provide its functionality. A native Combine operator acts as both a Subscriber of upstream data and a Publisher to downstream subscribers.

For this example, we’ll create a modulus operator implemented natively in Combine. The modulus is a mathematical operator which gives the remainder of a division as an absolute value and is represented by the percent sign, %. So, for example, 10 % 3 = 1, or 10 modulo 3 is 1 (10 ➗ 3 = 3 Remainder 1).

Let’s look at the complete code for this native Combine operator, how to use it, and then discuss how it works.

// 1
struct ModulusOperator<Upstream: Publisher>: Publisher where Upstream.Output: SignedInteger {
    typealias Output = Upstream.Output // 2
    typealias Failure = Upstream.Failure

    let modulo: Upstream.Output
    let upstream: Upstream

    // 3
    func receive<S>(subscriber: S) where S : Subscriber, Self.Failure == S.Failure, Self.Output == S.Input {
        let bridge = ModulusOperatorBridge(modulo: modulo, downstream: subscriber)
        upstream.subscribe(bridge)
    }
}

extension ModulusOperator {
    // 4
    struct ModulusOperatorBridge<S>: Subscriber where S: Subscriber, S.Input == Output, S.Failure == Failure {
        typealias Input = S.Input
        typealias Failure = S.Failure

        // 5
        let modulo: S.Input
        
        // 6
        let downstream: S

        //7
        let combineIdentifier = CombineIdentifier()

        // 8
        func receive(subscription: Subscription) {
            downstream.receive(subscription: subscription)
        }

        // 9
        func receive(_ input: S.Input) -> Subscribers.Demand {
            downstream.receive(abs(input % modulo))
        }

        func receive(completion: Subscribers.Completion<S.Failure>) {
            downstream.receive(completion: completion)
        }
    }

// Note: `where Output == Int` here limits the `modulus` operator to 
// only being available on publishers of Ints.
extension Publisher where Output == Int {
    // 10
    func modulus(_ modulo: Int) -> ModulusOperator<Self> {
        return ModulusOperator(modulo: modulo, upstream: self)
    }
}

As you can see, the modulus is always positive, and when evenly divisible it is equal to 0.

How does the code work?

Now we can discuss how the native Combine operator code works.

1. We define our new Combine operator as a Publisher with a constraint on some upstream Publishers output of type SignedInteger. Remember, our operator will be acting as both a Publisher and a Subscriber. Thus our input, the upstream, must be SignedIntegers.
2. Our ModulusOperator output, acting as a Publisher, will be the same as our input (i.e. SignedIntegers).
3. Required function implementation for Publisher. Creates a Subscription which acts as a bridge between the operators upstream Publisher and the downstream Subscriber.
4. The ModulusOperatorBridge can act as both a Subscription and a Subscriber. However, simple operators like this one can be a Subscriber without the need of being a Subscription. This is due to the upstream handling lifecycle necessities like Demand. The upstream behavior is acceptable for our operator, so there is no need to implement Subscription. The ModulusOperatorBridge also performs the primary tasks of the modulus operator.
5. Input parameter to the operator for the modulus that will be calculated.
6. References to the downstream Subscriber and the upstream Publisher.
7. CombineIdentifier for CustomCombineIdentifierConvertible conformance when a Subscription or Subject is implemented as a structure.
8. Required function implementations for Subscriber. Links the upstream Subscription to the bridge as a downstream Subscription in addition to lifecycle.
9. Receives input as a Subscriber, performs the modulus operation on this input, and then passes it along to the downstream Subscriber. The new demand for data, if any, from the downstream is relayed to the upstream.
10. Finally, an extension on Publisher makes our custom Combine operator available for use. The extension is limited to those upstream Publishers whose output is of type Int.

Putting this new modulus operator into action on a Publisher of Int would look like:

[-10, -9, -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10].publisher
    .modulus(3)
    .sink { modulus in
        print("modulus: \(modulus)")
    }
    .store(in: &cancellables)
modulus: 1
modulus: 0
modulus: 2
modulus: 1
modulus: 0
modulus: 2
modulus: 1
modulus: 0
modulus: 2
modulus: 1
modulus: 0
modulus: 1
modulus: 2
modulus: 0
modulus: 1
modulus: 2
modulus: 0
modulus: 1
modulus: 2
modulus: 0
modulus: 1

As you can see, the modulus operator will act upon a Publisher of Int. In this example, we’re taking the modulus of 3 for each Int value in turn.

Conclusion

Combine is a powerful declarative framework for the asynchronous processing of values over time. Its utility can be extended and customized even further through the creation of custom operators which act as processors in a pipeline of data. These operators can be created through composition, allowing for excellent reuse of common pipelines. They can also be created through direct implementation of the Combine Publisher, Subscriber, and Subscription protocols, which allows for the ultimate in flexibility and control over the flow of data.

Whenever you find yourself working with Combine, keep these techniques in mind and look for opportunities to create custom operators when relevant. A little time and effort creating a custom Combine operator can save you hours of work down the road.

Josh Justice

Reviewer Big Nerd Ranch

Josh Justice has worked as a developer since 2004 across backend, frontend, and native mobile platforms. Josh values creating maintainable systems via testing, refactoring, and evolutionary design, and mentoring others to do the same. He currently serves as the Web Platform Lead at Big Nerd Ranch.