protocol annotation in var declaration breaks code

Question

I am trying to call the method .enumerate() on an instance of a type which conforms to the protocol Sequence. According to Apple's documentation, this should be correct, since .enumerate is part of the Sequence protocol.

However, I receive this complaint from the compiler:

Member 'enumerated' cannot be used on value of type 'any Sequence<URL>'; consider using a generic constraint instead.

Yet, if I remove the type annotation, then it works.

Here is an example which reproduces the problem:

func example() -> URL? {
    let fm      : FileManager                       = FileManager.default
    let appDir  : FileManager.SearchPathDirectory = FileManager.SearchPathDirectory.applicationDirectory
    let domMask : FileManager.SearchPathDomainMask  = FileManager.SearchPathDomainMask.allDomainsMask
    let appResourceValues : [URLResourceKey] = [URLResourceKey.localizedNameKey]

    var appURLs : any Sequence<URL> = fm.urls(for: appDir, in: domMask)
    //var appURLs : Sequence<URL>     = fm.urls(for: appDir, in: domMask)
    //var appURLs                     = fm.urls(for: appDir, in: domMask)
    var appURL : URL? = appURLs.enumerated().first { (offset: Int, element: URL) in
            try! element.resourceValues(forKeys: Set(appResourceValues)).localizedName!.contains("App Store")
        }?.element
    return appURL
}

There are two commented lines in the code above, which are alternate ways to instantiate appURLs. If I use the first commented line, which is the old Swift syntax apparently, then I receive an error telling me that in order to add a type annotation which enforces a protocol, I need to use any protocolName, and not protocolName. (According to a comment on another post, this was a recent change in Swift: Use of protocol 'YourProtocol' as a type must be written 'any YourProtocol' Error, https://github.com/apple/swift-evolution/blob/main/proposals/0335-existential-any.md)

If I use the second commented line, which removes the protocol annotation altogether, then the code works.

Is this a bug in Swift? How can I apply an annotation to indicate that it must conform to Sequence<URL> without breaking the code?

I tried to declare a generic type parameter, but Swift won't let me. None of these work:

associatedtype does exactly what I want: it creates a generic type parameter. But it doesn't work outside a protocol.

`FileManager.urls(for:in:)` simply returns a `[URL]`. Why do you want it to be in a `let` constant of type `any Sequence` instead? Sure, it works, but as you can see, you cannot do certain things, like `enumerated`, as a result. — Sweeper, Feb 07 '23 at 01:50
@Sweeper This is a code sample. It is not the full program; it is a minimum-working-example which reproduces the issue. I only need the properties of `Sequence`, therefore it is good practice to only enforce that protocol. It is bad practice to type hint something as `[URL]` when you only intend to use the qualities encapsulated by `Sequence` protocol. The Apple documentation says that `enumerated` is a method of the `Sequence` protocol. Therefore, knowing that `appURLs` is of the `Sequence` protocol is sufficient for the compiler to know that invoking `enumerated` is possible. — Myridium, Feb 07 '23 at 02:19
My question is: How can I apply an annotation to indicate that it must conform to `Sequence` without breaking the code? — Myridium, Feb 07 '23 at 02:21
"Therefore, knowing that appURLs is of the Sequence protocol is sufficient for the compiler to know that invoking enumerated is possible." isn't true, and if you really don't have any other reason than "It is bad practice to type hint something as `[URL]` when you only intend to use the qualities encapsulated by `Sequence` protocol", I would suggest that you just remove all the type annotations - this is *not* bad practice at all in Swift. If you do have other reasons, that would make this an [XY problem](http://xyproblem.info/). — Sweeper, Feb 07 '23 at 02:24
And no, this is not a compiler bug. It is actually intentional. (introduced [here](https://github.com/apple/swift-evolution/blob/main/proposals/0309-unlock-existential-types-for-all-protocols.md)) I hope you are not trying to write Swift code in the style of some other language that you know. — Sweeper, Feb 07 '23 at 02:26
Example: I may want to use a method which compiles a enumerable list of `URLs`, but the way that these `URLs` are fetched will depend on runtime parameters (e.g., do I have internet access? Is an external drive currently mounted?). Depending on these parameters, it may be more efficient (or only be possible) to acquire the list of URLs as `[URL]` or as any other type which conforms to `Sequence`. In that case, the return type of such a function will be anything which conforms to `Sequence`. So, pretend I have such a function which defines `appURLs`. Then you will see the problem. — Myridium, Feb 07 '23 at 02:27
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/251679/discussion-between-sweeper-and-myridium). — Sweeper, Feb 07 '23 at 02:30

Sweeper · Answer 1 · 2023-02-07T06:13:19.250

If you annotate appURLs with the existential type any Sequence<URL>, then that means that you don't know what concrete type it actually stores. This is problematic for calling enumerated, because enumerated returns EnumeratedSequence<Self>:

func enumerated() -> EnumeratedSequence<Self>

Self means "type on which this is called" - the exact thing that you don't know. Sometimes, having an unknown Self is fine. e.g. if methods with these signatures existed in Sequence:

func f() -> Self
func g() -> (Self, Int)
func h(p: (Self) -> Void)
func i() -> [Self]
func j() -> [Int: Self]
func k() -> Self?

All of these are covariant positions. It remains type safe to substitute Self in these positions with any Sequence<URL>, which is why you can still call these methods. However, it is not safe to do the same with EnumeratedSequence<Self>, because even though SomeConcreteImplementationOfSequence is a any Sequence<URL>, EnumeratedSequence<SomeConcreteImplementationOfSequence> is not a EnumeratedSequence<any Sequence<URL>>. Generics are invariant in Swift - the Self in EnumeratedSequence<Self> is in an invariant position.

You can see they talk about when functions involving Self can be called in this SE proposal:

[...] but references to Self-rooted associated types will for the same reasons some Self references do today. As alluded to back in Inconsistent Language Semantics, references to covariant Self are already getting automatically replaced with the base object type, permitting usage of Self-returning methods on existential values [...]

This way, a protocol or protocol extension member (method/property/subscript/initializer) may be used on an existential value unless:

The type of the invoked member (accessor — for storage declarations), as viewed in context of the base type, contains references to Self or Self-rooted associated types in non-covariant position. [...]

They even use enumerated() as an example further down!

extension Sequence {
  public func enumerated() -> EnumeratedSequence<Self> {
    return EnumeratedSequence(_base: self)
  }
}

func printEnumerated(s: Sequence) {
  // error: member 'enumerated' cannot be used on value of type protocol type 'Sequence'
  // because it references 'Self' in invariant position; use a conformance constraint
  // instead. [fix-it: printEnumerated(s: Sequence) -> printEnumerated<S: Sequence>(s: S)]
  for (index, element) in s.enumerated() {
    print("\(index) : \(element)")
  }
}

Besides, EnumeratedSequence<any Sequence<URL>> isn't even a valid type! EnumeratedSequence requires its type parameter to be a Sequence, but any Sequence<URL> isn't one! Because Sequence has static requirements.

Responding to your comments,

It is bad practice to type hint something as [URL] when you only intend to use the qualities encapsulated by Sequence protocol

That is not bad practice. Rather, putting type annotations where they are not needed is considered not Swifty.

Example: I may want to use a method which compiles a enumerable list of URLs, but the way that these URLs are fetched will depend on runtime parameters (e.g., do I have internet access? Is an external drive currently mounted?). Depending on these parameters, it may be more efficient (or only be possible) to acquire the list of URLs as [URL] or as any other type which conforms to Sequence<URL>. In that case, the return type of such a function will be anything which conforms to Sequence<URL>

In that case, your function can return an AnySequence<URL>. Unlike any Sequence<URL>, this is a concrete type. You just need to do the extra step of wrapping other sequence types to it:

func fetchSomeURLs() -> AnySequence<URL> {
    if someCondition {
        return AnySequence([url1, url2]) // [URL]
    } else {
        return AnySequence(someOtherImplementationOfSequence)
    }
}

`AnySequence` does not work. See edit. Also, I do not care what is Swifty. I have a brain of my own. I can decide what is best practice for myself. I do not work with others in a team. — Myridium, Feb 07 '23 at 05:49
@Myridium The new code you showed is completely different from what you described. I thought you had a function that returns some kind of sequence? But the function you just added returns an `NSImage`, with a type parameter that I'm not sure what the purpose is... Anyway, at least I understand what you are trying to do in the fifth screenshot. Please refer to the code snippet in my answer - the conversion to `AnySequence` is not implicit, you need to wrap the entire expression with `AnySequence( ... )`. — Sweeper, Feb 07 '23 at 05:59
In case I wasn't clear, "the entire expression" means the part starting with `fm.urls` and ending with `lazy.joined()`. Also, since the code that you are showing is vastly different, I would suggest that you post a new question if you still want to ask about it, and explain *clearly* what you are trying to do. — Sweeper, Feb 07 '23 at 06:05
_How can I apply an annotation to indicate that it [appURLs] must conform to Sequence without breaking the code?_ I'm asking about a specific feature of the language. I want to know how to get the compiler to check for conformity to the protocol. The compiler should be creating a type variable `T1` for `appURLs` when it is first declared, then an associated type variable `T2` for the `EnumeratedSequence`. It can do type checking against this. I think the Swift compiler just isn't sophisticated enough to do what I want. — Myridium, Feb 07 '23 at 07:08
@Myridium You are not being clear enough about what you want. Is `T1` a concrete type? If it isn't (like in the case of `any Sequence`), then `EnumeratedSequence` is not a valid type according to the rules of Swift, for the reasons I mentioned in the answer. Whether `EnumeratedSequence` can *theoretically* be a valid type (e.g. if the compiler was designed differently) is another story, and I haven't looked into that. If `T1` *is* a concrete type, like `[URL]`, then you wouldn't have this problem in the first place, and you wouldn't need to annotate anything. — Sweeper, Feb 07 '23 at 07:20
Of course `EnumeratedSequence` can theoretically be a valid type! That's my point. The protocols define contracts of behaviour, and the specific implementation should be left up to the instance. Your suggestion to cast to `AnySequence` does work for this problem and it is kinda what I was looking for. However, it (presumably) erases the specific type behaviour. — Myridium, Feb 07 '23 at 07:50
The compiler has demonstrated that it is capable of checking that `appURLs` conforms to `Sequence`. But when I make the annotation to do that check, other things break! It can do A or B separately, but it can't do both A and B. — Myridium, Feb 07 '23 at 07:50

Myridium · Answer 2 · 2023-02-07T08:06:58.210

I have concluded that the Swift compiler isn't sophisticated enough to check conformity of a variable to a protocol. (There are some limited cases where it will work.)

A work-around in this case is the following:

extension Sequence {
    func enumerated_custom() -> any Sequence<(offset:Int, element:Iterator.Element)> {
        var index : Int = -1
        return self.lazy.map({ (el:Iterator.Element) in
            index = index+1 ; return (index+1, el)
        })
    }
}

Then, we can do appURLs.enumerated_custom() to get a Sequence<(Int, URL)> which mimics the behaviour of appURLs.enumerated(). I can then use the annotation appURLs : Sequence<URL> in order to check for conformity, and the code does not break.

protocol annotation in var declaration breaks code

2 Answers2