Reconciling the differences between BinaryIO and RawIOBase

Question

My modest goal: Correctly annotate a function argument as an object supporting read() which returns bytes.

If I use typing.BinaryIO, I cannot accept a serial.Serial object (which derives from io.RawIOBase).
If I use io.RawIOBase, I cannot accept a BinaryIO object (e.g. sys.stdin.buffer)

import io
import serial
from typing import BinaryIO, Optional
import sys

def foo_rawiobase(infile: io.RawIOBase) -> Optional[bytes]:
    return infile.read(42)

def foo_binaryio(infile: BinaryIO) -> bytes:
    return infile.read(42)


def foo_serial(dev: serial.Serial) -> None:
    # error: Argument 1 to "foo_binaryio" has incompatible type "Serial"; expected "BinaryIO"  [arg-type]
    foo_binaryio(dev)

    # ok
    foo_rawiobase(dev)

def foo_file(f: BinaryIO) -> None:
    # ok
    foo_binaryio(f)

    # error: Argument 1 to "foo_rawiobase" has incompatible type "BinaryIO"; expected "RawIOBase"  [arg-type]
    foo_rawiobase(f)

def foo_stdin() -> None:
    foo_file(sys.stdin.buffer)

typeshed definitions:

How do I properly handle both cases?

I once wrote a similar question, maybe it contains some insight: https://stackoverflow.com/questions/69570166/how-to-write-a-file-like-class-that-satisfies-typing-textio — mkrieger1, Mar 13 '23 at 19:56

score 1 · Answer 1 · answered Mar 13 '23 at 19:58

One solution to this is to use PEP 544 – Protocols: Structural subtyping (static duck typing).

Use typing.Protocol to define the required interface. Then, any compatible type can be used.

The mypy docs even include this exact example:

class SupportsRead(Protocol):
    def read(self, amount: int) -> bytes: ...

Using Protocol, the above problem could be solved like this:

import io
import serial
from typing import BinaryIO, Optional, Protocol
import sys

class Readable(Protocol):
    def read(self, n: int = -1) -> Optional[bytes]:
        ...

def foo_custom(infile: Readable) -> Optional[bytes]:
    return infile.read(42)


def foo_serial(dev: serial.Serial) -> None:
    foo_custom(dev)

def foo_file(f: BinaryIO) -> None:
    foo_custom(f)

def foo_stdin() -> None:
    foo_file(sys.stdin.buffer)

(Note that io.RawIOBase declares the return type of read as Optional[bytes], so I carry that forward here.)

Jonathon Reinhart · Answer 2 · 2023-03-14T15:01:24.873

Another solution to this is to accept a Union of both possible types. The type checker should properly deduce the type of infile.read.

import io
import serial
from typing import BinaryIO, Optional
import sys

def foo_union(infile: io.RawIOBase | BinaryIO) -> Optional[bytes]:
    return infile.read(42)


def foo_serial(dev: serial.Serial) -> None:
    foo_union(dev)

def foo_file(f: BinaryIO) -> None:
    foo_union(f)

def foo_stdin() -> None:
    foo_file(sys.stdin.buffer)

This prevents inappropriate use of methods or attributes that are present on only one of the input types:

def bad_use(infile: io.RawIOBase | BinaryIO) -> str:
    # error: Item "RawIOBase" of "Union[RawIOBase, BinaryIO]" has no attribute "name"
    return infile.name

Reconciling the differences between BinaryIO and RawIOBase

2 Answers2