(Pylance-only) Union of two functions with different total=False TypedDict args causes false positive error

[kkom]

Describe the bug

When I define two functions, each with a total=False TypedDict argument, I get an unexpected type error in Pylance when working with a union of these functions.

It seems like Pylance infers that the provided dictionary is one of the two, and then reports that it's incompatible with the other one.

PS: The error message about the incompatibility looks suspect to me as well – shouldn't it say that field_a is missing from type[TypedDictB] instead (because undeclared keys cannot be set on a TypedDict)? TypedDictB is not total, so field_b does not need to be set.

Code or Screenshots

A minimal example below:

from collections.abc import Callable
from typing import TypedDict


class TypedDictA(TypedDict, total=False):
    field_a: str


class TypedDictB(TypedDict, total=False):
    field_b: str


FunA = Callable[[TypedDictA], None]
FunB = Callable[[TypedDictB], None]
FunC = Callable[[TypedDictA | TypedDictB], None]


def fun_a(f: FunA) -> None:
    f({})


def fun_b(f: FunB) -> None:
    f({})


def fun_a_or_b(f: FunA | FunB) -> None:
    # Argument of type "TypedDictA" cannot be assigned to parameter of type "TypedDictB" "field_b" is missing from "type[TypedDictA]" Pylance(reportGeneralTypeIssues)
    f({})


def fun_c(f: FunC) -> None:
    f({})


def fun_str(a: str) -> None:
    # Expression of type "str" cannot be assigned to return type "None" Type cannot be assigned to type "None" Pylance(reportGeneralTypeIssues)
    return a

VS Code extension or command-line

I'm using strict mode.

When using the latest version of Pyright the unexpected error doesn't appear (only the trivial error at the bottom of the file is reported):

(example-prisma-py3.11) ➜  2023-07-prisma-example git:(main) ✗ npx pyright --version
pyright 1.1.319
(example-prisma-py3.11) ➜  2023-07-prisma-example git:(main) ✗ npx pyright -p pyproject.toml prisma_example/union_type_bug/typed_dict_argument.py
/Users/kkom/Repos/isometric/adhoc/2023-07-prisma-example/prisma_example/union_type_bug/typed_dict_argument.py
  /Users/kkom/Repos/isometric/adhoc/2023-07-prisma-example/prisma_example/union_type_bug/typed_dict_argument.py:37:12 - error: Expression of type "str" cannot be assigned to return type "None"
    Type cannot be assigned to type "None" (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations 

However, I get an error inside the IDE:

My version of the Python extension is v2023.12.0 and of the Pylance extension is v2023.7.40.

[kkom]

The fact that this is working fine in Pyright, but not in Pylance, gives me hope that maybe this was already fixed in Pyright but needs to be released in Pylance?

I am not familiar with the release cadence of both pieces of software, so reporting it here just in case it's more complicated than this.

[erictraut]

Yes, this was already fixed in pyright 1.1.319. The fix is included in the latest prerelease version of Pylance and will appear in this week's production release of Pylance.

We typically release a new version of pyright every Tuesday evening. Pylance typically releases on Wednesday, and it picks up the latest pyright changes for the weekly prerelease version. Those same changes are then released as the production version a week later.

[kkom]

Thank you for the quick response! Glad that #5547 fixes it – I confirm that everything is fine when using pre-releases Python/Pylance extensions.

I think that the other thing I flagged is still a problem:

from collections.abc import Callable
from typing import TypedDict


class TypedDictA(TypedDict, total=False):
    foo: str


class TypedDictB(TypedDict, total=False):
    foo: str
    bar: str


FunB = Callable[[TypedDictB], None]


def fun(d: TypedDictA, f: FunB) -> None:
    # Argument of type "TypedDictA" cannot be assigned to parameter of type "TypedDictB"
    # "bar" is missing from "type[TypedDictA]" (reportGeneralTypeIssues)
    f(d)

bar is not required in TypedDictB because it's not total. I'm happy to open a new issue for it – if you confirm that this is a real problem.

[erictraut]

Pyright is correct in emitting an error in this case. Refer to PEP 589 for details. Mypy generates the same error here.

You're correct that TypedDictB does not require bar, but if bar is present, it must be compatible with the type str. This cannot be guaranteed with a value that satisfies TypedDictA.

[kkom]

Thank you - I did not think about subtyping.

What if the TypedDict is final though?

Pyright already prevents both inheritance (error 1) and structural subtyping (error 3) in this case:

from collections.abc import Callable
from typing import TypedDict, final


@final
class TypedDictA(TypedDict, total=False):
    foo: str


# ERROR #1
# Base class "TypedDictA" is marked final and cannot be subclassed
class TypedDictAInherit(TypedDictA, total=False):
    bar: int


class TypedDictARedefine(TypedDict, total=False):
    foo: str
    bar: int


class TypedDictB(TypedDict, total=False):
    foo: str
    bar: str


FunB = Callable[[TypedDictB], None]


def fun_b(d: TypedDictB) -> None:
    pass


def call(f: FunB, d: TypedDictA) -> None:
    # ERROR #2
    #
    # Argument of type "TypedDictA" cannot be assigned to parameter of type "TypedDictB"
    # "bar" is missing from "type[TypedDictA]" (reportGeneralTypeIssues)
    f(d)


a = TypedDictA(
    foo="",
)

# not creating a_inherit because the typechecker prevents us from creating TypedDictAInherit

a_redefine = TypedDictARedefine(
    foo="",
    bar=0,
)

call(fun_b, a)

# ERROR #3
#
# Argument of type "TypedDictARedefine" cannot be assigned to parameter "d" of type "TypedDictA" in function "call"
# "TypedDictARedefine" is incompatible with "TypedDictA" because of a @final mismatch (reportGeneralTypeIssues
call(fun_b, a_redefine)

In this situation error 2 does feel overly cautious. Or am I missing another scenario in which extra keys may be present in a value that satisfies TypedDictA?

[kkom]

Oh, sorry, I am wrong.

I didn't realise that defining TypedDictARedefine in this way will get rid of error 3:

@final
class TypedDictARedefine(TypedDict, total=False):
    foo: str
    bar: int

So nothing prevents structural subtyping for @final TypedDicts. I assume that this is in line with how PEP 589 is defined (especially if Mypy also behaves in this way).

[erictraut]

TypedDict is a structural type, not a nominal type. @final isn't meaningful here because structural subtyping doesn't depend on inheritance hierarchies.