Bug Fix: PyPI's rfc3986 version 2.0.0 release seems to be older than this repository

[ColinKennedy]

Thank you for this excellent codebase, it's made dealing with URIs much easier.

I was messing around with mypy and got this error:

error: Skipping analyzing "rfc3986": module is installed, but missing library stubs or py.typed marker [import]

Which was surprising to me because I can see in this repository's latest tagged version for 2.0.0, released on January 9th 2022 has type hints.

But if you look at the source code .tar.gz file for the same release on PyPI, the 2.0.0 version, also released on January 9th 2022, does not have any type-hints: https://files.pythonhosted.org/packages/85/40/1520d68bfa07ab5a6f065a186815fb6610c86fe957bc065754e47f7b0840/rfc3986-2.0.0.tar.gz (from this page: https://pypi.org/project/rfc3986/2.0.0/#files)

My guess is that somehow the 1.5.0 version of the code was released to PyPI as 2.0.0 by mistake. Could we get another 2.0.1 release (or a reupload of 2.0.0) to fix it?

[sigmavirus24]

Hi @ColinKennedy,

That error is correct. py.typed was only added 2 months ago when we added the bulk of our type hints that are now verified and correct. We were working towards it during 2.0 development, but hadn't finalized it. There seems to be some final things that I missed in my review of #118 (namely including py.typed in our packaging). We need to update

rfc3986/setup.cfg

Lines 42 to 44 in 75e77ba

	[options.package_data]
	* =
	LICENSE

to add in

rfc3986 =
    py.typed

Would you be willing to send one or more PRs with that and maybe include:

• Release notes for a 2.1.0 release
• Add GitHub Workflows for releasing rfc3986 based off of urllib3's automation

cc @Sachaa-Thanasius

[Sachaa-Thanasius]

I'd be very surprised if setuptools doesn't pick up the py.typed file without that setup.cfg entry, though given that package-data is manually specified, that might override its automatic finding. I can attempt to tackle some of this.

Potential food for thought: #118 did not adjust any docstrings or documentation to account for incongruency with the annotations (e.g. str in docstring vs str | bytes in parameter annotation), so depending on what you consider as part of the "typed" contract, a bit of work might still need doing to consider this "properly typed" beyond the release notes and workflow additions.

[ColinKennedy]

Checking the pyproject.toml it looks like there was typing enforced by pyright. I'm not experienced with using an LSP as a type-hint enforcer. I'd always preferred mypy since it's maintained by the Python org. Mypy shows a number of issues. I skimmed the codebase quickly and the logs are consistent with what I checked.

src/rfc3986/compat.py:41: error: Item "None" of "bytes | None" has no attribute "decode"  [union-attr]
src/rfc3986/compat.py:63: error: Item "None" of "str | None" has no attribute "encode"  [union-attr]
src/rfc3986/_mixin.py:72: error: Incompatible return value type (got "dict[str, str | Any]", expected "_AuthorityInfo")  [return-value]
src/rfc3986/_mixin.py:75: error: Argument 1 to "match" of "Pattern" has incompatible type "str | None"; expected "str"  [arg-type]
src/rfc3986/_mixin.py:81: error: Returning Any from function declared to return "Validator"  [no-any-return]
src/rfc3986/_mixin.py:272: error: "URIMixin" has no attribute "normalize"  [attr-defined]
src/rfc3986/_mixin.py:295: error: "type[Self]" has no attribute "from_string"  [attr-defined]
src/rfc3986/_mixin.py:318: error: Argument 1 to "normalize_path" has incompatible type "str | None"; expected "str"  [arg-type]
src/rfc3986/_mixin.py:324: error: Argument 1 to "normalize_path" has incompatible type "str | None"; expected "str"  [arg-type]
src/rfc3986/_mixin.py:331: error: Incompatible types in assignment (expression has type "str | None", variable has type "str")  [assignment]
src/rfc3986/_mixin.py:408: error: "Self" has no attribute "_replace"  [attr-defined]
src/rfc3986/uri.py:107: error: Incompatible types in assignment (expression has type "Callable[[tuple[_T_co, ...]], int]", base class "object" defined the type as "Callable[[object], int]")  [assignment]
src/rfc3986/uri.py:116: error: Argument 1 to "from_string" of "URIReference" has incompatible type "object"; expected "str | bytes"  [arg-type]
src/rfc3986/uri.py:125: error: Argument 1 to "tuple" has incompatible type "object"; expected "Iterable[Never]"  [arg-type]
src/rfc3986/uri.py:126: error: Argument 1 to "normalized_equality" of "URIMixin" has incompatible type "object"; expected "URIReference"  [arg-type]
src/rfc3986/uri.py:164: error: Item "None" of "Match[str] | None" has no attribute "groupdict"  [union-attr]
src/rfc3986/parseresult.py:60: error: If x = b'abc' then f"{x}" or "{}".format(x) produces "b'abc'", not "abc". If this is desired behavior, use f"{x!r}" or "{!r}".format(x). Otherwise, decode the bytes  [str-bytes-safe]
src/rfc3986/parseresult.py:60: error: Incompatible types in assignment (expression has type "str", variable has type "bytes | None")  [assignment]
src/rfc3986/parseresult.py:62: error: Argument 1 to "normalize_authority" has incompatible type "tuple[str | None, str | None, bytes | None]"; expected "tuple[str | None, str | None, str | None]"  [arg-type]
src/rfc3986/parseresult.py:74: error: Returning Any from function declared to return "str"  [no-any-return]
src/rfc3986/parseresult.py:74: error: Returning Any from function declared to return "bytes"  [no-any-return]
src/rfc3986/parseresult.py:74: error: "ParseResultMixin[str]" has no attribute "unsplit"  [attr-defined]
src/rfc3986/parseresult.py:74: error: "ParseResultMixin[bytes]" has no attribute "unsplit"  [attr-defined]
src/rfc3986/parseresult.py:93: error: List or tuple literal expected as the second argument to "namedtuple()"  [misc]
src/rfc3986/parseresult.py:124: error: Too many arguments for "__new__" of "namedtuple@93"  [call-arg]
src/rfc3986/parseresult.py:237: error: Incompatible types in assignment (expression has type "int | str | None", target has type "str | None")  [assignment]
src/rfc3986/parseresult.py:246: error: Argument 3 to "ParseResult" has incompatible type "**dict[str, str | None]"; expected "int | None"  [arg-type]
src/rfc3986/parseresult.py:254: error: Need type annotation for "attr"  [var-annotated]
src/rfc3986/parseresult.py:255: error: "Never" has no attribute "encode"  [attr-defined]
src/rfc3986/parseresult.py:279: error: List or tuple literal expected as the second argument to "namedtuple()"  [misc]
src/rfc3986/parseresult.py:308: error: Too many arguments for "__new__" of "namedtuple@279"  [call-arg]
src/rfc3986/parseresult.py:408: error: Item "None" of "str | None" has no attribute "encode"  [union-attr]
src/rfc3986/parseresult.py:431: error: Item "None" of "int | str | None" has no attribute "encode"  [union-attr]
src/rfc3986/parseresult.py:435: error: Incompatible types in assignment (expression has type "dict[str, bytes | None]", variable has type "dict[str, Any | int | bytes | None]")  [assignment]
src/rfc3986/parseresult.py:435: note: "Dict" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
src/rfc3986/parseresult.py:435: note: Consider using "Mapping" instead, which is covariant in the value type
src/rfc3986/parseresult.py:437: error: Argument 1 to "_generate_authority" of "ParseResultMixin" has incompatible type "dict[str, Any | int | bytes | None]"; expected "dict[str, bytes | None]"  [arg-type]
src/rfc3986/parseresult.py:440: error: Argument 1 has incompatible type "Any | int | bytes | None"; expected "str | bytes"  [arg-type]
src/rfc3986/parseresult.py:442: error: Argument 1 has incompatible type "Any | int | bytes | None"; expected "str | bytes"  [arg-type]
src/rfc3986/parseresult.py:443: error: Argument 1 has incompatible type "Any | int | bytes | None"; expected "str | bytes"  [arg-type]
src/rfc3986/parseresult.py:444: error: Argument 1 has incompatible type "Any | int | bytes | None"; expected "str | bytes"  [arg-type]
src/rfc3986/parseresult.py:452: error: Argument 4 to "ParseResultBytes" has incompatible type "**dict[str, Any | int | bytes | None]"; expected "bytes | None"  [arg-type]
src/rfc3986/parseresult.py:452: error: Argument 4 to "ParseResultBytes" has incompatible type "**dict[str, Any | int | bytes | None]"; expected "int | None"  [arg-type]
src/rfc3986/parseresult.py:512: error: Argument 1 to "split_authority" has incompatible type "str | None"; expected "str"  [arg-type]
src/rfc3986/parseresult.py:522: error: Incompatible types in assignment (expression has type "int", variable has type "str | None")  [assignment]
src/rfc3986/parseresult.py:525: error: Incompatible return value type (got "tuple[str | None, str | None, str | None]", expected "tuple[str | None, str | None, int | None]")  [return-value]
src/rfc3986/iri.py:29: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
src/rfc3986/iri.py:32: error: Name "uri.URIMixin" is not defined  [name-defined]
src/rfc3986/iri.py:32: error: Class cannot subclass "URIMixin" (has type "Any")  [misc]
src/rfc3986/iri.py:66: error: Incompatible types in assignment (expression has type "Callable[[tuple[_T_co, ...]], int]", base class "object" defined the type as "Callable[[object], int]")  [assignment]
src/rfc3986/iri.py:75: error: Argument 1 to "from_string" of "IRIReference" has incompatible type "object"; expected "str | bytes"  [arg-type]
src/rfc3986/iri.py:84: error: Argument 1 to "tuple" has incompatible type "object"; expected "Iterable[Never]"  [arg-type]
src/rfc3986/iri.py:87: error: Argument 1 to "match" of "Pattern" has incompatible type "str | None"; expected "str"  [arg-type]
src/rfc3986/iri.py:103: error: Item "None" of "Match[str] | None" has no attribute "groupdict"  [union-attr]
src/rfc3986/iri.py:150: error: Argument 1 to "InvalidAuthority" has incompatible type "str | None"; expected "str | bytes"  [arg-type]
src/rfc3986/iri.py:164: error: Unsupported left operand type for + ("None")  [operator]
src/rfc3986/iri.py:170: error: Unsupported left operand type for + ("None")  [operator]
src/rfc3986/iri.py:170: note: Left operand is of type "str | None"
src/rfc3986/__init__.py:22: error: Module "rfc3986.api" does not explicitly export attribute "IRIReference"  [attr-defined]
src/rfc3986/__init__.py:23: error: Module "rfc3986.api" does not explicitly export attribute "URIReference"  [attr-defined]
src/rfc3986/builder.py:81: error: Function is missing a type annotation  [no-untyped-def]
Found 58 errors in 7 files (checked 14 source files)

This is mostly just to echo @Sachaa-Thanasius previous point about annotations vs docstrings but judging from the output this repository has some work before it is fully described. In the meantime I'll probably move forward with some temporary stubs and replace the stubs once types are available from here. Thank you for your replies!