This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author andreash
Recipients AlexWaygood, andreash, gvanrossum, kj, kumaraditya, python-dev
Date 2022-01-12.16:01:20
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1642003280.47.0.231004587762.issue46333@roundup.psfhosted.org>
In-reply-to
Content
Yeah, sure. The use-case is (de)serialization. Right now I use the library cattr, but there are many others. 

If you are interested there is related discussion in the cattr board [1].


The original problem is how to define the types for serialization.

1. If everything is a class, things work well, but not if type aliases are used: 

2. Type aliases sometimes have to be used - they cannot be avoided in all cases, 
   especially with recursive types. The famous example is 

      Json = Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ]

   (Note: even though mypy does not support this construct, pylance meanwhile does [2])

3. `typing.Annotated` seems to be made for specifying additional information such as value ranges, right to be used
   in (de)serialization + validation contexts. Often these will just be type aliases (not used as class members). 
   Combination is possible with typing.NewType.


The problem is that the implicit `ForwardRef('Json')` cannot be automatically resolved (as it only a name with no context). 
There is really no way this could be handle inside a library such as cattr.  

When one wants to separate interface from implementation this issue is even more complicated. The module where the serialization function is called is typically different from the module with the type definition (This is probably more the norm than the exception)



The option I expored is to explicitly create the ForwardRef and specify the module parameter (even though I have to admit that I also did read that the ForwardRef is only for internal use). 
   
    Json = Union[ List[ForwardRef(Json',module=__name__)], Dict[str, ForwardRef(Json',module=__name__)], int, float, bool, None ]

Ugly, but this is better than nothing.


A (worse) alternative is to do

    Json = Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ]
    typing._eval_type(Json, globals(), locals())

That works since it puts the ForwardRefs into "evaluated" state (self.__forward_value__ is then set)



A third, but future, alternative could be to automatically decompose the argument of ForwardRef into module and type. Then one could write

   Json = Union[ List[__name__+'.Json'], Dict[str, __name__+'.Json'], int, float, bool, None ]

and all above problems would be solved.

Then ForwardRef could stay internal and this is more readable. Even though, the static type checkers would need to understand `__name__+'TYPE'` constructs to be useful.


Anyhow, it would be nice to have a solution here.



[1]: https://github.com/python-attrs/cattrs/issues/201
[2]: https://devblogs.microsoft.com/python/pylance-introduces-five-new-features-that-enable-type-magic-for-python-developers/
History
Date User Action Args
2022-01-12 16:01:20andreashsetrecipients: + andreash, gvanrossum, python-dev, kj, kumaraditya, AlexWaygood
2022-01-12 16:01:20andreashsetmessageid: <1642003280.47.0.231004587762.issue46333@roundup.psfhosted.org>
2022-01-12 16:01:20andreashlinkissue46333 messages
2022-01-12 16:01:20andreashcreate