Friday, 6 October 2017

Mypy 0.530 Released

We’ve just uploaded mypy 0.530 to the Python Package Index (PyPI). This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

You can read the documentation for this release on ReadTheDocs.

Major New Features

We present several new features in this release: Protocols, TypedDict, Per-file Control of Strict Optional Checking and Module Aliases. The features are described below.

Protocols (structural subtyping, a.k.a. static duck typing)

Protocols are a new, experimental feature, intended to better support duck typing. Protocols are on a standardization track (PEP 544), but we are still working on the details of how protocols work. They are not yet included in the standard library typing module. You can import them from the new typing_extensions package instead (not to be confused with mypy_extensions).

To use protocols, first install the package typing_extensions:

    python3 -m pip install -U typing_extensions

Once this is installed you can start defining protocols by inheriting from typing_extensions.Protocol. Here’s a simple example:

    from typing import Iterable
    from typing_extensions import Protocol
    
    class SupportsClose(Protocol):
        def close(self) -> None: ...  # Use a literal '...' here
    
    class Resource:  # No SupportsClose base class!
        # ... some methods ...
    
        def close(self) -> None:
           self.resource.release()
    
    def close_all(items: Iterable[SupportsClose]) -> None:
        for item in items:
            item.close()
    
    close_all([Resource(), open('some/file')])  # Okay!

In this example, Resource is compatible with the SupportClose protocol since it defines a compatible close() method. Regular file objects returned by open() are similarly compatible with the protocol, as they support close().

Protocols were implemented by Ivan Levkivskyi. Here’s the documentation.

TypedDict

TypedDict has been an undocumented feature for a while; it’s now officially supported. It’s still experimental, but we think that it’s robust enough to invite everyone to kick the tires.

Python programs often use dictionaries with string keys to represent objects. Here is a typical example:

    movie = {'name': 'Blade Runner', 'year': 1982}

You can now define a TypedDict to give a precise type for objects like movie, where the type of each dictionary value depends on the key:

    from mypy_extensions import TypedDict
    Movie = TypedDict('Movie', {'name': str, 'year': int})
    favorite = {'name': 'Blade Runner', 'year': 1982}  # type: Movie
Movie is a TypedDict type with two items: 'name' (with type str) and 'year' (with type int). Note that we used an explicit type annotation for the favorite variable. This type annotation is important -- without it, mypy will try to infer a regular, uniform Dict type for favorite, which is not what we want here.

By default mypy ensures that a TypedDict object has exactly all the specified keys. You can provide the total=False argument to TypedDict(...) to allow keys to be left out when creating a TypedDict object:

    GuiOptions = TypedDict(
       'GuiOptions', {'language': str, 'color': str}, total=False)
    options = {}  # type: GuiOptions  # Okay
    options['language'] = 'en'

Python 3.6 also supports an alternative, class-based syntax to define a TypedDict:

       from mypy_extensions import TypedDict
    
       class Movie(TypedDict):
           name: str
           year: int

This syntax also supports the total=False flag, as follows:

    class GuiOptions(TypeDict, total=False):
        ...

Several people contributed to TypedDict, including David Foster (original implementation), Ivan Levkivskyi, Roy Williams, Max Moroz and Jukka Lehtosalo. Here’s the documentation.

Per-file Control of Strict Optional Checking

Mypy has supported the option --strict-optional for a while that causes mypy to detect errors due to None values. However, many users still haven’t switched to using this option as it can require a bit of refactoring to adopt, such as adding Optional[…] to annotations. Now mypy makes this easier as you can enable strict optional checking on a per-file basis, allowing you to gradually migrate your codebase to strict optional checking, while ensuring that previously migrated modules remain strict optional clean.

You can control this through a mypy config file. Example:

    ; Enable strict optional checking in acme.util
    [mypy-acme.util]
    strict_optional = True

This was implemented by David Fisher (PR 3206). Most of the work was actually completed for the 0.520/0.521 release but we withheld the announcement because we hadn’t had enough opportunity to test it.

Module Aliases

Mypy now can handle aliases for modules. Example:

    import subprocess
    sp = subprocess
    sp.check_call(['whoami'])  # Okay

This was contributed by Carl Meyer, with help from Ivan Levkivskyi.

More New Features

  • Support module-level __getattr__() in stubs (Jelle Zijlstra, PR 3647; Ethan Smith, PR 3779)
  • Generate note if reveal_type() is used in an unchecked function (Svyatoslav Ilinskiy, PR 3743)
  • Add flag --disallow-incomplete-defs to disallow partially annotated functions (Svyatoslav Ilinskiy, PR 3744)
  • Add flag --disallow-untyped-decorators to output error if a typed function is used with an untyped decorator (Svyatoslav Ilinskiy, PR 3555)
  • Tag Any types with the kind of their origin (Svyatoslav Ilinskiy, PR 3786)
  • Make disallow_subclassing_any a per-module option (Daniel Watkins, PR 3801)
  • Report sources of Any in html-report per-line as hovertext (Svyatoslav Ilinskiy, PR 3798)
  • Add --warn-unused-configs flag (PR 3865)
  • Support six.add_metaclass() (Elazar Gershuni, PR 3842)
  • Type check arguments for super() (PR 3919)
  • Allow creating type aliases for None (Ivan Levkivskyi, PR 3754)

Stubgen Improvements

  • Add -o <dir> option (Brian Bruggeman, PR 3902)
  • Preserve existing annotations (Daniel F Moisset, PR 3169)

Notable Bugs Fixed

  • Fix crashes and failures caused by forward references to types (Ivan Levkivskyi, PR 3952)
  • Make configuration file section ordering consistent on Python 3.5 and earlier (PR 3678)
  • Re-broaden variable type to Any on assignment if it started as Any but was narrowed in an enclosing frame (lincolnq, PR 3361)
  • Suppress errors found in unreachable parts of a Boolean expression (Roy Williams, PR 3666)
  • Give a useful message if we can't infer a TypeVar-based type for attribute with value restriction (PR 3651)
  • Support docstrings in a class-based TypedDict (Dominik Miedziński, PR 3749)
  • Do not follow imports from stubs with --follow-imports=skip (Svyatoslav Ilinskiy, PR 3738)
  • Support exported names starting with an underscore in __all__ (David Ross, PR 3746)
  • Search lib directory suffixed by ABI (32 or 64), which is used in at least Gentoo Linux (Shunsuke Shimizu, PR 3758)
  • Fix crash with unanalyzed branches and old-html report (Svyatoslav Ilinskiy, PR 3782)
  • Hide imported names in stubs unless as id is used (to align with PEP 484) (Ivan Levkivskyi, PR 3706)
  • Use correct configuration file path when searching for plugins (Tin Tvrtković, PR 3770)
  • Use types of actual arguments to infer lambda argument types if callee is a lambda (Elazar Gershuni, PR 3116)
  • Don't supply a type context to the expression in yield from (PR 3815)
  • Erase type variables of generic types when used in Type[C] (Daniel Watkins, PR 3833)
  • Fix using stale metadata in incremental mode when errors are no longer ignored (Mark Hahnenberg, PR 3793)
  • Infer better return type for some calls to overloaded functions with Any actual arguments (PR 3917)
  • Fix problem with overload resolution when there are union types in signatures (PR 3300)
  • Fix type checking decorated method overrides when decorator changes signature (PR 3918)
  • Fix crash caused by attempted serialization of a partial type (PR 3954)
  • Support user install on Windows (Ethan Smith, PR 4005)
  • Fix crashes related to partial types (Ivan Levkivskyi, PR 3995)
  • Don't crash on unexpected PY2 except syntax (PR 4011)

Dependencies Etc.

  • This mypy release requires typed-ast version 1.1.0 or higher. Since mypy 0.52[01] required a version less than 1.1.0, this is a mandatory upgrade. The main reason for the upgrade is that the new version supports Python 3.3 and 3.4 on Windows, which required some API changes.
  • This is the last mypy release with official support for running on Python 3.3, which reached its End Of Life last week. Note that future mypy versions will still support checking older Python versions — they just won’t run on 3.3 or earlier. It’s easy to have multiple Python versions installed, so this won’t prevent anyone from running mypy.

Other Changes

  • Many more mypy unit tests are now run via pytest (but the runtests.py script is still not dead yet).
  • Almost all of the mypy implementation is now type checked using strict optional checking (Ivan Levkiskyi).
  • Improvements to error messages (Dominik Miedziński, PR 3515; Elazar Gershuni, PR 3783; Svyatoslav Ilinskiy, PR 3707; Jesse Lord, PR 3411; Ethan Smith, PR 3862; Daniel Watkins, PR 3873, PR 3893, PR 3898, and PR 3910).

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Brian Bruggeman
  • Carl Meyer
  • Chen Rotem Levy
  • Daniel F Moisset
  • Daniel Watkins
  • David Ross
  • Dominik Miedziński
  • Edward Betts
  • Elazar Gershuni
  • Ethan Smith
  • Greg Price
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • Jesse Lord
  • lincolnq
  • Mark Hahnenberg
  • Max Moroz
  • Michael R. Crusoe
  • Roy Williams
  • Shunsuke Shimizu
  • Svyatoslav Ilinskiy
  • Thomas Grainger
  • Tin Tvrtković
  • Tobin Yehle

Additional thanks to all contributors to typeshed:

  • Adam Dangoor
  • Alex Ellis
  • Alex Grönholm
  • Amandine Lee
  • Artur Rodrigues
  • Ashwini Chaudhary
  • Ben Leslie
  • Brian C. Lane
  • Bruce Merry
  • Chris Kerr
  • Daniel Watkins
  • David Novakovic
  • Dominik Gabi
  • Elazar Gershuni
  • Ethan Smith
  • Evan Krall
  • FichteFoll
  • Garrett
  • Ivan Levkivskyi
  • Jaen
  • Jelle Zijlstra
  • John Reese
  • Jonathan Schoonhoven
  • Liran Nuna
  • Luka Sterbic
  • Martin DeMello
  • Matthias Kramm
  • Melvyn Sopacua
  • Michael Lee
  • Mikołaj Biel
  • Neil Pilgrim
  • Øystein Olsen
  • Pavel Pletenev
  • Pieter-Jan Briers
  • Robert T. McGibbon
  • Semyon Proshev
  • Svyatoslav Ilinskiy
  • Thomas Grainger
  • Tim Crawford
  • Zack Hsi
  • aubanel
  • blokeley
  • daid
  • eujing
  • rchen152
  • toejough
  • vim345

Tuesday, 25 July 2017

Mypy 0.521 Released

We’ve just uploaded mypy 0.521 to PyPI. This is a bugfix release containing no new features, only fixes for some important bugs and regressions in 0.520. You can install it as follows:

    python3 -m pip install --upgrade mypy

Mypy Bugs Fixed

  • Fix crash on name collision for self attributes (Ivan Levkivskyi, PRs 3700 and 3719)
  • Fix bug in cache updates and improve cache logging (PR 3708)
  • Fix per-file strict Optional interaction with default-None args (PR 3686)
  • Fix crash in --quick mode when aliases are re-exported (Ivan Levkivskyi, PR 3740)
  • Fix crash on incompatible redefinition in named tuple (Ivan Levkivskyi, PR 3760)

Typeshed Bugs Fixed

  • Fix regressions in csv.DictReader (Thomas Grainger, PRs 1475 and 1478)

Acknowledgments

First of all, I’d like to thank my employer Dropbox for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Ivan Levkivskyi

Additional thanks to all contributors to typeshed for this release:

  • Thomas Grainger

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)

Monday, 10 July 2017

Mypy 0.520 Released

We’ve just uploaded mypy 0.520 to PyPI. This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install --upgrade mypy

We are always happy to answer questions on Gitter — note that we’ve moved to a new channel, python/typing.

Major New Features

Fine-grained Control of Any Types

Mypy now has a --disallow-any option that gives you more fine-grained control over Any types in your code. In particular, it lets you disallow the following kinds of Any:

  • explicit disallows explicit Any in type positions such as type annotations and generic type parameters
  • expr disallows all expressions in the module that have type Any
  • unimported disallows Any types that come from unfollowed imports (they usually appear with --follow-imports=skip or --ignore-missing-imports)
  • generics disallows usage of generic types that do not specify explicit type parameters. For example, List (without type parameters) is disallowed but using List[int] (or even List[Any]) is allowed.
  • decorated disallows functions that have Any in their signature after decorator transformation

You can use the option from command-line as --disallow-any=expr,explicit or from config file as disallow_any = generics,expr.

To learn more about the --disallow-any option, read the documentation.

This was implemented by Svyatoslav Ilinskiy.

__setattr__ Support

Mypy now understands __setattr__. If a class defines it, mypy allows assignments to arbitrary attributes. Example:

    class A:
        ...
        # This will allow assignment to any A.x, if x is the same type as value.
        def __setattr__(self, name: str, value: int) -> None:
            ...
    a.foo = 42  # Works
    a.bar = 'Ex-parrot'  # Fails type checking

This was contributed by Ethan Smith (PR 3451).

Hash-based Incremental Cache

Previously the incremental cache (enabled by the -i option) detected changes in source files based on the file modification time and file size. This made it hard to share incremental cache files between systems, since modification times generally aren’t consistent across multiple repository checkouts. Now mypy requires a change to the md5 hash of a file to consider it changed. This makes it possible to “warm up” the incremental cache by downloading it from a central repository, which can speed up mypy runs significantly. Providing an implementation of such a repository is out of scope for mypy, though.

This was implemented by Max Moroz and Guido (PR 3437 and some others).

NamedTuple Methods

It is now possible to add methods to NamedTuples defined using the Python 3.6 class-based syntax:
    class Point(NamedTuple):
        x: float
        y: float

        def translate(self, dx: float, dx: float) -> 'Point':
            return Point(x=self.x + dx, y=self.y + dy)

    p = Point(1, 2).translate(Point(4, 5))
    print(p)

Note that runtime support for this feature requires typing 3.6.1.

This was contributed by Jelle Zijlstra (PR 3081).

Improved Error Messages

Various error messages generated by mypy have been improved. Here is a summary of what’s changed:

  • Use distinct formatting for regular types and type object types; previously they looked sometimes confusingly identical (Elazar Gershuni, PR 3374)
  • Display the union type and offending union item when reporting a missing attribute for a union type (Amir Rachum, PR 3402)
  • Include line number of previous definition when there is a redefinition error (Max Moroz, PR 3424)
  • Suggest a type annotation if the type of a returned variable is incompatible with the declared return type (Jesse Lord, PR 3428)
  • Improve error message for invalid package names passed to mypy (chernrick, PR 3447)
  • Fix several error messages that didn't display types correctly (Svyatoslav Ilinskiy, PR 3638)
  • Revamp TypedDict-related error messages (Jukka Lehtosalo)

More New Features

  • Experimental support for enabling strict optional checking on a per file basis (David Fisher, PR 3206)
  • Support simple aliases to modules (Carl Meyer, PR 3435)
  • Support six.with_metaclass (Guido, PR 3364)
  • Allow any subtype of typing.Mapping[str, Any] to be passed in as **kwargs (Roy Williams, PR 3604)
  • Add flag --no-implicit-optional to avoid interpreting arguments with a default of None as Optional — this may become the default behavior in the future (David Fisher, PR 3248)
  • Allow NewType that subclasses another NewType (Carl Meyer, PR 3465)
  • Infer better return types for certain common special signatures such as open (if the mode is a string literal), contextlib.contextmanager and ** with a literal exponent (Jukka Lehtosalo, PR 3299)
  • Add --include-private flag to stubgen to include private members (Chad Dombrova, PR 3367)
  • Add flag --any-exprs-report to generate report about Any expressions (Svyatoslav Ilinskiy, PR 3492)
  • Allow tuple indexing using non-literal expressions (David Fisher, PR 3514)
  • Normalize Type[Union[...]] internally to Union[Type[...], ...] (Max Moroz, PR 3209)
  • Improve handling of compile-time relational operators (Ryan Gonzalez, PR 3158)
  • Add --no-incremental flag to disable incremental mode (David Fisher, PR 3570)
  • Improve type inference of unconditional if statements without else block (Michael Lee, PR 3567)
  • Document the --shadow-file option (David Fisher, PR 3615)
  • Display unreachable code as Any in HTML coverage reports (Svyatoslav Ilinskiy, PR 3616)
  • Add --skip-version-check flag to ignore the mypy version when validating incremental cache metadata (Guido, PR 3641)
  • Add a very experimental (and still undocumented) plugin system for extending mypy (PR 3534 and Jukka Lehtosalo, PR 3517)
  • Many improvements to TypedDict and new documentation [this is still not an officially supported feature] (Jukka Lehtosalo, Max Moroz and Ivan Levkivskyi)

Notable Bugs Fixed

  • Make change of platform invalidate the cache (Guido, PR 3663)
  • Fix imported generic type aliases (Ivan Levkivskyi, PR 3524)
  • Fix Optional[...] type aliases (Ivan Levkivskyi, PR 3524)
  • Reject redefinition of type aliases (Ivan Levkivskyi, PR 3524)
  • Fix None slice bounds with --strict-optional (Dominik Miedziński, PR 3445)
  • Fix crashes related to referring to a type before the definition has been processed (Ivan Levkiskyi, PR 3322 and PR 3560)
  • Fix crash when displaying the location of a method definition (Dominik Miedziński, PR 3375)
  • Treat non-inplace augmented assignment as a simple assignment (Max Moroz, PR 3110)
  • Fixes to quick mode (Ivan Levkivskyi, PR 3304 and PR 3356)
  • Don't warn about returning Any if an Any return value is explicitly allowed (Ivan Levkivskyi, PR 3473)
  • Fix type of f-string with only {…} such as f'{x}' (lincolnq, PR 3390)
  • Make Type[X] compatible with the metaclass of X (Elazar Gershuni, PR 3346)
  • Support accessing modules imported in class bodies within methods (Carl Meyer, PR 3450)
  • Fix crash when serializing a type-ignored property setter (Jelle Zijlstra, PR 3493)
  • Fix spurious "cycle in function expansion" errors (Guido, PR 3504)
  • Fix to union type used in condition (David Fisher, PR 3510)
  • Fix crashes on lambda generators (Ivan Levkivskyi, PR 3523)
  • Fix accessing qualified import in incremental mode (Michael Lee, PR 3548)
  • Fix overload resolution for metaclass (Elazar Gershuni, PR 3511)
  • Allow instances of a class to access inner class definitions on that class (Roy Williams, PR 3636)
  • Fix to type checking calls to classmethods with self types (Elazar Gershuni, PR 3633)
  • Don't crash if isinstance() called with too few arguments (Jukka Lehtosalo, PR 3652)
  • Reject isinstance() with NewType (Jukka Lehtosalo, PR 3654)
  • Fix crash during deserialization of callable types (Jukka Lehtosalo, PR 3660)

Other Changes

  • Require version 1.0.4 of the typed-ast dependency to run mypy
  • Release version 0.3.0 of mypy_extensions
  • Publish a new mypy roadmap
  • The majority of the mypy implementation is now type checked using strict optional checking (Ivan Levkiskyi)

Acknowledgments

First of all, I’d like to thank my employer Dropbox for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Abhas Bhattacharya
  • Amir Rachum
  • Carl Meyer
  • Chad Dombrova
  • chernrick
  • Dominik Miedziński
  • Elazar Gershuni
  • Ethan Smith
  • fdeliege
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • Jesse Lord
  • lincolnq
  • Max Moroz
  • Michael Lee
  • Nathaniel Smith
  • Patrick Williams
  • Roy Williams
  • Ryan Gonzalez
  • Svyatoslav Ilinskiy
  • Thomas Grainger

Additional thanks to all contributors to typeshed:

  • Alex Grönholm
  • Alex Khomchenko
  • amstree
  • Antoine Reversat
  • Ashwini Chaudhary
  • Ask Solem
  • Atul Varma
  • Benjamin Starling
  • Carl Meyer
  • Cooper Lees
  • Daniel Li
  • David Euresti
  • DmitriyS
  • Doug Freed
  • Eklavya Sharma
  • Elliot Marsden
  • elmar bucher
  • Emily Morehouse
  • Ethan Smith
  • George Caley
  • Harmen
  • Ivan Levkivskyi
  • James Elford
  • Jan Hermann
  • Jason Fried
  • Jelle Zijlstra
  • Joe Bateson
  • John Reese
  • khyox
  • li-dan
  • Luka Sterbic
  • Lukasz Langa
  • macheins
  • Mahmoud Afaneh
  • Matthew Page
  • Matthias Kramm
  • Matěj Cepl
  • Max Moroz
  • Max Rozentsveyg
  • Michael Lee
  • Miguel Gaiowski
  • mistermocha
  • mpage
  • Nathan Henrie
  • Peter Vilim
  • pkordas
  • Ray Kraesig
  • rchen152
  • Rhys Parry
  • Roy Williams
  • Ryan Jarvis
  • Sebastian Meßmer
  • Semyon Proshev
  • Sergey Passichenko
  • Suren Nihalani
  • Svyatoslav Ilinskiy
  • Teemu R
  • Thomas Grainger
  • Yusuke Miyazaki

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)

Friday, 5 May 2017

Mypy 0.510 Released

We’ve just uploaded mypy 0.510 to PyPI. This release adds new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install --upgrade mypy

Note: Remember that the package name is now mypy (no longer mypy-lang).

Update (10 May): Released mypy 0.511 that fixes two crash bugs but is otherwise identical to 0.510.

Overloads in Source Files

Previously the @overload decorator was only supported in stub files. Now you can use it in regular source files as well. First write some @overload declarations that define the different signatures your function supports. These should have empty bodies and are only used by mypy. Put the actual runtime definition of the function after these declarations, without a decorator. Here’s an example from the mypy documentation:

    from typing import overload, Sequence, TypeVar, Union
    T = TypeVar('T')

    class MyList(Sequence[T]):

        # The @overload definitions are just for the type checker,
        # and overwritten by the real implementation below.
        @overload
        def __getitem__(self, index: int) -> T:
            pass  # Don't put code here

        # All overloads and the implementation must be adjacent
        # in the source file, and overload order may matter:
        # when two overloads may overlap, the more specific one
        # should come first.
        @overload
        def __getitem__(self, index: slice) -> Sequence[T]:
            pass  # Don't put code here

        # The implementation goes last, without @overload.
        # It may or may not have type hints; if it does,
        # these are checked against the overload definitions
        # as well as against the implementation body.
        def __getitem__(self, index):
            # This is exactly the same as before.
            if isinstance(index, int):
                ...  # Return a T here
            elif isinstance(index, slice):
                ...  # Return a sequence of Ts here
            else:
                raise TypeError(...)

This was contributed by Naomi Seyfer.

Extended Callable Types

As an experimental mypy extension, you can specify Callable types that support keyword arguments, optional arguments, and more. Where you specify the arguments of a Callable, you can choose to supply just the type of a nameless positional argument, or an "argument specifier" representing a more complicated form of argument, using helpers such as Arg and DefaultArg that are defined in the mypy_extensions module. This allows one to more closely emulate the full range of possibilities given by Python’s def statement.

As an example, here's a complicated function definition and the corresponding Callable:

    from typing import Callable
    from mypy_extensions import (Arg, DefaultArg, NamedArg,
                                 DefaultNamedArg, VarArg, KwArg)

    def func(__a: int, # This convention is for nameless arguments
             b: int,
             c: int = 0,
             *args: int,
             d: int,
             e: int = 0,
             **kwargs: int) -> int:
        ...

    F = Callable[[int,            # Or Arg(int); an argument without a name
                  Arg(int, 'b'),  # Argument with name and type
                  DefaultArg(int, 'c'),  # Argument that may be omitted
                  VarArg(int),    # *args argument
                  NamedArg(int, 'd'),  # Must be passed as a keyword argument
                  DefaultNamedArg(int, 'e'),  # Like above, but may be omitted
                  KwArg(int)],    # **kwargs argument
                 int]  # Return type

    f: F = func

To use this, you need to install mypy_extensions:

    python3 -m pip install --upgrade mypy_extensions

This was contributed by Naomi Seyfer.

ClassVar

You can mark names intended to be used as class variables with ClassVar. In a pinch you can also use ClassVar in # type comments. This is supported by the typing module that ships with Python 3.6 (it is part of PEP 526). Example:

    from typing import ClassVar

    class C:
        x: int # instance variable
        y: ClassVar[int] # class variable
        z = None # type: ClassVar[int]

        def foo(self) -> None:
            self.x = 0  # OK
            self.y = 0  # Error: Cannot assign to class variable "y" via instance

    C.y = 0 # This is OK

This was contributed by Dominik Miedziński.

Quick Mode

Run mypy with the --quick-and-dirty (or just --quick) option to try a new experimental, unsafe variant of incremental mode. Quick mode is faster than regular incremental mode, because it only re-checks modules that were modified since their cache file was last written; regular incremental mode also re-checks all modules that depend on one or more modules that were re-checked.

Quick mode is unsafe because it may miss problems caused by a change in a dependency, so you should not use it in a continuous integration build, and you should perhaps run mypy without quick mode before landing commits. Quick mode updates the cache, but regular incremental mode ignores cache files written by quick mode.

Functional API for Enum

Mypy now also supports the functional API for defining an Enum. Previously you had to define a class deriving from Enum. Example (other forms are also supported):

    Color = Enum('Color', 'red blue green')

    Color.red    # Ok
    Color.black  # "Color" has no attribute "black"

Improvements to None Handling

Mypy now treats the None type more consistently between the default checking mode and the strict optional checking mode. Unlike previously, None is largely treated as a regular type, and things like List[None] mean what you’d expect: a list containing None values. Additionally, this idiom no longer generates an error:

    def f() -> None: ...

    def g() -> None:
        return f()  # Okay

Generators and Async Comprehensions (PEP 525 and PEP 530)

Python 3.6 allows coroutines defined with async def to be generators — they may contain yield statements and expressions. It also introduces a syntax for asynchronous comprehensions. Mypy now supports these features. Here’s an example from the mypy documentation:

    from typing import AsyncIterator

    async def gen() -> AsyncIterator[bytes]:
        lst = [b async for b in gen()]  # Inferred type is "List[bytes]"
        yield 'no way'  # Error: Incompatible types (got "str", expected "bytes")

This was contributed by Jelle Zijlstra.

Functions Returning Generic Functions

Mypy now understands that a function may return a generic function. A typical use case is a function that returns a decorator:

    from typing import TypeVar, Callable, Any, cast

    T = TypeVar('T', bound=Callable[..., Any])

    def logged(description: str) -> Callable[[T], T]:
        def decorator(f: T) -> T:
            def wrapper(*args, **kwargs):
                print('entering:', description)
                value = f(*args, **kwargs)
                print('leaving:', description)
            return cast(T, wrapper)
        return decorator

    @logged('system initialization')
    def init() -> None:
        print('do something')

    init(1)  # Too many arguments (signature is correctly preserved)

This was contributed by Naomi Seyfer.

Don’t Simplify Unions Containing Any

Unions containing Any, such as Union[int, Any], are now handled consistently. Previously in some cases mypy would simplify these into just Any, which is not equivalent. This could result in mypy failing to generate errors for buggy code. All operations performed on a union-typed value much be valid for every union item. For example, adding a string to a Union[int, Any] value is an error, since you can’t add a string and an integer. If the union was simplified to Any, mypy would not detect this error, since anything can be added to Any.

Prohibit Parametrizing Built-in Types

You no longer can write list[int] (with lower case l) in type annotations, as this doesn’t work at runtime. The also applies to dict and other types that have capitalized aliases in typing. Use List, Dict and so on (defined in typing) in type annotations. Note that this may require many changes to existing type annotations.

This was contributed by Ivan Levkivskyi.

More New Features

  • Support arbitrary member access on Type[Any] so that it’s possible to access class methods and attributes. (Daniel F Moisset)
  • Allow keyword arguments in __call__. (Ivan Levkivskyi)
  • Infer types from issubclass() calls, similar to isinstance(). (Max Moroz)
  • Always write the cache for incremental checking (unless cache_dir is /dev/null).
  • Use types.ModuleType instead of module, since the latter doesn’t exist at runtime. (Max Moroz)
  • Make it an error to use a class-attribute type variable outside a type. (Naomi Seyfer)
  • Allow importing NoReturn from typing. (Ivan Levkivskyi)
  • Reject TypeVar with only one constraint, since these never make sense. (Michael Lee)
  • Allow instantiation of Type[A], if A is abstract. (Ivan Levkivskyi)
  • Allow unions and nested tuples in except handlers. (Jelle Zijlstra)
  • Treat type as equivalent to Type[Any]. (Ivan Levkivskyi)
  • Make Type[T] compatible with Callable[..., T]. (Dominik Miedziński)
  • Always use the default Python 3 version to parse stub files, so that Python 3.6 features can be used in stub files.
  • Support default values for items in NamedTuple class definitions. (Jelle Zijlstra)
  • Reject generic types as the second argument to isinstance or issubclass, since they don’t work at runtime. (Max Moroz)
  • Allow variable as a type in isinstance. (Max Moroz)
  • Support Type[x] and type with isinstance. (Max Moroz)
  • Allow isinstance/issubclass with nested tuples. (Max Moroz)
  • Improve type inference of containers containing callables or overloads. (Ivan Levkivskyi)
  • Add support for type aliases typing.ChainMap, typing.Counter, typing.DefaultDict and typing.Deque. (Jelle Zijlstra)
  • Improvements to TypedDict (this feature is still incomplete and undocumented). (Roy Williams, Max Moroz and Ivan Levkivskyi)

Removed or Deprecated Features

  • The old parser that was available through --no-fast-parser is no longer supported.
  • Deprecate --strict-boolean and don’t enable it with --strict. (Max Moroz)
  • Drop Python 3.2 as a type checking target, since it has reached its end of life.
  • The stubs for sqlalchemy were dropped since they were incomplete and caused problems with common idioms. The stubs now live in a separate repository.

Notable Bugs Fixed

  • Fixes to various crashes. (Ivan Levkivskyi, Max Moroz and Jelle Zijlstra)
  • Fixes to incremental mode.
  • Fix bugs related to narrowing down unions using runtime checks.
  • Fix unions containing both bool and float.
  • Make union simplification be independent of the order of union items.
  • Fixes to confusing error messages due to dict displays. (Daniel F Moisset)
  • Fix subtyping for type variables whose upper bound is a union. (Max Moroz)
  • Fix invalid complaints about f-string expressions. (Ingmar Steen)
  • Fix isinstance check with nested unions. (Ivan Levkivskyi)
  • Work around random failures on writing cache files on Windows.
  • Fixes to % formatting. (Jelle Zijlstra)
  • Fixes to nested classes. (Elazar Gershuni)
  • Disallow generic Enums as they don’t work. (Jelle Zijlstra)

Other Changes

  • Require version 1.0.3 of typed-ast to run mypy.
  • Mypy developers on Windows must run certain git commands as Administrator. This only affects the mypy and typeshed git repositories.
  • Add stubtest, a tool for comparing a stub file against the implementation. (Jared Garst)
  • Speed up incremental mode. (Max Moroz)
  • Many updates to the library stubs in typeshed. A few highlights (there are too many contributors to list them all here; see below for a full contributor list):
    • Many new stubs.
    • Many stubs that had separate Python 2 and Python 3 variants were merged into shared Python 2+3 stubs to improve consistency and to simplify maintenance.

Acknowledgments

First of all, I’d like to thank my employer Dropbox for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Ben Kuhn
  • Daniel F Moisset
  • Dominik Miedziński
  • Elazar Gershuni
  • Ethan
  • Garrett
  • Ingmar Steen
  • Ivan Levkivskyi
  • Jared Garst
  • Jelle Zijlstra
  • Łukasz Langa
  • Max Moroz
  • Michael Lee
  • Naomi Seyfer
  • Ryan Gonzalez
  • Tobin Yehle

Additional thanks to all contributors to typeshed. In particular, I’d like to single out Jelle Zijlstra and David Euresti who have done a huge amount work on improving typeshed. Here’s a list of all typeshed contributors who contributed to this release:

  • Alvaro Caceres
  • Andrey Vlasovskikh
  • Antoine Reversat
  • Ashwini Chaudhary
  • Bertrand Bonnefoy-Claudet
  • Carl Meyer
  • Cooper Lees
  • David Euresti
  • David Wetterau
  • Dominik Miedziński
  • Eddie Antonio Santos
  • Ethan
  • George King
  • Günther Noack
  • Hong Minhee
  • Ivan Levkivskyi
  • James Saryerwinnie
  • Jan Hermann
  • Jelle Zijlstra
  • Jeremy Apthorp
  • jkleint
  • John Reese
  • Josiah Boning
  • Luka Sterbic
  • Łukasz Langa
  • Manuel Krebber
  • Marcin Kurczewski
  • Martijn Pieters
  • Matthias Kramm
  • Max Moroz
  • Michael Walter
  • Michał Masłowski
  • Naomi Seyfer
  • Nathan Henrie
  • Nikhil Marathe
  • nimin98
  • rchen152
  • Richard Hansen
  • Roy Williams
  • Sam Dunster
  • Samuel Colvin
  • Sebastian Meßmer
  • Semyon Proshev
  • Shengpei Zhang
  • Stefan Urbanek
  • Tadeu Manoel
  • Teddy Sudol
  • Thomas Ballinger
  • Tim Abbott

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)

Thursday, 2 March 2017

Mypy 0.501 Released

We’ve just uploaded mypy 0.501 to PyPI. This release adds new features, bug fixes and library stub updates. You can install it as follows:

    python3 -m pip install --upgrade mypy

Note: Remember that the package name is now mypy (no longer mypy-lang).

Switch to New Parser

Mypy now uses the typed_ast parser by default, which is more correct and significantly faster. The old parser is available for now with --no-fast-parser, but we are planning to remove it in the next release. Previously you had to enable the new parser explicitly through --fast-parser.

Note that mypy 0.501 requires a new, backwards incompatible version of typed_ast (1.0.1 or higher). This should be taken care of automatically by the package dependencies if you install mypy using pip .

Default to Python 3.6 Mode

Supported Python 3.6 features are now enabled by default, such as variable annotations (PEP 526), f-strings (PEP 498), and underscores in numeric literals (PEP 515).

Simple Metaclasses

Mypy now supports simple metaclasses. Mypy can look up attributes and methods defined in the metaclass, but mypy still doesn’t understand more dynamic metaclass behavior. For example, this works now:

    class Meta(type):
        def __add__(self, x: str) -> str:
            return 'a' + x

    class C(metaclass=Meta):
        ...

    print(C + 'x')  # Okay

You can also use metaclasses in Python 2:

    ...

    class C(object):
        __metaclass__ = Meta

This was contributed by Elazar Gershuni with help from Tom Manderson.

Warn about Missing Return Statements by Default

The --warn-no-return flag is now on by default, which means mypy will warn about functions missing a return statement on one or more code paths, unless the function’s return type is None or Any. For example:

        def f(n: int) -> int:  # Missing return statement
            if n > 3:
                return n + 1

Mypy currently doesn’t warn about empty function bodies or bodies that only have an ellipsis (…), since these are sometimes used for abstract methods and aren’t necessarily a problem. The option can generate false positives if you rely on the implicit None return value when execution falls off the end of a function. This behavior can be disabled globally with --no-warn-no-return or on a per-module basis with the warn_no_return config option.

This was contributed by Reid Barton.

NoReturn

You can now annotate functions that never return using mypy_extensions.NoReturn. Mypy will check that those functions do, in fact, not return, and will understand that they cause code paths to never return when called. Mypy now also knows that certain library functions such as sys.exit() don’t return. For example:

    from mypy_extensions import NoReturn
    def bad_number() -> NoReturn:
        raise RuntimeError("Bad number.")

    def f(n: int) -> int:  # okay
        if n > 3:
            return n + 1
        else:
            bad_number()

Install mypy_extensions using pip to use NoReturn:

    python3 -m pip install --upgrade mypy-extensions  # For Python 3
    pip install --upgrade mypy-extensions  # For Python 2

Leaving Out Redundant Generic[T]

You can omit redundant Generic[T] in the base class list if you have another reference to T within base classes. For example, this now works:

    from typing import TypeVar, Iterable

    T = TypeVar('T')

    class MyCollection(Iterable[T]):
        ...
        def __iter__(self) -> Iterator[T]: ...

    def collection_to_list(c: MyCollection[int]) -> List[int]:
        return list(c)

For previous versions of mypy you would have needed to write the class definition like this:

    ...
    class MyCollection(Iterable[T], Generic[T]):
        ...

This was contributed by Ivan Levkivskyi.

Better Checking of Assignments in Class Body

Mypy finally can check if assignments within a class body are compatible with base classes. Now this will get flagged as an error:

    class A:
        x = 1

    class B(A):
        x = 'a'  # Error

This was contributed by TrueBrain.

Strict Mode

Mypy now has a strict mode which turns on most flags that add additional checks. Strict mode can be enabled with --strict. Some of these checks may be opinionated (like --strict-boolean). New checks will be added to strict mode over time.

Invertible Options

Most boolean flags can now be explicitly inverted. See mypy --help for the names of the inverted options. For example, the inverse of --disallow-untyped-defs is --allow-untyped-defs. This is useful when an option is enabled in the config file but you want to disable it for a specific run without editing the config file.

More New Features

  • Read configuration from setup.cfg if mypy.ini not present. (Dominik Miedziński)
  • Support type comments for with and for statements. (Alex Frieder)
  • Support type comments for async with and async for statements. (Alex Frieder)
  • Support E[<str>] when E is an Enum type.
  • Support @no_type_check in fast parser.
  • api.run now accepts a list of command-line arguments and returns a 3-tuple, including exit status. (Jacques de Hooge)
  • Add --strict-boolean flag that warns about non-bool values used as conditions. (Alex Frieder)
  • Add --warn-return-any flag that warns about returning a value with Any type. (lincolnq)
  • Add support for __getattribute__. (Jelle Zijlstra)
  • Issue warning for possibly malformed asserts. (Alex Frieder)
  • Make most fast parser errors non-fatal.
  • Improve fast parser error messages.
  • TypedDict improvements (TypedDict is still undocumented and experimental). (Ivan Levkivskyi)

Notable Bugs Fixed

  • Detect missing imports of List, Dict and Set. (Ivan Levkivskyi)
  • Don't complain about method override when the base class method has type Any. (Naomi Seyfer)
  • Fix stubgen crash on an extension module containing an alias of object. (Jelle Zijlstra)
  • Fix crash on access to local classes. (Ivan Levkivskyi)
  • Fix crash on expressions like "%d%d" % (*x,).
  • Generate errors on classmethod and staticmethod override conflicts. (Alex Frieder)
  • Fix crash on invalid type comments/annotations. (Ivan Levkivskyi)
  • Fix crash when assigning to a variable with a partial type. (Alex Frieder)
  • Treat varargs as valid context in default lambda argument. (Elazar Gershuni)
  • Always process modules/packages given with -m /-p.
  • Generate error when using deleted variable in a conditional. (Alex Frieder)
  • Make fast parser able to parse machine-generated code with deep nesting. (Łukasz Langa)
  • Fix bug with type variables defined in different scopes being treated as the same type variable. (with help from Ivan Levkivskyi)
  • Generate error on Any(...) which fails at runtime. (Alex Frieder)
  • Check for duplicate names in function definitions when using the fast parser. (Alex Frieder)
  • Disallow duplicate type comments/annotations in fast parser. (Alex Frieder)
  • Fix error context for indexed assignment with an annotation. (Jelle Zijlstra)
  • Fix return type of dict.get in strict optional mode.
  • Type check assert messages in fast parser. (Alex Frieder)
  • Fix warning about missing return statements in async def functions. (Jelle Zijlstra)
  • Order of union items no longer affects type sameness. (Alex Frieder)
  • Propagate inference type context through reveal_type for more precise revealed types.
  • Allow unicode docstrings in Python 2. (Jelle Zijlstra)
  • Fix several Python 2 bugs in fast parser.
  • Fix some additional crashes.

Other Changes

Acknowledgments

Thanks to all mypy contributors who contributed to this release:

  • Adrien Chauve
  • Alex Frieder
  • Brandon W Maister
  • Dominik Miedziński
  • Eduard-Cristian Stefan
  • Elazar Gershuni
  • Ivan Levkivskyi
  • Jacques de Hooge
  • Jelle Zijlstra
  • lincolnq
  • Łukasz Langa
  • Naomi Seyfer
  • Tom Manderson
  • TrueBrain
  • Wojciech Kaczmarek

Additional thanks to these contributors to typeshed:

  • 5j9
  • Adam Marszałek
  • Alex Frieder
  • Alexey (forestbiiird)
  • Andrey Vlasovskikh
  • aostiles
  • Avy Faingezicht
  • Bertrand Bonnefoy-Claudet
  • David Euresti
  • Dominik Miedziński
  • Dylan Jenkinson
  • Eklavya Sharma
  • Eric Moyer
  • Ethan (ethanhs)
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • lincolnq
  • Lucas Wiman
  • Łukasz Langa
  • Matt Kimball
  • Matthias Kramm
  • Mike Patek
  • Mohab Usama
  • rchen152
  • Richard Hansen
  • Roy Williams
  • Russ Allbery
  • Sunghyun Hwang
  • syrrim
  • Tim Abbott
  • Tom Manderson
  • Tomasz Elendt
  • Valérian Rousset

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)

Friday, 13 January 2017

Mypy 0.470 Released

We’ve just uploaded mypy 0.470 to PyPI. This release adds new features, bug fixes and library stub updates. You can install it as follows:

    python3 -m pip install mypy

Note: The package name is now “mypy” (no longer mypy-lang). If you have an older mypy version installed, remove it first, before installing the new package:

    python3 -m pip uninstall mypy-lang

Good news: New Package Name and Version Style

We have finally obtained ownership of the “mypy” package name on PyPI. In order to avoid conflicts with (hypothetical) users of the previous occupant of the “mypy” package name we’re changing the style of version numbers we’re using (at least until 1.0 comes along) — the new version is now 0.470. If you are using requirements.txt files, the proper incantation is now:

    mypy==0.470

The old “mypy-lang” package will no longer be upgraded (the last version there is 0.4.6).

New Import Handling Options

The “silent imports” functionality, useful to avoid noisy errors about missing modules when annotating a large codebase, has been refactored and improved. There are now two separate flags (see the online docs for the full scoop):

  • --follow-imports=arg takes an argument which must be one of the following:
    • normal — the default behavior
    • silent — tries to find, parse and check imported modules (same as normal) but suppresses all errors for those modules that were not given on the command line
    • skip — roughly equivalent to the old --silent-imports flag
    • error — roughly equivalent to the old --almost-silent flag
  • --ignore-missing-imports suppresses error messages about imports that cannot be resolved at all. This was previously implied by --silent-imports unless --almost-silent was given.

Fast Parser Supported on Windows

The --fast-parser option now works also on Windows. It’s faster than the default parser and supports more Python syntax, including new syntax introduced in Python 3.6. The required package typed_ast is now installed by default as a mypy dependency (see below for more). We are planning to deprecate the current default parser in a future mypy release. (Also, the mypy project now runs tests for Windows on Appveyor.)

Improved Type Inference for lambda Expressions

A lambda expression without type context is now inferred as a Callable returning whatever type can be inferred for the expression in the body of the lambda. Previously such lambda expressions were given type Any . Note that lambda ...: None is now considered to have type Callable[..., None] which cannot be called to provide a value, only as a procedure call. The arguments are given type Any, so that e.g. the following two are now inferring the same type for a, i.e. Callable[[Any], List[Any]]:

    a = lambda x: [x]

    def a(x) -> List[Any]: return [x]

(Contributed by Elazar Gershuni)

Support for callable()

A condition using e.g. if callable(x) now causes mypy to infer a Callable type for x in the block controlled by the condition, and a non-Callable type in the block controlled by its negation. For example, this now type-checks:

    def maybecall(x: Union[int, Callable[[], int]]) -> int:
        if callable(x):
            return x()
        else:
            return x

(Contributed by Alex Frieder)

New Package Dependencies

Mypy now has automatic dependencies on typing and typed_ast. Previously typed_ast had to be manually installed using pip and typing was bundled with the mypy package.

Note that typed_ast requires Python 3.5+ on Windows. We’re dropping support for running mypy on Windows using Python 3.4 or older. (But checking still supports 2.7 and 3.2+ targets, using the --python-version flag.)

Notable Bugs Fixed

  • Callable type compatibility checking rewritten with several improvements (Naomi Seyfer)
  • Fix Callable arguments in overloaded functions
  • Fix lambda inference with simple Union contexts
  • Fix spurious warnings caused by type redeclarations (Łukasz Langa)
  • Fix handling of empty tuple type aliases
  • Fix crash on retrieving TypeVar from class (Tom Manderson)
  • Ensure required keyword-only arguments are provided (Naomi Seyfer)
  • Consider types like Tuple[t1, t2, ...] more consistently as subtypes of plain tuple (Ivan Levkivskyi)
  • Compare cached options to current per-file options when using --incremental
  • Fix NamedTuple defined in a method

New Experimental Features

There is work-in-progress support for dictionaries with literal string keys (TypedDict). Different keys can have items with different types, and mypy will figure out the right type for each key. For example, it will be possible to give a precise type for this dictionary:

    d = {'path': 'file.txt', 'size': 1254}

TypedDict is still very tentative and there’s no official documentation yet. You need to import TypedDict from mypy_extensions; use pip install -U mypy_extensions to install it. (Contributed by David Foster)

Descriptors are now supported. We're still not quite sure how well they work in practice. (Contributed by Calen Pennington)

Other Changes in This Release

  • Turn --hide-error-context on by default to generate more concise output from mypy. Add --show-error-context to re-enable the old behavior.
  • Arguments starting with __ (two underscores) are now positional-only. (Naomi Seyfer)
  • Add an API for calling mypy directly from another python application. (Jacques de Hooge)
  • Add ignore_errors config option to selectively ignore all non-fatal errors in some files.
  • Add Python 3 cheat sheet. (Ethan Smith)
  • Speed up the mypy test suite. (Łukasz Langa)
  • Miscellaneous other bug fixes.
  • Many updates to the library stubs in typeshed.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Aleksander Vognild Burkow
  • Alex Frieder
  • Bertrand Bonnefoy-Claudet
  • Calen Pennington
  • Chris Oelmueller
  • David Euresti
  • David Foster
  • Elazar Gershuni
  • Ethan (ethanhs)
  • Ivan Levkivskyi
  • Jacques de Hooge
  • Jakub Stasiak
  • Jelle Zijlstra
  • Josiah Boning
  • Juanvulcano
  • Łukasz Langa
  • Naomi Seyfer
  • Roy Williams
  • Tetsuya Morimoto
  • Tom Manderson
  • TrueBrain

Additional thanks to these contributors to typeshed:

  • Alex Frieder
  • Alex Jurkiewicz
  • Anders Kaseorg
  • Bertrand Bonnefoy-Claudet
  • Cadel Watson
  • Calen Pennington
  • Daisuke Miyakawa
  • Danny Weinberg
  • David Euresti
  • Eric Moyer
  • George King
  • gotyaoi
  • Henri Dwyer
  • Hugo (hugovk)
  • Jason Fried
  • Jelle Zijlstra
  • Jon Dufresne
  • Joseph H Garvin
  • Josiah Boning
  • Kosaka Masayuki
  • lionel-github
  • Luiz Menezesf
  • Łukasz Langa
  • Madeleine Thompson
  • Mateusz Kurek
  • Matthias Kramm
  • Michael Lee
  • Mohab Usama
  • Naomi Seyfer
  • Nicolas Duchastel de Montrouge
  • Onno Kortmann
  • Peter Amstutz
  • Philip House
  • Reverb Chu
  • Richard Eames
  • Roy Williams
  • Ryan C. Thompson
  • rymdhund
  • Simon Ekstrand
  • Thomas Aynaud
  • Thomas Cellerier
  • Tom Manderson
  • TrueBrain
  • Wesley Bowman
  • z33ky

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)

Monday, 21 November 2016

Mypy 0.4.6 Released

We’ve just uploaded mypy 0.4.6 to PyPI. This release adds many new features, bug fixes and library stub updates. As always, you can install it as follows, being careful to install mypy-lang, not mypy:

    python3 -m pip install --upgrade mypy-lang

Generic Type Aliases

You can now define type aliases that are themselves generic types. For example, consider this definition of Vector:

    from typing import TypeVar, List

    T = TypeVar('T')

    Vector = Tuple[T, T, T]

Now Vector[float], for example, is equivalent to Tuple[float, float, float]. Generic type aliases can make your type annotations shorter and easier to read. The following example defines two equivalent functions. The first one uses a generic type alias to simplify the signature:

    def negate1(vecs: List[Vector[float]]) -> List[Vector[float]]:
        return [(-x, -y, -z) for x, y, z in vecs]

    def negate2(vecs: List[Tuple[float, float, float]]
               ) -> List[Tuple[float, float, float]]:
        return [(-x, -y, -z) for x, y, z in vecs]

Check out the documentation for more details.

This was contributed by Ivan Levkivskyi and he also implemented the related changes to typing.

Detecting Missing Return Statements

Mypy now can generate an error if a function is missing a return statement, unless the return type is None or Any. Use the --warn-no-return command line option to enable this, or the warn_no_return config file setting. When the option is set, mypy warns about this function, since it doesn’t return anything if the condition is false:

    def f(n: int) -> int:
        if n > 3:
            return n + 1  # Missing return statement

Mypy currently doesn’t warn about empty function bodies or bodies that only have an ellipsis (), since these are sometimes used for abstract methods and aren’t necessarily a problem. The option can generate false positives if you rely on the implicit None return value when execution falls off the end of a function.

This was contributed by Reid Barton.

Improved Type Checking of Import Cycles

Previous versions of mypy were prone to giving errors like ‘Cannot determine type of “foo”’ if your program had import cycles. Another common problem was that innocent-looking changes inside import cycles would cause mypy to emit a new set of unrelated errors.

This release has two changes that should resolve most of these issues. They are a bit technical, and it’s not important to understand how they work in order to enjoy the benefits. Anyway, the two following subsections have a summary for those who are interested in the nitty-gritty internal details. Feel free to skip them. For even more details, have a look at these PRs: #2264 and #2167

Deferred Checking within Import Cycles

Previously mypy type-checked each module only once, a single module at a time. This could be a problem with import cycles. For example, module a could depend on inferred types in b, and b could depend on types of a. No linear ordering of modules would produce clean output, unless the program had type annotations for all the problematic variables — this was only a problem for things whose types weren’t immediately obvious to mypy. Mypy now type-checks an import cycle in two passes. If there is a reference to a variable in another module that mypy hasn’t yet processed, mypy defers the type-checking of the surrounding function, and finally runs another type-checking pass for only deferred functions.

Programs without cycles aren’t affected, since mypy has type-checked single modules in two passes for some time. We’ve now generalized it to multiple modules in a cycle.

Stable Processing Order within Import Cycles

The erratic mypy behavior mentioned earlier was caused by changes in the order of processing modules within a cycle. Mypy goes through each module in a cycle in a linear order, and some errors are only triggered for certain orderings. In previous versions of mypy a small change in a program could trigger big changes in the module processing order. Mypy now better approximates the runtime module initialization order, which is usually pretty stable.

For example, if module a has a statement like from b import f, module b likely will be initialized before module a at runtime, and so mypy will type-check b before a. On the other hand, if an import statement is within an if TYPE_CHECKING: block, mypy will not use this heuristic, since typing.TYPE_CHECKING is false at runtime and the import statement won’t affect runtime behavior.

Self Types (Experimental)

[This feature is experimental — the implementation still has significant limitations or bugs, and the feature may change in future mypy versions in incompatible ways.]

Sometimes you have a method that returns a value with exactly the same type as self. Previously there was no way to do this. Now you can annotate the self argument with a type variable to express this. Here’s an example taken from the documentation:

    from typing import TypeVar

    T = TypeVar('T', bound='Shape')

    class Shape:
        def set_scale(self: T, scale: float) -> T:
            self.scale = scale
            return self

    class Circle(Shape):
        def set_radius(self, r: float) -> 'Circle':
            self.radius = r
            return self

    class Square(Shape):
        def set_width(self, w: float) -> 'Square':
            self.width = w
            return self

    circle = Circle().set_scale(0.5).set_radius(2.7)  # type: Circle
    square = Square().set_scale(0.5).set_width(3.2)  # type: Square

By declaring self as a type variable, the set_scale can return a Circle when called on a Circle object and a Square when called on a Square, while only having a single method definition in the Shape base class.

For a class method, you can also now use Type[T] in a similar way as the annotation for the cls argument.

This feature was contributed by Elazar Gershuni.

Type Applications

Mypy now supports a type application syntax for user-defined generic classes. For example, consider a generic Stack class:

    from typing import TypeVar, Generic

    T = TypeVar('T')

    class Stack(Generic[T]):
        ...

When constructing a Stack instance, you can use the type application syntax Stack[<type>] to specify the type arguments. This constructs a Stack[int] instance using a type application:

    stack = Stack[int]()

Previously you had to use a type annotation, which is more verbose:

    stack = Stack()  # type: Stack[int]

This is now practical since the latest typing implementation caches the result of Stack[int], so it will be quick enough even for a frequently called function. This was actually supported in early mypy releases, but it has been unsupported for a while, primarily due to performance concerns.

You still have to use the type annotation syntax for standard-library classes such as list and Queue since these classes don’t support the indexing operator.

This was contributed by Ivan Levkivskyi. He also implemented the related changes to typing.

New NamedTuple Syntax (Python 3.6)

You can use a new syntax for named tuples with Python 3.6b1 or newer:

    from typing import NamedTuple

    class Point(NamedTuple):
        x: int
        y: int

    p = Point(x=1, y=2)

Run mypy with --fast-parser --python-version 3.6 to use this. You’ll need to install a recent version of typed_ast (pip3 install --upgrade typed_ast).

This was contributed by Ivan Levkivskyi.

Config File Changes

The filename patterns in the mypy configuration file were changed to module name patterns. You’ll have to update your config files if you use this feature.

Here’s an example mypy.ini file that rejects functions without a type annotation in the frobnicate package but allows them elsewhere:

    [mypy-frobnicate.*]
    disallow_untyped_defs = True

You can configure additional module search path entries in the mypy configuration file through the mypy_path configuration file option. This can be useful with local stub files stored in a separate directory, for example. Previously you had to use the MYPYPATH environment variable. mypy_path was contributed by Filip Figiel.

New Command Line Options

  • Add --cobertura-xml-report for outputting Cobertura XML reports with type checking coverage information (Roy Williams).
  • Add support for using a custom typeshed directory (--custom-typeshed-dir).
  • Add --junit-xml=PATH option for generating JUnit XML files with type checking results.
  • Add --find-occurrences that finds all references to a class member using static type information (experimental).
  • Stubgen now supports --fast-parser (Ivan Levkivskyi).

Other Changes in This Release

  • Improvements to --strict-optional (please go ahead and try it — it now works pretty well).
  • Allow subclassing Tuple[...] outside stubs again (Ivan Levkivskyi).
  • Show column number for reveal_type() expressions.
  • Various updates to the bundled typing module.
  • Support conditional import in functions/classes (Mark Laws).
  • Support matrix multiplication operator @ (Elazar Gershuni).
  • Several miscellaneous bug fixes.
  • Several documentation improvements.
  • Lots of updates to the library stubs in typeshed.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Jurkiewicz
  • Byungwoo Jeon
  • Calen Pennington
  • Chris Lamb
  • David Foster
  • Elazar Gershuni
  • Filip Figiel
  • Herbert Ho
  • Ivan Levkivskyi
  • Kevin Yap
  • Mark Laws
  • Reid Barton
  • Roy Williams
  • Ryan Gonzalez
  • Ulric Aleksandrov

Additional thanks to these contributors to typeshed:

  • Alvaro Caceres
  • Calen Pennington
  • David Percy
  • Dima Gerasimov
  • Eklavya Sharma
  • Evan Hubinger
  • Gerhard Hagerer
  • Hong Minhee
  • Jakub Stasiak
  • Jon Dufresne
  • Jordan Pittier
  • Joshua Smock
  • Kai Lautaportti
  • Matthias Kramm
  • Onno Kortmann
  • Reiner Gerecke
  • Ruud van Asseldonk
  • Ryan C. Thompson
  • Sebastian Meßmer
  • TrueBrain
  • Xavier Mehrenberger
  • Yegor Roganov
  • Joseph H Garvin
  • nobuggy
  • paavoap

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)