17db96d56Sopenharmony_ci.. _devmode: 27db96d56Sopenharmony_ci 37db96d56Sopenharmony_ciPython Development Mode 47db96d56Sopenharmony_ci======================= 57db96d56Sopenharmony_ci 67db96d56Sopenharmony_ci.. versionadded:: 3.7 77db96d56Sopenharmony_ci 87db96d56Sopenharmony_ciThe Python Development Mode introduces additional runtime checks that are too 97db96d56Sopenharmony_ciexpensive to be enabled by default. It should not be more verbose than the 107db96d56Sopenharmony_cidefault if the code is correct; new warnings are only emitted when an issue is 117db96d56Sopenharmony_cidetected. 127db96d56Sopenharmony_ci 137db96d56Sopenharmony_ciIt can be enabled using the :option:`-X dev <-X>` command line option or by 147db96d56Sopenharmony_cisetting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. 157db96d56Sopenharmony_ci 167db96d56Sopenharmony_ciSee also :ref:`Python debug build <debug-build>`. 177db96d56Sopenharmony_ci 187db96d56Sopenharmony_ciEffects of the Python Development Mode 197db96d56Sopenharmony_ci====================================== 207db96d56Sopenharmony_ci 217db96d56Sopenharmony_ciEnabling the Python Development Mode is similar to the following command, but 227db96d56Sopenharmony_ciwith additional effects described below:: 237db96d56Sopenharmony_ci 247db96d56Sopenharmony_ci PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python3 -W default -X faulthandler 257db96d56Sopenharmony_ci 267db96d56Sopenharmony_ciEffects of the Python Development Mode: 277db96d56Sopenharmony_ci 287db96d56Sopenharmony_ci* Add ``default`` :ref:`warning filter <describing-warning-filters>`. The 297db96d56Sopenharmony_ci following warnings are shown: 307db96d56Sopenharmony_ci 317db96d56Sopenharmony_ci * :exc:`DeprecationWarning` 327db96d56Sopenharmony_ci * :exc:`ImportWarning` 337db96d56Sopenharmony_ci * :exc:`PendingDeprecationWarning` 347db96d56Sopenharmony_ci * :exc:`ResourceWarning` 357db96d56Sopenharmony_ci 367db96d56Sopenharmony_ci Normally, the above warnings are filtered by the default :ref:`warning 377db96d56Sopenharmony_ci filters <describing-warning-filters>`. 387db96d56Sopenharmony_ci 397db96d56Sopenharmony_ci It behaves as if the :option:`-W default <-W>` command line option is used. 407db96d56Sopenharmony_ci 417db96d56Sopenharmony_ci Use the :option:`-W error <-W>` command line option or set the 427db96d56Sopenharmony_ci :envvar:`PYTHONWARNINGS` environment variable to ``error`` to treat warnings 437db96d56Sopenharmony_ci as errors. 447db96d56Sopenharmony_ci 457db96d56Sopenharmony_ci* Install debug hooks on memory allocators to check for: 467db96d56Sopenharmony_ci 477db96d56Sopenharmony_ci * Buffer underflow 487db96d56Sopenharmony_ci * Buffer overflow 497db96d56Sopenharmony_ci * Memory allocator API violation 507db96d56Sopenharmony_ci * Unsafe usage of the GIL 517db96d56Sopenharmony_ci 527db96d56Sopenharmony_ci See the :c:func:`PyMem_SetupDebugHooks` C function. 537db96d56Sopenharmony_ci 547db96d56Sopenharmony_ci It behaves as if the :envvar:`PYTHONMALLOC` environment variable is set to 557db96d56Sopenharmony_ci ``debug``. 567db96d56Sopenharmony_ci 577db96d56Sopenharmony_ci To enable the Python Development Mode without installing debug hooks on 587db96d56Sopenharmony_ci memory allocators, set the :envvar:`PYTHONMALLOC` environment variable to 597db96d56Sopenharmony_ci ``default``. 607db96d56Sopenharmony_ci 617db96d56Sopenharmony_ci* Call :func:`faulthandler.enable` at Python startup to install handlers for 627db96d56Sopenharmony_ci the :const:`SIGSEGV`, :const:`SIGFPE`, :const:`SIGABRT`, :const:`SIGBUS` and 637db96d56Sopenharmony_ci :const:`SIGILL` signals to dump the Python traceback on a crash. 647db96d56Sopenharmony_ci 657db96d56Sopenharmony_ci It behaves as if the :option:`-X faulthandler <-X>` command line option is 667db96d56Sopenharmony_ci used or if the :envvar:`PYTHONFAULTHANDLER` environment variable is set to 677db96d56Sopenharmony_ci ``1``. 687db96d56Sopenharmony_ci 697db96d56Sopenharmony_ci* Enable :ref:`asyncio debug mode <asyncio-debug-mode>`. For example, 707db96d56Sopenharmony_ci :mod:`asyncio` checks for coroutines that were not awaited and logs them. 717db96d56Sopenharmony_ci 727db96d56Sopenharmony_ci It behaves as if the :envvar:`PYTHONASYNCIODEBUG` environment variable is set 737db96d56Sopenharmony_ci to ``1``. 747db96d56Sopenharmony_ci 757db96d56Sopenharmony_ci* Check the *encoding* and *errors* arguments for string encoding and decoding 767db96d56Sopenharmony_ci operations. Examples: :func:`open`, :meth:`str.encode` and 777db96d56Sopenharmony_ci :meth:`bytes.decode`. 787db96d56Sopenharmony_ci 797db96d56Sopenharmony_ci By default, for best performance, the *errors* argument is only checked at 807db96d56Sopenharmony_ci the first encoding/decoding error and the *encoding* argument is sometimes 817db96d56Sopenharmony_ci ignored for empty strings. 827db96d56Sopenharmony_ci 837db96d56Sopenharmony_ci* The :class:`io.IOBase` destructor logs ``close()`` exceptions. 847db96d56Sopenharmony_ci* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to 857db96d56Sopenharmony_ci ``True``. 867db96d56Sopenharmony_ci 877db96d56Sopenharmony_ciThe Python Development Mode does not enable the :mod:`tracemalloc` module by 887db96d56Sopenharmony_cidefault, because the overhead cost (to performance and memory) would be too 897db96d56Sopenharmony_cilarge. Enabling the :mod:`tracemalloc` module provides additional information 907db96d56Sopenharmony_cion the origin of some errors. For example, :exc:`ResourceWarning` logs the 917db96d56Sopenharmony_citraceback where the resource was allocated, and a buffer overflow error logs 927db96d56Sopenharmony_cithe traceback where the memory block was allocated. 937db96d56Sopenharmony_ci 947db96d56Sopenharmony_ciThe Python Development Mode does not prevent the :option:`-O` command line 957db96d56Sopenharmony_cioption from removing :keyword:`assert` statements nor from setting 967db96d56Sopenharmony_ci:const:`__debug__` to ``False``. 977db96d56Sopenharmony_ci 987db96d56Sopenharmony_ciThe Python Development Mode can only be enabled at the Python startup. Its 997db96d56Sopenharmony_civalue can be read from :data:`sys.flags.dev_mode <sys.flags>`. 1007db96d56Sopenharmony_ci 1017db96d56Sopenharmony_ci.. versionchanged:: 3.8 1027db96d56Sopenharmony_ci The :class:`io.IOBase` destructor now logs ``close()`` exceptions. 1037db96d56Sopenharmony_ci 1047db96d56Sopenharmony_ci.. versionchanged:: 3.9 1057db96d56Sopenharmony_ci The *encoding* and *errors* arguments are now checked for string encoding 1067db96d56Sopenharmony_ci and decoding operations. 1077db96d56Sopenharmony_ci 1087db96d56Sopenharmony_ci 1097db96d56Sopenharmony_ciResourceWarning Example 1107db96d56Sopenharmony_ci======================= 1117db96d56Sopenharmony_ci 1127db96d56Sopenharmony_ciExample of a script counting the number of lines of the text file specified in 1137db96d56Sopenharmony_cithe command line:: 1147db96d56Sopenharmony_ci 1157db96d56Sopenharmony_ci import sys 1167db96d56Sopenharmony_ci 1177db96d56Sopenharmony_ci def main(): 1187db96d56Sopenharmony_ci fp = open(sys.argv[1]) 1197db96d56Sopenharmony_ci nlines = len(fp.readlines()) 1207db96d56Sopenharmony_ci print(nlines) 1217db96d56Sopenharmony_ci # The file is closed implicitly 1227db96d56Sopenharmony_ci 1237db96d56Sopenharmony_ci if __name__ == "__main__": 1247db96d56Sopenharmony_ci main() 1257db96d56Sopenharmony_ci 1267db96d56Sopenharmony_ciThe script does not close the file explicitly. By default, Python does not emit 1277db96d56Sopenharmony_ciany warning. Example using README.txt, which has 269 lines: 1287db96d56Sopenharmony_ci 1297db96d56Sopenharmony_ci.. code-block:: shell-session 1307db96d56Sopenharmony_ci 1317db96d56Sopenharmony_ci $ python3 script.py README.txt 1327db96d56Sopenharmony_ci 269 1337db96d56Sopenharmony_ci 1347db96d56Sopenharmony_ciEnabling the Python Development Mode displays a :exc:`ResourceWarning` warning: 1357db96d56Sopenharmony_ci 1367db96d56Sopenharmony_ci.. code-block:: shell-session 1377db96d56Sopenharmony_ci 1387db96d56Sopenharmony_ci $ python3 -X dev script.py README.txt 1397db96d56Sopenharmony_ci 269 1407db96d56Sopenharmony_ci script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> 1417db96d56Sopenharmony_ci main() 1427db96d56Sopenharmony_ci ResourceWarning: Enable tracemalloc to get the object allocation traceback 1437db96d56Sopenharmony_ci 1447db96d56Sopenharmony_ciIn addition, enabling :mod:`tracemalloc` shows the line where the file was 1457db96d56Sopenharmony_ciopened: 1467db96d56Sopenharmony_ci 1477db96d56Sopenharmony_ci.. code-block:: shell-session 1487db96d56Sopenharmony_ci 1497db96d56Sopenharmony_ci $ python3 -X dev -X tracemalloc=5 script.py README.rst 1507db96d56Sopenharmony_ci 269 1517db96d56Sopenharmony_ci script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> 1527db96d56Sopenharmony_ci main() 1537db96d56Sopenharmony_ci Object allocated at (most recent call last): 1547db96d56Sopenharmony_ci File "script.py", lineno 10 1557db96d56Sopenharmony_ci main() 1567db96d56Sopenharmony_ci File "script.py", lineno 4 1577db96d56Sopenharmony_ci fp = open(sys.argv[1]) 1587db96d56Sopenharmony_ci 1597db96d56Sopenharmony_ciThe fix is to close explicitly the file. Example using a context manager:: 1607db96d56Sopenharmony_ci 1617db96d56Sopenharmony_ci def main(): 1627db96d56Sopenharmony_ci # Close the file explicitly when exiting the with block 1637db96d56Sopenharmony_ci with open(sys.argv[1]) as fp: 1647db96d56Sopenharmony_ci nlines = len(fp.readlines()) 1657db96d56Sopenharmony_ci print(nlines) 1667db96d56Sopenharmony_ci 1677db96d56Sopenharmony_ciNot closing a resource explicitly can leave a resource open for way longer than 1687db96d56Sopenharmony_ciexpected; it can cause severe issues upon exiting Python. It is bad in 1697db96d56Sopenharmony_ciCPython, but it is even worse in PyPy. Closing resources explicitly makes an 1707db96d56Sopenharmony_ciapplication more deterministic and more reliable. 1717db96d56Sopenharmony_ci 1727db96d56Sopenharmony_ci 1737db96d56Sopenharmony_ciBad file descriptor error example 1747db96d56Sopenharmony_ci================================= 1757db96d56Sopenharmony_ci 1767db96d56Sopenharmony_ciScript displaying the first line of itself:: 1777db96d56Sopenharmony_ci 1787db96d56Sopenharmony_ci import os 1797db96d56Sopenharmony_ci 1807db96d56Sopenharmony_ci def main(): 1817db96d56Sopenharmony_ci fp = open(__file__) 1827db96d56Sopenharmony_ci firstline = fp.readline() 1837db96d56Sopenharmony_ci print(firstline.rstrip()) 1847db96d56Sopenharmony_ci os.close(fp.fileno()) 1857db96d56Sopenharmony_ci # The file is closed implicitly 1867db96d56Sopenharmony_ci 1877db96d56Sopenharmony_ci main() 1887db96d56Sopenharmony_ci 1897db96d56Sopenharmony_ciBy default, Python does not emit any warning: 1907db96d56Sopenharmony_ci 1917db96d56Sopenharmony_ci.. code-block:: shell-session 1927db96d56Sopenharmony_ci 1937db96d56Sopenharmony_ci $ python3 script.py 1947db96d56Sopenharmony_ci import os 1957db96d56Sopenharmony_ci 1967db96d56Sopenharmony_ciThe Python Development Mode shows a :exc:`ResourceWarning` and logs a "Bad file 1977db96d56Sopenharmony_cidescriptor" error when finalizing the file object: 1987db96d56Sopenharmony_ci 1997db96d56Sopenharmony_ci.. code-block:: shell-session 2007db96d56Sopenharmony_ci 2017db96d56Sopenharmony_ci $ python3 script.py 2027db96d56Sopenharmony_ci import os 2037db96d56Sopenharmony_ci script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> 2047db96d56Sopenharmony_ci main() 2057db96d56Sopenharmony_ci ResourceWarning: Enable tracemalloc to get the object allocation traceback 2067db96d56Sopenharmony_ci Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> 2077db96d56Sopenharmony_ci Traceback (most recent call last): 2087db96d56Sopenharmony_ci File "script.py", line 10, in <module> 2097db96d56Sopenharmony_ci main() 2107db96d56Sopenharmony_ci OSError: [Errno 9] Bad file descriptor 2117db96d56Sopenharmony_ci 2127db96d56Sopenharmony_ci``os.close(fp.fileno())`` closes the file descriptor. When the file object 2137db96d56Sopenharmony_cifinalizer tries to close the file descriptor again, it fails with the ``Bad 2147db96d56Sopenharmony_cifile descriptor`` error. A file descriptor must be closed only once. In the 2157db96d56Sopenharmony_ciworst case scenario, closing it twice can lead to a crash (see :issue:`18748` 2167db96d56Sopenharmony_cifor an example). 2177db96d56Sopenharmony_ci 2187db96d56Sopenharmony_ciThe fix is to remove the ``os.close(fp.fileno())`` line, or open the file with 2197db96d56Sopenharmony_ci``closefd=False``. 220