There are currently three supported optimization levels (0, 1, and 2). Briefly summarized, they do the following.

    0: no optimizations
    1: remove assert statements and __debug__ blocks
    2: remove docstrings, assert statements, and __debug__ blocks

The current compile() tests for optimization levels in Lib/test/test_builtin.py covers the assert and docstring cases, but it doesn't test that __debug__ code blocks are included or excluded based on the optimization level.

For example, if you change Python/compile.c to always include __debug__ blocks regardless of the optimization level, the existing compile() tests will continue to pass.

$ git diff Python/compile.c
diff --git a/Python/compile.c b/Python/compile.c
index 280ddc39e3..d65df098bb 100644
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -4143,7 +4143,7 @@ expr_constant(struct compiler *c, expr_ty e)
         /* optimize away names that can't be reassigned */
         id = PyUnicode_AsUTF8(e->v.Name.id);
         if (id && strcmp(id, "__debug__") == 0)
-            return !c->c_optimize;
+            return 1;
         return -1;
     case NameConstant_kind: {
         PyObject *o = e->v.NameConstant.value;

New changeset 543386b7f077d210ea0722079d68beb6c066730a by Mariatta (diana) in branch 'master':
bpo-31657: Add test coverage for the __debug__ case (GH-3450)
https://github.com/python/cpython/commit/543386b7f077d210ea0722079d68beb6c066730a

Thanks for this enhancement, Diana.

Since PR has been merged, and backport is not needed, I think this can be closed now. :)