在我的，而不是库代码中停止

I'm developing an app using a Python library urllib and it is sometimes rising exceptions due to not being able to access an URL.

我正在开发一个使用Python库urllib的应用程序，有时由于无法访问URL而引发异常。

However, the exception is raised almost 6 levels into the standard library stack:

但是，在标准库堆栈中有6个级别的异常:

/home/user/Workspace/application/main.py in call(path)
     11                                  headers={'content-type': 'application/json'},
     12                                  data=b'')
---> 13     resp = urllib.request.urlopen(req)          ####### THIS IS MY CODE
     14     return json.loads(resp.read().decode('utf-8'))

/usr/lib/python3.4/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    159     else:
    160         opener = _opener
--> 161     return opener.open(url, data, timeout)
    162 
    163 def install_opener(opener):

/usr/lib/python3.4/urllib/request.py in open(self, fullurl, data, timeout)
    461             req = meth(req)
    462 
--> 463         response = self._open(req, data)
    464 
    465         # post-process response

/usr/lib/python3.4/urllib/request.py in _open(self, req, data)
    479         protocol = req.type
    480         result = self._call_chain(self.handle_open, protocol, protocol +
--> 481                                   '_open', req)
    482         if result:
    483             return result

/usr/lib/python3.4/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    439         for handler in handlers:
    440             func = getattr(handler, meth_name)
--> 441             result = func(*args)
    442             if result is not None:
    443                 return result

/usr/lib/python3.4/urllib/request.py in http_open(self, req)
   1208 
   1209     def http_open(self, req):
-> 1210         return self.do_open(http.client.HTTPConnection, req)
   1211 
   1212     http_request = AbstractHTTPHandler.do_request_

/usr/lib/python3.4/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1182                 h.request(req.get_method(), req.selector, req.data, headers)
   1183             except OSError as err: # timeout error
-> 1184                 raise URLError(err)
   1185             r = h.getresponse()
   1186         except:

URLError: <urlopen error [Errno 111] Connection refused>

I usually run the code in ipython3 with the %pdb magic turned on so in case there is an exception I can inspect it immediately. However for this I have to go down the stack 6 levels to get to my code.

我通常在ipython3中使用%pdb魔术来运行代码，所以如果有异常，我可以立即检查它。然而，为了实现这一点，我必须在堆栈的6层中找到我的代码。

Is it achievable that my app crashes pointing to my code directly?

我的应用程序崩溃，直接指向我的代码是否可以实现?

5 个解决方案

#1

I would go with modifying the code:

我会修改代码:

try:
    resp = urllib.request.urlopen(req)

except Exception as e:
    raise RuntimeError(e)

That way:

这种方式:

• %pdb moves you to your code,
• %pdb移动到您的代码，
• the original exception is preserved as argument of the "secondary" exception.
• 原始异常被保留为“次要”异常的参数。

You may also monkeypatch urllib.request.urlopen() function:

您也可以monkeypatch urllib.request.urlopen()函数:

class MonkeyPatchUrllib(object):
    def __enter__(self):
        self.__urlopen = urllib.request.urlopen
        urllib.request.urlopen = self
    def __exit__(self, exception_type, exception_value, traceback):
        urllib.request.urlopen = self.__urlopen
    def __call__(self, *args, **kwargs):
        try:                                  
            return self.__urlopen(*args, **kwargs)
        except Exception as e:
            raise RuntimeError(e)

Any time you have an exception raised in urlibopen() call within the context manager scope:

当您在urlibopen()调用中遇到上下文管理器范围内的异常时:

with MonkeyPatchUrllib():
    #your code here

%pdb will move you only 1 level away from your code.

%pdb将只将您的代码移动1级。

[EDIT]

(编辑)

With sys.exc_info() it is possible to preserve a more verbose context of the original exception (like its traceback).

使用sys.exc_info()可以保存原始异常的更详细的上下文(如其回溯)。

#2

pdb has only incremental frame positioning (moving up or down the list of frames).

pdb只有增量帧定位(上下移动帧列表)。

To get the feature you want, you can try trepan (github repository). It has an IPython extension here. You then use the command frame -1 once the exception shows up:

要获得所需的特性，可以尝试trepan (github存储库)。它在这里有一个IPython扩展。当异常出现时，您将使用命令帧-1:

Frame (absolute frame positioning)

(绝对帧定位)

frame [thread-Name*|*thread-number] [frame-number]

框架(线程名字* | *线程数量)(帧数)

Change the current frame to frame frame-number if specified, or the current frame, 0, if no frame number specified.

如果指定当前帧数，则将其更改为帧数;如果没有指定帧数，则更改为当前帧数0。

If a thread name or thread number is given, change the current frame to a frame in that thread. Dot (.) can be used to indicate the name of the current frame the debugger is stopped in.

如果给定一个线程名或线程号，则将当前帧更改为该线程中的一个帧。可以使用Dot(.)表示调试器停止的当前帧的名称。

A negative number indicates the position from the other or least-recently-entered end. So frame -1 moves to the oldest frame, and frame 0 moves to the newest frame. Any variable or expression that evaluates to a number can be used as a position, however due to parsing limitations, the position expression has to be seen as a single blank-delimited parameter. That is, the expression (5*3)-1 is okay while (5 * 3) - 1) isn’t.

负数表示来自另一个或最近最少输入的端点的位置。因此，帧-1移动到最老的帧，帧0移动到最新的帧。任何对数字进行计算的变量或表达式都可以被用作一个位置，但是由于解析限制，位置表达式必须被看作是一个单一的空格分隔的参数。也就是说，表达式(5*3)-1是可以的，而(5*3)-1不是。

Once you are in the desired frame, you can use edit to modify your code.

一旦您处于所需的框架中，您可以使用edit来修改您的代码。

You may find the command backtrace useful too as it gives a stack trace with the less recent call at the bottom.

您可能会发现命令回溯也很有用，因为它在底部提供了一个堆栈跟踪和最近的调用。

trepan depends on uncompyle6 available here.

trepan依赖于这里可用的uncompyle6。

pydb provides a similar feature but was unfortunately not ported to Python3.

pydb提供了一个类似的特性，但不幸的是它没有移植到Python3中。

Otherwise, you may decide to be patient and wait for improvements. In IPython/core/debugger.py:

否则，你可能会选择耐心等待改进。在IPython /核心/ debugger.py:

"""
Pdb debugger class.

Modified from the standard pdb.Pdb class to avoid including readline, so that
the command line completion of other programs which include this isn't damaged.

In the future, this class will be expanded with improvements over the standard pdb.
[...]
"""

#3

It can be done with some hacking. These docs show how you can turn on post-mortem debugging with the following code in the entry point:

这可以通过一些黑客技术来实现。这些文档展示了如何使用以下代码在入口点启动后期调试:

import sys
from IPython.core import ultratb
sys.excepthook = ultratb.FormattedTB(mode='Verbose',
                                     color_scheme='Linux', call_pdb=1)

Stepping through this hook after an exception is raised shows that we need to tinker with the debugger method. Unfortunately I can see no better way to do this other than to copy the entire method and modify it where needed (I tried modifying self.tb but traceback objects are read only and can't be used with copy.deepcopy). Here's a demo:

在出现异常之后，通过这个钩子可以看到我们需要修改调试器方法。不幸的是，除了复制整个方法并在需要时修改它(我尝试修改self)之外，我看不到更好的方法。但是回溯对象是只读的，不能与copy.deepcopy一起使用)。这里有一个演示:

import json
import sys
from IPython.core import debugger, ultratb
from IPython.core.display_trap import DisplayTrap

class CustomTB(ultratb.FormattedTB):
    def debugger(self, force=False):
        if force or self.call_pdb:
            if self.pdb is None:
                self.pdb = debugger.Pdb(
                    self.color_scheme_table.active_scheme_name)
            # the system displayhook may have changed, restore the original
            # for pdb
            display_trap = DisplayTrap(hook=sys.__displayhook__)
            with display_trap:
                self.pdb.reset()
                # Find the right frame so we don't pop up inside ipython itself
                if hasattr(self, 'tb') and self.tb is not None:
                    etb = self.tb
                else:
                    etb = self.tb = sys.last_traceback

                # only modification is here -----+
                #                                |
                #                                V
                while self.tb is not None and '/lib/python3' not in self.tb.tb_next.tb_frame.f_code.co_filename:
                    self.tb = self.tb.tb_next
                if etb and etb.tb_next:
                    etb = etb.tb_next
                self.pdb.botframe = etb.tb_frame
                self.pdb.interaction(self.tb.tb_frame, self.tb)

        if hasattr(self, 'tb'):
            del self.tb

sys.excepthook = CustomTB(mode='Verbose',
                          color_scheme='Linux', call_pdb=1)

def foo():
    bar()

def bar():
    json.dumps(json)

foo()

As you can see it stops searching through the traceback when it's about to reach library code. Here's the result:

如您所见，当它即将到达库代码时，它将停止通过回溯进行搜索。结果:

TypeErrorTraceback (most recent call last)
/Users/alexhall/Dropbox/python/sandbox3/sandbox.py in <module>()
     40     json.dumps(json)
     41 
---> 42 foo()
        global foo = <function foo at 0x1031358c8>

/Users/alexhall/Dropbox/python/sandbox3/sandbox.py in foo()
     35 
     36 def foo():
---> 37     bar()
        global bar = <function bar at 0x103135950>
     38 
     39 def bar():

/Users/alexhall/Dropbox/python/sandbox3/sandbox.py in bar()
     38 
     39 def bar():
---> 40     json.dumps(json)
        global json.dumps = <function dumps at 0x10168b268>
        global json = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
     41 
     42 foo()

/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py in dumps(obj=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw={})
    228         cls is None and indent is None and separators is None and
    229         default is None and not sort_keys and not kw):
--> 230         return _default_encoder.encode(obj)
        global _default_encoder.encode = <bound method JSONEncoder.encode of <json.encoder.JSONEncoder object at 0x10166e8d0>>
        obj = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
    231     if cls is None:
    232         cls = JSONEncoder

/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/encoder.py in encode(self=<json.encoder.JSONEncoder object>, o=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
        chunks = undefined
        self.iterencode = <bound method JSONEncoder.iterencode of <json.encoder.JSONEncoder object at 0x10166e8d0>>
        o = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
        global _one_shot = undefined
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/encoder.py in iterencode(self=<json.encoder.JSONEncoder object>, o=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>, _one_shot=True)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
        _iterencode = <_json.Encoder object at 0x1031296d8>
        o = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/encoder.py in default(self=<json.encoder.JSONEncoder object>, o=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>)
    178 
    179         ""
--> 180         raise TypeError(repr(o) + " is not JSON serializable")
        global TypeError = undefined
        global repr = undefined
        o = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
    181 
    182     def encode(self, o):

TypeError: <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'> is not JSON serializable
> /Users/alexhall/Dropbox/python/sandbox3/sandbox.py(40)bar()
     38 
     39 def bar():
---> 40     json.dumps(json)
     41 
     42 foo()

ipdb> down
> /Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py(230)dumps()
    228         cls is None and indent is None and separators is None and
    229         default is None and not sort_keys and not kw):
--> 230         return _default_encoder.encode(obj)
    231     if cls is None:
    232         cls = JSONEncoder

ipdb> 

Basically the full traceback is still printed out but ipdb starts at your own code. If you enter the down command you find yourself in a library frame.

基本上，完整的回溯仍然打印出来，但是ipdb从您自己的代码开始。如果您输入down命令，您将发现自己处于库框架中。

#4

I think the answer is no.

我认为答案是否定的。

pdb stops at the exception and shows you the stack.

pdb在异常处停止并显示堆栈。

Why would it be useful to hide the real source of the exception?

为什么隐藏异常的真正来源是有用的呢?

If it worked as you seem to be requesting and hides the 6 layers of stack how would you work out what to fix?

如果它像您所期望的那样工作并且隐藏了6层堆栈，您将如何解决什么问题呢?

If this is still not on topic please add to your question.

如果这还不是主题，请补充你的问题。

#5

urllib can raise a lot of exceptions.

urllib可以引发很多例外。

You need to put a try block around the call into urllib and figure how to handle the exceptions for example:

您需要在调用urllib时使用try块，并考虑如何处理异常，例如:

try:
    resp = urllib.request.urlopen(req)   

except URLError as e:
    # analyse e to figure out the detail
    ...

Certainly under python2's urllib lots of other exceptions are thrown. I'm not sure about python3's urllib.

当然，在python2的urllib下抛出了许多其他的异常。我不太清楚毕氏3的小肚皮。