RuntimeError: can't start new thread

时间:2022-08-26 19:35:45

明明我只是简单跑了一个数据清洗28W数据的python脚本,不知道怎么就报错如下:

RuntimeError: can't start new thread

too many threads running within your python process

The "can't start new thread" error almost certainly due to the fact that you have already have too many threads running within your python process,

and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you're creating; the maximum number you will be able to create will be determined by your environment,

but it should be in the order of hundreds at least. It would probably be a good idea to re-think your architecture here;

seeing as this is running asynchronously anyhow, perhaps you could use a pool of threads to fetch resources from another site instead of always starting up a thread for every request. Another improvement to consider is your use of Thread.join and Thread.stop; this would probably be better accomplished by providing a timeout value to the constructor of HTTPSConnection.

RuntimeError: can't start new thread的更多相关文章

  1. Python 3.5 RuntimeError: can't start new thread

    /*********************************************************************** * Python 3.5 RuntimeError: ...

  2. tcp端口扫描(python多线程)

    1 使用单线程扫描单台主机 首先实现的是对单台主机中0-1024端口的扫描,发现差不多每秒扫描一个端口,很慢. import socket def tcp_scanner(host,port): cl ...

  3. 我的Keras使用总结(3)——利用bottleneck features进行微调预训练模型VGG16

    Keras的预训练模型地址:https://github.com/fchollet/deep-learning-models/releases 一个稍微讲究一点的办法是,利用在大规模数据集上预训练好的 ...

  4. [线程]Thead 中传参数RuntimeError: thread.__init__() not called

    在写一个多线程类的时候调用报错 RuntimeError: thread.__init__() not called class NotifyTread(threading.Thread): def ...

  5. tensorflow_目标识别object_detection_api,RuntimeError: main thread is not in main loop,fig = plt.figure(frameon=False)_tkinter.TclError: no display name and no $DISPLAY environment variable

    最近在使用目标识别api,但是报错了: File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/script_o ...

  6. 解决async 运行多线程时报错RuntimeError: There is no current event loop in thread 'Thread-2'

    原来使用: loop = asyncio.get_event_loop()task = asyncio.ensure_future(do_work(checker))loop.run_until_co ...

  7. Python thread local

    由于GIL的原因,笔者在日常开发中几乎没有用到python的多线程.如果需要并发,一般使用多进程,对于IO Bound这种情况,使用协程也是不错的注意.但是在python很多的网络库中,都支持多线程, ...

  8. [Python]Threading.Thread之Daemon线程

    之前对Daemon线程理解有偏差,特记录说明: 一.什么是Daemon A thread can be flagged as a "daemon thread". The sign ...

  9. python threading.thread

    Thread 是threading模块中最重要的类之一,可以使用它来创建线程.有两种方式来创建线程:一种是通过继承Thread类,重写它的run方法:另一种是创建一个threading.Thread对 ...

随机推荐

  1. IOS照片颠倒分析及PHP服务端的处理

    前言: 因朋友的PHP小项目, 而去帮忙解决了一个小问题, 现在来总结概括一下. 也不知道大家在使用和开发的过程中有没有遇到类似的场景, IPhone手机上传照片后, 发现图片方向颠倒了, 甚至各种姿 ...

  2. `cocos2dx 非完整` UI解析模块

    昨天在cocos2dx的一个群里,遇到一位匿名为x的朋友询问的问题,是关于ui的.他使用c++写了不少的ui封装节点,用来实现游戏中的各种不同效果.然后现在想改用lua,于是尝试使用最小代价去复用自己 ...

  3. git查看某个文件的修改历史及具体修改内容

    有时候在比对代码时,看到某些改动,但不清楚这个改动的作者和原因,也不知道对应的BUG号,也就是说无从查到这些改动的具体原因了- [注]:某个文件的改动是有限次的,而且每次代码修改的提交都会有commi ...

  4. Visual Studio 2012 update3 安装后的问题及解决

    安装之后可能遇到的问题: 安装完时,打开Help Viewer时,出现了一个错误提示:”a content file required by the help viewer is missing or ...

  5. sublime text3 支持终端打开文件

    sudo ln -s /Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl /usr/local/bin/subl

  6. A+B problems

    这几道习题大概是ACM输入输出格式的普及 P1:---------------------------------------------------------------------------- ...

  7. spring-boot-starter-actouator2.1.4与c3p0版本0.9.1.2冲突

    报错前的pom文件: <?xml version="1.0" encoding="UTF-8"?><project xmlns="h ...

  8. HTML5 五大特性

    一.正则表达式: 相信大家都会非常喜欢这个特性,无须服务器端的检测,使用浏览器的本地功能就可以帮助你判断电子邮件的格式,URL,或者是电话格式,防止用户输入错误的信息,通过使用HTML5的patter ...

  9. Cannot retrieve repository metadata &lpar;repomd&period;xml&rpar; for repository&colon; base&period; Please verify its path and try again YUM报错

        1.挂盘 ----- 2.# mount /dev/sr0 /media/ mount: block device /dev/sr0 is write-protected, mounting ...

  10. MySQL学习(四)查询

    一.group_concat()函数.把groupby的分组中字段数据组合显示出来 select s_id , GROUP_CONCAT(要显示的字段名)  from table group by 分 ...