2

I probably learned that the error was due to the use of coroutine io in pyppeteer and requests_html, which conflicted with multithreading, but I can't find a way to fix this.I don't speak English very much, I use google translation. 

import asyncio
from pyppeteer import launch
from requests_html import HTMLSession

# Simulation using requests_html

def test1():
    session = HTMLSession()
    _r = session.get('http://bbs.tianya.cn/post-free-6085404-1.shtml' )
    _r.html.render()
    html = _r.html.html

    print(html)


# main
async def main():
    browser = await launch()
    page = await browser.newPage()
    await page.goto('http://example.com')
    await page.screenshot({'path': 'example.png'})
    await browser.close()


##The pyppeteer method is called at work
def aJob(arg):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_until_complete(main())


## Multi-threaded task generation
def multiThread():
    from multiprocessing.dummy import Pool as ThreadPool
    cpus = 1  # 线程池大小

    pool = ThreadPool(cpus)
    _lstParam = range(0, 3)
    pool.map(aJob, _lstParam)
    pool.close()
    pool.join()


if __name__ == "__main__":
    loop = asyncio.new_event_loop()
    multiThread()

I want to call pyppeteer or requests_html to simulate browsing the webpage in multithreading, but I always get the error "ValueError: signal only works in main thread" or "RuntimeError: There is no current event loop in thread 'Thread-1'." I have tried many methods, but I have been unable to run successfully.Please give me help, thank you!

eyllanesc
  • 235,170
  • 19
  • 170
  • 241
Gigi Dai
  • 21
  • 2
  • Does this answer your question? [Running pypupeteer in FLASK gives ValueError: signal only works in main thread](https://stackoverflow.com/questions/53679905/running-pypupeteer-in-flask-gives-valueerror-signal-only-works-in-main-thread) – Calumah May 04 '20 at 22:24

1 Answers1

0

Pyppeteer use signal to close the browser process but signal only works in main thread. If you don't really need this feature just set handleSIGINT,handleSIGTERM,handleSIGHUP to False on pyppeteer.launch method

Dozy Sun
  • 71
  • 6