Code Monkey home page Code Monkey logo

Comments (11)

Fogapod avatar Fogapod commented on May 14, 2024 3

I managed to run session using uvloop setting log_file argument to os.devnull

from arsenic.

Fogapod avatar Fogapod commented on May 14, 2024 1

Could you post the exact error message when not using uvloop?

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/arsenic/__init__.py", line 16, in __aenter__
    self.session = await start_session(self.service, self.browser, self.bind)
  File "/usr/local/lib/python3.6/dist-packages/arsenic/__init__.py", line 28, in start_session
    driver = await service.start()
  File "/usr/local/lib/python3.6/dist-packages/arsenic/services.py", line 99, in start
    self.log_file
  File "/usr/local/lib/python3.6/dist-packages/arsenic/services.py", line 34, in subprocess_based_service
    process = await impl.start_process(cmd, log_file)
  File "/usr/local/lib/python3.6/dist-packages/arsenic/subprocess.py", line 61, in start_process
    stdin=DEVNULL,
  File "/usr/lib/python3.6/asyncio/subprocess.py", line 225, in create_subprocess_exec
    stderr=stderr, **kwds)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1192, in subprocess_exec
    bufsize, **kwargs)
  File "/usr/lib/python3.6/asyncio/unix_events.py", line 200, in _make_subprocess_transport
    **kwargs)
  File "/usr/lib/python3.6/asyncio/base_subprocess.py", line 39, in __init__
    stderr=stderr, bufsize=bufsize, **kwargs)
  File "/usr/lib/python3.6/asyncio/unix_events.py", line 706, in _start
    universal_newlines=False, bufsize=bufsize, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 667, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib/python3.6/subprocess.py", line 1184, in _get_handles
    c2pwrite = stdout.fileno()
io.UnsupportedOperation: fileno

from arsenic.

Fogapod avatar Fogapod commented on May 14, 2024 1

I added log_file parameter, now I have different issue. I get the following output:

2018-05-28 05:42.02 request                        body={"desiredCapabilities": {"browserName": "firefox", "marionette": true, "acceptInsecureCerts": true, "firefoxOptions": {"args": ["-headless"]}}} method=POST url=http://localhost:43633/session
2018-05-28 05:42.02 response                       body={"desiredCapabilities": {"browserName": "firefox", "marionette": true, "acceptInsecureCerts": true, "firefoxOptions": {"args": ["-headless"]}}} data={'value': {'error': 'unknown error', 'message': 'Process unexpectedly closed with status 1', 'stacktrace': ''}} method=POST response=<ClientResponse(http://localhost:43633/session) [500 Internal Server Error]>
<CIMultiDictProxy('Connection': 'close', 'Content-Type': 'application/json; charset=utf-8', 'Cache-Control': 'no-cache', 'Content-Length': '105', 'Date': 'Mon, 28 May 2018 05:42:02 GMT')>
 url=http://localhost:43633/session

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/arsenic/__init__.py", line 16, in __aenter__
    self.session = await start_session(self.service, self.browser, self.bind)
  File "/usr/local/lib/python3.6/dist-packages/arsenic/__init__.py", line 29, in start_session
    return await driver.new_session(browser, bind=bind)
  File "/usr/local/lib/python3.6/dist-packages/arsenic/webdriver.py", line 57, in new_session
    raise SessionStartError(err_resp['error'], err_resp.get('message', ''), original_response)
arsenic.errors.SessionStartError: unknown error: Process unexpectedly closed with status 1

Log file says I don't have display variable defined, but I'm using headless mode

1527486122031   geckodriver     INFO    geckodriver 0.20.1
1527486122039   geckodriver     INFO    Listening on 127.0.0.1:43633                                                    1527486122544   mozrunner::runner       INFO    Running command: "/usr/bin/firefox" "-marionette" "-profile" "/tmp/rust_mozprofile.INA3ouudlnMM"
Error: no DISPLAY environment variable specified

My code:

from arsenic import get_session, services, browsers


service = services.Geckodriver(log_file=open('file.txt','w'))
browser = browsers.Firefox(firefoxOptions={ 'args': ['-headless'] })
async with get_session(service, browser) as session:
  await session.get('https://google.com')

from arsenic.

ojii avatar ojii commented on May 14, 2024 1

Looks like the options changed (either in FF or geckodriver). this worked for me:

browser = browsers.Firefox(**{'moz:firefoxOptions': { 'args': ['-headless'] }})

from arsenic.

ojii avatar ojii commented on May 14, 2024

same exception happens without using uvloop

Could you post the exact error message when not using uvloop? uvloop isn't supported right now because the last time I tried there were some issues that I couldn't work around.

from arsenic.

ojii avatar ojii commented on May 14, 2024

As a workaround, pass a log_file to Geckodriver (or whatever driver you are using). It should be an object that is valid for stdout/stderr as described in https://docs.python.org/3/library/asyncio-subprocess.html#asyncio.AbstractEventLoop.subprocess_exec.

from arsenic.

Fogapod avatar Fogapod commented on May 14, 2024

New issue occurred. Log file output:

1527489279485   geckodriver     INFO    geckodriver 0.20.1
1527489279495   geckodriver     INFO    Listening on 127.0.0.1:54365
1527489279997   mozrunner::runner       INFO    Running command: "/usr/bin/firefox" "-marionette" "-headless" "-profile" "/tmp/rust_mozprofile.Tq4p7Bgte7hq"
*** You are running in headless mode.
Assertion failure: [unhandlable oom] Failed to allocate object while tenuring., at /build/firefox-lojCEZ/firefox-60.0.1+build2/js/src/vm/JSContext.cpp:1662

I'm able to get page with synchronous selenium library, so Firefox browser and driver are working

from arsenic.

ojii avatar ojii commented on May 14, 2024

unhandlable oom

You ran out of RAM? (OOM = Out Of Memory)

from arsenic.

Fogapod avatar Fogapod commented on May 14, 2024

Yes, seems like previous firefox sessions remained in memory and ate it. Is this a bug?
Output ps -aux | grep fire:

kiwi       385  0.1 10.2 2151272 103512 pts/1  Sl+  06:29   0:04 /usr/lib/firefox/firefox -marionette --headless -profile /tmp/rust_mozprofile.IDL1dqR12NKe
kiwi       452  0.0  2.1 1538776 21312 pts/1   Sl+  06:29   0:00 /usr/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -intPrefs 42:0|44:0|74:0| -boolPrefs 5:1|61:1|276:0|299:0| -stringPrefs 285:36;df6fb09a-00ad-4dfd-a53c-ef0582a18525| -schedulerPrefs 0001,2 -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 385 true tab
kiwi       505  0.0  1.6 1524400 16328 pts/1   Sl+  06:29   0:00 /usr/lib/firefox/firefox -contentproc -childID 2 -isForBrowser -intPrefs 42:0|44:0|74:0| -boolPrefs 5:1|61:1|276:0|299:0| -stringPrefs 285:36;df6fb09a-00ad-4dfd-a53c-ef0582a18525| -schedulerPrefs 0001,2 -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 385 true tab
kiwi       538  0.2 10.8 2179688 109572 pts/1  Sl+  06:30   0:05 /usr/lib/firefox/firefox -marionette --headless -profile /tmp/rust_mozprofile.7monsLLmwYcX
kiwi       588  0.0  9.2 1813584 93352 pts/1   Sl+  06:30   0:01 /usr/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -intPrefs 42:0|44:0|74:0| -boolPrefs 5:1|61:1|276:0|299:0| -stringPrefs 285:36;2fd196cf-10f5-46d7-b129-6416e05f7d59| -schedulerPrefs 0001,2 -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 538 true tab
kiwi       627  0.0  1.6 1524400 16348 pts/1   Sl+  06:30   0:00 /usr/lib/firefox/firefox -contentproc -childID 2 -isForBrowser -intPrefs 42:0|44:0|74:0| -boolPrefs 5:1|61:1|276:0|299:0| -stringPrefs 285:36;2fd196cf-10f5-46d7-b129-6416e05f7d59| -schedulerPrefs 0001,2 -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 538 true tab
kiwi      1569  0.0  0.1  14856  1120 pts/3    S+   07:14   0:00 grep --color=auto fire

After killing these I was able to navigate and get screenshot. Thanks for the support

from arsenic.

ojii avatar ojii commented on May 14, 2024

Yes, seems like previous firefox sessions remained in memory and ate it. Is this a bug?

I don't know. I ran the code and it worked fine. Maybe it failed to clean up in the cases where it triggered one of the bugs/problems you ran into? I had to use docker since I don't have a native linux handy, so it cleaned up after each run.

from arsenic.

ojii avatar ojii commented on May 14, 2024

I managed to run session using uvloop setting log_file argument to os.devnull

Awesome! I'll try to add it as a fully supported loop then (with CI etc)

from arsenic.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.