Code Monkey home page Code Monkey logo

Comments (15)

cbdelavenne avatar cbdelavenne commented on July 18, 2024 13

Having the same issue on Mac OS X and Windows Server 2012.

This is my workaround:

import psutil

def terminate_browsermob_processes(self):
    self._browsermob_proxy.close()
    self._browsermob_server.stop()

    # Find BrowserMob-Proxy processes that may still be alive and kill them
    for process in psutil.process_iter():
        try:
            process_info = process.as_dict(attrs=['name', 'cmdline'])
            if process_info.get('name') in ('java', 'java.exe'):
                for cmd_info in process_info.get('cmdline'):
                    if cmd_info == '-Dapp.name=browsermob-proxy':
                        process.kill()
        except psutil.NoSuchProcess:
            pass

from browsermob-proxy-py.

alex-ber avatar alex-ber commented on July 18, 2024 5

I've investigate this issue deeply. What happens, than when you start browsermobproxy server (I will reffer to it as bmp_daemon, see https://medium.com/@arkadyt/setting-up-programmatic-aws-access-for-mfa-protected-federated-identities-3be22bcecf4b for details) it uses subprocess.Popen with command that contains bat-file. So, what happen on Windows Python process start's cmd that executes bat-file that starts Java process. When you're calling close() function on bmp_daemon, it successfully kills cmd process. On Windows (and on Mac as per #76) OS doesn't kills grand process automatically (I don't know what's happen on Linux, but I suspect it works their), that is close() function kills cmd process, but on Windows (and Mac) the Java process survives.

Inspired by @dasshark response above, I have witted the following helper function that I'm using calling instead of bmp_daemon.stop().

import psutil, signal
from contextlib import suppress

def closeBmpDaemon(bmp_daemon):
    if bmp_daemon is not None and bmp_daemon.process is not None:
        childs_process = None
        try:
            cmd_process = psutil.Process(bmp_daemon.process.pid)
            childs_process = cmd_process.children(recursive=True)
            childs_process = [*childs_process, cmd_process]

            bmp_daemon.stop()
        finally:
            for child in childs_process:
                # we can't accidentally kill newly created process
                # we can kill only the process we have cached earlier
                # if process was already finished we will get NoSuchProcess
                # that we're just suppressing
                with suppress(psutil.NoSuchProcess):
                    child.send_signal(signal.SIGTERM)

EDIT I have added some comments and make some minor changes.

from browsermob-proxy-py.

avatar commented on July 18, 2024 2

On Debian 8 (not Windows)

Python: Selenium v3.0.2, python3.4, browsermobproxy v0.5.0
JAVA: BrowserMob Proxy version 2.1.4

When run, it will create these two applicable lines in the ps aux:
/bin/sh /usr/local/browsermobproxy/bin/browsermob-proxy --port=8080
and:
java -Dapp.name=browsermob-proxy -Dbasedir=/usr/local/browsermobproxy -jar /usr/local/browsermobproxy/lib/browsermob-dist-2.1.4.jar --port=8080

But all of the various suggestions (above) to stop the BMP Server always leave this in the ps aux output:

java -Dapp.name=browsermob-proxy -Dbasedir=/usr/local/browsermobproxy -jar /usr/local/browsermobproxy/lib/browsermob-dist-2.1.4.jar --port=8080

I need to get rid of this proc: How? Suggestions?
Something akin to @cbdelavenne 's solution is not possible, it would destroy the ongoing main process.

The shutdown code:

        if(self.route == 'proxify'):
            if(debug):
                logging.info(Fore.YELLOW +'Turning off the browsermob-proxy server')
            #self.proxy.close()
            self.proxy_server.stop()
        if(debug):
            logging.info(Fore.YELLOW +'Turning off the Selenium WebDriver')
        self.driver.quit()
        if(debug):
            logging.info(Fore.YELLOW +'Turning off the X-Window Frameless Visual Buffer (headless browsing)')
        self.display.stop()

Produces this debugging output:

INFO:root:Turning off the browsermob-proxy server
INFO:root:Turning off the Selenium WebDriver
DEBUG:selenium.webdriver.remote.remote_connection:DELETE http://127.0.0.1:43014/session/8e48269a5c3ef89499e896e7bdb59449 {"sessionId": "8e48269a5c3ef89499e896e7bdb59449"}
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
INFO:root:Turning off the X-Window Frameless Visual Buffer (headless browsing)
DEBUG:pyvirtualdisplay.abstractdisplay:unset DISPLAY
DEBUG:easyprocess:stopping process (pid=29711 cmd="['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '800x600x24', ':1001']")
DEBUG:easyprocess:process is active -> sending SIGTERM
DEBUG:easyprocess:process has ended
DEBUG:easyprocess:return code=0
DEBUG:easyprocess:stdout=
DEBUG:easyprocess:stderr=

server.log:

Running BrowserMob Proxy using LittleProxy implementation. To revert to the legacy implementation, run the proxy with the command-line option '--use-littleproxy false'.
[INFO  2017-02-02T19:10:15,742 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4 
[INFO  2017-02-02T19:10:15,784 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT 
[INFO  2017-02-02T19:10:15,852 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null} 
[INFO  2017-02-02T19:10:16,140 org.eclipse.jetty.util.log] (main) Started [email protected]:8080 
[INFO  2017-02-02T19:10:17,643 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Starting proxy at address: 0.0.0.0/0.0.0.0:8081 
[INFO  2017-02-02T19:10:17,685 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy listening with TCP transport 
[INFO  2017-02-02T19:10:17,831 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy started at address: /0:0:0:0:0:0:0:0:8081 
[INFO  2017-02-02T19:10:49,754 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Shutting down proxy server gracefully 
[INFO  2017-02-02T19:10:49,755 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Closing all channels (graceful) 
[INFO  2017-02-02T19:10:49,782 org.littleshoot.proxy.impl.ServerGroup] (qtp359742806-16) Shutting down server group event loops (graceful) 
[INFO  2017-02-02T19:10:51,991 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Done shutting down proxy server

bmp.log:

[INFO  2017-02-02T19:10:15,742 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4 
[INFO  2017-02-02T19:10:15,784 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT 
[INFO  2017-02-02T19:10:15,852 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null} 
[INFO  2017-02-02T19:10:16,140 org.eclipse.jetty.util.log] (main) Started [email protected]:8080 
[INFO  2017-02-02T19:10:17,643 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Starting proxy at address: 0.0.0.0/0.0.0.0:8081 
[INFO  2017-02-02T19:10:17,685 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy listening with TCP transport 
[INFO  2017-02-02T19:10:17,831 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy started at address: /0:0:0:0:0:0:0:0:8081 
[INFO  2017-02-02T19:10:49,754 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Shutting down proxy server gracefully 
[INFO  2017-02-02T19:10:49,755 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Closing all channels (graceful) 
[INFO  2017-02-02T19:10:49,782 org.littleshoot.proxy.impl.ServerGroup] (qtp359742806-16) Shutting down server group event loops (graceful) 
[INFO  2017-02-02T19:10:51,991 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Done shutting down proxy server

from browsermob-proxy-py.

andreyrusanov avatar andreyrusanov commented on July 18, 2024 1

It seems I found the issue;
It seems that Browsermob process should be started as a group of processes and then has to be terminated as a group. I will try to make a pull request.

from browsermob-proxy-py.

fpoolev avatar fpoolev commented on July 18, 2024

This problem still exists on Windows 7 as described above.

from browsermob-proxy-py.

DarthOpto avatar DarthOpto commented on July 18, 2024

try proxy.close() prior to the server.stop() that helped me, although if the test fails mid way through it still doesn't close the process.

from browsermob-proxy-py.

dshuga avatar dshuga commented on July 18, 2024

Calling proxy.close() prior to server.stop() didn't resolve this issue for me on Windows 7.

from browsermob-proxy-py.

andreyrusanov avatar andreyrusanov commented on July 18, 2024

Any updates on it? I have the same problem(reproduced on CentOS7 and Ubuntu 16.04)

from browsermob-proxy-py.

andreyrusanov avatar andreyrusanov commented on July 18, 2024

PR is made(for Linux only since I don't have Windows machine to test a fix for Windows.

However, I have on assumption on possible fix(see PR description)

from browsermob-proxy-py.

michelebenolli avatar michelebenolli commented on July 18, 2024

Why is this marked as closed? The issue is currently present under Windows.

from browsermob-proxy-py.

darukavishnikov avatar darukavishnikov commented on July 18, 2024

+1
still have the same issue under python

server.stop() ..\..\..\..\..\browsermobproxy\server.py:133: in stop if self.process.poll() is not None: E AttributeError: 'NoneType' object has no attribute 'poll'

Could it be windows permission case?

from browsermob-proxy-py.

dasshark avatar dasshark commented on July 18, 2024

This can be solved by modifying the stop() method and adding a windows handler like this:

import psutil, signal

parent_process = psutil.Process(self.process.pid)
child_process = parent_process.children(recursive=True)
for child in child_process:
  child.send_signal(signal.SIGTERM)
try:
  parent_process.send_signal(signal.SIGTERM)
except psutil.NoSuchProcess:
  pass

from browsermob-proxy-py.

alex-ber avatar alex-ber commented on July 18, 2024

I have extracted this function (and more!) above into separate package.
Here https://github.com/alex-ber/selenium-support/blob/main/alexber/seleniumsupport/_impl.py#L46 is source-code.
Here https://pypi.org/project/selenium-support/ description of this package at PyPI
Here https://alex-ber.medium.com/selenium-support-19330843c63a is detail description of this and another useful utility function for Selenium.

from browsermob-proxy-py.

KB00100100 avatar KB00100100 commented on July 18, 2024

For Ubuntu 16.04, my personal solution:

  1. add import signal
  2. (line 112) small change
self.process = subprocess.Popen(self.command,
                                        close_fds=True,
                                        preexec_fn=os.setsid,
                                        stdout=self.log_file,
                                        stderr=subprocess.STDOUT)
  1. (line 141) replace self.process.kill() with os.killpg(self.process.pid,signal.SIGUSR1)

from browsermob-proxy-py.

alex-ber avatar alex-ber commented on July 18, 2024

@KB00100100 hopefully my code is cross-platform. In about 2 month I'm going to test my code on Ubuntu. If my code will fail, I will try to use your code snippet. Thanks for sharing.

from browsermob-proxy-py.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.