browserup / browsermob-proxy-rb Goto Github PK

I am trying to implement browsermob-proxy-rb to block selenium from making API calls to google maps. I don't understand the below line from the readme:

server = BrowserMob::Proxy::Server.new("/path/to/download/browsermob-proxy")

What am I supposed to be linking to here?

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

when I use proxy.new_har 'new_page' , I got

from /Users/jacksontom/.rvm/gems/ruby-2.5.1/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'

AND proxy detail is

#<BrowserMob::Proxy::Client:0x00007f9c9731bbc8
 @host="localhost",
 @port=8081,
 @resource=#<RestClient::Resource:0x00007f9c9731bd30 @block=nil, @options={}, @url="http://localhost:8080/proxy/8081">>

require 'selenium/webdriver'
require 'browsermob/proxy'

path = "/Users/jacksontom/Documents/browserup-proxy-1.2.1/bin/browserup-proxy"
server = BrowserMob::Proxy::Server.new(path) #=> #<BrowserMob::Proxy::Server:0x000001022c6ea8 ...>
server.start

proxy = server.create_proxy() #=> #<BrowserMob::Proxy::Client:0x0000010224bdc0 ...>

profile = Selenium::WebDriver::Firefox::Profile.new #=> #<Selenium::WebDriver::Firefox::Profile:0x000001022bf748 ...>
profile.proxy = proxy.selenium_proxy

driver = Selenium::WebDriver.for :firefox, :profile => profile

proxy.new_har "google"  # ERROR:
driver.get "https://baidu.com"

har = proxy.har #=> #<HAR::Archive:0x-27066c42d7e75fa6>
har.entries.first.request.url #=> "http://google.com"
har.save_to "/tmp/google.har"

proxy.close
driver.quit

similar problem: #38
browsermob-proxy-0.3.1/lib/browsermob/proxy/client.rb:57:in `new_har'
I have been installed: java-13-openjdk and the program did not work. Then i installed the version: java-13-jdk (yaourt -S jre) and it worked. The program does not work with openjdk. You need to install the official version of java.

I only see post requests to OCSP and nothing else. I've configured true for all three criteria (capture content, capture headers, capture binary data) when setting up the driver:

require 'selenium/webdriver' 
require 'browsermob/proxy'
server = BrowserMob::Proxy::Server.new("/Users/pairing/Downloads/browsermob-proxy-2.1.2/bin/browsermob-proxy") 

proxy = server.create_proxy

profile = Selenium::WebDriver::Firefox::Profile.new 
profile.proxy = proxy.selenium_proxy

driver = Selenium::WebDriver.for(:firefox, :profile => profile)
opts = {:capture_binary_content=> true, :capture_content => true, :capture_headers => true}
proxy.new_har("netflix", opts)

driver.get "https://www.netflix.com"

har = proxy.har 
sleep(10) #=> how long do you want the website to load
jsonHar = har.to_json

any advice on why it's not capturing the actual site content?

@jarib @agile I can able to create and set proxy for firefox profile but not on chrome.

I don't know which options i have to use for chrome to set proxy.

Am using the following code:-

For firefox:-

require 'selenium/webdriver'
require 'browsermob/proxy'

server = BrowserMob::Proxy::Server.new("/path/to/downloads/browsermob-proxy/bin/browsermob-proxy") #=> #<BrowserMob::Proxy::Server:0x000001022c6ea8 ...>
server.start

proxy = server.create_proxy #=> #<BrowserMob::Proxy::Client:0x0000010224bdc0 ...>

profile = Selenium::WebDriver::Firefox::Profile.new #=> #<Selenium::WebDriver::Firefox::Profile:0x000001022bf748 ...>
profile.proxy = proxy.selenium_proxy

driver = Selenium::WebDriver.for :firefox, :profile => profile

proxy.new_har "google"
driver.get "http://google.com"

har = proxy.har #=> #<HAR::Archive:0x-27066c42d7e75fa6>
har.entries.first.request.url #=> "http://google.com"
har.save_to "/tmp/google.har"

proxy.close
driver.quit

For chrome:-

require 'selenium/webdriver'
require 'browsermob/proxy'

server = BrowserMob::Proxy::Server.new("/path/to/downloads/browsermob-proxy/bin/browsermob-proxy") #=> #<BrowserMob::Proxy::Server:0x000001022c6ea8 ...>
server.start

proxy = server.create_proxy #=> #<BrowserMob::Proxy::Client:0x0000010224bdc0 ...>

profile = Selenium::WebDriver::Chrome::Profile.new #=>
profile.proxy = proxy.selenium_proxy

driver = Selenium::WebDriver.for :chrome, :prefs => profile

proxy.new_har "google"
driver.get "http://google.com"

har = proxy.har #=> #<HAR::Archive:0x-27066c42d7e75fa6>
har.entries.first.request.url #=> "http://google.com"
har.save_to "/tmp/google.har"

proxy.close
driver.quit

In chrome, errors throws on the following line

profile.proxy = proxy.selenium_proxy

Error:- NoMethodError: undefined method `proxy=' for #Selenium::WebDriver::Chrome::Profile:0x000000042d53a8

How to set proxy on chrome driver profile ?

Hi jarib,

is it possible to use browsermob-proxy-rb with selenium grid?

My use case would be one Linux Server running the proxy and being the selenium hub, with a few nodes.

Any advice would be greatly appreciated.

Regards
Matt

I am trying to block all API requests to google from my selenium browser session using browsermob. Below is my code:

Capybara.register_driver :selenium_without_google do |app|
  filename = File.join(Rails.root, "lib", "browsermob", "bin", "browsermob-proxy")

  server = BrowserMob::Proxy::Server.new(filename) #=> #<BrowserMob::Proxy::Server:0x000001022c6ea8 ...>
  server.start

  proxy = server.create_proxy #=> #<BrowserMob::Proxy::Client:0x0000010224bdc0 ...>
  proxy.blacklist(/google/, '200')

  profile = Selenium::WebDriver::Firefox::Profile.new #=> #<Selenium::WebDriver::Firefox::Profile:0x000001022bf748 ...>
  profile.proxy = proxy.selenium_proxy
  Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile)
end

It is still sending out the API calls. Any direction you can give me on this?

Hi, I am trying to capture mobile app network traffic in appium. I am able to start browser mob server and save har file but it only contains default header no traffic related to mobile app. here's my code. Kindly help

server = BrowserMob::Proxy::Server.new('/Downloads/browsermob-proxy/bin/browsermob-proxy')
server.start
proxy = server.create_proxy #=> #<BrowserMob::Proxy::Client:0x0000010224bdc0 ...>
proxy.new_har "my_har"
capabilities = capabilities.merge(deviceName: ENV['DEVICE'], proxy: proxy.selenium_proxy)
har = proxy.har
har.save_to "/Downloads/my_har.har"

When creating 2 proxy servers there is an error.

Here is the simplest code that will produce the problem:
server1 = BrowserMob::Proxy::Server.new(\"browsermob-proxy-2.0-beta-9/bin/browsermob-proxy\", {port:9000, log:true})
server1.start
proxy1 = server1.create_proxy

server2 = BrowserMob::Proxy::Server.new(\"browsermob-proxy-2.0-beta-9/bin/browsermob-proxy\",{port:9001, log:true})
server2.start
proxy2 = server2.create_proxy

The log warning says Failed to start: [email protected]:9091. That port value is not the same one I specified, but it is the same one used for the first proxy server, so perhaps that is the source of the error.

The specific error is:
/Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient/abstract_response.rb:48:in return!: 500 Internal Server Error (RestClient::InternalServerError)
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient/request.rb:495:in process_result
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient/request.rb:421:in block in transmit
from /Users/henri/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/net/http.rb:853:in start
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient/request.rb:413:in transmit
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient/request.rb:176:in execute
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient/request.rb:41:in execute
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/rest-client-1.7.2/lib/restclient.rb:69:in post
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/browsermob-proxy-0.1.9/lib/browsermob/proxy/client.rb:13:in from
from /Users/henri/.rvm/gems/ruby-2.1.3/gems/browsermob-proxy-0.1.9/lib/browsermob/proxy/server.rb:47:in create_proxy
from bug_test.rb:20:in <main>

@jarib Is this fully compatible with the latest version of the bmp? We were forced to upgrade (version we had been using used SSLv3 and was incompatible with the latest OS upgrade we did) and I think that will require us to rewrite a few of the REST calls we're using in conjunction with this. Being short on time, I'd like to revert to the legacy (Jetty) implementation we were using to see if it just gets things working again.

https://github.com/lightbody/browsermob-proxy#rest-api says you can specify a simple command line argument, --use-littleproxy false to revert to this. In server.rb, I see the only argument accepted is the path and port number: process = ChildProcess.new(@path, "--port", @port.to_s)

Is it possible to use the above flag through the server_options hash or could an optional argument be added for additional flags to the create_process function?

Hi,

Apologies if this is a really simple question and i'm missing something obvious.

I'm trying out browsermob-proxy-rb and so far it is working well. The only problem i have is that I am unable to obtain any header information.

Is this something that is available within this project?

so far https://groups.google.com/forum/?fromgroups#!topic/browsermob-proxy/GKDeQVsmXv0 is the closet I have got to an answer.

Thanks
Matt

I don't have the capacity to maintain this library. If someone wants to take it over, it's up for grabs.

Can someone please provide exact steps to generate SSL certificate, how to self-sign it and how to use it in ruby (Same as mentioned here for Java)

When I try to run the browsermob-proxy it show me this error:
Unable to activate browsermob-proxy-0.3.1, because childprocess-3.0.0 conflicts with childprocess (~> 0.5) (Gem::ConflictError)
Can anyone fix this issue or give me directions on how to solve this?