estately / rets Goto Github PK

#98 added a SAX processor that reduced memory footprint quite a bit, as you don't have to convert the response XML first into a Nokogiri object tree before converting back to a more simple hash array. Yay!

However, the results still need to be coalesced into that array before you can return doc.results from the parser. While the memory requirements for hashes are not too crazy, it's still pretty wasteful and results in memory bloat (which is bad since Ruby doesn't give back memory).

What would be better is to provide back an Enumerable from the search that would let us declaratively handle the results and keep a clamp on memory usage. I'm currently playing with Nokogiri::XML::Reader to make this happen, but I'm curious if there have been any experiments with this in the past. I know RETS isn't exactly the most sane standard, so are there problems with this approach? It might require some wrangling to make it work, but it seems worth the effort.

Would be helpful to have in the README also. I'm in the analysis phase of a project that provides a RETS v2.2 interface and I can't tell whether this library supports that or not.

Well, ruby-rets is no longer so I've found this great gem. Thanks for building and maintaining it!

It's a new world for me here and was just wondering if anyone had any "find" examples. I'm able to create a client and login successfully but every find query I try returns:

Rets::InvalidRequest: Got error code 20203 (Invalid Search Type)

or other various syntax errors.

This is what I'm trying based on what I can gather from the code.

@client.find(:first, {:Bedrooms => 3})

Not sure where to enter the search type or property class and I assume that's an issue.

I am getting two errors, and I'm also just generally confused on how to use the gem to get data from the MLS systems. I am currently trying to receive specific types of data from the MLS using this dictionary as my guide

http://www.reso.org/assets/Certifications/resodatadictionarycertificationoverview_v1.0.3_2015_0330.pdf

and so right now I'm trying to get address, but I'm not getting anywhere. Just an FYI the property number 123, is fake, I'm just trying to test if I could receive data. If this has an effect on my error messages let me know and ill try to get a real mls_id. What I want to do its use mL_ids of houses to look up their specific information, including timestamps/addresses etc..

So my code looks like this.

require 'rets'
# Pass the :login_url, :username, :password and :version of RETS
client = Rets::Client.new({
                              login_url: 'http://ris.retsiq.com/rets/login',
                              username: 'Deleted for Github',
                              password: 'Deleted for Github',
                              version: 'RETS/1.5'
                          })

begin
  client.login
rescue => e
  puts 'Error: ' + e.message
  exit!
end
puts 'We connected! Lets log out...'
client.logout

address = client.objects '*', {
                               resource: 'Property',
                               object_type: 'Address',
                               resource_id: '123'
                           }
puts address
client.logout

And the errors I'm receiving look like this.

/Users/Zoe/.rvm/rubies/ruby-2.2.1/bin/ruby -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) /Users/Zoe/Documents/planomatic-photoplan/ruby/apiconnector/lib/mls/ri_mls.rb
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
We connected! Lets log out...
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
/Users/Zoe/.rvm/gems/ruby-2.2.1/gems/rets-0.9.0/lib/rets/client.rb:310:in `capability_url': unknown capability GetObject, available capabilities ["membername", "user", "broker", "metadataversion", "minmetadataversion", "timeoutseconds", "login", "logout", "x-selector"] (Rets::UnknownCapability)
    from /Users/Zoe/.rvm/gems/ruby-2.2.1/gems/rets-0.9.0/lib/rets/client.rb:250:in `fetch_object'
    from /Users/Zoe/.rvm/gems/ruby-2.2.1/gems/rets-0.9.0/lib/rets/client.rb:193:in `objects'
    from /Users/Zoe/Documents/planomatic-photoplan/ruby/apiconnector/lib/mls/ri_mls.rb:19:in `<top (required)>'
    from -e:1:in `load'
    from -e:1:in `<main>'

Process finished with exit code 1

So the Url_compatibility error is the obvious one I do not understand. The Cookie#Domain is apparently from http client, I'm still unsure why its there. Any help would be greatly appreciated.

The version on rubygems has a dependency on net-http-persistent, so I think it's real old, and I am trying to use a (lame) service that only allows gems to be pulled from rubygems.org.

Are you guys seeing this issue? This past week, we've been getting this message intermittently from Rapattoni RETS servers. I can make it happen fairly consistently in BAREIS by running a specific rspec test multiple times in a row. The query for data goes just fine, but when you use GetObject to get binary photos (urls are not available), the first several work just fine, but then this response comes from the rest:

HTTP/1.1 401 RETS-Session-ID is invalid.

There's one header value from the server that changes between successful and failed requests:

X-Powered-By: ASP.NET (L004)

to this:

X-Powered-By: ASP.NET (L002)

My theory is a load-balancer is passing us from one server to another, and the RETS-Session-ID is not recognized on the new server. This might really be a server side issue, but my question is do you feel like that is a valid server response that may happen from time to time for whatever reason, and we should be handling that somehow? It looks like the server dictates this cookie on the login response, then we just include it in responses from then on.

Right now, the gem retries that getobject three times, then does a new login, but the same RETS-Session-ID in the header continues to be used, so the getobject fails again three times, and that repeatedly happens for all subsequent attempts.

Maybe that RETS-Session-ID should be purged when we get that error response? Not sure how we would get a new one going forward though. Maybe that error requires a reset session with a whole new login?

Just wanted your feedback before I spend time poking around the code.

Some RETS servers (at least a couple Marketlinx boards) handle pagination badly, and so require seriously huge queries if you don't want to miss anything. This will blow up a delayed job using standard DOM Nokogiri.parse. Stream parsing, especially with a lazy #each, makes this less painful.

Check out https://github.com/josephholsten/rets4r/blob/master/lib/rets4r/client/parsers/compact_nokogiri.rb for a parser that returns a hash for each row. It's under the ruby license, so you're welcome to use it under your MIT.

https://rubygems.org/gems/rets/versions

@dougcole could we get this version out pretty please? :)

Can anyone in this project share a little knowledge on the state of existing and libraries for the existing and coming APIs as regulated by RESO's upcoming Web API specification? The RESO organization regulates all MLS platforms and mandates this new API spec, due for deadline in several days. I'm really liking this library and I'm feeling reluctant to move to new the RESTful APIs for various reasons.

Hoping general questions are welcome in this issue queue, but let me know otherwise and I'll delete!

see this gist

https://gist.github.com/jondruse/b2a68c514b10875b2b8b

Thanks!

PS. 0.4.0 seems to be working just fine from rubygems.org.

Looks like there's an issue with the short/long format metadata printers:

# rets http://rets.tulsa.mlsrets.com/rets/login -U 'ME' -P 'SEKRET' -m
rets:129: undefined method `[]' for #<Rets::Metadata::Root:0x10b1a4a08> (NoMethodError)
    from rets:127:in `each'
    from rets:127

I'm seeing this on GTAR (Tulsa), CCAR (Corpus Christi), LAR (Lubbock), and NTREIS (Dallas).

Anyone seeing this on other servers? I'm happy to help get someone else access if they want to help debug.

Login works great, metadata works great, the following search works great

client.find(:all, {
  search_type: 'Property',
  class: 'Property',
  query: '(ID=*)'
})

but if I try filtering

client.find(:all, {
  search_type: 'Property',
  class: 'Property',
  query: '(LastUpdated=2016-02-18T00:00:00+)',
})

BAM Got error code 20206 (Invalid Query Syntax)

This same query works great over curl. Am I missing something?

@dougcole @hfaulds

Could we start tagging releases of rets? That way it's easier to see when a release was made and at what commit 😋

It seems like you have merged lots of pull requests since your last minor update. Is there something hanging you up on that? Maybe we can help? Thanks!

@dougcole
This is not directly related to the gem, but wanted to get your thoughts/experience on the matter.

Given that most Rets servers return open house datetimes relative to the listings location (no utc offset), and open house datetimes should be shown relative to the listings timezone (not the users's), how do you handle storage of datetimes? Do you convert to UTC before storing? Store a tz with the listing? Leave as is?

While it's best to store datetimes as UTC, open houses are an interesting problem.

Thanks!!

Hi,

I'm trying to connect to NTREIS (RETS/1.5) with the rets gem. Logging in works fine, but when I try to retrieve a property I get the Rets::InvalidRequest error.

property = client.find :first, {search_type: 'Property', class: 'RES', query: '(178=ACT)'}

According to the 1.5 spec the Well-Known Class Name for residential properties is RES. But I've tried other class names such as 'Residential-Property', 'ResidentialProperty', '1', etc. all to no avail.

This is my first time working with RETS, so it's probably something obvious that I'm missing.

Also I see the the Client Class supports logging. How do I turn that on ? I tried:

client = Rets::Client.new({ login_url: '[url]', username: '[username]', password: '[password]', version: 'RETS/1.5', logger: true })

but got the following error, when I ran client.login

NoMethodError: undefined method debug?' for true:TrueClass

Help appreciated!

Thanks,
YH

FMLS has stopped returning many fields when we query for a property using MLS Number. Instead, now they return unique identifers of Agent/Office(e.g. ListAgent_MUI, ListOffice_MU, SellingAgent_MUI, SellingOffice_MUI). Currently we only have examples for Property and Photos. Can someone please help how to fetch Agent/Office info now since they have stopped returning ListAgentFullName, SellingAgentFullName, etc with Property fields? Any help would be highly appreciated.

Doing a bunch of different rets mappings this week....

I ran into one that requires a proxy and I'm getting an HTTP error when logging in. My code:

@client = Rets::Client.new({
    metadata_serializer: Rets::Metadata::JsonSerializer.new,
    login_url: @login_url,
    http_proxy: "http://proxy:[email protected]",
    username: @username,
    password: @password
    })

The proxy service doesn't require a password (or it's actually included in the url itself or something)

It returns this error:

Rets::HttpError: HTTP status: 407, body: invalid
	from /app/vendor/bundle/ruby/2.3.0/gems/rets-0.10.0/lib/rets/parser/error_checker.rb:50:in `check'
	from /app/vendor/bundle/ruby/2.3.0/gems/rets-0.10.0/lib/rets/http_client.rb:62:in `http_get'
	from /app/vendor/bundle/ruby/2.3.0/gems/rets-0.10.0/lib/rets/client.rb:333:in `http_get'
	from /app/vendor/bundle/ruby/2.3.0/gems/rets-0.10.0/lib/rets/client.rb:31:in `login'
	from (irb):45
	from /app/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7/lib/rails/commands/console.rb:110:in `start'
	from /app/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7/lib/rails/commands/console.rb:9:in `start'
	from /app/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7/lib/rails/commands/commands_tasks.rb:68:in `console'
	from /app/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
	from /app/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7/lib/rails/commands.rb:17:in `<top (required)>'
	from bin/rails:8:in `require'
	from bin/rails:8:in `<main>'

Hey guys, I've been poking around this gem for some time now, but still having trouble figuring out the API.

Making the requests from the examples folder works fine, and calling client.metadata also appears to make a request and populate the client with some info. But how does one then take advantage of that?

client.find appears to return simple hashes, how is one supposed to use the metadata to expand on properties like "OwnershipType"=>"3"?

Give bogus id/pwd to NWMLS Matrix RETS server and you don't get an auth failure:

[Dan@lilmac2 rets (master*)]$ bin/rets http://rets.matrix.nwmls.com/rets/login.ashx -U bogus -P bogus -m
/Users/Dan/Dev/rets/lib/rets/client.rb:313:in capability_url': unknown capability GetMetadata, available capabilities [] (Rets::UnknownCapability) from /Users/Dan/Dev/rets/lib/rets/client.rb:291:inretrieve_metadata_type'
from /Users/Dan/Dev/rets/lib/rets/client.rb:285:in block in retrieve_metadata' from /Users/Dan/Dev/rets/lib/rets/client.rb:284:ineach'
from /Users/Dan/Dev/rets/lib/rets/client.rb:284:in retrieve_metadata' from /Users/Dan/Dev/rets/lib/rets/client.rb:278:inmetadata'
from bin/rets:118:in `

It looks like in Client::ErrorChecker, if the status_code is 401, but the response body is valid XHTML with no RETS element, it just returns nothing. I believe it should ignore valid XHTML if it has no RETS element and continue on, then treat the status_code 401 as an AuthorizationFailure.

I've got a PR I'll submit to fix this.

I've been using this code for several years on 2 successful apps, and now launching a new app. This code works in my development environment for the new app, but not on the staging app hosted on heroku. On heroku I get the following error on login:

Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
Error: Got error code 20513 (Unexpected error, please try again)

Login code:

client = Rets::Client.new({
   login_url: LOGIN_URL,
   username: LOGIN,
   password: PASSWORD,
   version: VERSION
   })

 begin
    client.login
 rescue => e
    puts 'Error: ' + e.message
    exit!
 end

Hello estately/rets maintainers -- I am interested in adding Update functionality to this rets library.

At my company I have a (not-quite-immediate) need for this functionality and would like to start working on this in the coming weeks/months. It seems logical to add it to your existing gem rather than to start from scratch.

I am wondering if you have any input on this. Would you be open to accepting a pull request, assuming there are tests and that it conforms to the conventions of the existing ruby API? Do you have any thoughts about how this should work before I dive in?

Thanks for any feedback.

Recently I update my version of ruby in windows

When I try to load the gem I discover the version nokogiri is quite old.

The problem is this gem dont have the DLL precompiled for ruby 2.1

irb(main):001:0> require 'rets'
LoadError: cannot load such file -- nokogiri/2.1/nokogiri
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:144:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/nokogiri-1.5.11-x86-mingw32/lib/nokogiri/nokogiri.rb:1:in `<top (required)>'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/nokogiri-1.5.11-x86-mingw32/lib/nokogiri.rb:28:in `<top (required)>'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:144:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rets-0.6.0/lib/rets.rb:3:in `<top (required)>'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
        from C:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:144:in `require'
        from (irb):1
        from C:/RailsInstaller/Ruby2.1.0/bin/irb:11:in `<main>'

I think if the version for nokogiri is change to one more new the problem could be solved.

Im going to play with this a little ... to try to find out a solution.

Getting: "412 - Precondition set by the client " on current 0.10 version, when switching to current master it doesn't happen, is there an option to do another release.

We're attempting to use this library for the RLS IDX Feed from Stratus Data for REBNY (New York). Seems like everything should be straight forward (at least login and logout). But this is what we see:

> client = Rets::Client.new({ version: 'RETS/1.5', login_url: 'http://rebny-idx.stratusdata.com:XXX/rets/login', username: 'XXX', password: 'XXX'})
=> #<Rets::Client:0x007ffe73d6fc58 @options={:version=>"RE ... }>
> client.login
=> {"broker"=>"NONE", "membername"=>"XXX", "metadataversion"=>"1.00.00001", "minmetadataversion"=>"1.00.00001", "user"=>"XXXX", "changepassword"=>"http://rebny-idx.stratusdata.com:XXX/rets/changePassword", "login"=>"http://rebny-idx.stratusdata.com:XXX/rets/login", "logout"=>"http://rebny-idx.stratusdata.com:XXX/rets/logout", "search"=>"http://rebny-idx.stratusdata.com:XXX/rets/search", "getmetadata"=>"http://rebny-idx.stratusdata.com:XXX/rets/getMetadata", "getobject"=>"http://rebny-idx.stratusdata.com:XXX/rets/getObject", "update"=>"http://rebny-idx.stratusdata.com:XXX/rets/update", "balance"=>"XXX", "timeoutseconds"=>"1800"}
> client.logout
=> ...HTTP Status 401 Unauthorized... This request requires HTTP authentication ()....

So essentially login is fine but logout fails, and other queries I've tried (i.e. find) show an unauthorized response as well. Are we using this client incorrectly or might REBNY be incompatible for some reason?

we are expecting issues in photo listings. They are not coming up a couple weeks ago. Any idea?
https://gooneworld.com Please check

Hi,
I'm using the rets gem. I have done with rets mapping to my application. Now trying to fetching the images from MLS server. But I'm getting the issues like (DMQL: Field 'ListingKey' is not known) Please help me with this issue. I'm sharing the screenshots also please have a look at this.

Thanks.

Hey guys,

Thanks for this awesome gem. It saved me a lot of work. I have some questions though:

1. Do you have a roadmap or new release dates planning? I've seen you've already made lots of improvements in the master branch.
2. Are there plans to support STANDARD-XML format for search? I'm using version 0.9.0, and when I try to use this format, it returns nil when using the search transaction (find method in the gem context).

I also saw you need some help with the Readme. I've been playing a lot with the gem to get the desired results, and I could add info like retrieving lookup types for a resource instance, tables for a rets_class instance (good for understanding attributes metadata like name and types), or even how to search with more options. If you have content ideas or guidelines for the usage, you tell me and I can write something.

Cheers.

I think I'm having trouble connecting to BAREIS because the User Agent isn't acceptable (getting 401 errors). haven't found a method for setting it explicitly so I wondered if there is one and I'm missing it or if I'm just up in the night? I'm new to dealing with RETS.

I'm working with an MLS that seems to lean heavily on using data from lookups. I'm having a little trouble accessing the data in the metadata though. Is there an easy way to lookup on the fly using the metadata that it pulls during the client login?

Here's an example of a value for the status field of a particular listing: UA8R267V0JV and I'll need to use the lookup (in the metadata) feature.

What I know about the attribute itself is this:

LIST_15	ListingStatus	Status	status	100	Character	1	Lookup	20090725042206214793000000	0	0
</DATA>

If I manually grep around, into the metadata that I pull by using client.metadata, I can see that that value is Active. I'd love to do this lookup while I'm creating the new listing in my db while parsing the response. It seems like that is possible with this gem but I'm a little stuck in finding the right syntax to do it...

Thanks in advance!

At least in the RETS version I am working on, the RETS-RESPONSE element does not exist. The response is just under the RETS element.

e.g.

<RETS ReplyCode="0" ReplyText="Login Successful">
  MemberName=XXX
  User=XXX
  Broker=XXX
  <!-- Ect. -->
</RETS>

This causes an error when trying to perform an action because the lib does not recognize any capabilities.

I can work around it but defining capabilities when initializing the Client. However, Should this method be expanded/improved? Or is the work around sufficient?

Hi, I'm trying to get all of the photos downloaded using this example, however I get this error when trying to run the script.

ruby get-photos.rb

require 'rets'

client = Rets::Client.new({
  login_url: 'domainURL',
  username: 'username',
  password: 'password',
  version: 'RETS/1.7.2' 
})

begin
    client.login
rescue => e
    puts 'Error: ' + e.message
    exit!
end

puts 'We connected! Lets get all the photos for a property...'

photos = client.find (:first), {
  search_type: 'Media',
  class: 'Media',
  query: '(MediaType=Image)'

}

photos.each_with_index do |data, index|
  File.open("property-#{index.to_s}.jpg", 'w') do |file|
    file.write data.body
  end
end

puts photos.length.to_s + ' photos saved.'
client.logout

Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
We connected! Lets get all the photos for a property...
current_index: 0
get-photos.rb:54:in `block (2 levels) in <main>': undefined method `body' for ["FileSize", ""]:Array (NoMethodError)
	from get-photos.rb:52:in `open'
	from get-photos.rb:52:in `block in <main>'
	from get-photos.rb:46:in `each'
	from get-photos.rb:46:in `each_with_index'
	from get-photos.rb:46:in `<main>'

any idea what that means? Thanks

Looks like nokogiri needs to be updated:

Bundler could not find compatible versions for gem "nokogiri":
  In Gemfile:
    rets (>= 0) ruby depends on
      nokogiri (~> 1.5.2) ruby

    rails (= 4.2.0) ruby depends on
      actionmailer (= 4.2.0) ruby depends on
        rails-dom-testing (>= 1.0.5, ~> 1.0) ruby depends on
          nokogiri (1.6.0)

I tried here but bundle update doesn't seem to change the nokogiri version. I'm unfamiliar with hoe, so I'd appreciate if anyone could point me in the right direction: https://github.com/openlistings/rets/commit/0985c054d178486cbc329ec64bb4a7b65e633af0

Sorry, I dug a bit deeper into the rets_object metadata and realized my question/issue was based on an incorrect understanding.
As you were.
😄

The types argument added here: #156, creates an edge case where if you request a different list of metadata types the cached data will still be returned, ignoring the new types argument. In practice I doubt this will affect many users, but it's worth fixing before we release a new version.

Right now the cookie caching code assumes the cookie storage is on a local disk. This works fine if you run all your rets interactions on a single machine or have some sort of shared filesystem, but it's not terribly flexible.

I'd prefer a caching mechanism that accepts an object that responds to read(name) and write(name) or something similar, then we can write plugins for this for nearly any storage system and simplify the process of running rets clients on multiple machines.

It doesn't seem correct that when receiving a reply code "20403" -- "No Object Found" that the request should be retried two more times. Would it be safe to skip retrying in this case?

Regarding your photo example:

# Get all photos (*) for MLS ID 'mls_id'
# Pass :object_id (ie '0', '1,2', wildcard '*')
# The pass :resource (Property, Agent, MetaData, ...), :object_type (Photo, PhotoLarge), :rescource_id (ID of agent, MLS, ...)
photos = client.objects '*', {
  resource: 'Property',
  object_type: 'Photo',
  resource_id: 'mls_id'
}

1 - Is there any way to get one response for multiple records? (based on mls_ids or other categories)

I have tried using commas, colons and semicolon to separate different "mls_ids" but it doesn't work

2 - what do the values represent for object_id?

In lib/rets/client.rb:319 - use of /dev/null makes it not initialize appropriately on Windows.

I am working with the Rhode island MLS system and they are saying they have double authentication. After I login to the RETS server through your system I can not go any further because I need to login for a second time with a user-agent username and password. Do you have any idea how I might set this up with your system? Or have you dealt with any other double authentication system?

I had an incident today where I hooked HTTPScoop up to my traffic and noticed the following: each search would consist of a POST request with search criteria but no Authorization header, a response with an auth failure ReplyText, another POST request with the same exact search criteria and an Authorization header, a response with the requested data. So each search is actually a pair of requests, one with valid Authorization, one without. A valid search gives a valid result, but it takes twice as many steps as it should.

The log file generated by the gem does not reflect this. It shows one request and response for each search. This fact, combined with looking through the gem code, leads me to believe that the Authorization header is automatically handled at a lower level, in the HTTPClient. You can see this using the bin/rets example script, so I've eliminated my code causing the issue.

I'm wondering if you guys have noticed this, if it's just an artifact of abstracting the request/response cycle to HTTPClient, if there's something setting I don't know about, etc. I'm happy to work on a pull request if I can figure it out, but just wanted to get your opinion first.

I'm getting the following error from the httpclient gem (I believe) lately when trying to run a query like the one below:

(GLVAR is weird. 144 is ListPrice and class '1' is Residential)

property = client.find :first, {search_type: 'Property', class: '1', query: '144=990000-1000000'}

TypeError: no implicit conversion of nil into String
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:803:in `initialize'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:803:in `new'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:803:in `create_socket'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:752:in `block in connect'
    from /Users/seth/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/timeout.rb:91:in `block in timeout'
    from /Users/seth/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/timeout.rb:101:in `call'
    from /Users/seth/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/timeout.rb:101:in `timeout'
    from /Users/seth/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/timeout.rb:127:in `timeout'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:751:in `connect'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:609:in `query'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:164:in `query'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1083:in `do_get_block'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:887:in `block in do_request'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:981:in `protect_keep_alive_disconnected'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:886:in `do_request'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:774:in `request'
... 4 levels...
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/http_client.rb:16:in `http_get'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:321:in `http_get'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:63:in `login'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:281:in `capabilities'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:285:in `capability_url'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:132:in `find_every'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:116:in `find_with_retries'
    from /Users/seth/.rvm/gems/ruby-2.1.1/bundler/gems/rets-90092ccbf806/lib/rets/client.rb:104:in `find'
    from (irb):19
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/railties-4.1.0/lib/rails/commands/console.rb:90:in `start'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/railties-4.1.0/lib/rails/commands/console.rb:9:in `start'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/railties-4.1.0/lib/rails/commands/commands_tasks.rb:69:in `console'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/railties-4.1.0/lib/rails/commands/commands_tasks.rb:40:in `run_command!'
    from /Users/seth/.rvm/gems/ruby-2.1.1/gems/railties-4.1.0/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:4:in `require'

The system I'm connecting to limits the amount of listings returned to 100. I know that there are about 11,000 total.
If the system wants to return more than 100 listings, then the xml comes with a pagination tag that gives the number of pages and the offset, etc.

Is there functionality to loop and request the rest of the pages?

Thanks!

I got a RETS provider who return rets_class_name as integer so the method 'find_rets_class' cannot find any classes.
Should be fixed to:
rets_classes.detect {|rc| rc.name == rets_class_name.to_s }

Thanks.

Ran into a snag running find with httpclient 2.6 and http-cookie in the Gemfile:

NoMethodError: undefined method `match?' for #<WebAgent::Cookie:0x0000010a0b1de0>
    from [path]/lib/rets/http_client.rb:86:in `block in http_cookie'
    from [path]/lib/rets/http_client.rb:85:in `each'
    from [path]/lib/rets/http_client.rb:85:in `http_cookie'
    from [path]/lib/rets/http_client.rb:76:in `rets_extra_headers'
    from [path]/lib/rets/http_client.rb:26:in `http_post'
    from [path]/lib/rets/client.rb:346:in `http_post'
    from [path]/lib/rets/client.rb:146:in `find_every'
    from [path]/lib/rets/client.rb:119:in `find_with_given_retry'
    from [path]/lib/rets/client.rb:114:in `find_with_retries'
    from [path]/lib/rets/client.rb:103:in `find'

The httpclient 2.6 CHANGELOG (https://github.com/nahi/httpclient/blob/master/CHANGELOG.md) indicates it will use http-cookie if it exists, so assume that's what's happening (and it did in my case, thanks to the mechanize gem). Also received a number of debug log messages: Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.

If I specify httpclient 2.5.3 in the Gemfile, all is well, just figured I should bring it up.

Thanks for everyone's hard work on this excellent gem.

I've created retscli to ease integration with RETs servers. It is using the rets gem underneath, so I wanted to thank you guys for your work on this 🍻 . Hopefully retscli will be of some use to you!

I'm still making improvements and adding features so any feedback/ideas you all have is greatly appreciated. Would you mind mentioning retscli in your README? Hopefully this is helpful to others, working with the rets gem. Looking to make integration as smooth as possible based on everyone's experiences.

Thanks!

Hi,
I want to first thank you for providing this library.
It makes querying mls data a lot easier!!
I just want to know if this is a crmls specific behavior or I am using your library incorrectly.
When I try to query all open house for a given property from CRMLS, I always get a HTTP exception, HTTP status: 401 (Invalid or missing session. Use the login URL to login to this RETS server.), which seems weird to me.
I feel there should be no exception thrown in this case unless it is from CRMLS directly.

Any help is much appreciated!