I dont see any support for opensea api keys which would be needed for prod environments.

See here: https://docs.opensea.io/reference/getting-assets

As collection offers just arrived and are not get by the API I would like to be able to scrape collection activity to check if there is collection offers.

Do you take new stuff on the roadmap or just maintaining it?

If I try to run the script on Linux (computer in the same network) it stops right here without any errors like it's frozen. It works perfectly fine on Mac and it's the same code. It worked a week ago and nothing was changed.

✅ === OpenseaScraper.rankings() ===
=== OpenseaScraper.rankings() ===
...fetching 1 pages (= top 100 collections)
...opening url: https://opensea.io/rankings?sortBy=one_day_volume
...🚧 waiting for cloudflare to resolve
...exposing helper functions through script tag
...scrolling to bottom and fetching collections.
-> freezes here <-

There are no errors. It seems like its stuck in an infinite loop

Edit: After a lot of investigation i finally found what causes the freezing:

If I change the url in the rankings.js from "https://opensea.io/rankings?sortBy=total_volume" to "https://opensea.io/rankings?sortBy=one_day_volume" the script freezes, it works perfectly with the total_volume though. Seven_day_volume does also cause freezing. It is weird because it also worked with one_day_volume a few days ago

Hey everybody,

I noticed that opensea increased the number of times it checked my browser when opening it.
I then noticed that the script can't fetch anymore collections.

✅ === OpenseaScraper.rankings() ===
=== OpenseaScraper.rankings() ===
...fetching 1 pages (= top 100 collections)
...opening url: https://opensea.io/rankings?sortBy=one_day_volume
...🚧 waiting for cloudflare to resolve
...exposing helper functions through script tag
...scrolling to bottom and fetching collections.
...🥳 DONE. Total Collections fetched: 0
scraped 0 collections:

Is this the same for you guys? Does anyone have an idea on how to fix this?

Thanks

Is it possible to fetch rankings from more than 1 page?

The result cannot get prices anymore

Hey everyone so I have been trying to scrape some page from Opensea. And I have used the offersByScrollingByUrl() but I was unsuccessful.
When I use these methods offersByScrollingByUrl() even though I provide the resultSize it stop once it reaches the bottom of page.
and once it reaches the bottom it returns empty resuly.
{ offers: [], stats: { totalOffers: undefined } }

my code:
`const options = {
debug: false,
logs: true,
sort: false,
additionalWait: 0,
browserInstance: undefined,
};

(async () => {
try {
let resultSize=1
const urlWithFilters = 'https://opensea.io/collection/wilderness-to-blockchain-editions?search[sortAscending]=false&search[sortBy]=LAST_SALE_DATE&search[toggles][0]=BUY_NOW';
const res = await OpenseaScraper.offersByScrollingByUrl(urlWithFilters,resultSize, options);
OpenseaScraper.
// const res = await OpenseaScraper.offersByUrl(urlWithFilters,options)
console.log(res);
} catch (err) {
console.log(err);
}
})();`

Any help??
thanks
using the latest v 6.5.2.

Hey
What could be nice is adding testnet option to scrap opensea testnet website :)

I would like to be able to scrape all of the cards on a page for Id information, but I found that as currently implemented, this scraper only actually grabs the top 2-6 with this line in OpenseaScraper.js const cardsNodeList = document.querySelectorAll(".Asset--anchor");. This works great for finding the floor, but what if I wanted to grab all of the cards, would this be possible? I tried a couple other options like querySelectorAlling on '[role="gridcell"]' but ultimately found no improvement - would you have any ideas on why it isn't able to grab the whole page worth of cards? Thanks so much!

This is what I get running on GCP using offersByScrolling: TimeoutError: waiting for selector ``.cf-browser-verification`` to be hidden failed: timeout 30000ms exceeded
It seems it is sometimes working and sometimes failing on this error.
Any idea what's happening there ?

I need to provide a browserInstance as my code runs in an alpine docker container. I was having weird issues, like empty resultsets for offersByScrolling and an error for offers because a split on some part of the html could not be found...
I eventually found out that my browserInstance was detected by opensea, which was serving a warning page instead of the content and tripping opensea-scraper
Don't you thing the doc should have a warning note about browserInstance explaining that it needs to implement puppeteer-extra-plugin-stealth ?

Is there a way to apply opensea's filters?

ex) https://opensea.io/collection/boredapeyachtclub?search[sortBy]=PRICE&search[sortAscending]=true&search[stringTraits][0][name]=Background&search[stringTraits][0][values][0]=Purple

I cannot get any result for floor price from a NFT on opensea. I used the same code as before (it worked), but now it does not work. I would wonder - is it caused by opensea website upgrade? Thanks.

Theres a faster way of getting floor prices I just found. When you have a collection endpoint with query params (meaning filtering is enabled), opensea exposes a __wired__ variable to the window with a lot of information, also including the floor prices. Simply run:

Object.values(__wired__.records)
  .filter(o => o.__typename === "AssetQuantityType")
  .filter(o => o.quantityInEth)
  .map(o => o.quantity / 1000000000000000000);

Some Remarks

• ✅ this does work on pages that have filtering enabled (BUY NOW). Example
• ❌ does not work on the /collection/{slug} route without query params. example
• this new method improves speed because you don't need to scroll
• only the top 32 offers are availible, so if you need more than that you would still need to scroll

Leaving this here for the moment because inside the __wired__ variable there is a lot of information that I currently cannot make sense of. I think there might be more hidden such that we can maybe also find the tokenId or name in this object. If someone wants to do some digging feel free @Jethro87 @robertistok @Androz2091 .

I implemented a method in the branch alternative-method-to-get-floor-prices but not sure yet whether to replace the old way of scraping that data (through the cards), or leave both methods coexist so the developer can try both functions (not ideal) or implement a fallback machanism (try method 1, if it fails try method2).

Will leave open for discussion for now :)

Hello!

In case one would like to run the scraper in a worker, to check for example the offers for multiple projects asynchronously, the current solution would launch a new puppeteer instance for each call.

There are two solutions that came to my mind:

1. Add the third argument, object named opts, to the functions, where users would be able to bring their own puppeteer instance. This is not the most elegant solution IMO, as it would be more complex for the caller.
2. Export an OpenseaScraper class, that would need to be instantiated. In the constructor, we would launch a puppeteer instance and pass it down to the functions, as described in the first point. We could export both the functions as it is now and the new class.

What do you think? I think it would be valuable to have this functionality, as it would add lots of efficiencies when the functions are called one after another.

I'd be happy to submit a PR for this!

@alex-pcln needs functionality to scrape more than 32 offers, so we decided to bring back the old method where we scrape offers by scrolling thorugh the page.

see issue #2

I've added addtionalWait (up to 1 minute) to get html.content(), but it seems that opensea blocks me from it.

Note, it works when getting nft information from opensea testnet. Any tips?

Is it possible to scrapper the owner address using this tool from any specific project

I know from smart contract we can easily do that.

But collections which I am at is created using opensea store contract

Is there a way to scrape the ranking outside of the top 100 from the latest version?

Hi,

I used two scripts below for getting prices of nfts, but all failed. I remembered that I can get prices by using function "offers" or "offersByUrl" before, but they do not work now.
Script 1:
let result_testnet = await OpenseaScraper.offersByUrl("https://testnets.opensea.io/collection/doodles-officias", options); console.dir(result_testnet, {depth: null}); // result object contains keys stats and offers

Script 2:
let result = await OpenseaScraper.offers(nft, options); console.dir(result, {depth: null}); // result object contains keys stats and offers

Any suggestions for that? Thanks.

This functionality no longer works, getting an axios error from this function alone. It responds with error code 404 which is file not found.

Would you consider adding a feature of reading from an activity page (sales/listings)? e.g. https://opensea.io/collection/sandbox?tab=activity&search[isSingleCollection]=true&search[eventTypes][0]=AUCTION_CREATED

Some slugs, like lostpoets would have multiple currencies.

The floor for example is at .969 ETH as of this writing.
https://opensea.io/collection/lostpoets

The first listing is selling it for 160 "ASH"

When this scraper scrapes, it returns 160 as floor, instead of .969.

One of 2 ways to handle I think: kill outliers in first few floor listings and take the minimum. Or just convert whatever currency to ETH.

Hey,

if I try to use any function it returns "Promise pending".

I also removed the "await" from the functions because I get this error if i keep it:

const ranking = await OpenseaScraper.rankings(nbrOfPages, options);
                ^^^^^

SyntaxError: await is only valid in async functions and the top level bodies of modules

I would appreciate your help :)

Code used:

const OpenseaScraper = require("opensea-scraper");

// options
const options = {
  debug: false,
  logs: false,
  sort: true,
  browserInstance: undefined,
}

const nbrOfPages = 1;
const ranking =  OpenseaScraper.rankings(nbrOfPages, options);
console.dir(ranking)

I noticed that the function offersByScrolling() and offersByScrollingByUrl() is not working properly. Most of the offers are not scraped (a lot of them are skipped for some reason, approximately 75% of the offers are not saved). This leads to the function being stuck for a long time, as it takes a lot longer to scrape the desired amount of offers when 75% of the offers are not scraped.

it crashes on https://github.com/dcts/opensea-scraper/blob/main/src/functions/offers.js#L101 because html does not contain window.__wired__=
Proabably this is because the page contains This website is using a security service to protect itself from online attacks.. I'm providing a custom puppeteer instance...

Hi- OpenSea employee here. We've added the floor_price to the /stats endpoint, and this should reflect the same up-to-date price as on our website:

GET /api/v1/collection/cool-cats-nft/stats
HTTP 200 OK
Allow: OPTIONS, GET
Content-Type: application/json
Vary: Accept

{
    "stats": {
        "one_day_volume": 227.8588,
        "one_day_change": 0.7904841231799217,
        "one_day_sales": 34.0,
        "one_day_average_price": 6.701729411764706,
        "seven_day_volume": 1070.25705670901,
        "seven_day_change": -0.42636165444007446,
        "seven_day_sales": 140.0,
        "seven_day_average_price": 7.6446932622072135,
        "thirty_day_volume": 8349.7626205981,
        "thirty_day_change": -0.546823704869074,
        "thirty_day_sales": 761.0,
        "thirty_day_average_price": 10.97209279973469,
        "total_volume": 52723.9290518944,
        "total_sales": 22636.0,
        "total_supply": 9933.0,
        "count": 9933.0,
        "num_owners": 4940,
        "average_price": 2.329206973488885,
        "num_reports": 1,
        "market_cap": 75934.73817350426,
        "floor_price": 5.45
    }
}

https://api.opensea.io/api/v1/collection/cool-cats-nft/stats
https://twitter.com/apiopensea/status/1455918328397144069

This is not working for me when I deploy the app on heroku but working fine in local.
Is that the same for you?

Wanna use this package because a collection can have different categories of NFT and OpeanSea's API don't provide to filter by trait