Code Monkey home page Code Monkey logo

Comments (4)

alkhachatryan avatar alkhachatryan commented on June 20, 2024

@sudheer-ranga hi :)

from aliexpress-product-scraper.

rafaelsg-01 avatar rafaelsg-01 commented on June 20, 2024

Change the aliexpress-product-scraper/src/aliexpressProductScraper.js file code to this one:

const puppeteer = require('puppeteer');
const cheerio = require('cheerio');

const Variants = require('./variants');
const Feedback = require('./feedback');

async function AliexpressProductScraper(productId, feedbackLimit) {
  const FEEDBACK_LIMIT = feedbackLimit || 10;
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  /** Scrape the aliexpress product page for details */
  await page.goto(`https://www.aliexpress.com/item/${productId}.html`);
  const aliExpressData = await page.evaluate(() => runParams);

  const data = aliExpressData.data;

  /** Scrape the description page for the product using the description url */
  const descriptionUrl = data.productDescComponent.descriptionUrl;
  await page.goto(descriptionUrl);
  const descriptionPageHtml = await page.content();

  /** Build the AST for the description page html content using cheerio */
  const $ = cheerio.load(descriptionPageHtml);
  const descriptionData = $('body').html();

  /** Fetch the adminAccountId required to fetch the feedbacks */
  const adminAccountId = await page.evaluate(() => adminAccountId);
  await browser.close();

  let feedbackData = [];

  if (data.feedbackComponent.totalValidNum > 0) {
    feedbackData = await Feedback.get(
      data.productInfoComponent.id,
      adminAccountId,
      data.feedbackComponent.totalValidNum,
      FEEDBACK_LIMIT
    );
  }

  /** Build the JSON response with aliexpress product details */
  const json = {
    title: data.productInfoComponent.subject,
    categoryId: data.productInfoComponent.categoryId,
    productId: data.productInfoComponent.id,
    totalAvailableQuantity: data.inventoryComponent.totalAvailQuantity,
    description: descriptionData,
    orders: data.tradeComponent.formatTradeCount,
    storeInfo: {
      name: data.sellerComponent.storeName,
      companyId: data.sellerComponent.companyId,
      storeNumber: data.sellerComponent.storeNum,
      followers: data.sellerComponent.followingNumber,
      ratingCount: data.sellerComponent.positiveNum,
      rating: data.sellerComponent.positiveRate
    },
    ratings: {
      totalStar: 5,
      averageStar: data.feedbackComponent.averageStar,
      totalStartCount: data.feedbackComponent.totalValidNum,
      fiveStarCount: data.feedbackComponent.fiveStarNum,
      fourStarCount: data.feedbackComponent.fourStarNum,
      threeStarCount: data.feedbackComponent.threeStarNum,
      twoStarCount: data.feedbackComponent.twoStarNum,
      oneStarCount: data.feedbackComponent.oneStarNum
    },
    images:
      (data.imageModule &&
        data.imageModule.imagePathList) ||
      [],
    feedback: feedbackData,
    variants: Variants.get(data.priceComponent.skuPriceList),
    specs: data.productPropComponent.props,
    currency: data.currencyComponent.currencyCode,
    originalPrice: {
      min: data.priceComponent.origPrice.minAmount.value,
      max: data.priceComponent.origPrice.maxAmount.value
    },
    salePrice: {
      min: data.priceComponent.discountPrice.minActivityAmount 
        ? data.priceComponent.discountPrice.minActivityAmount.value 
        : data.priceComponent.discountPrice.minAmount.value,
      max: data.priceComponent.discountPrice.maxActivityAmount 
        ? data.priceComponent.discountPrice.maxActivityAmount.value 
        : data.priceComponent.discountPrice.maxAmount.value,
    }
  };

  return json;
}

module.exports = AliexpressProductScraper;

This code updates the file to the new Aliexpress object names. 🙂

from aliexpress-product-scraper.

sudheer-ranga avatar sudheer-ranga commented on June 20, 2024

@alkhachatryan I have released a new version v2.0.2. There were breaking changes due to aliexpress changing their website layout and variables. Please update to the latest version.

from aliexpress-product-scraper.

sudheer-ranga avatar sudheer-ranga commented on June 20, 2024

Closing this issue for now. Feel free to reopen or create new issue if the latest update is not working.

from aliexpress-product-scraper.

Related Issues (14)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.