markt.io's Introduction

markt.io

Markt.io is a stock market tracking interface

Installation Instructions

Use NPM to install these node packages after installing source code.

npm puppeteer
npm express
npm node-fetch
npm body-parser
npm node-html-parser

Scraping Method for basic stock data

  const getCurrentStockPrice = await page.evaluate(() => {
      const stock = window.location.href;
      const stockSymbol = stock.substring(stock.indexOf('quote/')+6, stock.length);
      const currentStockPrice = document.querySelector(`[data-symbol="${stockSymbol}"][data-field="regularMarketPrice"]`);
      return currentStockPrice.innerHTML;
    });

Query Stock Scrraping Concept (Light Weight) (Good for Live Market Price Ticker use)

await page.goto("https://query1.finance.yahoo.com/v8/finance/chart/aapl" + stockSymbol);

const getCurrentStockPrice = await page.evaluate(() => {

  let currentStockPrice = document.querySelector('body');
  const stockPrice = currentStockPrice.innerHTML.toString();
  const stockPriceString = stockPrice.substring(stockPrice.indexOf('"regularMarketPrice":')+21, stockPrice.indexOf(',"chartPrevi'));

  const findError = stockPrice.substring(stockPrice.indexOf('":{"code":"')+11, stockPrice.indexOf('","descripti'));
  if(findError === "Not Found")
  {
    return "notValid";
  }
  else {
    return stockPriceString.toString().replace(/\B(?=(\d{3})+(?!\d))/g, ",");
  }


});

Live Stock Data Queries

await page.goto("https://query1.finance.yahoo.com/v7/finance/quote?symbols=(STOCK SYMBOL)");

await page.goto("https://query1.finance.yahoo.com/v6/finance/quote?symbols=(STOCK SYMBOL)");

markt.io's People

Contributors

Watchers

markt.io's Issues

Loading Stock data during pre and active market hours

  const getAfterMarketPriceChange = await page.evaluate(() => {
      const stock = window.location.href;
      const stockSymbol = stock.substring(stock.indexOf('quote/')+6, stock.length);
      const afterMarketStockPrice = document.querySelector(`[data-symbol="${stockSymbol}"][data-field="postMarketChange"] span`);
      return afterMarketStockPrice.innerHTML;
    });


    const getAfterMarketStockPrice = await page.evaluate(() => {
      const stock = window.location.href;
      const stockSymbol = stock.substring(stock.indexOf('quote/')+6, stock.length);
      const afterMarketStockPrice = document.querySelector(`[data-symbol="${stockSymbol}"][data-field="postMarketPrice"]`);
      return afterMarketStockPrice.text;
    });

These divs do not exist during pre and regular market hours

Loading Stock Pull Data

Use await function on top of the the res.send in the Ajax post section in order to force a delay until the stock data is sent to the front end.

Scrolling Concept

http://jsfiddle.net/nimittshah/ahvsq6np/

Scraping

//MAKE SURE TO IMPORT
const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch({headless: true});
const page = await browser.newPage();
console.log("Loading Listing Data..");
await page.goto("https://www.bmwofbeverlyhills.com/certified/BMW/2018-BMW-M3-05c63cf00a0e09b14df8966073b470a6.htm");
await page.screenshot({path: "listing.png"});

const getPrice = await page.evaluate(() => {
const priceTag = document.querySelector(".price-value");
return priceTag.innerHTML;
});

const getMiles = await page.evaluate(() => {
const milesTag = document.querySelector(".line-height-condensed dd span");
return milesTag.innerHTML;
});

const getTitle = await page.evaluate(() => {
const titleStart = document.querySelector(".vehicle-title span");
const titleEnd = document.querySelector(".vehicle-title span:last-child span");

return titleStart.innerHTML + " " + titleEnd.innerHTML;

});

const getPageHTML = await page.evaluate(() => {
const bodyTag = document.querySelector("body");
return bodyTag.innerHTML;
});

//console.log(getPageHTML);
//let substring = "price-value";
//let priceHtmlLocation = getPageHTML.toLocaleLowerCase().indexOf(substring.toLocaleLowerCase());
//let priceHTML = getPageHTML.slice(priceHtmlLocation-20, priceHtmlLocation+20);
//console.log(priceHTML);

console.log(getTitle);
console.log(getPrice);
console.log(getMiles);

await browser.close();
})();

Recommend Projects