Code Monkey home page Code Monkey logo

scraper's People

Contributors

binsarjr avatar bochilgaming avatar dependabot[bot] avatar fjr-api avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

scraper's Issues

Anti-Bot

Look like blocked lol functions "Lyrics"

image

return null

i combine express js and scraper.

image

in console.log its return the url link download
image

but in json respon its null,
any insight ?
thanks

Error: Cannot find module '../constant.js'

Error: Cannot find module '../constant.js'
Require stack:

  • C:\Bot\azbotzd\node_modules@bochilteam\scraper\lib\cjs\social-media\types.js
  • C:\Bot\azbotzd\node_modules@bochilteam\scraper\lib\cjs\social-media\aiovideodl.js
  • C:\Bot\azbotzd\node_modules@bochilteam\scraper\lib\cjs\social-media\index.js
  • C:\Bot\azbotzd\node_modules@bochilteam\scraper\lib\cjs\index.js
  • C:\Bot\azbotzd\plugins\downloader-yta.js
  • C:\Bot\azbotzd\main.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:939:15)
    at Module._load (node:internal/modules/cjs/loader:780:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object. (C:\Bot\azbotzd\node_modules@bochilteam\scraper\lib\cjs\social-media\types.js:5:23)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Module._load (node:internal/modules/cjs/loader:827:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [
    'C:\Bot\azbotzd\node_modules\@BochilTeam\scraper\lib\cjs\social-media\types.js',
    'C:\Bot\azbotzd\node_modules\@BochilTeam\scraper\lib\cjs\social-media\aiovideodl.js',
    'C:\Bot\azbotzd\node_modules\@BochilTeam\scraper\lib\cjs\social-media\index.js',
    'C:\Bot\azbotzd\node_modules\@BochilTeam\scraper\lib\cjs\index.js',
    'C:\Bot\azbotzd\plugins\downloader-yta.js',
    'C:\Bot\azbotzd\main.js'
    ]
    }

Scrape Error

.igdl error : ScraperError: Cannot find execute code
/js-response/(function(){
function del(){while(document.body.firstChild){document.body.removeChild(document.body.firstChild);}};

if(window.location.hostname.search(/(?:^|.|@)(savefrom.net|sfrom.net|ssyoutube.com)$/i) == -1) {
alert('Please go to http://savefrom.net/ to get direct links');
del();
return;
}
function showResult(){
var success = false;
try {
var d = window.parent.document;
if(d && d.getElementById) {
var e = d.getElementById('sf_result');
if(e && e.innerHTML) {
window.parent.sf.finishRequest(true);;
e.innerHTML = '';
window.parent.sf.result.showEmptyResult({"html":"Something went wrong. Please try to reload the page or use another browser.","invalid_request":true,"resource":"","success":false,"source_url":"https://www.instagram.com/reel/Cu1m_2sOX1N/?igshid=MzRlODBiNWFlZA=="},false,true,false);;
window.parent.sf.enableElement('sf_submit', true);
success=true;
}
}
} catch(err){}
del();
if(!success) {
var alt = '';
if(alt) alert(alt);
}
}
showResult();
})();/js-response/

If this is bug pls report to https://github.com/BochilTeam/scraper

HTTPError: Response code 403 (Forbidden)

node:events:505
throw er; // Unhandled 'error' event
^

ZodError: [
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
"sd"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
"hd"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
"hosting"
],
"message": "Required"
}
]

facebookdl and facebookdlv2 error

error:

/node_modules/zod/lib/index.mjs:538
const error = new ZodError(ctx.common.issues);
^

ZodError: [
{
"code": "invalid_type",
"expected": "number",
"received": "string",
"path": [
"duration"
],
"message": "Expected number, received string"
}
]
at get error [as error] (/node_modules/zod/lib/index.mjs:538:31)
at ZodObject.parse (/node_modules/zod/lib/index.mjs:638:22)
at facebookdl (/node_modules/@bochilteam/scraper-sosmed/lib/esm/src/facebook.js:48:37)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async file /index.js:14:14 {
issues: [
{
code: 'invalid_type',
expected: 'number',
received: 'string',
path: [ 'duration' ],
message: 'Expected number, received string'
}
],
addIssue: [Function (anonymous)],
addIssues: [Function (anonymous)],
errors: [
{
code: 'invalid_type',
expected: 'number',
received: 'string',
path: [ 'duration' ],
message: 'Expected number, received string'
}
]
}

Node.js v20.0.0

import { facebookdl, facebookdlv2 } from '@bochilteam/scraper-sosmed'

const data = await facebookdl('https://www.facebook.com/reel/841811270967738')
console.log(data) // JSON
const data2 = await facebookdlv2('https://www.facebook.com/reel/841811270967738')
console.log(data2) // JSON

Request for Contact Information for Private Conversation

Hello @BochilGaming,

My name is Sanuja and I'm reaching out to you regarding the code segment related to the generateHash function. in "https://github.com/BochilTeam/scraper/blob/master/packages/scraper-youtube/src/util.ts". I have questions regarding the generateHash functionality. Unfortunately, I am unable to find contact information of you to initiate a private conversation.

Could you please provide me with a means of contacting you privately? I am eager to engage in a discussion about the codebase and would greatly appreciate the opportunity to communicate directly.
If it is more convenient for you, please feel free to reach out to me directly via Telegram @SanujaNS

I look forward to hearing from you soon.

Thank you.

Error in Instagram

It gives error when trying to download any type of video/photos from Instagram

error

const { savefrom } = require('@bochilteam/scraper'); (async () => { savefrom('https://www.instagram.com/p/CrvIUf8omdg/').then(res => { console.log(res); }).catch(err => { console.log(err); }) })();

ZodError: [
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
"sd"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
"hd"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
"hosting"
],
"message": "Required"
}
]

Error: Cannot find execute code

/js-response/(function(){
function del(){while(document.body.firstChild){document.body.removeChild(document.body.firstChild);}};

if(window.location.hostname.search(/(?:^|.|@)(savefrom.net|sfrom.net|ssyoutube.com)$/i) == -1) {
alert('Please go to http://savefrom.net/ to get direct links');
del();
return;
}
function showResult(){
var success = false;
try {
var d = window.parent.document;
if(d && d.getElementById) {
var e = d.getElementById('sf_result');
if(e && e.innerHTML) {
window.parent.sf.finishRequest(true);;
e.innerHTML = '';
window.parent.sf.result.showEmptyResult({"html":"Something went wrong. Please try to reload the page or use another browser.","invalid_request":true,"resource":"","success":false,"source_url":"https://www.instagram.com/p/CvFbwPqRxd3"},false,true,false);;
window.parent.sf.enableElement('sf_submit', true);
success=true;
}
}
} catch(err){}
del();
if(!success) {
var alt = '';
if(alt) alert(alt);
}
}
showResult();
})();/js-response/

its not working. please take a look,

url- https://www.instagram.com/p/CvFbwPqRxd3

Gagal dalam mendownload tiktok no wm

Screenshot_2024-02-17-18-35-28-70
Screenshot_2024-02-17-18-35-10-77

ScraperError: Cannot find execute code/js-response/(function(){function del(){while(document.body.firstChild){document.bo dy.removeChild(document.body.firstChild);}};if(window.location.hostname.search(/(?:^|.|@)(savefrom.ne|sfrom.net|ssyoutube.com)$/1) == -1) {alert('Please go to http://savefrom.net/ to get direct links');del();return;function showResult(){var success = false;try {var d = window.parent.document;if(d && d.getElementById) {var e= d.getElementById('sf_result');if(e && e.innerHTML) {window.parent.sf.finishRequest(true);;e.innerHTML =window.parent.sf.result.showEmptyResult({"html":"Something went wrong. Please try to reload the page or use an other browser.","invalid_request":true,"resource": "", "succes s":false,"source_url":"https://vt.tiktok.com/ZSFjKtYas/"},false, true, false);;window.parent.sf.enableElement('sf_submit', true); success=true;Type a command}StartRestartStop}} } catch(err){}del();if(!success) {var alt="if(alt) alert(alt);} }showResult();})();/js-response/If this is bug pls report to https://github.com/BochilTeam/s craperat savefrom /home/container/node_modules/@bochilteam/sc raper/lib/cjs/social-media/savefrom.js:34:15at process.processTicksAndRejections (node:internal/process/task queues: 95:5)date: 2024-02-17T11:34:07.989Z} unhandledRejection

Video Downloading issue

function showResult(){
var success = false;
try {
var d = window.parent.document;
if(d && d.getElementById) {
var e = d.getElementById('sf_result');
if(e && e.innerHTML) {
window.parent.sf.finishRequest(true);;
e.innerHTML = '';
window.parent.sf.result.showEmptyResult({"html":"Something went wrong. Please try to reload the page or use another browser.","invalid_request":true,"resource":"","success":false,"source_url":"https://www.facebook.com/stories/2388064231225152/UzpfSVNDOjY2NDcyNzU3OTUzODgxNjg=/?view_single=1"},false,true,false);;
window.parent.sf.enableElement('sf_submit', true);
success=true;
}
}
} catch(err){}
del();
if(!success) {
var alt = '';
if(alt) alert(alt);
}
}
showResult();
})();/js-response/

//

"use strict";

var _express = _interopRequireWildcard(require("express"));

var _scraper = _interopRequireWildcard(require("@bochilteam/scraper"));

function _getRequireWildcardCache(nodeInterop) { if (typeof WeakMap !== "function") return null; var cacheBabelInterop = new WeakMap(); var cacheNodeInterop = new WeakMap(); return (_getRequireWildcardCache = function (nodeInterop) { return nodeInterop ? cacheNodeInterop : cacheBabelInterop; })(nodeInterop); }

function _interopRequireWildcard(obj, nodeInterop) { if (!nodeInterop && obj && obj.__esModule) { return obj; } if (obj === null || typeof obj !== "object" && typeof obj !== "function") { return { default: obj }; } var cache = _getRequireWildcardCache(nodeInterop); if (cache && cache.has(obj)) { return cache.get(obj); } var newObj = {}; var hasPropertyDescriptor = Object.defineProperty && Object.getOwnPropertyDescriptor; for (var key in obj) { if (key !== "default" && Object.prototype.hasOwnProperty.call(obj, key)) { var desc = hasPropertyDescriptor ? Object.getOwnPropertyDescriptor(obj, key) : null; if (desc && (desc.get || desc.set)) { Object.defineProperty(newObj, key, desc); } else { newObj[key] = obj[key]; } } } newObj.default = obj; if (cache) { cache.set(obj, newObj); } return newObj; }

const app = (0, _express.default)();
app.use((0, _express.json)());
const PORT = process.env.PORT || 3000;
app.get('/abc', (req, res) => {
const url = req.query.url;
console.log(url);
(0, _scraper.savefrom)(url).then(data => {
res.json(transform(data));
}).catch(err => {
res.json("not found");
});
});
app.listen(PORT, () => console.log(App listening at ppport ${PORT}));

function transform(data) {
return {
img: data.thumb,
urls: data.url.map(u => ({
url: u.url,
quality: u.quality,
ext: u.ext,
size: u.filesize
}))
};
}

Bug twitterdl

node:events:505
throw er; // Unhandled 'error' event
^

ScraperError: No results found!

aa

TikTok Downloader v2 Issue

Version 2 was the only one that i could get to work in the first place and now its down

HTTPError: Response code 502 (Bad Gateway)
at Request. (/Users/../../node_modules/got/dist/source/as-promise/index.js:118:42)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
code: 'ERR_NON_2XX_3XX_RESPONSE',
timings: {
start: 1672879198171,
socket: 1672879198173,
lookup: 1672879198281,
connect: 1672879198317,
secureConnect: 1672879198355,
upload: 1672879198355,
response: 1672879199347,
end: 1672879199348,
error: undefined,
abort: undefined,
phases: {
wait: 2,
dns: 108,
tcp: 36,
tls: 38,
request: 0,
firstByte: 992,
download: 1,
total: 1177
}
}
}

youtubedl & youtubedlv2 error

const { youtubedl } = require('@bochilteam/scraper')

(async () => {
youtubedl("https://youtube.com/watch?v=HpUcMGmZbko").then(res => {
console.log(res) })
})();

output:
HTTPError: Response code 403 (Forbidden)
at Request. (/home/container/node_modules/got/dist/source/as-promise/index.js:118:42)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
code: 'ERR_NON_2XX_3XX_RESPONSE',
timings: {
start: 1700022630672,
socket: 1700022630672,
lookup: 1700022630675,
connect: 1700022630677,
secureConnect: 1700022630681,
upload: 1700022630681,
response: 1700022630690,
end: 1700022630691,
error: undefined,
abort: undefined,
phases: {
wait: 0,
dns: 3,
tcp: 2,
tls: 4,
request: 0,
firstByte: 9,
download: 1,
total: 19
}
}
}

youtubedl dan youtubedlv2 output na sama

TypeError: Cannot read properties of undefined (reading 'accessibilityData')

Beberapa query bikin error gini

TypeError: Cannot read properties of undefined (reading 'accessibilityData')
    at file:///home/MuhammadRestu/Rain-BotV2/node_modules/@bochilteam/scraper/lib/esm/social-media/youtube-search.js:71:122
    at Array.forEach (<anonymous>)
    at youtubeSearch (file:///home/MuhammadRestu/Rain-BotV2/node_modules/@bochilteam/scraper/lib/esm/social-media/youtube-search.js:24:71)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.handler (file:///home/MuhammadRestu/Rain-BotV2/plugins/youtube-play.js:10:14)
    at async Object.handler (file:///home/MuhammadRestu/Rain-BotV2/handler.js?update=1670815978468:716:21)

igstory issue

const { instagramStory } = require('@bochilteam/scraper')
await instagramStory("Persija").then(res => {
console.log(res)
})

output:
HTTPError: Response code 500 (Internal Server Error)
at Request. (/node_modules/got/dist/source/as-promise/index.js:118:42)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
code: 'ERR_NON_2XX_3XX_RESPONSE',
timings: {
start: 1696041980886,
socket: 1696041980887,
lookup: 1696041980900,
connect: 1696041980912,
secureConnect: 1696041980929,
upload: 1696041980929,
response: 1696041985238,
end: 1696041985335,
error: undefined,
abort: undefined,
phases: {
wait: 1,
dns: 13,
tcp: 12,
tls: 17,
request: 0,
firstByte: 4309,
download: 97,
total: 4449
}
}
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.