russss / barclayscrape Goto Github PK

View Code? Open in Web Editor NEW

56.0 56.0 22.0 282 KB

A small app to programmatically mainpulate Barclays online banking

License: GNU General Public License v3.0

JavaScript 100.00%

bank-account barclays ofx puppeteer scraper

barclayscrape's People

Contributors

Stargazers

Watchers

Forkers

jwheare dpslwk benjie jamiemcnaught synctext rukor bassu carpii robhayward kirun1 osgirl pinklite34 blueracle ryanwwhittaker markus851 onlywayisup castaway

barclayscrape's Issues

New login page layer (aka No node found for selector: #lastDigits0)

Looks like we need to add something to click on the "PINsentry card reader" tab

Happy to have a go later (if I can figure out the code..)

Example?

I was wondering if there was an example of use for the library?

Thanks!

Timeout error on login ..

Error when clicking button[title="Next Step"] on URL https://bank.barclays.co.uk/olb/authlogin/loginAppContainer.do#/identification: TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded

I'm getting this when running
"$ENV{BARCLAYSCRAPE}/barclayscrape.js --otp $ENV{CODE} get_ofx $ENV{ACCESS_HOME}"

consistently .. logging in manually see,s to work fine.. any ideas?

Login by memorable word broken

Hi,

This is a great app, I have used for years, thank you :)

However... it seems the latest faffing at Barclays end has broken it, I had a look, and released I am well out of my depth, when you have time, if you can fix it would be very much appreciated :)

Thank you again

Scott

[Ignore] Understanding the different markup

Ignore this issue, I'm just using it as a scratchboard to record the vastly different markup people are reporting

If you're having an issue with logging in via this library, feel free to post here using the following template...

Account Type: Business/Personal/Both
Passcode Enabled: Yes/No
PinSentry Enabled: Yes/No
Mobile PINSentry Enabled: Yes/No
 
### Step 2 Images...

Don't post other comments or discussion here please, use #39 instead

barclays-pinsentry - 4 digit PIN

barclays-pinsentry does not work with a 4 digit PIN is there any chance you can help look at this, it's well beyond my skillset.

Cheers!

Last commit which used casper.js?

Hey Russ,

I'm trying to find the last commit which used casper.js, before you started migrating to Puppeteer. Is this the barclayscrape-2 branch?

I want to carry my previous contributions across to the new library, but not sure where to start.

Thanks

Available Balance

Would it be possible to have the choice of including the current available balance in the export_csv.js option?

Mobile PINSentry not being selected during login

Hi,
I'm trying to connect using 'motp' however during login (and when showing browser - ie. no-headless) I can see that the Mobile PINSentry tab is not being selected on the screen that appears after the initial screen. I am supplying my surname and membership no. in that first screen.
Running this on a Mac and opening in Chromium.
Launching in terminal within VS Code.

Here is the trace in terminal:
Error: Node is either not visible or not an HTMLElement
at ElementHandle._clickablePoint (/Users/ryan/VSCode/barclayscrape/node_modules/puppeteer/lib/cjs/puppeteer/common/JSHandle.js:329:19)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async ElementHandle.click (/Users/ryan/VSCode/barclayscrape/node_modules/puppeteer/lib/cjs/puppeteer/common/JSHandle.js:390:26)
at async DOMWorld.click (/Users/ryan/VSCode/barclayscrape/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:277:9)
at async Object.exports.fillFields (/Users/ryan/VSCode/barclayscrape/node_modules/barclayscrape/utils.js:33:5)
at async Session.loginMOTP (/Users/ryan/VSCode/barclayscrape/node_modules/barclayscrape/session.js:65:5)
at async auth (/Users/ryan/VSCode/barclayscrape/node_modules/barclayscrape/barclayscrape.js:222:5)
at async Command. (/Users/ryan/VSCode/barclayscrape/node_modules/barclayscrape/barclayscrape.js:30:14)

Any ideas?

Thanks
Ryan

Online Banking on Big Sur and Chrome?

Sorry this isn't related directly to the barclayscrape lib.
Just wondering if we have any users here using OSX Big Sur, and Chrome browser?

Since upgrading to Big Sur, Chrome seems to be having quite a few problems ('View All Online Services' loads without any CSS, is invalid HTML anyway and so also has a bunch of jQuery errors.)

Also some serverside errors when trying to view Account -> Show Recent Transactions -> View All Transactions

Error 500: javax.servlet.ServletException: Filter [securityFilter]: filter is unavailable

Safari and Firefox on the same machine all working great, not a cache/extension/cookie problem wither.
Is this a 'me' thing, or anyone else seeing this too?

Thanks

[phantom] Wait timeout of 5000ms expired, exiting.

Hi,

Thank you @carpii for looking at this, I have been out of town and was keen to get it up and running again.

Sadly, my lack of knowledge has me stuck again, every time I run it I get -

[error] [phantom] Wait timeout of 5000ms expired, exiting.
Wait timeout of 5000ms expired, exiting.

I tried increasing the casperjs timeout, same result, just takes longer to fail...

I have even spun up clean VM (CentOS7) and started from scratch, same results.

If you are able to offer any pointers as to where I am going wrong, would be much appreciated.

Google Chrome version error

Hey all, last few days getting this issue (and no downloads, presumably cos need to press the button on the warning) . Have checked / ran npm ci (and then npm install just in case, same error. Do we need to update puppeteer or similar?

"#pinsentryCode0" field missing

Thank you for this fine tool. However, when I try to log in I get:

Couldn't find selector "#pinsentryCode0" on page https://bank.barclays.co.uk/olb/authlogin/loginAppContainer.do#/authentication.

(or "#mobilePinsentryCode0 for mobile PINsentry").

Am I doing something wrong? This is a business account and I'm wondering if the arrangement of the login screens is more complicated than for a personal account. After I've typed my name and customer number and confirmed, I then have to click one of two buttons: "Log in with PINsentry" or "Log in with passcode" before I get to a screen with the "pinsentryCode0" or "mobilePinsentryCode0" field. I can't see any way to bypass the screen that asks you to choose between PINsentry and passcode.

Seems busted again, just me?

casperjs '--ignore-ssl-errors=true' '--ssl-protocol=any' /usr/local/root/acc_mon2/barclayscrape/get_data.js '--mcode=XXXXXX' '--pcode=XXXXX'
[error] [phantom] Timeout waiting for a#logout
[error] [phantom] Failed to save screenshot to /usr/local/root/acc_mon2/waitfor-timeout.png; please check permissions
[error] [phantom] Screenshot saved to waitfor-timeout.png.
CasperError: Cannot dispatch mousedown event on nonexistent selector: button[title="Log in to Online Banking"]
/usr/local/root/acc_mon2/phantomjs:/platform/casper.js:1517 in mouseEvent
/usr/local/root/acc_mon2/phantomjs:/platform/casper.js:558 in click
/usr/local/root/acc_mon2/phantomjs:/platform/barclayscrape.js:201 in loginTimeout
/usr/local/root/acc_mon2/phantomjs:/platform/casper.js:2477 in _check
[error] [phantom] Failed to save screenshot to /usr/local/root/acc_mon2/error.png; please check permissions
[error] [phantom] CasperError: Cannot dispatch mousedown event on nonexistent selector: button[title="Log in to Online Banking"]
CasperError: Cannot dispatch mousedown event on nonexistent selector: button[title="Log in to Online Banking"]

Whole new shiny UI :(

Guessing I'm not the only one that noticed.. Mebbe I should learn how to pupetteer

is there an issue with syncprompt?

[carpii@flake:~/dev/barclayscrape]$ node -v && npm -v && npx -v
v12.1.0
6.9.0
6.9.0

Im trying to check out the git repo, for development purposes, but I cannot get far enough to run it....

Is there an issue with syncprompt at the moment, or am I just doing something wrong?

[carpii@flake:~/dev]$ git clone https://github.com/russss/barclayscrape.git
Cloning into 'barclayscrape'...
remote: Enumerating objects: 335, done.
remote: Total 335 (delta 0), reused 0 (delta 0), pack-reused 335
Receiving objects: 100% (335/335), 240.34 KiB | 776.00 KiB/s, done.
Resolving deltas: 100% (190/190), done.

[carpii@flake:~/dev]$ cd barclayscrape

[carpii@flake:~/dev/barclayscrape]$ npm i

> [email protected] install /Users/carpii/dev/barclayscrape/node_modules/puppeteer
> node install.js

Downloading Chromium r599821 - 82.9 Mb [====================] 99% 0.0s
Chromium downloaded to /Users/carpii/dev/barclayscrape/node_modules/puppeteer/.local-chromium/mac-599821

> [email protected] install /Users/carpii/dev/barclayscrape/node_modules/syncprompt
> node-gyp rebuild

  CXX(target) Release/obj.target/sync_prompt/src/sync_prompt.o
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:221:
In file included from ../../nan/nan_converters.h:67:
../../nan/nan_converters_43_inl.h:22:1: warning: 'ToBoolean' is deprecated: ToBoolean can never throw. Use Local version. [-Wdeprecated-declarations]
X(Boolean)
^
../../nan/nan_converters_43_inl.h:18:12: note: expanded from macro 'X'
      val->To ## TYPE(isolate->GetCurrentContext())                            \
           ^
<scratch space>:201:1: note: expanded from here
ToBoolean
^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:2523:3: note: 'ToBoolean' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("ToBoolean can never throw. Use Local version.",
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:221:
In file included from ../../nan/nan_converters.h:67:
../../nan/nan_converters_43_inl.h:40:1: warning: 'BooleanValue' is deprecated: BooleanValue can never throw. Use Isolate version. [-Wdeprecated-declarations]
X(bool, Boolean)
^
../../nan/nan_converters_43_inl.h:37:15: note: expanded from macro 'X'
  return val->NAME ## Value(isolate->GetCurrentContext());                     \
              ^
<scratch space>:208:1: note: expanded from here
BooleanValue
^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:2561:3: note: 'BooleanValue' has been explicitly marked deprecated here
  V8_DEPRECATED("BooleanValue can never throw. Use Isolate version.",
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:307:29: note: expanded from macro 'V8_DEPRECATED'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:222:
In file included from ../../nan/nan_new.h:189:
../../nan/nan_implementation_12_inl.h:103:42: error: no viable conversion from 'v8::Isolate *' to 'Local<v8::Context>'
  return scope.Escape(v8::Function::New( isolate
                                         ^~~~~~~
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:183:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'v8::Isolate *' to 'const v8::Local<v8::Context> &'
      for 1st argument
class Local {
      ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:183:7: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'v8::Isolate *' to 'v8::Local<v8::Context> &&' for
      1st argument
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:187:13: note: candidate template ignored: could not match 'Local<type-parameter-0-0>' against 'v8::Isolate *'
  V8_INLINE Local(Local<S> that)
            ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:4126:22: note: passing argument to parameter 'context' here
      Local<Context> context, FunctionCallback callback,
                     ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:222:
In file included from ../../nan/nan_new.h:189:
../../nan/nan_implementation_12_inl.h:337:37: error: too few arguments to function call, expected 2, have 1
  return v8::StringObject::New(value).As<v8::StringObject>();
         ~~~~~~~~~~~~~~~~~~~~~      ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:5380:3: note: 'New' declared here
  static Local<Value> New(Isolate* isolate, Local<String> value);
  ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:222:
In file included from ../../nan/nan_new.h:189:
../../nan/nan_implementation_12_inl.h:337:58: error: expected '(' for function-style cast or type construction
  return v8::StringObject::New(value).As<v8::StringObject>();
                                         ~~~~~~~~~~~~~~~~^
../../nan/nan_implementation_12_inl.h:337:60: error: expected expression
  return v8::StringObject::New(value).As<v8::StringObject>();
                                                           ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1063:44: error: no matching member function for call to 'ToString'
      v8::Local<v8::String> string = from->ToString();
                                     ~~~~~~^~~~~~~~
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:2528:44: note: candidate function not viable: requires single argument 'context', but no arguments were provided
  V8_WARN_UNUSED_RESULT MaybeLocal<String> ToString(
                                           ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:2544:35: note: candidate function not viable: requires single argument 'isolate', but no arguments were provided
                    Local<String> ToString(Isolate* isolate) const);
                                  ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1073:37: error: cannot initialize a parameter of type 'v8::Isolate *' with an lvalue of type 'char *'
        length_ = string->WriteUtf8(str_, static_cast<int>(len), 0, flags);
                                    ^~~~
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:2738:26: note: passing argument to parameter 'isolate' here
  int WriteUtf8(Isolate* isolate, char* buffer, int length = -1,
                         ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1847:28: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
    New(persistentHandle)->Set(New(key).ToLocalChecked(), value);
                           ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1853:28: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
    New(persistentHandle)->Set(key, value);
                           ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1859:28: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
    New(persistentHandle)->Set(index, value);
                           ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:3367:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1865:32: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
        New(persistentHandle)->Get(New(key).ToLocalChecked()));
                               ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1871:48: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
    return scope.Escape(New(persistentHandle)->Get(key));
                                               ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
../../nan/nan.h:1876:48: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
    return scope.Escape(New(persistentHandle)->Get(index));
                                               ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:3416:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(uint32_t index));
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:2690:
../../nan/nan_object_wrap.h:24:25: error: no member named 'IsNearDeath' in 'Nan::Persistent<v8::Object, v8::NonCopyablePersistentTraits<v8::Object> >'
    assert(persistent().IsNearDeath());
           ~~~~~~~~~~~~ ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
    (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
                        ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:2690:
../../nan/nan_object_wrap.h:67:18: warning: 'MarkIndependent' is deprecated: Weak objects are always considered independent. Use TracedGlobal when trying to use EmbedderHeapTracer. Use a strong handle
      when trying to keep an object alive. [-Wdeprecated-declarations]
    persistent().MarkIndependent();
                 ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8.h:563:3: note: 'MarkIndependent' has been explicitly marked deprecated here
  V8_DEPRECATED(
  ^
/Users/carpii/.node-gyp/12.1.0/include/node/v8config.h:307:29: note: expanded from macro 'V8_DEPRECATED'
  declarator __attribute__((deprecated(message)))
                            ^
In file included from ../src/sync_prompt.cc:1:
In file included from ../../nan/nan.h:2690:
../../nan/nan_object_wrap.h:124:26: error: no member named 'IsNearDeath' in 'Nan::Persistent<v8::Object, v8::NonCopyablePersistentTraits<v8::Object> >'
    assert(wrap->handle_.IsNearDeath());
           ~~~~~~~~~~~~~ ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
    (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
                        ^
9 warnings and 8 errors generated.
make: *** [Release/obj.target/sync_prompt/src/sync_prompt.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:196:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:256:12)
gyp ERR! System Darwin 18.6.0
gyp ERR! command "/usr/local/Cellar/node/12.1.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/carpii/dev/barclayscrape/node_modules/syncprompt
gyp ERR! node -v v12.1.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/carpii/.npm/_logs/2019-06-30T22_40_35_307Z-debug.log

Export to CSV does not work anymore

Export to OFX doesnt work

Exporting to OFX isn't currently working when using the mobile pinsentry option, nothing appears to get exported.

$ ./export_csv.js 
Identify with mobile pin sentry: 
32388690
Successfully logged in
Exporting hacman account: 20254363237958 (37 rows)
$ ./get_ofx.js 
Identify with mobile pin sentry: 
11950028
Successfully logged in
$ ./get_combined_ofx.js 
Identify with mobile pin sentry: 
41475645
Successfully logged in
$

Couldn't find selector ".accounts-body" on page https://bank.barclays.co.uk/olb/balances/digital/btr/home/business

Nothing interesting in the error.png - just looks like the main "post login" page.

I logged in and it looks like (now, for me) the "Show recent transactions" link on the main page now makes a modal dialog (which still contains a link to see everything..)

This has been happening last couple of days - possibly longer, but I managed mess up my emvtool for a bit so got different errors ;)

Any ideas?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.