Seems busted again, just me? about barclayscrape HOT 10 CLOSED

I haven't had this particular error, but I agree, something is not quite right with the master branch

Mine sometimes fails, although re-running it often succeeds. Have you tried running it again, or it fails each and every time?

Not exactly sure whats causing it at the moment... I get the impression Barclays might be using measures to try to sabotage any automation, it's just a case of figuring out how to circumvent it

Either way, you should fix your filesystem permissions of possible.
It seems your script does not have write access to dump screenshots to the run dir

Also "/usr/local/root" not sure if this implies you are running it under root account, but this is always a bad idea :-)

from barclayscrape.

Hi,

The error repeats, in fact I have not seen it work in a few weeks, I have my suspicions that the page is not loading in time (I am currently based in Asia so +latency) if there is any merit to this theory, is there any way to add an (adjustable?) delay to the script to allow the page to load fully before commencing 'magic' ?

Of course I may be way off the mark :)

from barclayscrape.

It does not run as root, I just never got around to moving after initial setup as root - I tend to run things as root while I test, so if it break when run as non root I can be fairly certain I have hit a permissions issue, which is a frequent occurrence....

from barclayscrape.

@Sc0th "It does not run as root"

Ok cool. Just thought it was worth mentioning

from barclayscrape.

@Sc0th "I am currently based in Asia so +latency"

That may factor into it, although I think there's other issues too. Im in UK on super fast broadband, and even mine fails intermittently.

I'll try to take a look at the weekend and see if I can make the base timeout configurable, doubt it will fix it 100% but it may be a step in the right direction for you

Presumably you have no issues logging in manually with passcode and memorable word? Its possible Barclays are treating your connection as higher risk simply because its overseas

from barclayscrape.

Thank you, appreciate you taking the time as always, if there is anything I can do to help....

I will at least fix the permissions issue (school boy error!) and see if they dump provide any insight

from barclayscrape.

@Sc0th that error is consistent with what happens if the login fails. Without a screenshot there's not much info here, so if you can get one next time then I might be able to take a look.

There have been some general issues the last few days because Barclays has been fitfully trying to roll their SSL certificate. They keep pushing the new out without the intermediate certificate which breaks validation on phantomjs, but then rolling it back 6-12 hours later when they realise... @carpii - this is probably related to the problems you're seeing.

@Sc0th: as you're running with --ignore-ssl-errors=true this isn't related to your problem. However I strongly recommend you don't use this option as it severely degrades your security.

from barclayscrape.

@russss Interesting, thanks. Would explain why my cronned scrape has been failing, but then often works later when I rerun it manually.

@Sc0th have to agree with russs, it's really important not to use --ignore-ssl-errors (especially connecting from Asia).
Bear in mind Barclays definitely don't approve of this access method, so if your account was breached and they had reason to believe you had been automating it without PINSentry, there's no guarantee they will reimburse any lost funds

from barclayscrape.

Hi!

Finally found some time to take a look at this, for anyone else seeing the same issue, defo seems to be a timeout thing, I upped the timeout value on the following lines in barclayscrape.js and all is well

215       }, 50000);
216     }, 50000);

Would be great if this could be added as a variable to be fed in.....

from barclayscrape.

@Sc0th I made a bit of progress on this, but to pass the timeout for all operations (logging in, statement downloads etc), is not easy in its current form.

I will try to revisit it at some point, but in the meantime all I can suggest is to host your own edits

from barclayscrape.