Couldn't find selector ".accounts-body" on page https://bank.barclays.co.uk/olb/balances/digital/btr/home/business about barclayscrape HOT 30 CLOSED

thanks, I'll look into porting these changes into my site_redesign branch

This feels like our best bet.. I'm not sure russ is still maintaining this repo, and it's becoming very difficult to cover every edge case Barclays throws at us

Can you give me an idea how you are using the get_ofx feature? Maybe there's a better way to approach all of this

from barclayscrape.

awesome, thanks for confirming.

I'll merge site_redesign into master so you can switch to using that shortly

Mind closing these two issues? Any new issues should be reported on my repo, as I think russ is handing it over

from barclayscrape.

.. retrying @carpii 's branch..

from barclayscrape.

Sadly that gets me our ole friend:
Error: No element found for selector: input[name="lastDigits"] at assert (/mnt/allthespace/usrsrc/extern/barclayscrape/node_modules/puppeteer-core/lib/cjs/puppeteer/util/assert.js:28:15)

from barclayscrape.

What commandline are you using (loginmethod and feature)?

from barclayscrape.

system("$ENV{BARCLAYSCRAPE}/barclayscrape.js --otp $ENV{CODE} get_ofx $ENV{ACCESS_HOME}");

from barclayscrape.

You might like to apply this patch from my branch

carpii@fdc2b20

It sounds like they have moved you over to the new site design, but you are still being presented with the old login markup. I don't have a chip and PIN machine available for testing currently

from barclayscrape.

Hmm, tried that, got Couldn't find selector "#membership0" on page https://bank.barclays.co.uk/olb/authlogin/loginAppContainer.do#/identification. Screenshot saved to ./error.png.

tried site_redesign branch yet again (cos clearly something's changed again today), got: Couldn't find selector "#welcomePMsg" on page https://bank.barclays.co.uk/olb/balances/digital/btr/home/business. Screenshot saved to ./error.png.

Isnt this fun..

from barclayscrape.

I don't know what to suggest at this stage, sorry.

You may need to start debugging it yourself, or find a dev who is seeing the same problem as you
Not sure many are using this library though, so might be tricky

Is it Barclays Business Banking you are using, or just personal account(s)?

from barclayscrape.

No worries I'll find some time to have a go.. is there a way to test it other than to keep relaunching the whole thing / browser?

Its a business account

from barclayscrape.

I normally debug it straight from vscode

Add a ./vscode/launch.json with whatever args you need

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "name": "plogin / get_ofx_combined",
            "skipFiles": [
                "<node_internals>/**"
            ],
            "program": "${workspaceFolder}/barclayscrape.js",
            "args": ["--plogin", "get_ofx_combined"]
        }
    ]
}

Its a business account

ok, that explains a lot

from barclayscrape.

Hmm, well this is fun.. the biz page for listing accounts has a different set of classes etc than (presumably) the personal ones.. mine now works (for get_ofx at least) with session.js like this:

--- a/session.js
+++ b/session.js
@@ -53,7 +53,11 @@ class Session {
   async ensureLoggedIn() {
     // Check that we're looking at the logged in homepage and throw an
     // error if we aren't.
-    await u.wait(this.page, '.accounts-body');
+    // await u.wait(this.page, '.accounts-body');
+    await u.wait(this.page, 'div.c-section.c-section--primary');
     this.logged_in = true;
   }
 
@@ -145,13 +149,14 @@ class Session {
   }
 
   async accounts() {
-    let accData = await this.page.$$eval('.o-account-list__item', accounts => {
+//    let accData = await this.page.$$eval('.o-account-list__item', accounts => {
+    let accData = await this.page.$$eval('.c-account__content', accounts => {
       return accounts.map(acc => {
         return [
-          acc.querySelector('.my-account-link').getAttribute('href'),
-          acc.querySelector('.o-account').getAttribute('id').replace(/[^0-9]/g,
 ''),
-          acc.querySelector('.my-account-link').textContent.trim(),
-          acc.querySelector('.o-account__balance-head') !== null ? acc.querySelector('.o-account__balance-head').textContent.trim().replace(/[£$€]/g, '') : ''
+          acc.querySelector('.c-account__body a').getAttribute('href'),
+          Array.from(acc.querySelectorAll('.c-account__detail--multi span')).map((span) => span.textContent.replace(/[^0-9]/g, '')).join(''),
+          acc.querySelector('.c-account__body a').textContent.trim(),
+          acc.querySelector('.c-account__balance') !== null ? acc.querySelector('[description="Available balance"]').textContent.trim().replace(/[^0-9]/g, '') : ''
         ]
       });
     });
@@ -176,7 +181,8 @@ class Session {
 
   async home() {
     await u.click(this.page, '[aria-label="Home"]');
-    await u.wait(this.page, '.accounts-body');
+    // await u.wait(this.page, '.accounts-body');
+    await u.wait(this.page, 'div.c-section.c-section--primary');
   }
 }

there's probably less ugly ways to do the account data fetching.. its a bit of a mess tho!

from barclayscrape.

I'm running a daily cronjob, which calls a perl script, and in that perl script I run the line I showed above: system("$ENV{BARCLAYSCRAPE}/barclayscrape.js --otp $ENV{CODE} get_ofx $ENV{ACCESS_HOME}");

After that dumps a .ofx in the current directory, I rename it to YYYY-MM-DD.ofx, extract the transactions, and toss em in a database. I mostly need the date, amount, and "ref" fields.

from barclayscrape.

This feels like our best bet.. I'm not sure russ is still maintaining this repo, and it's becoming very difficult to cover every edge case Barclays throws at us

Yeah, I'm no longer using this, and it does seem like it's getting more complicated. If you're willing to keep an eye on it for the moment, perhaps it's best if I archive this repository and leave a link to yours?

from barclayscrape.

I dunno why this only just occurred to me - but does Barclays support openbanking? I wonder if it would be less faff to use one of the budgeting thingies that support that, and export the data from there? (Anyone trust any of the budgeting things.. ?)

from barclayscrape.

Yes, Barclays does support Open Banking (they're required to), but you'll need to find an open banking third party which gives you access to the data. I haven't seen one yet which doesn't involve a large amount of hassle.

from barclayscrape.

required and reality dont necessarily match (looking at you, co-op bank).. I shall research!

from barclayscrape.

Actually the one Im using for personal stuff lets you export transactions fairly easily - Moneyhub

from barclayscrape.

Yeah, I'm no longer using this, and it does seem like it's getting more complicated. If you're willing to keep an eye on it for the moment, perhaps it's best if I archive this repository and leave a link to yours?

I dont plan on migrating to an Open Banking platform, so I'll be maintaining this lib for as long as I can. If you want to link to my repo, I'm happy to look after it

from barclayscrape.

Error: No element found for selector: input[name="lastDigits"] .. of course today's email says...

from barclayscrape.

Error: No element found for selector: input[name="lastDigits"] .. of course today's email says...

I don't understand what this means?

from barclayscrape.

I think it means my account now has the login page updated again / as well..

from barclayscrape.

Could you try running my site_redesign branch, on https://github.com/carpii/barclayscrape (with no custom changes)?
The diff you provided above, seems to include all the login changes already on that branch (

Run it using node --trace-warnings barclayscrape.js etc etc and then paste the stack trace

This branch doesn't include your changes you made to the accounts() routine yet, but would show whether the login feature is now working for you

from barclayscrape.

Oops missed this one, yup can do! (tho it worked again today.. sigh)

from barclayscrape.

Yup, it seems login works, it gets to the Home page, and then stops on: Couldn't find selector "#welcomePMsg" on page https://bank.barclays.co.uk/olb/balances/digital/btr/home/business screenshot shows the Home page.

trace warnings didnt seem to output anything extra tho?

node --version
v18.12.1

CODE=``emvtool -r 1 -p <pin> cap``; node --trace-warnings ./barclayscrape.js --otp $CODE list

from barclayscrape.

#welcomePMsg

you seem using an outdated version of my site_redesign branch, try a git pull

This selector was added in December, but then reverted in favour of div.c-section.c-section--primary

I've now applied your accounts() diff to my branch also, please could you give it a test?

from barclayscrape.

ooh that works, entirely in fact!

from barclayscrape.

from barclayscrape.

Is it still not appearing for you? I enabled the Issues tab yesterday.

from barclayscrape.

It is now, all good ;)

from barclayscrape.