artpi / roam-research-private-api Goto Github PK

Private API to enable API access for Roam Research. Now you can connect Roam to your other projects.

Home Page: https://piszek.com/roam/roam-api/

JavaScript 100.00%

roam-research roamcult roam api-wrapper puppeteer chrome

roam-research-private-api's Introduction

Roam Private API

This project exposes command line tool (roam-api) and a node library to connect Roam Research to your other software. You can use it in bash scripts, Github actions, or as a dependency of your project.

How does it work?

It looks like Roam is not providing a REST API any time soon. If you want to bridge Roam with your other software, you can do so from within Roam (with JavaScript), but that has limited number of use cases. Without a REST API, this project launches an invisible browser and performs automated actions, just as you would manually. No need to install chrome, this library comes with one. It uses your login and password, so this won't work if you are using Google login. It wraps around import/export functionality and actions exposed via roamAlphaApi.

Command line tool `roam-api`

This package exposes a roam-api tool in your system. You can use it to automate Roam and bridge other systems with your Roam graph.

Installation:

This entire library is build on node, so you need node v12 and npm v6 in your system. You can install the package the following way:

npm i -g roam-research-private-api

Now you can use a variety of commands. All command take the following arguments, which you can also set as environmental variables:

-g, --graph or env variable ROAM_API_GRAPH - this is your graph name
-e, --email or env variable ROAM_API_EMAIL - email to log into your Roam
-p, --password or env variable ROAM_API_PASSWORD - password to your Roam.

`roam-api export` will export your Roam graph to a directory of your choice.

This example will export the graph to your desktop. It will appear as "db.json".

roam-api export ~/Desktop

It can also push the new version of the graph to an URL of your choosing. That way, you can upload the graph to some other system or use it with Zapier and similar tools.

roam-api export ~/Desktop http://secret.url?token=secret_token.

`roam-api search` will search your Roam graph for a phrase:

roam-api search "potatoes"

Result will be JSON array of objects { blockUid, pageTitle, string }

`roam-api-query` will let you do a full Datalog query.

This will find all block uids in your database which have the content "Import".

roam-api query '[:find ?uid :where [?b :block/string "Import"] [?b :block/uid ?uid]]'

Check out this fantastic article to know more about the Roam data structure.

`roam-api create` create a block under specified uid. If no uid is provided, it will be inserted into your daily page:

roam-api create "This will be prepended to my daily page"

Library to use in your project.

As mentioned, this is also a library that you can use within your project. Here are examples on how to do so:

All the functionality in the roam-api tool
Sync your Roam graph to Evernote - you can also use command-line utility roam-evernote-sync.
Import arbitrary data into any note
Send note to Roam using the Quick Capture feature

Pull requests welcome and I take no responsibility in case this messes up your Roam Graph :).

roam-research-private-api's People

Contributors

Stargazers

Watchers

roam-research-private-api's Issues

Constructor 'options' loader error

Hey! I am trying to use this API to run some queries but am having issues initializing the API instance.
Every time I try to run it, I get an issue about needing a webpack loader for the options param -- no idea why it would flag that. Any advice? Do I need to add a JSON loader? Thanks in advance!!

waitFor is deprecated and will be removed in a future release

After calling the roam-quick command a notification was displayed:
waitFor is deprecated and will be removed in a future release. See puppeteer/puppeteer#6214 for details and how to migrate your code.

Documentation for more use cases

I'd love to see more documentation here (or on your blog). Is it possible to use this API to add text to a particular note? Or create a new note?
Thanks!

Cannot load the login screen

After running roam-api export ~/Downloads/ -g $ROAM_API_GRAPH -e $ROAM_API_EMAIL -p $ROAM_API_PASSWORD got this

Cannot load the login screen!
(node:6165) UnhandledPromiseRejectionWarning: TimeoutError: waiting for selector `input[name=email]` failed: timeout 60000ms exceeded
    at new WaitTask (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:505:34)
    at DOMWorld.waitForSelectorInPage (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:416:26)
    at Object.internalHandler.waitFor (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/QueryHandler.js:31:77)
    at DOMWorld.waitForSelector (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:312:29)
    at Frame.waitForSelector (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:842:51)
    at Page.waitForSelector (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1285:33)
    at RoamPrivateApi.logIn (/usr/local/lib/node_modules/roam-research-private-api/RoamPrivateApi.js:197:20)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:6165) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:6165) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Error when login in

It looks like your script has an issue finding the correct input field in the login form.

/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26
        throw new Error(message);
              ^

Error: No node found for selector: input[name=email]
    at Object.exports.assert (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26:15)
    at DOMWorld.type (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:306:21)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at async RoamPrivateApi.logIn (/usr/local/lib/node_modules/roam-research-private-api/RoamPrivateApi.js:55:3)
    at async RoamPrivateApi.quickCapture (/usr/local/lib/node_modules/roam-research-private-api/RoamPrivateApi.js:87:3)

Export of all available formats

Love this, thank you for your work Artur!

Would it be possible to add an option for getting all export formats in one? So JSON, Markdown and EDN?

Puppeteer won't install on M1 Mac

There is no puppeteer version for M1 macs, so it fails with

npm WARN old lockfile Error: Integrity checksum failed when using sha512: wanted sha512-5GCgdeevIXwR3cW4Qj5XWC5MO1iSCz8+IPn0mMw6awAt/PBiey8yyO7MhePRsaMqghJAhg6Q3QLYWSnUHWkG6A== but got sha512-7PtVymN48hGcO4fGjybyBSIWDsLU4H4XlvOHfq91pz9kkGlonzwTfYkaIEwiRg/dAJF9YlbsduBAgtYLi+8cFg==.
npm WARN old lockfile     at /Users/artpi/.nvm/versions/node/v17.0.0/lib/node_modules/npm/node_modules/pacote/lib/registry.js:145:39
npm WARN old lockfile     at async Array.<anonymous> (/Users/artpi/.nvm/versions/node/v17.0.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:741:9)
npm WARN old lockfile  Could not fetch metadata for [email protected] Error: Integrity checksum failed when using sha512: wanted sha512-5GCgdeevIXwR3cW4Qj5XWC5MO1iSCz8+IPn0mMw6awAt/PBiey8yyO7MhePRsaMqghJAhg6Q3QLYWSnUHWkG6A== but got sha512-7PtVymN48hGcO4fGjybyBSIWDsLU4H4XlvOHfq91pz9kkGlonzwTfYkaIEwiRg/dAJF9YlbsduBAgtYLi+8cFg==.
npm WARN old lockfile     at /Users/artpi/.nvm/versions/node/v17.0.0/lib/node_modules/npm/node_modules/pacote/lib/registry.js:145:39
npm WARN old lockfile     at async Array.<anonymous> (/Users/artpi/.nvm/versions/node/v17.0.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:741:9) {
npm WARN old lockfile   code: 'EINTEGRITY'

The solution is to install with env PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true npm install
And download Chromium separately via this comment puppeteer/puppeteer#6622 (comment)

Then the puppeteer will have to be pointed to by command line

Import doesn't work anymore

My script to import files in roam uses this library, since a couple of days there is an error:

ERROR occured:  Error: Node is either not visible or not an HTMLElement
    at ElementHandle._clickablePoint (/home/runner/work/road-to-roam/road-to-roam/node_modules/puppeteer/lib/cjs/puppeteer/common/JSHandle.js:329:19)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async ElementHandle.click (/home/runner/work/road-to-roam/road-to-roam/node_modules/puppeteer/lib/cjs/puppeteer/common/JSHandle.js:390:26)
    at async DOMWorld.click (/home/runner/work/road-to-roam/road-to-roam/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:277:9)
    at async RoamPrivateApi.import (/home/runner/work/road-to-roam/road-to-roam/node_modules/roam-research-private-api/RoamPrivateApi.js:233:3)

The error occurs here (line 233):

roam-research-private-api/RoamPrivateApi.js

Lines 230 to 236 in 0fe3579

    
           // Sets the value of the file input to fileToUpload 
        
           inputUploadHandle.uploadFile( fileName ); 
        
           await this.page.waitForSelector( '.bp3-dialog .bp3-intent-primary' ); 
        
           await this.page.click( '.bp3-dialog .bp3-intent-primary' ); 
        
           await this.page.waitForTimeout( 3000 ); 
        
           await this.removeImportBlockFromDailyNote(); 
        
           return;

At first glance I couldn't find anything, since in Roam the class names didn't change (as far as I can tell). Not sure what the problem is.

Doesn't work if you do Google SSO

I think that this won't work if you use your Google account to auth with Roam, right? Maybe good to drop a note about that?

May I wrap this project on Python?

I'm developing my own automation system for manage my data resources using python.
Mostly, I stacked all my informations on Roam Research. But roam isnt support any public api to call.
If you allow to wrap this project into python, I'll notice this project as a source of my library.
Please consider it.

Log in with Google

The current api requires you to use email and password as tags when using the API.

This is a struggle for those who created their account with Google (or some other social sign-in) because they cannot log in. Is there any way that I can sign in with Google instead of regular?

Unexpected token ; when running from Command Line and from /examples/

I attempted to run from both the command line for search and download. Consistently getting the below error when using the CLI and running the example files.

(base)  roam-research-private-api/examples % node download.js        
/usr/local/lib/node_modules/roam-research-private-api/RoamPrivateApi.js:13
	options;
	       ^

SyntaxError: Unexpected token ;
    at Module._compile (internal/modules/cjs/loader.js:721:23)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:690:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/usr/local/lib/node_modules/roam-research-private-api/examples/download.js:1:24)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)

TimeoutError: waiting for selector `.bp3-icon-more` failed: timeout 30000ms exceeded

In the same call as #5 an additional error showed: I have the feeling they are not connected, but could be wrong. ....

Node: version 14.15
npm version 6.14.10

TimeoutError: waiting for selector .bp3-icon-more failed: timeout 30000ms exceeded

(node:10362) UnhandledPromiseRejectionWarning: TimeoutError: waiting for selector .bp3-icon-more failed: timeout 30000ms exceeded
at new WaitTask (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:505:34)
at DOMWorld.waitForSelectorInPage (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:416:26)
at Object.internalHandler.waitFor (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/QueryHandler.js:31:77)
at DOMWorld.waitForSelector (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:312:29)
at Frame.waitForSelector (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:842:51)
at Page.waitForSelector (/usr/local/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1285:33)
at RoamPrivateApi.logIn (/usr/local/lib/node_modules/roam-research-private-api/RoamPrivateApi.js:59:19)
at async RoamPrivateApi.quickCapture (/usr/local/lib/node_modules/roam-research-private-api/RoamPrivateApi.js:87:3)
(Use node --trace-warnings ... to show where the warning was created)
(node:10362) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:10362) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

puppeteer / macos

I'm running into an issue when installing on macos. Help is very much appreciated!

npm ERR! code 1
npm ERR! path /opt/homebrew/lib/node_modules/roam-research-private-api/node_modules/puppeteer
npm ERR! command failed
npm ERR! command sh -c node install.js
npm ERR! The chromium binary is not available for arm64:
npm ERR! If you are on Ubuntu, you can install with:
npm ERR!
npm ERR! apt-get install chromium-browser
npm ERR!
npm ERR! /opt/homebrew/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:112
npm ERR! throw new Error();
npm ERR! ^
npm ERR!
npm ERR! Error
npm ERR! at /opt/homebrew/lib/node_modules/roam-research-private-api/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:112:19
npm ERR! at FSReqCallback.oncomplete (node:fs:198:21)
npm ERR!
npm ERR! Node.js v17.0.1

Missing option for inputs

Hi! Thanks for creating this library - it's super useful.

Datalog queries take a second parameter, "inputs", that allows concise pattern matching on specific value. The roamAlphaAPI interface provides access to this, but it's missing from the interface provided by this library.

We should add this interface. I'll make a PR to do so.

	// Sets the value of the file input to fileToUpload
	inputUploadHandle.uploadFile( fileName );
	await this.page.waitForSelector( '.bp3-dialog .bp3-intent-primary' );
	await this.page.click( '.bp3-dialog .bp3-intent-primary' );
	await this.page.waitForTimeout( 3000 );
	await this.removeImportBlockFromDailyNote();
	return;

artpi / roam-research-private-api Goto Github PK

roam-research-private-api's Introduction

Roam Private API

How does it work?

Command line tool roam-api

Installation:

roam-api export will export your Roam graph to a directory of your choice.

roam-api search will search your Roam graph for a phrase:

roam-api-query will let you do a full Datalog query.

roam-api create create a block under specified uid. If no uid is provided, it will be inserted into your daily page: