Error with basic CLI usage about a11ywatch HOT 10 CLOSED

Thanks @j-mendez - absolutely right about Ubuntu. Debian flavoured, but that's not always close enough.

I had run a11ywatch start. That seemed to basically work which is why I was a bit surprised with the failure.

I'll have to give this a deeper try later. Ran that script and still got an error when running it.

from a11ywatch.

Also, not sure if this is expected:

$ a11ywatch scan --url https://a11ywatch.com -s
{"code":0,"data":null,"message":"","success":false}

Maybe I installed it incorrectly... Looking up I did find this error in the terminal when installing:

  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  run pkg_config fail: "Could not run `\"pkg-config\" \"--libs\" \"--cflags\" \"openssl\"`\nThe pkg-config command could not be found.\n\nMost likely, you need to install a pkg-config package for your OS.\nTry `apt install pkg-config`, or `yum install pkg-config`,\nor `pkg install pkg-config`, or `apk add pkgconfig` depending on your distribution.\n\nIf you've already installed it, ensure the pkg-config command is one of the\ndirectories in the PATH environment variable.\n\nIf you did not expect this build to link to a pre-installed system library,\nthen check documentation of the openssl-sys crate for an option to\nbuild the library from source, or disable features or dependencies\nthat require pkg-config."

  --- stderr
  thread 'main' panicked at '

  Could not find directory of OpenSSL installation, and this `-sys` crate cannot
  proceed without this knowledge. If OpenSSL is installed and this crate had
  trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
  compilation process.

  Make sure you also have the development packages of openssl installed.
  For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

  If you're in a situation where you think the directory *should* be found
  automatically, please open a bug at https://github.com/sfackler/rust-openssl
  and include information about your system as well as this message.

  $HOST = x86_64-unknown-linux-gnu
  $TARGET = x86_64-unknown-linux-gnu
  openssl-sys = 0.9.79


  It looks like you're compiling on Linux and also targeting Linux. Currently this
  requires the `pkg-config` utility to find OpenSSL but unfortunately `pkg-config`
  could not be found. If you have OpenSSL installed you can likely fix this by
  installing `pkg-config`.

  ', /home/mike/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.79/build/find_normal.rs:191:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `a11ywatch_cli v0.8.4`, intermediate artifacts can be found at `/tmp/cargo-installzumqXz`

    at ChildProcess.exithandler (child_process.js:303:12)
    at ChildProcess.emit (events.js:315:20)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5) {
  killed: false,
  code: 101,
  signal: null,
  cmd: 'cargo install a11ywatch_cli --vers 0.8.4 '
}
+ [email protected]
updated 1 package in 14.905s

from a11ywatch.

Hi @mgifford , thanks for reporting the issue. Installing the CLI on ubuntu(I think you are referring to not debian) needs pk-config and openssl installed. The project uses system dependancies and depending on the flavor of linux. OpenSSL is needed for secure network IO. Here is an example of setting up the crawler-ubuntu for ubuntu.

The command bellow should get you up and running.
apt-get update && apt upgrade -y \ && apt-get install -y --no-install-recommends \ build-essential gcc cmake libc6 libssl-dev pkg-config.

If you are using docker make sure to leave some RAM outside since the IO work is dependent on the system kernel host.

from a11ywatch.

I will update the docs on the Ubuntu getting started and ideal specs and etc. The command a11ywatch start is also required before running scans in order to bring up the server locally.

from a11ywatch.

@mgifford I can re-open the issue if issues persist. The default for the A11yWatch start command uses docker we made some changes to the images that support for the CLI above 0.6 is preferred since it defaults to using gRPC streams instead of http.

Here is an example curl that can be used to test the full e2e of the system.

Basic health check:

curl --location --request GET 'http://localhost:3280/_internal_/healthcheck'

Full e2e:

curl --location --request POST 'http://localhost:3280/api/crawl' \
--header 'Content-Type: application/json' \
--data-raw '{
    "websiteUrl": "https://www.a11ywatch.com",
    "subdomains": false,
    "tld": false,
    "robots": false
}'

from a11ywatch.

Looks like it is healthy on both my Mac & Ubuntu:

% curl --location --request GET 'localhost:3280/internal/healthcheck' \
--header 'Content-Type: application/x-www-form-urlencoded'
--data-urlencode 'password=drake'
{"status":"healthy"}%

The "Full e2e" on both systems just sits there with no results.

from a11ywatch.

Looks like it is healthy on both my Mac & Ubuntu:

% curl --location --request GET 'localhost:3280/internal/healthcheck' \ --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'password=drake' {"status":"healthy"}%

The "Full e2e" on both systems just sits there with no results.

That sounds like good news!

Ok, let us try a basic single page scan to make sure there is not an issue with the micro services / chrome area.

curl --location --request POST 'http://localhost:3280/api/scan' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://a11ywatch.com"
}'

The next step would be to try using the crawl instead of scan.

If the system is just hanging it is most likely an issue with the crawler and the native platform.

The latest CLI changes some of the defaults on the a11ywatch/crawler image which allow more universal appropriate builds. Here is a link to all the crawlers. You can use any depending on the OS on older version of the CLI by using the env var CRAWLER_IMAGE.

from a11ywatch.

In order to change the env config for the docker startup use a11ywatch --find-tmp-dir and create a .env file if it is missing with the var CRAWLER_IMAGE=debian, ubuntu, alpine etc. This is only required on older versions of the CLI or the stale npm version of the CLI that was outdated until a couple minutes ago.

from a11ywatch.

So I deleted the tmp directory (not that you said I should) and created a .env file with:
var CRAWLER_IMAGE=debian, ubuntu, alpine in it.

From Ubuntu I am now seeing:

curl --location --request POST 'http://localhost:3280/api/scan' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>     "url": "https://a11ywatch.com"
> }'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /api/scan</pre>
</body>
</html>

from a11ywatch.

@mgifford the env var needs to be inside a .env file in the format CRAWLER_IMAGE= after = one of the image tags not the list. The file also needs to be inside the tmp folder that gets created on a11ywatch build.

from a11ywatch.