Code Monkey home page Code Monkey logo

cctail's Introduction

Salesforce Commerce Cloud log tail

Remote tail Salesforce Commerce Cloud logs via webdav. Allow to monitor more logs at once, merging the content of all the files in a single stream. Reports either to console or a FluentD collector.

Features

  • Authentication using API Client (recommended) OR Business Manager (deprecated)
  • Interactive prompt for logs selection OR selection of logs by config file.
  • Supports configuration of multiple instances OR standard dw.json config file
  • Outputs to console OR FluentD collector
  • Multiple log tailing, with merging of log entries
  • In Console mode:
    • Sorts log entries by timestamp
    • Colors output based on log levels
    • Converts log timestamp to local timezone

Installation

$ npm i -g cctail

Requirements

  • Node >= 10

Configuration

Requires one of the following configuration files:

  • a log.conf.json file with multiple environments configured. This may be used if you want to easily switch between multiple instances
  • a standard dw.json file, tipically pointing to your working sandbox.

cctail requires a correctly configured API client id/secret OR Business Manager username/password for accessing logs via webdav. API client authentication is recommended, because it is faster after the initial authorization, and Business Manager authentication to WebDAV has been deprecated by SalesForce.

Optional Configurations

  • "profiles": - Standard log types: analytics, api, console, customdebug, customerror, customfatal, custominfo, customwarn, dbinit-sql, debug, deprecation, error, fatal, info, jobs, migration, performance, quota, sql, staging, sysevent, syslog, warn
    • "log_types": ["log", "types", "array"] (default: all log types) - In non-interactive mode, defining this will limit the log types that cctail collects to this list.
      • Standard log types: analytics, api, codeprofiler, console, customdebug, customerror, customfatal, custominfo, customwarn, dbinit-sql, debug, deprecation, error, fatal, info, jobs, migration, performance, quota, sql, staging, sysevent, syslog, warn
    • "polling_interval": nnn (default: 3) - Frequency (seconds) with which cctail will poll the logs.
      • If you are using non-interactive mode to pipe the logs elsewhere (i.e. FluentD), a longer interval is recommended (i.e. 30 or 60).
    • "refresh_loglist_interval": nnn (default: 600) - In non-interactive mode, this is the frequency (seconds) in which cctail will check the WebDAV server for new logs that match your log_types criteria.
  • "interactive": true|false (default: true) - Interactive mode asks which logs you will want to tail. If false, cctail will tail all of today's logs by default.

FluentD

NOTE: All configurations for fluentD are optional, except enabled must be set to true if you want to use it.

  • fluent:
    • "enabled": true|false (default: false) - If enabled, logs will be directed to the Fluent collector.
    • "host": "fluentd.yourco.com" (default: localhost) - FluentD collector host
    • "port": nnn (default: 24224) - FluentD collector port
    • "reconnect_interval": nnn (default: 600) - If the collector can't be reached, cctail will try to reconnect again in nnn seconds.
    • "timeout": nnn (default: 3) - Timeout to connect to FluentD collector
    • "tag_prefix": "your_tag_prefix" (default: sfcc) - All logs sent to FluentD will have this prefix, followed by the log type (i.e. "sfcc.customerror").

Sample configuration files

Sample dw.json:

{
  "hostname": "dev01-mysandbox.demandware.net",
  "client_id": "a12464ae-b484-4b90-4dfe-17e20844e9f0",
  "client_secret": "mysupersecretpassword"
}

Sample log.conf.json:

{
  "profiles": {
    "dev01-api-client-example": {
      "hostname": "dev01-mysandbox.demandware.net",
      "client_id": "a12345ae-b678-9b01-2dfe-34e56789e0f1",
      "client_secret": "mysupersecretsecret",
      "polling_interval": 30,
			"refresh_loglist_interval": 900
    },
    "dev02-bm-example": {
      "hostname": "dev02-mysandbox.demandware.net",
      "username": "[email protected]",
      "password": "mysupersecretpassword",
      "log_types": [ "customerror", "customwarn", "error", "jobs", "warn" ],
      "polling_interval": 60
    }
  },
  "interactive": false,
  "fluent": {
    "enabled": true
  }
}

If multiple instances are configured, you may directly pass the name of the instance for skipping the interactive selection prompt, e.g.:

$ cctail dev02

API client configuration

The API client id must be created in the account.demandware.com console. Before being able to use cctail you must grant the required permissions to that client id for accessing the logs folder through WebDAV in any target SFCC instance.

To do so, access Business Manager and add the following to Administration -> Organization -> WebDAV Client Permissions, replacing the client_id value with your client id. Note: you may need to merge these settings with existing ones.

{
  "clients": [
    {
      "client_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "permissions": [
        {
          "path": "/logs",
          "operations": ["read_write"]
        }
      ]
    }
  ]
}

Usage

$ cctail

Run cctail in a folder containing either a log.conf-json or dw.json config file. The tool will display the list of available logs in order to let you interactively select the ones you want to monitor.

License

Released under the MIT license.

cctail's People

Contributors

fgiust avatar fgiustmw avatar sschwartzman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

cctail's Issues

Support for the security log on SFCC

Hi There,

I wonder if you have considered pulling the security logs (/on/demandware.servlet/webdav/Sites/Securitylogs) at all?
https://documentation.b2c.commercecloud.salesforce.com/DOC4/index.jsp?topic=%2Fcom.demandware.dochelp%2Fcontent%2Fb2c_commerce%2Ftopics%2Fsite_development%2Fb2c_understanding_log_files.html

I think they would be useful to our use case but I don't see any reference to them in the current logging time. Forgive me if i've missed something.

Many thanks.

George

cctail pulling all the logs even after mentioning the specific log types in the config file

We have created 7 different docker containers for pulling different log types but we could see duplicates in the NR as each container is pulling all the other log types as well rather the ones mentioned in the config file
these are the different log types used by each different containers

"log_types": ["customwarn", "warn"]
"log_types": ["customerror", "customfatal"]
log_types: ["error", "fatal"]
"log_types": ["sysevent", "syslog", "system"]
"log_types": ["service", "jobs"]
"log_types": ["custom"]
log_types: ["custominfo", "info"]

The sample config file of cctail is below::

/ # cat ./log.conf.json
{
"profiles": {
"sfcc": {
"hostname": "ENV_SFCC_HOSTNAME",
"client_id": "ENV_SFCC_CLIENT_ID",
"client_secret": "ENV_SFCC_CLIENT_SECRET",
"polling_interval": 30,
"log_types": ["service", "jobs"]
}
},
"interactive": false,
"fluent": {
"enabled": true
}
}

HTTP 401 not causing code to fall into 'error' path

Hi @fgiust ! I'm excited to use cctail, I'm actually looking into having it pipe the logs to an ingestion tool like fluentd, but that's besides the point right now. ;) For right now though, I'm just playing with it to see how it works today. For an SFCC sandbox for which I don't have an API Client create yet, I tried my personal client_id and secret_id. I would expect this to fail since it is expecting an API Client creds. However when I run it this way, it doesn't throw any 401 failures, it just comes back with an empty log list, like this:

seth:cctail $ node dist/cctail.js
cctail - v1.2.0 - (c) openmind

Authenticating using oauth - client_id REDACTED

✔ Select logs on [REDACTED.demandware.net] ›
No log selected, exiting.

I added a couple log messages to logfetcher.ts to see what's going on, here's the result:

seth:cctail $ node dist/cctail.js
cctail - v1.2.0 - (c) openmind

Authenticating using oauth - client_id REDACTED

Response:
{
  error_description: 'Client authentication failed',
  error: 'invalid_client'
}
Status Code: 401

Response: 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
	<title>401 - Authorization Required.</title>
	<link rel="STYLESHEET" type="text/css" href="/waroot/style.css">
</head>
<body>
	<h1>Authorization Required</h1>
	<img src="/waroot/system_arrow.gif" width="21" height="21" alt="" border="0">
	<p class="system_info">
		This request requires HTTP authentication.
	</p>
	<hr>
</body>
</html>
Status Code: 401

✔ Select logs on [REDACTED.demandware.net] ›
No log selected, exiting.

So it seems the Axios requests are getting 401 errors back (which I would expect!), but the catch blocks aren't catching them. I'll keep playing with the code, I suspect it's because it's to do with await or how Axios handles error codes.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.