currently we enforce a minimum length of 6 for both username / password. this is too long for a username, and too short for a password.

We should enable the following:

• Minimum 3 for username
• Minimum 10 for password

There should probably be a max also?

Perhaps the Logon screen can display the rules.. ?

After deploying Stored Programs to SASjs Server, it is helpful to be able to navigate / edit / execute those programs.

As such, we need the following functionality under the /SASjsDrive endpoint:

• Explorer window (folders & files)
• Ability to VIEW files (default action when clicking is to VIEW)
• Ability to EDIT files (button action)
• Ability to EXECUTE files (button action)

Explorer

It should be possible to expand and collapse folders, to see a list of content. It should be clear from the list what the content type is (eg Stored Program or Subfolder etc)

Viewer

The file content should be displayed in a content box, with buttons to EDIT and to EXECUTE

Editor

In edit mode, when edits are made, the EXECUTE button should be greyed out. After clicking SAVE, the EXECUTE button can be re-enabled.

Executor

When clicking EXECUTE, a NEW window (or tab) is opened with the file location in the _program url param. For instance, if opening /Public/subfolder/someprog then the url would be: /SASjsExecutor?_program=/Public/subfolder/someprog.

Unsure whether this belongs to /server or /cli. When running sasjs cbd -t server on the https://github.com/sasjs/angular-seed-app repo, the following error is returned:

 ERROR  Auth error: {                                                                                                                                               23:28:48
  "status": "failure",
  "message": "Provided not supported data format.",

The cause of the error is not shown.

The message should also be updated, eg: "The format of the build-pack JSON is invalid", ideally it would show WHY (or WHERE) it was invalid.

Most likely due to the getFile API update

To avoid the need for having a SAS exe, we can emulate one ourselves

This can be placed in the sasPath, and all it needs to do is read in the SAS Program and write it to the LOG location.

To emulate the "hot session" behaviour in #18, this program will also need to remove the initial SYSIN program, then wait for it to re-appear

Definition of done:

• an executable JS file that:
  • removes the code.sas file (location in the SYSIN command variable)
  • loops until it reappears
  • Copies the file to log.log (location in the LOG command variable)
  • exit with return code
• a test that calls "SAS" and confirms that the result contains the input string

To reproduce, deploy an app with CORS=enable and WHITELIST=

Expected - the app would still work when the domain is the same

Nice to have - if WHITELIST is empty then ALL domains are accessible (it would be fine to throw an ERROR or WARNING in the console for this)

Instead of having multiple variables for Session e.g. ready, inUse, consumed, completed.

Define state and it's types.

Hi Team,

Is it possible to set up a key-shortcut for running selected code? For example F3? or Ctrl+Enter?

All the best
Bart

@sasjs/server should be able to create internal files accordignally to JSON file created by sasjs build command.

We should update the -CFG file to ensure that SASjs sessions are always UTF-8

This will improve forwards compatibility with Viya as well as addressing the shortcomings of default WLATIN1 (character set incompatibilities)

Currently there is no way to delete a file

Developing apps will require a workflow as follows:

1. Build app locally, designate the source in streamConfig. This folder is deployed to $appLoc/services/$streamConfig.streamWebFolder.
2. Configure the app remotely, as follows:

{
  "AppStreamName1": {
    "appLoc": "/some/other/location/services/web",
    "mainLogo": "/path/to/logo/in/sasjs/drive"
  },
  "AppStreamName2": {
    "appLoc": "$/some/location/services/webv",
    "mainLogo": "/path/to/logo/in/sasjs/drive"
  }
}

The above could be modified using an API also.

The User Experience will then be as follows:

1. Navigate to "serverUrl:/appstream"
2. They are presented with a series of logos
3. Clicking the logo opens the app at /appStream/$appStreamName

something broke in the last update - currently _webin_name has the uploaded path:

But it should be the actual file name:

SASjs Server is composed of a "web" and an "api" component. All web content is public, with requests being made to the API (same as other clients). The API supports a single authentication mechanism - Authorization Code Flow. There are three supported providers for this:

• Self Hosted (Username/Password)
• Viya authorization-flow
• Auth0 authorization-flow

In each case, the client is provided the CLIENT_ID, and the client uses this to fetch an AUTH_CODE. The backend (API) uses this to fetch the ACCESS/REFRESH tokens which are provided to the client in JWT form. This JWT is provided to each API request either in the header, or in a cookie. The client must call the /SASjsApi/auth/refresh endpoint before ACCESS_TOKEN expiry to obtain a refreshed JWT.

This ticket covers the Self Hosted (username/password) flow.

When starting SASjs Server for the first time, a default admin user/password is created. This enables immediate use of the APIs.

The following components are necessary:

• A /SASjsLogon application, allowing a user to simply log in with username and password
• A /SASjsApi/oauth/authorize endpoint, accepting a valid login, plus a CLIENT_ID, and returning an AUTH_CODE (used by frontend)
• A /SASjsApi/oauth/token endpoint, accepting a client/secret/auth code and returning access / refresh tokens (used by backend, and maybe by other apps in future)
• A /SASjsApi/auth/submit endpoint, accepting a CLIENT_ID and AUTH_CODE and returning a JWT (calls the previous endpoint at backend)
• A /SASjsApi/auth/refresh endpoint for refreshing the JWT
• A /SASjsApi/user/changePassword endpoint allowing a password to be changed
• A /SASjsApi/user/add endpoint allowing a new user to be created, with the following attributes: UserId, UserName, Password, isAdmin, isActive. Only admins can create new users. The response object contains a randomly generated password for the new user.
• A lookup table for commonly accessed attributes such as userName, UserId, Refresh Token (a user id could have multiple refresh tokens, one for each client id).
• The ability to authenticate each API request so that only "active" users are allowed

The following is not necessary (for this ticket):

• Auth0 authentication
• Viya authentication
• Forcing the user to change their password on first login

The stpsrv_header() function is used in SAS 9 to alter the http headers sent to the frontend (along with the request body).
Examples from official documentation: https://documentation.sas.com/doc/en/itechcdc/9.4/stpug/srvhead.htm (could be used as test cases)

In sasjs/server we can achieve this by using the fcmp function described here: https://core.sasjs.io/mfs__httpheader_8sas.html

This involves setting a macro variable in the autoexec ( %let sasjs_stpsrv_header_loc=%sysfunc(pathname(work))/../stpsrv_header.txt;) which the location of a text file that is written to the session folder.

If the file exists in the session folder, response headers are set according to the file contents. If the file has duplicates, the last one is used. If the final record has an empty VALUE then the header is removed.

The actual response depends on the way that the request is invoked.

• If GET /SASjsApi/stp/execute then the actual http response headers are set accordingly
• If POST /SASjsApi/stp/execute then the header table is simply inserted into the JSON response (alongside log and webout), eg: {headers:[{header1:value1},{header2:value2}],log:{},webout:{}}

%let sasjs_stpsrv_header_loc=%sysfunc(pathname(work))/../stpsrv_header.txt;

%mfs_httpheader(Content-type,application/csv)
%mfs_httpheader(Expires,%str(Thu, 18 Nov 1999 12:23:34 GMT))

data _null_;
  infile "&sasjs_stpsrv_header_loc";
  input;
  putlog _infile_;
run;

Currently to upload a new app to SASjs server it is necessary to open the swagger docs

It would be nice if there was an upload, or "+" button on the AppStream page itself that did the same thing

Hi Team!

First of all: Great Product! I love it!

Is it possible to set an autoexec.sas file in dedicated directory?

All the best
Bart

potentially related to: https://stackoverflow.com/questions/60474110/why-is-my-console-logging-timeoutoverflowwarning-4294967296000-does-not-fit-i/62003170#62003170

in this code snippet:

After running some code in SASjs Studio, the log should open at the bottom (not the top)

Getting "filePath" is required regardless of presence of filepath

Should have link to /SASjsAPI/
Should have loggedin user's section at top right.
Should have requests modal similar to react seed app.

When running in Studio, it's a pain to provide the relevant CORE macros each time.

Given that we are on the server, and we have access to the filesystem, we could instead just put the macros there. Then make them available to all SAS sessions in a single line of code.

The steps would be:

• Create a "core" subfolder in the DRIVE_PATH
• copy ALL the files from the folders below, into the above folder. They would have to be provided some other way in the final (built) version.
  • node_modules/@sasjs/core/base
  • node_modules/@sasjs/core/ddl
  • node_modules/@sasjs/core/fcmp
  • node_modules/@sasjs/core/lua
  • node_modules/@sasjs/core/server
• add the following line of code to the SAS autoexec:
  • options insert=(SASAUTOS="$(DRIVE_PATH)/core"); -> replacing $DRIVE_PATH with the actual path

when we click open in studio button. It actually opens the studio tab but in app bar does not change the selected indicator from drive to studio.

When uploading files to SASjs server we should pass the filenames into the _WEBIN_FILENAME series of variables.

Currently these do not appear to be populated.

The information is visible from the request header, eg:

In the above scenario we'd expect the following code generated:

%let _webin_filename=DC687632.MPE_X_TEST.xlsx;
%let _webin_filename1=DC687632.MPE_X_TEST.xlsx;
%let _webin_filename2=DC687632.MPE_X_TEST.xlsx.csv;

When files are uploaded to SAS (/SASjsExecutor endpoint) we need to make them available to the SAS session. Ideally they will be loaded into SASWORK, but until we find a nice way to handle a running SAS Session, we should proceed as follows:

1. Create a temporary directory in which to store the files
2. Insert some SAS code to enable the session to reference those files
3. After execution, delete the directory.

The SAS code to be inserted consists of the following variables:

• _WEBIN_FILE_COUNT - contains an integer, from 0 to the number of files to being provided. This variable is always created.
• _WEBIN_FILENAME1 - The full path to the first file (including the filename). The path to the second file will be _WEBIN_FILENAME2 (and so on, same for the below)
• _WEBIN_FILEREF1 - An 8 letter code, starting with an underscore and finishing with 7 random alphanumeric characters
• _WEBIN_NAME1 - the value that was specified in the NAME attribute by the frontend

We will also create the fileref(s) using the filename statement.

To illustrate with an example - we are uploading two files, F1 and F2. The following SAS code will be generated, and inserted at the beginning of the executed program:

filename _SJS0001 "/some/temp/location/file1";
filename _SJS0002 "/some/temp/location/file2";
%let _WEBIN_FILE_COUNT=2;
%let _WEBIN_FILENAME1=/some/temp/location/file1;
%let _WEBIN_FILENAME2=/some/temp/location/file2;
%let _WEBIN_FILEREF1=_SJS0001;
%let _WEBIN_FILEREF2=_SJS0002;
%let _WEBIN_NAME1=F1;
%let _WEBIN_NAME1=F2;

If there are no files uploaded, only the following code will be generated:

%let _WEBIN_FILE_COUNT=0;

Ideally we would run sasjs cb on a project and then provide the exact same JSON file to the sasjsapi/drive/deploy service - which would then create the files in SASjs Drive as well as configure the app in AppStream.

Currently the following error can be found during parallel runs:

ERROR: Resource is write-locked by another user. File =/home/muhsaa/server/api/code.lst. System Error Code = 11.

An explanation can be found here: https://support.sas.com/documentation/cdl/en/hostunx/61879/HTML/default/viewer.htm#o-p3.htm

The invocation of SAS needs the PRINT system option applied

To fit our migration use case (and the streaming app use case), we need to continue to support the approach whereby the webout result is streamed directly to the browser.

To ease the integration with the adapter (and other API clients) we should also support a JSON response.

This ticket proposes that GET requests should result in a streamed output, and POST requests result in JSON output. This is now documented in https://server.sasjs.io/api

Currently, the LOG is returned as a single encoded string.

This can cause memory problems when we have very large logs. It is also problematic to parse within certain instances of SAS.

Instead, we should send the log in an array (ie, line by line).

This affects the following endpoints:

There is likely to also be an impact on the adapter / cli, which ingests the log int the request history.

example format:

{
  log:{
    "1":"reallyh long ",
    "2":" log|",
    "3":"sasoasdfasdfasdfasdf"
  }
}

• Readme should be updated to say that CORS is enabled by default on DESKTOP mode.

We should have no security vulnerabilities. If there is a way we can test for this as part of a merge request / github action, we should do so:

To reproduce - build DC in development mode and run sasjs cbd -t server

Currently certificates are only supported in 'dev mode' (by cloning the repo)

it should be possible for an admin to configure https using the final (built) release zip files

Due to limitations in SAS, we need to avoid the use of very wide strings.

Currently in POST /SASjsApi/drive/file we send a payload in the following format:

{
  "filePath": "/Public/somefolder/some.file",
  "fileContent": "Contents of the File"
}

Instead, fileContent should be sent as a raw file in the request body. This will also enable forward compatibility when we upload other file types (such as images). The filePath can be a URL parameter.

It would be super convenient for SAS programmers (as consumers of the sasjs/server apis) to be able to use the JSON libname engine for reading the webout response.

Currently that is not possible for large webouts, due to a 32767 character string length limitation.

So, when the HTTP header content-type: application/json is set AND the request is POST (not GET) SASjs/stp/execute, AND the webout contains valid JSON, the webout response should be passed back as an object rather than stringified text.

The description of the swagger API should also be updated to reflect the above logic.

It is common for SAS Developers to be denied access to the SAS Web Server.

To enable web development regardless, SASjs enables 'streaming apps' on both SAS 9 and Viya. This involves loading web content to the logical SAS folder (eg metadata or SAS drive) and adjusting the source files to use the modified paths.

We ought to support the same approach for SASjs server, enabling both frontend and backend to be deployed using sasjs deploy, where streamConfig.streamWeb==true.

The path inserted into SASAUTOS is not valid in the released version of the app (it works only in development mode)

To validate, run the following code in Studio on a published version:

%put %sysfunc(getoption(SASAUTOS));

And check whether that folder exists / has the core macros.

In order to execute SAS 9 code on SASjs Server we will need to enable some additional functions. This will require the addition of some system precode when executing each service.

Currently we already insert some dynamic lines of code before each service, eg:

filename _webout "C:\Users\YuryShkoda\projects\server\tmp\webouts\testJob-20210819080309.json";
%let var2=val2;
%let varname=value;
* Job Variables start;
...

The above contains:

• execution specific dynamic variable (from sasjs/server), ie temporary webout location
• invocation specific variables (from the user), ie macro variables
• actual job code (from the filesystem / database), ie the static SAS code

We now need to insert two more things:

• An additional entry in the execution-specific part, ie the temporary header file location
• An additional section between the above, and the invocation specific variables (a 'system init' program)

The systemInit program will be stored in the repo as follows:

/**
  @file
  @brief The systemInit program
  @details This program is inserted into every sasjs/server program invocation,
  _before_ any user-provided content.

  <h4> SAS Macros </h4>
  @li mcf_stpsrv_header.sas

**/


proc fcmp outcat=work.sasjs.utils;
%mcf_stpsrv_header()
quit;

During npm run build of sasjs/server this systemInit program will be compiled, and the compiled version will be inserted whenever deploying jobs/services/tests/etc.

So, a job that looks like this in the database/filesystem:

* Job Variables start;
*Job Variables end;
* Dependencies start;
* Dependencies end;
* Programs start;
*Programs end;
* Job start;
data _null_;
  file _webout;
  put 'test Jobhere is some end user SAS code';
run;
* Job end;

Will look like this when it is executed in SAS:

/* runtime vars */
%let sasjs_stpsrv_header_loc = C:\Users\YuryShkoda\projects\server\tmp\webouts\header-20210819080309.txt;
filename _webout "C:\Users\YuryShkoda\projects\server\tmp\webouts\testJob-20210819080309.json";

/* compiled systemInit */
SYSTEM INIT COMPILED PROGRAM

/* dynamic user-provided vars */
%let var2=val2;
%let varname=value;

/* actual job code */
* Job Variables start;
*Job Variables end;
* Dependencies start;
* Dependencies end;
* Programs start;
*Programs end;
* Job start;
data _null_;
  file _webout;
  put 'test Jobhere is some end user SAS code';
run;
* Job end;

For development in non-SAS environments (eg running tests in github pipelines) it will be very helpful to have mocked versions of the Viya endpoints as they pertain to SASjs.

This will enable a more robust (and faster) test suite for the CLI and adapter, as well as various web apps. It will also mean that the tests can still be easily run against an actual viya instance on an ad-hoc basis.

The following entry will be enabled in the .env file:

# optional - enable mocked server endpoints for testing
# options: [disable|enable] default: `disable`
MOCK=

When enabled, various Viya / SAS 9 endpoints will be made available on SASjs server. The endpoints will provide a mix of actual and hard-coded responses, and varying degrees of actual / mocked functionality, in order to fit the needs of the test cases we have so far.

Definition of done:

• The sasjs/adapter and sasjs/cli test suites can be executed using the "SASVIYA" serverType and the sasjs/server serverUrl.
• The sasjs/adapter and sasjs/cli test suites can be executed using the "SAS9" serverType And sasjs/server serverUrl

and gives following result
{"status":"failure","message":"Job execution failed.","error":"TypeError [ERR_INVALID_ARG_TYPE]: The \"file\" argument must be of type string. Received undefined"}

Currently it is not possible to navigate to SASjs Studio on a mobile (unless the mobile is tipped sideways)

It should be possible to navigate the menus on small screens

Unless the _debug parameter is 131 or greater

We should probably stop calling it "tmp" (as it contains permanent Drive files)

see:

It is a common requirement to alter the startup options for a SAS session. There are many ways to achieve this as illustrated by teh diagram below:

In our case we should use the SAS_OPTIONS environment variable. This will override all other settings (except for the invocation arguments).

Excerpt from docs:

SAS_OPTIONS operating system environment variable
This environment variable, if defined, contains a string of option specifications for any other SAS system options that you want to process each time you invoke SAS. For example, this environment variable might contain -obs 2m -sasinitialfolder c:\myfolder -linesize max.

Implementation

Enable an optional SAS_OPTIONS variable in the .env file

When hovering over webout it would be nice to have a tooltip with the following text:

"Displays content from the _webout fileref"

The NOSPLASH argument is not valid on linux - we should remove it

For analytium, the binary is here: /opt/sas/sas9/SASHome/SASFoundation/9.4/sas

For maximum responsiveness we should pre-spawn SAS instances so that they can execute immediately when the file arrives.

We can do this by implementing the following code in the autoexec (which will wait 15 minutes for the file to appear before continuing):

data _null_;
  /* remove the dummy SYSIN */
  length fname $8;
  rc=filename(fname,getoption('SYSIN') );
  if rc = 0 and fexist(fname) then rc=fdelete(fname);
  rc=filename(fname);
  /* now wait for the real SYSIN */
  slept=0;
  do until ( fileexist(getoption('SYSIN')) or slept>(60*15) );
    slept=slept+sleep(0.1,1);
  end;
run;

SAS can then be invoked as follows:

# make dedicated folder
mkdir /tmp/mydir

# create autoexec
cat > /tmp/mydir/autoexec.sas <<'EOL'
data _null_;
  /* remove the dummy SYSIN */
  length fname $8;
  rc=filename(fname,getoption('SYSIN') );
  if rc = 0 and fexist(fname) then rc=fdelete(fname);
  rc=filename(fname);
  /* now wait for the real SYSIN */
  slept=0;
  do until ( fileexist(getoption('SYSIN')) or slept>(60*15) );
    slept=slept+sleep(0.1,1);
  end;
run;
EOL

# create dummy SYSIN
touch /tmp/mydir/code.sas

# invoke SAS
/opt/sas/sas9/SASHome/SASFoundation/9.4/sasexe/sas -LOG /tmp/mydir/demo.log  -SYSIN /tmp/mydir/code.sas -AUTOEXEC /tmp/mydir/autoexec.sas -WORK /tmp/mydir

In another shell, execute the following:

# create sasfile
cat > /tmp/mydir/code.sas <<'EOL'
%put start NOW;
EOL

And the first shell should finish & execute.