qcb-collaboratory / telescope Goto Github PK

Description

If parsing a password through the config.ini file, an InterpolationSyntaxError is given by the configparser library.

Method to Repeat

Create a password with a '%' character and run the test through the server, 'http://localhost:4000/experiment?expID=1'.

Work around

Current work around is to tell python that it is the character, '%' by typing '%%'. An alternative solution is the use 'RawConfigParser' instead of configparser.
See this link.

One of the most important features would be keep track of memory usage over time. This can be partially achieved with the -j flag of qstat:

qstat -j <JOB_ID>

The item maxvmem gives you the maximum amount of RAM your job used when it was running. Not sure yet how to calculate the precise instantaneous amount of RAM being used.

cc @juandelahoz

Is it possible to send notifications (e.g., an email) if one machine is down? That is, when a machine was rebooted or turned off.

This is the list of features per job that we would like to see in the database:

• job number, user-defined name, job ID...
• command
• directory of job submission
• current status (queue, running, finished)
• time requested (and time running)
• memory requested (and current memory use)
• max memory used until now
• time of job completion, and reason: (Completed, Killed, Aborted (by whom?))
• the node in which the job is running
• is it running in parallel? (on how many cores?)

For some reason, after paramiko sends a command to the remote server, sometimes it responds with a None instead of a string containing the server's response. I could not find a way to reproduce this behavior deterministically. However, if you set up a small script that uses paramiko and then run it several times in a row, it eventually happens.

Paramiko is only used in the sshKernel module. We could:

• Fix it, if we can guarantee this doesn't happen anymore
• Wrap a try/error around it
• Change how we connect via SSH

Here is a minimal example of a Python script to test this error. This should print the output of uname -a of the remote server. Just try and run it several times in a row.

import paramiko

username = 'USERNAME'
address = 'ADDRESS'

def query(command):
        # Check if connection is made previously
        if (sshClient):
            stdin, stdout, stderr = sshClient.exec_command(command)
            while not stdout.channel.exit_status_ready():
                # Print stdout data
                if stdout.channel.recv_ready():
                    stdin.close()
                    std_out = stdout.readlines()
                    return ''.join(std_out)

                    # the method below, although recommended in the docs,
                    # is returning intermitent Nones with HIGHER frequency
                    # alldata = stdout.channel.recv(1024)
                    # while stdout.channel.recv_ready():
                    #     alldata += stdout.channel.recv(1024)
                    #
                    # # Print as string with utf8 encoding
                    # string = str(alldata).encode("utf-8")
                    # return string

        else:
            return "No connection."


sshClient = paramiko.client.SSHClient()
sshClient.set_missing_host_key_policy( paramiko.client.AutoAddPolicy() )
sshClient.connect( address, username=username, look_for_keys=True )

print( query( 'uname -a' ) )