Hi,
I have trouble when i try to execute integration test on minikube cluster on mac os.
It failed during the test:

kubectl exec peer1-hlf-peer-66bdfc4657-wmmfs -n peers -- peer channel list

The error from the peer:

[grpc] HandleSubConnStateChange -> DEBU 039 pickfirstBalancer: HandleSubConnStateChange: 0xc42001a390, CONNECTING
Error: error getting endorser client for channel: endorser client failed to connect to peer1-hlf-peer:7051: failed to create new connection: context deadline exceeded
command terminated with exit code 1

The connection failed because there is an incorrect value set in CORE_PEER_ADDRESS:

root@peer1-hlf-peer-66bdfc4657-wmmfs:/# env | grep CORE_PEER_ADD
CORE_PEER_ADDRESSAUTODETECT=true
CORE_PEER_ADDRESS=peer1-hlf-peer:7051

My test environment :
Mac OS 10.13.6
Minikube

⇒  kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:28:14Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
⇒  helm version
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}

Can you help me ? How can i remove this variable it's not necessary because CORE_PEER_ADDRESSAUTODETECT=true ?

Depends on #16

Depends on #17

An helper function is required to obtain orderers from the nephos_config.yaml, such that we replace:

opts["orderers"]

with something like this:

get_orderers(opts, msp_name)

Depends on #18

Depends on #15 and #24

This example will demonstrate completion of the milestone where we are able to do multi organisation deployment using Let's Encrypt certificate for TLS across different K8S clusters

Is it possible to provide an example of how I can install chaincode in the defined network?

Create TLS secrets from generated crypto material under crypto (and naturally also fabric and deploy) command with deploy.py

Once completed add example in #21

Hey,

Thank you for this great project, really easier to install the stack compared to the workshop, (which I failed in early steps).

I've still struggled a little with pre-requisites (and still have some unit tests failed on Ubuntu for Windows), and wonder how I will automate the process for CI/CD.

Do you have any docker image somewhere, containing all needed tools to run Nephos ? Or would you consider building one ?

Thanks 😃

This will make dependabot flow faster.

Current message:

2019-06-25 10:03:02,686 misc       INFO     Input None
<your string path to bna file/n>
2019-06-25 10:03:15,472 misc       WARNING  Replaced <default_path_to.bna> with <you_string_to.bna>

When executing composer, we get a logging info meessage saying "Input None" and it waits for input. You paste the path to the bna file and then it states that has replaced a default value with another value as a warning.

I think this is a good case where we can use printed to stdout messages and not the logger function, we don't need the input requested in the logs.

Example output could be:

2019-06-25 10:03:02,686 misc       INFO     Starting composer
2019-06-25 10:03:02,686 misc       INFO     Default bna file <path_to_default_bna>
### Please input full path to your BNA file (default: /root/[email protected]) ###
### insert path: _ <- You wait for input here


2019-06-25 10:03:15,472 misc       INFO  Processing bna file -> /defined_path/file.bna

This will require to create functions to generate TLS crypto material from the CA and save them as secrets on K8S

Once completed add example in #30

Depends on #14

Ideally we need to support multiple channels. Currently our nephos_config.yaml has only support for 1 channel, to which all peers are joined.

Instead, we should add a section in the config for channels, and reference these in the peer section by name.

We can start by separating the channel into its own section, even if we don't immediately support multiple channels, and then add the extra support, as this will require changes in several places, including:

• the configuration file itself
• the Helm Charts for the Peers
• the Nephos code that deals with creating channels
• the Nephos code that deals with joining peers to channels

Occasionally commands will fail with a Kubernetes/Helm timeout, we should have a robust mechanism to avoid breaking execution on a first fail (e.g. some kind of exponential backoff, or use execute_until_success with a limited number of tries.)

After deploying a network and leaving it for some time without activity, the orderer log that check it is functioning stalls.

We should be able to use the native python logging module to provide useful logging of information, warnings and errors.

This issue depends on #12 and #13 being completed

Move orderers and peers inside Organisation MSPs

msps:
  OrdererMSP:
    namespace: orderers
    org_admin: ordadmin
    # org_passwd:  # Set implicitly
    orderers:
        ...
  
  PeerMSP:
    namespace: peers
    org_admin: peeradmin
    # org_passwd:  # Set implicitly
    peers:
         ...

Nephos is currently using stable/postgresql chart as backend of few services, running deploy with the --upgrade flag, ignores postgresql version.

Should it be added to the workflow? Otherwise if the user is not aware, may keep old versions of postgreSQL across time.

The instruction for the manual upgrade are in the helm chart repo, it will vary depending on the chart version -> https://github.com/helm/charts/tree/master/stable/postgresql

Depends on #16

This update will require several moving pieces:

• Update the helm charts to Fabric 2.0
• Enable using RAFT as the ordering service
• Ensuring Nephos deployment and configuration is compatible with this upgrade.

In Fabric 1.4 we may need to add the "hf.Registrar.Roles" attribute to allow correct registration/enrolling.

The current CA aliveness check may fail if CA is alive for long, so we should enable a more robust one.

I am learning Hyperledger Fabric and pretty new to Kubernetes (which is the problem). I found about Nephos and started to try it, I must say Nephos abstracts away a lot of complex parts. I followed steps in documentation to run test example in my minikube and it works well.
Now, I need to deploy .bna file to the cluster developed using Composer. Then, I also find the presence of 'hl-composer' chart which should be able to install chaincodes using Composer on the K8s cluster but I couldn't find any example on how can I do so. I got stuck now. So, It will nice, for the beginner like me, to have a few more instructions in documentation to be able to install chaincode using hl-composer and run rest-server, and playground in minikube.

Thanks in advance for any possible directions!

Depends on #14

This will enable multi org deployment where separate organisations granting custody of their resources to a central administrator.

An helper function is required to obtain peers from the nephos_config.yaml, such that we replace:

opts["peers"]

with something like this:

get_peers(opts, msp_name)

Depends on #20

Do you plan to upgrade this for Hyperledger Fabric 2.0?

Depends on #16

Once completed add example in #23

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Investigate the library PyHelm to replace the "helm" commands launched from Python using subprocess.

• Could be cleaner to handle credentials to access helm in the cluster.
• Maybe we can reduce the complexity of unit-testing using this library.
• Adding the third party library we may reduce the maintenance load, make the code more clear to handle using only python code and not having to assemble cli commands, and what is expected from the api call could be more reliable.
• Reduce the error handling special characters and how we are escaping them.

Module documentation: https://pypi.org/project/pyhelm/

Example of how we execute:

• We create a command string:

command = "helm install {repo}/{app} -n {name} --namespace {ns}".format(

• Execute method:

  if show_command:
       print(t.magenta(command))
   try:
       # TODO: Can we do this with a different command than check_output (Bandit security issue)
       result = check_output(command, stderr=STDOUT, shell=True)
       decoded = result.decode("utf-8")
       if verbose:
           print(decoded)
       return decoded, None
   except CalledProcessError as e:
       error_text = e.output.decode("utf-8")
       if show_errors:
           print(t.red("Command failed with CalledProcessError:"))
           print(error_text)
return None, error_text

This could help to fix #26

Composer install and setup scripts are calling the function get_namespace with a direct access to a dictionary subkey that may not be present in older versions of nephos deployments.

This makes the upgrade of older deployments to fail when trying to get the namespace passing the "msp" parameter to get_namespace function.

Traceback (most recent call last):
  File "./donate/setup_da.py", line 215, in <module>
    main()
  File "/tools/deployer/venv/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/tools/deployer/venv/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/tools/deployer/venv/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tools/deployer/venv/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "./donate/setup_da.py", line 208, in main
    install_api(env, opts, upgrade=upgrade, verbose=verbose)
  File "./donate/setup_da.py", line 108, in install_api
    env_vars = get_api_vars(env, opts, verbose=verbose)
  File "./donate/setup_da.py", line 32, in get_api_vars
    composer_data = get_composer_data(opts, verbose=verbose)
  File "/tools/deployer/venv/lib/python3.7/site-packages/nephos/composer/install.py", line 41, in get_composer_data
    peer_namespace = get_namespace(opts, opts["peers"]["msp"])
KeyError: 'msp'

The problem is not in the get_namespace function, this will allow None for the msp parameter.

https://github.com/hyperledger-labs/nephos/blob/master/nephos/fabric/settings.py#L51
It's gracefully handled in get_namespace if the value is 'None' if it's None it falls back to get the namespace defined in the core config of the nephos_config.yaml file.

def get_namespace(opts, msp=None, ca=None):
    """Get relevant namespace where MSP or CA should be located.
    Args:
        opts (dict): Nephos options dict.
        msp (str): Name of Membership Service Provider (MSP).
        ca (str): Name of Certificate Authority (CA).
    Returns:
        str: Namespace relating to either an MSP or a CA.
    """

It fails when is calling with the following code:
https://github.com/hyperledger-labs/nephos/blob/master/nephos/composer/install.py#L47
https://github.com/hyperledger-labs/nephos/blob/master/nephos/composer/upgrade.py#L34

    peer_namespace = get_namespace(opts, opts["peers"]["msp"])

If the key/value doesn't exist in the dictionary. The dictionary is populated with the nephos_config.yaml file.

In the previous version of Nephos I could work it around using a simple workaround:

diff --git a/nephos/composer/install.py b/nephos/composer/install.py
index 40ddd52..e6dde03 100644
--- a/nephos/composer/install.py
+++ b/nephos/composer/install.py
@@ -44,7 +44,12 @@ def get_composer_data(opts, verbose=False):
     Returns:
         dict: Data related to the Composer deployment (URI & API key)
     """
-    peer_namespace = get_namespace(opts, opts["peers"]["msp"])
+    try:
+        msp = opts["peers"]["msp"]
+    except KeyError:
+        msp = None
+
+    peer_namespace = get_namespace(opts, msp)
     composer_name = opts["composer"]["name"] + "-hl-composer-rest"
     data = get_app_info(
         peer_namespace,

For new versions with multi-org, we should check what's the best wat to go ahead.

Hi everyone, I tried to deploy the examples/qa in a real cluster and I struggled with some changes of Kubernetes v1.16.2.

I encountered the following issues:

The following command helm install stable/cert-manager -n cert-manager --namespace cert-manager does no longer work since [email protected] requires some additional repos to be added in helm.

Also theirapiVersion has changed from: certmanager.k8s.io/v1alpha1 to cert-manager.io/v1alpha2

Their acme resolvers now require additional solvers fields etc..

I encountered these issues and I managed to fix them, I would be happy to create a PR regarding this if you agree.

root@ubuntu:~# pip install nephos
Collecting nephos
  Could not find a version that satisfies the requirement nephos (from versions: )
No matching distribution found for nephos

I see it on PyPi, but for whatever reason cannot pull it on any of my computers.

In the Installation section of the README, there is a reference to the aidtechnology organization. This should be changed to the hyperledger-labs organization.

git clone https://github.com/aidtechnology/nephos.git

should be

git clone https://github.com/hyperledger-labs/nephos.git

Commonly, we see Integration Test failures that result in us seeing the following errors, costing considerable time since we need to restart the build:

As described here (https://realpython.com/python-f-strings/), we should migrate to f-strings, which are more elegant and compact than the format() function on strings.

This is a very easy issue to implement, and good for newcomers who want to contribute something useful to the repository.

Remove the certs that can be generated dynamically from /examples.

Add documentation to be followed during example to generate them.

This is the main reason why most of the integration test fails and have to be repeated. When CA server is not yet ready it sends a 503 error for the command fabric-ca-client identity list but still the function check_id() returns true.

I am trying to expose order, peer and ca so I can interact with the applications from outside world by enabling for them an ingress controller. Doing so, at the moment without a tls since I cannot find any documentation on how to properly enable it except from the helm charts configuration:

  ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: nginx
    # TODO: When working locally, we may wish to comment the cluster-issuer
    #certmanager.k8s.io/cluster-issuer: "letsencrypt-production"
  path: /
  hosts:
    # TODO: Change this to your Domain Name if not working locally
    - beta.nephos.local

I am doing also the same for order too.

The deploy fails to create the channel with the following error:

2020-01-19 11:59:34,968 misc       INFO     kubectl exec beta-peer1-hlf-peer-7f647985fc-tpbz7 -n beta -- peer channel list
2020-01-19 11:59:36,035 misc       ERROR    Command failed with CalledProcessError:
2020-01-19 11:59:36,036 misc       ERROR    2020-01-19 10:59:35.162 UTC [main] InitCmd -> WARN 001 CORE_LOGGING_LEVEL is no longer supported, please use the FABRIC_LOGGING_SPEC environment variable
2020-01-19 10:59:35.165 UTC [main] SetOrdererEnv -> WARN 002 CORE_LOGGING_LEVEL is no longer supported, please use the FABRIC_LOGGING_SPEC environment variable
Error: error getting endorser client for channel: endorser client failed to connect to beta.nephos.local:443: failed to create new connection: connection error: desc = "transport: error while dialing: dial tcp: lookup beta.nephos.local on 10.96.0.10:53: no such host"
command terminated with exit code 1

I cannot figure it out why it keep looking for the host beta.nephos.local and not to ping the internal kubernetes ip

We should modify the message log level for different messages. I'll add an example bellow, but we should verify all the log messages.

i.e.:
The example is from the setup_ca method -> https://github.com/hyperledger-labs/nephos/blob/master/nephos/fabric/ca.py#L163

2019-06-25 09:22:59,598 misc       INFO     helm status rca-pg
2019-06-25 09:23:00,655 misc       ERROR    Command failed with CalledProcessError:
2019-06-25 09:23:00,656 misc       ERROR    Error: getting deployed release "rca-pg": release: "rca-pg" not found

2019-06-25 09:23:00,656 misc       INFO     helm install stable/postgresql -n rca-pg --namespace cas --version 3.18.3 -f /path_to/helm_values/postgres-ca/rca-pg.yaml
2019-06-25 09:23:04,962 k8s        INFO     Ensuring that all pods are running
2019-06-25 09:23:04,963 misc       INFO     kubectl get pods -n cas -l "app=postgresql,release=rca-pg" -o jsonpath="{.items[*].status.phase}"
.2019-06-25 09:23:15,357 misc       INFO     kubectl get pods -n cas -l "app=postgresql,release=rca-pg" -o jsonpath="{.items[*].status.phase}"
.2019-06-25 09:23:25,603 misc       INFO     kubectl get pods -n cas -l "app=postgresql,release=rca-pg" -o jsonpath="{.items[*].status.phase}"

So, for example in this case, going line by line

• INFO helm status rca-pg
  This message could be divided into two, info "Checking for existing helm app <>" and a debug message with the command.

• ERROR Command failed with CalledProcessError:

• ERROR Error: getting deployed release "rca-pg": release: "rca-pg" not found
  These message shouldn't be error, but debug, I would even move into the future to a "try" structure, with python we shouldn't ask for before.

• INFO helm install stable/postgresql -n rca-pg --namespace cas --version 3.18.3 -f /tools/deployer/test_deployments/at-e2e/qa/helm_values/postgres-ca/rca-pg.yaml
  This is a debug message as well, but should be replaced with the command in debug and another INFO saying something like "Installing helm posgresql chart"

• INFO Ensuring that all pods are running
  It's good as it is.

• kubectl get pods -n cas -l "app=postgresql,release=rca-pg" -o jsonpath="{.items[*].status.phase}"
  Also to be divided into two messages, an info and a debug.

Details

Error while installing Hyperledger Composer Business Network:

Error: Error trying to ping. Error: make sure the chaincode hlfv1 has been successfully instantiated and try again: chaincode hlfv1 not found Command failed

How to fix

Remove the network when setting up the Network Admin card.

The command to create the Network Admin Card (PeerAdmin@hlfv1) starts with:

composer network install -n hlfv1 -p ...

It should actually be

composer network install -p ...

Once completed add example in #24