wso2 / puppet-apim Goto Github PK

View Code? Open in Web Editor NEW

41.0 152.0 42.0 6.27 MB

Puppet 30.92% Shell 51.25% HTML 17.83%

puppet-apim's Introduction

Puppet Modules for WSO2 API Manager

This repository contains the Puppet modules for profiles related to WSO2 API Manager and API Manager Analytics.

Supported Puppet Versions

Puppet 7.31.0

Quick Start Guide

Download a product package. Product packages can be downloaded and copied to a local directory, or downloaded from a remote location.
- Local: Download wso2am-4.3.0.zip from here and copy it to the <puppet_environment>/modules/apim_common/files/packs directory in the Puppetmaster.
- Remote:
  1. Change the value $pack_location variable in <puppet_environment>/modules/apim_common/manifests/params.pp to remote.
  2. Change the value $remote_pack variable of the relevant profile in <puppet_environment>/modules/apim_common/manifests/params.pp to the URL in which the package should be downloaded from, and remove it as a comment.
Set up the JDK distribution as follows:

The Puppet modules for WSO2 products use Amazon Corretto as the JDK distribution. However, you can use any supported JDK distribution. The JDK Distribution can be downloaded and copied to a local directory, or downloaded from a remote location.
- local: Download Amazon Corretto for Linux x64 from here and copy .tar into the <puppet_environment>/modules/apim_common/files/jdk directory.
- remote: Change the value $remote_jdk variable in <puppet_environment>/modules/apim_common/manifests/params.pp to the URL in which the JDK should be downloaded from, and remove it as a comment.
- To use a different jdk distribution, reassign the $jdk_name and the $java_home variables in <puppet_environment>/modules/apim_common/manifests/params.pp accordingly.

Run the relevant profile on the Puppet agent.

Default profile:

export FACTER_profile=apim
puppet agent -vt

Gateway profile:

export FACTER_profile=apim_gateway
puppet agent -vt

Control Plane profile:

export FACTER_profile=apim_control_plane
puppet agent -vt

Traffic Manager profile:

export FACTER_profile=apim_tm
puppet agent -vt

Performance Tuning

System configurations can be changed through Puppet to optimize OS level performance. Performance tuning can be enabled by changing $enable_performance_tuning in <puppet_environment>/modules/apim_common/manifests/params.pp to true.

System files that will be updated when performance tuning is enabled are available in <puppet_environment>/modules/apim_common/files/system. Update the configuration values according to the requirements of your deployment.

Manifests in a module

The run stages for Puppet are described in <puppet_environment>/manifests/site.pp, and they are of the order Main -> Custom.

Each Puppet module contains the following .pp files.

Main
- params.pp: Contains all the parameters necessary for the main configuration and template.
- init.pp: Contains the main script of the module.
Custom
- custom.pp: Used to add custom configurations to the Puppet module.

puppet-apim's People

Contributors

Stargazers

Watchers

Forkers

thanu pubudu538 niranjan-k rmsamitha chris2fire chamilaadhi anuruddhal chankami imesh chamilad chrisarnal asanthi vidurananayakkara nu1silva isuruw chamndesilva test-coverage-enforce-bot mushthaq33 kamidu irunika chirangaalwis jsdjayanga vinujan59 gdrdabarera savidude ibaqu shafanas codeturbine kaumini ridmir nishikadesilva vimukthiperera ctienshi rasanjaya85 rakhitharr heshansudarshana kugelblitzz isurumaduranga pasant9 jparedesr91 saipriya435github dinithherath

puppet-apim's Issues

TimestampSkew default vaue need to set 0 in identity.xml

Description:
When we generate access token first time it created with expiry time 3600 seconds as expected but when we invoke same command after few seconds expire time got increased to 3890 seconds which is not expected behavior.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

[C4] Update Puppet modules for new Deployment Patterns

Description:
Need to update Puppet Modules for the latest deployment patterns [1].

[1]. https://docs.wso2.com/display/AM210/Deployment+Patterns#DeploymentPatterns-Olderdeploymentpatterns

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Packages are reinstalled when running Puppet Agent again

Description:
There is an issue of the package getting reinstalled when running puppet agent multiple times

Affected Product Version:
Puppet 5 Manifests for API Manager 2.6.0

Steps to reproduce:
Run puppet agent multiple times

my-node-id.bat file is missing in puppet

analytics server need to be configured with my-node-id.bal file. need to make id unique for each deployment. otherwise it will be generated automatically when clean deployment is executed.
then there will be some issue sin analytics.

Add HA clustering specific puppet parameterization into analytics-module

Currently, analytics clustering related puppet module parameterization is not included into the wso2am_analytics module. It would be better to add this.

Use of apply_publisher_specific_configurations

Looks like the following variables are used to selectively apply certain values in the APIM templates, specifically api-manager.xml and jndi.properties templates.

wso2::apply_publisher_specific_configurations: false
wso2::apply_store_specific_configurations: false
wso2::apply_gateway_specific_configurations: false

One such use in the api-manager.xml is as follows. There, the provided values for GatewayEndpoint is applied only if apply_publisher_specific_configurations is set to true.

    <APIGateway>
	    <Environments>
                 <%- @environments.each do |environment_name, environment| -%>
                    <Environment type="hybrid" api-console="true">
                        <%- if (@apply_publisher_specific_configurations or @apply_store_specific_configurations) and environment['name'] -%>
                          <Name><%= environment['name'] %></Name>
                        <%- else -%>
                          <Name>Production and Sandbox</Name>
                        <%- end -%>
                        <%- if (@apply_publisher_specific_configurations or @apply_store_specific_configurations) and environment['description'] -%>
                          <Description><%= environment['description'] %></Description>
                        <%- else -%>
                          <Description>This is a hybrid gateway that handles both production and sandbox token traffic.</Description>
                        <%- end -%>

                        <%- if (@apply_publisher_specific_configurations or @apply_store_specific_configurations) and
                        environment['host'] and environment['port']-%>
                          <ServerURL>https://<%= environment['host'] %>:<%= environment['port'] %>/services/</ServerURL>
                        <%- else -%>
                          <ServerURL>https://${carbon.local.ip}:${mgt.transport.https.port}${carbon.context}services/</ServerURL>
                        <%- end -%>

                        <%- if (@apply_publisher_specific_configurations or @apply_store_specific_configurations) and
                        environment['username'] -%>
                          <Username><%= environment['username'] %></Username>
                        <%- else -%>
                          <Username>${admin.username}</Username>
                        <%- end -%>

                        <%- if (@apply_publisher_specific_configurations or @apply_store_specific_configurations) and
                        environment['password'] -%>
                          <Password><%= environment['password'] %></Password>
                        <%- else -%>
                           <Password>${admin.password}</Password>
                        <%- end -%>

                        <%- if (@apply_publisher_specific_configurations or @apply_store_specific_configurations) and environment['api_endpoint_host'] and
                          environment['api_endpoint_port'] and  environment['secure_api_endpoint_port']-%>
                          <GatewayEndpoint>http://<%= environment['api_endpoint_host'] %>:<%= environment['api_endpoint_port'] %>,https://<%= environment['api_endpoint_host'] %>:<%= environment['secure_api_endpoint_port'] %></GatewayEndpoint>
                        <%- else -%>
                          <GatewayEndpoint>http://${carbon.local.ip}:${http.nio.port},https://${carbon.local.ip}:${https.nio.port}</GatewayEndpoint>
                        <%- end -%>
                    </Environment>
                    <%- end -%>
        </Environments>
    </APIGateway>

However, this logic is not always true. The value for GatewayEndpoint should be changed to a publicly resolvable name or IP irrespective of the deployment pattern. The same goes for other values that are used to generate URLs in the frontend.

The logic should be that, if a value for GatewayEndpoint is available, apply it, else keep the default in the template.

WSO2 performance tunings are not addressed in puppet scripts

Description:
WSO2 performance tuning[1] should be addressed by the puppet scripts. These configuration are production recommended tunings so puppet modules should incorporate those changes to make them production ready.

[1] https://docs.wso2.com/display/AM2xx/Tuning+Performance

Affected Product Version:
apim2.1.0

Analytics DAS receivers should accept multiple server names and ports

The following Hieradata only accepts one Thrift Receiver whereas multiple receivers are possible to be added [1].

The entries server_host and server_port should probably change to a list of objects that accepts multiple receivers.

wso2::analytics:
  receivers:
    -
      host: host1
      port: port1
    -
      host: host2
      port: port2

[1] - https://github.com/wso2/puppet-apim/blob/master/wso2am_runtime/hieradata/dev/wso2/wso2am_runtime/pattern-2/default.yaml#L247-L254

Specify common configurations which are specific to remote database configuration

Description:
Document: Using Puppet Modules to Set up WSO2 API-M with Pattern 6
Step 5 - Update the clustering related configurations

Specify some common configurations which are specific to remote database configuration such as below.

How to grant permission to remote hosts to get connected in the database server?
How to remove the restrictions which are applied to localhost only.
Provide solutions to common errors such as Communications link failures.
Provide information on which configuration should be exactly done in puppet master in order to connect to the remote database server.

Suggested Labels:
AM2xx Docs
Priority/High
Type/Improvement

Affected Product Version:
Ubuntu 14.04

OS, DB, other environment details and versions:
DB: MySQL

Allow port offset to be change in the Puppet modules

Current Puppet modules are tested against default port offset on all the profiles. Once the port offset is changes the communication among components not working as expected. So need to test the modules against different port offset for different profiles.

SkipDBSchemaCreation should be set to true on Traffic-Manager profile.

Description:
According to [1] apim profiles should be started on that specific order. When we start TM in second place, an exception[2] is occured. To prevent this below configuration[3] should be enable in identity.xml on traffic-manager puppet modules.

[1] https://docs.wso2.com/display/AM2xx/Using+Puppet+Modules+to+Set+up+WSO2+API-M+with+Pattern+6#UsingPuppetModulestoSetupWSO2API-MwithPattern6-Step7-RunthePuppetAgents

[2]

org.wso2.carbon.identity.base.IdentityRuntimeException: java.io.FileNotFoundException: /mnt/wso2am-2.1.0/dbscripts/identity/mysql.sql (No such file or directory)
at org.wso2.carbon.identity.base.IdentityRuntimeException.error(IdentityRuntimeException.java:71)
at org.wso2.carbon.identity.core.persistence.IdentityDBInitializer.executeSQLScript(IdentityDBInitializer.java:273)
at org.wso2.carbon.identity.core.persistence.IdentityDBInitializer.createIdentityDatabase(IdentityDBInitializer.java:141)
at org.wso2.carbon.identity.core.persistence.JDBCPersistenceManager.initializeDatabase(JDBCPersistenceManager.java:112)
at org.wso2.carbon.identity.core.internal.IdentityCoreServiceComponent.activate(IdentityCoreServiceComponent.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:514)
at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

[3]

<SkipDBSchemaCreation>true</SkipDBSchemaCreation>

Affected Product Version:
2.1.0

OS, DB, other environment details and versions:
MySql 5.5.58, Ubuntu 14.04

Steps to reproduce:

Setup api-manager-2.1.0 pattern-6 using puppet
Strat traffic-manager agent.

Pass JDK parameters

After the following PRs are merged to wso2/puppet-common and wso2/puppet-base, wso2base puppet module interface changes to include additional Java related parameters. They should be added to Hieradata and params.pp should properly read the inputs and assign to variables.

wso2/puppet-common#13
wso2/puppet-base#19

Specify where should the mysql-connector jar be placed in puppet master in wso2is_prepacked

Document: Using Puppet Modules to Set up WSO2 API-M with Pattern 6
Step 5 - Update the clustering related configurations
3)b)ii

Specify mysql-connector jar should be placed in following location in puppet master under wso2is_prepacked.
/etc/puppet/environments/production/modules/wso2is_prepacked/files/configs/repository/components/lib in puppet master

Suggested Labels:
AM2xx docs
AM2xxPuppetModules

Affected Product Version:
AM2xx docs

Enable continuous update delivery to API Manager through WUM

Description:
Develop puppet setup to deliver updates to API Manager using in-place client.

Suggested Labels:
Type/Improvement

Suggested Assignees:
@Savidude

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Specify mysql connector version needs to be updated in wso2is_prepacked - default.yaml

Description:
Document: Using Puppet Modules to Set up WSO2 API-M with Pattern 6
Step 5 - Update the clustering related configurations

Specify that mysql connector version needs to be updated in default.yaml which is placed in the following location.
/etc/puppet/hieradata/production/wso2/wso2is_prepacked/pattern-1/default.yaml
Specify that following should be uncommented in the above same file.
"repository/components/lib/%{hiera('wso2::datasources::mysql::connector_jar')}"

Suggested Labels:
AM2xx puppet modules

Affected Product Version:
AM2xx docs

[AMHACK] No error logs or warnings when you are pointing to an incorrect database url in puppet common.yaml heira data

Description:
If you didn't update the correct db url configuration in common.yaml file in hiera data, you won't see any error or warning logs when starting the copied pack.

Suggested Labels:
puppet

Suggested Assignees:

Affected Product Version:
APIM 2.1.0

OS, DB, other environment details and versions:
Mysql 5.5
ubuntu 14.04

Steps to reproduce:
Go to the common.yaml file in hiera data
point to an invalid database url.
run the setup.sh
configuration will copied to the puppet agent.
when starting the pack, there's no error or warnings.
but apim pack won't running. In certain point it will keep as stacked
it's better to have some meaningful error logs for this case.

Related Issues:

Spark errors are thrown with missing tables in analytics instance created from puppet

Description:
Spark errors are thrown with missing tables in analytics instance created from puppet pattern 3

Suggested Labels:
Bug

Suggested Assignees:
N/A

Affected Product Version:
APIM 2.1.0
IS as a KM 5.3.0
WUM updated

OS, DB, other environment details and versions:
Ubuntu 14.04
MySQL 5.6

Steps to reproduce:

Setup APIM distributed setup with pattern-3 of [1] with IS as the KM and analytics

[1] https://github.com/wso2/puppet-apim

Can observe the following logs in the analytics instance.

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
	at scala.Option.foreach(Option.scala:236)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1915)
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelation.writeDataFrameToDAL(AnalyticsRelation.java:233)
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelation.insert(AnalyticsRelation.java:223)
	at org.apache.spark.sql.execution.datasources.InsertIntoDataSource.run(InsertIntoDataSource.scala:40)
	at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58)
	at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56)
	at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
	at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
	at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55)
	at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55)
	at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:145)
	at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:130)
	at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52)
	at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:817)
	at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:914)
	... 11 more
Caused by: java.lang.RuntimeException: Error while inserting data into table LOGANALYZER_CLASS_LEVEL_ERROR_DAILY: [-1234:LOGANALYZER_CLASS_LEVEL_ERROR_DAILY] does not exist
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsWritingFunction.recordsPutNormal(AnalyticsWritingFunction.java:143)
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsWritingFunction.recordsPut(AnalyticsWritingFunction.java:128)
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsWritingFunction.apply(AnalyticsWritingFunction.java:119)
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsWritingFunction.apply(AnalyticsWritingFunction.java:45)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:89)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
	... 3 more
Caused by: org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsTableNotAvailableException: [-1234:LOGANALYZER_CLASS_LEVEL_ERROR_DAILY] does not exist
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.readTableInfo(AnalyticsDataServiceImpl.java:455)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.lookupTableInfo(AnalyticsDataServiceImpl.java:810)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.preprocessRecordBatch(AnalyticsDataServiceImpl.java:864)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.preprocessRecords(AnalyticsDataServiceImpl.java:790)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.put(AnalyticsDataServiceImpl.java:877)
	at org.wso2.carbon.analytics.spark.core.sources.AnalyticsWritingFunction.recordsPutNormal(AnalyticsWritingFunction.java:139)
	... 11 more

Full log:
wso2carbon.log

Related Issues:
N/AJDBC deadlock detected when trying to create API

Missing config in default.yaml

The following code needs to be added to [1] similar to [2].

#wso2::file_list:
# - "repository/components/lib/%{hiera('wso2::datasources::mysql::connector_jar')}"

The latter mentioned change is needed because we instruct the user to uncomment this configuration in the default.yaml in the Confluence Wiki docs under the datasource configuration section, which comes under the Puppet Modules [3] section.

[1] https://github.com/wso2/puppet-apim/blob/master/hieradata/dev/wso2/wso2am/pattern-1/default.yaml
[2] https://github.com/wso2/puppet-esb/blob/master/hieradata/dev/wso2/wso2esb/pattern-1/default.yaml
[3] https://docs.wso2.com/display/MHAM/Configuring+Datasources

No deployment pattern that uses KM in a distributed HA deployment

Description:
No deployment pattern that uses KM in a distributed HA deployment

Suggested Labels:
Improvement

Suggested Assignees:
N/A

Affected Product Version:
APIM 210 and IS 530

OS, DB, other environment details and versions:
N/A

Steps to reproduce:
We need to have a deployment pattern that uses KM in a distributed HA deployment

Related Issues:
N/A

DB name should be configurable in the hieradata

In the hieradata for the datasources, different DBs are used for AM DB, User DB etc. These should be configurable through a different set of keys.

[Doc] Create document on setting up wso2am-analytics server using puppet modules

There is no documentation on setting up analytics server using puppet modules. Puppet modules are already there in [1].

[1] https://github.com/wso2/puppet-apim/releases/tag/v2.1.0

[AMHACK] [Puppet] Driver version is not copied via puppet scripts

Description:
Mysql driver is not copied using puppet scripts

Suggested Labels:
puppet

Suggested Assignees:

Affected Product Version:
APIM 2.1.0

OS, DB, other environment details and versions:
Ubuntu 14.04
JAVAA-jdk 1.8.0_121
Mysql 5.5

Steps to reproduce:

Copied the Mysql driver version to /etc/puppet/environments/production/modules/wso2am_analytics/files/configs/repository/components/lib
Update the driver version in /etc/puppet/hieradata/production/wso2/wso2am_analytics/pattern-1/default.yaml

When starting the pack, it will throws an error saying driver is not copied.

{code}
ERROR {org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceComponent} - Error in activating analytics data service: Error in looking up database type: com.mysql.jdbc.Driver cannot be found by jdbc-pool_7.0.34.wso2v2
org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException: Error in looking up database type: com.mysql.jdbc.Driver cannot be found by jdbc-pool_7.0.34.wso2v2
at org.wso2.carbon.analytics.datasource.rdbms.RDBMSUtils.lookupDatabaseInfo(RDBMSUtils.java:73)
at org.wso2.carbon.analytics.datasource.rdbms.RDBMSUtils.lookupCurrentQueryConfigurationEntry(RDBMSUtils.java:89)
at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.init(RDBMSAnalyticsRecordStore.java:100)
at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.initARS(AnalyticsDataServiceImpl.java:289)
at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.(AnalyticsDataServiceImpl.java:135)
at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceComponent.activate(AnalyticsDataServiceComponent.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.ntask.core.internal.TasksDSComponent.activate(TasksDSComponent.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.completeInitialization(StartupFinalizerServiceComponent.java:199)
at org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.serviceChanged(StartupFinalizerServiceComponent.java:288)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.server.admin.internal.ServerAdminServiceComponent.activate(ServerAdminServiceComponent.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:514)
at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
at org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1269)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1182)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1072)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5362)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5660)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1694)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1684)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: com.mysql.jdbc.Driver cannot be found by jdbc-pool_7.0.34.wso2v2
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:254)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:701)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:635)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:188)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:127)
at org.wso2.carbon.analytics.datasource.rdbms.RDBMSUtils.lookupDatabaseInfo(RDBMSUtils.java:66)
... 121 more
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver cannot be found by jdbc-pool_7.0.34.wso2v2
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:501)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:246)
... 127 more
{code}

Related Issues:

common.yaml needs to be updated

The common.yaml needs to be updated as it is missing some of the data sources defined in <APIM_HOME>/repository/conf/datasources/master-datasources.xml, such as the STATS DB.
A majority of the data source URLs stated need to change. For example, [1] needs to change to [2]. The DB name will change based on which approach is used for Hiera files.
[1] url: jdbc:mysql://192.168.100.1:3306/APIM_DB?autoReconnect=true
[2] jdbc:mysql://apimgtdb.mysql-wso2.com:3306/apimgtdb?autoReconnect=true
The DB and data source names defined in common.yaml is different to the names specified in the <APIM_HOME>/repository/conf/datasources/master-datasources.xml. The DB name difference is evident from the data source URLs.

Need puppet 5 manifests for each profile in API Manager 2.6.0

Description:
Need to create puppet 5 modules for API Manager 2.6.0 profiles.

Remove optional KM pointing puppet configs in gateway APIs- _AuthorizeAPI_, _RevokeAPI_, _TokenAPI_, _UserInfoAPI_

This is an issue with https://github.com/wso2/puppet-apim modules(v2.1.0).

The templates for APIs [1] AuthorizeAPI, RevokeAPI, TokenAPI, UserInfoAPI are having optional parameterized puppet confgs which pointing the KM nodes.

i.e.

<%- if @apim_keymanager and @apim_keymanager['host'] and @apim_keymanager['port'] -%>
                     <http uri-template="https://<%= @apim_keymanager['host'] %>:<%= @apim_keymanager['port'] %>/oauth2/token">
                     <%- else -%>
                     <http uri-template="https://{uri.var.hostname}:{uri.var.portnum}/oauth2/token">
<%- end -%>

In the above segment, remove the "<%- if" condition section. Just leave the <%- else -%> section, as this KM pointing parameterizing is not required.

[1] https://github.com/wso2/puppet-apim/tree/master/wso2am_runtime/templates/repository/deployment/server/synapse-configs/default/api

Configuration value repeated in the pattern-3 gateway-manager.yaml

Description:
Configuration value repeated in the pattern-3 gateway-manager.yaml

Suggested Labels:
Bug

Suggested Assignees:
N/A

Affected Product Version:
N/A

OS, DB, other environment details and versions:
N/A

Steps to reproduce:

Goto [1], There the configuration for membership_scheme: wka is repeated.

[1] https://github.com/wso2/puppet-apim/blob/master/wso2am_runtime/hieradata/dev/wso2/wso2am_runtime/pattern-3/gateway-manager.yaml

Related Issues:
N/A

The service cannot be found for the endpoint reference (EPR) thrown when generating keys with prepackaged IS

Description:
The service cannot be found for the endpoint reference (EPR) when generating keys with prepackaged IS

Suggested Labels:
Bug

Suggested Assignees:
N/A

Affected Product Version:
APIM 2.1.0 and IS as a KM 5.3.0 (wum downloaded packs)

OS, DB, other environment details and versions:
Ubuntu 14.04

Steps to reproduce:

Setup the IS as the KM using [1]
Get the wum updated prepacked IS with KM features.
add this file to the <PUPPET_HOME>/modules/wso2is_prepacked/files
deploy the KM using puppet
Get the wum update APIM 210 pack
Add this APIM 210 pack to <PUPPET_HOME>/modules/wso2am_runtime/files
deploy the Store using puppet.
login to the API store
Go to the Applications > Default Application and click on generate keys.

Actual Error log from IS instance.

TID: [-1234] [] [2017-11-23 08:25:20,053] ERROR {org.apache.axis2.engine.AxisEngine} -  The service cannot be found for the endpoint reference (EPR) https://km.dev.wso2.org:9443/services/APIKeyMgtSubscriberService 
org.apache.axis2.AxisFault: The service cannot be found for the endpoint reference (EPR) https://km.dev.wso2.org:9443/services/APIKeyMgtSubscriberService
	at org.apache.axis2.engine.DispatchPhase.checkPostConditions(DispatchPhase.java:78)
	at org.apache.axis2.engine.Phase.invoke(Phase.java:329)
	at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
	at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
	at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173)
	at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:147)
	at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
	at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
	at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91)
	at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:60)
	at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
	at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
	at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
	at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
	at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
	at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
	at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

[1] https://github.com/wso2/puppet-apim/tree/master/wso2is_prepacked
N/A

full log:
wso2carbon.log.2017-11-24.log

Specify in which file MySQL datasources configuration should be updated in puppet master

Description:
Document: Using Puppet Modules to Set up WSO2 API-M with Pattern 6
Step 5 - Update the clustering related configurations

Specify in which file, MySQL datasources configuration should be updated in puppet master. Which details should be updated?

Suggested Labels:
Blocker

Affected Product Version:
am2xx docs

OS, DB, other environment details and versions:
Ubuntu 14.04

custom.pp mostly useless?

Description:
custom.pp does not inherit the params for the profile in which it runs. It seems unlikely anyone can do much useful without this.

It should probably inherit from the params class...?

Make install path configurable

Description:
Install path is hard-coded to /usr/lib64/wso2/

Could be externalized to allow for alternate installation locations?

Current code:
$install_path = "/usr/lib64/wso2/${product}/${product_version}"

Possible alternative:
$install_path = "${install_dir}/${product}/${product_version}"

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

No puppet configuration for data purging configuration

there is no puppet configuration for analytics data purging.

Specify mysql connector version needs to be updated in wso2am_analytics - default.yaml

Description:
Document: Using Puppet Modules to Set up WSO2 API-M with Pattern 6
Step 5 - Update the clustering related configurations

Specify that mysql connector version needs to be updated in default.yaml which is placed in the following location.
/etc/puppet/hieradata/production/wso2/wso2am_analytics/pattern-1/default.yaml
Specify that following should be uncommented in the above same file.
"repository/components/lib/%{hiera('wso2::datasources::mysql::connector_jar')}"

Suggested Labels:
AM2xx puppet modules

Affected Product Version:
AM2xx docs

Proposal to rename wso2is_prepacked to wso2is-as-km

Description:
It might be better to rename wso2is_prepacked to wso2is-as-km to make it more self readable and follow the product naming convention.

Affected Product Version:
v2.1.0.1

Provide support for all popular databases

Description:
While MySQL is a good first choice, Oracle, Postgres and SQL server are few of widely used databases. Hence, it is better if our puppet scripts has OOTB support for these databases. This is a requirement for testgrid as well since it needs to test API-M against all these databases.

Currently, API-M supports following database setup scripts:

mysql 5.5 - 5.7
mysql 5.7+
mssql
oracle
oracle_rac
postgresql
informix

Suggested Labels:
Type/New Feature
Priority/High

Affected Product Version:
v2.1.0.1

OS, DB, other environment details and versions:
N/A

Steps to reproduce:
N/A

Related Issues:
N/A

Need puppet 5 manifests for each profile in API Manager 2.5.0

Description:
Currently, puppet 5 modules are available only for the all-in-one profile of API Manager 2.5.0

But they need to be created for following profiles in order to complete the puppet 5 manifests for APIM 2.5.0

Gateway-Manager
Key-Manager
Traffic-Manager
Publisher
Store

Suggested Assignees:
@irunika

Affected Product Version:
API Manager 2.5.0

No configured pattern for Traffic Manager in HA

Description:
Could not find a pattern that would enable HA for the traffic manager.

Suggested Labels:
Improvement

Suggested Assignees:
N/A

Affected Product Version:
2.1.0

OS, DB, other environment details and versions:
Ubuntu 14.04

Steps to reproduce:
Could not find a pattern to make the Traffic Manager instance HA. Had to add the following to pattern-3 traffic-manager.yaml to get it to HA.

wso2::clustering:
  enabled: true
  domain: tm.am.wso2.domain
  sub_domain: mgt
  local_member_host: "%{::ipaddress}"
  local_member_port: 4000
# WKA membership scheme
  membership_scheme: wka
  wka:
    members:
      -
        hostname: 192.168.48.33
        port: 4000
      -
        hostname: 192.168.48.34
        port: 4000

Related Issues:
N/A

Hard coded IP addresses in the hieradata

DB configurations have IP addresses hard coded in the hieradata in almost all patterns and profiles. These should be consolidated to a single data key.

[AMHACK] ServletException when 'Service Testing' option selected from the tenant management console

Description:
ServletException when tenant user click the "Service Testing" option from the carbon console

Suggested Labels:
Bug

Affected Product Version:
APIM wum updated - wso2am-2.1.0.1510928473100.zip

OS, DB, other environment details and versions:
Standalone
Oracle 15.10
JDK - jdk1.8.0_66
DB - H2
Browser - Chrome Version 58.0.3029.96 (64-bit)

Steps to reproduce:

Create a tenant in APIM console
Log in to carbon console from the tenant user
Click the "Service Testing" option

Observations
Redirection failed and blank page appearing resulting the following error message

[2017-11-22 11:33:17,435] ERROR - ApplicationDispatcher Servlet.service() for servlet bridgeservlet threw exception
javax.servlet.ServletException: File &quot;/tryit/index.jsp&quot; not found
	at org.apache.jasper.servlet.JspServlet.handleMissingResource(JspServlet.java:417)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:384)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155)
	at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
	at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:743)
	at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:603)
	at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:542)
	at org.eclipse.equinox.http.servlet.internal.RequestDispatcherAdaptor.include(RequestDispatcherAdaptor.java:37)
	at org.eclipse.equinox.http.helper.ContextPathServletAdaptor$RequestDispatcherAdaptor.include(ContextPathServletAdaptor.java:369)
	at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:905)
	at org.apache.jasper.runtime.PageContextImpl.doInclude(PageContextImpl.java:688)
	at org.apache.jasper.runtime.PageContextImpl.include(PageContextImpl.java:682)
	at sun.reflect.GeneratedMethodAccessor124.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.apache.tiles.jsp.context.JspUtil.doInclude(JspUtil.java:87)
	at org.apache.tiles.jsp.context.JspTilesRequestContext.include(JspTilesRequestContext.java:88)
	at org.apache.tiles.jsp.context.JspTilesRequestContext.dispatch(JspTilesRequestContext.java:82)
	at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:465)
	at org.apache.tiles.jsp.taglib.InsertAttributeTag.render(InsertAttributeTag.java:140)
	at org.apache.tiles.jsp.taglib.InsertAttributeTag.render(InsertAttributeTag.java:117)
	at org.apache.tiles.jsp.taglib.RenderTagSupport.execute(RenderTagSupport.java:171)
	at org.apache.tiles.jsp.taglib.RoleSecurityTagSupport.doEndTag(RoleSecurityTagSupport.java:75)
	at org.apache.tiles.jsp.taglib.ContainerTagSupport.doEndTag(ContainerTagSupport.java:80)
	at org.apache.jsp.admin.layout.template_jsp._jspx_meth_tiles_005finsertAttribute_005f7(template_jsp.java:717)
	at org.apache.jsp.admin.layout.template_jsp._jspService(template_jsp.java:380)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:439)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.ui.JspServlet.service(JspServlet.java:155)
	at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
	at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:743)
	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:485)
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:410)
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:337)
	at org.eclipse.equinox.http.servlet.internal.RequestDispatcherAdaptor.forward(RequestDispatcherAdaptor.java:30)
	at org.eclipse.equinox.http.helper.ContextPathServletAdaptor$RequestDispatcherAdaptor.forward(ContextPathServletAdaptor.java:362)
	at org.apache.tiles.servlet.context.ServletTilesRequestContext.forward(ServletTilesRequestContext.java:198)
	at org.apache.tiles.servlet.context.ServletTilesRequestContext.dispatch(ServletTilesRequestContext.java:185)
	at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:419)
	at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:370)
	at org.wso2.carbon.ui.action.ActionHelper.render(ActionHelper.java:52)
	at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:101)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37)
	at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:743)
	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:485)
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:410)
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:337)
	at org.eclipse.equinox.http.servlet.internal.RequestDispatcherAdaptor.forward(RequestDispatcherAdaptor.java:30)
	at org.wso2.carbon.redirector.servlet.ui.filters.AllPagesFilter.doFilter(AllPagesFilter.java:156)
	at org.eclipse.equinox.http.helper.FilterServletAdaptor.service(FilterServletAdaptor.java:37)
	at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.wso2.carbon.ui.filters.cache.URLBasedCachePreventionFilter.doFilter(URLBasedCachePreventionFilter.java:57)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
	at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
	at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
	at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:63)
	at org.wso2.carbon.event.receiver.core.internal.tenantmgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:52)
	at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
	at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
	at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
	at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)

[Doc] Gateway env1 and env2 does not exists in Puppet modules

Document[1] refers gateway-manager-env1 and gateway-manager-env2 as node types. But there are no such node type(hieradata files) inside puppet modules listed in github[2]. Instead it has gateway-manager-lan and gateway-manager-dmz. This needs to be fix in all the patterns listed in document[1].

[1] https://docs.wso2.com/display/AM210/Using+Puppet+Modules+to+Setup+WSO2+API-M
[2] https://github.com/wso2/puppet-apim/releases/tag/v2.1.0

SQL Exceptions while starting up the analytics instances in HA

Description:
SQL Exceptions while starting up the analytics instances in HA

Suggested Labels:
Bug

Suggested Assignees:
N/A

Affected Product Version:
AM Analytics 2.1.0

OS, DB, other environment details and versions:
Ubuntu 14.04, MySQL 5.6

Steps to reproduce:

Create a clustered Analytics deployment using [1].
add a wum updated wso2am-analytics-2.1.0 pack in <PUPPET_HOME>/modules/wso2am_analytics/files
Start the cluster trough the setup.sh deployment script.

[1] https://github.com/wso2/puppet-apim/tree/master/wso2am_analytics

TID: [-1] [] [2017-11-27 06:56:09,495]  INFO {Remoting} -  Starting remoting {Remoting}
TID: [-1] [] [2017-11-27 06:56:09,713]  INFO {Remoting} -  Remoting started; listening on addresses :[akka.tcp://[email protected]:13001] {Remoting}
TID: [-1] [] [2017-11-27 06:56:10,557] ERROR {org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine} -  Error in writing data to spark meta table: Error in adding records: SQL String can not be NULL {org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine}
org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException: Error in adding records: SQL String can not be NULL
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:164)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.putSimilarRecordBatch(AnalyticsDataServiceImpl.java:889)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.put(AnalyticsDataServiceImpl.java:879)
	at org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine.persist(AnalyticsPersistenceEngine.java:79)
	at org.apache.spark.deploy.master.PersistenceEngine.addApplication(PersistenceEngine.scala:57)
	at org.apache.spark.deploy.master.Master$$anonfun$receive$1.applyOrElse(Master.scala:253)
	at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:116)
	at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:204)
	at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
	at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:215)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: SQL String can not be NULL
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:963)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:896)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:885)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
	at com.mysql.jdbc.PreparedStatement.<init>(PreparedStatement.java:819)
	at com.mysql.jdbc.JDBC4PreparedStatement.<init>(JDBC4PreparedStatement.java:45)
	at com.mysql.jdbc.JDBC42PreparedStatement.<init>(JDBC42PreparedStatement.java:39)
	at sun.reflect.GeneratedConstructorAccessor75.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:761)
	at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1471)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4167)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4071)
	at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.wso2.carbon.ndatasource.rdbms.ConnectionRollbackOnReturnInterceptor.invoke(ConnectionRollbackOnReturnInterceptor.java:51)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:67)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState.invoke(ConnectionState.java:153)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.TrapException.invoke(TrapException.java:41)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:80)
	at com.sun.proxy.$Proxy16.prepareStatement(Unknown Source)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilarSequentially(RDBMSAnalyticsRecordStore.java:315)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilar(RDBMSAnalyticsRecordStore.java:303)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.addRecordsSimilar(RDBMSAnalyticsRecordStore.java:187)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:159)
	... 12 more
TID: [-1] [] [2017-11-27 06:56:10,567] ERROR {org.apache.spark.rpc.netty.Inbox} -  Ignoring error {org.apache.spark.rpc.netty.Inbox}
java.lang.RuntimeException: Error in writing data to spark meta table: Error in adding records: SQL String can not be NULL
	at org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine.persist(AnalyticsPersistenceEngine.java:83)
	at org.apache.spark.deploy.master.PersistenceEngine.addApplication(PersistenceEngine.scala:57)
	at org.apache.spark.deploy.master.Master$$anonfun$receive$1.applyOrElse(Master.scala:253)
	at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:116)
	at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:204)
	at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
	at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:215)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException: Error in adding records: SQL String can not be NULL
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:164)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.putSimilarRecordBatch(AnalyticsDataServiceImpl.java:889)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.put(AnalyticsDataServiceImpl.java:879)
	at org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine.persist(AnalyticsPersistenceEngine.java:79)
	... 9 more
Caused by: java.sql.SQLException: SQL String can not be NULL
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:963)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:896)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:885)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
	at com.mysql.jdbc.PreparedStatement.<init>(PreparedStatement.java:819)
	at com.mysql.jdbc.JDBC4PreparedStatement.<init>(JDBC4PreparedStatement.java:45)
	at com.mysql.jdbc.JDBC42PreparedStatement.<init>(JDBC42PreparedStatement.java:39)
	at sun.reflect.GeneratedConstructorAccessor75.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:761)
	at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1471)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4167)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4071)
	at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.wso2.carbon.ndatasource.rdbms.ConnectionRollbackOnReturnInterceptor.invoke(ConnectionRollbackOnReturnInterceptor.java:51)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:67)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState.invoke(ConnectionState.java:153)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.TrapException.invoke(TrapException.java:41)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:80)
	at com.sun.proxy.$Proxy16.prepareStatement(Unknown Source)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilarSequentially(RDBMSAnalyticsRecordStore.java:315)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilar(RDBMSAnalyticsRecordStore.java:303)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.addRecordsSimilar(RDBMSAnalyticsRecordStore.java:187)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:159)
	... 12 more
TID: [-1] [] [2017-11-27 06:56:17,965]  INFO {org.wso2.carbon.event.processor.manager.core.internal.CarbonEventManagementService} -  Starting polling event receivers {org.wso2.carbon.event.processor.manager.core.internal.CarbonEventManagementService}
TID: [-1] [] [2017-11-27 06:56:30,232] ERROR {org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine} -  Error in writing data to spark meta table: Error in adding records: SQL String can not be NULL {org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine}
org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException: Error in adding records: SQL String can not be NULL
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:164)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.putSimilarRecordBatch(AnalyticsDataServiceImpl.java:889)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.put(AnalyticsDataServiceImpl.java:879)
	at org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine.persist(AnalyticsPersistenceEngine.java:79)
	at org.apache.spark.deploy.master.PersistenceEngine.addApplication(PersistenceEngine.scala:57)
	at org.apache.spark.deploy.master.Master$$anonfun$receive$1.applyOrElse(Master.scala:253)
	at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:116)
	at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:204)
	at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
	at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:215)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: SQL String can not be NULL
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:963)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:896)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:885)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
	at com.mysql.jdbc.PreparedStatement.<init>(PreparedStatement.java:819)
	at com.mysql.jdbc.JDBC4PreparedStatement.<init>(JDBC4PreparedStatement.java:45)
	at com.mysql.jdbc.JDBC42PreparedStatement.<init>(JDBC42PreparedStatement.java:39)
	at sun.reflect.GeneratedConstructorAccessor75.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:761)
	at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1471)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4167)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4071)
	at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.wso2.carbon.ndatasource.rdbms.ConnectionRollbackOnReturnInterceptor.invoke(ConnectionRollbackOnReturnInterceptor.java:51)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:67)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState.invoke(ConnectionState.java:153)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.TrapException.invoke(TrapException.java:41)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:80)
	at com.sun.proxy.$Proxy16.prepareStatement(Unknown Source)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilarSequentially(RDBMSAnalyticsRecordStore.java:315)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilar(RDBMSAnalyticsRecordStore.java:303)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.addRecordsSimilar(RDBMSAnalyticsRecordStore.java:187)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:159)
	... 12 more
TID: [-1] [] [2017-11-27 06:56:30,233] ERROR {org.apache.spark.rpc.netty.Inbox} -  Ignoring error {org.apache.spark.rpc.netty.Inbox}
java.lang.RuntimeException: Error in writing data to spark meta table: Error in adding records: SQL String can not be NULL
	at org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine.persist(AnalyticsPersistenceEngine.java:83)
	at org.apache.spark.deploy.master.PersistenceEngine.addApplication(PersistenceEngine.scala:57)
	at org.apache.spark.deploy.master.Master$$anonfun$receive$1.applyOrElse(Master.scala:253)
	at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:116)
	at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:204)
	at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
	at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:215)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException: Error in adding records: SQL String can not be NULL
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:164)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.putSimilarRecordBatch(AnalyticsDataServiceImpl.java:889)
	at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceImpl.put(AnalyticsDataServiceImpl.java:879)
	at org.wso2.carbon.analytics.spark.core.deploy.AnalyticsPersistenceEngine.persist(AnalyticsPersistenceEngine.java:79)
	... 9 more
Caused by: java.sql.SQLException: SQL String can not be NULL
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:963)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:896)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:885)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
	at com.mysql.jdbc.PreparedStatement.<init>(PreparedStatement.java:819)
	at com.mysql.jdbc.JDBC4PreparedStatement.<init>(JDBC4PreparedStatement.java:45)
	at com.mysql.jdbc.JDBC42PreparedStatement.<init>(JDBC42PreparedStatement.java:39)
	at sun.reflect.GeneratedConstructorAccessor75.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:761)
	at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1471)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4167)
	at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4071)
	at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.wso2.carbon.ndatasource.rdbms.ConnectionRollbackOnReturnInterceptor.invoke(ConnectionRollbackOnReturnInterceptor.java:51)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:67)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState.invoke(ConnectionState.java:153)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.TrapException.invoke(TrapException.java:41)
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:109)
	at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:80)
	at com.sun.proxy.$Proxy16.prepareStatement(Unknown Source)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilarSequentially(RDBMSAnalyticsRecordStore.java:315)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.insertAndUpdateRecordsSimilar(RDBMSAnalyticsRecordStore.java:303)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.addRecordsSimilar(RDBMSAnalyticsRecordStore.java:187)
	at org.wso2.carbon.analytics.datasource.rdbms.RDBMSAnalyticsRecordStore.put(RDBMSAnalyticsRecordStore.java:159)

Related Issues:
N/A

Full Log:
wso2carbon.log

Params are not truly externalized

Description:
init.pp inherits from manifests.pp - the intent is to be able to modify parameters without forking the codebase (without editing a manifest under source control). In puppet this is typically achieved using hiera.

Unfortunately the manifests are written is such a way that does not allow us to override the parameters without editing your manifests. This defeats the purpose of attempting to externalize the parameters in the first place.

In previous versions this was catered for, for example here.
You will note the parameter list in the class definition - that allowed for default values to be overridden by hiera.

You can see this pattern has been removed in later versions, for example here.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Missing declaration for java_prefs_system_root and java_prefs_user_root

Description:
In 2.6.x apim_publisher module there are parameter references to java_prefs_system_root and java_prefs_user_root. However, these parameters are not defined in params.pp for the same module. Hence these parameters are by default null.

Suggested Labels:
bug

Suggested Assignees:
Irunika

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

[Doc] Explain the role of worker/manager setup when using Puppet modules

If someone setup API Manager with given puppet modules the given Management node does not suppose to handle traffic (because the management node has disabled the JMS Connection to TM). If that is the case, Shall we put this info in docs ?

https://docs.wso2.com/display/AM210/Using+Puppet+Modules+to+Setup+WSO2+API-M+with+Pattern+4#5732aa976fc649e0bbbcdd020963d041

Dependency failure when adding an new pack and deploying the puppet deployment

Description:
Dependency failure when adding an new pack and deploying the puppet deployment

Suggested Labels:
Bug

Suggested Assignees:
N/A

Affected Product Version:
puppet-apim 2.1.0

OS, DB, other environment details and versions:
Ubuntu 14.04
MySQL 5.6

Steps to reproduce:

Setup puppet as per the README in [1] for wso2am_runtime
Copy an wum updated pack to the "/etc/puppet/environments/production/modules/wso2am_runtime/files" location.
start the deployment using ./setup.sh (see logs below)
when running ./setup.sh the second time this works.

[1] https://github.com/wso2/puppet-apim

The following is observed when setting up.

root@ce-publisher:/opt# ./setup.sh 
#####################################################
                   Starting cleanup 
#####################################################
#####################################################
               Setting up environment 
#####################################################
#####################################################
 Installing 
#####################################################
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1139:in `issue_deprecation_warning')
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for ce-publisher.openstacklocal
Info: Applying configuration version '1512016970'
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Clean/Exec[remove_lock_file_/mnt/wso2am-2.1.0]/returns: executed successfully
Info: /Stage[main]/Wso2am_runtime/Wso2base::Clean/Exec[remove_lock_file_/mnt/wso2am-2.1.0]: Scheduling refresh of Exec[stop_process_/mnt/wso2am-2.1.0]
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Clean/Exec[stop_process_/mnt/wso2am-2.1.0]/returns: sh: 1: kill: No such process
Error: /Stage[main]/Wso2am_runtime/Wso2base::Clean/Exec[stop_process_/mnt/wso2am-2.1.0]: Failed to call refresh: kill -9 `cat /mnt/wso2am-2.1.0/wso2carbon.pid` returned 1 instead of one of [0]
Error: /Stage[main]/Wso2am_runtime/Wso2base::Clean/Exec[stop_process_/mnt/wso2am-2.1.0]: kill -9 `cat /mnt/wso2am-2.1.0/wso2carbon.pid` returned 1 instead of one of [0]
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Ensure_directory_structures[/mnt]/Exec[create_directory_structure_/mnt/wso2am-2.1.0//mnt]/returns: executed successfully
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Ensure_directory_structures[/mnt/packs]/Exec[create_directory_structure_/mnt/wso2am-2.1.0//mnt/packs]/returns: executed successfully
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/File[/tmp/wso2am-2.1.0.zip]/ensure: defined content as '{md5}5e07e2e0d8d953a515d5e375bda013ca'
Info: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/File[/tmp/wso2am-2.1.0.zip]: Scheduling refresh of Exec[check_diff]
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/Exec[check_diff]/returns: executed successfully
Info: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/Exec[check_diff]: Scheduling refresh of Exec[clean_temp_pack]
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/Exec[check_diff]: Triggered 'refresh' from 1 events
Info: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/Exec[check_diff]: Scheduling refresh of Exec[clean_temp_pack]
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/Exec[clean_temp_pack]/returns: executed successfully
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Wso2base::Clean_deployment[clean_on_pack_change]/Exec[clean_temp_pack]: Triggered 'refresh' from 2 events
Error: Could not set 'file' on ensure: No such file or directory - /mnt/packs/wso2am-2.1.0.zip20171130-10064-wh8gi9.lock at 95:/etc/puppet/environments/production/modules/wso2base/manifests/install.pp
Error: Could not set 'file' on ensure: No such file or directory - /mnt/packs/wso2am-2.1.0.zip20171130-10064-wh8gi9.lock at 95:/etc/puppet/environments/production/modules/wso2base/manifests/install.pp
Wrapped exception:
No such file or directory - /mnt/packs/wso2am-2.1.0.zip20171130-10064-wh8gi9.lock
Error: /Stage[main]/Wso2am_runtime/Wso2base::Install/File[/mnt/packs/wso2am-2.1.0.zip]/ensure: change from absent to file failed: Could not set 'file' on ensure: No such file or directory - /mnt/packs/wso2am-2.1.0.zip20171130-10064-wh8gi9.lock at 95:/etc/puppet/environments/production/modules/wso2base/manifests/install.pp
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Exec[extract_/mnt/packs/wso2am-2.1.0.zip]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Install/Exec[extract_/mnt/packs/wso2am-2.1.0.zip]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Exec[set_ownership_/mnt/wso2am-2.1.0]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Install/Exec[set_ownership_/mnt/wso2am-2.1.0]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Install/Exec[set_permissions_/mnt/wso2am-2.1.0]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Install/Exec[set_permissions_/mnt/wso2am-2.1.0]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/api-manager.xml]/File[/mnt/wso2am-2.1.0/repository/conf/api-manager.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/api-manager.xml]/File[/mnt/wso2am-2.1.0/repository/conf/api-manager.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/log4j.properties]/File[/mnt/wso2am-2.1.0/repository/conf/log4j.properties]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/log4j.properties]/File[/mnt/wso2am-2.1.0/repository/conf/log4j.properties]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_files[repository/components/lib/mysql-connector-java-5.1.39-bin.jar]/File[/mnt/wso2am-2.1.0/repository/components/lib/mysql-connector-java-5.1.39-bin.jar]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_files[repository/components/lib/mysql-connector-java-5.1.39-bin.jar]/File[/mnt/wso2am-2.1.0/repository/components/lib/mysql-connector-java-5.1.39-bin.jar]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/registry.xml]/File[/mnt/wso2am-2.1.0/repository/conf/registry.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/registry.xml]/File[/mnt/wso2am-2.1.0/repository/conf/registry.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/carbon.xml]/File[/mnt/wso2am-2.1.0/repository/conf/carbon.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/carbon.xml]/File[/mnt/wso2am-2.1.0/repository/conf/carbon.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_files[repository/resources/security/client-truststore.jks]/File[/mnt/wso2am-2.1.0/repository/resources/security/client-truststore.jks]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_files[repository/resources/security/client-truststore.jks]/File[/mnt/wso2am-2.1.0/repository/resources/security/client-truststore.jks]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/tomcat/catalina-server.xml]/File[/mnt/wso2am-2.1.0/repository/conf/tomcat/catalina-server.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/tomcat/catalina-server.xml]/File[/mnt/wso2am-2.1.0/repository/conf/tomcat/catalina-server.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/identity/identity.xml]/File[/mnt/wso2am-2.1.0/repository/conf/identity/identity.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/identity/identity.xml]/File[/mnt/wso2am-2.1.0/repository/conf/identity/identity.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/identity/application-authentication.xml]/File[/mnt/wso2am-2.1.0/repository/conf/identity/application-authentication.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/identity/application-authentication.xml]/File[/mnt/wso2am-2.1.0/repository/conf/identity/application-authentication.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_files[repository/resources/security/wso2carbon.jks]/File[/mnt/wso2am-2.1.0/repository/resources/security/wso2carbon.jks]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_files[repository/resources/security/wso2carbon.jks]/File[/mnt/wso2am-2.1.0/repository/resources/security/wso2carbon.jks]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/identity/EndpointConfig.properties]/File[/mnt/wso2am-2.1.0/repository/conf/identity/EndpointConfig.properties]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/identity/EndpointConfig.properties]/File[/mnt/wso2am-2.1.0/repository/conf/identity/EndpointConfig.properties]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/jndi.properties]/File[/mnt/wso2am-2.1.0/repository/conf/jndi.properties]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/jndi.properties]/File[/mnt/wso2am-2.1.0/repository/conf/jndi.properties]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[bin/wso2server.sh]/File[/mnt/wso2am-2.1.0/bin/wso2server.sh]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[bin/wso2server.sh]/File[/mnt/wso2am-2.1.0/bin/wso2server.sh]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/security/authenticators.xml]/File[/mnt/wso2am-2.1.0/repository/conf/security/authenticators.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/security/authenticators.xml]/File[/mnt/wso2am-2.1.0/repository/conf/security/authenticators.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/user-mgt.xml]/File[/mnt/wso2am-2.1.0/repository/conf/user-mgt.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/user-mgt.xml]/File[/mnt/wso2am-2.1.0/repository/conf/user-mgt.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/axis2/axis2.xml]/File[/mnt/wso2am-2.1.0/repository/conf/axis2/axis2.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/axis2/axis2.xml]/File[/mnt/wso2am-2.1.0/repository/conf/axis2/axis2.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/File[/mnt/wso2am-2.1.0/repository/components/patches]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/File[/mnt/wso2am-2.1.0/repository/components/patches]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/File[/mnt/wso2am-2.1.0/repository/components/patches/.gitkeep]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/File[/mnt/wso2am-2.1.0/repository/components/patches/.gitkeep]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/datasources/master-datasources.xml]/File[/mnt/wso2am-2.1.0/repository/conf/datasources/master-datasources.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/datasources/master-datasources.xml]/File[/mnt/wso2am-2.1.0/repository/conf/datasources/master-datasources.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/datasources/am-datasources.xml]/File[/mnt/wso2am-2.1.0/repository/conf/datasources/am-datasources.xml]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Configure/Wso2base::Push_templates[repository/conf/datasources/am-datasources.xml]/File[/mnt/wso2am-2.1.0/repository/conf/datasources/am-datasources.xml]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Service/Service[wso2am]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Service/Service[wso2am]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2am_runtime/Wso2base::Service/Notify[Successfully started WSO2 service [name] wso2am, [CARBON_HOME] /mnt/wso2am-2.1.0]: Dependency File[/mnt/packs/wso2am-2.1.0.zip] has failures: true
Warning: /Stage[main]/Wso2am_runtime/Wso2base::Service/Notify[Successfully started WSO2 service [name] wso2am, [CARBON_HOME] /mnt/wso2am-2.1.0]: Skipping because of failed dependencies
Notice: /Stage[main]/Wso2base::Java/File[/mnt/jdk-8u131]/ensure: created
Notice: /Stage[main]/Wso2base::Java/Exec[move_java-wso2base::java]/returns: executed successfully
Notice: Finished catalog run in 46.86 seconds

Related Issues:
N/A

Specify how to create configdb, govregdb and userdb

Description:
Document: Using Puppet Modules to Set up WSO2 API-M with Pattern 6
Step 5 - Update the clustering related configurations

In /etc/puppet/hieradata/production/wso2/wso2is_prepacked/pattern-1/default.yaml file, database configurations for configdb, govregdb and userdb databases are listed. Please provide information about creating these databases. Please mention which scripts to be run when creating them.

Suggested Labels:
AM2xx puppet modules
Blocker

Affected Product Version:
AM2xx

OS, DB, other environment details and versions:
Ubuntu 14.04

Adapt secure vault config changes from puppet base

Applying secure vault configurations via puppet templates is removed in puppet base. Instead, we simply copy the secure vault files via puppet file resources. This change needs to be adapted in order to ensure the functionality of secure vault

Typing error: axis2.xml.erb line 179 has an extral "l" for <messageFormatter contentType="application/octet-streaml"

Description:
In all of the modules the template file templates/carbon-home/repository/conf/axis2/axis2.xml.erb has an error on line 179. The value of contentType has an extra "l" : application/octet-streaml

<messageFormatter contentType="application/octet-streaml" 
                          class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

It should be

<messageFormatter contentType="application/octet-stream" 
                          class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

The actual config file in v2.6.0 is OK. So just an issue with the puppet templete.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
all

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

WSO2_CARBON_DB configuration hardcoded

Description:
The template master-datasources.xml.erb includes hard-coded database configuration for the WSO2_CARBON_DB.

This obviosuly makes it impossible for anyone to reconfigure the database without hacking files under source control.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Credentials are hardcoded in v2.0.0 jndi.properties file

Reported by Isabelle in puppet-modules:
https://github.com/wso2/puppet-modules/issues/96

Please refer following file, we may need to parameterize these credentials:
https://github.com/wso2/puppet-apim/blob/master/templates/repository/conf/jndi.properties.erb#L19

[Pattern 6] NPE thrown when "DataPublisher" is set to false in Gateway Manager Node

Description:
When a cluster is configured using the pattern 6 scripts [1]. A NPE is thrown on every api invocation. Please find the stacktrace below.

Was able to overcome this by setting the "DataPublisher" to true [2] in gateway manager's apim-manager.xml. Refer dev mail - "Error while publishing throttling events to global policy server"

TID: [-1234] [] [2018-01-26 14:07:41,822] ERROR {org.wso2.carbon.apimgt.gateway.throttling.publisher.ThrottleDataPublisher} -  Error while publishing throttling events to global policy server {org.wso2.carbon.apimgt.gateway.throttling.publisher.ThrottleDataPublisher}
java.lang.NullPointerException
    at org.wso2.carbon.apimgt.gateway.throttling.publisher.ThrottleDataPublisher.publishNonThrottledEvent(ThrottleDataPublisher.java:119)
    at org.wso2.carbon.apimgt.gateway.handlers.throttling.ThrottleHandler.doRoleBasedAccessThrottlingWithCEP(ThrottleHandler.java:349)
    at org.wso2.carbon.apimgt.gateway.handlers.throttling.ThrottleHandler.doThrottle(ThrottleHandler.java:512)
    at org.wso2.carbon.apimgt.gateway.handlers.throttling.ThrottleHandler.handleRequest(ThrottleHandler.java:460)
    at org.apache.synapse.rest.API.process(API.java:325)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:90)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:69)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:304)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:78)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:330)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:159)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

[1] https://github.com/wso2/puppet-apim/blob/v2.1.0/wso2am_runtime/hieradata/dev/wso2/wso2am_runtime/pattern-6

[2] https://github.com/wso2/puppet-apim/blob/v2.1.0/wso2am_runtime/hieradata/dev/wso2/wso2am_runtime/pattern-6/gateway-manager.yaml#L98

Suggested Labels:
Bug, APIM 2.1.0, Pattern 6

Affected Product Version:
APIM 2.1.0 (wum updated 25/01/2018)

wso2 / puppet-apim Goto Github PK

puppet-apim's Introduction

Puppet Modules for WSO2 API Manager

Supported Puppet Versions

Quick Start Guide

Performance Tuning

Manifests in a module

puppet-apim's People

Contributors

Stargazers

Watchers

Forkers

puppet-apim's Issues

Recommend Projects

Recommend Topics

Recommend Org