biemond / biemond-orawls-vagrant Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 8.0 3.36 MB

WebLogic 10.3.6 cluster with Puppet & Vagrant

Shell 0.21% Puppet 12.84% Ruby 58.50% Groff 0.06% HTML 28.30% Pascal 0.06% Python 0.03%

biemond-orawls-vagrant's People

Contributors

Stargazers

Watchers

Forkers

skibum55 dever860 dportabella bassista sebykrueger sridharsuravarapu jespinoz veriton ayoneeyee

biemond-orawls-vagrant's Issues

implement the resouce types: orawls:machine, orawls:managed_server and orawls:cluster ?

we call orawls::wlstexec in order to define machines, managed_servers and clusters.

for instance, to create a machine we call

orawls::wlstexec { 'createMachine_node1':
     script:            'createMachine.py'
     params:
        machineName:    'Node1'
        machineDnsName: '10.10.10.100'   
}

this is not valid syntay, but just for the sake of illustration. see below for the complete example.

the problem is that 'machines_instances' is not actually a list of machines instances ('Node1'), but a list of how to create our machines instances ('createMachine_node1'),
and also the parameters are not early verified by puppet, but by the createMachine.py script.

could it be possible to create the resource orawls:machine, orawls:managed_server and orawls:cluster (which they would call orawls:wlstexec)?

then, orawls:machine would define a list of valid parameters, such as machineName and machineDnsName.

or did you see a problem with this design?

example taken from this vagrant:

puppet/manifests//site.pp:
class machines{
  require userconfig

  notify { 'class machines':} 
  $default_params = {}
  $machines_instances = hiera('machines_instances', [])
  create_resources('orawls::wlstexec',$machines_instances, $default_params)
}


puppet/hieradata//admin.example.com.yaml
machines_instances:
  'createMachine_node1':
     log_output:           *logoutput
     weblogic_type:        "machine"
     weblogic_object_name: "Node1"
     script:               'createMachine.py'
     params:
        - "machineName      = 'Node1'"
        - "machineDnsName   = '10.10.10.100'"
  'createMachine_node2':
     log_output:           *logoutput
     weblogic_type:        "machine"
     weblogic_object_name: "Node2"
     script:               'createMachine.py'
     params:
        - "machineName      = 'Node2'"
        - "machineDnsName   = '10.10.10.200'"

remove default value of wls_server listenaddress?

the parameter orawls/lib/puppet/type/wls_server/listenaddress.rb
has a default value of 'localhost'.

listenaddress is an optional value,
however, if I specify an undefined value, I will get 'localhost'.

a workaround is to specify an empty string '',
but this approach is not very clean,
and it didn't work in a complex situation (that I am still looking into).

does it make sense to remove this 'localhost' default value?

Use orawls::domain instead of orawls::copydomain in the non-admin nodes

Hi,

Here are two related questions:
#1- Use orawls::domain instead of orawls::copydomain in the non-admin nodes

The example vagrant machines uses the weblogic pack/unpack mechanism to set-up the domain in the non-admin nodes.

Instead, I would prefer to use orawls::domain also in the non-admin nodes,
and then enroll them to the admin server.

I've tested this approach manually,
and now I'd like to modify the domain.pp to include a parameter specifying whether to install a local admin server or to enroll to a defined one.

Do you see any problem with this approach?
#2- weblogic unpack does ignore a modified setDomainEnv.sh

I've posted this issue here: https://community.oracle.com/thread/3517602
basically, if we modify setDomainEnv.sh in the admin node, pack the domain, and unpack it to a non-admin node,
the result is that the setDomainEnv.sh in the non-admin node is not what we wanted (we get the base file of weblogic).

I modified setDomainEnv.sh to call an external setCustomDomainEnv.sh,
and I see that this is not really propagated to the non-admin nodes.

I am wondering if your modification of "debugFlag" in setDomainEnv.sh really gets propagated in the non-admin nodes?

According to the answer to this thread (although not an official answer),
weblogic unpack recreates this file from scratch, and so it is normal that it is not propagated-
https://community.oracle.com/message/12265267#12265267

What do you think about this?
If this is true, that would also justify using the approach in question 1 (to use orawls::domain instead of orawls::copydomain in the non-admin nodes)

not working on puppet 3.4.3 (Puppet Enterprise 3.2.0)?

I re-post the issue.

I've updated to puppet 3.4.3 (Puppet Enterprise 3.2.0), and it is now not working.
It installs weblogic, it creates the domain, but it then complains on wls_machine.

Notice: class machines
Notice: /Stage[main]/Machines/Notify[class machines]/message: defined 'message' as 'class machines'
Debug: /Stage[main]/Machines/Notify[class machines]: The container Class[Machines] will propagate my refresh event
Debug: Prefetching simple resources for wls_machine
Info: index wls_machine 
Info: Executing: wlstScript
Error: Could not set 'present' on ensure: wrong number of arguments (1 for 0)
/tmp/vagrant-puppet/modules-0/orawls/lib/puppet/type/wls_machine.rb:27:in `block (2 levels) in <module:Puppet>'
/tmp/vagrant-puppet/modules-0/easy_type/lib/easy_type/provider.rb:94:in `call'
/tmp/vagrant-puppet/modules-0/easy_type/lib/easy_type/provider.rb:94:in `build_from_type'
/tmp/vagrant-puppet/modules-0/easy_type/lib/easy_type/provider.rb:59:in `create'
/opt/puppet/lib/ruby/site_ruby/1.9.1/puppet/property/ensure.rb:16:in `block in defaultvalues'
...

i am trying to see if it is a problem of version puppet 3.4.2 vs 3.4.3, or a problem of puppet normal vs entreprise.

do you know about it?

vagrant example with multiple domains

at first I didn't understand much why you defined apps outside hiera and used the loadyaml function: #2 (comment)
but now my hiera data file is becoming too big, difficult to split into smaller files, and I now see your point now.

I still have a doubt though; we have ~10 testing domains on a server, so I wonder how to configure this.
in your vagrant example: https://github.com/biemond/biemond-orawls-vagrant/blob/master/puppet/hieradata/admin.example.com.yaml
you can define default values (domain_name, domain_adminserver, domain_adminserver_address...) when you have just one domain on a server.

for instance, wls_server create.py.erb uses a weblogicConnectUrl value, coming from a default hiera('domain_adminserver_address').
how would you do this when there are more than one domain on a server?
(that is, how would you define default values for each domain; how to avoid repeating those values for each domain and type (wls_server, wls_machine, wls_cluster...))

do you have an example vagrant with more than one domain?

puppet/modules/orawls/files/providers/wls_server/create.py.erb
  connect('<%= weblogicUser %>','<%= weblogicPassword %>'  ,'<%= weblogicConnectUrl %>')

puppet/modules/orawls/lib/utils/wls_access.rb
      def weblogicConnectUrl
        setting_for('connect_url')

puppet/modules/orawls/lib/utils/settings.rb
  def setting_for(key)
    settings[key]

  $domain_address = hiera('domain_adminserver_address')
  $domain_port    = hiera('domain_adminserver_port')

./puppet/manifests/site.pp
  wls_setting { 'default':
    user               => hiera('wls_os_user'),
    weblogic_home_dir  => hiera('wls_weblogic_home_dir'),
    connect_url        => "t3://${domain_address}:${domain_port}",

innecessary script templates?

there are a lot of templates for scripts. for instance:
puppet/modules/orawls/templates/wlst/wlstexec/createServer.py.erb

# create a managed server

wlsUser    = '<%= @weblogic_user %>'
password   = sys.argv[1] 
machine    = '<%= @adminserver_address %>'
portNumber = '<%= @adminserver_port %>'

user_config = '<%= @userConfigFile %>'
user_key    = '<%= @userKeyFile %>'

listenAddress = ''

#wlsServerName, javaArguments, machineName, listenAddress,listenPort, nodeMgrLogDir
<% @params.each do |param| -%>   
<%= param %>                    
<% end -%>                      

def createBootPropertiesFile(directoryPath,fileName, username, password):
  serverDir = File(directoryPath)
  bool = serverDir.mkdirs()
[...]

these templates are only used to pass variables to the script.
in this case: wlsUser, machine, portNumber, user_config, user_key and params.

wouldn't it be simpler to have a static script, and pass all these parameters when executing the script, as it is done for password?

(I agree that it is a bit more difficult for the case of the params array, but it should be possible to make it nicely)

wls_cluster/modify does not dettach servers from the cluster when requested

wls_cluster iterates on the list of clusterNodesTargets, and assign each node to the cluster.
however, if a node has been removed from clusterNodesTargets, wls_cluster/modify will not handle the issue.
https://github.com/biemond/biemond-orawls-vagrant/blob/master/puppet/modules/orawls/files/providers/wls_cluster/modify.py.erb

I see two solutions:
1- wls_cluster/modify should iterate on all servers, and:

if that server is on clusterNodesTargets, assign it to the cluster
if not, and the cluster of that server is the current wls_cluster, then unset the cluster for that node.

2- use assign: http://docs.oracle.com/cd/E13222_01/wls/docs90/config_scripting/reference.html
assign('Server', 'ms1,ms2','Cluster','wlsCluster')

I am not an expert of weblogic, but option2 seems good to me.
what do you think?
should I do it and send you a pull request?

running orawls::packdomain just after orawls::domain?

In the vangrant example,
orawls::packdomain is run after the full domain configuration, including machines, serverrs, clusters and jmxs.

is there a problem to run the orawls::packdomain just after orawls::domain,
and then configure the rest of the domain (machines, servers, clusters and jmxs)?

if we run the current vagrant example, and then we add/update/remove servers, the orawls::packdomain will not be executed again anyway, and the admin server will make sure to update this new configuration all their nodes. so, why not running orawls::packdomain just after orawls::domain anyway?

hi biemond,

I installed orawls module. I am having this issue.
Debug: importing '/etc/puppetlabs/puppet/development/modules/weblogic_orawls/manifests/init.pp' in environment development
Debug: Automatically imported weblogic_orawls from weblogic_orawls into development
Debug: importing '/etc/puppetlabs/puppet/development/modules/orawls/manifests/init.pp' in environment development
Debug: importing '/etc/puppetlabs/puppet/development/modules/orawls/manifests/weblogic.pp' in environment development
Debug: Automatically imported orawls::weblogic from orawls/weblogic into development
Debug: hiera(): Hiera YAML backend starting
Debug: hiera(): Looking up orawls::weblogic::fmw_infra in YAML backend
Debug: hiera(): Looking for data source defaults
Debug: hiera(): Cannot find datafile /var/lib/hiera/defaults.yaml, skipping
Debug: hiera(): Looking for data source dot-pap-spr-d03.local
Debug: hiera(): Looking for data source development
Debug: hiera(): Looking for data source global
Error: Could not run: (): block sequence entries are not allowed in this context at line 13 column 24
The global.yaml file is here

  2 messageDefault: 'default'
  3 messageEnv:     'global'
  4
  5
  6
  7
  8 hosts:
  9   'localhost':
 10     ip:                "127.0.0.1"
 11     host_aliases:      - 'localhost.localdomain'
 12   'dot-pap-spr-d03.local':
 13     ip:                "10.234.0.38"
 14     host_aliases:      - 'admin'
 15
 16
 17 apps_config_dir:    "/u01/app/oracle/product/apps"
 18
 19
 20 # global WebLogic vars

Permissions with install of WLS 12c

I am getting this os permission error on trying to install the server..

Does this have something to do with the ssh password that you have hashed?


Notice: /Stage[main]/Orawls::Weblogic/Orawls::Utils::Structure[weblogic structure 1212]/File[/opt/oracle]/mode: mode changed '0755' to '0775'
Notice: /Stage[main]/Orawls::Weblogic/Orawls::Utils::Structure[weblogic structure 1212]/File[/opt/oracle/oraInventory]/ensure: created
Notice: /Stage[main]/Orawls::Weblogic/File[/data/install/wls_121200.jar]/ensure: defined content as '{md5}66380e1b5143949f0169e9b62e9272fa'
Notice: /Stage[main]/Orawls::Weblogic/File[/data/install/weblogic_silent_install.xml]/ensure: created
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Extracting files......................
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Starting Oracle Universal Installer
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: 
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Checking if CPU speed is above 300 MHz.   Actual 2055.771 MHz    Passed
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Checking swap space: must be greater than 512 MB.   Actual 9404400 MB    Passed
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Checking if this platform requires a 64-bit JVM.   Actual 64    Passed (64-bit not required)
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Checking temp space: must be greater than 300 MB.   Actual 25237 MB    Passed
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: 
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: 
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Preparing to launch the Oracle Universal Installer from /tmp/OraInstall2013-11-22_08-58-01PM
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Log: /opt/oracle/oraInventory/logs/install2013-11-22_08-58-01PM.log
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Copyright (c) 1996, 2013, Oracle and/or its affiliates. All rights reserved.
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Reading response file..
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: Verifying data......
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: [VALIDATION] [ERROR]:INST-07008: Validation of Oracle Home location failed. User does not have permission to create home/instance location
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: [VALIDATION] [SUGGESTION]:Make sure that you have the required permission to create the directory
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: installation Failed. Exiting installation due to data validation failure.
Notice: /Stage[main]/Orawls::Weblogic/Exec[install weblogic 1212]/returns: The Oracle Universal Installer failed.  Exiting.

Common YAML

messageDefault: 'default'
messageEnv:     'common'


# global WebLogic vars
wls_oracle_base_home_dir: &wls_oracle_base_home_dir "/opt/oracle"
wls_weblogic_user:        &wls_weblogic_user        "welcome1"
wls_weblogic_home_dir:    &wls_weblogic_home_dir    "/oracle/product/Middleware12c/wlserver"
wls_middleware_home_dir:  &wls_middleware_home_dir  "/oracle/product/Middleware12c"
wls_version:              &wls_version              1212

# global OS vars
wls_os_user:              &wls_os_user              "oracle"
wls_os_group:             &wls_os_group             "dba"
wls_download_dir:         &wls_download_dir         "/data/install"
wls_source:               &wls_source               "/vagrant"
wls_jdk_home_dir:         &wls_jdk_home_dir         "/usr/java/jdk1.7.0_45"
wls_log_dir:              &wls_log_dir              "/data/logs"


#WebLogic installation variables 
orawls::weblogic::version:              *wls_version
orawls::weblogic::filename:             "wls_121200.jar"
orawls::weblogic::middleware_home_dir:  *wls_middleware_home_dir
orawls::weblogic::log_output:           false

# hiera default anchors
orawls::weblogic::jdk_home_dir:         *wls_jdk_home_dir
orawls::weblogic::oracle_base_home_dir: *wls_oracle_base_home_dir
orawls::weblogic::os_user:              *wls_os_user
orawls::weblogic::os_group:             *wls_os_group
orawls::weblogic::download_dir:         *wls_download_dir
orawls::weblogic::source:               *wls_source

admin.example.com.yaml

messageEnv: 'admin'

orawls::weblogic::log_output:   true


logoutput:                     &logoutput                     true

# when you have just one domain on a server
domain_name:                "Wls12c"
domain_adminserver:         "AdminServer"
domain_adminserver_address: "10.10.10.10"
domain_adminserver_port:    7001
domain_nodemanager_port:    5556
domain_wls_password:        "welcome1"
domain_user_config_file:    "/home/oracle/oracle-Wls1036-WebLogicConfig.properties"
domain_user_key_file:       "/home/oracle/oracle-Wls1036-WebLogicKey.properties"


# create a standard domain
domain_instances:
  'wlsDomain':
     domain_template:          "standard"
     development_mode:         false
     log_output:               *logoutput
#     repository_database_url: "jdbc:oracle:thin:@xxxx.alfa.local:1521/test.oracle.com"
#     repository_prefix:       "DEV"
#     repository_password:     "Welcome01"



# create and startup the nodemanager
nodemanager_instances:
  'nodemanager':
     log_output:           *logoutput

# startup adminserver for extra configuration
control_instances:
  'startWLSAdminServer':
     domain_dir:           "/opt/oracle/middleware11g/user_projects/domains/Wls12c"
     server_type:          'admin'
     target:               'Server'
     server:               'AdminServer'
     action:               'start'
     log_output:           *logoutput

remove configuration duplication

thanks for sharing this weblogic puppet module! :)

one question, in the example provided there is quite a lot of duplication.
for instance:
cluster_instances defines 'clusterOne', with a list of clusterNodes (servers): wlsServer1, wlsServer2.
then, servers_instances needs to specify again the name of cluster 'clusterOne'.

If we would want to change the name of cluster from "clusterOne" to "clusterHello", we would need to modify this name in both two files. if we would only modify that name in one of the files, nothing would work.
that is, this is a duplication of information.

we see this issue in several places.

is it possible to remove this duplication?

puppet/apps/one/clusters/one_clusters.yaml
clusters_instances:
   - clusterOne:
      cluster_web:
         weblogic_object_name: "WebCluster"
         log_output:           *logoutput
         weblogic_type:        "cluster"
         script:               'createCluster.py'
         params:
            - "clusterName      = 'WebCluster'"
            - "clusterNodes     = 'wlsServer1,wlsServer2'"



puppet/apps/one//servers/one_servers.yaml
servers_instances:
   - clusterOne:
      global_parameters:
         log_output:           *logoutput
         weblogic_type:        "server"
         script:               'createServer.py'
         params:
            - "listenPort       = 8001"
            - "nodeMgrLogDir    = '/data/logs'"
      wlsServer1_node1:
         weblogic_object_name: "wlsServer1"
         params:
            - "javaArguments    = '-XX:PermSize=256m -XX:MaxPermSize=256m -Xms752m -Xmx752m -Dweblogic.Stdout=/data/logs/wlsServer1.out -Dweblogic.Stderr=/data/logs/wlsServer1_err.out'"
            - "wlsServerName    = 'wlsServer1'"
            - "machineName      = 'Node1'"
            - "listenAddress    = '10.10.10.100'"
      wlsServer2_node2:
         weblogic_object_name: "wlsServer2"
         params:
            - "javaArguments    = '-XX:PermSize=256m -XX:MaxPermSize=256m -Xms752m -Xmx752m -Dweblogic.Stdout=/data/logs/wlsServer2.out -Dweblogic.Stderr=/data/logs/wlsServer2_err.out'"
            - "wlsServerName    = 'wlsServer2'"
            - "machineName      = 'Node2'"
            - "listenAddress    = '10.10.10.200'"

pass $domain_dir directly to orawls::domain

define orawls::domain(...) {
  if $::override_weblogic_domain_folder == undef {
    $domain_dir = "${middleware_home_dir}/user_projects/domains"
    $app_dir    = "${middleware_home_dir}/user_projects/applications"
  } else {
    $domain_dir = "${::override_weblogic_domain_folder}/domains"
    $app_dir    = "${::override_weblogic_domain_folder}/applications"
  }
...
}

with this code, the parent directory of all domains must be named "domains".

this setup does not work in our case. our sysadmin decided to have a structure like this.
/srv/domain_group1/mydomain1
/srv/domain_group1/mydomain2

so, the easiest to pass the $domain_dir as parameter to orawls::domain (instead of $middleware_home_dir or ${::override_weblogic_domain_folder}).

but before doing so, I wanted to know if there was a reason why you did it that way,
and if you see any problem with my suggestion? would you accept such a pull request?
what is this "applications" folder for?

orawls/manifests/domain.pp setDebugFlagOnFalse?

Why this code in orawls/manifests/domain.pp?

    if $::kernel == "SunOS" {
      exec { "setDebugFlagOnFalse ${domain_name} ${title}":
        command => "sed -e's/debugFlag=\"true\"/debugFlag=\"false\"/g' ${domain_dir}/${domain_name}/bin/setDomainEnv.sh > /tmp/domain.tmp && mv /tmp/domain.tmp ${domain_dir}/${domain_name}/bin/setDomainEnv.sh",
...
    } else {
      exec { "setDebugFlagOnFalse ${domain_name} ${title}":
        command => "sed -e's/debugFlag=\"true\"/debugFlag=\"false\"/g' ${domain_dir}/${domain_name}/bin/setDomainEnv.sh",
...

In case of SunOS, the exec command effectively changes debugFlag="true" to false,
but if not SunOS, the exec command does not have any effect (and so the exec command could be removed).

Also, what is the intent of this exec?
A typical setDomainEnv.sh file, such as this one http://blog.sina.com.cn/s/blog_53e89d4f01019m3x.html
has two lines with debugFlag:

...
while [ $# -gt 0 ]
do
    case $1 in
    nodebug)
        debugFlag="false"
...
if [ "${PRODUCTION_MODE}" = "true" ] ; then
    debugFlag="false"
...

and, by default, they are already set to false.